article thumbnail

Fundamentals of Data Engineering


The following is a review of the book Fundamentals of Data Engineering by Joe Reis and Matt Housley, published by O’Reilly in June of 2022, and some takeaway lessons. This book is as good for a project manager or any other non-technical role as it is for a computer science student or a data engineer.

article thumbnail

Data engineers vs. data scientists

O'Reilly Media - Data

It’s important to understand the differences between a data engineer and a data scientist. Misunderstanding or not knowing these differences are making teams fail or underperform with big data. I think some of these misconceptions come from the diagrams that are used to describe data scientists and data engineers.


Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Delivering Modern Enterprise Data Engineering with Cloudera Data Engineering on Azure


After the launch of CDP Data Engineering (CDE) on AWS a few months ago, we are thrilled to announce that CDE, the only cloud-native service purpose built for enterprise data engineers, is now available on Microsoft Azure. . Prerequisites for deploying CDP Data Engineering on Azure can be found here.

article thumbnail

Building a Robust Data Engineering Pipeline in the Streaming Media Industry: An Insider’s Perspective

Dzone - DevOps

In this detailed and personal account, the author shared his journey of building and evolving data pipelines in the rapidly transforming streaming media industry. In the last two decades, data engineering has dramatically transformed industries.

article thumbnail

The Next-Generation Cloud Data Lake: An Open, No-Copy Data Architecture

In an effort to be data-driven, many organizations are looking to democratize data. However, they often struggle with increasingly larger data volumes, reverting back to bottlenecking data access to manage large numbers of data engineering requests and rising data warehousing costs.

article thumbnail

Why Data Engineering And AI Are Mutually Beneficial

Daffodil Software

Artificial Intelligence (AI) and data engineering are closely interlinked. On one hand, making sense of unstructured data is the process known as data science or data engineering.

article thumbnail

Make the leap to Hybrid with Cloudera Data Engineering


When we introduced Cloudera Data Engineering (CDE) in the Public Cloud in 2020 it was a culmination of many years of working alongside companies as they deployed Apache Spark based ETL workloads at scale. Each unlocking value in the data engineering workflows enterprises can start taking advantage of. Usage Patterns.

article thumbnail

Data & Analytics Maturity Model Workshop Series

Speaker: Dave Mariani, Co-founder & Chief Technology Officer, AtScale; Bob Kelly, Director of Education and Enablement, AtScale

Workshop video modules include: Breaking down data silos. Integrating data from third-party sources. Developing a data-sharing culture. Combining data integration styles. Translating DevOps principles into your data engineering process. Using data models to create a single source of truth.

article thumbnail

The Evolution of the Data Team: Lessons Learned From Growing a Team From 3 to 20

Speaker: Mindy Chen, Director of Decision Science, Hudl

Mindy Chen, Director of Decision Science at Hudl, will take us on a journey through the challenges and opportunities she has seen when building a data team from scratch. Growing from 3 data engineers to a robust team of 20, Hudl has been on a journey to establish their data capability.