article thumbnail

Fundamentals of Data Engineering

Xebia

The following is a review of the book Fundamentals of Data Engineering by Joe Reis and Matt Housley, published by O’Reilly in June of 2022, and some takeaway lessons. This book is as good for a project manager or any other non-technical role as it is for a computer science student or a data engineer.

article thumbnail

The early returns on gen AI for software development

CIO

Generative AI is already having an impact on multiple areas of IT, most notably in software development. Still, gen AI for software development is in the nascent stages, so technology leaders and software teams can expect to encounter bumps in the road.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

The Good and the Bad of Databricks Lakehouse Platform

Altexsoft

What is Databricks Databricks is an analytics platform with a unified set of tools for data engineering, data management , data science, and machine learning. It combines the best elements of a data warehouse, a centralized repository for structured data, and a data lake used to host large amounts of raw data.

article thumbnail

The Good and the Bad of Apache Kafka Streaming Platform

Altexsoft

The technology was written in Java and Scala in LinkedIn to solve the internal problem of managing continuous data flows. The number of possible applications tends to grow due to the rise of IoT , Big Data analytics , streaming media, smart manufacturing, predictive maintenance , and other data-intensive technologies.

article thumbnail

The Good and the Bad of Apache Spark Big Data Processing

Altexsoft

These seemingly unrelated terms unite within the sphere of big data, representing a processing engine that is both enduring and powerfully effective — Apache Spark. Before diving into the world of Spark, we suggest you get acquainted with data engineering in general. How data engineering works in a nutshell.

article thumbnail

Ascend.io lands $31M to automate data pipeline orchestration

TechCrunch

Ascend.io , a company developing data automation products for enterprise customers, has raised $31 million in a Series B round led by Tiger Global with participation from Shasta Ventures and existing investor Accel, it announced today. Rather, it was the ability to scale the productivity of the people who work with data.

Data 230
article thumbnail

The Good and the Bad of Apache Airflow Pipeline Orchestration

Altexsoft

You can hardly compare data engineering toil with something as easy as breathing or as fast as the wind. The platform went live in 2015 at Airbnb, the biggest home-sharing and vacation rental site, as an orchestrator for increasingly complex data pipelines. How data engineering works. Airflow architecture.