article thumbnail

How to Use Apache Iceberg in CDP’s Open Lakehouse

Cloudera

The general availability covers Iceberg running within some of the key data services in CDP, including Cloudera Data Warehouse ( CDW ), Cloudera Data Engineering ( CDE ), and Cloudera Machine Learning ( CML ). Cloudera Data Engineering (Spark 3) with Airflow enabled. 8 2001 5967780. 1 2008 7009728.

How To 91
article thumbnail

How to Save Time and Money by Testing Spark Locally

Xebia

Data Engineers were tempted by the pressure of the moment to give up on testing all together. There was no need for generating your own data; just take a percentage of production data. In many cases, these tasks ended up on the shoulders of the Data Engineers themselves. Overly restrictive governance.

Testing 130
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

Technology Trends for 2024

O'Reilly Media - Ideas

Before that, cloud computing itself took off in roughly 2010 (AWS was founded in 2006); and Agile goes back to 2000 (the Agile Manifesto dates back to 2001, Extreme Programming to 1999). Rust is a relatively young language that stresses memory safety and performance. 2023 was one of those rare disruptive years.

Trends 119
article thumbnail

NetFlow, sFlow, and Flow Extensibility, Part 2

Kentik

In this post we’ll look at how sFlow works compared to NetFlow, and then consider where flow data protocols are headed next. sFlow, which has been available in switches and routers since 2001, is the brainchild of InMon Corporation, whose continued control over the protocol is both benevolent and absolute. The sFlow difference.

article thumbnail

Technology Trends for 2022

O'Reilly Media - Ideas

A quick look at bigram usage (word pairs) doesn’t really distinguish between “data science,” “data engineering,” “data analysis,” and other terms; the most common word pair with “data” is “data governance,” followed by “data science.” All three are now coming onto the map. So what can we conclude?

Trends 111