The Citus Data

MARCH 31, 2023

A developer friend of mine prefers to read about what to expect at upcoming events in the narrative form of a blog, rather than having to click in and out of different abstracts on a schedule page. Additional IO Observability in Postgres with pg stat io , by Melanie Plageman on the Postgres Open Source team at Microsoft.

Azure 85

Azure Open Source Virtualization Software Engineering 85

Cloudera

SEPTEMBER 29, 2020

Performance is one of the key, if not the most important deciding criterion, in choosing a Cloud Data Warehouse service. In today’s fast changing world, enterprises have to make data driven decisions quickly and for that they rely heavily on their data warehouse service. . Cloudera Data Warehouse vs HDInsight.

Azure 116

Azure Data Comparison Virtualization 116

Cloudera

OCTOBER 5, 2020

Cloudera Data Platform Powered by NVIDIA RAPIDS Software Aims to Dramatically Increase Performance of the Data Lifecycle Across Public and Private Clouds. In his GTC 2020 keynote , NVIDIA CEO Jensen Huang revealed that NVIDIA and Cloudera are teaming up to accelerate the Cloudera Data Platform. with Spark 3.0

Enterprise 88

Enterprise Cloud Data Machine Learning 88

Cloudera

JUNE 18, 2022

As use cases matured, we saw the need for both efficient, interactive BI analytics and transactional semantics to modify data. The first generation of the Hive Metastore attempted to address the performance considerations to run SQL efficiently on a data lake. Iterations of the lakehouse. How are we embracing Iceberg?

Data 90

Data Analytics Open Source Architecture 90

CTOvision

JUNE 15, 2015

You know Spark, the free and open source complement to Apache Hadoop that gives enterprises better ability to field fast, unified applications that combine multiple workloads, including streaming over all your data. Spark is still new and frankly there have been issues with performance from time to time.

Open Source 117

Open Source Big Data Machine Learning Artificial Inteligence 117

Cloudera

JANUARY 19, 2022

We have been working hard to build our cloud-native data services on Cloudera Data Platform (CDP), which include CDP Data Warehouse, CDP Operational Database, CDP Machine Learning, CDP Data Engineering and CDP Data Flow. Download the reports to see the detailed scores .

Report 87

Report Technical Advisors Cloud Analytics 87

Cloudera

SEPTEMBER 21, 2021

Many customers looking at modernizing their pipeline orchestration have turned to Apache Airflow, a flexible and scalable workflow manager for data engineers. But now it has become very simple and secure with our release of the Cloudera Airflow provider , which gives users the best of Airflow and CDP data services. What’s Next.

Off-The-Shelf 95

Off-The-Shelf Data Engineering Virtualization Cloud 95

Confluent

FEBRUARY 6, 2019

Building a scalable, reliable and performant machine learning (ML) infrastructure is not easy. Uber expanded Michelangelo “to serve any kind of Python model from any source to support other Machine Learning and Deep Learning frameworks like PyTorch and TensorFlow [instead of just using Spark for everything].”.

Machine Learning 106

Machine Learning Artificial Inteligence Scalability Data Engineering 106

Altexsoft

AUGUST 25, 2021

But the same principle of calculating probability of word sequences can create language models that can perform impressive results in mimicking human speech. For example, tokenization (splitting text data into words) and part-of-speech tagging (labeling nouns, verbs, etc.) are successfully performed by rules. Speech recognition.

Tools 139

Tools Artificial Inteligence Technical Review Systems Review 139

Cloudera

FEBRUARY 25, 2021

They need strong data exploration and visualization skills, as well as sufficient data engineering chops to fix the gaps they find in their initial study. This AMP benchmarks multiple state of the art algorithms, with a front end for comparing their performance. Deep Learning for Image Analysis.

Machine Learning 82

Machine Learning Artificial Inteligence Telecommunications Enterprise 82

Netflix Tech

MARCH 5, 2019

As a micro-service owner, a Netflix engineer is responsible for its innovation as well as its operation, which includes making sure the service is reliable, secure, efficient and performant. In the Performance space, our data teams currently focus on the quality of experience on Netflix-enabled devices.

Infrastructure 73

Infrastructure Scalability Cloud Data 73

Cloudera

DECEMBER 10, 2020

Drawing on more than a decade of experience in building and deploying massive scale data platforms on economical budgets, Cloudera has designed and delivered a cost-cutting cloud-native solution – Cloudera Data Warehouse (CDW), part of the new Cloudera Data Platform (CDP). CDW is built for high performance.

Data 98

Data Technical Review Storage Systems Review 98

Cloudera

SEPTEMBER 1, 2020

Solr is a standard and open source, commonly adopted text search engine with rich query APIs for performing analytics over text and other unstructured data. For the examples presented in this blog, we assume you have a CDP account already. Before you Get Started.

Data 67

Data Backup Disaster Recovery Storage 67

AWS Machine Learning - AI

JANUARY 26, 2024

We are committed to expanding our capabilities to allow you to meet your digital sovereignty needs, without compromising on the performance, innovation, security, or scale of the AWS Cloud. Understand the security posture of your AI, ML, and generative AI workloads by performing a Well-Architected Framework review.

Artificial Inteligence 120

Artificial Inteligence Generative AI Security Applications 120

Xebia

FEBRUARY 25, 2024

In this blog post, we’ll try to demystify MLOps and take you through the process of going from a notebook to your very own industry-grade ML application. Data science is generally not operationalized Consider a data flow from a machine or process, all the way to an end-user. What is this thing called MLOps ?

Applications 130

Applications Technical Review Software Review Azure 130

Cloudera

MAY 22, 2018

With Experiments, data scientists can run a batch job that will: create a snapshot of model code, dependencies, and configuration parameters necessary to train the model. track model metrics, performance, and any model artifacts the user specifies. Data scientists often develop models using a variety of Python/R open source packages.

Data 41

Data Load Balancer Machine Learning Artificial Inteligence 41

Cloudera

MARCH 23, 2022

Please join us on March 24 for Future of Data meetup where we do a deep dive into Iceberg with CDP . Apache Iceberg is a high-performance, open table format, born-in-the cloud that scales to petabytes independent of the underlying storage layer and the access engine layer. 2: Open formats. 3: Open Performance.

Data 88

Data Open Source Storage Machine Learning 88

Cloudera

SEPTEMBER 9, 2022

analyst Sumit Pal, in “Exploring Lakehouse Architecture and Use Cases,” published January 11, 2022: “Data lakehouses integrate and unify the capabilities of data warehouses and data lakes, aiming to support AI, BI, ML, and data engineering on a single platform.” According to Gartner, Inc.

Architecture 97

Architecture Innovation Data Open Source 97

Datavail

JUNE 8, 2023

From DBA to Data Engineer—The Strategic Role of DBAs in the Cloud Over the past few years, the IT landscape has experienced significant disruptions. The roles and duties traditionally performed by DBAs have changed as cloud adoption and automation become commonplace. Plus, we gave away an awesome LEGO NASA Space Shuttle!

Database Administration 52

Database Administration Data Artificial Inteligence Analytics 52

Cloudera

OCTOBER 7, 2022

dbt allows data teams to produce trusted data sets for reporting, ML modeling, and operational workflows using SQL, with a simple workflow that follows software engineering best practices like modularity, portability, and continuous integration/continuous development (CI/CD). Introduction. dbt-impala . dbt-spark-livy.

SDLC 79

SDLC Data Open Source Handbook 79

Cloudera

NOVEMBER 19, 2020

This type of framework requires a streaming environment that provides continuous updates coupled with model performance monitoring to ensure consistent performance. . The Public Sector data challenge. Cloudera Data Platform (CDP) is a solution that integrates open-source tools with security and cloud compatibility.

Government 115

Government Artificial Inteligence Machine Learning Data 115

Cloudera

FEBRUARY 22, 2022

This unprecedented level of big data workloads hasn’t come without its fair share of challenges. The data architecture layer is one such area where growing datasets have pushed the limits of scalability and performance. Apache Iceberg is a new open table format targeted for petabyte-scale analytic datasets.

Data 105

Data Analytics Disaster Recovery Travel 105

Altexsoft

NOVEMBER 7, 2022

You can hardly compare data engineering toil with something as easy as breathing or as fast as the wind. The platform went live in 2015 at Airbnb, the biggest home-sharing and vacation rental site, as an orchestrator for increasingly complex data pipelines. How data engineering works. Source: Apache Airflow.

Weak Development Team 52

Weak Development Team Technical Review Software Review Systems Review 52

Cloudera

AUGUST 24, 2022

It unifies self-service data science and data engineering in a single, portable service as part of an enterprise data cloud for multi-function analytics on data anywhere. Data professionals who use CML spend the vast majority of their time in an isolated compute session that comes pre-loaded with an editor UI.

Machine Learning 79

Machine Learning Artificial Inteligence Open Source Windows 79

Cloudera

AUGUST 17, 2021

When we announced the GA of Cloudera Data Engineering back in September of last year, a key vision we had was to simplify the automation of data transformation pipelines at scale. And by being purely python based, Apache Airflow pipelines are accessible to a wide range of users, with a strong open source community.

Data 111

Data Open Source Government Analytics 111

Cloudera

JULY 13, 2023

A range of Iceberg table analysis such as listing table’s data file, selecting table snapshot, partition filtering, and predicate filtering can be delegated through Iceberg Java API instead, obviating the need for each query engine to implement it themself. However, Iceberg Java API calls are not always cheap.

Weak Development Team 69

Weak Development Team Engineering Analytics Storage 69

Xebia

JANUARY 9, 2024

This gives some high-level information, however it is hard to determine what the actual SQL is that was executed, and on which exact records a data quality test failed or raised a warning. Logs will purely show information about a single run, so you don’t have an easy way to see how a test performs over time.

Testing 130

Testing Data Open Source Google Cloud 130

Cloudera

DECEMBER 20, 2023

We see AI applications like chatbots being built on top of closed-source or open source foundational models. Those models are trained or augmented with data from a data management platform. The data management platform, models, and end applications are powered by cloud infrastructure and/or specialized hardware.

Artificial Inteligence 70

Artificial Inteligence Artificial Intelligence Enterprise Machine Learning 70

O'Reilly Media - Ideas

JANUARY 25, 2024

Rachel Stephens provides two fascinating pieces of the puzzle in a recent article on the RedMonk blog , but those pieces don’t fit together exactly. Our own theory is that it’s a reaction to GPT models leaking proprietary code and abusing open source licenses; that could cause programmers to be wary of public code repositories.

Trends 119

Trends Technical Review Technology Artificial Inteligence 119

Cloudera

JUNE 30, 2022

Today’s general availability announcement covers Iceberg running within key data services in the Cloudera Data Platform (CDP) — including Cloudera Data Warehousing ( CDW ), Cloudera Data Engineering ( CDE ), and Cloudera Machine Learning ( CML ). Read why the future of data lakehouses is open.

Data 84

Data Analytics Machine Learning Artificial Inteligence 84

Altexsoft

FEBRUARY 10, 2021

Similar to how DevOps once reshaped the software development landscape, another evolving methodology, DataOps, is currently changing Big Data analytics — and for the better. DataOps is a relatively new methodology that knits together data engineering, data analytics, and DevOps to deliver high-quality data products as fast as possible.

Analytics 75

Analytics DevOps Development Software Review 75

Cloudera

APRIL 26, 2023

Application modernization initiatives have led to cloud native architectures gaining popularity on premises, making it a sensible choice to extend to your data platform. It offers features such as data ingestion, storage, ETL, BI and analytics, observability, and AI model development and deployment. What is cloud native exactly?

Cloud 63

Cloud Data Load Balancer Storage 63

Cloudera

APRIL 3, 2023

In this blog, we will share with you in detail how Cloudera integrates core compute engines including Apache Hive and Apache Impala in Cloudera Data Warehouse with Iceberg. We will publish follow up blogs for other data services. Iceberg basics Iceberg is an open table format designed for large analytic workloads.

Data 79

Data Travel Analytics Virtualization 79

OpenCredo

JANUARY 26, 2023

The second step is to extract the data into a staging area (which can be sourced from a combination of multiple sources, but in our case, we simply have a single “big” dataset). As the last step (for this blogpost) we will load the data into a graph database where you can do further transformations and analysis as you like.

Big Data 52

Big Data Data Software Engineering Data Engineering 52

Cloudera

APRIL 22, 2021

Storage plays one of the most important roles in the data platforms strategy, it provides the basis for all compute engines and applications to be built on top of it. Businesses are also looking to move to a scale-out storage model that provides dense storages along with reliability, scalability, and performance.

Data 98

Data Storage Architecture Big Data 98

Altexsoft

OCTOBER 21, 2022

Similar to Google in web browsing and Photoshop in image processing, it became a gold standard in data streaming, preferred by 70 percent of Fortune 500 companies. Apache Kafka is an open-source, distributed streaming platform for messaging, storing, processing, and integrating large data volumes in real time.

Weak Development Team 93

Weak Development Team Technical Review Systems Review Software Review 93

Cloudera

AUGUST 20, 2021

The Cloudera Data Platform comprises a number of ‘data experiences’ each delivering a distinct analytical capability using one or more purposely-built Apache open source projects such as Apache Spark for Data Engineering and Apache HBase for Operational Database workloads. A Robust Security Framework.

Strategy 94

Strategy Data Technical Review Weak Development Team 94