Cloudera

OCTOBER 11, 2021

Modak, a leading provider of modern data engineering solutions, is now a certified solution partner with Cloudera. Customers can now seamlessly automate migration to Cloudera’s Hybrid Data Platform — Cloudera Data Platform (CDP) to dynamically auto-scale cloud services with Cloudera Data Engineering (CDE) integration with Modak Nabu.

Data Engineering 91

Data Engineering Engineering Data Cloud 91

Cloudera

SEPTEMBER 17, 2020

With growing disparate data across everything from edge devices to individual lines of business needing to be consolidated, curated, and delivered for downstream consumption, it’s no wonder that data engineering has become the most in-demand role across businesses — growing at an estimated rate of 50% year over year.

Data Engineering 102

Data Engineering Engineering Data Tools 102

Mobilunity

AUGUST 3, 2021

The cloud offers excellent scalability, while graph databases offer the ability to display incredible amounts of data in a way that makes analytics efficient and effective. Who is Big Data Engineer? Big Data requires a unique engineering approach. Big Data Engineer vs Data Scientist.

Big Data 52

Big Data Data Engineering Engineering Data 52

Netflix Tech

DECEMBER 21, 2020

At this scale, we can gain a significant amount of performance and cost benefits by optimizing the storage layout (records, objects, partitions) as the data lands into our warehouse. We built AutoOptimize to efficiently and transparently optimize the data and metadata storage layout while maximizing their cost and performance benefits.

Storage 81

Storage Data Resources Data Engineering 81

Cloudera

JANUARY 12, 2018

I mentioned in an earlier blog titled, “Staffing your big data team, ” that data engineers are critical to a successful data journey. That said, most companies that are early in their journey lack a dedicated engineering group. Image 1: Data Engineering Skillsets.

Data Engineering 54

Data Engineering Big Data Engineering Data 54

Cloudera

FEBRUARY 12, 2024

In this blog, we’ll take you through our tried and tested best practices for setting up your DNS for use with Cloudera on Azure. In a Cloudera deployment scenario, only storage accounts, PostgreSQL DB, and Key Vault support service endpoints. There is at present no option to disable private DNS zones for data engineering.

Azure 52

Azure Firewall Data Engineering Storage 52

Perficient

FEBRUARY 16, 2024

In the world of big data processing, efficient and scalable file systems play a crucial role. In this blog post, we’ll explore into what DBFS is, how it works, and provide examples to illustrate its usage. DBFS provides a unified interface to access data stored in various underlying storage systems. What is DBFS?

System 52

System Storage Azure Big Data 52

Cloudera

APRIL 1, 2024

Cloudera, a leader in big data analytics, provides a unified Data Platform for data management, AI, and analytics. Our customers run some of the world’s most innovative, largest, and most demanding data science, data engineering, analytics, and AI use cases, including PB-size generative AI workloads.

Artificial Inteligence 102

Artificial Inteligence Cloud Generative AI Analytics 102

CIO

FEBRUARY 14, 2023

Organizations have balanced competing needs to make more efficient data-driven decisions and to build the technical infrastructure to support that goal. However, a data execution strategy has to evolve for real-time AI to scale with speed. Learn more about moving to cloud-native applications and data with Kubernetes in this blog post.)

Artificial Inteligence 287

Artificial Inteligence Strategy Data Machine Learning 287

Cloudera

NOVEMBER 28, 2023

Cloudera Private Cloud Data Services is a comprehensive platform that empowers organizations to deliver trusted enterprise data at scale in order to deliver fast, actionable insights and trusted AI. This means you can expect simpler data management and drastically improved productivity for your business users.

Data 81

Data Scalability Analytics Quality Assurance 81

Tenable

NOVEMBER 3, 2022

When our data engineering team was enlisted to work on Tenable One, we knew we needed a strong partner. When Tenable’s product engineering team came to us in data engineering asking how we could build a data platform to power the product, we knew we had an incredible opportunity to modernize our data stack.

Data 98

Data AWS Storage Data Engineering 98

Cloudera

SEPTEMBER 28, 2021

The Ranger Authorization Service (RAZ) is a new service added to help provide fine-grained access control (FGAC) for cloud storage. We covered the value this new capability provides in a previous blog. Let’s say that both Jon and Remi belong to the Data Engineering group. Adding a user to multiple groups.

Groups 62

Groups Cloud AWS Data 62

Cloudera

JULY 11, 2023

In this blog we will explain how to integrate them together to achieve separation of compute (i.e. Impala) and storage (i.e. This also enables advanced scenarios where customers can connect multiple CDW Virtual Clusters to different real-time data mart clusters to connect to a Kudu cluster specific for their workloads.

Data 70

Data Virtualization Pharmaceuticals Storage 70

Cloudera

JUNE 26, 2023

Here are some tips and tricks of the trade to prevent well-intended yet inappropriate data engineering and data science activities from cluttering or crashing the cluster. For data engineering and data science teams, CDSW is highly effective as a comprehensive platform that trains, develops, and deploys machine learning models.

Tools 76

Tools Data Engineering Analytics Testing 76

Cloudera

APRIL 26, 2023

Application modernization initiatives have led to cloud native architectures gaining popularity on premises, making it a sensible choice to extend to your data platform. At its core, CDP Private Cloud Data Services (“the platform”) is an end-to-end cloud native platform that provides a private open data lakehouse.

Cloud 64

Cloud Data Load Balancer Storage 64

Altexsoft

FEBRUARY 11, 2023

Data architecture is the organization and design of how data is collected, transformed, integrated, stored, and used by a company. If you are not familiar with the above-mentioned concepts, we suggest you to follow the links above to learn more about each of them in our blog posts. Feel free to enjoy it.

Data 87

Data Data Engineering Big Data Architecture 87

Cloudera

APRIL 22, 2021

Today’s enterprise data analytics teams are constantly looking to get the best out of their platforms. Storage plays one of the most important roles in the data platforms strategy, it provides the basis for all compute engines and applications to be built on top of it. Supports Disaggregation of compute and storage.

Data 99

Data Storage Architecture Big Data 99

Cloudera

FEBRUARY 9, 2021

Cloudera Operational Database is now available in three different form-factors in Cloudera Data Platform (CDP). . If you are new to Cloudera Operational Database, see this blog post. In this blog post, we’ll look at both Apache HBase and Apache Phoenix concepts relevant to developing applications for Cloudera Operational Database.

Applications 102

Applications Development Storage Database Administration 102

Cloudera

FEBRUARY 1, 2024

The other half of the equation requires your team’s emphasis to shift to sustained excellence in managing and optimizing your data ecosystem — better known as Day 2 operations. In this blog, we’ll cover the highlights of our recently published Day 2 Operations Guide and why it matters to enterprises.

Backup 82

Backup Cloud Architecture Resources 82

Cloudera

MAY 18, 2021

Second, since IaaS deployments replicated the on-premises HDFS storage model, they resulted in the same data replication overhead in the cloud (typical 3x), something that could have mostly been avoided by leveraging modern object store. Storage costs. using list pricing of $0.72/hour hour using a r5d.4xlarge

Cloud 84

Cloud Technical Review Storage Backup 84

InnovationM

MAY 14, 2023

In this article, I will share practical insights and technologies utilized in building and harnessing the potential of data lakes. Demystifying Data Lakes Data lakes serve as flexible storage repositories, enabling organizations to store raw and diverse data types, breaking away from the constraints of traditional data warehouses.

Data 52

Data Storage Construction Business Intelligence 52

Perficient

FEBRUARY 28, 2023

This is the final blog in a series that explains how organizations can prevent their Data Lake from becoming a Data Swamp, with insights and strategy from Perficient’s Senior Data Strategist and Solutions Architect, Dr. Chuck Brooks. Once data is in the Data Lake, the data can be made available to anyone.

Data 110

Data Analytics Google Cloud Cloud 110

Mobilunity

MAY 8, 2023

Key zones of an Enterprise Data Lake Architecture typically include ingestion zone, storage zone, processing zone, analytics zone, and governance zone. Ingestion zone is where data is collected from various sources and ingested into the data lake. Storage zone is where the raw data is stored in its original format.

Engineering 52

Engineering Data Storage AWS 52

Cloudera

NOVEMBER 1, 2022

In the private sector, excluding highly regulated industries like financial services, the migration to the public cloud was the answer to most IT modernization woes, especially those around data, analytics, and storage. The post When Private Cloud is the Right Fit for Public Sector Missions appeared first on Cloudera Blog.

Cloud 86

Cloud Government Storage Analytics 86

Cloudera

MAY 17, 2021

CDP (Cloudera Data Platform) Private Cloud 1.2 release blog ). In this blog, we’ll cover the complete range of new capabilities and updates for CDP Private Cloud as a whole (the platform) as well as for both the CDW (Cloudera Data Warehouse) and CML (Cloudera Machine Learning) services. Platform – In-place Updates.

Cloud 89

Cloud Machine Learning Artificial Inteligence Storage 89

Cloudera

JULY 14, 2023

Depending on the size and usage patterns of the data, several different strategies could be pursued to achieve a successful migration. In this blog, I will describe a few strategies one could undertake for various use cases. This will be discussed in a later blog. Relatively fast as the underlying data files are kept in place.

Backup 70

Backup Data Engineering Engineering Data 70

Perficient

APRIL 1, 2024

Traditionally, organizations used to provision multiple services of Azure Services, like Azure Storage, Azure Databricks, etc. This data needs to be ingested on a real-time basis into the warehouse, for further usage by the data scientist & data analyst team. Terminologies Used Below are few terms used in the blog.

Data 64

Data Azure Case Study Data Engineering 64

Cloudera

JANUARY 15, 2021

Some examples of recent optimizations in Impala include: New multithreading model (see dedicated blog post ). Remote read optimizations: IMPALA-8341: Data cache for remote reads. Impala use of KRPC (see dedicated blog post ). Parquet page indexes (see dedicated blog post ). IMPALA-8690: Add LIRS cache eviction algorithm.

Performance 93

Performance Cloud Data Storage 93

Cloudera

SEPTEMBER 14, 2020

Everybody needs more data and more analytics, with so many different and sometimes often conflicting needs. Data engineers need batch resources, while data scientists need to quickly onboard ephemeral users. The post Fundamentals for Success in Cloud Data Management appeared first on Cloudera Blog.

Cloud 108

Cloud Data Compliance Government 108

Cloudera

MARCH 23, 2022

Please join us on March 24 for Future of Data meetup where we do a deep dive into Iceberg with CDP . Apache Iceberg is a high-performance, open table format, born-in-the cloud that scales to petabytes independent of the underlying storage layer and the access engine layer. What is Apache Iceberg? 1: Multi-function analytics .

Data 90

Data Open Source Storage Machine Learning 90

Cloudera

AUGUST 26, 2020

Informatica and Cloudera deliver a proven set of solutions for rapidly curating data into trusted information. Informatica’s comprehensive suite of Data Engineering solutions is designed to run natively on Cloudera Data Platform — taking full advantage of the scalable computing platform.

Data 81

Data Machine Learning Artificial Inteligence Disaster Recovery 81

Cloudera

OCTOBER 13, 2021

These companies have instead opted to leverage their existing data centre investment. In addition to the Cloudera Data Warehouse (CDW) and Cloudera Machine Learning (CML), CDP Private Cloud now also includes Cloudera Data Engineering (CDE). Breaking down the on-prem monolith.

Data Center 95

Data Center Cloud Data How To 95

Cloudera

SEPTEMBER 21, 2021

Many customers looking at modernizing their pipeline orchestration have turned to Apache Airflow, a flexible and scalable workflow manager for data engineers. A provider could be used to make HTTP requests, connect to a RDBMS, check file systems (such as S3 object storage), invoke cloud provider services, and much more.

Off-The-Shelf 96

Off-The-Shelf Data Engineering Virtualization Cloud 96

Altexsoft

MARCH 31, 2020

It means you must collect transactional data and move it from the database that supports transactions to another system that can handle large volumes of data. And, as is common, to transform it before loading to another storage system. But how do you move data? The simplest illustration for a data pipeline.

Data 76

Data Storage Analytics Data Center 76

Cloudera

JUNE 30, 2022

Today’s general availability announcement covers Iceberg running within key data services in the Cloudera Data Platform (CDP) — including Cloudera Data Warehousing ( CDW ), Cloudera Data Engineering ( CDE ), and Cloudera Machine Learning ( CML ). Read why the future of data lakehouses is open.

Data 85

Data Analytics Machine Learning Artificial Inteligence 85

Cloudera

SEPTEMBER 29, 2020

Performance is one of the key, if not the most important deciding criterion, in choosing a Cloud Data Warehouse service. In today’s fast changing world, enterprises have to make data driven decisions quickly and for that they rely heavily on their data warehouse service. . Cloudera Data Warehouse vs HDInsight.

Azure 117

Azure Data Comparison Virtualization 117

Cloudera

JANUARY 6, 2021

Python is used extensively among Data Engineers and Data Scientists to solve all sorts of problems from ETL/ELT pipelines to building machine learning models. Apache HBase is an effective data storage system for many workflows but accessing this data specifically through Python can be a struggle.

Machine Learning 132

Machine Learning Artificial Inteligence Data Applications 132