Cloudera

JUNE 26, 2023

Here are some tips and tricks of the trade to prevent well-intended yet inappropriate data engineering and data science activities from cluttering or crashing the cluster. For data engineering and data science teams, CDSW is highly effective as a comprehensive platform that trains, develops, and deploys machine learning models.

Tools 75

Tools Data Engineering Analytics Testing 75

The Citus Data

MARCH 31, 2023

A developer friend of mine prefers to read about what to expect at upcoming events in the narrative form of a blog, rather than having to click in and out of different abstracts on a schedule page. Additional IO Observability in Postgres with pg stat io , by Melanie Plageman on the Postgres Open Source team at Microsoft.

Azure 85

Azure Open Source Virtualization Software Engineering 85

Cloudera

OCTOBER 5, 2020

Cloudera Data Platform Powered by NVIDIA RAPIDS Software Aims to Dramatically Increase Performance of the Data Lifecycle Across Public and Private Clouds. This exciting initiative is built on our shared vision to make data-driven decision-making a reality for every business. Compared to previous CPU-based architectures, CDP 7.1

Enterprise 88

Enterprise Cloud Data Machine Learning 88

Xebia

FEBRUARY 25, 2024

In this blog post, we’ll try to demystify MLOps and take you through the process of going from a notebook to your very own industry-grade ML application. Data science is generally not operationalized Consider a data flow from a machine or process, all the way to an end-user. What is this thing called MLOps ?

Applications 130

Applications Technical Review Software Review Azure 130

CTOvision

JUNE 15, 2015

You know Spark, the free and open source complement to Apache Hadoop that gives enterprises better ability to field fast, unified applications that combine multiple workloads, including streaming over all your data. They also launched a plan to train over a million data scientists and data engineers on Spark.

Open Source 117

Open Source Big Data Machine Learning Artificial Inteligence 117

Cloudera

JULY 27, 2021

Data has become the critical fuel that drives growth and automation, the twin guide points for an industry stretching for new revenue sources and cost reduction, all the while maintaining governance on precious customer data. Edge analytics by definition require in-network deployment.

Telecommunications 61

Telecommunications Data Architecture Big Data 61

O'Reilly Media - Data

JULY 10, 2018

Data scientists, data engineers, AI and ML developers, and other data professionals need to live ethical values, not just talk about them. The hard thing about being an ethical data scientist isn’t understanding ethics. It’s doing good data science. How do we manage any sensitive data that we acquire?

Data 173

Data Weak Development Team Software Review Culture 173

Cloudera

FEBRUARY 25, 2021

While there are many factors that can contribute to this inefficiency, one of the most prevalent hurdles to overcome has to do with simply getting projects off the ground and selecting the right approaches, algorithms, and applications that will lead to fast results and trustworthy decision making. .

Machine Learning 82

Machine Learning Artificial Inteligence Telecommunications Enterprise 82

ParkMyCloud

DECEMBER 11, 2020

If you know where to look, open-source learning is a great way to get familiar with different cloud service providers. . With the combined knowledge from our previous blog posts on free training resources for AWS and Azure , you’ll be well on your way to expanding your cloud expertise and finding your own niche.

Google Cloud 97

Google Cloud Training Resources Cloud 97

Confluent

FEBRUARY 6, 2019

Uber expanded Michelangelo “to serve any kind of Python model from any source to support other Machine Learning and Deep Learning frameworks like PyTorch and TensorFlow [instead of just using Spark for everything].”. It allows real-time data ingestion, processing, model deployment and monitoring in a reliable and scalable way.

Machine Learning 106

Machine Learning Artificial Inteligence Scalability Data Engineering 106

Cloudera

SEPTEMBER 1, 2020

It is hard to believe if you have had previous experience with setting up, sizing, and deploying a distributed search engine service that this is possible. Imagine how many times IT has lost valuable time spending hours trying to understand Apache Solr application requirements and map them into how to best size and deploy the Solr service.

Data 67

Data Backup Disaster Recovery Storage 67

Altexsoft

AUGUST 25, 2021

Some common applications of text classification include the following. This is not an exhaustive list of all NLP use cases by far, but it paints a clear picture of its diverse applications. NLP techniques open tons of opportunities for human-machine interactions that we’ve been exploring for decades. Assessing text data quality.

Tools 139

Tools Artificial Inteligence Technical Review Systems Review 139

Netflix Tech

MARCH 5, 2019

While our engineering teams have and continue to build solutions to lighten this cognitive load (better guardrails, improved tooling, …), data and its derived products are critical elements to understanding, optimizing and abstracting our infrastructure. Give us a holler if you are interested in a thought exchange.

Infrastructure 73

Infrastructure Scalability Cloud Data 73

Erik Bernhardsson

OCTOBER 27, 2015

Anyway, reposting the full interview: As part of my interviews with Data Scientists I recently caught up with Erik Bernhardsson who is famous in the world of ‘Big Data’ for his open source contributions, his leading of teams at Spotify, and his various talks at various conferences.

Data 100

Data Big Data Machine Learning Artificial Inteligence 100

Erik Bernhardsson

OCTOBER 27, 2015

Anyway, reposting the full interview: As part of my interviews with Data Scientists I recently caught up with Erik Bernhardsson who is famous in the world of ‘Big Data’ for his open source contributions, his leading of teams at Spotify, and his various talks at various conferences.

Data 100

Data Big Data Machine Learning Artificial Inteligence 100

Kentik

OCTOBER 2, 2017

Open Connections Can Build Powerful Combinations. The Application Programming Interface ( API ) has been around since the 1960s, but in today’s world of heterogeneous environments and distributed systems it’s become indispensable. For an easy-to-understand example, take the “Skills” that are available for Amazon’s Alexa.

Open Source 40

Open Source Metrics Network System 40

Altexsoft

MARCH 16, 2021

In this article, we will explain the concept and usage of Big Data in the healthcare industry and talk about its sources, applications, and implementation challenges. Definitely, the topic is way too extensive to be covered in a blog post, so we’re only going to make a succinct overview. Staffing problems.

Big Data 116

Big Data Healthcare Applications Data 116

Cloudera

MARCH 23, 2022

Through Cloudera’s contributions, we have extended support for Hive and Impala, delivering on the vision of a data architecture for multi-function analytics from large scale data engineering (DE) workloads and stream processing (DF) to fast BI and querying (within DW) and machine learning (ML). . 2: Open formats.

Data 88

Data Open Source Storage Machine Learning 88

O'Reilly Media - Ideas

JANUARY 25, 2024

Who wants to learn about design patterns or software architecture when some AI application may eventually do your high-level design? Whether it will be able to do high-level design is an open question—but as always, that question has two sides: “Will AI do our design work?” This change is apparently not an error in the data.

Trends 119

Trends Technical Review Technology Artificial Inteligence 119

Xebia

JANUARY 9, 2024

To solve this problem it is possible to build a subsequent step in the form of a script or application to parse these artifact files and store them somewhere. However, this requires a lot of custom engineering work and is not an easy task. Luckily, there is an open-source solution for this called Elementary Data.

Testing 130

Testing Data Open Source Google Cloud 130

Cloudera

OCTOBER 7, 2022

dbt allows data teams to produce trusted data sets for reporting, ML modeling, and operational workflows using SQL, with a simple workflow that follows software engineering best practices like modularity, portability, and continuous integration/continuous development (CI/CD). Introduction. dbt-impala . dbt-spark-livy.

SDLC 79

SDLC Data Open Source Handbook 79

Datavail

JUNE 8, 2023

From DBA to Data Engineer—The Strategic Role of DBAs in the Cloud Over the past few years, the IT landscape has experienced significant disruptions. Additionally, emerging data types and modern applications drive enterprises to adopt new data platforms. Plus, we gave away an awesome LEGO NASA Space Shuttle!

Database Administration 52

Database Administration Data Artificial Inteligence Analytics 52

Cloudera

OCTOBER 19, 2023

Integrating these capabilities into a data platform gives telcos the flexibility to navigate changing conditions while enforcing data security, compliance with regulations, and delivering novel products. However, Cloudera Data Platform (CDP) is a best-in-class platform that is 100 percent compliant with upstream OSS projects.

Data 73

Data Compliance Architecture Data Engineering 73

Cloudera

DECEMBER 20, 2023

In this post we’ll give you an overview of these new and expanded partnerships and how we see them fitting into the emerging AI technology stack that supports the AI application lifecycle. We see AI applications like chatbots being built on top of closed-source or open source foundational models.

Artificial Inteligence 70

Artificial Inteligence Artificial Intelligence Enterprise Machine Learning 70

Cloudera

NOVEMBER 19, 2020

Cloudera Data Platform (CDP) is a solution that integrates open-source tools with security and cloud compatibility. Governance: With a unified data platform, government agencies can apply strict and consistent enterprise-level data security, governance, and control across all environments.

Government 115

Government Artificial Inteligence Machine Learning Data 115

Cloudera

APRIL 26, 2023

Can you achieve similar outcomes with your on-premises data platform? Application modernization initiatives have led to cloud native architectures gaining popularity on premises, making it a sensible choice to extend to your data platform. You absolutely can. What is cloud native exactly?

Cloud 63

Cloud Data Load Balancer Storage 63

Altexsoft

MAY 14, 2021

Variety is the vector showing the diversity of Big Data. This data isn’t just about structured data that resides within relational databases as rows and columns. It comes in all sorts of forms that differ from one application to another, and most of Big Data is unstructured. Source: phoenixNAP.

Big Data 78

Big Data Analytics Tools Applications 78

Altexsoft

NOVEMBER 7, 2022

You can hardly compare data engineering toil with something as easy as breathing or as fast as the wind. The platform went live in 2015 at Airbnb, the biggest home-sharing and vacation rental site, as an orchestrator for increasingly complex data pipelines. How data engineering works. Source: Apache Airflow.

Weak Development Team 52

Weak Development Team Technical Review Software Review Systems Review 52

Cloudera

AUGUST 24, 2022

It unifies self-service data science and data engineering in a single, portable service as part of an enterprise data cloud for multi-function analytics on data anywhere. Data professionals who use CML spend the vast majority of their time in an isolated compute session that comes pre-loaded with an editor UI.

Machine Learning 79

Machine Learning Artificial Inteligence Open Source Windows 79

Cloudera

AUGUST 17, 2021

When we announced the GA of Cloudera Data Engineering back in September of last year, a key vision we had was to simplify the automation of data transformation pipelines at scale. And by being purely python based, Apache Airflow pipelines are accessible to a wide range of users, with a strong open source community.

Data 111

Data Open Source Government Analytics 111

Altexsoft

FEBRUARY 10, 2021

Similar to how DevOps once reshaped the software development landscape, another evolving methodology, DataOps, is currently changing Big Data analytics — and for the better. DataOps is a relatively new methodology that knits together data engineering, data analytics, and DevOps to deliver high-quality data products as fast as possible.

Analytics 75

Analytics DevOps Development Software Review 75

Cloudera

JUNE 30, 2022

Today’s general availability announcement covers Iceberg running within key data services in the Cloudera Data Platform (CDP) — including Cloudera Data Warehousing ( CDW ), Cloudera Data Engineering ( CDE ), and Cloudera Machine Learning ( CML ). Read why the future of data lakehouses is open.

Data 84

Data Analytics Machine Learning Artificial Inteligence 84

Cloudera

APRIL 3, 2023

In this blog, we will share with you in detail how Cloudera integrates core compute engines including Apache Hive and Apache Impala in Cloudera Data Warehouse with Iceberg. We will publish follow up blogs for other data services. Iceberg basics Iceberg is an open table format designed for large analytic workloads.

Data 79

Data Travel Analytics Virtualization 79

Confluent

FEBRUARY 22, 2019

Our quickly expanding business also means our platform needs to keep ahead of the curve to accommodate the ever-growing volumes of data and increasing complexity of our systems. The Deliveroo Engineering organisation is in the process of decomposing a monolith application into a suite of microservices.

Data 79

Data Software Review Weak Development Team Systems Review 79

Cloudera

APRIL 22, 2021

Today’s enterprise data analytics teams are constantly looking to get the best out of their platforms. Storage plays one of the most important roles in the data platforms strategy, it provides the basis for all compute engines and applications to be built on top of it. Cisco Data Intelligence Platform.

Data 98

Data Storage Architecture Big Data 98

Cloudera

OCTOBER 14, 2020

Kubernetes offers a simplified way to manage infrastructure and applications with a practical approach to isolate workloads, limit the use of resources, deploy on-demand resources, and auto-scaling capabilities as needed. Lack of first-class application concept . Application concept. Natively support Big Data workloads.

Policies 114

Policies Resources Systems Review Technical Review 114

BPM

JANUARY 18, 2019

So BPM is today another form of low-code application development. Expect true zero code delivery, best of breed open source, 100% event-driven architecture, 100% microservices based, SDKs in all major languages, and a simple, fast, sleek new design. All of them virtually not true. It will be wrong for most of its users.

Artificial Inteligence 111

Artificial Inteligence Artificial Intelligence Machine Learning Weak Development Team 111