Remove apache-kafka-deployments-and-systems-reliability-part-1
article thumbnail

Machine Learning with Python, Jupyter, KSQL and TensorFlow

Confluent

Building a scalable, reliable and performant machine learning (ML) infrastructure is not easy. It allows real-time data ingestion, processing, model deployment and monitoring in a reliable and scalable way. It takes much more effort than just building an analytic model with Python and your favorite machine learning framework.

article thumbnail

Apache Kafka Deployments and Systems Reliability – Part 1

Cloudera

There are many ways that Apache Kafka has been deployed in the field. In our Kafka Summit 2021 presentation, we took a brief overview of many different configurations that have been observed to date. Serial and Parallel Systems Reliability . Serial Systems Reliability.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

Data governance beyond SDX: Adding third party assets to Apache Atlas

Cloudera

While Cloudera Data Platform (CDP) already supports the entire data lifecycle from ‘Edge to AI’, we at Cloudera are fully aware that enterprises have more systems outside of CDP. Apache Atlas as a fundamental part of SDX. can be part of a superior class, allowing the creation of a tree-like, structured storage for assets.

article thumbnail

Scaling Kafka at Honeycomb

Honeycomb

For the entirety of Honeycomb’s existence, we have used Apache Kafka to perform this buffering function in our observability pipeline. A brief overview of Kafka at Honeycomb. How does Kafka help us achieve these results? Adapting Kafka for our use case. Adapting Kafka for our use case. Let’s dive in.

AWS 145
article thumbnail

Microservices, Apache Kafka, and Domain-Driven Design

Confluent

Microservices have a symbiotic relationship with domain-driven design (DDD)—a design approach where the business domain is carefully modeled in software and evolved over time, independently of the plumbing that makes the system work. I see this pattern coming up more and more in the field in conjunction with Apache Kafka ®.

article thumbnail

Building a Scalable Search Architecture

Confluent

It involves many moving parts, from data preparation to building indexing and query pipelines. It also requires both systems to always be available, so no maintenance windows are possible. Distributed transactions are very hard to implement successfully, which is why we’ll introduce a log-inspired system such as Apache Kafka ®.

article thumbnail

Reliable Data Exchange with the Outbox Pattern and Cloudera DiM

Cloudera

In this post, I will demonstrate how to use the Cloudera Data Platform (CDP) and its streaming solutions to set up reliable data exchange in modern applications between high-scale microservices, and ensure that the internal state will stay consistent even under the highest load. Introduction Many modern application designs are event-driven.

Data 87