Cloudera

MARCH 14, 2023

We just announced the general availability of Cloudera DataFlow Designer , bringing self-service data flow development to all CDP Public Cloud customers. In our previous DataFlow Designer blog post , we introduced you to the new user interface and highlighted its key capabilities.

Agile 80

Agile Development Data Serverless 80

Confluent

AUGUST 27, 2019

We are excited to announce the release of Confluent Cloud Schema Registry in general availability (GA), available in Confluent Cloud , our fully managed event streaming service based on Apache Kafka ®. Confluent Schema Registry provides a serving layer for your metadata and a RESTful interface for storing and retrieving Avro schemas.

Cloud 18

Cloud Product Management AWS Storage 18

Confluent

JULY 17, 2019

Together, MongoDB and Apache Kafka ® make up the heart of many modern data architectures today. Integrating Kafka with external systems like MongoDB is best done though the use of Kafka Connect. The official MongoDB Connector for Apache Kafka is developed and supported by MongoDB engineers. Getting started.

eCommerce 21

eCommerce Architecture System Guidelines 21

Cloudera

MARCH 11, 2022

With the launch of CDP Public Cloud 7.2.14, Cloudera Streams Messaging for Data Hub deployments has gotten some powerful new features! In this release , the Streams Messaging templates in Data Hub will come with Apache Kafka 2.8 Kafka & Cruise Control Updates. Kafka Updates: Deployments with Kafka 2.5

Cloud 100

Cloud Policies Infrastructure Authentication 100

Confluent

MAY 21, 2019

The profile service will publish the changes in profiles, including address changes to an Apache Kafka ® topic, and the quote service will subscribe to the updates from the profile changes topic, calculate a new quote if needed and publish the new quota to a Kafka topic so other services can subscribe to the updated quote event.

Microservices 110

Microservices Architecture Software Review Insurance 110

Netflix Tech

AUGUST 1, 2022

Data Mesh?—?A A Data Movement and Processing Platform @ Netflix By Bo Lei , Guilherme Pires , James Shao , Kasturi Chatterjee , Sujay Jain , Vlad Sydorenko Background Realtime processing technologies (A.K.A After evaluating the options , the team has decided to create Data Mesh as our next generation data pipeline solution.

Data 108

Data Transportation Systems Review System 108

Confluent

FEBRUARY 22, 2019

The requirements for fast and reliable data pipelines are growing quickly at Deliveroo as the business continues to grow and innovate. We have delivered an event streaming platform which gives strong guarantees on data quality, using Apache Kafka ® and Protocol Buffers.

Data 80

Data Software Review Weak Development Team Systems Review 80

Cloudera

AUGUST 10, 2022

Cloudera Stream Processing (CSP), powered by Apache Flink and Apache Kafka, provides a complete stream management and stateful processing solution. In CSP, Kafka serves as the storage streaming substrate, and Flink as the core in-stream processing engine that supports SQL and REST interfaces. Apache Kafka and SMM.

Applications 85

Applications Scalability Examples Data 85

Confluent

JUNE 26, 2019

I see this pattern coming up more and more in the field in conjunction with Apache Kafka ®. In these projects, microservice architectures use Kafka as an event streaming platform. Apache Kafka – An event streaming platform for microservices. So why is Kafka such a good fit? Apache Kafka and domain-driven microservices.

Microservices 109

Microservices Software Review Technical Review Architecture 109

Confluent

JUNE 18, 2019

It involves many moving parts, from data preparation to building indexing and query pipelines. Luckily, this task looks a lot like the way we tackle problems that arise when connecting data. Distributed transactions are very hard to implement successfully, which is why we’ll introduce a log-inspired system such as Apache Kafka ®.

Scalability 96

Scalability Architecture Machine Learning Artificial Inteligence 96

Instaclustr

AUGUST 11, 2021

In Part 1 of this two-part blog series we discovered that Debezium is not a new metallic element (but the name was inspired by the periodic table), but a distributed open source CDC technology, and that CDC (in this context) stands for Change Data Capture (not Centers for Disease Control or the Control Data Corporation).

Data 68

Data Systems Review Scalability Off-The-Shelf 68

OpenCredo

JUNE 8, 2023

Navigating the General Data Protection Regulation (GDPR) maze might be daunting. If you’re using Kafka for your data storage and processing, you might be wondering if you’re GDPR compliant, particularly with respect to the Right to Erasure or “Right to be Forgotten” (RTBF).

Compliance 52

Compliance Backup Storage Technical Review 52

Confluent

APRIL 17, 2019

In particular, the management and monitoring capabilities that we added to Confluent Control Center have evolved it into an indispensable tool for anyone working with Apache Kafka ®. Part 2: Managing Kafka Configurations at Scale with Confluent Control Center. Edit schemas as your data evolves.

DevOps 75

DevOps Policies Comparison Applications 75

Confluent

MAY 22, 2019

In light of this, we’ll share an emerging machine-to-machine (M2M) architecture pattern in which MQTT, Apache Kafka ® , and Scylla all work together to provide an end-to-end IoT solution. The explosive number of devices generating, tracking and sharing data across a variety of networks is overwhelming to most data management solutions.

IoT 97

IoT Google Cloud Scalability Microservices 97

Confluent

JULY 16, 2019

How cool would it be to build your own burglar alarm system that can alert you before the actual event takes place simply by using a few network-connected cameras and analyzing the camera images with Apache Kafka ® , Kafka Streams, and TensorFlow? I will show how to implement this use case in this blog post.

Machine Learning 101

Machine Learning Artificial Inteligence Software Review Training 101

Confluent

MARCH 12, 2019

On the heels of part one in this blog series, Spring for Apache Kafka – Part 1: Error Handling, Message Conversion and Transaction Support , here in part two we’ll focus on another project that enhances the developer experience when building streaming applications on Kafka: Spring Cloud Stream. What is Spring Cloud Stream?

Cloud 111

Cloud Applications Groups Examples 111

Confluent

APRIL 17, 2019

In particular, the management and monitoring capabilities that we added to Confluent Control Center have evolved it into an indispensable tool for anyone working with Apache Kafka ®. Part 2: Managing Kafka Configurations at Scale with Confluent Control Center. Edit schemas as your data evolves.

DevOps 40

DevOps Policies Comparison Applications 40

Confluent

JULY 3, 2019

From a data perspective, the World Cup represents an interesting source of information. The idea in this blog post is to mix information coming from two distinct channels: the RSS feeds of sport-related newspapers and Twitter feeds of the FIFA Women’s World Cup. Data sources. Ingesting Twitter data. connector.state].

Analysis 21

Analysis Data Sport Windows 21

Confluent

AUGUST 14, 2019

These secrets may include passwords, such as the values for ssl.key.password , ssl.keystore.password , and ssl.truststore.password configuration parameters (as shown below), or any other sensitive data in the configuration files or log files. Before Confluent Platform 5.3, path/to/secrets.txt ).

Examples 12

Examples Software Review Backup Linux 12

Confluent

MAY 9, 2019

Storing events in a stream and connecting streams via stream processors provide a generic, data-centric, distributed application runtime that you can use to build ETL, event streaming applications, applications for recording metrics and anything else that has a real-time data requirement. Avro or Protobuf ). Control plane.

Microservices 93

Microservices Metrics Systems Review Architecture 93

Confluent

OCTOBER 24, 2019

I’ve written an event sourcing bank simulation in Clojure (a lisp build for Java virtual machines or JVMs) called open-bank-mark , which you are welcome to read about in my previous blog post explaining the story behind this open source example. All messages use a String for the key and Avro for the value. The bank application.

Banking 18

Banking Software Review Testing Applications 18

Confluent

SEPTEMBER 6, 2019

In the above scenario, we would have to update all five services to connect to Apache Kafka ® , create the event in all the appropriate places inside each service, and then produce that event to a Kafka topic. The first point to consider is if connecting to Kafka from all services is even possible.

Examples 22

Examples Applications Banking Data 22

Henrik Warne

JULY 27, 2019

What a great book Designing Data-Intensive Applications is! There are three parts in the book: Foundations of Data Systems (chapters 1 – 4), Distributed Data (chapters 5 – 9), and Derived Data (chapters 10 – 12). Foundations of Data Systems. Foundations of Data Systems.

Systems Review 214

Systems Review Software Review Applications Data 214

Confluent

AUGUST 19, 2019

It points to best practices for anyone writing Kafka Connect connectors. In a nutshell, the document states that sources and sinks are verified as Gold if they’re functionally equivalent to Kafka Connect connectors. Over the years, we’ve since seen wide adoption of Kafka Connect.

Programming 15

Programming Software Review Technical Review Systems Review 15

OpenCredo

JULY 30, 2019

This did however serve as the inspiration that has resulted in me writing this blog – essentially providing a brief recap on the key components of Kafka Connect, and then providing a step by step guide how these relate to developing a Source Connector. Kafka Connect Summary. Kafka Connect was added in the Kafka 0.9.0

Software Review 15

Software Review Systems Review Technical Review System 15