Confluent

APRIL 30, 2019

With the release of Apache Kafka ® 2.1.0, Kafka Streams introduced the processor topology optimization framework at the Kafka Streams DSL layer. This framework opens the door for various optimization techniques from the existing data stream management system (DSMS) and data stream processing literature.

Applications 90

Applications Construction Software Review Examples 90

Altexsoft

OCTOBER 21, 2022

Kafka can continue the list of brand names that became generic terms for the entire type of technology. Similar to Google in web browsing and Photoshop in image processing, it became a gold standard in data streaming, preferred by 70 percent of Fortune 500 companies. What is Kafka? What Kafka is used for.

Weak Development Team 93

Weak Development Team Technical Review Systems Review Software Review 93

Bernd Rucker

NOVEMBER 18, 2019

How and why to bring workflow to Apache Kafka In the last year I had a lot of contact with the community around Kafka and Confluent (the company behind Apache Kafka)?—?a For example, at Kafka Summit New York City earlier this year, I was impressed how many big banks attended, that currently modernize their architecture.

Microservices 8

Microservices Architecture Scalability Course 8

Confluent

JULY 10, 2019

As discussed in part 2, I created a GitHub repository with Docker Compose functionality for starting a Kafka and Confluent Platform environment, as well as the code samples mentioned below. Spock allows us to use the @Unroll annotation to define data tables for data-driven testing, defined using the where: clause in the spec below.

Lambda 87

Lambda Analytics Testing Groups 87

Cloudera

MARCH 15, 2023

In this post, I will demonstrate how to use the Cloudera Data Platform (CDP) and its streaming solutions to set up reliable data exchange in modern applications between high-scale microservices, and ensure that the internal state will stay consistent even under the highest load. InventoryService) or processing a payment (eg.

Data 88

Data Microservices Storage Applications 88

Confluent

JULY 16, 2019

How cool would it be to build your own burglar alarm system that can alert you before the actual event takes place simply by using a few network-connected cameras and analyzing the camera images with Apache Kafka ® , Kafka Streams, and TensorFlow? I will show how to implement this use case in this blog post.

Machine Learning 101

Machine Learning Artificial Inteligence Software Review Training 101

Confluent

JULY 24, 2019

Using Jaeger tracing, I’ve been able to answer an important question that nearly every Apache Kafka ® project that I’ve worked on posed: how is data flowing through my distributed system? Eventually, this GitHub repository will come in handy. Distributed tracing with Apache Kafka and Jaeger. What does this all mean?

System 54

System Applications Examples Storage 54

Confluent

MARCH 12, 2019

On the heels of part one in this blog series, Spring for Apache Kafka – Part 1: Error Handling, Message Conversion and Transaction Support , here in part two we’ll focus on another project that enhances the developer experience when building streaming applications on Kafka: Spring Cloud Stream.

Cloud 111

Cloud Applications Groups Examples 111

Bernd Rucker

APRIL 6, 2020

I want to answer all open questions in this blog post. you can find one example in Create Readable Process Models. Q: Does the Camunda Process Engine in its open-source version supports the entire and complete BPMN 2.0 Some of them were answered, but a lot of them were not. Unfortunately this post got a bit long?—?sorry,

Microservices 52

Microservices Architecture Technical Review Engineering 52

Confluent

FEBRUARY 13, 2019

Once settled on the event streaming approach, I’ll provide a high-level dataflow of how we design systems for payment processing at scale using this approach. The paradigm shift to the event streaming architecture often leads to a lightbulb moment of inspiration and clarity. Event-driven, streaming architecture.

Architecture 90

Architecture Programming Microservices Serverless 90

Cloudera

MAY 4, 2021

This article is the first of a multipart series to showcase the power and expressibility of FlinkSQL applied to market data. Code and data for this series are available on github. Flink SQL is a data processing language that enables rapid prototyping and development of event-driven and streaming applications.

Marketing 77

Marketing Data Windows Groups 77

Confluent

MAY 29, 2019

In part 1 , we discussed an event streaming architecture that we implemented for a customer using Apache Kafka ® , KSQL from Confluent, and Kafka Streams. In part 3, we’ll explore using Gradle to build and deploy KSQL user-defined functions (UDFs) and Kafka Streams microservices. Introduction. KSQL primer.

Groups 95

Groups Windows Testing Microservices 95

Altexsoft

NOVEMBER 7, 2022

You can hardly compare data engineering toil with something as easy as breathing or as fast as the wind. The platform went live in 2015 at Airbnb, the biggest home-sharing and vacation rental site, as an orchestrator for increasingly complex data pipelines. How data engineering works. What is Apache Airflow?

Weak Development Team 52

Weak Development Team Technical Review Software Review Systems Review 52

O'Reilly Media - Ideas

JANUARY 25, 2024

The data used in this report covers January through November in 2022 and 2023. Who wants to learn about coding practices when you’re letting GitHub Copilot write your code for you? Our data shows that most topics in software architecture and design are down year-over-year. Or will it add complexity all its own?

Trends 114

Trends Technical Review Technology Artificial Inteligence 114

Instaclustr

AUGUST 4, 2021

Centers for Disease Control and Prevention Control Data Corporation Change Data Capture. (1) Centers for Disease Control and Prevention ( cdc.gov ) has certainly been in the forefront of the news, and their Apache Kafka Covid-19 pipeline was the inspiration for my ongoing Kafka Connect streaming data pipeline series.

Data 59

Data Open Source Analytics System 59

Gorilla Logic

NOVEMBER 6, 2019

As software engineers we tend to like green field development— the process of starting from scratch without reusing existing code—because it allows us to start fresh with new sets of technologies and proper architectures that are more closely tied to the requirements of the business. Beyond Scripts: A Cool New Way to Pipe Data.

Data 51

Data How To Lambda AWS 51

Confluent

SEPTEMBER 20, 2019

As a distributed system for collecting, storing, and processing data at scale, Apache Kafka ® comes with its own deployment complexities. To simplify all of this, different providers have emerged to offer Apache Kafka as a managed service. How do you spot a true fully managed service for Apache Kafka?

Software Review 21

Software Review Technical Review Storage Cloud 21

Confluent

JULY 3, 2019

From a data perspective, the World Cup represents an interesting source of information. The idea in this blog post is to mix information coming from two distinct channels: the RSS feeds of sport-related newspapers and Twitter feeds of the FIFA Women’s World Cup. Data sources. Ingesting Twitter data. connector.state].

Analysis 21

Analysis Data Sport Windows 21

Confluent

AUGUST 19, 2019

This post specifically outlines the process by which we verify partner integrations, and is a means of letting the world know about our partner’s contributions to our connector ecosystem. It points to best practices for anyone writing Kafka Connect connectors. Over the years, we’ve since seen wide adoption of Kafka Connect.

Programming 15

Programming Software Review Technical Review Systems Review 15

OpenCredo

JULY 30, 2019

This did however serve as the inspiration that has resulted in me writing this blog – essentially providing a brief recap on the key components of Kafka Connect, and then providing a step by step guide how these relate to developing a Source Connector. Kafka Connect Summary. Kafka Connect was added in the Kafka 0.9.0

Software Review 15

Software Review Systems Review Technical Review System 15

Confluent

OCTOBER 24, 2019

I’ve written an event sourcing bank simulation in Clojure (a lisp build for Java virtual machines or JVMs) called open-bank-mark , which you are welcome to read about in my previous blog post explaining the story behind this open source example. Using Java interop, these classes make working with the data in Clojure easier.

Banking 18

Banking Software Review Testing Applications 18

Dzone - DevOps

FEBRUARY 10, 2023

Building Apache Kafka connectors is a good example. In this blog post, I will share the details of a very nice GitHub repository that does exactly this. Kafka Connectors: What, Why, and When Apache Kafka by itself is just a dumb pipe.

Open Source 94

Open Source System Examples Data 94

Cloudera

NOVEMBER 27, 2023

Flink Forward is a great opportunity to learn about the cutting edge of streaming and stream processing technologies. This blog is a summary of what we observed there for anyone who was unable to attend or just wants to stay on top of what’s happening in streaming. Takeaway No. The tech itself now boasts 1.4

Open Source 79

Open Source Conference Storage Architecture 79