Confluent

AUGUST 29, 2019

We know that Apache Kafka ® is great when you’re dealing with streams, allowing you to conveniently look at streams as tables. Kafka already allows you to look at data as streams or tables; graphs are a third option, a more natural representation with a lot of grounding in theory for some use cases.

Social 55

Social Network Applications Cloud 55

Confluent

AUGUST 27, 2019

We are excited to announce the release of Confluent Cloud Schema Registry in general availability (GA), available in Confluent Cloud , our fully managed event streaming service based on Apache Kafka ®. In contrast, Confluent Cloud Schema Registry is backed by Confluent-managed Kafka clusters, where schemas are stored.

Cloud 18

Cloud Product Management AWS Storage 18

Confluent

SEPTEMBER 26, 2019

In the early days, many companies simply used Apache Kafka ® for data ingestion into Hadoop or another data lake. However, Apache Kafka is more than just messaging. Some Kafka and Rockset users have also built real-time e-commerce applications , for example, using Rockset’s Java, Node.js

Analytics 21

Analytics Serverless Blockchain Microservices 21

Confluent

MAY 21, 2019

This leads us to event streaming microservices patterns. In this case, a simple solution will be to publish the exact same information that it used to send in the HTTP request: { user_id: 53, timestamp: 1497842472, address: “2 Elm St. An example can be the backend architecture for an insurance product. Chattanooga, TN” }.

Microservices 110

Microservices Architecture Software Review Insurance 110

Confluent

JUNE 11, 2019

For event streaming application developers, it is important to continuously update the streaming pipeline based on the need for changes in the individual applications in the pipeline. It is also important to understand some of the common streaming topologies that streaming developers use to build an event streaming pipeline.

Continuous Delivery 85

Continuous Delivery Applications Cloud Construction 85

Instaclustr

OCTOBER 13, 2021

In Part 6 and Part 7 of the pipeline series we took a different path in the pipe/tunnel and explored PostgreSQL and Apache Superset, mainly from a functional perspective—how can you get JSON data into PostgreSQL from Kafka Connect , and what does it look like in Superset. Scaling Kafka Connect, Elasticsearch, and PostgreSQL Pipelines.

Performance 83

Performance Metrics Systems Review Testing 83

Confluent

MAY 30, 2019

Following part 1 and part 2 of the Spring for Apache Kafka Deep Dive blog series, here in part 3 we will discuss another project from the Spring team: Spring Cloud Data Flow , which focuses on enabling developers to easily develop, deploy, and orchestrate event streaming pipelines based on Apache Kafka ®.

Cloud 94

Cloud Data Applications Windows 94

Altexsoft

OCTOBER 21, 2022

Kafka can continue the list of brand names that became generic terms for the entire type of technology. In this article, we’ll explain why businesses choose Kafka and what problems they face when using it. What is Kafka? How Apache Kafka streams relate to Franz Kafka’s books. What Kafka is used for.

Weak Development Team 93

Weak Development Team Technical Review Systems Review Software Review 93

Honeycomb

NOVEMBER 30, 2021

For the entirety of Honeycomb’s existence, we have used Apache Kafka to perform this buffering function in our observability pipeline. In this post, we’ll review the history of how we got here, why we’re so picky about Kafka software and hardware, and how we qualified and adopted the new AWS Graviton2-based storage instances.

AWS 145

AWS Storage Technical Review Hardware 145

Confluent

APRIL 4, 2019

Event-driven architecture means just that: It’s all about the events. In a microservices architecture, events drive microservice actions. No event, no shoes, no service. In the most basic scenario, microservices that need to take action on a common stream of events all listen to that stream.

Microservices 108

Microservices IoT Applications Architecture 108

Cloudera

SEPTEMBER 20, 2021

There are many ways that Apache Kafka has been deployed in the field. In our Kafka Summit 2021 presentation, we took a brief overview of many different configurations that have been observed to date. In this blog series, we will discuss each of these deployments and the deployment choices made along with how they impact reliability.

Systems Review 115

Systems Review System Hardware Software Review 115

Confluent

JULY 10, 2019

As discussed in part 2, I created a GitHub repository with Docker Compose functionality for starting a Kafka and Confluent Platform environment, as well as the code samples mentioned below. gradlew composeUp. KSQL user-defined functions. While the CASE syntax now available since KSQL 5.2.2

Lambda 87

Lambda Analytics Testing Groups 87

O'Reilly Media - Ideas

JANUARY 25, 2024

Each graph is scaled so that the topic with the greatest usage is 1. Software Development Most of the topics that fall under software development declined in 2023. The web is over 30 years old; the Netscape browser appeared in 1994, and it wasn’t the first. 2023 was one of those rare disruptive years. What will those changes be?

Trends 114

Trends Technical Review Technology Artificial Inteligence 114

Confluent

MAY 28, 2019

Part 1 of this blog series by Gwen Shapira explained the benefits of schemas, contracts between services, and compatibility checking for schema evolution. There are three common types of operational mistakes we have observed customers make in self-managing their own Schema Registry: Bad designs. Inconsistent configurations.

Weak Development Team 83

Weak Development Team Backup Disaster Recovery Applications 83

Confluent

JUNE 18, 2019

Distributed transactions are very hard to implement successfully, which is why we’ll introduce a log-inspired system such as Apache Kafka ®. Building an indexing pipeline at scale with Kafka Connect. Software projects of all sizes and complexities have a common challenge: building a scalable solution for search. Direct indexing.

Scalability 96

Scalability Architecture Machine Learning Artificial Inteligence 96

Gorilla Logic

JUNE 26, 2023

If you want to build event-based applications, it’s important to explore the foundations of Spring Cloud Stream. In this post, we’ll learn more about the main building blocks of Spring Cloud Stream, its core components, and the basic configuration that enables us to produce and consume events through different applications.

Cloud 66

Cloud Systems Review Groups Software Review 66

Cloudera

JANUARY 11, 2021

I want to thank you all for joining and attending these events! I received hundreds of questions during these events, and my colleagues and I tried to answer as many as we could. Interactive demo sessions and live Q&A are what we all need these days when working remotely from home is now a norm.

Policies 59

Policies Load Balancer Windows Resources 59

Instaclustr

APRIL 20, 2023

How Do Kafka Concurrency and Throughput Work? Apache Kafka ® is a high-throughput, low-latency distributed streaming platform. It enables messages to be sent from multiple distributed producers via the distributed Kafka cluster and topics, to multiple distributed consumers. Is there a limit?

Scalability 57

Scalability Performance Software Review Groups 57

Instaclustr

AUGUST 4, 2021

Centers for Disease Control and Prevention ( cdc.gov ) has certainly been in the forefront of the news, and their Apache Kafka Covid-19 pipeline was the inspiration for my ongoing Kafka Connect streaming data pipeline series. Centers for Disease Control and Prevention Control Data Corporation Change Data Capture. (1) CDC 6600 [link].

Data 59

Data Open Source Analytics System 59

Instaclustr

APRIL 29, 2019

In this tenth and final blog of the Anomalia Machina series we tune the anomaly detection system and succeed in scaling the application out from 3 to 48 Cassandra nodes, and get some impressive numbers: 574 CPU cores (across Cassandra, Kafka, and Kubernetes clusters), 2.3 The Scaling Journey. The suitors all tried and failed.

Scalability 49

Scalability Systems Review Metrics AWS 49

Confluent

JUNE 25, 2019

It’s official: Apache Kafka ® 2.3 Core Kafka. In order to keep your data safe, Kafka creates several replicas of it on different brokers. Kafka will not allow writes to proceed unless the partition has a minimum number of in-sync replicas. specifies how long an old value may possibly live in a compacted topic.

Metrics 111

Metrics Windows Performance Video 111

Bernd Rucker

APRIL 6, 2020

I want to answer all open questions in this blog post. Not only was I overwhelmed by the number of attendees, but we also got a huge list of interesting questions before and especially during the webinar. Some of them were answered, but a lot of them were not. Some of them were answered, but a lot of them were not. descriptive symbol set?

Microservices 52

Microservices Architecture Technical Review Engineering 52

Confluent

FEBRUARY 13, 2019

Part 1 of this series discussed why you need to embrace event-first thinking, while this article builds a rationale for different styles of event-driven architectures and compares and contrasts scaling, persistence and runtime models. Event-driven architecture. Event-driven, reactive architecture. Being event first.

Architecture 90

Architecture Programming Microservices Serverless 90

Confluent

FEBRUARY 27, 2019

What is more, as the world adopts the event-driven streaming architecture, how does it fit with serverless? When it comes to the emerging serverless world, It makes sense to validate how Apache Kafka ® fits in considering that it is mission critical in 90 percent of companies. Event-first FaaS? What is FaaS? Next Steps.

Serverless 109

Serverless Lambda AWS Systems Review 109

Confluent

MAY 9, 2019

Event-first thinking enables us to build a new atomic unit: the event. Four pillars of event streaming. Pillar 4 – Operational plane: Event logging, DLQs and automation. To read the other articles in this series, see: Journey to Event Driven – Part 1: Why Event-First Thinking Changes Everything.

Microservices 93

Microservices Metrics Systems Review Architecture 93

Instaclustr

OCTOBER 14, 2019

A complete rundown leading up to ApacheCon USA, starting a week before the final event. This blog provides a brief of things I learnt during the 4-day event, including a summary of talks I attended and presented. programming language, starts with “J”, type of coffee, etc.) IoT, geospatial, ML, streaming, etc.),

Technical Review 36

Technical Review Technology Report Open Source 36

Instaclustr

OCTOBER 14, 2019

A complete rundown leading up to ApacheCon USA, starting a week before the final event. This blog provides a brief of things I learnt during the 4-day event, including a summary of talks I attended and presented. . Countdown begins: The week before ApacheCon. I’m looking forward to ApacheCon 2019 next week in Las Vegas.

Technical Review 28

Technical Review Technology Report Open Source 28

Confluent

SEPTEMBER 20, 2019

As a distributed system for collecting, storing, and processing data at scale, Apache Kafka ® comes with its own deployment complexities. To simplify all of this, different providers have emerged to offer Apache Kafka as a managed service. Before Confluent Cloud was announced , a managed service for Apache Kafka did not exist.

Software Review 21

Software Review Technical Review Storage Cloud 21

Altexsoft

MAY 14, 2021

Big Data enjoys the hype around it and for a reason. But the understanding of the essence of Big Data and ways to analyze it is still blurred. The truth is, there’s more to this term than just the size of information generated. This post will draw a full picture of what Big Data analytics is and how it works. Key Big Data characteristics.

Big Data 78

Big Data Analytics Tools Applications 78

Confluent

JULY 3, 2019

France, Brazil, and the USA are the favourites, and this year Italy is present at the event for the first time in 20 years. There’s a lot of dedicated press coverage, as well as the standard social media excitement following any kind of big event. confluent-hub install jcustenborder/kafka-connect-twitter:latest.

Analysis 21

Analysis Data Sport Windows 21

Confluent

OCTOBER 24, 2019

I’ve written an event sourcing bank simulation in Clojure (a lisp build for Java virtual machines or JVMs) called open-bank-mark , which you are welcome to read about in my previous blog post explaining the story behind this open source example. Either way, both are accomplished with event sourcing. The bank application.

Banking 18

Banking Software Review Testing Applications 18

Confluent

SEPTEMBER 6, 2019

First, what is event sourcing? We can answer all those questions because the individual events that make up our balance are stored. In fact, it’s the summation of these events that result in our current account balance. This, in a nutshell, is event sourcing. Event sourcing: primary vs. derivative.

Examples 22

Examples Applications Banking Data 22

OpenCredo

JULY 30, 2019

This did however serve as the inspiration that has resulted in me writing this blog – essentially providing a brief recap on the key components of Kafka Connect, and then providing a step by step guide how these relate to developing a Source Connector. Kafka Connect Summary. Kafka Connect was added in the Kafka 0.9.0

Software Review 15

Software Review Systems Review Technical Review System 15