Instaclustr

MARCH 12, 2019

In the previous blog , we deployed the Anomalia Machina application in a production-like Kubernetes environment. In this blog, we test it out and see many anomalies it can detect at scale on small Kafka and Cassandra Instaclustr production clusters. Can we ride out the storm and scale the application? Yes, in theory.

Scalability 31

Scalability Metrics Resources AWS 31

Cloudera

OCTOBER 28, 2020

Cloudera services logs offer a breadth of information to assist in cluster maintenance; from assisting in security checks, auditing tasks, and validation for performance tuning and testing tasks – to name a few. . However, log records generated by these services do not hold the same value for every organisation. More processors.

Examples 64

Examples Groups Performance Applications 64

Honeycomb

NOVEMBER 30, 2021

When you send telemetry into Honeycomb, our infrastructure needs to buffer your data before processing it in our “retriever” columnar storage database. For the entirety of Honeycomb’s existence, we have used Apache Kafka to perform this buffering function in our observability pipeline. Adapting Kafka for our use case.

AWS 145

AWS Storage Technical Review Hardware 145

Instaclustr

OCTOBER 13, 2021

In Part 6 and Part 7 of the pipeline series we took a different path in the pipe/tunnel and explored PostgreSQL and Apache Superset, mainly from a functional perspective—how can you get JSON data into PostgreSQL from Kafka Connect , and what does it look like in Superset. Scaling Kafka Connect, Elasticsearch, and PostgreSQL Pipelines.

Performance 83

Performance Metrics Systems Review Testing 83

Confluent

MAY 21, 2019

At first, you may think to use REST APIs—most programming languages have frameworks that make it very easy to implement REST APIs, so this is a common first choice. If you evaluate architectures by how easy they are to extend, then this architecture gets an A+. An example can be the backend architecture for an insurance product.

Microservices 110

Microservices Architecture Software Review Insurance 110

O'Reilly Media - Ideas

JANUARY 25, 2024

The data used in this report covers January through November in 2022 and 2023. Generative AI is the wild card: Will it help developers to manage complexity? And the real question that will change our industry is “How do we design systems in which generative AI and humans collaborate effectively?”

Trends 114

Trends Technical Review Technology Artificial Inteligence 114

Confluent

JULY 10, 2019

As discussed in part 2, I created a GitHub repository with Docker Compose functionality for starting a Kafka and Confluent Platform environment, as well as the code samples mentioned below. We are using the Spock framework for writing our test specifications. gradlew composeUp. text1 : defaultvalue } [.] }.

Lambda 87

Lambda Analytics Testing Groups 87

Instaclustr

JULY 22, 2021

Having explored one fork in the path (Elasticsearch and Kibana) in the previous pipeline blog series ( here is part 5 ), in this blog we backtrack to the junction to explore the alternative path ( PostgreSQL and Apache Superset). The first thing is to decide how to connect to it for testing.

Open Source 92

Open Source Data AWS Software Review 92

Altexsoft

OCTOBER 21, 2022

Kafka can continue the list of brand names that became generic terms for the entire type of technology. Similar to Google in web browsing and Photoshop in image processing, it became a gold standard in data streaming, preferred by 70 percent of Fortune 500 companies. What is Kafka? What Kafka is used for.

Weak Development Team 93

Weak Development Team Technical Review Systems Review Software Review 93

Bernd Rucker

APRIL 6, 2020

I want to answer all open questions in this blog post. If you want to keep the event names from your application, we could model: I do agree that this might be the most accurate way of modeling this situation. You can read about it in this blog post , the whole tooling is open source! Unfortunately this post got a bit long?—?sorry,

Microservices 52

Microservices Architecture Technical Review Engineering 52

Honeycomb

AUGUST 8, 2019

AWS bills are notoriously complicated, and the Amazon Cost Explorer doesn’t always make it easy to understand exactly where your money is going. Configure your AWS account to generate Cost & Usage Reports. Then, turn on report generation. Wait a day or two to for your reports to generate, and then… 2.

AWS 10

AWS Report Infrastructure Engineering 10

Altexsoft

FEBRUARY 10, 2021

Unless you meet it in the article saying that “only 13 percent data science projects make it into production.” This sounds really ominous — especially, for companies heavily investing in data-driven transformations. New approaches arise to speed up the transformation of raw data into useful insights.

Analytics 75

Analytics DevOps Development Software Review 75

Altexsoft

NOVEMBER 7, 2022

You can hardly compare data engineering toil with something as easy as breathing or as fast as the wind. The platform went live in 2015 at Airbnb, the biggest home-sharing and vacation rental site, as an orchestrator for increasingly complex data pipelines. How data engineering works. What is Apache Airflow?

Weak Development Team 52

Weak Development Team Technical Review Software Review Systems Review 52

Confluent

MAY 29, 2019

In part 1 , we discussed an event streaming architecture that we implemented for a customer using Apache Kafka ® , KSQL from Confluent, and Kafka Streams. In part 3, we’ll explore using Gradle to build and deploy KSQL user-defined functions (UDFs) and Kafka Streams microservices. Sample repository. gradlew composeUp.

Groups 95

Groups Windows Testing Microservices 95

Cloudera

OCTOBER 22, 2021

With the latest release of Cloudera DataFlow for the Public Cloud (CDF-PC) we added new CLI capabilities that allow you to automate data flow deployments, making it easier than ever before to incorporate Apache NiFi flow deployments into your CI/CD pipelines. Understanding the data flow development lifecycle.

Cloud 91

Cloud Data KPI How To 91

Confluent

MAY 9, 2019

Storing events in a stream and connecting streams via stream processors provide a generic, data-centric, distributed application runtime that you can use to build ETL, event streaming applications, applications for recording metrics and anything else that has a real-time data requirement. Why a payment system, you ask?

Microservices 93

Microservices Metrics Systems Review Architecture 93

Confluent

SEPTEMBER 20, 2019

As a distributed system for collecting, storing, and processing data at scale, Apache Kafka ® comes with its own deployment complexities. To simplify all of this, different providers have emerged to offer Apache Kafka as a managed service. Before Confluent Cloud was announced , a managed service for Apache Kafka did not exist.

Software Review 21

Software Review Technical Review Storage Cloud 21

Confluent

OCTOBER 24, 2019

I’ve written an event sourcing bank simulation in Clojure (a lisp build for Java virtual machines or JVMs) called open-bank-mark , which you are welcome to read about in my previous blog post explaining the story behind this open source example. Either way, both are accomplished with event sourcing. The bank application.

Banking 18

Banking Software Review Testing Applications 18

Confluent

AUGUST 6, 2019

One of KSQL’s most powerful features is allowing users to build their own KSQL functions for processing real-time streams of data. The previous blog post How to Build a UDF and/or UDAF in KSQL 5.0 Unlike UDFs, which operate on a single row at a time, UDAFs can be used for computing aggregates against multiple rows of data.

Software Review 17

Software Review Testing Examples Systems Review 17

Confluent

AUGUST 1, 2019

The Kafka Streams API boasts a number of capabilities that make it well suited for maintaining the global state of a distributed system. At Imperva, we took advantage of Kafka Streams to build shared state microservices that serve as fault-tolerant, highly available single sources of truth about the state of objects in our system.

Microservices 20

Microservices Systems Review System Software Review 20