Cloudera

JANUARY 31, 2024

Streams Replication Manager (SRM) is an enterprise-grade replication solution that enables fault tolerant, scalable, and robust cross-cluster Kafka topic replication. Introduction Kafka as an event streaming component can be applied to a wide variety of use cases. Replication can be dynamically enabled for topics and consumer groups.

Policies 68

Policies Metrics Load Balancer Software Review 68

Confluent

APRIL 10, 2019

Previously in 3 Ways to Prepare for Disaster Recovery in Multi-Datacenter Apache Kafka Deployments , we provided resources for multi-datacenter designs, centralized schema management, prevention of cyclic repetition of messages, and automatic consumer offset translation to automatically resume applications.

Data 86

Data Disaster Recovery Metrics Applications 86

Instaclustr

SEPTEMBER 28, 2021

In the last installment of the pipeline blog series , we explored writing streaming JSON data into PostgreSQL using Kafka Connect. In this blog, it’s time to find out if our fears of encountering mutant monsters in the gloom of an abandoned uranium mine were warranted or not. What Is Apache Superset? Approach 1.

Data 68

Data Virtualization Metrics Open Source 68

Confluent

JUNE 12, 2019

Confluent Cloud, a fully managed event cloud-native streaming service that extends the value of Apache Kafka ® , is simple, resilient, secure, and performant, allowing you to focus on what is important—building contextual event-driven applications, not infrastructure. KSQL and Kafka Connect example. and Helm/Tiller 2.8.2+

Cloud 93

Cloud How To Examples Infrastructure 93

Cloudera

OCTOBER 4, 2022

This blog post will provide guidance to administrators currently using or interested in using Kafka nodes to maintain cluster changes as they scale up or down to balance performance and cloud costs in production deployments. Kafka brokers contained within host groups enable the administrators to more easily add and remove nodes.

Data 76

Data Groups Cloud Big Data 76

Cloudera

AUGUST 10, 2021

With Apache Ozone on the Cloudera Data Platform (CDP) , they can implement a scale-out model and build out their next generation storage architecture without sacrificing security, governance and lineage. In this article, we’ll focus on generating and viewing lineage that includes Ozone assets from Apache Atlas. With CDP 7.1.4

Storage 103

Storage Policies Architecture Data 103

Instaclustr

OCTOBER 18, 2021

In Part 6 and Part 7 of the pipeline series we took a different path in the pipe/tunnel and explored PostgreSQL and Apache Superset, mainly from a functional perspective—how can you get JSON data into PostgreSQL from Kafka Connect, and what does it look like in Superset. Comparing Elasticsearch and PostgreSQL Replication Settings.

Performance 84

Performance Comparison Systems Review Metrics 84

Cloudera

DECEMBER 9, 2022

Cloudera has been providing enterprise support for Apache NiFi since 2015, helping hundreds of organizations take control of their data movement pipelines on premises and in the public cloud. What if there was a way to not require developers to manage their own Apache NiFi installation without putting that burden on platform administrators?

Testing 95

Testing Azure Cloud AWS 95

Cloudera

JULY 15, 2021

This blog post provides an overview of best practice for the design and deployment of clusters incorporating hardware and operating system configuration, along with guidance for networking and security as well as integration with existing enterprise infrastructure. Introduction and Rationale. Recommended deployment patterns.

Architecture 96

Architecture Cloud Data Technical Advisors 96

Cloudera

DECEMBER 10, 2020

In the previous post, we talked about Kerberos authentication and explained how to configure a Kafka client to authenticate using Kerberos credentials. In this post we will look into how to configure a Kafka client to authenticate using LDAP, instead of Kerberos. We use the Kafka-console-consumer for all the examples below.

Authentication 52

Authentication How To Examples Network 52

Cloudera

OCTOBER 13, 2020

Data Hub – has expanded to support all stages of the data lifecycle: Collect – Flow Management (Apache NiFi), Streams Management (Apache Kafka) and Streaming Analytics (Apache Flink). Enrich – Data Engineering (Apache Spark and Apache Hive). Predict – Data Engineering (Apache Spark).

Cloud 90

Cloud Machine Learning Artificial Inteligence Azure 90

Cloudera

FEBRUARY 22, 2023

As apart of Cloudera Streaming Analytics it enables users to easily write, run, and manage real-time SQL queries on streams with a smooth user experience, while it attempts to expose the full power of Apache Flink. The post Implementing and Using UDFs in Cloudera SQL Stream Builder appeared first on Cloudera Blog.

Analytics 81

Analytics Examples Data Programming 81

Cloudera

MARCH 9, 2021

In this blog, we’ll highlight the key CDP aspects that provide data governance and lineage and show how they can be extended to incorporate metadata for non-CDP systems from across the enterprise. Apache Atlas as a fundamental part of SDX. The example 1_typedef-server.json describes the server typedef used in this blog. .

Government 93

Government Data Applications Storage 93

Instaclustr

JUNE 4, 2019

At Instaclustr, we constantly push ourselves to test the limits of the performance and scalability of the open source technologies our Managed Platform offers. Kafka is a good choice for fast scalable ingestion of streaming data. “The limits of the possible can only be defined by going beyond them into the impossible.”.

Scalability 29

Scalability Open Source AWS Applications 29

Bernd Rucker

APRIL 20, 2021

Camunda Cloud Architecture Blog Post Series?—?Part One of your first tasks would be to sketch the basic architecture of your solution, and this blog post will guide you through some important early questions such as how to connect the workflow engine Zeebe with your application or with remote systems? in July 2021. Prior to the 1.1

Architecture 63

Architecture Engineering Cloud Programming 63

Cloudera

SEPTEMBER 30, 2021

Our recent blog discussed the four paths to get from legacy platforms to CDP Private Cloud Base. In this blog and accompanying video, we will deep dive into the mechanics of running an in-place upgrade from CDH5 or CDH6 to CDP Private Cloud Base. Exporting Sentry policies ready for Apache Ranger. Replication Manager checks.

Cloud 69

Cloud Systems Review Backup Software Review 69

Altexsoft

FEBRUARY 11, 2023

It serves as a foundation for the entire data management strategy and consists of multiple components including data pipelines; , on-premises and cloud storage facilities – data lakes , data warehouses , data hubs ;, data streaming and Big Data analytics solutions ( Hadoop , Spark , Kafka , etc.);

Data 87

Data Data Engineering Big Data Architecture 87

Cloudera

OCTOBER 28, 2020

Cloudera services logs offer a breadth of information to assist in cluster maintenance; from assisting in security checks, auditing tasks, and validation for performance tuning and testing tasks – to name a few. . Any records from Kafka, only IF the record represents users publishing to certain Kafka topics. Assumptions.

Examples 63

Examples Groups Performance Applications 63

Instaclustr

MARCH 12, 2019

In the previous blog , we deployed the Anomalia Machina application in a production-like Kubernetes environment. In this blog, we test it out and see many anomalies it can detect at scale on small Kafka and Cassandra Instaclustr production clusters. Here are some blogs on how to run the Cluster Autoscaler on AWS.

Scalability 31

Scalability Metrics Resources AWS 31

Datavail

JULY 14, 2022

Healthcare companies generate critical, immediately relevant information every moment, as doctors and patients interact, tests are run and results are shared, and treatment plans are created.

Data 52

Data AWS Analytics Programming 52

TIBCO - Connected Intelligence

MAY 28, 2021

Included under TIBCO’s evaluative strengths were its dominating worldwide presence, the time-tested and well-proven data management and analytics capabilities of its digital platforms, and TIBCO’s strong traditional integration brand and reputation. Once, Twice, Three Times a Leader. Let TIBCO Take the Lead in Your Business’s Transformation.

Research 29

Research Innovation Cloud Leadership 29

Confluent

JULY 17, 2019

Together, MongoDB and Apache Kafka ® make up the heart of many modern data architectures today. Integrating Kafka with external systems like MongoDB is best done though the use of Kafka Connect. The official MongoDB Connector for Apache Kafka is developed and supported by MongoDB engineers. Getting started.

eCommerce 21

eCommerce Architecture System Guidelines 21

Bernd Rucker

FEBRUARY 8, 2022

In today’s blog post I want to explain the reasoning behind this move and why we recommend a remote engine. They used Apache Kafka and MongoDB. REST, AMQP, Kafka). I dedicated a blog post to this topic: “ Achieving consistency without transaction managers.” Refer to zeebe-process-test for details.

Engineering 52

Engineering Architecture Programming Cloud 52

Honeycomb

NOVEMBER 30, 2021

For the entirety of Honeycomb’s existence, we have used Apache Kafka to perform this buffering function in our observability pipeline. A brief overview of Kafka at Honeycomb. How does Kafka help us achieve these results? Adapting Kafka for our use case. Let’s dive in.

AWS 145

AWS Storage Technical Review Hardware 145

Modus Create

JULY 7, 2020

This article expands on the most advanced technique in my earlier blog post, New Relic and JMeter Performance Remediation – Part 3 , Introduce transactional asynchronous messaging. The Apache project has a kitchen sink full of messaging systems: ActiveMQ (see Amazon MQ ), Apollo , Kafka , Pulsar , QPid , RocketMQ , and Synapse.

Performance 84

Performance Hotels AWS Systems Review 84

OpenCredo

APRIL 2, 2020

Environment-aware : A variation on release cycle integration, a good data catalogue should also take into account the promotion of code and data between dev, test, UAT/preprod environments. Streaming data platforms such as Kafka and Pulsar mean that data can be delivered in a reliable, scalable and immutable way. Data visualisation.

Development Team Review 40

Development Team Review Systems Review Comparison Government 40

Instaclustr

OCTOBER 13, 2021

In Part 6 and Part 7 of the pipeline series we took a different path in the pipe/tunnel and explored PostgreSQL and Apache Superset, mainly from a functional perspective—how can you get JSON data into PostgreSQL from Kafka Connect , and what does it look like in Superset. Scaling Kafka Connect, Elasticsearch, and PostgreSQL Pipelines.

Performance 83

Performance Metrics Systems Review Testing 83

Confluent

MAY 30, 2019

Following part 1 and part 2 of the Spring for Apache Kafka Deep Dive blog series, here in part 3 we will discuss another project from the Spring team: Spring Cloud Data Flow , which focuses on enabling developers to easily develop, deploy, and orchestrate event streaming pipelines based on Apache Kafka ®.

Cloud 94

Cloud Data Applications Windows 94

Cloudera

FEBRUARY 28, 2024

This blog post walks you through how you can use prefixless replication with Streams Replication Manager (SRM) to aggregate Kafka topics from multiple sources. To be specific, we will be diving deep into a prefixless replication scenario that involves the aggregation of two topics from two separate Kafka clusters into a third cluster.

Testing 68

Testing Big Data Tools Data 68

Xebia

MARCH 15, 2023

In this blog Simon Karman and Matt Watson will dive into monitoring an Amazon MSK cluster. Amazon MSK is the AWS managed offering for Apache Kafka. Identify important metrics To identify which metric(s) could be used when a cluster is overloaded, we did a load test.

Metrics 130

Metrics Systems Review Comparison AWS 130

Confluent

MAY 21, 2019

The profile service will publish the changes in profiles, including address changes to an Apache Kafka ® topic, and the quote service will subscribe to the updates from the profile changes topic, calculate a new quote if needed and publish the new quota to a Kafka topic so other services can subscribe to the updated quote event.

Microservices 110

Microservices Architecture Software Review Insurance 110

Instaclustr

JULY 22, 2021

Having explored one fork in the path (Elasticsearch and Kibana) in the previous pipeline blog series ( here is part 5 ), in this blog we backtrack to the junction to explore the alternative path ( PostgreSQL and Apache Superset). The first thing is to decide how to connect to it for testing.

Open Source 92

Open Source Data AWS Software Review 92

Altexsoft

OCTOBER 21, 2022

Kafka can continue the list of brand names that became generic terms for the entire type of technology. In this article, we’ll explain why businesses choose Kafka and what problems they face when using it. In this article, we’ll explain why businesses choose Kafka and what problems they face when using it. What is Kafka?

Weak Development Team 93

Weak Development Team Technical Review Systems Review Software Review 93

Confluent

JULY 10, 2019

As discussed in part 2, I created a GitHub repository with Docker Compose functionality for starting a Kafka and Confluent Platform environment, as well as the code samples mentioned below. We are using the Spock framework for writing our test specifications. gradlew composeUp. return Utils.textMatch(expression, search1, ignorecase) ?

Lambda 87

Lambda Analytics Testing Groups 87

Instaclustr

JULY 7, 2019

Massively Scalable Geospatial Anomaly Detection with Apache Kafka and Cassandra. In this blog we discover that we’ve been trapped in Flatland, and encounter a Dimension of the Third Kind. The winner will be revealed in the next blog. Do 3D geohashes impact partition cardinality?

3D 28

3D Systems Review Software Review Examples 28

Cloudera

DECEMBER 2, 2020

This is the first installment in a short series of blog posts about security in Apache Kafka. Secured Apache Kafka clusters can be configured to enforce authentication using different methods, including the following: SSL – TLS client authentication. sasl.kerberos.service.name=kafka. # TLS truststore.

Authentication 65

Authentication How To Load Balancer Firewall 65

Confluent

JUNE 11, 2019

Here in part 4 of the Spring for Apache Kafka Deep Dive blog series, we will cover: Common event streaming topology patterns supported in Spring Cloud Data Flow. Create and manage event streaming pipelines, including a Kafka Streams application using Spring Cloud Data Flow.

Continuous Delivery 85

Continuous Delivery Applications Cloud Construction 85