Apache Kafka

Apache Kafka: A Distributed Streaming Platform.

kafka apache-kafka

Spark Streaming for Beginners

Spark is deemed to be a highly fast engine to process high volumes of data and is found to be 100 times faster than MapReduce. It is so as it uses distributed data processing through which it breaks the data into smaller pieces so that the chunks of data can be computed in parallel across the machines which saves time

Diving Deep into Kafka

The objective of this blog is to build some more understanding of Apache Kafka concepts such as Topics, Partitions, Consumer, and Consumer Groups. Kafka's basic concepts have been covered in my previous article. Kafka Topic & Partitions As we know, messages in Kafka are categorized or stored inside Topics. In simple terms, Topic can be construed as a Database table.

Kafka on Kubernetes, the Strimzi Way (Part 2)

Learn how to run Kafka on Kubernetes using an open source operator.We kicked off the the first part of the series by setting up a single node Kafka cluster which was accessible to only internal clients within the same Kubernetes cluster, had no encryption, authentication or authorization and used temporary persistence.

Kafka on Kubernetes, the Strimzi Way! (Part 1)

In this article, learn how to run Kafka on Kubernetes.Some of my previous blog posts (such as Kafka Connect on Kubernetes, the easy way!), demonstrate how to use Kafka Connect in a Kubernetes-native way. This is the first in a series of blog posts which will cover Apache Kafka on Kubernetes using the Strimzi Operator.

Auto-scale Kafka applications on Kubernetes with KEDA

This tutorial will demonstrate auto-scaling Kafka based consumer applications on Kubernetes using KEDA

Kafka on Kubernetes, the Strimzi Way (Part 3)

Over the course of the first two parts of this blog series, we setup a single-node Kafka cluster on Kubernetes, secured it using TLS encryption and accessed the broker using both internal and external clients. Let's keep iterating! In this post, we will continue the Kafka on Kubernetes journey with Strimzi and cover:

Using Kafka to optimize data flow of your Twitter Stream

A guide to processing big data from Twitter efficiently. I recently worked on a big data analytics project where I collected live streaming data of around 50–60 trending topics from Twitter.

Data Pipeline Using MongoDB and Kafka Connect on Kubernetes

In this blog, you'll learn how you can run Kafka Connect on Kubernetes to create data pipelines using MongoDB.

Event-Driven Architecture as a Strategy

Event-driven architecture provides five key benefits to modern application architecture: scalability, resilience, agility, data sharing, and cloud enabling.

Setting the Scene for Apache Kafka

It is said that Kafka is a messaging system. Many people think of it as a message bus that moves messages from one place to another. Surely it is true but in reality it is more than that. Kafka is an open source, distributed event streaming platform, which can handle all of the data and all of the events within the entire system. It is a platform on which real time applications can be built and it can integrate different systems together driven by the power of events.

Stop Building Your Platform Around Kafka

How your Kafka architecture is setting you up for critical failure. Stop building central dependencies and build for reliable components. Before you get the wrong idea, I’m a big fan of Kafka. It does a great job but all too often it’s setting your architecture up for failure. High performance and throughput — Kafka is great at handling large volumes of messages concurrently and dealing with disparate consumers sending and receiving from it

How to Set Up and Use Kafka Connect on Kubernetes using Strimzi

Learn how to set up and use Kafka Connect on Kubernetes using Strimzi. Kafka Connect is a tool for scalably and reliably streaming data between Apache Kafka and other systems using source and sink connectors. I love the fact that Strimzi enables a Kubernetes-native way of doing this using the Operator pattern with the help of Custom Resource Definitions.

How to Setup Change Data Capture using Debezium, Kafka and PostrgeSQL

Hands-on tutorial on how to setup Change Data Capture Architecture using Debezium, Kafka and PostrgeSQL. Change Data Capture (CDC) is a technique used to track row-level changes in database tables in response to create, update and delete operations.

Apache Kafka and Blockchain: Friends, Enemies, or Frenemies?

Use cases and architectures for Event Streaming, Apache Kafka, Distributed Ledger (DLT) and Blockchain. Comparison: Kafka-native vs Hyperledger vs Ethereum.

A Brief Introduction to Apache Kafka

A little overview of the Apache Kafka platform. During the last years, technologies for building real-time data pipelines and event streaming apps have emerged, promoting also the horizontal scalability and the fault tolerance of a system.

Spring Boot Application With Kafka, Elasticsearch, Redis With Enterprise Standards Part 1

Sample application by using spring boot, kafka, elastic search and Redis. How these are fitted into the enterprise applications.. I am going to discuss the aspects of the regular enterprise development scenarios and how each person has to responsible for each level of software development.

Event Streaming and Apache Kafka in Telco Business (OSS/BSS)

In this article, we discuss the need for businesses to leverage Apache Kafka to better implement scalable, real-time infrastructures for event streaming.

Spring Boot Application With Kafka, Elasticsearch, Redis With Enterprise Standards Part 1

There are many resources online when we want to learn about different tools for standard enterprise applications. But there are no specific resources which scatter all the enterprise standards. There are no clear tutorials on how the requirement is seen at different levels of software development.

Spring Boot Application With Kafka, Elasticsearch, Redis With Enterprise Standards Part 1

There are many resources online when we want to learn about different tools for standard enterprise applications. But there are no specific resources which scatter all the enterprise standards. There are no clear tutorials on how the requirement is seen at different levels of software development.

Azure Event Hubs “Role Based Access Control” in action

A Question and Answer session about Apache Kafka with guests: Ricardo Ferreira, Anna McDonald, Otavio Santana, and Larry Garfield.