Centralized Logging for Kafka on Kubernetes With Grafana, Loki and Promtail

Centralized Logging for Kafka on Kubernetes With Grafana, Loki and Promtail

If there are multiple zookeeper and Kafka pods, a single window would be a boon for administrators. Grafana provides Loki and Promtail this functionality. Centralized Logging for Kafka on Kubernetes With Grafana, Loki, and Promtail.

If there are multiple zookeeper and Kafka pods, a single window would be a boon for administrators. Grafana provides Loki and Promtail this functionality.

Introduction

In one of my another articles, I discussed how to set up strimzi (also known as Kafka on Kubernetes) on minikube. Also, we discussed how to set up Grafana and Prometheus to fetch metrics from Kafka and zookeeper instances. But wouldn't it have been more helpful and more administrator-friendly if Grafana could also be used to monitor logs of all the pods? If there are multiple zookeeper and Kafka pods, a single window would certainly be a boon for administrators and management.

Grafana provides Loki and Promtail a functionality to aggregate logs and view them from the same Grafana UI.

I have tested this in Fedora 33, minikube version: v1.15.1.

kubernetes kafka grafana minikube strimzi

What is Geek Coin

What is GeekCash, Geek Token

Best Visual Studio Code Themes of 2021

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

50+ Useful Kubernetes Tools for 2020 - Part 2

Our original Kubernetes tool list was so popular that we've curated another great list of tools to help you improve your functionality with the platform.

Grafana and Prometheus Setup With Strimzi, a.k.a. Kafka on Kubernetes

A software developer gives a quick tutorial on how to use Strimzi to integrate Prometheus into a Java-based application. Grafana and Prometheus Setup With Strimzi, a.k.a. Kafka on Kubernetes.

Kafka on Kubernetes, the Strimzi Way! (Part 1)

In this article, learn how to run Kafka on Kubernetes.Some of my previous blog posts (such as Kafka Connect on Kubernetes, the easy way!), demonstrate how to use Kafka Connect in a Kubernetes-native way. This is the first in a series of blog posts which will cover Apache Kafka on Kubernetes using the Strimzi Operator.

Kafka on Kubernetes, the Strimzi Way (Part 2)

Learn how to run Kafka on Kubernetes using an open source operator.We kicked off the the first part of the series by setting up a single node Kafka cluster which was accessible to only internal clients within the same Kubernetes cluster, had no encryption, authentication or authorization and used temporary persistence.

Kafka on Kubernetes, the Strimzi Way! (Part 4)

In this article, we'll configure Strimzi to add persistence to a Kubernetes cluster, explore various components, and expand our cluster's storage size.