Morioh
Login
Arlo McLaughlin

Arlo McLaughlin

3 years ago

Introduction to Kafka Dead Letter Queue (DLQ) and its implementation in Python

Dead Letter Queue is a secondary Kafka topic which receives the messages for which the Kafka Consumer failed to process due to certain errors like improper deserialization of message, improper message format, etc.

Image for post

Installation

There are various libraries in Python which can be used to connect to Kafka Cluster. Some of them are:

  1. kafka-python
  2. confluent-kafka
  3. PyKafka

I’ll be using kafka-python to connect to Kafka Cluster and to create Kafka Producer and Consumer Clients.

#programming #python #kafka

Dead Letter Queue (DLQ) in Kafka

towardsdatascience.com

Dead Letter Queue (DLQ) in Kafka

Introduction to Kafka Dead Letter Queue (DLQ) and its implementation in Python. Dead Letter Queue is a secondary Kafka topic which receives the messages for which the Kafka Consumer failed to process due to certain errors like improper deserialization of message, improper message format, etc. There are various libraries in Python which can be used to connect to Kafka Cluster: kafka-python, confluent-kafka, PyKafka. I’ll be using kafka-python to connect to Kafka Cluster and to create Kafka Producer and Consumer Clients.

3.95 GEEK