In this previous blog I showed how to consume messages from Kafka declaratively. Today I am going to show you how to do the opposite, how to produce messages into Kafka aka how to send Messages to a Kafka topic declaratively.

Head over to the previous blog for context. Basically we are trying to write Kubernetes manifests to declare how we send and receive or produce/consume messages to a Kafka topic. The diagram above shows it all. For the confusion part of this blog note that a KafkaSink produces an event to a Kafka topic and therefore is a Kafka source from the POV of the Kafka cluster :) and therefore the KafkaSource consumes an event from a Kafka topic and therefore is a Kafka sink from the POV of the Kafka cluster, it would have been to easy otherwise !!!

Why ?

Because it would be much easier than writing my own Kafka clients, compiling, packaging, deploying etc. If only I could write a bit of config and let a system like Kubernetes manage it for me that would be great.

How ?

We install two controllers and a CRD for a new kind called KafkaSink and for good measures (otherwise shit hits the fan) we install the knative eventing CRD (we will get that fixed because it would be nice to avoid having to do that step…).

#kubernetes