From releases, to Kafka Improvement Proposals (KIPs), to blogs, check out what’s going on in the Apache Kafka community for May 2020.

In this 28th edition of the Kafka Monthly Digest, I’ll cover what happened in the Apache Kafka community in May 2020.

For last month’s digest, see Kafka Monthly Digest: April 2020.

Releases

2.6.0: On May 5, Randall Hauch volunteered to run the 2.6.0 release. KIP freeze happened on May 20, and feature freeze was on May 27. The release date is currently expected for the end of June. For the full details, see the release plan in the wiki.

2.5.1: On May 29, John Roesler volunteered to be the release manager for 2.5.1. The release plan for this bugfix version is available on the wiki.

KIPs

Last month, the community submitted 16 KIPs (KIP-604 to KIP-619), and these are the ones that caught my eye.

• KIP-604: Remove ZooKeeper Flags from the Administrative Tools. As work to remove Zookeeper is under way, this KIP is for removing the --zookeeper flag from all command line tools in the next major release, 3.0. This flag is already deprecated, and all tools can use the --bootstrap-server flag to directly connect to Kafka.

• KIP-612: Ability to Limit Connection Creation Rate on Brokers. Receiving new client connections uses some CPU on brokers. While this is normally negligible, a connection storm can apply significant load onto brokers. Connection storms can be caused by “well-behaved” clients, for example, when a large application with dozens of clients restarts. This KIP aims at providing a new configuration, max.connection.creation.rate, to enable cluster administrators to limit client connections creations.

• KIP-613: Add end-to-end latency metrics to Streams. At the moment, Kafka Streams does not provide end-to-end latency metrics. This makes it hard to build and operate realtime applications. This KIP aims at adding a number of end-to-end latency metrics to understand how long it takes for records to flow through a topology.

• KIP-618: Atomic commit of source connector records and offsets. This KIP’s goal is to strengthen the delivery guarantees for source connectors. In addition to source records, source connectors also write their current offsets to Kafka. Currently, these 2 writes are done independently. This can cause some record duplication, for example, if a source record is produced but the associated offset is not. This KIP proposes grouping these 2 writes in a transaction to provide exactly-once semantics for source connectors.

#apache kafka #messaging #apache