If you face the problems, when:
- Kafka Consumer is not involving in rebalance process
- Kafka Consumer is not reading messages
- Partitions are not allocated to the Kafka Consumer, but the Consumer is subscribed to the topic
- Kafka Consumer connection gets stuck in CHECKING_API_VERSIONS state
- Kafka Producer connection gets stuck in CHECKING_API_VERSIONS state
Then this is a bug in the kafka-clients library. You need to upgrade Kafka to versions 3.0.1, 3.2.0, 3.1.1.
It is described in Kafka’s Jira: https://issues.apache.org/jira/browse/KAFKA-13388. A lot of thanks to David Hoffman and David Mao for sharing and solving this problem.
You can monitor connections between your Kafka Consumer and Kafka Brokers using code from tis article: https://mchesnavsky.tech/how-to-get-kafka-consumer-connection-states.
Telegram channel
If you still have any questions, feel free to ask me in the comments under this article or write me at promark33@gmail.com.
If I saved your day, you can support me 🤝