KafkaTimeoutError: Timeout during ApiVersions check

[berrfred]

First of all thanks for your incredible work since kafka-python allows me to use KAFKA on a very particular system (HPE NonStop) and also for your support, I still remember one year ago when you pointed out that rebalancing on my consumer was due to the heartbeat not running properly and a consequence of the GIL on CPython ... solved with a small time sleep after each poll.

Now I have another issue again on the consumer while trying to upgrade my environment from version 2.2.10 to the latest 3.0.8 .
It is not reading anything and the attached debug log shows some errors, any idea what's going wrong ?

gwkmt_kafka_gmtk2.log

[dpkp]

It looks like a similar issue. In 3.0.8 all network IO is processed on a background thread (in 2.2 it was only the consumer heartbeat). There is a ~10sec default timeout on connections and it seems like you may be hitting that due to similar GIL problems. It looks like we haven't actually exposed that 10sec timeout configuration to clients, but you should be able to monkeypatch it like so:

consumer._manager.config['socket_connection_setup_timeout_ms'] = 30000  # 30sec connection timeout

(also the intent of that timeout is that it increases w/ exponential backoff, but in this case that is not working -- i'll work on a fix for that as well).

[berrfred]

I tried your suggestion setting the timeout to 30s and I do not see the previous ERROR on ApiVersions check but I think we are still facing the same issue ...

2026-07-14 10:35:03,284 - WARNING - $GMTK2 - Task <kafka.net.selector.Task object at 0x8dbce10> ran for 5.004s (>0.100s threshold). It is blocking the event loop -- likely a tight sync loop inside a coroutine. Other pollers will time out.

gwkmt_kafka_gmtk2_20260714.log

[dpkp]

Yes, I saw that warning in the first set of logs. In your case I suspect it is not actually the coroutine that is blocking but more likely it is the GIL lock problem that you raised in the prior issue. If your app code holds the GIL for 5secs then it will prevent the event loop from processing any coroutines, and then when the GIL is unlocked and the event loop starts processing again it will log this warning.

Are you seeing any other errors or issues besides this warning?

[berrfred]

No, I do not see any other error, it is only logging Timeout during ApiVersions check without increasing the connection timeout
or SASL Authentication timed out when increasing the timeout to 30 seconds. I also tried adding a time.sleep(0.001) right after the consumer creation but that does not change anything.

gwkmt_kafka_gmtk1_timeout_10s.log
gwkmt_kafka_gmtk1_timeout_30s.log