summaryrefslogtreecommitdiff
path: root/example.py
diff options
context:
space:
mode:
authorJeff Widman <jeff@jeffwidman.com>2018-11-13 13:10:19 -0800
committerJeff Widman <jeff@jeffwidman.com>2018-11-18 15:33:02 -0800
commit232a2d6b428069722fd7f4dc9fba77e76f418594 (patch)
treee96e229974dfef1e093e9d22c2ccda06e8369143 /example.py
parentcc8e91426907f8ccadd60eedc4dc53b8729a84ec (diff)
downloadkafka-python-232a2d6b428069722fd7f4dc9fba77e76f418594.tar.gz
Fix list_consumer_groups() to query all brokers
Previously, this only queried the controller. In actuality, the Kafka protocol requires that the client query all brokers in order to get the full list of consumer groups. Note: The Java code (as best I can tell) doesn't allow limiting this to specific brokers. And on the surface, this makes sense... you typically don't care about specific brokers. However, the inverse is true... consumer groups care about knowing their group coordinator so they don't have to repeatedly query to find it. In fact, a Kafka broker will only return the groups that it's a coordinator for. While this is an implementation detail that is not guaranteed by the upstream broker code, and technically should not be relied upon, I think it very unlikely to change. So monitoring scripts that fetch the offsets or describe the consumers groups of all groups in the cluster can simply issue one call per broker to identify all the coordinators, rather than having to issue one call per consumer group. For an ad-hoc script this doesn't matter, but for a monitoring script that runs every couple of minutes, this can be a big deal. I know in the situations where I will use this, this matters more to me than the risk of the interface unexpectedly breaking.
Diffstat (limited to 'example.py')
0 files changed, 0 insertions, 0 deletions