diff options
| author | David Arthur <mumrah@gmail.com> | 2013-09-09 00:44:36 -0400 |
|---|---|---|
| committer | David Arthur <mumrah@gmail.com> | 2013-09-09 00:47:28 -0400 |
| commit | f67ad27f72aca077f24fa801a9d2d3075d6d5b60 (patch) | |
| tree | b6c76cfe70429b079a147d6e84952b1f63352dc9 /kafka/util.py | |
| parent | 40d8e9e550b48755e2f40cfd0877a5b848a3254f (diff) | |
| download | kafka-python-f67ad27f72aca077f24fa801a9d2d3075d6d5b60.tar.gz | |
Auto-adjusting consumer fetch size
Related to #42
Adds new ConsumerFetchSizeTooSmall exception that is thrown when
`_decode_message_set_iter` gets a BufferUnderflowError but has not yet
yielded a message
In this event, SimpleConsumer will increase the fetch size by 1.5 and
continue the fetching loop while _not_ increasing the offset (basically
just retries the request with a larger fetch size)
Once the consumer fetch size has been increased, it will remain
increased while SimpleConsumer fetches from that partition
Diffstat (limited to 'kafka/util.py')
| -rw-r--r-- | kafka/util.py | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/kafka/util.py b/kafka/util.py index 11178f5..bdda7ed 100644 --- a/kafka/util.py +++ b/kafka/util.py @@ -73,6 +73,8 @@ class BufferUnderflowError(Exception): class ChecksumError(Exception): pass +class ConsumerFetchSizeTooSmall(Exception): + pass class ReentrantTimer(object): """ |