summaryrefslogtreecommitdiff
path: root/kafka/consumer/fetcher.py
Commit message (Collapse)AuthorAgeFilesLines
* Use Fetch/Produce API v2 for brokers >= 0.10 (uses message format v1) (#694)Dana Powers2016-05-221-1/+6
|
* KAFKA-3025: Message v1 -- add timetamp and use relative offset in compressed ↵Dana Powers2016-05-221-4/+20
| | | | messagesets
* Fix throttle_time_ms sensorZack Dever2016-04-251-1/+2
| | | Fixes #665
* instrument metrics for fetch requestsZack Dever2016-04-131-63/+91
|
* Changing some commented out metrics to avoid future gotchas.Zack Dever2016-04-131-1/+1
|
* KAFKA-2136: support Fetch and Produce v1 (throttle_time_ms)kafka-2136Dana Powers2016-04-061-2/+4
|
* Merge pull request #634 from dpkp/fetchDana Powers2016-04-061-18/+32
|\ | | | | Small improvements to fetching logic
| * Dont send FetchRequest for (obviously) pending datafetchDana Powers2016-04-061-3/+12
| |
| * Increase coverage of StopIteration check in _unpack_message_setDana Powers2016-04-061-15/+15
| |
| * Log debug messages when skipping fetched messages due to offset checksDana Powers2016-04-061-0/+5
| |
* | Use version-indexed lists for request/response protocol structsprotocol_versionsDana Powers2016-04-051-2/+2
|/
* KAFKA-3318: clean up consumer logging and error messagesDana Powers2016-04-051-5/+5
|
* Update imports from kafka.common -> kafka.errors / kafka.structsDana Powers2016-04-051-2/+2
|
* Drop now-redundant ready() check in Fetcher._handle_offset_responseDana Powers2016-04-041-3/+0
|
* Issue 545: Convert deserializer StopIteration errors to raw ExceptionsDana Powers2016-02-151-1/+8
|
* KAFKA-3170: default consumer config for fetch_min_bytes should be 1Dana Powers2016-02-021-2/+2
|
* Add KafkaConsumer.highwater(partition) to get highwater offsets from ↵Dana Powers2016-01-311-1/+3
| | | | FetchResponses
* Fetcher iterator should check for pause and seek resetsDana Powers2016-01-311-3/+5
|
* Fetcher should filter compressed messages with offsets lower than were requestedDana Powers2016-01-311-0/+8
|
* Fix Fetch._create_fetch_requests docstringDana Powers2016-01-251-1/+1
|
* Ok to sleep in blocking poll for metadata updateDana Powers2016-01-251-1/+1
|
* Attempt to pipeline fetchrequests in iteratorDana Powers2016-01-121-1/+2
|
* Use private deque to track in-flight fetchrequestsDana Powers2016-01-121-0/+16
|
* Update docstring and comments in _create_fetch_requests re KAFKA-2978Dana Powers2016-01-101-5/+2
|
* Reorganize init_fetches calls during iterationDana Powers2016-01-101-5/+17
| | | | | | | Generally should not init_fetches while the generator has pending messages; this revision adds an explicit check / noop to the public interface, and uses a private method internally to attempt to pipeline fetch requests.
* Check for assignment changes before yielding new recordDana Powers2016-01-101-0/+9
|
* Fetcher logging should be debug or trace (left higher during testing)Dana Powers2016-01-101-9/+11
|
* Fix debug logging call in PR 500Dana Powers2016-01-101-1/+1
|
* KAFKA-2978: consumer stops fetching when consumed and fetch positions get ↵Dana Powers2016-01-101-48/+38
| | | | out of sync
* Disable pylint error on __iter__ (next is provided via six.Iterator)Dana Powers2016-01-031-1/+1
|
* Batched message methods now return dict of listsDana Powers2016-01-021-2/+2
|
* Improve iterator interfaceDana Powers2016-01-021-6/+22
| | | | | | | | - Support single message consumption via next(consumer) in py2/py3 - batch message methods (Fetcher.fetched_records / KafkaConsumer.poll) are incompatible with iterators -- message generator state keeps messages internally after they are popped from _records, but before subscription_state is updated.
* Improve request pipelining in consumer iteratorDana Powers2015-12-311-13/+25
|
* Support message decompressionDana Powers2015-12-311-19/+15
|
* Support simple message iteration in Fetcher and new KafkaConsumerDana Powers2015-12-301-0/+49
|
* Prefer assert or more-specific error to IllegalState / IllegalArgumentDana Powers2015-12-301-13/+7
|
* More Docstring ImprovementsDana Powers2015-12-301-39/+94
|
* Switch configs from attributes to dict to make passing / inspecting easierDana Powers2015-12-291-25/+25
|
* New class: kafka.consumer.fetcher.FetcherDana Powers2015-12-281-0/+523
- manages fetch requests via async client and subscription_state - based on upstream java client
View Lorry Controller Status