summaryrefslogtreecommitdiff
path: root/kafka/consumer
Commit message (Collapse)AuthorAgeFilesLines
* Support configuration of custom kafka client for Admin/Consumer/Producer (#2144)Tincu Gabriel2020-12-021-1/+3
|
* KafkaConsumer: Exit poll if consumer is closed (#2152)Rauli Ikonen2020-11-151-2/+4
|
* Only try to update sensors fetch lag if the unpacked list contains elements ↵Keith So2020-11-051-2/+3
| | | | | (#2158) Previously, if the `unpacked` list was empty, the call to `unpacked[-1]` would throw an `IndexError: list index out of range`
* Fix typo (#2096)KimDongMin2020-09-071-1/+1
|
* Fix docs by adding SASL mentionJeff Widman2020-03-021-1/+2
| | | | | This was previously supported but wasn't documented. Fix #1990.
* Set length of header value to 0 if Nonekvfi2020-03-021-1/+3
|
* Optionally return OffsetAndMetadata from consumer.committed(tp) (#1979)Dana Powers2019-12-293-6/+13
|
* Raise AssertionError if consumer closed in poll() (#1978)Dana Powers2019-12-291-0/+3
|
* Enable SCRAM-SHA-256 and SCRAM-SHA-512 for sasl (#1918)Swen Wenzel2019-12-291-5/+5
|
* Improve docs for reconnect_backoff_max_ms (#1976)Dana Powers2019-12-281-5/+6
|
* Fix typosCarson Ip2019-11-081-2/+2
|
* Remove SimpleClient, Producer, Consumer, Unittest (#1196)Jeff Widman2019-10-115-999/+1
| | | | | | | | | | | | | | | | | | In the 2.0 release, we're removing: * `SimpleClient` * `SimpleConsumer` * `SimpleProducer` * Old partitioners used by `SimpleProducer`; these are superceded by the `DefaultPartitioner` These have been deprecated for several years in favor of `KafkaClient` / `KafkaConsumer` / `KafkaProducer`. Since 2.0 allows breaking changes, we are removing the deprecated classes. Additionally, since the only usage of `unittest` was in tests for these old Simple* clients, this also drops `unittest` from the library. All tests now run under `pytest`.
* Follow up to PR 1782 -- fix tests (#1914)Dana Powers2019-09-301-1/+2
|
* Improve/refactor bootstrap_connectedDana Powers2019-09-301-3/+1
|
* Added a function to determine if bootstrap is successfully connected (#1876)PandllCom2019-09-301-0/+6
|
* Issue #1780 - Consumer hang indefinitely in fetcher._retrieve_offsets() due ↵Commander Dishwasher2019-09-301-7/+21
| | | | to topic deletion while rebalancing (#1782)
* Send socket data via non-blocking IO with send buffer (#1912)Dana Powers2019-09-291-2/+6
|
* Do not use wakeup when sending fetch requests from consumer (#1911)Dana Powers2019-09-291-1/+1
|
* Wrap consumer.poll() for KafkaConsumer iteration (#1902)Dana Powers2019-09-282-10/+69
|
* Reduce internal client poll timeout for consumer iterator interface (#1824)Dana Powers2019-08-161-3/+1
| | | More attempts to address heartbeat timing issues in consumers, especially with the iterator interface. Here we can reduce the `client.poll` timeout to at most the retry backoff (typically 100ms) so that the consumer iterator interface doesn't block for longer than the heartbeat timeout.
* Fix minor typo (#1865)Carson Ip2019-07-141-1/+1
|
* Update link to upstream Kafka docsJeff Widman2019-07-111-1/+1
| | | the new consumer is now the standard consumer, so they dropped the `new_` from the anchor
* Update docs for api_version_auto_timeout_ms (#1812)Jeff Widman2019-05-241-1/+1
| | | | | | | | | The docs for `api_version_auto_timeout_ms` mention setting `api_version='auto'` but that value has been deprecated for years in favor of `api_version=None`. Updating the docs for now, and will remove support for `'auto'` in next major version bump.
* Fix typo in _fetch_all_topic_metadata function (#1809)Brian Sang2019-05-231-1/+1
|
* Make partitions_for_topic a read-through cache (#1781)Brian Sang2019-05-221-8/+25
| | | If the cluster metadata object has no info about the topic, then issue a blocking metadata call to fetch it.
* Update link to kafka docsJeff Widman2019-05-171-1/+1
| | | Now that the old zookeeper consumer has been completely deprecated/removed, these are no longer the "new consumer configs" but rather simply the "consumer configs"
* Update sasl configuration docstringsDana Powers2019-03-231-5/+5
|
* Support SASL OAuthBearer Authentication (#1750)Phong Pham2019-03-221-1/+4
|
* Allow configuration of SSL Ciphers (#1755)Dana Powers2019-03-211-0/+6
|
* Error if connections_max_idle_ms not larger than request_timeout_ms (#1688)Jeff Widman2019-03-141-3/+7
|
* 1701 use last offset from fetch v4 if available (#1724)Keith So2019-03-132-0/+24
|
* Do network connections and writes in KafkaClient.poll() (#1729)Dana Powers2019-03-081-12/+1
| | | | | | * Add BrokerConnection.send_pending_requests to support async network sends * Send network requests during KafkaClient.poll() rather than in KafkaClient.send() * Dont acquire lock during KafkaClient.send if node is connected / ready * Move all network connection IO into KafkaClient.poll()
* Remove unused `skip_double_compressed_messages`Jeff Widman2019-01-132-16/+0
| | | | | | | | | | This `skip_double_compressed_messages` flag was added in https://github.com/dpkp/kafka-python/pull/755 in order to fix https://github.com/dpkp/kafka-python/issues/718. However, grep'ing through the code, it looks like it this is no longer used anywhere and doesn't do anything. So removing it.
* #1681 add copy() in metrics() to avoid thread safety issues (#1682)Tosi Émeric2018-12-271-2/+2
|
* Be explicit with tuples for %s formattingJeff Widman2018-11-184-10/+10
| | | | Fix #1633
* Expose record headers in ConsumerRecordsHeikki Nousiainen2018-09-271-3/+5
|
* add kerberos domain name config for gssapi sasl mechanism handshake (#1542)the-sea2018-08-311-1/+4
|
* Document connections_max_idle_msJeff Widman2018-06-261-0/+5
| | | | | This was added in #1068 but never documented. Fix #1497
* Don't use `kafka.common` internally1.3.5Jeff Widman2018-06-052-5/+5
| | | | This finishes the split from `kafka.common` to `kafka.errors`/`kafka.structs`.
* Stop using deprecated log.warn()Jeff Widman2018-05-101-3/+3
|
* Fix KafkaConsumer docstring for request_timeout_ms default (#1459)Dana Powers2018-03-231-1/+1
|
* Do not validate api_version against known versions (#1434)Dana Powers2018-03-101-2/+1
|
* Avoid tight poll loop in consumer when brokers are down (#1415)Dana Powers2018-03-081-1/+1
|
* Use local copies in Fetcher._fetchable_partitions to avoid mutation errors ↵Dana Powers2018-03-071-3/+6
| | | | (#1400)
* Validate max_records in KafkaConsumer.poll (#1398)Dana Powers2018-02-271-0/+2
|
* Fix KafkaConsumer compacted offset handling (#1397)Dana Powers2018-02-261-8/+9
|
* Correctly respect timeouts in consumer poll interface (#1384)Braedon Vickers2018-02-211-1/+1
|
* use absolute imports everywhere (#1362)Kevin Tindall2018-02-063-8/+8
|
* Avoid consuming duplicate compressed messages from mid-batch (#1367)Dana Powers2018-02-051-2/+11
|
* KAFKA-3949: Avoid race condition when subscription changes during rebalance ↵Dana Powers2018-02-023-22/+24
| | | | (#1364)