Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | Remove string support in `api_version`Remove-support-for-api_version-strings | Jeff Widman | 2019-05-24 | 3 | -22/+2 |
| | | | | | | A long time ago, `api_version` supported strings. That has been deprecated for years in favor of tuples. Time to remove support for the strings. | ||||
* | Update docs for api_version_auto_timeout_ms (#1812) | Jeff Widman | 2019-05-24 | 4 | -7/+29 |
| | | | | | | | | | The docs for `api_version_auto_timeout_ms` mention setting `api_version='auto'` but that value has been deprecated for years in favor of `api_version=None`. Updating the docs for now, and will remove support for `'auto'` in next major version bump. | ||||
* | Fix typo in _fetch_all_topic_metadata function (#1809) | Brian Sang | 2019-05-23 | 1 | -1/+1 |
| | |||||
* | Make partitions_for_topic a read-through cache (#1781) | Brian Sang | 2019-05-22 | 1 | -8/+25 |
| | | | If the cluster metadata object has no info about the topic, then issue a blocking metadata call to fetch it. | ||||
* | Remove unused imports (#1808) | Jeff Widman | 2019-05-22 | 2 | -7/+3 |
| | |||||
* | Use futures to parallelize calls to _send_request_to_node() (#1807) | Lou-Cipher | 2019-05-21 | 1 | -34/+75 |
| | | | | | Use `futures` to parallelize calls to `_send_request_to_node()` This allows queries that need to go to multiple brokers to be run in parallel. | ||||
* | Update link to kafka docs | Jeff Widman | 2019-05-17 | 1 | -1/+1 |
| | | | Now that the old zookeeper consumer has been completely deprecated/removed, these are no longer the "new consumer configs" but rather simply the "consumer configs" | ||||
* | A little python cleanup (#1805) | Jeff Widman | 2019-05-17 | 1 | -4/+2 |
| | | | | | | 1. Remove unused variable: `partitions_for_topic` 2. No need to cast to list as `sorted()` already returns a list 3. Using `enumerate()` is cleaner than `range(len())` and handles assigning `member` | ||||
* | Bump version for development of next release | Dana Powers | 2019-04-03 | 1 | -1/+1 |
| | |||||
* | Release 1.4.61.4.6 | Dana Powers | 2019-04-02 | 3 | -1/+91 |
| | |||||
* | Do not call state_change_callback with lock (#1775) | Dana Powers | 2019-04-02 | 3 | -31/+40 |
| | |||||
* | Additional BrokerConnection locks to synchronize protocol/IFR state (#1768) | Dana Powers | 2019-04-02 | 1 | -61/+85 |
| | |||||
* | Return connection state explicitly after close in connect() (#1778) | Dana Powers | 2019-04-02 | 1 | -1/+3 |
| | |||||
* | Reset reconnect backoff on SSL connection (#1777) | Dana Powers | 2019-04-02 | 1 | -0/+1 |
| | |||||
* | Fix possible AttribueError during conn._close_socket (#1776) | Dana Powers | 2019-04-01 | 1 | -1/+1 |
| | |||||
* | Dont treat popped conn.close() as failure in state change callback (#1773) | Dana Powers | 2019-04-01 | 2 | -5/+13 |
| | |||||
* | Avoid race condition on client._conns in send() (#1772) | Dana Powers | 2019-03-31 | 1 | -2/+3 |
| | | | There was a very small possibility that between checking `self._can_send_request(node_id)` and grabbing the connection object via `self._conns[node_id]` that the connection could get closed / recycled / removed from _conns and cause a KeyError. This PR should prevent such a KeyError. In the case where the connection is disconnected by the time we call send(), we should expect conn.send() simply to fail the request. | ||||
* | lock client.check_version (#1771) | Dana Powers | 2019-03-31 | 1 | -0/+5 |
| | |||||
* | Dont wakeup during maybe_refresh_metadata -- it is only called by poll() (#1769) | Dana Powers | 2019-03-30 | 2 | -6/+6 |
| | |||||
* | Fixups to benchmark scripts for py3 / new KafkaFixture interfacebenchmark_fixups | Dana Powers | 2019-03-30 | 2 | -4/+4 |
| | |||||
* | Revert 703f0659 / fix 0.8.2 protocol quick detection (#1763) | Dana Powers | 2019-03-27 | 2 | -6/+9 |
| | |||||
* | Send pending requests before waiting for responses (#1762) | Dana Powers | 2019-03-27 | 1 | -2/+4 |
| | |||||
* | Rename ssl.keystore.location and ssl.truststore.location config files | Dana Powers | 2019-03-24 | 16 | -32/+36 |
| | |||||
* | Dont do client wakeup when sending from sender thread (#1761) | Dana Powers | 2019-03-24 | 2 | -6/+10 |
| | |||||
* | Update sasl configuration docstrings | Dana Powers | 2019-03-23 | 5 | -24/+24 |
| | |||||
* | Support SASL OAuthBearer Authentication (#1750) | Phong Pham | 2019-03-22 | 7 | -6/+117 |
| | |||||
* | Fix flaky conn tests that use time.time (#1758) | Dana Powers | 2019-03-21 | 1 | -9/+12 |
| | |||||
* | Add py to requirements-dev | Dana Powers | 2019-03-21 | 1 | -0/+1 |
| | |||||
* | Maintain shadow cluster metadata for bootstrapping (#1753) | Dana Powers | 2019-03-21 | 2 | -35/+21 |
| | |||||
* | Allow configuration of SSL Ciphers (#1755) | Dana Powers | 2019-03-21 | 4 | -1/+28 |
| | |||||
* | Wrap SSL sockets after connecting (#1754) | Dana Powers | 2019-03-21 | 1 | -19/+11 |
| | |||||
* | Generate SSL certificates for local testing (#1756) | Dana Powers | 2019-03-21 | 1 | -0/+29 |
| | | | This doesn't fully implement SSL fixtures, but as a first step it should help with automatically generating required certificates / keystores / etc. My hope is that this helps generate more community support for SSL testing! | ||||
* | Fix race condition in protocol.send_bytes (#1752) | Filip Stefanak | 2019-03-21 | 1 | -1/+2 |
| | |||||
* | Forgot compatibility section in docs/changelog.rst | Dana Powers | 2019-03-14 | 1 | -0/+8 |
| | |||||
* | Bump version for development | Dana Powers | 2019-03-14 | 1 | -1/+1 |
| | |||||
* | Release 1.4.51.4.5 | Dana Powers | 2019-03-14 | 4 | -2/+102 |
| | |||||
* | Error if connections_max_idle_ms not larger than request_timeout_ms (#1688) | Jeff Widman | 2019-03-14 | 2 | -5/+13 |
| | |||||
* | Retry bootstrapping after backoff when necessary (#1736) | Dana Powers | 2019-03-14 | 3 | -129/+112 |
| | | | | | | | The current client attempts to bootstrap once during initialization, but if it fails there is no second attempt and the client will be inoperable. This can happen, for example, if an entire cluster is down at the time a long-running client starts execution. This commit attempts to fix this by removing the synchronous bootstrapping from `KafkaClient` init, and instead merges bootstrap metadata with the cluster metadata. The Java client uses a similar approach. This allows us to continue falling back to bootstrap data when necessary throughout the life of a long-running consumer or producer. Fix #1670 | ||||
* | Fix default protocol parser version | Dana Powers | 2019-03-13 | 1 | -0/+3 |
| | |||||
* | Minor updates to client_async.py | Dana Powers | 2019-03-13 | 1 | -4/+4 |
| | |||||
* | Mock dns lookups in test_conn (#1739) | Dana Powers | 2019-03-13 | 1 | -1/+8 |
| | | | Small change to avoid doing dns resolution when running local connection tests. This fixture always returns a broker on localhost:9092, so DNS lookups don't make sense here. | ||||
* | Recheck connecting nodes sooner when refreshing metadata (#1737) | Dana Powers | 2019-03-13 | 2 | -4/+2 |
| | |||||
* | Don't recheck version if api_versions data is already cached (#1738) | Dana Powers | 2019-03-13 | 1 | -0/+3 |
| | | | I noticed during local testing that version probing was happening twice when connecting to newer broker versions. This was because we call check_version() once explicitly, and then again implicitly within get_api_versions(). But once we have _api_versions data cached, we can just return it and avoid probing versions a second time. | ||||
* | Attempt to join heartbeat thread during close() (#1735) | Dana Powers | 2019-03-13 | 1 | -3/+6 |
| | | | | | | | Underlying issue here is a race on consumer.close() between the client, the connections/sockets, and the heartbeat thread. Although the heartbeat thread is signaled to close, we do not block for it. So when we go on to close the client and its underlying connections, if the heartbeat is still doing work it can cause errors/crashes if it attempts to access the now closed objects (selectors and/or sockets, primarily). So this commit adds a blocking thread join to the heartbeat close. This may cause some additional blocking time on consumer.close() while the heartbeat thread finishes. But it should be small in average case and in the worst case will be no longer than the heartbeat_timeout_ms (though if we timeout the join, race errors may still occur). Fix #1666 | ||||
* | 1701 use last offset from fetch v4 if available (#1724) | Keith So | 2019-03-13 | 3 | -0/+28 |
| | |||||
* | Catch thrown OSError by python 3.7 when creating a connection (#1694) | Daniel Johansson | 2019-03-12 | 1 | -0/+3 |
| | |||||
* | Update travis test coverage: 2.7, 3.4, 3.7, pypy2.7 (#1614) | Jeff Widman | 2019-03-12 | 6 | -11/+40 |
| | | | | | * Use xenial dist for travis builds * Use openjdk8 for all travis tests * Update python build matrix -- add 3.7, drop 3.5/3.6 (keep 2.7, 3.4, pypy2.7) | ||||
* | Ignore lookup_coordinator result in commit_offsets_async (#1712) | Faqa | 2019-03-12 | 1 | -1/+2 |
| | |||||
* | Synchronize puts to KafkaConsumer protocol buffer during async sends | Dana Powers | 2019-03-12 | 2 | -25/+60 |
| | |||||
* | Do network connections and writes in KafkaClient.poll() (#1729) | Dana Powers | 2019-03-08 | 6 | -57/+84 |
| | | | | | | * Add BrokerConnection.send_pending_requests to support async network sends * Send network requests during KafkaClient.poll() rather than in KafkaClient.send() * Dont acquire lock during KafkaClient.send if node is connected / ready * Move all network connection IO into KafkaClient.poll() |