summaryrefslogtreecommitdiff
path: root/test/test_consumer_integration.py
Commit message (Collapse)AuthorAgeFilesLines
* Add logic for inferring newer broker versions (#2038)Tincu Gabriel2020-05-051-3/+12
| | | | | | | | | | | | | | * Add logic for inferring newer broker versions - New Fetch / ListOffsets request / response objects - Add new test cases to inferr code based on mentioned objects - Add unit test to check inferred version against whatever resides in KAFKA_VERSION - Add new kafka broker versions to integration list - Add more kafka broker versions to travis task list - Add support for broker version 2.5 id * Implement PR change requests: fewer versions for travis testing, remove unused older versions for inference code, remove one minor version from known server list Do not use newly created ACL request / responses in allowed version lists, due to flexible versions enabling in kafka actually requiring a serialization protocol header update Revert admin client file change
* Remove SimpleClient, Producer, Consumer, Unittest (#1196)Jeff Widman2019-10-111-495/+3
| | | | | | | | | | | | | | | | | | In the 2.0 release, we're removing: * `SimpleClient` * `SimpleConsumer` * `SimpleProducer` * Old partitioners used by `SimpleProducer`; these are superceded by the `DefaultPartitioner` These have been deprecated for several years in favor of `KafkaClient` / `KafkaConsumer` / `KafkaProducer`. Since 2.0 allows breaking changes, we are removing the deprecated classes. Additionally, since the only usage of `unittest` was in tests for these old Simple* clients, this also drops `unittest` from the library. All tests now run under `pytest`.
* Add KAFKA 2.3.0 to test matrix (#1915)Dana Powers2019-09-301-1/+5
|
* Follow up to PR 1782 -- fix tests (#1914)Dana Powers2019-09-301-3/+2
|
* Convert remaining `KafkaConsumer` tests to `pytest` (#1886)Jeff Widman2019-08-221-256/+245
| | | | | | | | This makes it so the only remaining use of `unittest` is in the old tests of the deprecated `Simple*` clients. All `KafkaConsumer` tests are migrated to `pytest`. I also had to bump the test iterations up on one of the tests, I think there was a race condition there that was more commonly hit under pytest , planning to cleanup that in a followup PR. See https://github.com/dpkp/kafka-python/pull/1886#discussion_r316860737 for details.
* Cleanup handling of KAFKA_VERSION env var in tests (#1887)Jeff Widman2019-08-221-21/+21
| | | | | | | | | | | | | Now that we are using `pytest`, there is no need for a custom decorator because we can use `pytest.mark.skipif()`. This makes the code significantly simpler. In particular, dropping the custom `@kafka_versions()` decorator is necessary because it uses `func.wraps()` which doesn't play nice with `pytest` fixtures: - https://github.com/pytest-dev/pytest/issues/677 - https://stackoverflow.com/a/19614807/770425 So this is a pre-requisite to migrating some of those tests to using pytest fixtures.
* Use test.fixtures.version not test.conftest.version to avoid warnings (#1731)Dana Powers2019-03-061-2/+1
|
* Migrate from `Unittest` to `pytest` (#1620)Jeff Widman2018-11-101-9/+10
|
* Cleanup fixture importsJeff Widman2018-10-271-2/+2
| | | | | | | | | | | | | | | `random_string` now comes from `test.fixtures` and was being transparently imported via `test.testutil` so this bypasses the pointless indirect import. Similarly, `kafka_version` was transparently imported by `test.testutil` from `test.fixtures`. Also removed `random_port()` in `test.testutil` because its unused as its been replaced by the one in `test.fixtures`. This is part of the pytest migration that was started back in a1869c4be5f47b4f6433610249aaf29af4ec95e5.
* Vendor `six` consistentlyJeff Widman2018-10-221-5/+5
| | | | Use vendored `six`, and also `six.moves.range` rather than `xrange`
* Add codec validators to record parser and builder for all formats (#1447)Taras2018-04-181-1/+23
|
* Fix skipped integration tests if KAFKA_VERSION unset (#1453)Dana Powers2018-03-221-0/+3
|
* Close KafkaConsumer instances during tests (#1410)Dana Powers2018-03-081-0/+7
|
* Introduce new fixtures to prepare for migration to pytest.Andre Araujo2018-02-211-21/+24
| | | | | | | This commits adds new pytest fixtures in prepation for the migration of unittest.TestCases to pytest test cases. The handling of temporary dir creation was also changed so that we can use the pytest tmpdir fixture after the migration.
* Increase some integration test timeouts (#1374)Dana Powers2018-02-081-4/+6
|
* KAFKA-3888 Use background thread to process consumer heartbeats (#1266)Dana Powers2017-12-211-1/+2
|
* Refactor MessageSet and Message into LegacyRecordBatch to later support v2 ↵Taras2017-10-111-2/+4
| | | | message format
* remove beginning/end offsets request version limitLiao Jiayi2017-09-041-6/+0
|
* Added unit tests for fetcher's `_reset_offset` and related functions.Taras Voinarovskiy2017-08-071-1/+1
|
* Added `beginning_offsets` and `end_offsets` API's and fixed @jeffwidman ↵Taras Voinarovskiy2017-08-071-1/+46
| | | | review issues
* Changed retrieve_offsets to allow fetching multiple offsets at onceTaras Voinarovskiy2017-08-071-4/+41
|
* Fix test for older brokersTaras Voinarovskiy2017-08-071-2/+4
|
* Added basic support for offsets_for_times API. Still needs to group by nodes ↵Taras Voinarovskiy2017-08-071-1/+45
| | | | and send in parallel.
* Fix typoJeff Widman2017-07-011-1/+1
|
* Do not test fetched_size with fetch_max_bytes since 0.11 brokers will return ↵Dana Powers2017-06-181-3/+0
| | | | more data than 0.10
* Increase max_buffer_size for test_large_messagesDana Powers2017-06-181-2/+8
|
* Fix fetch_max_bytes=1 consumer integration testDana Powers2017-06-181-10/+10
|
* Fix integration test that requires consumer groupDana Powers2017-03-071-1/+4
|
* Added `max_bytes` option and FetchRequest_v3 usage. (#962)Taras Voinarovskyi2017-03-061-0/+46
| | | | * Added `max_bytes` option and FetchRequest_v3 usage. * Add checks for versions above 0.10 based on ApiVersionResponse
* KAFKA-3007: KafkaConsumer max_poll_records (#831)Dana Powers2016-09-281-0/+1
|
* * [SimpleConsumer] Fix legacy SimpleConsumer when using compressed messages* ↵Paul Cavallaro2016-05-101-1/+18
| | | | | [Legacy Protocol] Update legacy protocol to handle compressed messages * [SimpleConsumer] Fix legacy SimpleConsumer when using compressed messages
* Update imports from kafka.common -> kafka.errors / kafka.structsDana Powers2016-04-051-4/+2
|
* Support setting kafka instance port explicitly in fixtureDana Powers2016-02-151-2/+4
|
* Fix missing importDana Powers2016-01-301-0/+1
|
* Skip MultiProcessConsumer integration testsDana Powers2016-01-301-0/+5
|
* Merge branch '0.9'Dana Powers2016-01-071-45/+57
|\ | | | | | | | | | | | | | | Conflicts: kafka/codec.py kafka/version.py test/test_producer.py test/test_producer_integration.py
| * Dont use consumer_timeout_ms in simple kafka_consumer testDana Powers2016-01-041-2/+1
| |
| * Dont use consumer_timeout_ms in kafka blocking testDana Powers2016-01-031-7/+4
| |
| * Update consumer integration tests to use new (group) KafkaConsumerDana Powers2016-01-031-29/+31
| | | | | | | | | | | | | | | | | | | | - Remove debug call to deprecated .offsets() method - Manually assign TopicPartition to avoid group subscription overhead - Use next(consumer), not consumer.next() - consumer_timeout_ms now raises StopIteration, not ConsumerTimeout - auto_commit_enable is now enable_auto_commit - auto_offset_reset -> earliest, not smallest - new consumer does not support auto_commit_interval_messages
| * Drop bytes encoding of consumer group in consumer integration testDana Powers2015-12-101-1/+1
| |
| * Fix consumer integration test that assumed a single brokerDana Powers2015-12-101-4/+6
| |
| * Fix zk_chroot in consumer integration kafka fixturesDana Powers2015-12-101-2/+3
| |
| * Drop kafka_bytestringDana Powers2015-12-101-1/+1
| |
| * Configure consumer group in consumer_integration testsDana Powers2015-12-101-3/+13
| |
| * Default consumer integration tests should not use offset commitsDana Powers2015-12-091-2/+3
| |
| * Update references to kafka.common Request/Response (now Payload)Dana Powers2015-12-091-2/+2
| |
* | Refactor kafka_versions to support arbitrary operators (> >= < <= ! =)kafka_version_testsDana Powers2015-12-101-20/+5
|/
* Consumers get_messages: allow blocking until some messages are receivedOskari Saarenmaa2015-09-181-0/+18
| | | | | | | | | Modified MultiProcessConsumer's and SimpleConsumer's `block` argument to allow integer value which defines the number of messages to block for. This allows callers to ask for a high number of messages and block only until some of them are received. Otherwise callers would have to request messages one by one or block for some time.
* fix #410 SimpleConsumer cannot seek to an absolute offset.haosdent2015-06-201-0/+14
|
* Add kafka 0.8.2.1 to integration tests, remove 0.8.2.0Dana Powers2015-06-101-5/+5
|
View Lorry Controller Status