summaryrefslogtreecommitdiff
path: root/kafka/protocol
Commit message (Collapse)AuthorAgeFilesLines
* Core Protocol: Add support for flexible versions (#2151)Tincu Gabriel2020-12-025-16/+294
| | | | | | | - Add support for new request and response headers, supporting flexible versions / tagged fields - Add List / Alter partition reassignments APIs - Add support for varints - Add support for compact collections (byte array, string, array)
* Hotfix: TypeError: object of type 'dict_itemiterator' has no len() (#2167)Krzysztof Grabowski2020-11-191-2/+3
| | | | | | | * Hotfix: TypeError: object of type 'dict_itemiterator' has no len() * Avoid looping over items 2x Co-authored-by: Grabowski <chris@crawlinski.com>
* Add Kafka 2.6.0 to tests and protocol compatibility matrix (#2162)Lourens Naudé2020-11-153-1/+55
| | | | * Co-authored-by: Andrew Brown <andrew.brown@shopify.com> * Co-authored-by: Aaron Brady <aaron.brady@shopify.com>
* Feature: delete consumergroups (#2040)Swen Wenzel2020-09-171-0/+41
| | | | | * Add consumergroup related errors * Add DeleteGroups to protocol.admin * Implement delete_groups feature on KafkaAdminClient
* Add support for `zstd` compression (#2021)Tincu Gabriel2020-09-071-3/+7
|
* Add logic for inferring newer broker versions (#2038)Tincu Gabriel2020-05-053-4/+287
| | | | | | | | | | | | | | * Add logic for inferring newer broker versions - New Fetch / ListOffsets request / response objects - Add new test cases to inferr code based on mentioned objects - Add unit test to check inferred version against whatever resides in KAFKA_VERSION - Add new kafka broker versions to integration list - Add more kafka broker versions to travis task list - Add support for broker version 2.5 id * Implement PR change requests: fewer versions for travis testing, remove unused older versions for inference code, remove one minor version from known server list Do not use newly created ACL request / responses in allowed version lists, due to flexible versions enabling in kafka actually requiring a serialization protocol header update Revert admin client file change
* Add `log_start_offset` to message protocol parsing (#2020)Tincu Gabriel2020-03-251-2/+77
| | | This is in preparation for adding `zstd` support.
* Implement methods to convert a Struct object to a pythonic object (#1951)Tyler Lubeck2020-02-062-1/+37
| | | Implement methods to convert a Struct object to a pythonic object
* Remove some dead codeJeff Widman2020-02-051-8/+0
|
* Admin protocol updates (#1948)Tyler Lubeck2019-12-291-20/+239
|
* Remove SimpleClient, Producer, Consumer, Unittest (#1196)Jeff Widman2019-10-112-480/+0
| | | | | | | | | | | | | | | | | | In the 2.0 release, we're removing: * `SimpleClient` * `SimpleConsumer` * `SimpleProducer` * Old partitioners used by `SimpleProducer`; these are superceded by the `DefaultPartitioner` These have been deprecated for several years in favor of `KafkaClient` / `KafkaConsumer` / `KafkaProducer`. Since 2.0 allows breaking changes, we are removing the deprecated classes. Additionally, since the only usage of `unittest` was in tests for these old Simple* clients, this also drops `unittest` from the library. All tests now run under `pytest`.
* Revert 703f0659 / fix 0.8.2 protocol quick detection (#1763)Dana Powers2019-03-271-3/+9
|
* Fix response error checking in KafkaAdminClient send_to_controllerDana Powers2019-01-031-3/+3
| | | | | | | | | | | | Previously we weren't accounting for when the response tuple also has a `error_message` value. Note that in Java, the error fieldname is inconsistent: - `CreateTopicsResponse` / `CreatePartitionsResponse` uses `topic_errors` - `DeleteTopicsResponse` uses `topic_error_codes` So this updates the `CreateTopicsResponse` classes to match. The fix is a little brittle, but should suffice for now.
* Add protocols for {Describe,Create,Delete} AclsUlrik Johansson2018-11-191-0/+185
|
* Be explicit with tuples for %s formattingJeff Widman2018-11-183-4/+4
| | | | Fix #1633
* Pre-compile pack/unpack function callsbillyevans2018-10-291-13/+29
| | | | | | | | | | | | | | | | I noticed that pack/unpack functions from https://github.com/dpkp/kafka-python/blob/master/kafka/protocol/types.py might be slightly improved. I made pre-compilation for them. It gives about 10% better performance compared to the current implementation. Consumption of 100msg: ``` 239884 0.187 0.000 0.287 0.000 types.py:18(_unpack) # new version 239884 0.192 0.000 0.323 0.000 types.py:17(_unpack) ``` I also made some profiling for producers/consumers. It gives about 1-1.5% time savings.
* Add KafkaAdmin classRichard Lee2018-10-241-0/+5
| | | | | | Requires cluster version > 0.10.0.0, and uses new wire protocol classes to do many things via broker connection that previously needed to be done directly in zookeeper.
* Don't use `kafka.common` internally1.3.5Jeff Widman2018-06-051-3/+2
| | | | This finishes the split from `kafka.common` to `kafka.errors`/`kafka.structs`.
* Added AlterConfigs and DescribeConfigs apis (#1472)Stephen SORRIAUX2018-04-181-1/+96
|
* Fix CreatePartitionsRequest_v0 (#1469)Stephen SORRIAUX2018-04-051-1/+1
|
* Fix error var name in _unpack (#1403)j2gg0s2018-03-051-1/+1
| | | Change-Id: I6527da4b70ebec9e08fa50aca9eba717b8361f19
* Fix response schemaAlex Eftimie2018-02-201-1/+2
|
* Add Admin CreatePartitions API callAlex Eftimie2018-02-202-1/+32
|
* Use raw in case string overriden (#1373)Jeff Widman2018-02-081-2/+2
|
* use absolute imports everywhere (#1362)Kevin Tindall2018-02-0612-27/+27
|
* Add Request/Response structs for kafka broker 1.0.0Dana Powers2018-02-065-36/+166
|
* KAFKA-3888 Use background thread to process consumer heartbeats (#1266)Dana Powers2017-12-211-1/+1
|
* Minor Exception cleanupJeff Widman2017-12-121-1/+1
|
* Raise better struct pack/unpack errors (#1320)Jeff Widman2017-12-121-4/+8
|
* Fix two bugs in printing bytes instanceAndre Araujo2017-11-271-1/+1
| | | | | | | | | Bug 1: When `value` is None, trying to call `len(None)` throws an exception. Bug 2: When len(`value`) <= 100, the code currently prints b'' rather than `value`.
* Added limit of 100 bytes for Bytes debug printslegacy_records_refactorTaras Voinarovskiy2017-10-141-0/+4
|
* Added minor fixes for PR reviewTaras2017-10-121-1/+2
|
* Refactor MessageSet and Message into LegacyRecordBatch to later support v2 ↵Taras2017-10-114-16/+20
| | | | message format
* Add kafka.protocol.parser.KafkaProtocol w/ receive and send (#1230)Dana Powers2017-10-101-0/+177
|
* BrokerConnection receive bytes pipe (#1032)Dana Powers2017-08-152-3/+34
|
* Added basic support for offsets_for_times API. Still needs to group by nodes ↵Taras Voinarovskiy2017-08-071-1/+3
| | | | and send in parallel.
* Add private map of api key -> min/max versions to BrokerConnection (#1169)Dana Powers2017-08-061-0/+37
|
* Protocol updates for 0.11.0.0 (#1127)Dana Powers2017-07-097-41/+508
|
* Derive all api classes from Request / Response base classes (#1030)Dana Powers2017-03-148-68/+133
|
* Avoid re-encoding for message crc check (#1027)Dana Powers2017-03-131-6/+12
|
* Fix kwarg handing in kafka.protocol.struct.Struct (#1025)Dana Powers2017-03-091-1/+6
|
* Fixed couple of "leaks" when gc is disabled (#979)Max Baryshnikov2017-03-073-15/+24
|
* CreateTopicsRequest / Response v1 (#1012)Dana Powers2017-03-041-3/+34
|
* Add send_list_offset_request for searching offset by timestamp (#1001)charsyam2017-03-021-0/+29
|
* PEP-8: Spacing & removed unused imports (#899)Jeff Widman2017-02-094-16/+10
|
* Add MetadataRequest_v2 and MetadataResponse_v2 structures for KIP-78 (#974)Taras Voinarovskyi2017-02-091-2/+34
|
* KIP-88 / KAFKA-3853: OffsetFetch v2 structs (#971)Jeff Widman2017-02-081-2/+30
|
* DRY-up the MetadataRequest_v1 struct (#966)Jeff Widman2017-02-081-4/+5
|
* Add JoinGroup v1 structs (#965)Jeff Widman2017-02-081-2/+25
| | | The JoinGroupRequest struct added a rebalance_timeout as part of KIP-62 / KAFKA-3888
* DRY-up the OffsetCommitResponse Structs (#970)Jeff Widman2017-02-031-17/+5
|
View Lorry Controller Status