summaryrefslogtreecommitdiff
path: root/kafka
Commit message (Collapse)AuthorAgeFilesLines
...
| * Make more methods private(ish) in AbstractCoordinatorDana Powers2015-12-291-13/+13
| |
| * Remove recursive client.poll call in ConsumerCoordinatorDana Powers2015-12-291-2/+3
| |
| * Pass auto commit parameters from KafkaConsumer to ConsumerCoordinatorDana Powers2015-12-292-1/+3
| |
| * Fix broken SubscriptionState.is_assigned callDana Powers2015-12-291-1/+1
| |
| * Fix HeartbeatTask request_in_flight handlingDana Powers2015-12-291-1/+1
| |
| * kafka.consumer.group.KafkaConsumer: 0.9 coordinator-aware consumerDana Powers2015-12-281-797/+480
| | | | | | | | | | | | | | - use ConsumerCoordinator to manage group membership - dynamically assign partitions via group using topic subscription - use upstream configuration parameters as __init__ kwargs - metrics and SSL are still TODO
| * ConsumerCoordinator (based on upstream Java client)Dana Powers2015-12-284-0/+703
| | | | | | | | | | | | - Use RoundRobinPartitionAssignor by default - Define AbstractPartitionAssignor for custom assignors - metrics still TODO
| * AbstractCoordinator (modelled after upstream Java class)Dana Powers2015-12-284-0/+690
| | | | | | | | | | - Heartbeat class manages HeartbeatRequest / Response state - metrics are still TODO
| * Define ConsumerRebalanceListener abstract classDana Powers2015-12-281-0/+84
| |
| * Add UNKNOWN_MEMBER_ID to JoinGroupRequestDana Powers2015-12-281-0/+1
| |
| * Add DEFAULT_GENERATION_ID and DEFAULT_RETENTION_TIME to OffsetCommitRequest_v2Dana Powers2015-12-281-0/+2
| |
| * New class: kafka.consumer.fetcher.FetcherDana Powers2015-12-281-0/+523
| | | | | | | | | | - manages fetch requests via async client and subscription_state - based on upstream java client
| * Improvements to kafka.future.FutureDana Powers2015-12-281-8/+37
| | | | | | | | | | | | | | | | - log exceptions in callbacks/errors - dont raise - guarantee future.exception is an instance, not a class/type - *args, **kwargs in add_callback / errback (functools.partial) - add_both(f, *args, **kwargs) to add same f as callback and errback - chain(new_future) to trigger new_future on success / failure of this
| * New class SubscriptionState, based on upstream Java clientDana Powers2015-12-281-0/+304
| |
| * Add OffsetAndMetadata namedtupleDana Powers2015-12-281-0/+3
| |
| * Add Message.validate_crc() methodDana Powers2015-12-281-0/+7
| |
| * Add OffsetResetStrategy enum class to kafka.protocol.offsetDana Powers2015-12-281-0/+5
| |
| * Use attributes on exception classes to signal retriable and invalid metadata ↵Dana Powers2015-12-282-13/+33
| | | | | | | | errors
| * Rename TopicAndPartition -> TopicPartitionDana Powers2015-12-283-7/+7
| |
| * New async KafkaClient, based on java NetworkClient and ConsumerClientDana Powers2015-12-282-0/+510
| | | | | | | | | | | | | | - Support async client bootstrap retries after exponential backoff - Attempt to finish_connect if still connecting in KafkaClient.ready(node_id) - Run full async loop while waiting for futures in KafkaClient.poll() - Add more fallbacks to KafkaClient.least_loaded_node; worst case, retry boostrap
| * Update BrokerConnection for use with async clientDana Powers2015-12-282-39/+78
| | | | | | | | | | | | | | | | | | - use descriptive names for ConnectionStates enum values - Change default send_buffer_bytes config to 131072 - add can_send_more() and max_in_flight_requests_per_connection config - add blacked_out() and reconnect_backoff_ms config - last_attempt and last_failure are now public attributes - raise TooManyInFlightRequests in conn.send() if cant send more
| * Add ClusterMetadata.add_group_coordinator()Dana Powers2015-12-281-0/+39
| |
| * Rename Cluster -> ClusterMetadata; align with upstream Metadata classDana Powers2015-12-282-66/+125
| |
| * Switch BrokerConnection to (mostly) non-blocking IO.Dana Powers2015-12-175-119/+339
| | | | | | | | | | | | - return kafka.Future on send() - recv is now non-blocking call that completes futures when possible - update KafkaClient to block on future completion
| * new 0.9 error codes plus descriptions.Zack Dever2015-12-152-10/+172
| | | | | | | | | | reference: https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-ErrorCodes
| * Check for no in-flight-requests in BrokerConnection.recvDana Powers2015-12-101-0/+3
| |
| * Only log topic names (and broker metadata) in KafkaClient load_metadata ↵Dana Powers2015-12-101-1/+1
| | | | | | | | debug logging
| * Move Request / Response logging from KafkaClient to BrokerConnectionDana Powers2015-12-102-4/+4
| | | | | | | | and reenable kafka.conn debug logging in tests
| * pylint fixesDana Powers2015-12-104-4/+5
| | | | | | | | | | | | - temporarily ignore kafka/consumer/group.py - temporarily ignore test/test_protocol.py - disable-msg deprecated; use disable= instead
| * Remove KafkaProtocol._decode_messageDana Powers2015-12-101-32/+0
| |
| * Add pylint hints to AbstractType because we cant mark a classmethod as abstractDana Powers2015-12-101-2/+2
| |
| * Handle python3 in kafka.protocol.pickleDana Powers2015-12-101-2/+6
| |
| * Drop kafka_bytestringDana Powers2015-12-108-48/+19
| |
| * Convert OffsetCommit and OffsetFetch protocol encode/decodeDana Powers2015-12-105-98/+62
| |
| * Add next_correlation_id_recv + send helper methods to BrokerConnectionDana Powers2015-12-091-0/+8
| |
| * Support requests that do not expect a response in BrokerConnectionDana Powers2015-12-091-2/+3
| |
| * Handle PartialMessage / ConsumerFetchSizeTooSmall in SimpleConsumerDana Powers2015-12-091-21/+24
| |
| * Update references to kafka.common Request/Response (now Payload)Dana Powers2015-12-094-31/+27
| |
| * Refactor KafkaClient to use BrokerConnections and new Request/Response structsDana Powers2015-12-091-148/+131
| |
| * Exception handling cleanup in BrokerConnection (also catch struct.errors in ↵Dana Powers2015-12-091-4/+4
| | | | | | | | recv)
| * Add client_id and correlation_id to BrokerConnection constructor kwargsDana Powers2015-12-091-3/+7
| |
| * Change KafkaProtocol to encode/decode Structs, not bytesDana Powers2015-12-092-252/+112
| | | | | | | | | | - add Payload to kafka.common Request/Responses namedtuples - OffsetFetch and OffsetCommit still need to be converted
| * Support pickling of Structs -- _encode_self instance method needs some magicDana Powers2015-12-092-0/+26
| |
| * Return PartialMessage object in MessageSet.decode if message is truncated by ↵Dana Powers2015-12-091-3/+10
| | | | | | | | max_bytes
| * Fallback to simple repr() in Schema.repr()Dana Powers2015-12-091-7/+10
| |
| * Fix BytesIO import in kafka.protocol.messageDana Powers2015-12-091-1/+3
| |
| * Fix _mp_consume queue variable name conflictDana Powers2015-12-091-2/+2
| |
| * Handle special __init__ signature in Message decode()Dana Powers2015-12-091-0/+8
| |
| * Handle decoding partial messages in MessageSet - caused by FetchRequest ↵Dana Powers2015-12-091-6/+24
| | | | | | | | max_bytes
| * Merge branch 'master' into 0.9Zack Dever2015-12-074-17/+34
| |\
View Lorry Controller Status