summaryrefslogtreecommitdiff
path: root/kafka
Commit message (Collapse)AuthorAgeFilesLines
...
| * | Moving to **kwargs for MPConsumer optionsViktor Shlapakov2015-03-241-1/+2
| | |
| * | Moved additional MP consumer options to **kwargsViktor Shlapakov2015-03-131-3/+3
| | |
| * | Using additional params for MP consumer child processViktor Shlapakov2015-03-131-26/+32
| | | | | | | | | | | | | | | | | | | | | - Moved the events params to a separate param for consistency - Passing additional params to internal SimpleConsumer worker for multiprocessing high-level consumer. It allows to use non-default consumer settings (fetch_size_bytes, buffer_size, max_buffer_size).
* | | Merge pull request #350 from ecanzonieri/enable_commit_on_seekDana Powers2015-03-291-1/+1
|\ \ \ | | | | | | | | Enable commit on seek
| * | | Increase count_since_commit on seekEnrico Canzonieri2015-03-201-1/+1
| |/ / | | | | | | | | | | | | | | | When auto_commit is False this change enables an explicit call to commit() to actually commit the offsets. Otherwise a consumer won't be able to commit until at least one message is read.
* | | Merge pull request #338 from dpkp/structs_reorgDana Powers2015-03-291-23/+26
|\ \ \ | | | | | | | | Fixup indentation in kafka/common.py; add links to protocol definitions
| * | | Fixup indentation in kafka/common.py; add links to protocol definitionsDana Powers2015-03-081-23/+26
| | |/ | |/|
* | | Merge pull request #337 from dpkp/gzip_contextDana Powers2015-03-291-23/+32
|\ \ \ | | | | | | | | Use context managers in gzip_encode / gzip_decode
| * | | Take the linter to kafka/codec.pyDana Powers2015-03-091-11/+10
| | | |
| * | | Gzip context manager not supported in py2.6, so use try/finally insteadDana Powers2015-03-091-2/+17
| | | |
| * | | Use context managers in gzip_encode / gzip_decodeDana Powers2015-03-081-12/+7
| |/ /
* | | Merge pull request #329 from vshlapakov/feature-batch-msg-keysDana Powers2015-03-292-5/+5
|\ \ \ | |_|/ |/| | Correct message keys for async batching mode
| * | Correct message keys for async batching modeViktor Shlapakov2015-02-252-5/+5
| | |
* | | Merge pull request #335 from scrapinghub/fix-mp-consumer-distributionDana Powers2015-03-121-10/+12
|\ \ \ | |_|/ |/| | Wrong partitions distribution logic for MP Consumer
| * | Used thread-safe dict.copy().keys() for MP consumer partitionsViktor Shlapakov2015-03-121-1/+4
| | |
| * | Cleaned code for MP consumer chunkingViktor Shlapakov2015-03-111-10/+9
| | |
| * | Fixing distribution for MP ConsumerViktor Shlapakov2015-03-031-1/+1
| |/
* | Merge branch 'vshlapakov-feature-async-threading'Dana Powers2015-03-082-26/+29
|\ \ | | | | | | | | | | | | | | | | | | PR 330: Threading for async batching Conflicts: kafka/producer/base.py
| * | Async producer stop() fixViktor Shlapakov2015-02-261-2/+2
| | |
| * | Using threading.Event to stop async producer threadViktor Shlapakov2015-02-261-5/+10
| | |
| * | Returned original tests, rm dirty flag, name fixesViktor Shlapakov2015-02-252-25/+17
| | |
| * | Make KafkaConnection copies usable across threadsAlexey Borzenkov2015-02-251-0/+4
| | |
| * | Trying to use threading for async batchingViktor Shlapakov2015-02-251-3/+5
| | |
* | | Merge branch 'jobevers-master'Dana Powers2015-03-081-1/+3
|\ \ \ | | | | | | | | | | | | PR 302: Add failed_payloads attribute to FailedPayloadsError
| * | | Add failed_payloads attribute to FailedPayloadsErrorJob Evers2015-03-081-1/+3
| | | |
* | | | Merge pull request #332 from dpkp/kafka_client_editsDana Powers2015-03-072-14/+11
|\ \ \ \ | | | | | | | | | | Kafka client edits
| * | | | KafkaClient.get_partition_ids_for_topic now returns empty list for unknown ↵Dana Powers2015-03-021-1/+1
| | | | | | | | | | | | | | | | | | | | topics
| * | | | Add comment about no server response case in _send_broker_aware_requestDana Powers2015-03-021-0/+5
| | | | |
| * | | | Avoid topic_partitions KeyError in KafkaClientDana Powers2015-03-021-1/+1
| | | | |
| * | | | Remove KafkaConsumer.BYTES_CONFIGURATION_KEYSDana Powers2015-03-011-11/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - rely on KafkaClient to handle client_id string/bytes encoding - accept either string or bytes for group_id - convert group_id string to utf-8 bytes internally
| * | | | Accept KafkaClient client_id as string or bytes; encode string client_id as ↵Dana Powers2015-03-011-1/+2
| | |_|/ | |/| | | | | | | | | | utf-8 bytes internally
* | | | Merge pull request #324 from chmduquesne/masterDana Powers2015-03-022-0/+9
|\ \ \ \ | | | | | | | | | | Properly destroying the objects that contain daemonized threads
| * | | | Stopping daemonized threads when references to these threads are lostChristophe-Marie Duquesne2015-03-022-0/+9
| |/ / /
* | | | Merge pull request #328 from wkiser/validate_offsetDana Powers2015-03-022-0/+8
|\ \ \ \ | | | | | | | | | | Fixes consumer/kafka and consumer/simple to only yield messages if the m...
| * | | | Fixes consumer/kafka and consumer/simple to only yield messages if the ↵wkiser2015-03-022-0/+8
| |/ / / | | | | | | | | | | | | message's offset is greater than or equal to the consumer offset.
* | | | Merge pull request #296 from ecanzonieri/validate_consumer_offsetDana Powers2015-03-021-3/+54
|\ \ \ \ | |/ / / |/| | | Validate consumer offset in SimpleConsumer
| * | | Retry failed partitions and add integration testsEnrico Canzonieri2015-01-261-0/+2
| | | |
| * | | Merge branch 'master' of github.com:mumrah/kafka-python into ↵Enrico Canzonieri2015-01-2616-342/+381
| |\ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | validate_consumer_offset Conflicts: kafka/consumer/simple.py
| * | | | Make SimpleConsumer auto_offset_reset more like KafkaConsumerEnrico Canzonieri2015-01-261-4/+19
| | | | |
| * | | | use a list in send_offset_requestEnrico Canzonieri2015-01-141-3/+4
| | | | |
| * | | | Implement offsets reset when OffsetOutOfRangeErrorEnrico Canzonieri2015-01-141-3/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This slightly changes the SimpleConsumer interface adding the default option use_latest_offsets. The fetch behaviour is also changed since it does not raise OffsetOutOfRangeError anymore. Resetting the offsets automatically is especially useful in MultiprocessConsumer, where an explicit seek call is not possible.
* | | | | Merge pull request #314 from dpkp/keyed_producer_failoverDana Powers2015-02-194-8/+9
|\ \ \ \ \ | |_|_|_|/ |/| | | | Handle keyed producer failover
| * | | | Always use the initial partition list when hashing keys in KeyedProducerDana Powers2015-02-091-1/+1
| | | | |
| * | | | make partitions arg optional in *Partitioner.partition, defaulting to ↵Dana Powers2015-02-093-7/+8
| | | | | | | | | | | | | | | | | | | | self.partitions from init
* | | | | Merge pull request #317 from sontek/update_consumer_docsDana Powers2015-02-101-2/+5
|\ \ \ \ \ | |/ / / / |/| | | | Updated documentation for Consumers to prefer KafkaConsumer instead
| * | | | Updated documentation for Consumers to prefer KafkaConsumer insteadJohn Anderson2015-02-091-2/+5
| | | | |
* | | | | Always return sorted partition ids in KafkaClient.get_partition_ids_for_topic()Dana Powers2015-02-091-1/+1
|/ / / /
* | | | 2014 -> 2015David Arthur2015-02-031-1/+1
| | | |
* | | | Removing queue.pyDavid Arthur2015-02-031-215/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | It's just collecting dust and throwing off the coverage report. I pushed a branch queue-fixup in case someone wants to take a stab at refactoring it.
* | | | Fix python2.6 threading.Event bug in ReentrantTimerDana Powers2015-02-031-1/+5
| | | |
View Lorry Controller Status