summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* Starting work on fixing offset commitsissue-26David Arthur2013-06-012-5/+63
| | | | | | | | * Update the "public" offset before yielding the message * Add an option to SimpleConsumer.commit that excludes the current offset Ref #26
* PEP8-ify most of the filesMahendra M2013-05-297-128/+309
| | | | consumer.py and conn.py will be done later after pending merges
* Removing the bit about offsetsDavid Arthur2013-05-291-5/+0
|
* Minor bug fixesMahendra M2013-05-292-12/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * When you initiate a producer with a non-existant queue, the queue is created. However this partition info is not reflected in KafkaClient() immediately. So, we wait for a second and try loading it again. Without this fix, if we do producer.send_messages() after creating a new queue, the library will throw a StopIteration exception. * In SimpleConsumer(), the defaults are not as mentioned in the comments. Fix this (or do we change the documentation?) * There was a problem with the way the consumer iterator worked. for eg: assume that there were 10 messages in the queue/topic and you iterate over it as - for msg in consumer: print (msg) At the end of this, 'offset' that is saved is 10. So, if you run the above loop again, the last message (10) is repeated. This can be fixed by adjusting the offset counter before fetching the message * Avoid some code repeat in consumer.commit() * Fix a bug in send_offset_commit_request() invocation in consumer.py * Fix missing imports
* Merge branch 'issue-22'David Arthur2013-05-282-0/+45
|\ | | | | | | | | Conflicts: kafka/consumer.py
| * Closes #22David Arthur2013-05-282-20/+21
| | | | | | | | Removed get_messages API, added test for get_pending
| * Missed a doc stringMahendra M2013-05-271-0/+2
| |
| * New API for getting a specified set of messagesMahendra M2013-05-271-0/+18
| | | | | | | | | | | | | | | | | | This will be easier to use in some cases where we have to get only a specified set of messages. This API uses the __iter__ API internally, but maintains the state to give back only the required set of messages API is - get_messages(count=1)
| * New API for checking pending message countMahendra M2013-05-271-0/+24
| |
* | Adding a debug statementDavid Arthur2013-05-281-1/+2
| |
* | Merge pull request #23 from mahendra/autocommitDavid Arthur2013-05-282-13/+23
|\ \ | |/ |/| Auto commit timer is not periodic
| * Auto commit timer is not periodicMahendra M2013-05-282-13/+23
|/ | | | | | | The auto commit timer is one-shot. After the first commit, it does not fire again. This ticket fixes the issue. Also, in util.ReentrantTimer(), some duplicate code was cleaned up
* Update README.mdDavid Arthur2013-05-071-1/+5
|
* Fixes #14David Arthur2013-04-111-1/+1
|
* A few fixes for offset APIs in 0.8.1David Arthur2013-04-022-3/+1
|
* Update README and setup.py with new versionDavid Arthur2013-04-022-44/+57
|
* Update kafka-src to latest trunk, enable 0.8.1 featuresDavid Arthur2013-04-024-5/+3
|
* Typo in example.pyDavid Arthur2013-04-021-1/+1
|
* Update kafka-src to latest 0.8David Arthur2013-04-024-5/+17
| | | | Fix a broken test (100k was too much to send in one batch)
* Update README with new APIsDavid Arthur2013-04-021-94/+19
|
* Bring acks and timeout down to the clientDavid Arthur2013-04-023-10/+11
|
* Refactoring a bit, cleanup for 0.8David Arthur2013-04-028-183/+215
| | | | Marking some stuff as not compatible for 0.8 (will be added in 0.8.1)
* Big code re-orgDavid Arthur2013-04-0211-673/+868
|
* Some work on a simple consumerDavid Arthur2013-04-023-11/+91
|
* Started on a simple producer and consumerDavid Arthur2013-04-023-7/+84
|
* Removing __main__ stuff from client.pyDavid Arthur2013-04-021-66/+0
|
* Integration tests passingDavid Arthur2013-04-028-1384/+761
|
* Protocol and low-level client done, adding testsDavid Arthur2013-04-026-272/+396
|
* Fix a bunch of bugsDavid Arthur2013-04-022-65/+138
|
* First pass of cleanup/refactoringDavid Arthur2013-04-023-225/+487
| | | | Also added a bunch of docstrings
* Starting work on 0.8 compatDavid Arthur2013-04-022-0/+562
|
* Adding client_fetch_size to queue interfaceDavid Arthur2013-04-022-3/+57
| | | | Also more docs
* Fix some typos, closes #12David Arthur2013-01-261-2/+2
|
* Merge pull request #11 from anentropic/masterDavid Arthur2012-11-221-2/+2
|\ | | | | fix docs: iter_messages doesn't (any longer?) take a FetchRequest
| * fix docs: iter_messages doesn't (any longer?) take a FetchRequestanentropic2012-11-221-2/+2
|/ | | ...it takes the same args and instantiates its own one
* Add some docs and KafkaQueue configDavid Arthur2012-11-192-19/+48
| | | | Ref #8
* Add a Queue-like producer/consumerDavid Arthur2012-11-193-0/+148
| | | | | | | | | | | | | | | | | Creates a producer process and one consumer process per partition. Uses `multiprocessing.Queue` for communication between the parent process and the producer/consumers. ```python kafka = KafkaClient("localhost", 9092) q = KafkaQueue(kafka, client="test-queue", partitions=[0,1]) q.put("test") q.get() q.close() kafka.close() ``` Ref #8
* Update README.mdDavid Arthur2012-11-161-2/+4
|
* Add Snappy support0.1-alphaDavid Arthur2012-11-166-21/+144
| | | | Fixes #2
* Update LICENSE with author+yearDavid Arthur2012-11-161-2/+1
|
* Integration test writes to a file instead of stdoutDavid Arthur2012-10-301-5/+6
|
* Clean up imports in client, fixed #5David Arthur2012-10-301-3/+0
|
* Replace socket.send with socket.sendall, Fixes #6David Arthur2012-10-301-5/+5
|
* Adding a test for 10k messagesDavid Arthur2012-10-301-0/+10
|
* Merge pull request #4 from benfred/masterDavid Arthur2012-10-291-1/+1
|\ | | | | exception handling fix
| * error handling fixBen Frederickson2012-10-251-1/+1
|/
* Fixing install instructionsDavid Arthur2012-10-031-2/+3
|
* Adding link to KafkaDavid Arthur2012-10-031-0/+2
|
* Updating kafka-src to 0.7.2David Arthur2012-10-032-1/+1
|
* Isn't it nice when tests actually find bugsDavid Arthur2012-10-022-6/+51
|
View Lorry Controller Status