summaryrefslogtreecommitdiff
path: root/lib
Commit message (Collapse)AuthorAgeFilesLines
...
* improved testing to test the actual async handling of the pool. there are ↵Sebastian Thiel2010-06-073-5/+30
| | | | still inconsistencies that need to be fixed, but it already improved, especially the 4-thread performance which now is as fast as the dual-threaded performance
* task: Fixed incorrect handling of channel closure. Performance is alright ↵Sebastian Thiel2010-06-072-25/+39
| | | | for up to 2 threads, but 4 are killing the queue
* Moved pool utilities into util module, fixed critical issue that caused ↵Sebastian Thiel2010-06-074-126/+176
| | | | havok - lets call this a safe-state
* added high-speed locking facilities, allowing our Queue to be faster, at ↵Sebastian Thiel2010-06-072-58/+186
| | | | least in tests, and with multiple threads. There is still an sync bug in regard to closed channels to be fixed, as the Task.set_done handling is incorrecft
* Added task order cache, and a lock to prevent us walking the graph while ↵Sebastian Thiel2010-06-072-6/+23
| | | | | | changing tasks Now processing more items to test performance, in dual-threaded mode as well, and its rather bad, have to figure out the reason for this, probably gil, but queues could help
* changed scheduling and chunksize calculation in respect to the ↵Sebastian Thiel2010-06-071-76/+137
| | | | task.min_count, to fix theoretical option for a deadlock in serial mode, and unnecessary blocking in async mode
* pool.consumed_tasks: is now a queue to be thread safe, in preparation for ↵Sebastian Thiel2010-06-072-11/+23
| | | | | | multiple connected pools Reduced waiting time in tests to make them complete faster
* pool: First version which works as expected in async mode. Its just using a ↵Sebastian Thiel2010-06-074-54/+55
| | | | single task for now, but next up are dependent tasks
* channel.read: enhanced to be sure we don't run into non-atomicity issues ↵Sebastian Thiel2010-06-061-17/+72
| | | | related to our channel closed flag, which is the only way not to block forever on read(0) channels which were closed by a thread 'in the meanwhile'
* Plenty of fixes in the chunking routine, made possible by a serialized ↵Sebastian Thiel2010-06-063-17/+58
| | | | chunking test. Next up, actual async processing
* First step of testing the pool - tasks have been separated into a new module ↵Sebastian Thiel2010-06-065-111/+246
| | | | including own tests, their design improved to prepare them for some specifics that would be needed for multiprocessing support
* thread: adjusted worker thread not to provide an output queue anymore - this ↵Sebastian Thiel2010-06-063-37/+127
| | | | | | | is handled by the task system graph: implemented it including test according to the pools requirements pool: implemented set_pool_size
* Improved pool design and started rough implementation, top down to learn ↵Sebastian Thiel2010-06-063-68/+290
| | | | while going. Tests will be written soon for verification, its still quite theoretical
* Renamed mp to async, as this is a much better name for what is actually ↵Sebastian Thiel2010-06-054-0/+0
| | | | going on. The default implementation uses threads, which ends up being nothing more than async, as they are all locked down by internal and the global interpreter lock
* Moved multiprocessing modules into own package, as they in fact have nothing ↵Sebastian Thiel2010-06-054-13/+26
| | | | to do with the object db. If that really works the way I want, it will become an own project, called async
* Initial pool design added, allowing for lazy channel based evaluation of ↵Sebastian Thiel2010-06-052-0/+105
| | | | inter-dependent tasks
* A code donation: Donating a worker thread implementation inclduding tests to ↵Sebastian Thiel2010-06-052-0/+204
| | | | Git-Python. I have the feeling it can do much good here :)
* Added basic channel implementation including testSebastian Thiel2010-06-052-6/+111
| | | | restructured odb tests, they are now in an own module to keep the modules small
* Removed compression flag from IStream and OStream types, as a valid object ↵Sebastian Thiel2010-06-052-27/+17
| | | | | | will always be compressed if generated by the system ( even future memory db's will compress it ) loose db: implemented direct stream copy, indicated by a sha set in the IStream, including test. This will be the case once Packs are exploded for instance
* Implemented stream tests, found a bug on the way, slowly a test-framework ↵Sebastian Thiel2010-06-042-31/+52
| | | | for streams starts to show up, but its not yet there
* Merge branch 'odb'Sebastian Thiel2010-06-0415-1174/+2428
|\ | | | | | | | | Conflicts: lib/git/cmd.py
| * Fixed implementation after design change to deal with it - all tests run, ↵Sebastian Thiel2010-06-049-151/+211
| | | | | | | | but next there will have to be more through testing
| * initial version of new odb design to facilitate a channel based ↵Sebastian Thiel2010-06-045-262/+465
| | | | | | | | multi-threading implementation of all odb functions
| * db: implemented GitObjectDB using the git command to make sure we can lookup ↵Sebastian Thiel2010-06-043-24/+41
| | | | | | | | everything. Next is to implement pack-file reading, then alternates which should allow to resolve everything
| * Fixed compatability issues with python 2.5, made sure all tests runSebastian Thiel2010-06-031-2/+2
| |
| * commit.create_from_tree now uses pure python implementation, fixed message ↵Sebastian Thiel2010-06-037-64/+104
| | | | | | | | | | | | parsing which truncated newlines although it was ilegitimate. Its up to the reader to truncate therse, nowhere in the git code I could find anyone adding newlines to commits where it is written Added performance tests for serialization, it does about 5k commits per second if writing to tmpfs
| * Added performance comparison to cgit ... and yes, git-python is faster :)Sebastian Thiel2010-06-031-4/+27
| |
| * odb: fixed streamed decompression reader ( specific tests would still be ↵Sebastian Thiel2010-06-032-10/+16
| | | | | | | | missing ) and added performance tests which are extremely promising
| * odb: implemented loose object streaming, which is impossible to do ↵Sebastian Thiel2010-06-035-55/+327
| | | | | | | | efficiently considering that it copies string buffers all the time
| * added frame for object reading, including simple testSebastian Thiel2010-06-023-15/+70
| |
| * initial version of loose object writing and simple cached object lookup ↵Sebastian Thiel2010-06-024-28/+208
| | | | | | | | appears to be working
| * Added first design and frame for object database. In a first step, loose ↵Sebastian Thiel2010-06-023-9/+137
| | | | | | | | | | | | objects will be written using our utilities, and certain object retrieval functionality moves into the GitObjectDatabase which is used by the repo instance Added performance test for object database access, which shows quite respectable tree parsing performance, and okay blob access. Nonetheless, it will be hard to beat the c performance using a pure python implementation, but it can be a nice practice to write it anyway to allow more direct pack manipulations. Some could benefit from the ability to write packs as these can serve as local cache if alternates are used
| * git.cmd: added test for stream section constraint used in git command, found ↵Sebastian Thiel2010-06-021-1/+6
| | | | | | | | bug of course which just didn't kick in yet
| * commit: redesigned revlist and commit parsing, commits are always retrieved ↵Sebastian Thiel2010-06-021-58/+40
| | | | | | | | | | | | from their object information directly. This is faster, and resolves issues with the rev-list format and empty commit messages Adjusted many tests to go with the changes, as they were still mocked. The mock was removed if necessary and replaced by code that actually executes
| * commit: refactored existing code to decode commits from streams - ↵Sebastian Thiel2010-06-025-680/+797
| | | | | | | | | | | | | | performance is slightly better git.cmd: added method to provide access to the content stream directly. This is more efficient if large objects are handled, if it is actually used test.helpers: removed unnecessary code
| * commit: initial version of commit_from_tree which could create commit ↵Sebastian Thiel2010-06-023-510/+679
| | | | | | | | objects if it could serialize itself
* | git.cmd: using communicate in the main branch of execution, which might not ↵Sebastian Thiel2010-06-031-9/+16
| | | | | | | | | | | | make a big difference, but perhaps its smarter about broken pipes. Adjusted code to selectively strip terminating newline, only if they are there. The previous code would effectively duplicate the string and strip whitespace from both ends even though there was no need for it. Its a bit faster now as the tests proclaim
* | git.cmd: moved hardcoded chunksize when duplicating stream data into ↵Sebastian Thiel2010-06-031-3/+9
|/ | | | easy-to-change class member variable
* gitcmd: may now receive extra keyword arguments to be passed directly to the ↵Sebastian Thiel2010-05-311-11/+9
| | | | subproces.Popen invocation. It could be used to pass custom environments, without changing the own one (#26)
* cmd: By default, on linux, the parent file handles will be closed to leave ↵Sebastian Thiel2010-05-271-0/+1
| | | | the child less cluttered, and make it easier to debug as it will only have the file descriptors we set. It appears to be more stable regarding the stdin-is-closed-but-child-doesn't-realize-this issue
* index: index-add fixed to always append a newline after each item. In git ↵Sebastian Thiel2010-05-261-23/+10
| | | | | | has unified its way it reads from stdin, now it wants all items to be terminated by a newline usually. Previously, it could have been that it really didn't want to have a termination character when the last item was written to the file. Bumped the minimum requirements to 1.7.0 to be sure it is working as I think it will. Still, I have to admit that sometime it just appears the closed pipe will not stop git from waiting for more input, at least with the previous implementation
* refs: a Reference can now be created by assigning a commit or object (for ↵Sebastian Thiel2010-05-261-11/+40
| | | | convenience)
* BlockingLockFile: added sanity check that raises IOError if the directory ↵Sebastian Thiel2010-05-261-4/+15
| | | | containing the lock was removed. This is unlikely to happen in a production envrironment, but may happen during testing, as folders are moved/deleted once the test is complete. Daemons might still be waiting for something, and they should be allowed to terminate instead of waiting for a possibly long time
* diff: by limiting the splitcount to 5, a subtle bug was introduced as the ↵0.2.0-beta1Sebastian Thiel2010-05-251-0/+1
| | | | | | newline at the end of the split line was not split away automatically. Added test for this, and the trivial fix Wow, at least two people reviewd the code, but it slipped through anyway :)
* Repo: Added comparison operators and hash operator including testSebastian Thiel2010-05-122-2/+22
| | | | Cmd: AutoInterrupt handles boundary cases more gracefully as it can be that the os module suddenly becomes None if the interpreter is going down
* IndexFile.add: Fixed incorrect path handling if path rewriting was desired ↵Sebastian Thiel2010-05-113-9/+10
| | | | | | | | and absolute paths were given Commit.create_from_tree: fixed critical bug that would cause it to create a branch named master by default, instead of the reference actually set ( which is master in many, but not all cases ) - in fact it could be detached as well, we would fail ungracefully although we could assume master then ... although we cant really make the decision Repo.is_dirty: improved its abiility to deal with empty repositories and a missing head. Weird thing is that the test always worked fine with the previous code, but it didn't work for me in a similar situation without this change at least
* Handle filenames with embedded spaces when generating diffsRick Copeland2010-05-101-1/+1
|
* index.add: added index path rewrite functionality, which allows to store a ↵Sebastian Thiel2010-05-101-1343/+1402
| | | | | | | different path in the index than the actual one on disk ( from which the object will be created ) Fixed bug the way newlines were handled, which hopefully fixes occasional hangs as well. It works fine with git 1.7.1 Most of the changes are due to the tab-space conversion - its weird once more as I thought it was all in spaces before ... .
* repo: added test with some basic assertions for empty repositories theseSebastian Thiel2010-05-104-1358/+1347
| | | | | | | | repo.is_dirty: Will not fail on empty repo ( anymore ) index.entries: will just be empty if the repository is empty refs: added to_full_path method which can be used to create fully synthetic instances of Reference types, added a test for it Converted all touched files to spaces, which is why git reports so many changed files. Actually I was thinking every file would use spaces, but apparently not
* TODO: Removed all entries but left a mesage about where to find the issuee ↵Sebastian Thiel2010-05-041-1/+1
| | | | | | | on lighthouse. README/intro.rst: added information about the new repository at github tree: added marker to indicate that submodules would have to be returned there