summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
...
| | * task: now deletes itself once its done - for the test this doesn't change a ↵Sebastian Thiel2010-06-083-49/+35
| | | | | | | | | | | | thing as the task deletes itself too late - its time for a paradigm change, the task should be deleted with its RPoolChannel or explicitly by the user. The test needs to adapt, and shouldn't assume anything unless the RPoolChannel is gone
| | * Its getting better already - intermediate commit before further chaning the ↵Sebastian Thiel2010-06-087-41/+52
| | | | | | | | | | | | task class
| | * queue: adjusted queue to be closable ( without own testing yet, except for ↵Sebastian Thiel2010-06-082-50/+62
| | | | | | | | | | | | the pool which runs it ) - its not yet stable, but should be solvable.
| | * The new channeldesign actually works, but it also shows that its located at ↵Sebastian Thiel2010-06-085-90/+93
| | | | | | | | | | | | the wrong spot. The channel is nothing more than an adapter allowing to read multiple items from a thread-safe queue, the queue itself though must be 'closable' for writing, or needs something like a writable flag.
| | * both versions of the async queue still have trouble in certain situations, ↵Sebastian Thiel2010-06-082-26/+65
| | | | | | | | | | | | at least with my totally overwritten version of the condition - the previous one was somewhat more stable it seems. Nonetheless, this is the fastest version so far
| | * test implementation of async-queue with everything stripped from it that ↵Sebastian Thiel2010-06-082-5/+54
| |/ | | | | | | didn't seem necessary - its a failure, something is wrong - performance not much better than the original one, its depending on the condition performance actually, which I don't get faster
| * Merge branch 'stasks' into asyncSebastian Thiel2010-06-075-9/+85
| |\
| | * Task scheduled items lock now uses a dummy lock in serial mode, improving ↵Sebastian Thiel2010-06-072-1/+19
| | | | | | | | | | | | | | | | | | its performance considerably. Channels now use the AsyncQueue, boosting their throughput to about 5k items / s - this is something one can work with, considering the runtime of each item should be large enough to keep the threads busy. This could be a basis, further testing needed
| | * Channel now uses the AsyncQueue, boosting performance by factor 4, its a startSebastian Thiel2010-06-071-2/+2
| | |
| | * introduced a new counter keeping track of the scheduled tasks - this prevent ↵Sebastian Thiel2010-06-074-7/+65
| |/ | | | | | | unnecessary tasks to be scheduled as we keep track of how many items will be produced for the task at hand. This introduces additional locking, but performns well in multithreaded mode. Performance of the master queue is still a huge issue, its currently the limiting factor, as bypassing the master queue in serial moode gives 15x performance, wich is what I would need
| * improved testing to test the actual async handling of the pool. there are ↵Sebastian Thiel2010-06-074-10/+54
| | | | | | | | still inconsistencies that need to be fixed, but it already improved, especially the 4-thread performance which now is as fast as the dual-threaded performance
| * task: Fixed incorrect handling of channel closure. Performance is alright ↵Sebastian Thiel2010-06-073-25/+43
| | | | | | | | for up to 2 threads, but 4 are killing the queue
| * Moved pool utilities into util module, fixed critical issue that caused ↵Sebastian Thiel2010-06-074-126/+176
| | | | | | | | havok - lets call this a safe-state
| * added high-speed locking facilities, allowing our Queue to be faster, at ↵Sebastian Thiel2010-06-073-65/+199
| | | | | | | | least in tests, and with multiple threads. There is still an sync bug in regard to closed channels to be fixed, as the Task.set_done handling is incorrecft
| * Added task order cache, and a lock to prevent us walking the graph while ↵Sebastian Thiel2010-06-073-15/+54
| | | | | | | | | | | | changing tasks Now processing more items to test performance, in dual-threaded mode as well, and its rather bad, have to figure out the reason for this, probably gil, but queues could help
| * changed scheduling and chunksize calculation in respect to the ↵Sebastian Thiel2010-06-071-76/+137
| | | | | | | | task.min_count, to fix theoretical option for a deadlock in serial mode, and unnecessary blocking in async mode
| * pool.consumed_tasks: is now a queue to be thread safe, in preparation for ↵Sebastian Thiel2010-06-073-18/+31
| | | | | | | | | | | | multiple connected pools Reduced waiting time in tests to make them complete faster
| * pool: First version which works as expected in async mode. Its just using a ↵Sebastian Thiel2010-06-076-75/+96
| | | | | | | | single task for now, but next up are dependent tasks
| * channel.read: enhanced to be sure we don't run into non-atomicity issues ↵Sebastian Thiel2010-06-062-18/+73
| | | | | | | | related to our channel closed flag, which is the only way not to block forever on read(0) channels which were closed by a thread 'in the meanwhile'
| * Plenty of fixes in the chunking routine, made possible by a serialized ↵Sebastian Thiel2010-06-065-40/+207
| | | | | | | | chunking test. Next up, actual async processing
| * First step of testing the pool - tasks have been separated into a new module ↵Sebastian Thiel2010-06-068-113/+329
| | | | | | | | including own tests, their design improved to prepare them for some specifics that would be needed for multiprocessing support
| * thread: adjusted worker thread not to provide an output queue anymore - this ↵Sebastian Thiel2010-06-065-41/+209
| | | | | | | | | | | | | | is handled by the task system graph: implemented it including test according to the pools requirements pool: implemented set_pool_size
| * Improved pool design and started rough implementation, top down to learn ↵Sebastian Thiel2010-06-065-76/+314
| | | | | | | | while going. Tests will be written soon for verification, its still quite theoretical
| * Renamed mp to async, as this is a much better name for what is actually ↵Sebastian Thiel2010-06-058-3/+3
| | | | | | | | going on. The default implementation uses threads, which ends up being nothing more than async, as they are all locked down by internal and the global interpreter lock
| * Moved multiprocessing modules into own package, as they in fact have nothing ↵Sebastian Thiel2010-06-058-16/+29
| | | | | | | | to do with the object db. If that really works the way I want, it will become an own project, called async
| * Initial pool design added, allowing for lazy channel based evaluation of ↵Sebastian Thiel2010-06-055-2/+117
| | | | | | | | inter-dependent tasks
| * A code donation: Donating a worker thread implementation inclduding tests to ↵Sebastian Thiel2010-06-053-0/+251
| | | | | | | | Git-Python. I have the feeling it can do much good here :)
| * Added basic channel implementation including testSebastian Thiel2010-06-058-147/+344
|/ | | | restructured odb tests, they are now in an own module to keep the modules small
* Removed compression flag from IStream and OStream types, as a valid object ↵Sebastian Thiel2010-06-053-33/+66
| | | | | | will always be compressed if generated by the system ( even future memory db's will compress it ) loose db: implemented direct stream copy, indicated by a sha set in the IStream, including test. This will be the case once Packs are exploded for instance
* Implemented stream tests, found a bug on the way, slowly a test-framework ↵Sebastian Thiel2010-06-045-251/+490
| | | | for streams starts to show up, but its not yet there
* Merge branch 'odb'Sebastian Thiel2010-06-0429-1863/+3676
|\ | | | | | | | | Conflicts: lib/git/cmd.py
| * Fixed implementation after design change to deal with it - all tests run, ↵Sebastian Thiel2010-06-0414-180/+275
| | | | | | | | but next there will have to be more through testing
| * initial version of new odb design to facilitate a channel based ↵Sebastian Thiel2010-06-049-273/+476
| | | | | | | | multi-threading implementation of all odb functions
| * db: implemented GitObjectDB using the git command to make sure we can lookup ↵Sebastian Thiel2010-06-043-24/+41
| | | | | | | | everything. Next is to implement pack-file reading, then alternates which should allow to resolve everything
| * Fixed compatability issues with python 2.5, made sure all tests runSebastian Thiel2010-06-033-4/+4
| |
| * commit.create_from_tree now uses pure python implementation, fixed message ↵Sebastian Thiel2010-06-0315-82/+230
| | | | | | | | | | | | parsing which truncated newlines although it was ilegitimate. Its up to the reader to truncate therse, nowhere in the git code I could find anyone adding newlines to commits where it is written Added performance tests for serialization, it does about 5k commits per second if writing to tmpfs
| * Added performance comparison to cgit ... and yes, git-python is faster :)Sebastian Thiel2010-06-032-11/+87
| |
| * odb: fixed streamed decompression reader ( specific tests would still be ↵Sebastian Thiel2010-06-033-10/+107
| | | | | | | | missing ) and added performance tests which are extremely promising
| * odb: implemented loose object streaming, which is impossible to do ↵Sebastian Thiel2010-06-036-55/+371
| | | | | | | | efficiently considering that it copies string buffers all the time
| * added frame for object reading, including simple testSebastian Thiel2010-06-024-15/+83
| |
| * initial version of loose object writing and simple cached object lookup ↵Sebastian Thiel2010-06-025-30/+237
| | | | | | | | appears to be working
| * Added first design and frame for object database. In a first step, loose ↵Sebastian Thiel2010-06-025-9/+210
| | | | | | | | | | | | objects will be written using our utilities, and certain object retrieval functionality moves into the GitObjectDatabase which is used by the repo instance Added performance test for object database access, which shows quite respectable tree parsing performance, and okay blob access. Nonetheless, it will be hard to beat the c performance using a pure python implementation, but it can be a nice practice to write it anyway to allow more direct pack manipulations. Some could benefit from the ability to write packs as these can serve as local cache if alternates are used
| * git.cmd: added test for stream section constraint used in git command, found ↵Sebastian Thiel2010-06-022-289/+369
| | | | | | | | bug of course which just didn't kick in yet
| * commit: redesigned revlist and commit parsing, commits are always retrieved ↵Sebastian Thiel2010-06-025-265/+217
| | | | | | | | | | | | from their object information directly. This is faster, and resolves issues with the rev-list format and empty commit messages Adjusted many tests to go with the changes, as they were still mocked. The mock was removed if necessary and replaced by code that actually executes
| * Added performance testing foundation library, reworked existing performance ↵Sebastian Thiel2010-06-022-0/+46
| | | | | | | | tests to work on larger repositories
| * Added commit-iteration testSebastian Thiel2010-06-021-40/+56
| |
| * commit: refactored existing code to decode commits from streams - ↵Sebastian Thiel2010-06-029-718/+810
| | | | | | | | | | | | | | performance is slightly better git.cmd: added method to provide access to the content stream directly. This is more efficient if large objects are handled, if it is actually used test.helpers: removed unnecessary code
| * commit: initial version of commit_from_tree which could create commit ↵Sebastian Thiel2010-06-025-613/+815
| | | | | | | | objects if it could serialize itself
* | git.cmd: using communicate in the main branch of execution, which might not ↵Sebastian Thiel2010-06-031-9/+16
| | | | | | | | | | | | make a big difference, but perhaps its smarter about broken pipes. Adjusted code to selectively strip terminating newline, only if they are there. The previous code would effectively duplicate the string and strip whitespace from both ends even though there was no need for it. Its a bit faster now as the tests proclaim
* | git.cmd: moved hardcoded chunksize when duplicating stream data into ↵Sebastian Thiel2010-06-031-3/+9
|/ | | | easy-to-change class member variable
View Lorry Controller Status