| Commit message (Collapse) | Author | Age | Files | Lines |
| |
|
|
|
|
| |
the rule of trying not to cache possibly heavy data. The data_stream method should be used instead
|
|
|
|
| |
the submodules's naming conventions
|
|
|
|
| |
replaced them by a real test which actually executes code, and puts everything into the tmp directory
|
| |
|
| |
|
| |
|
|
|
|
| |
to using git-read-tree to keep the stat information when merging one tree in. After all this is what needed to be implemented in python as well
|
|
|
|
| |
The default is to write the physical index, which is the behaviour you would expect
|
|
|
|
|
|
| |
was actually empty. This is a rare case that can happen during stream testing. Theoretically there shouldn't be any empty streams of course, but practically they do exist sometimes ;); fixed stream.seek implementation, which previously used seek on standard output
Improved GitCmd error handling
|
|
|
|
|
|
| |
not implemented causing incorrect merge results. Added test to cover this issue
Diff: added NULL_BIN_SHA constant for completeness
|
| |
|
|
|
|
| |
including simple test, it may be simple as the methods it uses are throroughly tested
|
| |
|
|
|
|
| |
in progress
|
|
|
|
|
|
|
| |
can do much more than we can ( and faster assumably ), the .new method is used to create new index instances from up to 3 trees.
Implemented multi-tree traversal to facilitate building a stage list more efficiently ( although I am not sure whether it could be faster to use a dictionary together with some intensive lookup ), including test
Added performance to learn how fast certain operations are, and whether one should be preferred over another
|
|
|
|
| |
IO will only be done when required. A possible disadvantage though is that time is spent on compressing the trees, although only the raw data and their shas would theoretically be needed. On the other hand, compressing their data uses less memory. An optimal implementation would just sha the data, check for existance, and compress it to write it to the database right away. This would mean more specialized code though, introducing redundancy. If IStreams would know whether they contain compressed or uncompressed data, and if there was a method to get a sha from data, this would work nicely in the existing framework though
|
| |
|
|
|
|
|
|
|
| |
correctly, a test to explicitly compare the git version with the python implementation is still missing
Tree and Index internally use 20 byte shas, converting them only as needed to reduce memory footprint and processing time
objects: started own 'fun' module containing the most important tree functions, more are likely to be added soon
|
|
|
|
| |
faster as it removes one level of indirection, and makes the main file smaller, improving maintainability
|
|
|
|
| |
python version is about as fast, but could support multithreading using async
|
| |
|
|
|
|
|
|
| |
is actually more efficient than the previous implementation
Index now locks its file for reading, and properly uses LockedFD when writing
|
|
|
|
| |
fast, while staying compatible with serialization which requires it to be sorted
|
|
|
|
|
|
|
| |
according to a simple test
( presort still needs implementation )
submodule: added stub to allow the tree to return something, its not implemented though
|
|
|
|
|
|
| |
was added instead
Adjusted all imports to deal with the changed package names
|
| |
|
|
|
|
| |
on starting with just the main thread
|
|
|
|
|
|
| |
anymore, but are abstract.
Added IteratorReader, implementing the reader interface from an iterator. The implementation moved from the TaskIterator to the channel
|
|
|
|
| |
default. It shows that there must be the notion of a producer, which can work if there are no items read
|
| |
|
|
|
|
| |
works on py2.4, 2.5 and 2.6
|
|
|
|
| |
printing thanks to the underlying threading implementation, we can at least make sure that the interpreter doesn't block during shutdown. Now it appears to be running smoothly
|
|
|
|
| |
like to see
|
|
|
|
| |
utilities, as well as the tests themselves. File became too large
|
|
|
|
| |
lock is not actually released or they are not actually notifyied, staying in a beautysleep. This glitch is probably caused by some detail not treated correctly in the thread python module, which is something we cannot fix. It works most of the time as expected though - maybe some cleanup is not done correctly which causes this
|
|
|
|
| |
well as concurrency issues. Now it works okay, but the thread-shutdown is still an issue, as it causes incorrect behaviour making the tests fail. Its good, as it hints at additional issues that need to be solved. There is just a little more left on the feature side, but its nearly there
|
|
|
|
| |
closed only when there is no one else writing to it. This assures that all tasks can continue working, and put their results accordingly. Shutdown is still not working correctly, but that should be solvable as well. Its still not perfect though ...
|
|
|
|
|
|
| |
readers and writers, a ready is not connected to its writer anymore. This changes the refcounting of course, which is why the auto-cleanup for the pool is currently broken.
The benefit of this are faster writes to the channel, reading didn't improve, refcounts should be clearer now
|
|
|
|
| |
fully deterministic as tasks still run into the problem that they try to write into a closed channel, it was closed by one of their task-mates who didn't know someone else was still computing
|
|
|
|
| |
the same and different pools
|
|
|
|
| |
allows the pool to work as expected. Many more tests need to be added, and there still is a problem with shutdown as sometimes it won't kill all threads, mainly because the process came up with worker threads started, which cannot be
|
|
|
|
| |
reference counting mechanism, causing references to the pool to be kepts via cycles
|
|
|
|
| |
channel implementation, one of which is used as base by the Pool Read channel, releasing it of the duty to call these itself. The write channel with callback subclass allows the transformation of the item to be written
|
|
|
|
|
|
| |
makes it easier to constomize
pool: in serial mode, created channels will be serial-only, which brings 15% of performance
|
|
|
|
| |
and it runs faster as well, about 2/3 of the performance we have when being in serial mode
|
|
|
|
|
|
| |
Condition implementation, related to the notify method not being treadsafe. Although I was aware of it, I missed the first check which tests for the size - the result could be incorrect if the whole method wasn't locked.
Testing runs stable now, allowing to move on \!
|
|
|
|
| |
one more level of indirection. Clearly this not good from a design standpoint, as a Condition is no Deque, but it helps speeding things up which is what this is about. Could make it a hidden class to indicate how 'special' it is
|
|
|
|
|
| |
queue: Queue now derives from deque directly, which safes one dict lookup as the queue does not need to be accessed through self anymore
pool test improved to better verify threads are started correctly
|
|
|
|
|
|
| |
thread-safe, causing locks to be released multiple times. Now it runs very fast, and very stable apparently.
Now its about putting previous features back in, and studying their results, before more complex task graphs can be examined
|