summaryrefslogtreecommitdiff
path: root/taskflow/conductors/backends
Commit message (Collapse)AuthorAgeFilesLines
* Remove sixTakashi Kajinami2022-05-182-5/+2
| | | | | | | | This library no longer supports Python 2, thus usage of six can be removed. This also removes workaround about pickle library used in Python 2 only. Change-Id: I19d298cf0f402d65f0b142dea0bf35cf992332a9
* Import modules, not classesSean McGinnis2020-05-011-6/+2
| | | | | | | | This addresses comments from I24b13c8654a33d99bf687a44a36cfdace39e3866 to follow guidelines to only import modules, not classes. Change-Id: I0e66c08bfcdff306bac0121602d45b56cfb2c712 Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
* set default python to python3Doug Hellmann2018-04-171-1/+1
| | | | | | | | | | | | | | | | | | Set the default python to python3 except for the py27 environment. We have to set that explicitly to override the new default. Have py27 and py3* jobs run same commands by removing the commands entry for py27. The pep8 rules under python 3 are different, so we need to fix a couple of issues with blank space. The E721 rule is ignored entirely because we are not yet using a version of flake8 that supports ignoring individual rules on specific lines, although the instructions for that are added so that when we upgrade flake8 to a newer version we can remove the global rule. Change-Id: Id236b0632a83a188958130d04f1e9c476c5ace65 Signed-off-by: Doug Hellmann <doug@doughellmann.com>
* Remove kwarg timeout in executor conductorchenghuiyu2017-09-011-3/+1
| | | | | | | The method stop returns immediately regardless of whether the conductor has been stopped, so kwarg timeout is no longer necessary, and it is deprecated for removal. Change-Id: I50e5cbb1a00eb9b6f358f150716d2a5d682a49c9
* Remove deprecated things for 2.0 releaseJoshua Harlow2016-05-101-20/+0
| | | | Change-Id: Id9adbc50bd51adc77ce88f698ad0ea2ee63fc5e2
* Make conductor.stop stop the running engine gracefullyGreg Hill2016-05-031-2/+18
| | | | | | | | | | | | Previously stopping the conductor would only prevent it from accepting new jobs. This change makes it abort the current job by telling the engine to stop processing tasks after the current ones finish. This allows for conductors to gracefully exit when receiving a kill signal (although the signal handling is not done automatically yet). Change-Id: Ie6ddcbb2df4508ad1e3f6698c6f4cb2fc26a278f
* Add periodic jobboard refreshing (incase of sync issues)Joshua Harlow2016-03-151-2/+20
| | | | | | Related-Bug: #1557107 Change-Id: I42672ef63ef02ec5ec6a842d263d0db83d91fe45
* Merge "Move 'convert_to_timeout' to timing type as a helper function"Jenkins2015-12-241-12/+1
|\
| * Move 'convert_to_timeout' to timing type as a helper functionJoshua Harlow2015-11-121-12/+1
| | | | | | | | | | | | | | | | Also adds some basic tests to this newly exposed helper function so it continues to operate as expected and better docstrings on the timeout types methods. Change-Id: I9fa4c7d313084800d49cfc77f6ca93afcff1169d
* | Avoid recreating notify details for each dispatch iterationJoshua Harlow2015-12-141-10/+7
| | | | | | | | | | | | | | | | We can just create the notification details once and reuse it for each iteration, instead of recreating it for each iteration; saves some resources... Change-Id: Ie8ad61647afe2eef813af87a915710cf91de8e46
* | Adding notification points for job completionMin Pae2015-11-161-0/+24
|/ | | | | | | Adding notifications for job completion, both consumed and abandoned, so that a listener can take some action based on job completion. Change-Id: I826285d4bfccd2406df7b59e53a9b724702ed094
* Add a executor backed conductor and have existing impl. use itJoshua Harlow2015-11-123-241/+419
| | | | | | | | | | | | | | | | | | | This adds a executor backed job dispatching base class and has the existing blocking executor use it by running jobs and dispatching jobs into a sync executor. It also allows for dispatching jobs into a thread executor, or other executor via a new '_executor_factory' method that can generate executors (it can be overriden in the non-blocking executor to provide your own executors instances). This does alter the behavior in that now that jobs are dispatched into an executor we no longer can immediatly know if a job was dispatched and raised an exception or whether it will raise an exception in the future, so we now alter the 'local_dispatched' to just be a boolean that is used to determine if any dispatches happened (failure or not). Change-Id: I485770e8f4c85d3833892a453c9fb5168d8f0407
* Register conductor information on jobboardtonytan4ever2015-10-191-0/+26
| | | | Change-Id: I3bf935280a6e8b265045b09fde43d0ec7dc56f07
* feat: add max_dispatches arg to conductor's runSriram Madapusi Vasudevan2015-10-151-7/+27
| | | | | | | | | | | | | - This will cause the conductor to only do 'n' number of dispatches, after which it stops dispatching jobs. - This will allow the code that call the conductor, to monitor conductor.dispatching, and make a decision on what is to be done with it. Eg: Decomission a conductor, restart the conductor etc - Backward Compatible. Change-Id: I3386c7050806806b5ee44a74ba93e50515a5ab7b
* Update all removal_version from being ? to being 2.0Joshua Harlow2015-07-121-1/+1
| | | | | | | | Before the 2.0 release it would be great to get these removed so let's mark that version as the version that will no longer have these in it. Change-Id: I66a74d270bf95db005e9febfce1a5e211c7a49f6
* Merge "Notify on the individual engine steps"Jenkins2015-07-081-2/+26
|\
| * Notify on the individual engine stepsJoshua Harlow2015-07-081-2/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When a conductor is running it is quite useful to be able to how long each engine step takes. To enable this information being output, add a notifier to the base conductor and use it in the blocking conductor to emit events around engine activities. This makes it possible to track the timing (or other information that can be gathered from these events) in a non-intrusive manner. In the `99_bottles.py` demo we also now use this to be able to easily see what the conductor is actively doing (without having to enable the more verbose DEBUG level logging). Change-Id: Ifd8ff38f82fc8135fe5fec4c8e41f0e06f4fdee3
* | Merge "Handle conductor ctrl-c more appropriately"Jenkins2015-07-081-0/+4
|\ \ | |/
| * Handle conductor ctrl-c more appropriatelyJoshua Harlow2015-06-181-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | When a conductor program is interrupted via ctrl-c or equivalent it is much nicer log that that has happened and to reraise that exception. This also slightly tweaks the 99 bottles song to make it even better, by having more pieces/tasks, which makes it possible to kill the program during each task and see how the resumption works when a flow is composed of segments. Change-Id: I5d242eba9a043ef96646ba74ea5928daa0691ed0
* | Merge "Remove 2.6 classifier + 2.6 compatibility code"Jenkins2015-06-291-2/+3
|\ \
| * | Remove 2.6 classifier + 2.6 compatibility codeJoshua Harlow2015-06-211-2/+3
| |/ | | | | | | | | | | | | | | Fixes bug 1445827 Depends-On: I02e3c9aacef0b295a2f823a5cbaf11768a90cb82 Change-Id: I1db681803598ac1bc917fd74a99458bc61edf3f1
* | Use sphinx deprecated docstring markupJoshua Harlow2015-06-101-4/+6
|/ | | | | | | | | | Along with using the `debtcollector` library we should also be marking the docstrings of deprecated functions/methods/properties with the appropriate deprecated information so that the generated docs also show what is deprecated. Change-Id: I8a1960731fd53434bf04d22cc6ff208998adb4f3
* Remove listener stack and replace with exit stackJoshua Harlow2015-05-111-3/+8
| | | | Change-Id: Ic9150b8208420d29865af297ae8d0c656becb81e
* Move to the newer debtcollector provided functionsJoshua Harlow2015-03-201-3/+2
| | | | | | | | Use the debtcollector library provided variants of these functions and remove our versions to avoid more duplications. Change-Id: I526789f44899afd284803b127bec5bb349d1aa2b
* Ensure we register & deregister conductor listenersJoshua Harlow2015-03-131-3/+12
| | | | | | | | | | Instead of just registering engine listeners that were returned, make sure we also deregister them when the engine has either finished or failed. This ensures that if a listener has hold of any resources (or other) that it can clean those up and be sure that its deregister call will be made. Change-Id: Ia1420c435156362698702fed2bda11c2a0fef803
* Stick to one space after a periodJoshua Harlow2015-02-241-2/+2
| | | | Change-Id: I72666a24883e5b42e1ef76d0146d3da40d376ff8
* Allow loading conductors via entrypointsJoshua Harlow2015-02-162-0/+220
Add an easy to use helper function that is exposed to fetch a conductor in a way that matches how engines and persistence backends and jobboards are loaded. This also adjusts the single threaded conductor to be now named blocking conductor and exposes an entrypoint backend fetch() function to load current and future conductors. Change-Id: Id146d847c329d3e8510a8f24c3ec8b918680ddb5
View Lorry Controller Status