| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
| |
This library no longer supports Python 2, thus usage of six can be
removed. This also removes workaround about pickle library used in
Python 2 only.
Change-Id: I19d298cf0f402d65f0b142dea0bf35cf992332a9
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
With the removal of Python 2.x we can remove the unittest2 compat
wrappers and switch to assertCountEqual instead of assertItemsEqual
We have been able to use them since then, because
testtools required unittest2, which still included it. With testtools
removing Python 2.7 support [3][4], we will lose support for
assertItemsEqual, so we should switch to use assertCountEqual.
[1] - https://bugs.python.org/issue17866
[2] - https://hg.python.org/cpython/rev/d9921cb6e3cd
[3] - testing-cabal/testtools#286
[4] - testing-cabal/testtools#277
Change-Id: Iaa8251a1e9965a00fe99b7a740a104c011260340
|
| |
|
|
|
|
|
|
|
|
|
| |
The networkx 2.x series has been out for two years now and supports
python 3.6 and greater[1]. This patch updates TaskFlow to require
a minimum of networkx 2.1. It also updates the code to support
recent deprecation expiration introduced in the 2.4 release.
[1] https://networkx.github.io/documentation/stable/news.html
Change-Id: Ife31d353ba80824ebc63c8b21ee90943badc8da3
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
With the release of NetworkX 2.0 the reporting API was
moved to view/iterator model. Many methods were moved from
reporting lists or dicts to iterating over the information.
Methods that used to return containers now return views and
methods that returned iterators have been removed in networkx.
Because of this change in NetworkX 2.0 , taskflow code
have to be changed also to support networkx > 2.0
Change-Id: I23c226f37bd85c1e38039fbcb302a2d0de49f333
Closes-Bug: #1778115
|
| |
|
|
|
|
|
|
|
|
| |
assertRaisesRegexp was renamed to assertRaisesRegex in Py3.2
For more details, please check:
https://docs.python.org/3/library/
unittest.html#unittest.TestCase.assertRaisesRegex
Change-Id: I89cce19e80b04074aab9f49a76c7652acace78b3
Closes-Bug: #1436957
|
| |
|
|
|
|
|
|
|
|
| |
Following OpenStack Style Guidelines[1]:
http://docs.openstack.org/developer/hacking/#unit-tests-and-assertraises
[H203] Unit test assertions tend to give better messages for more
specific assertions. As a result, assertIsNone(...) is preferred
over assertEqual(None, ...) and assertIs(..,None).
Change-Id: I52ecf3f4945c4cb99f6350afca1c51d88b16bb8d
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
For a local process based executor usage currently to ensure
that task emitted notifications are proxied we use the multi
processing library and use its queue concept. This sadly creates
a proxy process that gets associated, and this proxy process
handles the queue and messages sent to and from it. Instead of
doing this we can instead just create a temporary local socket
using a random socket and have tasks (which are running in
different processes) use that to communicate back any emitted
notifications instead (and we can use the asyncore module to handle
the emitted notifications since it handles the lower level socket
reading, polling and dispatching).
To ensure that the socket created is somewhat secure we use a
similar process as the multi-processing library uses where we
sign all messages with a hmac that uses a one time key that only
the main process and the child process know about (and reject
any messages that do not validate using this key).
Change-Id: Iff9180054bf14495e5667af00ae2fafbdbc23791
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It can be quite nice to expose a basic set of metrics
about the internals of an engine, including the time
in each state, and how long the engine is active for
and likely more in the future.
To start add a engine statistics property and gather
some basic timing data and place this data into this
property for access (and/or introspection) by users.
Part of blueprint gather-engine-statistics
Change-Id: Ibc3c78755bd8ae779b52fc4772519f243a521576
|
| |
|
|
|
|
|
|
| |
Christmas came early.
Closes-Bug: #1479466
Change-Id: I931d826690c925f022dbfffe9afb7bf41345b1d0
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Instead of having the compiler do any validation on the
graph it has created instead have the compiler just compile
and have the engine that uses that compiled result do any
post compilation validation instead.
This makes it more clear that the compiler just compiles a
flow (and tasks and nested flows) into a graph, and that is
all that it does.
Change-Id: I96a35d732dc2be9fc8bc8dc6466256a19ac2df6d
|
| |
|
|
|
|
|
|
| |
If someone really wants to provide a flow to run that is
empty that is there prerogative so it doesn't seem that valuable
to blow up if they do this.
Change-Id: I0ad89b0ade85a64f6ec107e2686454ef6dc97353
|
| |
|
|
|
|
|
| |
First parameter should be the expected value.
Partial-Bug: #1357117
Change-Id: I2c7345171571a063b649a319a18b1cd712ac6275
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Instead of the added complexity of discarding flow nodes
we can simplify the compilation process by just retaining
them and jumping over them in further iteration and graph
and tree runtime usage.
This change moves toward a model that does just this, which
makes it also easier to in the future use the newly added
flow graph nodes to do meaningful things (like use them as
a point to change which flow_detail is used).
Change-Id: Icb1695f4b995a0392f940837514774768f222db4
|
| |
|
|
|
|
|
|
|
| |
Just directly use the built machine in the action engine and
avoid having another layer of abstraction that does not provide
that much value. This makes the code cleaner, and more easy to
understand (and so-on).
Change-Id: Iae1279098112254338258c1941c15889f1ad1a79
|
| |
|
|
|
|
|
|
|
|
|
|
| |
To make it easily possible to change the retry
atom execution from being in the engine thread this
creates a retry executor (which is similar to the task
executor) and provide that a serial executor (which it will
use to execute with). This makes the retry and task actions
closer to being the same and makes the surrounding code that
much similar (which makes understanding it easier).
Change-Id: I993e938280df3bd97f8076293183ef21989e2dba
|
| |\ |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Instead of having our own inbuilt fsm and table library
used to print the fsm states just use the newly created
and released automaton that contains the same/similar
code but as a released library that others can use and
benefit from.
Library @ http://pypi.python.org/pypi/automaton
Change-Id: I1ca40a0805e704fbb37b0106c1831a7e45c6ad68
|
| |\ \
| |/
|/| |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
When a atom is reverted it can be useful to retain the
result of that 'revert' method being called, so that it
can be later analyzed (or used for various purposes) so
adjust the storage, and actions to enable it to be stored.
Change-Id: I38a9a5f3bf7550e924468bb4a86652cb8beb306c
|
| |/
|
|
| |
Change-Id: If89baa042695f19e42b6368034f3ccf22c2cf0aa
|
| |\ |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
There is really no need to seperate off the runner into
a unit that builds a state-machine and then provides a tiny
utility function, both of these can just be in the same class
and code so that it is easier to understand/read.
Change-Id: I18b97514e230451ef804a878a0edcea4d0b2ad20
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
| |
Split the states that are not task states (but are retry
states) into there own additional set and then use that set
and a new function to validate the transition at other locations
in the code-base.
This makes the transitions that are valid for tasks/retries
easily viewable, more easy to read and understand, and
more correct (instead of being a mix of task + retry atom
transitions and states).
Change-Id: I9515c19daf59a21e581f51e757ece2050f348214
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Instead of using instance methods to perform the
compilation of tasks and flows instead move the
logic of those functions into tiny helper classes
that just perform the compilation of specific types
they handle.
This makes it more easy to document and understand
how a flow or task type is compiled down into a graph
and a tree node (which may or may not be attached to
a parent tree node).
Change-Id: Ib74edcad44556a897a4300132066e89d97739814
|
| |
|
|
|
|
|
|
| |
Make both of these finding functions use similar routines
that the utility module now provides since the logic that
both use can be shared.
Change-Id: Ib941b99945d42f5c0d791e9b2a0696d0e62a2388
|
| |
|
|
|
|
|
|
|
|
|
| |
Instead of having a nominally useful single-threaded storage
unit that uses a dummy r/w lock and a multi-threaded storage
unit just have the storage unit by default protect itself from
multi-threading calls being used on it (with the appropriate
reader/writer locks being activated to make this work
correctly).
Change-Id: Ib6879edb465156a8e54fd5b4002550d1cec49137
|
| |
|
|
|
|
|
|
|
|
|
| |
Recreate a very simple eventlet utility module that
has only a few features; one function checks if eventlet is
available and if not raise an exception; and a constant that
can be used by calling code (such as tests or other optional
functionality) to check if eventlet is useable before
proceeding.
Change-Id: I32df0702eeae7c7c78972c9796156dd824b2f123
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
To enable a parallel process executor to be used without
having to pass in a futures executor allow for the executor
option to be a string 'processes' (or similar) that will cause
the default parallel process executor to be used automatically.
Also allow for a 'threads' string that ensure a parallel thread
executor is used to match the ability to uses processes.
This also adjusts the WBE engine to have a similar executor fetching
function (which in the WBE case now validates a provided executor to
be of the desired type).
Change-Id: I54a82584c32c697922507b4f6e01ea7b8acc73c6
|
| |
|
|
|
|
|
|
| |
Since the scoping is an action_engine implementation detail
its test should belong in the action_engine test cases folder
instead of being at the top level unit test folder.
Change-Id: Ic436c534103d1d9fafad95299bd2632cc7ee5634
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Instead of linking nodes which have elements to
predecessors which do not we should search backwards
through the prior predecessors and link to one that
does have nodes; this ensures that we do not create
bad workflows when empty flows are injected.
Fixes bug 1392650
Change-Id: Ic362ef3400f9c77e60ed07b0097e3427b999d1cd
|
| |
|
|
|
|
|
|
| |
Internally we should be using the new location and not the
deprecated location wherever possible. This avoids emitting
warnings messages on our own code, which is a dirty habit.
Change-Id: Icc389e61613bc78e64083f0086f6f23aabd243d1
|
| |\ |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Move the storage ensuring logic from being split
across the engine and the storage layer and expose
only a single `ensure_atom` function that does
the work instead.
This also removes the access to the `ensure_task`
and `ensure_retry` methods as the internals of the
`ensure_atom` function is now the only location
that needs to use these two functions.
This reduces the need to do type specific atom
checks in the non-storage components (which we
want to reduce overall).
Breaking change: removes the public methods named
`ensure_task` and `ensure_retry` (which should not
be used externally anyway) from the storage object
and makes those internal/private methods instead.
Change-Id: I3a0f1f0dd777a1633b4937e16b50030275c84d1d
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
To complement the future changes in patterns we also want
to allow the execution of patterns to be affected in a similar
manner so that symbol lookup is no longer as confined as it was.
This change adds in the following:
- Symbol lookup by walking through an atoms contained scope/s.
- Better error messaging when symbols are not found.
- Adjusted & new tests (existing ones work).
- Better logging of the symbol lookup mechanism (helpful
during debugging, although it is very verbose...)
Part of blueprint taskflow-improved-scoping
Change-Id: Id921a4abd9bf2b7b5c5a762337f8e90e8f1fe194
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Instead of having a ad-hoc state-machine being used to perform
the various runtime actions (performed when a engine is ran) we
can gain a much more explict execution model by translating that
ad-hoc state machine to an explicit one instead...
This commit does just that, it adds a new fsm type that can be
used to create, define and run state-machines that respond to
various events (internal or external) and uses it in the runner action
engine module to run the previously ad-hc/implicit state-machine.
Implements blueprint runner-state-machine
Change-Id: Id35633a9de707f3ffb1a4b7e9619af1be009317f
|
|
|
To match how we have a worker_based subdirectory for
its specific tests lets start moving pieces of the action
engine specific tests to its own directory as well and
move more in the future as well.
Change-Id: I003b07a95259ba18b961834515121243e27d7456
|
