summaryrefslogtreecommitdiff
path: root/taskflow/tests/unit/persistence
Commit message (Collapse)AuthorAgeFilesLines
* Remove sixTakashi Kajinami2022-05-181-3/+2
| | | | | | | | This library no longer supports Python 2, thus usage of six can be removed. This also removes workaround about pickle library used in Python 2 only. Change-Id: I19d298cf0f402d65f0b142dea0bf35cf992332a9
* Use assertIs(Not)None to check for Nonehowardlee2016-11-071-1/+1
| | | | | | | | | | | | | [H203] Use assertIs(Not)None to check for None (off by default) Unit test assertions tend to give better messages for more specific assertions. As a result, assertIsNone(...) is preferred over assertEqual(None, ...) and assertIs(None, ...), and assertIsNotNone(...) is preferred over assertNotEqual(None, ...) and assertIsNot(None, ...). Off by default. Trivial fix Change-Id: I5b6ac7d99f0689843eb98cb3e9b9b10531322640
* Remove deprecated things for 2.0 releaseJoshua Harlow2016-05-102-27/+16
| | | | Change-Id: Id9adbc50bd51adc77ce88f698ad0ea2ee63fc5e2
* Fix currently broken and inactive mysql testsJoshua Harlow2015-12-221-4/+1
| | | | | | | | | | | | | | | | Commit 868dd8485456ce78a396b687ec7744dc365f06be added a migration script that caused there to be two heads (starting points) of migrations which would cause alembic to be unable to do the initial schema creation (and upgrade). So this fixes that by correctly making one head only and not skipping tests for a database if it can not be upgraded for whatever reason (which was silencing the test failures). Closes-bug: #1528683 Change-Id: Id571072eec1dc3b6cbb9e868854f6db0b271e5f8
* Merge "Add in-memory backend delete() in recursive/non-recursive modes"Jenkins2015-12-151-1/+38
|\
| * Add in-memory backend delete() in recursive/non-recursive modesJoshua Harlow2015-10-141-1/+38
| | | | | | | | Change-Id: I957c875bf493b2e38bf82af6faccbab8a4861ade
* | Fix order of assertEqual for unit.persistencelin-hua-cheng2015-10-171-9/+9
|/ | | | | | | First parameter should be the expected value. Change-Id: If255cc908cf013b6c58f5bf666ccaa50f04f7759 Partial-Bug: #1357117
* Fix how the dir persistence backend was not listing logbooksJoshua Harlow2015-09-041-0/+21
| | | | | | | | | | Due to the usage of the os.path.islink check this means that no logbooks would be returned when get_logbooks was called, which is not the behavior we want. Closes-Bug: #1492403 Change-Id: Ife6a5bec777c9e2d820391914ce2c6fbbadf4f79
* Rename logbook module -> models moduleJoshua Harlow2015-07-081-51/+51
| | | | | | | | | | | Since this module contains more than the logbook class and really is a our generic models that are used to hold the runtime structure it is more appropriate to place it under a models module and deprecate the usage of the old module by placing a warning there (so that when it is imported that warning is triggered). Change-Id: I79def5ee08f560d38f2c9dcefd0b33becc2a4d36
* Merge "Remove testing using persistence sqlalchemy backend with 'mysqldb'"Jenkins2015-06-141-74/+37
|\
| * Remove testing using persistence sqlalchemy backend with 'mysqldb'Joshua Harlow2015-05-291-74/+37
| | | | | | | | | | | | | | | | | | | | It appears no longer recommended to test against and/or use mysqldb as a backend that is tested against (since it lacks real support and development) and it is now better to just test against pymysql which is under development (and has better eventlet integration and py3.x support). Change-Id: I08f7ea67fbeb2e8b2976aef442ed442e33c77e09
* | Merge "Use a lru cache to limit the size of the internal file cache"Jenkins2015-06-101-9/+52
|\ \
| * | Use a lru cache to limit the size of the internal file cacheJoshua Harlow2015-06-101-9/+52
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Instead of having an unbounded internal file cache which will eventually absorb all memory of the running/containing python process have the size of that cache be limited by an optionally provided size (and have eviction be based on how recent a cached entry was used). Fixes bug 1458248 Change-Id: I5e59efc4edd51b05cfb1e67d3e7014e378e352aa
* | | Add a new `ls_r` methodJoshua Harlow2015-06-041-1/+59
|/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Instead of having a `ls` method that when used recursively *always* returns the absolute path of items in the fake in memory storage tree and *relative* paths (when used in non-recursive mode) add a new `ls_r` method that can return absolute *or* relative paths. In the future it is highly likely that the the `ls` recursive keyword argument will be removed (so preferring and moving to the `ls_r` should occur earlier rather than later), so this also adds a debtcollector removed keyword argument decorator over the existing `ls` to ensure that users are aware of this change (as well as a adjusted docstring). Fixes bug 1458114 Change-Id: Id2a5869e94ac44679020a14297d1073d1dc2718f
* | Ensure empty paths raise a value errorJoshua Harlow2015-05-111-0/+6
|/ | | | | | | | | | | Empty paths should not be allowed to be set or fetched or normalized so check for those and raise an error when one is encountered. Also add some basic test conditions that ensure the normpath method raises when it should. Change-Id: I7f7e6600f03c67376ba310ab231b2e33cd7528db
* Merge "Add in memory filesystem clearing"Jenkins2015-04-081-0/+11
|\
| * Add in memory filesystem clearingJoshua Harlow2015-03-291-0/+11
| | | | | | | | Change-Id: Ibd22eddeec87f660e729099f702f52cdd6b828ea
* | Allow ls() to list recursively (using breadth-first)Joshua Harlow2015-03-311-0/+29
|/ | | | | | | | | | | | Make it easier to list the contents of the full in-memory filesystem by allowing the ls() function to take a recursive keyword argument (which by default is false). This makes it easier to analyze the contents of the full in memory filesystem; saving people from creating similar code themselves. Change-Id: I573797945255cb81728e7a86b58768b848110ee7
* Add a set of tests to the in memory fake filesystemJoshua Harlow2015-03-251-0/+62
| | | | | | | | To ensure this continues to work as expected add some basic functional tests that ensure the existing operations work (and continue to work). Change-Id: I2903bf29a77936d850d667d1fbff179a14a57adb
* lazy loading for logbooks and flowdetailsDan Krause2015-03-111-0/+29
| | | | | | | | | You can now specify a "lazy" argument when asking a persistence backend for a logbook for flowdetail. This will return a logbook with no flowdetails in it, or a flowdetail with no atomdetails in it, respectively. Change-Id: Ie81a19a2107b44e2c35c2a92507bec03eee55c50
* Switch to using 'oslo_utils' vs 'oslo.utils'Joshua Harlow2015-01-142-2/+2
| | | | | | | | Prefer the non-deprecated 'oslo_utils' instead of the namespaced 'oslo.utils' wherever it was previously used. Change-Id: I9a78150ef5266e1ff22147278162fe3cfe1b2e3f
* Move over to using oslo.utils [reflection, uuidutils]Joshua Harlow2014-12-182-2/+3
| | | | | | | | | | | | | | The reflection module is now part of oslo.utils so we should remove our local version and use that version instead; this also goes for the uuidutils module which is now part of oslo.utils as well so we no longer need our local version copied from the incubator... Note that one reflection method `find_subclasses` which was to specific to taskflow is now moved to the misc utility module instead of its prior home in the reflection module. Change-Id: I069881c80b0b2916cc0c414992b80171f7eeb79f
* Merge "Fix split on "+" for connection strings that specify dialects"Jenkins2014-12-151-4/+12
|\
| * Fix split on "+" for connection strings that specify dialectsJoshua Harlow2014-12-041-4/+12
| | | | | | | | | | | | Fixes bug 1399486 Change-Id: I3b7e6331751f25d9c4221393e8329934925791e7
* | Ensure that the zookeeper backend creates missing atomsJoshua Harlow2014-11-241-1/+42
|/ | | | | | | | | | | | | When 'create_missing' is true the atom should be created instead of raising an exception; this is used when a flow detail is updated with a new detail and then saved. This also adds test cases that verify this happens so that we verify this on an ongoing basis. Fixes bug 1395812 Change-Id: I4851a08ff1ab4101dbec4a6656177908095c3c52
* Remove direct usage of the deprecated failure locationJoshua Harlow2014-10-211-8/+7
| | | | | | | | Internally we should be using the new location and not the deprecated location wherever possible. This avoids emitting warnings messages on our own code, which is a dirty habit. Change-Id: Idac5a772eca7529d92542ada3be1cea092880e25
* Switch to a custom NotImplementedError derivativeJoshua Harlow2014-10-081-2/+5
| | | | | | | | | | When a feature or method is not implemented it's useful to throw our own derivative of a NotImplementedError error so that we can distingush these types of errors vs actual usage of NotImplementedError which could be thrown from driver or user code. Change-Id: I8d5dfb56a254f315c5509dc600a078cfef2dde0b
* Remove db locks and use random db names for testsBrian Jarrett2014-08-121-57/+67
| | | | | | | | | | | | | | | | | | Removed database locks against the database openstack_citest and implemented random name generation for new databases created just for persistence testing. Once the locks were removed, using 'template1' as the initial db for postgres connections was problematic, because postgres refuses to create databases when there are multiple connections to 'template1'. Switched to using 'postgres' as an initial db to use. Changed _reset_database to _init_db since we are always working with a brand new database, and removed the 'drop database' before a 'create database. Added a _remove_db method to remove the database once testing was finished. Change-Id: Iaf1c101df9c268da48db7432bcbc0467f6486bcd Closes-Bug: 1327469
* Enabled hacking checks H305 and H307Christian Berendt2014-07-171-2/+1
| | | | | | | * H305 imports not grouped correctly * H307 like imports should be grouped together Change-Id: If1dd9c89f65ede6959865a885777cb08c263eca0
* Fix traces left in zookeeperJoshua Harlow2014-06-271-8/+25
| | | | | | | | | | | | We currently are leaving the root node in zookeeper whenever this test is ran, we should instead be removing the full directory and any children to avoid leaving test data behind in zookeeper (aka, cleanup our dirty laundry). Fixes bug 1330807 Change-Id: I56d9dabd9926463506e1710ddf0a6c4831d5dc57
* Upgrade hacking version and fix some of the issuesJoshua Harlow2014-06-131-3/+1
| | | | | | | | | | | | | | | | | Update hacking to the new requirements version and fix about half of the new reported issues. The other hacking issues are for now ignored until fixed by adjusting our tox.ini file. This commit fixes the following new hacking errors: H405 - multi line docstring summary not separated with an empty line E265 - block comment should start with '# ' F402 - import 'endpoint' from line 21 shadowed by loop variable Change-Id: I6bae61591fb988cc17fa79e21cb5f1508d22781c
* Cleanup zookeeper integration testingJoshua Harlow2014-05-092-70/+33
| | | | | | | | | | | | In the persistence tests only use zake when zookeeper is not available (and of the right version). When zookeeper is available skip running zake. In the jobboard tests split out the tests which are not specific for zake into a base class (allowing for a future commit to add a zookeeper integration test). Change-Id: I50d51639a7f6c03c29d559c485676fddb9a7cf20
* Revert "Move taskflow.utils.misc.Failure to its own module"Jeremy Stanley2014-03-311-8/+8
| | | | | | | | | | | | This reverts commit 42ca240e8157b840c298d14fbf478ae570376633 which was a breaking change in a library consumed by other OpenStack projects with no deprecation or backwards compatibility considerations. It was able to merge because openstack/taskflow is apparently not yet part of the integrated gate via the proposed I202f4809afd689155e2cc4a00fc704fd772a0e92 change. Change-Id: I96cf36dc317499df91e43502efc85221f8177395 Closes-Bug: #1300161
* Move taskflow.utils.misc.Failure to its own moduleIvan A. Melnikov2014-03-311-8/+8
| | | | | | | | | | Failure class is important part of TaskFlow API, so it should be more visible and accessible. Breaking change: any client that used taskflow.utils.misc.Failure should be updated. Change-Id: Ib30000c9246bbcb227b34dfb0aba4d0b950bf926
* Merge "Persistence cleanup part one"Jenkins2014-03-281-27/+29
|\
| * Persistence cleanup part oneJoshua Harlow2014-03-261-27/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Convert the various functions that take a task detail into ones that take atom details (since this is now the generic type they should take). - Don't expose the detail type strings as part of the atom detail api, leave those as private hidden strings and provide conversion functions from string<->class instead. - Have the logbook objects contain the following new methods to reduce the dependence on persistence_utils to do the same. - to_dict() which converts the current object into a dict - from_dict() which converts the provided dict into a object - merge() which merges a incoming objects data with the current objects - Have the persistence backends + storage + action engine use these new methods instead of there current usage. - Don't compare to logbook.RETRY_DETAIL or logbook.TASK_DETAIL since python has the isinstance function just use it (ideally we should fix the code so that this isn't even needed, usage of isinstance means something is not designed/structured right). - In storage tests we can't assume that failures will be non-lossy since under certain backends when a failure is stored information about the internally held exc_info is lost, so take this into account when testing by using matches() where applicable. Change-Id: Ie8a274cfd4cb4e64e87c355dc99d466d74a4e82c
* | tests: Discover absence of zookeeper fasterIvan A. Melnikov2014-03-271-7/+7
|/ | | | | | | | | | | | | | It turns out that when no zookeeper server is available for tests python ./setup.py test spends almost half of it's running time looking for it. This change addresses that and makes unit tests run roughly two times faster on my zookeeperless machine: - client starting timeout reduced from default 15 seconds to 3 seconds, which should be more than enough to connect to localhost; - zookeeper availability is checked only once, on module import, and then saved for later re-use. Change-Id: I11932e7cca12436cde0bcf0a20cb2adb96fbb27a
* Merge "Removed copyright from empty files"Jenkins2014-03-231-15/+0
|\
| * Removed copyright from empty filesyangxurong2014-02-221-15/+0
| | | | | | | | | | | | | | | | | | According to policy change in HACKING: http://docs.openstack.org/developer/hacking/#openstack-licensing empty files should no longer contain copyright notices. Change-Id: I1f825add9036623ef626b48c66206ac2b452c592 Partial-Bug: #1262424
* | Add atom intentions for tasks and retriesAnastasia Karpinska2014-03-181-0/+31
| | | | | | | | | | | | | | | | | | | | | | | | Add atom intentions: REVERT, EXECUTE, RETRY and IGNORE. Intentions will be used by action engine to schedule tasks correctly. Add intention to task detail and extend storage to work with atom intentions. Add alembic migration to add intentions column to database. Change-Id: I79c9bb5f11861658dbfedfd64ef93eb92b29cb2d
* | Extend logbook and storage to work with retryAnastasia Karpinska2014-03-141-0/+48
| | | | | | | | | | | | | | | | - Store AtomType in TaskDetail (TASK_DETAIL or RETRY_DETAIL) - Extend storage to work with retries - Add alembic migration for database Change-Id: I07896d4b79ebe5de33f65e00a3bd32af543a09b4
* | Allow connection string to be just backend nameIvan A. Melnikov2014-03-052-15/+17
|/ | | | | | | | | | Having to specify backend connection as URI is somewhat confusing when other parts of uri (like host or path) are not used. This commit allows to specify just backend name as connection string. Partially implements blueprint persistence-uris Change-Id: Ic38d41d4f24dcd596cbdff33de78d1a137fb2e8f
* Fix few spelling and grammar errorsIvan A. Melnikov2014-02-202-4/+4
| | | | Change-Id: I8e678a9f76cba3d90053d28baf4ff42d34b4e390
* Remove extraneous vim configuration commentsyangxurong2014-02-147-14/+0
| | | | | | | | | Remove line containing comment - # vim: tabstop=4 shiftwidth=4 softtabstop=4 Change-Id: I7581cc88b8de433d5609ed06c6570b0b45c13573 Closes-Bug:#1229324
* Merge "Run zookeeper tests if localhost has a compat. zookeeper server"Jenkins2014-02-051-14/+48
|\
| * Run zookeeper tests if localhost has a compat. zookeeper serverJoshua Harlow2014-02-041-14/+48
| | | | | | | | | | | | | | | | | | If a local zookeeper server is running then using that zookeeper server we are able to run the zookeeper tests, so instead of always turning these tests off; only turn them off if no zookeeper server is found. Change-Id: I27bbc67b4f2d03a4469882fdc3b8015324ad7e3b
* | Merge "Some zookeeper persistence improvements/adjustments"Jenkins2014-02-041-8/+7
|\ \ | |/
| * Some zookeeper persistence improvements/adjustmentsJoshua Harlow2014-02-021-8/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Only use zake in unit-tests by providing a client to the backend as a fake client. This avoids cases where a real client should be provided; and if not provided then a error should be thrown. This avoids potential misuse of zake (which really is only designed for testing usage and not for actual usage in production). - Add a kazoo_utils file that will be shared to make kazoo clients and to parse hosts into a kazoo friendly format for other usages of kazoo clients that will be appearing soon. - Ensure basic assertions when creating the zk backend, don't allow empty paths, require absolute paths. - Update exc_wrapper to have slightly more descriptive messages that identify more of what the original errors category was. - Create initial paths on upgrade() to mirror what other backends are doing in their upgrade() function. Other backends do their initial steps to upgrade there backends in this function (ie sqlalchemy does its migrations here so it seems to be consistent place to make sure zookeeper paths are created correctly). Change-Id: Iabafe73c059b4719617f01bd1ee35795f71ca21d
* | Be more resilent around import/detection/setup errorsJoshua Harlow2014-02-031-26/+45
|/ | | | | | | | | | | It appears that some of our current testr failures are due to this module not handling exception cases very well; this is making it hard to debug and hard to determine what the errors are, so handle more of the general exceptions better in more of the places where they are problematic. Change-Id: I7c71d3a68852c0093f27f3b16d6475cc0a8bf8ef
* Merge "Use addCleanup instead of tearDown in test_sql_persistence"Jenkins2014-01-301-9/+2
|\
View Lorry Controller Status