summaryrefslogtreecommitdiff
path: root/taskflow/persistence/backends/impl_sqlalchemy.py
Commit message (Collapse)AuthorAgeFilesLines
* Prepare taskflow for sqlalchemy2Gregory Thiemonge2023-02-271-13/+21
| | | | | | | | | * convert_unicode engine option is no longer supported * use positional arguments for select, lists are no longer supported * insert no longer accepts keyword constructor arguments * explicitly convert Rows to dicts Change-Id: I858d60b3d142e8ade8b2c0154872c8bfc19a42ba
* Remove sixTakashi Kajinami2022-05-181-7/+6
| | | | | | | | This library no longer supports Python 2, thus usage of six can be removed. This also removes workaround about pickle library used in Python 2 only. Change-Id: I19d298cf0f402d65f0b142dea0bf35cf992332a9
* Stop to use the __future__ module.Hervé Beraud2020-06-021-2/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | The __future__ module [1] was used in this context to ensure compatibility between python 2 and python 3. We previously dropped the support of python 2.7 [2] and now we only support python 3 so we don't need to continue to use this module and the imports listed below. Imports commonly used and their related PEPs: - `division` is related to PEP 238 [3] - `print_function` is related to PEP 3105 [4] - `unicode_literals` is related to PEP 3112 [5] - `with_statement` is related to PEP 343 [6] - `absolute_import` is related to PEP 328 [7] [1] https://docs.python.org/3/library/__future__.html [2] https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html [3] https://www.python.org/dev/peps/pep-0238 [4] https://www.python.org/dev/peps/pep-3105 [5] https://www.python.org/dev/peps/pep-3112 [6] https://www.python.org/dev/peps/pep-0343 [7] https://www.python.org/dev/peps/pep-0328 Change-Id: Ie7c18f2f1b6294e3d19c36112d35f4472ce77e7b
* python3.0 has deprecated LOG.warnxhzhf2017-03-071-3/+3
| | | | | | | | python3.0 has deprecated LOG.warn https://docs.python.org/3/library/logging.html#logging.warning Closes-Bug: #1650843 Change-Id: Ib8473cbdb84547f385e619fe08ed723b1ee7bcdb
* Replace retrying with tenacityGevorg Davoian2016-11-031-9/+7
| | | | | | | | | | | | This patch replaces the legacy retrying library with the newer and more convenient tenacity one, taking into account that: 1) retrying uses milliseconds for wait times, but tenacity uses seconds; 2) retrying has a lot of numeric arguments for specifying behaviour of decorated functions, while tenacity has a few of them, which are specialized objects, thus making the retry-decorator more flexible. Change-Id: I4b165d37b2ecc210f2b94c103b73eaab51529261 Closes-Bug: #1635404
* Ensure upgrade for sqlalchemy is protected by a lockJoshua Harlow2016-03-201-13/+17
| | | | | | | | | Make sure that upgrade() is thread-safe (as it appears not to be) by using a lock that all connections from the same engine will use. Change-Id: I2b2b1be9e797099c8412fc6819465e550b1b934a Closes-Bug: #1559496
* Merge "Use the retrying lib. to do basic sqlalchemy engine validation"Jenkins2016-01-091-47/+41
|\
| * Use the retrying lib. to do basic sqlalchemy engine validationJoshua Harlow2016-01-051-47/+41
| | | | | | | | | | | | | | | | Instead of a custom loop and backoff just use the retrying library to do this same code and delegate the loop complexity to it instead. Change-Id: Iaf02cc728d2a2cfc7077300e03d7ef25522717b7
* | Move all internal blather usage/calls to trace usage/callsJoshua Harlow2015-12-171-1/+1
|/ | | | Change-Id: I415a81d3b6b15b17a9a91cc2a0681c159172a4e1
* Merge "Address concurrent mutation of sqlalchemy backend"Jenkins2015-07-171-13/+8
|\
| * Address concurrent mutation of sqlalchemy backendJoshua Harlow2015-06-151-13/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In order to prevent a thread from closing a backend while another thread is getting a connection (which can result in an engine being created) stop this kind of concurrent mutation by creating a engine (if it was not user provided) in the constructor. In the close the engine dispose is called (which will according to the docs just create a new pool anyway) so there is no need to recreate the full engine object from its same configuration again. Change-Id: Id1fa3001b3ebbe76bbcdb08ed4add6a9e16ea96b
* | Rename logbook module -> models moduleJoshua Harlow2015-07-081-5/+5
| | | | | | | | | | | | | | | | | | | | | | Since this module contains more than the logbook class and really is a our generic models that are used to hold the runtime structure it is more appropriate to place it under a models module and deprecate the usage of the old module by placing a warning there (so that when it is imported that warning is triggered). Change-Id: I79def5ee08f560d38f2c9dcefd0b33becc2a4d36
* | Make it possible to see the queries executed (in BLATHER mode)Joshua Harlow2015-06-201-0/+13
|/ | | | | | Part of blueprint make-things-speedy Change-Id: I3ac092e98a492d3a70c7780efb4647d593fa70d1
* Move implementations into there own sub-sectionsJoshua Harlow2015-05-011-2/+0
| | | | | | | | Also fixes up some inline-code/examples docs to correctly display in the generated docs (and tweaks some URI capitalization). Change-Id: I001ef2460eb5e9a884ca6db6e8d6f72864191fe7
* Use the ability to chain exceptions correctlyJoshua Harlow2015-04-031-31/+43
| | | | | | | | | | | In the zookeeper jobboard (and persistence backends) we are associating the cause of a new exception when raising a exception. Using the new exception helper we can make this work better on py2.x and py3.x so that the py3.x version has the chain setup correctly (while the py2.x version just uses the class 'cause' attribute instead). Change-Id: Ieeac2f70e1834d4612556565762ffd3be3e5b5a1
* Merge "lazy loading for logbooks and flowdetails"Jenkins2015-03-121-11/+27
|\
| * lazy loading for logbooks and flowdetailsDan Krause2015-03-111-11/+27
| | | | | | | | | | | | | | | | | | You can now specify a "lazy" argument when asking a persistence backend for a logbook for flowdetail. This will return a logbook with no flowdetails in it, or a flowdetail with no atomdetails in it, respectively. Change-Id: Ie81a19a2107b44e2c35c2a92507bec03eee55c50
* | Put underscore in-front of alchemist helperJoshua Harlow2015-03-111-2/+2
|/ | | | | | | | | Since this is a internal helper object we should put a underscore infront of its class name to make sure this is obvious (the docstring says the same thing but this will help). Change-Id: I3aeb6a178c8ceda0c36d5f5bb57bb765260cbd3c
* Persistence backend refactorDan Krause2015-03-091-4/+17
| | | | | | | Factors lots of duplicate code out of persistence backends Adds get_flows_for_book to all backends Change-Id: I0434bd4931cd9274876f9e9c92909531f244bcac
* add get_flow_details and get_atom_details to all backendsDan Krause2015-02-191-0/+30
| | | | Change-Id: Ic21a883b4eee827d90065f8ecfc5af2599ebece5
* Rework the sqlalchemy backendJoshua Harlow2015-02-101-247/+183
| | | | | | | | | | | | | | | We can just simplify the usage of an sqlalchemy if we just use (we already have our own ORM like objects anyway) sqlalchemy core in the first place and have a very tiny layer that converts back and forth from our very limited object model that we use in our persistence layer. This change makes that adjustment, which makes it easier to read and understand the actions the sqlalchemy backend is doing when saving, reading and updating data, and avoids yet another layer that isn't useful for our purposes anyway. Change-Id: I911c509f65e7845aee86fed1622eaa56970741f2
* Merge "Move the persistence base to the parent directory"Jenkins2015-01-211-1/+1
|\
| * Move the persistence base to the parent directoryJoshua Harlow2014-12-191-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | In order to match the directory/module layout of the other pluggable backends better move the persistence base module that defines the base abstract classes up into the parent directory. This makes it easier to look at the taskflow code-base and understand the common layout. Change-Id: I7887cb0241b8fe65cbdfee32c101c3df5f05d27c
* | Merge "Move implementation(s) to there own sections"Jenkins2015-01-211-4/+4
|\ \
| * | Move implementation(s) to there own sectionsJoshua Harlow2015-01-161-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Instead of putting implementation(s) under the interfaces section put the implementation(s) under there own section. This also includes some other tweaks to refer to those implementation(s) where appropriate. Change-Id: Iffdc0439c843e7f70cf873e5a75501feb51f96c7
* | | Merge "Add back a 'eventlet_utils' helper utility module"Jenkins2015-01-191-2/+2
|\ \ \ | |/ / |/| |
| * | Add back a 'eventlet_utils' helper utility moduleJoshua Harlow2015-01-161-2/+2
| |/ | | | | | | | | | | | | | | | | | | | | Recreate a very simple eventlet utility module that has only a few features; one function checks if eventlet is available and if not raise an exception; and a constant that can be used by calling code (such as tests or other optional functionality) to check if eventlet is useable before proceeding. Change-Id: I32df0702eeae7c7c78972c9796156dd824b2f123
* | Switch to using 'oslo_utils' vs 'oslo.utils'Joshua Harlow2015-01-141-1/+1
|/ | | | | | | | Prefer the non-deprecated 'oslo_utils' instead of the namespaced 'oslo.utils' wherever it was previously used. Change-Id: I9a78150ef5266e1ff22147278162fe3cfe1b2e3f
* Merge "Remove default setting of 'mysql_traditional_mode'"Jenkins2014-12-181-2/+0
|\
| * Remove default setting of 'mysql_traditional_mode'Joshua Harlow2014-11-251-2/+0
| | | | | | | | | | | | | | | | | | | | | | The sqlalchemy versions we are using/supporting (0.6+) have the detection of the mysql mode built-in so only activate the connect setting if we are somehow overriden by a user who knows what they are doing. Fixes bug 1396278 Change-Id: If2226d3e9f921a1c5f62a6727016fe86cd50a9b5
* | Add and use a new simple helper logging moduleJoshua Harlow2014-12-081-1/+1
|/ | | | | | | | | Add a new logging BLATHER level to easily allow its usage for messages that are below the normal DEBUG level such as compilation information and scope lookup info which can be very verbose in logs if always enabled. Change-Id: I828211403bd02bfd6777b10cdcfe58fb0637a52c
* Remove direct usage of the deprecated failure locationJoshua Harlow2014-10-211-1/+2
| | | | | | | | Internally we should be using the new location and not the deprecated location wherever possible. This avoids emitting warnings messages on our own code, which is a dirty habit. Change-Id: Idac5a772eca7529d92542ada3be1cea092880e25
* Add a futures type that can unify our future functionalityJoshua Harlow2014-10-181-2/+2
| | | | | | | | | | | | | | | | | Move the currently existing green future executor and associated code to a new futures types module so that it can be accessed from this new location (TODO: deprecate the old location and link the old to the new for one release so that we can remove the old link in N + 1 release). This unifies the API that the existing pool (thread or process) future executors and the green thread pool future executor, and the newly added synchronous executor (replacing the previous `make_completed_future` function) provide so there usage is as seamless as possible. Part of blueprint top-level-types Change-Id: Ie5500eaa7f4425edb604b2dd13a15f82909a673b
* Switch to using oslo.utils and oslo.serializationJoshua Harlow2014-09-201-1/+1
| | | | | | | | | | | | Instead of copying modules from the incubator into taskflow we can now directly use these same modules from supported libraries instead so this moves the usage of everything except uuidutils which wasn't moved over to using those newly published libraries. Part of blueprint integrate-and-use-oslo-utils-serialization Change-Id: I1183bda96e1ddb062d9cab91990186f0f56f0a0e
* Merge "Remove misc.as_bool as oslo provides an equivalent"Jenkins2014-06-181-5/+18
|\
| * Remove misc.as_bool as oslo provides an equivalentJoshua Harlow2014-06-111-5/+18
| | | | | | | | | | | | | | | | | | There isn't a need to have a misc.as_bool function anymore now that we have imported the oslo incubator strutils module since that module provides a function that does *nearly* the same thing. Change-Id: I7afe141d5a37c50b0c926144743f9af71db95bbf
* | Upgrade hacking version and fix some of the issuesJoshua Harlow2014-06-131-3/+15
|/ | | | | | | | | | | | | | | | | Update hacking to the new requirements version and fix about half of the new reported issues. The other hacking issues are for now ignored until fixed by adjusting our tox.ini file. This commit fixes the following new hacking errors: H405 - multi line docstring summary not separated with an empty line E265 - block comment should start with '# ' F402 - import 'endpoint' from line 21 shadowed by loop variable Change-Id: I6bae61591fb988cc17fa79e21cb5f1508d22781c
* Merge "Include the function name on internal errors"0.3.21Jenkins2014-06-101-2/+5
|\
| * Include the function name on internal errorsJoshua Harlow2014-06-041-2/+5
| | | | | | | | | | | | | | | | | | To make it easier to debug and reason about the exception that occurred include the function name that was called during the sessions activation in the exception message (and associated LOG output). Change-Id: I21f9310f78968d1e60d88b1f77be0dc629e75525
* | Merge "Allow the mysql mode to be more than just TRADITIONAL"Jenkins2014-06-101-8/+19
|\ \
| * | Allow the mysql mode to be more than just TRADITIONALJoshua Harlow2014-06-041-8/+19
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To make it easier to later move to oslo.db match there ability to set a different mysql mode (this will default, unless configured explicitly off/differently to TRADITIONAL). To ensure that we retain backwards compatibility the prior existence of 'mysql_traditional_mode' as a boolean option will by default set the mode to TRADITIONAL, and a new configuration option 'mysql_sql_mode' can be used to provide a secondary overriding mode that will be used instead (incase someone wants to use something other than TRADITIONAL). Closes-Bug: 1326568 Change-Id: Ide34c27b12c26030c8842f3f4b0fcca43ce783a7
* | Add in default transaction isolation levelsJoshua Harlow2014-06-041-5/+29
|/ | | | | | | | | | | Apply a default setting for transaction isolation levels for mysql and postgresql to help avoid consistency issues that happen when two transactions occur at the same time on the same set of records. Closes-Bug: 1326507 Change-Id: I1819722889d0d66d938641af6aa6f79fcfd2deb4
* Use correct exception in the timing listenerStanislav Kudriashev2014-04-011-2/+2
| | | | | | | Used `StorageFailure` exception instead of non-existing `StorageError` in the timing listener. Change-Id: I83035b737f7507b760799a5d44d4c7d097103ae5
* Revert "Move taskflow.utils.misc.Failure to its own module"Jeremy Stanley2014-03-311-4/+2
| | | | | | | | | | | | This reverts commit 42ca240e8157b840c298d14fbf478ae570376633 which was a breaking change in a library consumed by other OpenStack projects with no deprecation or backwards compatibility considerations. It was able to merge because openstack/taskflow is apparently not yet part of the integrated gate via the proposed I202f4809afd689155e2cc4a00fc704fd772a0e92 change. Change-Id: I96cf36dc317499df91e43502efc85221f8177395 Closes-Bug: #1300161
* Move taskflow.utils.misc.Failure to its own moduleIvan A. Melnikov2014-03-311-2/+4
| | | | | | | | | | Failure class is important part of TaskFlow API, so it should be more visible and accessible. Breaking change: any client that used taskflow.utils.misc.Failure should be updated. Change-Id: Ib30000c9246bbcb227b34dfb0aba4d0b950bf926
* Persistence cleanup part oneJoshua Harlow2014-03-261-83/+92
| | | | | | | | | | | | | | | | | | | | | | | | | | | - Convert the various functions that take a task detail into ones that take atom details (since this is now the generic type they should take). - Don't expose the detail type strings as part of the atom detail api, leave those as private hidden strings and provide conversion functions from string<->class instead. - Have the logbook objects contain the following new methods to reduce the dependence on persistence_utils to do the same. - to_dict() which converts the current object into a dict - from_dict() which converts the provided dict into a object - merge() which merges a incoming objects data with the current objects - Have the persistence backends + storage + action engine use these new methods instead of there current usage. - Don't compare to logbook.RETRY_DETAIL or logbook.TASK_DETAIL since python has the isinstance function just use it (ideally we should fix the code so that this isn't even needed, usage of isinstance means something is not designed/structured right). - In storage tests we can't assume that failures will be non-lossy since under certain backends when a failure is stored information about the internally held exc_info is lost, so take this into account when testing by using matches() where applicable. Change-Id: Ie8a274cfd4cb4e64e87c355dc99d466d74a4e82c
* Adjust the exception hierachyJoshua Harlow2014-03-241-8/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Group the exceptions into the following groups * Storage * Jobs * Execution * Other (wrapped failure here) This grouping makes it easier to understand where one type of exception should be used vs using another type of exception. Backwards incompatible changes: * StorageError -> StorageFailure * AlreadyExists -> Duplicate * WrappedFailure now inherits from Exception and not directly from TaskFlowException since it wraps arbitrary other exceptions and is not specific to taskflow. Cleanups: * JobNotFound -> NotFound * EmptyFlow -> Empty * JobAlreadyExists -> AlreadyExists * InvariantViolation (X) * ConnectionFailure (X) Change-Id: I0e1e81b513fbbc7adb8bfaa1244993e345ab70d3
* Merge "Adjust logging levels and usage to follow standards"Jenkins2014-03-221-2/+2
|\
| * Adjust logging levels and usage to follow standardsJoshua Harlow2014-03-141-2/+2
| | | | | | | | | | | | | | | | | | | | A few of our logging levels and messages should be updated to better reflect the oslo logging standards created at https://wiki.openstack.org/LoggingStandards so update some of them to not report error when its really warning level and not report warning when debug is better. Change-Id: I8abdb1bcfa2893c6b97eda7e85ac779e456966b8
* | Add atom intentions for tasks and retriesAnastasia Karpinska2014-03-181-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | Add atom intentions: REVERT, EXECUTE, RETRY and IGNORE. Intentions will be used by action engine to schedule tasks correctly. Add intention to task detail and extend storage to work with atom intentions. Add alembic migration to add intentions column to database. Change-Id: I79c9bb5f11861658dbfedfd64ef93eb92b29cb2d
View Lorry Controller Status