summaryrefslogtreecommitdiff
path: root/tests/test_worker.py
Commit message (Collapse)AuthorAgeFilesLines
* worker.work() should be tested with burst=TrueSelwin Ong2020-06-071-2/+2
|
* Implement Customizable Serializer Support (#1219)Babatunde Olusola2020-04-161-0/+9
| | | | | | | | | | | | | | | | | | | | | | | | | * Implement Customizable Serializer Support * Refractor serializer instance methods * Update tests with other serializers * Edit function description * Edit function description * Raise appropriate exception * Update tests for better code coverage * Remove un-used imports and un-necessary code * Refractor resolve_serializer * Remove un-necessary alias from imports * Add documentation * Refractor tests, improve documentation
* fixing HerokuWorkerShutdownTestCase after #1194 (#1213)Samuel Colvin2020-03-091-2/+6
|
* Add a hard kill from the parent process with a 10% increased timeout … (#1169)mr-trouble2020-01-191-0/+27
| | | | | | | | * Add a hard kill from the parent process with a 10% increased timeout in case the forked process gets stuck and cannot stop itself. * Added test for the force kill of the parent process. * Changed 10% to +1 second, and other misc changes based on review comments.
* Job scheduling (#1163)Selwin Ong2020-01-041-14/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * First RQScheduler prototype * WIP job scheduling * Fixed Python 2.7 tests * Added ScheduledJobRegistry.get_scheduled_time(job) * WIP on scheduler's threading mechanism * Fixed test errors * Changed scheduler.acquire_locks() to instance method * Added scheduler.prepare_registries() * Somewhat working implementation of RQ scheduler * Only call stop_scheduler if there's a scheduler present * Use OSError rather than ProcessLookupError for PyPy compatibility * Added `auto_start` argument to scheduler.acquire_locks() * Make RQScheduler play better with timezone * Fixed test error * Added --with-scheduler flag to rq worker CLI * Fix tests on Python 2.x * More Python 2 fixes * Only call `scheduler.start` if worker is run in non burst mode * Fixed an issue where running worker with scheduler would fail sometimes * Make `worker.stop_scheduler()` more resilient to errors * worker.dequeue_job_and_maintain_ttl() should also periodically run maintenance tasks * Scheduler can now work with worker in both burst and non burst mode * Fixed scheduler logging message * Always log scheduler errors when running * Improve scheduler error logging message * Removed testing code * Scheduler should periodically try to acquire locks for other queues it doesn't have * Added tests for scheduler.should_reacquire_locks * Added queue.enqueue_in() * Fixes queue.enqueue_in() in Python 2.7 * First stab at documenting job scheduling * Remove unused methods * Remove Python 2.6 logging compatibility code * Remove more unused imports * Added convenience methods to access job registries from queue * Added test for worker.run_maintenance_tasks() * Simplify worker.queue_names() and worker.queue_keys() * Updated changelog to mention RQ's new job scheduling mechanism.
* Store worker's RQ and Python versions (#1125)Vladimir Protasov2019-10-081-0/+31
| | | | | | | | * Store worker version to Redis * Store worker's Python version to Redis * Store worker version in __init__ body as suggested in review
* Fix unreliable test (#1126)Vladimir Protasov2019-09-081-2/+5
| | | Also make error message more useful in case of future failures.
* Fixed test errors caused by _sentry_trace_headersSelwin Ong2019-07-201-4/+0
|
* Added logging statements to handle_job_success and handle_job_failure (#1112)Selwin Ong2019-06-221-5/+4
|
* add the ability to have the worker stop executing after a max amount of jobs ↵Paul Robertson2019-06-121-2/+13
| | | | | | | | | | (#1094) * add the ability to have the worker stop executing after a max amount of jobs * rename to max-jobs * updated logging messages
* Fix timeout adding job to StartedJobRegistry (#1086)Ted Summer2019-05-181-0/+17
| | | | | | | | | | * Fix timeout adding job to StartedJobRegistry * Fix prepare_job_execution handling neg timeout * Add test for inf job timeout in StartedJobRegistry * refactor(worker): simplify checking neg timeout
* RQ v1.0! (#1059)Selwin Ong2019-03-301-64/+120
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Added FailedJobRegistry. * Added job.failure_ttl. * queue.enqueue() now supports failure_ttl * Added registry.get_queue(). * FailedJobRegistry.add() now assigns DEFAULT_FAILURE_TTL. * StartedJobRegistry.cleanup() now moves expired jobs to FailedJobRegistry. * Failed jobs are now added to FailedJobRegistry. * Added FailedJobRegistry.requeue() * Document the new `FailedJobRegistry` and changes in custom exception handler behavior. * Added worker.disable_default_exception_handler. * Document --disable-default-exception-handler option. * Deleted worker.failed_queue. * Deleted "move_to_failed_queue" exception handler. * StartedJobRegistry should no longer move jobs to FailedQueue. * Deleted requeue_job * Fixed test error. * Make requeue cli command work with FailedJobRegistry * Added .pytest_cache to gitignore. * Custom exception handlers are no longer run in reverse * Restored requeue_job function * Removed get_failed_queue * Deleted FailedQueue * Updated changelog. * Document `failure_ttl` * Updated docs. * Remove job.status * Fixed typo in test_registry.py * Replaced _pipeline() with pipeline() * FailedJobRegistry no longer fails on redis-py>=3 * Fixes test_clean_registries * Worker names are now randomized * Added a note about random worker names in CHANGES.md * Worker will now stop working when encountering an unhandled exception. * Worker should reraise SystemExit on cold shutdowns * Added anchor.js to docs * Support for Sentry-SDK (#1045) * Updated RQ to support sentry-sdk * Document Sentry integration * Install sentry-sdk before running tests * Improved rq info CLI command to be more efficient when displaying lar… (#1046) * Improved rq info CLI command to be more efficient when displaying large number of workers * Fixed an rq info --by-queue bug * Fixed worker.total_working_time bug (#1047) * queue.enqueue() no longer accepts `timeout` argument (#1055) * Clean worker registry (#1056) * queue.enqueue() no longer accepts `timeout` argument * Added clean_worker_registry() * Show worker hostname and PID on cli (#1058) * Show worker hostname and PID on cli * Improve test coverage * Remove Redis version check when SSL is used * Bump version to 1.0 * Removed pytest_cache/README.md * Changed worker logging to use exc_info=True * Removed unused queue.dequeue() * Fixed typo in CHANGES.md * setup_loghandlers() should always call logger.setLevel() if specified
* Make logging in worker consitent. (#1030)Wolfgang Langner2019-01-301-4/+4
| | | Switch some messages from warn to info because it is normal requested bahavior.
* Update/add flag for description logging (#991)Finnci2019-01-221-0/+18
| | | | | | | | | | | | * test workers * indent * add docs and add option to the cli * rename flag for cli * logging
* skip test_1_sec_shutdown with pypy (#1020)Samuel Colvin2018-12-041-0/+1
| | | | | | * skip test_1_sec_shutdown with pypy, fix #1019 * skip all HerokuWorkerShutdownTestCase with pypy
* modify zadd calls for redis-py 3.0 (#1016)Darshan Rai2018-12-031-5/+5
| | | | | | | | | | | | | | | | | | | * modify zadd calls for redis-py 3.0 redis-py 3.0 changes the zadd interface that accepts a single mapping argument that is expected to be a dict. https://github.com/andymccurdy/redis-py#mset-msetnx-and-zadd * change FailedQueue.push_job_id to always push a str redis-py 3.0 does not attempt to cast values to str and is left to the user. * remove Redis connection patching Since in redis-py 3.0, Redis == StrictRedis class, we no longer need to patch _zadd and other methods. Ref: https://github.com/rq/rq/pull/1016#issuecomment-441010847
* Replace "timeout" argument in queue.enqueue() with "job_timeout" (#1010)Selwin Ong2018-11-221-3/+3
|
* Fixed a unicode test.Selwin Ong2018-10-261-8/+10
|
* SimpleWorker's ttl must always be longer than jobs. (#1002)Selwin Ong2018-10-201-5/+16
|
* worker.main_work_horse should always return 0Selwin Ong2018-05-071-8/+9
|
* add periodic worker heartbeats (#945)Thomas Kriechbaumer2018-05-061-1/+19
| | | | | | | | * add periodic worker heartbeats fixes #944 * improve worker default option handling
* Added the ability to fetch workers by queue (#911)Selwin Ong2017-12-181-2/+27
| | | | | | | | | | | | * job.exc_info is now compressed. * job.data is now stored in compressed format. * Added worker_registration.unregister. * Added worker_registration.get_keys(). * Modified Worker.all(), Worker.all_keys() and Worker.count() to accept "connection" and "queue" arguments.
* improve logging in worker.py (#902)Samuel Colvin2017-11-231-0/+24
| | | | | | * improve logging in worker * tests for log_result_lifespan
* Job compression (#907)Selwin Ong2017-11-231-8/+12
| | | | | | job.exc_info and job.data is now stored in compressed format in Redis. * job.data is now stored in compressed format.
* Fixed an issue where `birth` not present in Redis (#901)vanife2017-11-171-4/+9
| | | | | | | | * Fixed an issue where `birth` not present in Redis Fixed an issue where worker.refresh() may fail if `birth` is not present in Redis * added test coverage
* Fixed an issue where worker.refresh() may fail if last_heartbeat is not ↵Selwin Ong2017-11-031-2/+6
| | | | present in Redis.
* Worker statistics (#897)Selwin Ong2017-10-281-2/+45
| | | | | | | | | | | | | | * First stab at implementing worker statistics. * Moved worker data restoration logic to worker.refresh(). * Failed and successfull job counts are now properly incremented. * Worker now keeps track of total_working_time * Ensure job.ended_at is set in the case of unhandled job failure. * handle_job_failure shouldn't crash if job.started_at is not present.
* add milliseconds into timestamps, fix #721Samuel Colvin2017-09-091-3/+2
|
* Fixed #866 - Flak8 errorsTheo2017-09-081-16/+8
|
* Fixed #866 - Flak8 errorsTheo2017-09-081-12/+20
|
* remove python 2.6 supportSamuel Colvin2017-09-051-4/+2
|
* Worker.find_by_key should use hmget instead of repeated hget calls. (#826)Selwin Ong2017-04-291-4/+17
|
* Solve the UnicodeDecodeError while decode literal things. (#817)Peng Liu2017-04-191-0/+14
| | | | | | | * Solve the UnicodeDecodeError while decode literal things. * Add test case for when worker result is a unicode or str object that other than pure ascii content.
* Merge pull request #757 from jaywink/fix-unicode-decode-errorSelwin Ong2017-04-041-0/+22
|\ | | | | Fix UnicodeDecodeError when failing jobs
| * Fix UnicodeDecodeError when failing jobsJason Robinson2017-02-101-0/+22
| | | | | | | | | | | | | | | | Worker handle_exception and move_to_failed_queue couldn't handle a situation where the exception raised had non-ascii characters. This caused a UnicodeDecodeError when trying to format the exception strings. If on Python 2, ensure strings get decoded before building the exception string. Closes #482
* | correct heroku worker exit logicSamuel Colvin2017-03-091-1/+1
|/ | | | as per @Chronial's comment on b4b99f3
* Merge pull request #786 from jezdez/backend-class-overridesSelwin Ong2017-02-031-1/+1
|\ | | | | Allow passing backend classes from CLI and other APIs
| * Allow passing backend classes (job, queue, worker, connection) from CLI and ↵Jannis Leidel2017-01-251-1/+1
| | | | | | | | | | | | | | | | | | | | | | other APIs This includes: - a partial refactor of the CLI to organize the shared options - extends the tests in areas where passing custom backend classes makes sense - allow setting the core CLI options as env vars - minor cosmetic changes here and there
* | Add similar test for when the job failsBenjamin Root2017-01-271-2/+29
| |
* | Added integration test for the metadata persistence featureBenjamin Root2017-01-271-1/+21
|/
* cope with python 2.6 yawwwwwwnSamuel Colvin2016-12-201-2/+4
|
* skip test_run_scheduled_access_self on pypySamuel Colvin2016-12-201-0/+2
|
* allow for already dead horse on heroku worker terminationSamuel Colvin2016-12-201-3/+2
|
* Merge pull request #763 from jezdez/skip-heroku-macosSelwin Ong2016-11-301-0/+4
|\ | | | | Stop running some Heroku tests on macOS since it’s missing the appropriate signals
| * Stop running some Heroku tests on macOS since it’s missing the appropriate ↵Jannis Leidel2016-11-291-0/+4
| | | | | | | | | | | | signals Refs issue #584.
* | Merge branch 'master' into 739_fix_race_conditionStefan Hammer2016-11-191-0/+2
|\ \ | |/
| * Fix PEP8 complaintsVincent Driessen2016-11-181-0/+2
| |
* | refactored worker codeStefan Hammer2016-09-221-1/+1
| | | | | | | | | | Moved code into a new handle_job_success() method and reduced context of used pipelines.
* | added a test for the race conditionStefan Hammer2016-08-131-0/+36
|/
* use compat for PY2 test and add testsSamuel Colvin2016-07-251-2/+13
|
View Lorry Controller Status