diff options
| author | Selwin Ong <selwin.ong@gmail.com> | 2018-03-03 07:21:44 +0700 |
|---|---|---|
| committer | Selwin Ong <selwin.ong@gmail.com> | 2018-03-03 07:21:44 +0700 |
| commit | 28bbeca66aa2dd24f0ff6270c086bad55efe0fde (patch) | |
| tree | 818d81cac131d4bd1008a9f1fffe7cc8e37ed028 /docs/contrib | |
| parent | a6eb5d37eea6bf0de64e0c28723be5985f1684b4 (diff) | |
| download | rq-28bbeca66aa2dd24f0ff6270c086bad55efe0fde.tar.gz | |
Add docs folder.
Diffstat (limited to 'docs/contrib')
| -rw-r--r-- | docs/contrib/docs.md | 16 | ||||
| -rw-r--r-- | docs/contrib/github.md | 11 | ||||
| -rw-r--r-- | docs/contrib/index.md | 63 | ||||
| -rw-r--r-- | docs/contrib/testing.md | 16 | ||||
| -rw-r--r-- | docs/contrib/vagrant.md | 50 |
5 files changed, 156 insertions, 0 deletions
diff --git a/docs/contrib/docs.md b/docs/contrib/docs.md new file mode 100644 index 0000000..52c5da5 --- /dev/null +++ b/docs/contrib/docs.md @@ -0,0 +1,16 @@ +--- +title: "Documentation" +layout: contrib +--- + +### Running docs locally + +To build the docs, run [jekyll](http://jekyllrb.com/): + +``` +jekyll serve +``` + +If you rather use Vagrant, see [these instructions][v]. + +[v]: {{site.baseurl}}contrib/vagrant/ diff --git a/docs/contrib/github.md b/docs/contrib/github.md new file mode 100644 index 0000000..18c5bfd --- /dev/null +++ b/docs/contrib/github.md @@ -0,0 +1,11 @@ +--- +title: "Contributing to RQ" +layout: contrib +--- + +If you'd like to contribute to RQ, simply [fork](https://github.com/nvie/rq) +the project on GitHub and submit a pull request. + +Please bear in mind the philosiphy behind RQ: it should rather remain small and +simple, than packed with features. And it should value insightfulness over +performance. diff --git a/docs/contrib/index.md b/docs/contrib/index.md new file mode 100644 index 0000000..f503812 --- /dev/null +++ b/docs/contrib/index.md @@ -0,0 +1,63 @@ +--- +title: "RQ: Simple job queues for Python" +layout: contrib +--- + +This document describes how RQ works internally when enqueuing or dequeueing. + + +## Enqueueing internals + +Whenever a function call gets enqueued, RQ does two things: + +* It creates a job instance representing the delayed function call and persists + it in a Redis [hash][h]; and +* It pushes the given job's ID onto the requested Redis queue. + +All jobs are stored in Redis under the `rq:job:` prefix, for example: + + rq:job:55528e58-9cac-4e05-b444-8eded32e76a1 + +The keys of such a job [hash][h] are: + + created_at => '2012-02-13 14:35:16+0000' + enqueued_at => '2012-02-13 14:35:16+0000' + origin => 'default' + data => <pickled representation of the function call> + description => "count_words_at_url('http://nvie.com')" + +Depending on whether or not the job has run successfully or has failed, the +following keys are available, too: + + ended_at => '2012-02-13 14:41:33+0000' + result => <pickled return value> + exc_info => <exception information> + +[h]: http://redis.io/topics/data-types#hashes + + +## Dequeueing internals + +Whenever a dequeue is requested, an RQ worker does two things: + +* It pops a job ID from the queue, and fetches the job data belonging to that + job ID; +* It starts executing the function call. +* If the job succeeds, its return value is written to the `result` hash key and + the hash itself is expired after 500 seconds; or +* If the job failes, the exception information is written to the `exc_info` + hash key and the job ID is pushed onto the `failed` queue. + + +## Cancelling jobs + +Any job ID that is encountered by a worker for which no job hash is found in +Redis is simply ignored. This makes it easy to cancel jobs by simply removing +the job hash. In Python: + + from rq import cancel_job + cancel_job('2eafc1e6-48c2-464b-a0ff-88fd199d039c') + +Note that it is irrelevant on which queue the job resides. When a worker +eventually pops the job ID from the queue and notes that the Job hash does not +exist (anymore), it simply discards the job ID and continues with the next. diff --git a/docs/contrib/testing.md b/docs/contrib/testing.md new file mode 100644 index 0000000..8abcdc1 --- /dev/null +++ b/docs/contrib/testing.md @@ -0,0 +1,16 @@ +--- +title: "Testing" +layout: contrib +--- + +### Testing RQ locally + +To run tests locally; + +``` +tox +``` + +If you rather use Vagrant, see [these instructions][v]. + +[v]: {{site.baseurl}}contrib/vagrant/ diff --git a/docs/contrib/vagrant.md b/docs/contrib/vagrant.md new file mode 100644 index 0000000..c114c7c --- /dev/null +++ b/docs/contrib/vagrant.md @@ -0,0 +1,50 @@ +--- +title: "Using Vagrant" +layout: contrib +--- + +If you don't feel like installing dependencies on your main development +machine, you can use [Vagrant](https://www.vagrantup.com/). Here's how you run +your tests and build the documentation on Vagrant. + + +### Running tests in Vagrant + +To create a working Vagrant environment, use the following; + +``` +vagrant init ubuntu/trusty64 +vagrant up +vagrant ssh -- "sudo apt-get -y install redis-server python-dev python-pip" +vagrant ssh -- "sudo pip install --no-input redis hiredis mock" +vagrant ssh -- "(cd /vagrant; ./run_tests)" +``` + + +### Running docs on Vagrant + +``` +vagrant init ubuntu/trusty64 +vagrant up +vagrant ssh -- "sudo apt-get -y install ruby-dev nodejs" +vagrant ssh -- "sudo gem install jekyll" +vagrant ssh -- "(cd /vagrant; jekyll serve)" +``` + +You'll also need to add a port forward entry to your `Vagrantfile`; + +``` +config.vm.network "forwarded_port", guest: 4000, host: 4001 +``` + +Then you can access the docs using; + +``` +http://127.0.0.1:4001 +``` + +You also may need to forcibly kill Jekyll if you ctrl+c; + +``` +vagrant ssh -- "sudo killall -9 jekyll" +``` |
