Django integration with RQ, a Redis based Python queuing library. Django-RQ is a simple app that allows you to configure your queues in django's settings.py and easily use them in your project.

If you find django-rq useful, please consider supporting its development via Tidelift.

Django (2.0+)

RQ

Install django-rq (or download from PyPI ):

Add django_rq to INSTALLED_APPS in settings.py :

Configure your queues in django's settings.py :

Include django_rq.urls in your urls.py :

Django-RQ allows you to easily put jobs into any of the queues defined in settings.py. It comes with a few utility functions:

enqueue - push a job to the default queue:

get_queue - returns an Queue instance.

In addition to name argument, get_queue also accepts default_timeout, is_async, autocommit, connection and queue_class arguments. For example:

You can provide your own singleton Redis connection object to this function so that it will not create a new connection object for each queue definition. This will help you limit number of connections to Redis server. For example:

get_connection - accepts a single queue name argument (defaults to "default") and returns a connection to the queue's Redis server:

get_worker - accepts optional queue names and returns a new RQ Worker instance for specified queues (or default queue):

To easily turn a callable into an RQ task, you can also use the @job decorator that comes with django_rq :

You can pass in any arguments that RQ's job decorator accepts:

It's possible to specify default for result_ttl decorator keyword argument via DEFAULT_RESULT_TTL setting:

With this setting, job decorator will set result_ttl to 5000 unless it's specified explicitly.

django_rq provides a management command that starts a worker for every queue specified as arguments:

python manage.py rqworker high default low If you want to run rqworker in burst mode, you can pass in the --burst flag:

python manage.py rqworker high default low --burst If you need to use custom worker, job or queue classes, it is best to use global settings (see Custom queue classes and Custom job and worker classes ). However, it is also possible to override such settings with command line options as follows.

To use a custom worker class, you can pass in the --worker-class flag with the path to your worker:

python manage.py rqworker high default low --worker-class 'path.to.GeventWorker' To use a custom queue class, you can pass in the --queue-class flag with the path to your queue class:

python manage.py rqworker high default low --queue-class 'path.to.CustomQueue' To use a custom job class, provide --job-class flag.

With RQ 1.2.0. you can use built-in scheduler for your jobs. For example:

If you are using built-in scheduler you have to start workers with scheduler support:

python manage.py rqworker --with-scheduler Alternatively you can use RQ Scheduler. After install you can also use the get_scheduler function to return a Scheduler instance for queues defined in settings.py's RQ_QUEUES. For example:

You can also use the management command rqscheduler to start the scheduler:

python manage.py rqscheduler ### Support for django-redis and django-redis-cache

If you have django-redis or django-redis-cache installed, you can instruct django_rq to use the same connection information from your Redis cache. This has two advantages: it's DRY and it takes advantage of any optimization that may be going on in your cache setup (like using connection pooling or Hiredis.)

To use configure it, use a dict with the key USE_REDIS_CACHE pointing to the name of the desired cache in your RQ_QUEUES dict. It goes without saying that the chosen cache must exist and use the Redis backend. See your respective Redis cache package docs for configuration instructions. It's also important to point out that since the django-redis-cache ShardedClient splits the cache over multiple Redis connections, it does not work.

Here is an example settings fragment for django-redis:

django_rq also provides a dashboard to monitor the status of your queues at /django-rq/ (or whatever URL you set in your urls.py during installation.

You can also add a link to this dashboard link in /admin by adding RQ_SHOW_ADMIN_LINK = True in settings.py. Be careful though, this will override the default admin template so it may interfere with other apps that modifies the default admin template.

These statistics are also available in JSON format via /django-rq/stats.json, which is accessible to staff members. If you need to access this view via other HTTP clients (for monitoring purposes), you can define RQ_API_TOKEN and access it via /django-rq/stats.json/<API_TOKEN>.

Note: Statistics of scheduled jobs display jobs from RQ built-in scheduler, not optional RQ scheduler.

Additionally, these statistics are also accessible from  the command line.

Django-RQ >= 2.0 uses sentry-sdk instead of the deprecated raven library. Sentry should be configured within the Django settings.py as described in the Sentry docs.

You can override the default Django Sentry configuration when running the rqworker command by passing the sentry-dsn option:

`./manage.py rqworker --sentry-dsn=https://@sentry.io/222222`

This will override any existing Django configuration and reinitialise Sentry, setting the following Sentry options:

RQ uses Python's logging, this means you can easily configure rqworker 's logging mechanism in django's settings.py. For example:

Note: error logging to Sentry is known to be unreliable with RQ when using async transports (the default transport). Please configure Raven to use sync+https:// or requests+https:// transport in settings.py :

For more info, refer to Raven's documentation.

By default, every queue will use DjangoRQ class. If you want to use a custom queue class, you can do so by adding a QUEUE_CLASS option on a per queue basis in RQ_QUEUES :

or you can specify DjangoRQ to use a custom class for all your queues in RQ settings:

Custom queue classes should inherit from django_rq.queues.DjangoRQ.

If you are using more than one queue class (not recommended), be sure to only run workers on queues with same queue class. For example if you have two queues defined in RQ_QUEUES and one has custom class specified, you would have to run at least two separate workers for each queue.

Similarly to custom queue classes, global custom job and worker classes can be configured using JOB_CLASS and WORKER_CLASS settings:

Custom job class should inherit from rq.job.Job. It will be used for all jobs if configured.

Custom worker class should inherit from rq.worker.Worker. It will be used for running all workers unless overridden by rqworker management command worker-class option.

For an easier testing process, you can run a worker synchronously this way:

You can set the option ASYNC to False to make synchronous operation the default for a given queue. This will cause jobs to execute immediately and on the same thread as they are dispatched, which is useful for testing and debugging. For example, you might add the following after you queue configuration in your settings file:

Note that setting the is_async parameter explicitly when calling get_queue will override this setting.

To run django_rq 's test suite:

which django-admin.py test django_rq --settings=django_rq.tests.settings --pythonpath=. ## Deploying on Ubuntu

Create an rqworker service that runs the high, default, and low queues.

sudo vi /etc/systemd/system/rqworker.service

Enable and start the service

Add django-rq to your requirements.txt file with:

Update your Procfile to:

Commit and re-deploy. Then add your new worker with:

See CHANGELOG.md.