Change celery's \x06\x16 priority separator

2021-01-21

Celery has native support for task priorities when using rabbitmq as a broker and “emulated” priority support when using redis as a broker.

“Emulated” means that celery is putting the tasks into multiple internal priority queues and that it consumes from them in priority order, so that if the first queue always contains messages, the rest of the queues in the list will never be consumed from.

If you have only one celery queue then in the default priority mode celery will create 3 additional internal queues. Those queues will be named like this: celery, celery\\x06\\x163, celery\\x06\\x166 and celery\\x06\\x169. This naming schema makes it hard to use the redis-cli command line tool and it’s not easy on the eyes.

Since kombu >= 4.5.0 the priority separator is configurable using the sep transport option:

with Connection('redis://', transport_options={
        'sep': ':',
    }):
    # ...
    pass

If you are using django you can change it this way in your settings.py file:

CELERY_BROKER_TRANSPORT_OPTIONS = {
    'sep': ':',
}

Now your priority queues are shell friendly and look nice: celery, celery:3, celery:6 and celery:9 :-)

Snippetspythoncelerydjango

Django ForeignKey partial index

Print django SQL queries in development