Document usage of Redis + SSL

[davidjrice]

Ran into some issues getting setup with SSL using celery-singleton

As I couldn't find a working way to pass args necessary to the redis instance used I ended up creating a RedisSSLBackend class and configuring celery with the following:

app=Celery(
    __name__,
    broker=config.REDIS_URI,
    singleton_backend_class=RedisSSLBackend
    broker_use_ssl={
        "ssl_cert_reqs": SSL_VERIFY_MODE,
    },
)

import ssl
from typing import Any
from urllib.parse import urlparse

import redis
from celery_singleton.backends.redis import RedisBackend

from app import config

SSL_VERIFY_MODE: ssl.VerifyMode = ssl.CERT_NONE

class RedisSSLBackend(RedisBackend):
    def __init__(self, *_args: Any, **_kwargs: Any) -> None:
        redis_uri = urlparse(config.REDIS_URI)
        self.redis = redis.Redis(
            host=redis_uri.hostname,
            port=redis_uri.port,
            ssl=True,
            ssl_cert_reqs=SSL_VERIFY_MODE,
        )

[aemdy]

You can use CELERY_SINGLETON prefix to provide config options for the backend.

In your case it would be:

import ssl

CELERY_SINGLETON_BACKEND_KWARGS = {"ssl_cert_reqs": ssl.SSL_VERIFY_MODE}

[apoclyps]

@aemdy Is it possible to make use of the existing broker_use_ssl or is the preferred way to do this via singleton_backend_kwargs ?

Approach 1
Configuring singleton_backend_kwargs to passthrough the kwargs

import ssl
from typing import Final

from app import config
from lib.celery import Celery, celery_queue_name
from lib.celery.singleton import Singleton

def configure_celery(app: Celery) -> Celery:
    app.conf.singleton_backend_kwargs = {
        "ssl_cert_reqs": ssl.CERT_REQUIRED,
        "ssl_certfile": "/ca-certs/redis.crt",
        "ssl_keyfile": "/ca-certs/redis.key",
        "ssl_ca_certs": "/ca-certs/ca.crt",
    }

    return app


celery = configure_celery(
    app=Celery(
        __name__,
        broker="redis://localhost:6379",
        task_create_missing_queues=True,
        task_acks_late=True,
        task_time_limit=60,
    )
)

Aproach 2
Using broker_use_ssl to passthrough the kwargs

celery = Celery(
    __name__,
    broker="redis://localhost:6379",
    task_create_missing_queues=True,
    task_acks_late=True,
    task_time_limit=60,
    broker_use_ssl={
        "ssl_cert_reqs": ssl.CERT_REQUIRED,
        "ssl_certfile": "/ca-certs/redis.crt",
        "ssl_keyfile": "/ca-certs/redis.key",
        "ssl_ca_certs": "/ca-certs/ca.crt",
    }
)

[stuaxo]

Q: As an alternative, does using the "rediss:" URL (with the extra s) work ?