I'm running two EC2 instances connected to 1 Redis cache. Each EC2 instance is running a celery beat and 4 workers of celery 4.4.7 and celery-singleton 0.3.1.
I'm trying to use celery-singleton to only queue/run one copy of the task from whichever celery beat schedules first.
I have 14 tasks configured with base=Singleton and lock_expiry > task time.
I see only 10 singleton lock keys in my Redis.
Out of the 14 tasks, 9 of them run twice.
Out of the 14 tasks, 3 have parameters. One of those runs once, the other 2 run twice.
One of the tasks is configured with @app.task(base=Singleton, lock_expiry=60, raise_on_duplicate=True) and it still runs two tasks and does not raise an exception.
What could be going wrong so that my 14 configured CELERY_BEAT_SCHEDULE tasks are generating 23 tasks and 10 singleton locks? Why isn't raise_on_duplicate triggering?
I'm running two EC2 instances connected to 1 Redis cache. Each EC2 instance is running a celery beat and 4 workers of celery 4.4.7 and celery-singleton 0.3.1.
I'm trying to use celery-singleton to only queue/run one copy of the task from whichever celery beat schedules first.
I have 14 tasks configured with base=Singleton and lock_expiry > task time.
I see only 10 singleton lock keys in my Redis.
Out of the 14 tasks, 9 of them run twice.
Out of the 14 tasks, 3 have parameters. One of those runs once, the other 2 run twice.
One of the tasks is configured with
@app.task(base=Singleton, lock_expiry=60, raise_on_duplicate=True)and it still runs two tasks and does not raise an exception.What could be going wrong so that my 14 configured CELERY_BEAT_SCHEDULE tasks are generating 23 tasks and 10 singleton locks? Why isn't raise_on_duplicate triggering?