Celery : manage WorkerLostErrors
As of now, when a WorkerLostError happen, we do not have any traceback available. We do not know why the Worker crashed, We do not know which task was impacted, We know nothing. This has to change
Let's have a look at https://til.justinthurman.dev/how-to-add-context-to-celery-workerlosterror-logs and https://ankurdhuriya.medium.com/handling-failures-in-celery-workers-retries-timeouts-and-error-handling-97571b131267
This page describe how to use the Request object to manage WorkerLostErrors. But maybe we could also use the on_failure Task method to manage these ?
See :
- https://docs.celeryq.dev/en/stable/userguide/tasks.html#requests-and-custom-requests
- https://docs.celeryq.dev/en/stable/userguide/tasks.html#handlers
If this doesn't work, have a look at that too
Edited by Simon Panay