This might come as a shock to some developers, but we are moving away from Celery and will be replacing it with Python RQ.
We have had some bad experiences using Celery for background tasks. It is too complicated, takes a lot of time to debug and often feels like a black box.
Python RQ is really really simple. Because of its minimal feature set, we gain a lot more control and visibility of background jobs. It also has a great monitoring utility called rq-dashboard.
We will release Frappe 7 in about 2-3 weeks.
If your custom app has scheduled jobs like
daily, etc. mentioned in hooks, then you won’t have to change anything.
However, if you have background jobs decorated using
@celery_task that you called using
method.delay(), you might have to change the call to something like
frappe.utils.background_jobs.enqueue(method, **kwargs). Also, the
enqueue method will handle initialization of site, database connection, rollback/commit and cleanup, so that you only have to worry about your background job.
Along with this, we are also cleaning up the
bench installer (will be using ansible which is much much easier to maintain) and adding multi-bench capabilities so that you can host different sets of sites in different benches. I think those who really need this feature know the immense benefits of this. So donations are welcome
We will keep you posted on our progress.
PS. I would love to hear your thoughts on this.
Edit 1: We will also be dropping support for the Async Tasks feature. It wasn’t documented and didn’t work reliably. You can still call functions asynchronously by enqueuing them, but there will be no async magic available.