Hi everyone,
I’m facing an issue with a cron job in my Frappe application.
Here’s the scenario: I have a doctype for logging, which generates around 3 to 5 million records per month. On the first day of every month, we run a cron job (a background job) to aggregate these logs and save them in another table, essentially another doctype.
The problem is, sometimes this aggregation process takes 15-30 minutes, and occasionally even longer. To schedule this cron job, I’m using the Frappe scheduler. However, I’ve noticed that the worker starts but then closes without completing the task. The aggregation process doesn’t happen as expected.
Strangely, when I manually trigger the event from the console, everything works fine. How can I ensure that this process runs seamlessly without manual intervention?
Any insights or solutions would be greatly appreciated. Thanks!
Max timeout period for background job is for long queue tasks, i.e. 25 min.
So it tasks that exceed this period will be failed, so you have to manually change the timeout period of long queue or create a custom queue.
Refer for more information:
https://frappeframework.com/docs/user/en/api/background_jobs#queue
@Ninad1306 Thanks for the reply, but I was referring specifically to scheduled tasks, not background queues, Just to clarify, is there any time limit for scheduler tasks? As far as I know, there isn’t one, but I wanted to confirm with you
All the scheduler events in frappe are background jobs only and by default the worker assigned is default worker which has timeout 300s.
You can configure the worker assigned to the event.
Refer documentation.
@Ninad1306 Thank you very much for your reply. I had already glanced over the document, but your message prompted me to revisit it with greater attention to detail. Upon doing so, I noticed the distinctions you mentioned, particularly regarding the jobs running in the long worker.
I’ll adjust my scheduled tasks accordingly, setting them to run monthly as long jobs. Once I’ve made these changes, I’ll run some tests and confirm with you.
Thanks again for your guidance!
1 Like
