Creating a custom job queue and background worker

Frappe Framework
1

I’m trying to add a custom job queue to my app so I can run a very long job without interfering with other jobs. I followed the instructions in the documentation and added this to common_site_config.json:

 "workers": {
   "my_update": {
            "timeout": 100000,
            "background_workers": 1
     }
   }
}

However, while this appears to create the queue, I don’t believe it is starting the worker. I can enqueue my job on the “my_update” queue and see it queued in “Background Jobs” but it won’t actually run until I type “bench worker --queue my_update” on the command-line. Then the job will run to completion and any other jobs I add to the queue will run, as long as the worker is running.

How can I make Frappe start the worker for the custom queue automatically like it does for the other queues? Do I have to add it manually to config/supervisor.conf ?

2

Do I have to add it manually to config/supervisor.conf ?

In fact, that is what I did and after rebooting the worker thread is running and everything works as I expected. Simply restarting the supervisor after editing supervisor.conf was not enough to start the worker and I had to reboot instead.

1 Like
3

Where do i find that file : config/supervisor.conf ?

is it on etc? i cant seem to find it

any help would be appreciated. thank you

4

It should be in {bench}/config folder. That’s just symlinked to system folder.

1 Like
5

thank @ankush, i have found it