Gunicorn Worker how to increase

HI,

i was optimizing the server and found that gunicorn_workers is set to 2

i changed it according to requiments, and ran the following commands:

bench setup supervisor
sudo supervisorctl reread
sudo supervisorctl update

but still gunicorn process is running with old parameters.
i restarted the server as well. but still the same.

i found that supervisor.conf file is updated with the new parameters.

any help will be appreciated.

Regards,
Adnan

1 Like

Is this what you are after?

How to increase number of gunicorns workers - #2 by ccfiel

Yes Clarkj,

i tried the options from this thread, but didnt work for me.

is there anything else i need to check?

Thanks

Couldn’t say - I am just learning by suggesting.

Your next options are ask anand or look into the code…I can do that if need be.

Maybe you have found a bug?

Thanks for your support…

I played a bit - ‘bench setup supervisor’ seems to simply set the config/supervisor.conf default values!? I would need to check where these are read from…

For eg. this default
command=/home/frappe/frappe-bench/env/bin/gunicorn -b 127.0.0.1:8000 -w 1 -t 120 frappe.app:application --preload

So I simply directly edited config/supervisor.conf to set -w to 2 for eg.

command=/home/frappe/frappe-bench/env/bin/gunicorn -b 127.0.0.1:8000 -w 2 -t 120 frappe.app:application --preload

Then ran these:

sudo supervisorctl reread
sudo supervisorctl update

Then restarted the supervisor with this:

sudo supervisorctl stop all
sudo supervisorctl start all

Then confirm the results like this:

less logs/web.error.log

[2017-09-16 05:40:17 +0000] [19289] [INFO] Shutting down: Master
[2017-09-16 05:40:20 +0000] [19349] [INFO] Starting gunicorn 19.6.0
[2017-09-16 05:40:20 +0000] [19349] [INFO] Listening at: http://127.0.0.1:8000 (19349)
[2017-09-16 05:40:20 +0000] [19349] [INFO] Using worker: sync
[2017-09-16 05:40:20 +0000] [19366] [INFO] Booting worker with pid: 19366
[2017-09-16 05:40:20 +0000] [19367] [INFO] Booting worker with pid: 19367

ps aux | grep gunicorn

frappe@erpnext:~/frappe-bench$ ps aux | grep gunicorn
frappe 19349 0.0 2.3 33440 23988 ? S 05:40 0:01 /home/frappe/frappe-bench/env/bin/python /home/frappe/frappe-bench/env/bin/gunicorn -b 127.0.0.1:8000 -w 2 -t 120 frappe.app:application --preload
frappe 19366 0.0 2.0 33440 21232 ? S 05:40 0:00 /home/frappe/frappe-bench/env/bin/python /home/frappe/frappe-bench/env/bin/gunicorn -b 127.0.0.1:8000 -w 2 -t 120 frappe.app:application --preload
frappe 19367 0.0 2.4 35712 25244 ? S 05:40 0:00 /home/frappe/frappe-bench/env/bin/python /home/frappe/frappe-bench/env/bin/gunicorn -b 127.0.0.1:8000 -w 2 -t 120 frappe.app:application --preload
frappe 19629 0.0 0.0 4680 840 pts/0 S+ 06:47 0:00 grep --color=auto gunicorn
frappe@erpnext:~/frappe-bench$

I am not sure why there are 3 gunicorn processs here?

hth

seems like it worked for you.
the 3rd one is for the scheduler jobs…dont worry about that…

let me follow your procedure to test it with my server. will update you.

really appreciate your efforts and time to resolve this issue.

Regards,
Adnan

Hi clarkej,

i gues bydefult there are 3 workers for gunicorn. can you please try to increase this number from 2 to 5 or 10?

i guess that change didnt take effect. on my machine it was running with 3 workers…after all changes i tried it’s still three…

ok I changed the -w parameter (number of workers) from 1 to 5, and restarted the supervisor with these results below.

So the result is 6x gunicorn processes, each with 5x workers - I am not sure about why that number of processes?

frappe@erpnext:~/frappe-bench$ ps aux | grep gunicorn
frappe 20623 4.3 2.3 33440 23996 ? S 17:13 0:00 /home/frappe/frappe-bench/env/bin/python /home/frappe/frappe-bench/env/bin/gunicorn -b 127.0.0.1:8000 -w 5 -t 120 frappe.app:application --preload
frappe 20667 0.0 2.0 33440 21236 ? S 17:13 0:00 /home/frappe/frappe-bench/env/bin/python /home/frappe/frappe-bench/env/bin/gunicorn -b 127.0.0.1:8000 -w 5 -t 120 frappe.app:application --preload
frappe 20668 0.0 2.0 33440 21240 ? S 17:13 0:00 /home/frappe/frappe-bench/env/bin/python /home/frappe/frappe-bench/env/bin/gunicorn -b 127.0.0.1:8000 -w 5 -t 120 frappe.app:application --preload
frappe 20669 0.0 2.0 33440 21236 ? S 17:13 0:00 /home/frappe/frappe-bench/env/bin/python /home/frappe/frappe-bench/env/bin/gunicorn -b 127.0.0.1:8000 -w 5 -t 120 frappe.app:application --preload
frappe 20670 0.0 2.0 33440 21236 ? S 17:13 0:00 /home/frappe/frappe-bench/env/bin/python /home/frappe/frappe-bench/env/bin/gunicorn -b 127.0.0.1:8000 -w 5 -t 120 frappe.app:application --preload
frappe 20671 2.1 2.5 36312 25908 ? S 17:13 0:00 /home/frappe/frappe-bench/env/bin/python /home/frappe/frappe-bench/env/bin/gunicorn -b 127.0.0.1:8000 -w 5 -t 120 frappe.app:application --preload
frappe 20677 0.0 0.0 4680 836 pts/0 S+ 17:13 0:00 grep --color=auto gunicorn

Here’s the contents of logs/web.error.log

[2017-09-16 17:12:20 +0000] [19366] [INFO] Worker exiting (pid: 19366)
[2017-09-16 17:12:20 +0000] [19367] [INFO] Worker exiting (pid: 19367)
[2017-09-16 17:12:21 +0000] [19349] [INFO] Shutting down: Master
[2017-09-16 17:12:23 +0000] [20575] [INFO] Starting gunicorn 19.6.0
[2017-09-16 17:12:23 +0000] [20575] [INFO] Listening at: http://127.0.0.1:8000 (20575)
[2017-09-16 17:12:23 +0000] [20575] [INFO] Using worker: sync
[2017-09-16 17:12:23 +0000] [20592] [INFO] Booting worker with pid: 20592
[2017-09-16 17:12:23 +0000] [20593] [INFO] Booting worker with pid: 20593
[2017-09-16 17:12:23 +0000] [20594] [INFO] Booting worker with pid: 20594
[2017-09-16 17:12:23 +0000] [20595] [INFO] Booting worker with pid: 20595
[2017-09-16 17:12:23 +0000] [20596] [INFO] Booting worker with pid: 20596
[2017-09-16 17:12:41 +0000] [20575] [INFO] Handling signal: term
[2017-09-16 17:12:41 +0000] [20592] [INFO] Worker exiting (pid: 20592)
[2017-09-16 17:12:41 +0000] [20593] [INFO] Worker exiting (pid: 20593)
[2017-09-16 17:12:41 +0000] [20594] [INFO] Worker exiting (pid: 20594)
[2017-09-16 17:12:41 +0000] [20595] [INFO] Worker exiting (pid: 20595)
[2017-09-16 17:12:41 +0000] [20596] [INFO] Worker exiting (pid: 20596)
[2017-09-16 17:12:42 +0000] [20575] [INFO] Shutting down: Master
[2017-09-16 17:13:10 +0000] [20623] [INFO] Starting gunicorn 19.6.0
[2017-09-16 17:13:10 +0000] [20623] [INFO] Listening at: http://127.0.0.1:8000 (20623)
[2017-09-16 17:13:11 +0000] [20623] [INFO] Using worker: sync
[2017-09-16 17:13:11 +0000] [20667] [INFO] Booting worker with pid: 20667
[2017-09-16 17:13:11 +0000] [20668] [INFO] Booting worker with pid: 20668
[2017-09-16 17:13:11 +0000] [20669] [INFO] Booting worker with pid: 20669
[2017-09-16 17:13:11 +0000] [20670] [INFO] Booting worker with pid: 20670
[2017-09-16 17:13:11 +0000] [20671] [INFO] Booting worker with pid: 20671

Incidentally you do get output like below when you run this, correct? (I only ask as I have an instance that gives no output here for this command.)

frappe@erpnext:~/frappe-bench$ sudo supervisorctl start all
frappe-bench-frappe-schedule: started
frappe-bench-frappe-default-worker-0: started
frappe-bench-frappe-long-worker-0: started
frappe-bench-frappe-short-worker-0: started
frappe-bench-frappe-web: started
frappe-bench-node-socketio: started
frappe-bench-redis-queue: started
frappe-bench-redis-cache: started
frappe-bench-redis-socketio: started

Thanks for your help,

i got it working. i found where were the problem, you gave me hint that you directly change the supervisor.conf file.

i did the same thing not in config folder but in /etc/supervisor and it worked…

Thanks for your support…you made my day.

Regards,
Adnan

1 Like