Gunicorn Worker how to increase

Install / Update Deployment
1

HI,

i was optimizing the server and found that gunicorn_workers is set to 2

i changed it according to requiments, and ran the following commands:

bench setup supervisor
sudo supervisorctl reread
sudo supervisorctl update

but still gunicorn process is running with old parameters.
i restarted the server as well. but still the same.

i found that supervisor.conf file is updated with the new parameters.

any help will be appreciated.

Regards,
Adnan

1 Like
2

Is this what you are after?

How to increase number of gunicorns workers - #2 by ccfiel

3

Yes Clarkj,

i tried the options from this thread, but didnt work for me.

is there anything else i need to check?

Thanks

4

Couldnā€™t say - I am just learning by suggesting.

Your next options are ask anand or look into the codeā€¦I can do that if need be.

Maybe you have found a bug?

5

Thanks for your supportā€¦

6

I played a bit - ā€˜bench setup supervisorā€™ seems to simply set the config/supervisor.conf default values!? I would need to check where these are read fromā€¦

For eg. this default
command=/home/frappe/frappe-bench/env/bin/gunicorn -b 127.0.0.1:8000 -w 1 -t 120 frappe.app:application --preload

So I simply directly edited config/supervisor.conf to set -w to 2 for eg.

command=/home/frappe/frappe-bench/env/bin/gunicorn -b 127.0.0.1:8000 -w 2 -t 120 frappe.app:application --preload

Then ran these:

sudo supervisorctl reread
sudo supervisorctl update

Then restarted the supervisor with this:

sudo supervisorctl stop all
sudo supervisorctl start all

Then confirm the results like this:

less logs/web.error.log

[2017-09-16 05:40:17 +0000] [19289] [INFO] Shutting down: Master
[2017-09-16 05:40:20 +0000] [19349] [INFO] Starting gunicorn 19.6.0
[2017-09-16 05:40:20 +0000] [19349] [INFO] Listening at: http://127.0.0.1:8000 (19349)
[2017-09-16 05:40:20 +0000] [19349] [INFO] Using worker: sync
[2017-09-16 05:40:20 +0000] [19366] [INFO] Booting worker with pid: 19366
[2017-09-16 05:40:20 +0000] [19367] [INFO] Booting worker with pid: 19367

ps aux | grep gunicorn

frappe@erpnext:~/frappe-bench$ ps aux | grep gunicorn
frappe 19349 0.0 2.3 33440 23988 ? S 05:40 0:01 /home/frappe/frappe-bench/env/bin/python /home/frappe/frappe-bench/env/bin/gunicorn -b 127.0.0.1:8000 -w 2 -t 120 frappe.app:application --preload
frappe 19366 0.0 2.0 33440 21232 ? S 05:40 0:00 /home/frappe/frappe-bench/env/bin/python /home/frappe/frappe-bench/env/bin/gunicorn -b 127.0.0.1:8000 -w 2 -t 120 frappe.app:application --preload
frappe 19367 0.0 2.4 35712 25244 ? S 05:40 0:00 /home/frappe/frappe-bench/env/bin/python /home/frappe/frappe-bench/env/bin/gunicorn -b 127.0.0.1:8000 -w 2 -t 120 frappe.app:application --preload
frappe 19629 0.0 0.0 4680 840 pts/0 S+ 06:47 0:00 grep --color=auto gunicorn
frappe@erpnext:~/frappe-bench$

I am not sure why there are 3 gunicorn processs here?

hth

7

seems like it worked for you.
the 3rd one is for the scheduler jobsā€¦dont worry about thatā€¦

let me follow your procedure to test it with my server. will update you.

really appreciate your efforts and time to resolve this issue.

Regards,
Adnan

8

Hi clarkej,

i gues bydefult there are 3 workers for gunicorn. can you please try to increase this number from 2 to 5 or 10?

i guess that change didnt take effect. on my machine it was running with 3 workersā€¦after all changes i tried itā€™s still threeā€¦

9

ok I changed the -w parameter (number of workers) from 1 to 5, and restarted the supervisor with these results below.

So the result is 6x gunicorn processes, each with 5x workers - I am not sure about why that number of processes?

frappe@erpnext:~/frappe-bench$ ps aux | grep gunicorn
frappe 20623 4.3 2.3 33440 23996 ? S 17:13 0:00 /home/frappe/frappe-bench/env/bin/python /home/frappe/frappe-bench/env/bin/gunicorn -b 127.0.0.1:8000 -w 5 -t 120 frappe.app:application --preload
frappe 20667 0.0 2.0 33440 21236 ? S 17:13 0:00 /home/frappe/frappe-bench/env/bin/python /home/frappe/frappe-bench/env/bin/gunicorn -b 127.0.0.1:8000 -w 5 -t 120 frappe.app:application --preload
frappe 20668 0.0 2.0 33440 21240 ? S 17:13 0:00 /home/frappe/frappe-bench/env/bin/python /home/frappe/frappe-bench/env/bin/gunicorn -b 127.0.0.1:8000 -w 5 -t 120 frappe.app:application --preload
frappe 20669 0.0 2.0 33440 21236 ? S 17:13 0:00 /home/frappe/frappe-bench/env/bin/python /home/frappe/frappe-bench/env/bin/gunicorn -b 127.0.0.1:8000 -w 5 -t 120 frappe.app:application --preload
frappe 20670 0.0 2.0 33440 21236 ? S 17:13 0:00 /home/frappe/frappe-bench/env/bin/python /home/frappe/frappe-bench/env/bin/gunicorn -b 127.0.0.1:8000 -w 5 -t 120 frappe.app:application --preload
frappe 20671 2.1 2.5 36312 25908 ? S 17:13 0:00 /home/frappe/frappe-bench/env/bin/python /home/frappe/frappe-bench/env/bin/gunicorn -b 127.0.0.1:8000 -w 5 -t 120 frappe.app:application --preload
frappe 20677 0.0 0.0 4680 836 pts/0 S+ 17:13 0:00 grep --color=auto gunicorn

Hereā€™s the contents of logs/web.error.log

[2017-09-16 17:12:20 +0000] [19366] [INFO] Worker exiting (pid: 19366)
[2017-09-16 17:12:20 +0000] [19367] [INFO] Worker exiting (pid: 19367)
[2017-09-16 17:12:21 +0000] [19349] [INFO] Shutting down: Master
[2017-09-16 17:12:23 +0000] [20575] [INFO] Starting gunicorn 19.6.0
[2017-09-16 17:12:23 +0000] [20575] [INFO] Listening at: http://127.0.0.1:8000 (20575)
[2017-09-16 17:12:23 +0000] [20575] [INFO] Using worker: sync
[2017-09-16 17:12:23 +0000] [20592] [INFO] Booting worker with pid: 20592
[2017-09-16 17:12:23 +0000] [20593] [INFO] Booting worker with pid: 20593
[2017-09-16 17:12:23 +0000] [20594] [INFO] Booting worker with pid: 20594
[2017-09-16 17:12:23 +0000] [20595] [INFO] Booting worker with pid: 20595
[2017-09-16 17:12:23 +0000] [20596] [INFO] Booting worker with pid: 20596
[2017-09-16 17:12:41 +0000] [20575] [INFO] Handling signal: term
[2017-09-16 17:12:41 +0000] [20592] [INFO] Worker exiting (pid: 20592)
[2017-09-16 17:12:41 +0000] [20593] [INFO] Worker exiting (pid: 20593)
[2017-09-16 17:12:41 +0000] [20594] [INFO] Worker exiting (pid: 20594)
[2017-09-16 17:12:41 +0000] [20595] [INFO] Worker exiting (pid: 20595)
[2017-09-16 17:12:41 +0000] [20596] [INFO] Worker exiting (pid: 20596)
[2017-09-16 17:12:42 +0000] [20575] [INFO] Shutting down: Master
[2017-09-16 17:13:10 +0000] [20623] [INFO] Starting gunicorn 19.6.0
[2017-09-16 17:13:10 +0000] [20623] [INFO] Listening at: http://127.0.0.1:8000 (20623)
[2017-09-16 17:13:11 +0000] [20623] [INFO] Using worker: sync
[2017-09-16 17:13:11 +0000] [20667] [INFO] Booting worker with pid: 20667
[2017-09-16 17:13:11 +0000] [20668] [INFO] Booting worker with pid: 20668
[2017-09-16 17:13:11 +0000] [20669] [INFO] Booting worker with pid: 20669
[2017-09-16 17:13:11 +0000] [20670] [INFO] Booting worker with pid: 20670
[2017-09-16 17:13:11 +0000] [20671] [INFO] Booting worker with pid: 20671

Incidentally you do get output like below when you run this, correct? (I only ask as I have an instance that gives no output here for this command.)

frappe@erpnext:~/frappe-bench$ sudo supervisorctl start all
frappe-bench-frappe-schedule: started
frappe-bench-frappe-default-worker-0: started
frappe-bench-frappe-long-worker-0: started
frappe-bench-frappe-short-worker-0: started
frappe-bench-frappe-web: started
frappe-bench-node-socketio: started
frappe-bench-redis-queue: started
frappe-bench-redis-cache: started
frappe-bench-redis-socketio: started

10

Thanks for your help,

i got it working. i found where were the problem, you gave me hint that you directly change the supervisor.conf file.

i did the same thing not in config folder but in /etc/supervisor and it workedā€¦

Thanks for your supportā€¦you made my day.

Regards,
Adnan

1 Like