Frappe-bench-web:frappe-bench-node-socketio: ERROR (spawn error)

Ebraheem_Al-Dawlah · November 29, 2022, 11:23am

I have updated frappe/erpnext usning “bench update”.
then got this error
frappe-bench-web:frappe-bench-node-socketio: ERROR (spawn error)

when ran “bench restart”
still the same error.

Output of bench version

erpnext@vps:~/frappe-bench$ bench version
erpnext 14.8.0
frappe 14.16.0
payments 0.0.1

Steps to reproduce the issue

bench update
bench restart

Error on console

frappe@oclits:~/frappe-bench$ bench restart
$ supervisorctl restart frappe-bench-workers: frappe-bench-web:
frappe-bench-workers:frappe-bench-frappe-schedule: stopped
frappe-bench-workers:frappe-bench-frappe-default-worker-0: stopped
frappe-bench-workers:frappe-bench-frappe-short-worker-0: stopped
frappe-bench-workers:frappe-bench-frappe-long-worker-0: stopped
frappe-bench-web:frappe-bench-frappe-web: stopped
frappe-bench-workers:frappe-bench-frappe-schedule: started
frappe-bench-workers:frappe-bench-frappe-default-worker-0: started
frappe-bench-workers:frappe-bench-frappe-short-worker-0: started
frappe-bench-workers:frappe-bench-frappe-long-worker-0: started
frappe-bench-web:frappe-bench-frappe-web: started
frappe-bench-web:frappe-bench-node-socketio: ERROR (spawn error)
ERROR:
Traceback (most recent call last):
  File "/home/frappe/.local/bin/bench", line 8, in <module>
    sys.exit(cli())
  File "/home/frappe/.local/lib/python3.10/site-packages/bench/cli.py", line 127, in cli
    bench_command()
  File "/usr/lib/python3/dist-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3/dist-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python3/dist-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3/dist-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/home/frappe/.local/lib/python3.10/site-packages/bench/commands/utils.py", line 41, in restart
    Bench(".").reload(web, supervisor, systemd)
  File "/home/frappe/.local/lib/python3.10/site-packages/bench/utils/render.py", line 126, in wrapper_fn
    return fn(*args, **kwargs)
  File "/home/frappe/.local/lib/python3.10/site-packages/bench/bench.py", line 151, in reload
    restart_supervisor_processes(bench_path=self.name, web_workers=web)
  File "/home/frappe/.local/lib/python3.10/site-packages/bench/utils/bench.py", line 315, in restart_supervisor_processes
    bench.run(f"{sudo}supervisorctl restart {group}")
  File "/home/frappe/.local/lib/python3.10/site-packages/bench/bench.py", line 48, in run
    return exec_cmd(cmd, cwd=cwd or self.cwd)
  File "/home/frappe/.local/lib/python3.10/site-packages/bench/utils/__init__.py", line 158, in exec_cmd
    raise CommandFailedError
bench.exceptions.CommandFailedError

ankush · November 29, 2022, 11:30am

Can you share your config/supervisor.conf? (nothing secret in there)

Ebraheem_Al-Dawlah · November 29, 2022, 11:42am

Yes no problem

frappe@oclits:~/frappe-bench$ sudo nano config/supervisor.conf
  GNU nano 6.2                                              config/supervisor.conf
; Notes:
; priority=1 --> Lower priorities indicate programs that start first and shut down last
; killasgroup=true --> send kill signal to child processes too

[program:frappe-bench-frappe-web]
command=/home/frappe/frappe-bench/env/bin/gunicorn -b 127.0.0.1:8000 -w 5 --max-requests 5000 --max-requests-jitter 500 -t 120 frappe.ap>
priority=4
autostart=true
autorestart=true
stdout_logfile=/home/frappe/frappe-bench/logs/web.log
stderr_logfile=/home/frappe/frappe-bench/logs/web.error.log
user=frappe
directory=/home/frappe/frappe-bench/sites


[program:frappe-bench-frappe-schedule]
command=/home/frappe/.local/bin/bench schedule
priority=3
autostart=true
autorestart=true
stdout_logfile=/home/frappe/frappe-bench/logs/schedule.log
stderr_logfile=/home/frappe/frappe-bench/logs/schedule.error.log
user=frappe
directory=/home/frappe/frappe-bench

[program:frappe-bench-frappe-default-worker]
command=/home/frappe/.local/bin/bench worker --queue default
priority=4
autostart=true
autorestart=true
stdout_logfile=/home/frappe/frappe-bench/logs/worker.log
stderr_logfile=/home/frappe/frappe-bench/logs/worker.error.log
user=frappe
stopwaitsecs=1560

ankush · November 29, 2022, 11:53am

That looks like partial file. Socketio isn’t there at all in it. Try using cat to print it all.

Ebraheem_Al-Dawlah · November 29, 2022, 12:05pm

frappe@oclits:~/frappe-bench$ sudo cat config/supervisor.conf
[sudo] password for frappe:
; Notes:
; priority=1 --> Lower priorities indicate programs that start first and shut down last
; killasgroup=true --> send kill signal to child processes too

[program:frappe-bench-frappe-web]
command=/home/frappe/frappe-bench/env/bin/gunicorn -b 127.0.0.1:8000 -w 5 --max-requests 5000 --max-requests-jitter 500 -t 120 frappe.app:application --preload
priority=4
autostart=true
autorestart=true
stdout_logfile=/home/frappe/frappe-bench/logs/web.log
stderr_logfile=/home/frappe/frappe-bench/logs/web.error.log
user=frappe
directory=/home/frappe/frappe-bench/sites


[program:frappe-bench-frappe-schedule]
command=/home/frappe/.local/bin/bench schedule
priority=3
autostart=true
autorestart=true
stdout_logfile=/home/frappe/frappe-bench/logs/schedule.log
stderr_logfile=/home/frappe/frappe-bench/logs/schedule.error.log
user=frappe
directory=/home/frappe/frappe-bench

[program:frappe-bench-frappe-default-worker]
command=/home/frappe/.local/bin/bench worker --queue default
priority=4
autostart=true
autorestart=true
stdout_logfile=/home/frappe/frappe-bench/logs/worker.log
stderr_logfile=/home/frappe/frappe-bench/logs/worker.error.log
user=frappe
stopwaitsecs=1560
directory=/home/frappe/frappe-bench
killasgroup=true
numprocs=1
process_name=%(program_name)s-%(process_num)d

[program:frappe-bench-frappe-short-worker]
command=/home/frappe/.local/bin/bench worker --queue short
priority=4
autostart=true
autorestart=true
stdout_logfile=/home/frappe/frappe-bench/logs/worker.log
stderr_logfile=/home/frappe/frappe-bench/logs/worker.error.log
user=frappe
stopwaitsecs=360
directory=/home/frappe/frappe-bench
killasgroup=true
numprocs=1
process_name=%(program_name)s-%(process_num)d

[program:frappe-bench-frappe-long-worker]
command=/home/frappe/.local/bin/bench worker --queue long
priority=4
autostart=true
autorestart=true
stdout_logfile=/home/frappe/frappe-bench/logs/worker.log
stderr_logfile=/home/frappe/frappe-bench/logs/worker.error.log
user=frappe
stopwaitsecs=1560
directory=/home/frappe/frappe-bench
killasgroup=true
numprocs=1
process_name=%(program_name)s-%(process_num)d






[program:frappe-bench-redis-cache]
command=/usr/bin/redis-server /home/frappe/frappe-bench/config/redis_cache.conf
priority=1
autostart=true
autorestart=true
stdout_logfile=/home/frappe/frappe-bench/logs/redis-cache.log
stderr_logfile=/home/frappe/frappe-bench/logs/redis-cache.error.log
user=frappe
directory=/home/frappe/frappe-bench/sites

[program:frappe-bench-redis-queue]
command=/usr/bin/redis-server /home/frappe/frappe-bench/config/redis_queue.conf
priority=1
autostart=true
autorestart=true
stdout_logfile=/home/frappe/frappe-bench/logs/redis-queue.log
stderr_logfile=/home/frappe/frappe-bench/logs/redis-queue.error.log
user=frappe
directory=/home/frappe/frappe-bench/sites



[program:frappe-bench-redis-socketio]
command=/usr/bin/redis-server /home/frappe/frappe-bench/config/redis_socketio.conf
priority=1
autostart=true
autorestart=true
stdout_logfile=/home/frappe/frappe-bench/logs/redis-socketio.log
stderr_logfile=/home/frappe/frappe-bench/logs/redis-socketio.error.log
user=frappe
directory=/home/frappe/frappe-bench/sites



[program:frappe-bench-node-socketio]
command=/home/frappe/.nvm/versions/node/v16.15.0/bin/node /home/frappe/frappe-bench/apps/frappe/socketio.js
priority=4
autostart=true
autorestart=true
stdout_logfile=/home/frappe/frappe-bench/logs/node-socketio.log
stderr_logfile=/home/frappe/frappe-bench/logs/node-socketio.error.log
user=frappe
directory=/home/frappe/frappe-bench


[group:frappe-bench-web]
programs=frappe-bench-frappe-web,frappe-bench-node-socketio



[group:frappe-bench-workers]
programs=frappe-bench-frappe-schedule,frappe-bench-frappe-default-worker,frappe-bench-frappe-short-worker,frappe-bench-frappe-long-worker




[group:frappe-bench-redis]
programs=frappe-bench-redis-cache,frappe-bench-redis-queue,frappe-bench-redis-socketio

dj12djdjs · November 29, 2022, 5:13pm

Great! This should match the output of which node
Could you check?

ankush · December 1, 2022, 4:54am

Track here: Socket.io not working on production sites · Issue #19062 · frappe/frappe · GitHub

rahy · December 12, 2022, 1:25pm

Does this has anything to do with the deprecated command for bench setup socketio?