Update to Ubuntu 22.04.3 Jammy - Crashed the ERPNEXT site

smuzzamil · November 4, 2023, 6:05am

Hello, I recently updated my Ubuntu OS to jammy, after the Update the Site Gives the following error:

"Sorry, We will be back soon! Don’t panic. It’s not you, it’s us.

Most likely, our engineers are updating the code, and it should take a minute for the new code to load into memory."

Running bench status gives me the following error:


/home/frappe/frappe-bench/env/bin/python: Error while finding module specification for 'frappe.utils.bench_helper' (ModuleNotFoundError: No module named 'frappe')

/home/frappe/frappe-bench/env/bin/python: Error while finding module specification for 'frappe.utils.bench_helper' (ModuleNotFoundError: No module named 'frappe')

I have already tried restarting bench, apt-get update && apt-get upgrade … Kindly tell me how to fix this.

If you are going to propose reinstallation, please tell me the steps to backup everything properly as I had some custom app & print format work done earlier.

Thank you for your time & help in advance.

smuzzamil · November 4, 2023, 6:21am

bench restart

frappe@stt-desk:~/frappe-bench$ bench restart
$ sudo supervisorctl restart frappe-bench-workers: frappe-bench-web:
frappe-bench-workers:frappe-bench-frappe-schedule: stopped
frappe-bench-workers:frappe-bench-frappe-default-worker-0: stopped
frappe-bench-workers:frappe-bench-frappe-long-worker-0: stopped
frappe-bench-workers:frappe-bench-frappe-short-worker-0: stopped
frappe-bench-web:frappe-bench-node-socketio: stopped
frappe-bench-workers:frappe-bench-frappe-schedule: started
frappe-bench-workers:frappe-bench-frappe-default-worker-0: started
frappe-bench-workers:frappe-bench-frappe-short-worker-0: started
frappe-bench-workers:frappe-bench-frappe-long-worker-0: started
frappe-bench-web:frappe-bench-frappe-web: ERROR (spawn error)
frappe-bench-web:frappe-bench-node-socketio: started
ERROR: sudo supervisorctl restart frappe-bench-workers: frappe-bench-web:
subprocess.CalledProcessError: Command 'sudo supervisorctl restart frappe-bench-workers: frappe-bench-web:' returned non-zero exit status 7.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/bin/bench", line 8, in <module>
    sys.exit(cli())
  File "/home/frappe/.local/lib/python3.10/site-packages/bench/cli.py", line 132, in cli
    bench_command()
  File "/home/frappe/.local/lib/python3.10/site-packages/bench/commands/utils.py", line 41, in restart
    Bench(".").reload(web, supervisor, systemd)
  File "/home/frappe/.local/lib/python3.10/site-packages/bench/utils/render.py", line 126, in wrapper_fn
    return fn(*args, **kwargs)
  File "/home/frappe/.local/lib/python3.10/site-packages/bench/bench.py", line 152, in reload
    restart_supervisor_processes(bench_path=self.name, web_workers=web, _raise=_raise)
  File "/home/frappe/.local/lib/python3.10/site-packages/bench/utils/bench.py", line 321, in restart_supervisor_processes
    failure = bench.run(f"{sudo}supervisorctl restart {group}", _raise=_raise)
  File "/home/frappe/.local/lib/python3.10/site-packages/bench/bench.py", line 48, in run
    return exec_cmd(cmd, cwd=cwd or self.cwd, _raise=_raise)
  File "/home/frappe/.local/lib/python3.10/site-packages/bench/utils/__init__.py", line 158, in exec_cmd
    raise CommandFailedError(cmd) from subprocess.CalledProcessError(return_code, cmd)
bench.exceptions.CommandFailedError: sudo supervisorctl restart frappe-bench-workers: frappe-bench-web:

INFO: A newer version of bench is available: 5.18.0 → 5.19.0

bench update

frappe@stt-desk:~/frappe-bench$ bench update
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
From https://github.com/frappe/frappe
 * branch            version-13 -> FETCH_HEAD
WARN: shallow_clone is set in your bench config.
However without passing the --reset flag, your repositories will be unshallowed.
To avoid this, cancel this operation and run `bench update --reset`.

Consider the consequences of `git reset --hard` on your apps before you run that.
To avoid seeing this warning, set shallow_clone to false in your common_site_config.json
		
Backing up sites...                     
/home/frappe/frappe-bench/env/bin/python: Error while finding module specification for 'frappe.utils.bench_helper' (ModuleNotFoundError: No module named 'frappe')
INFO: A newer version of bench is available: 5.18.0 → 5.19.0

wale · November 4, 2023, 8:42am

Hi @smuzzamil

What version of Frappe / ERPNext are you on ?

If you weren’t already on V14 / Python 3.10 then you’ll likely need to update your virtual environment

Cheers

smuzzamil · November 4, 2023, 4:34pm

thank you for your response. I was already on v14 and python 3.10. However, I did try to install python3.11 and latest nvm

wale · November 6, 2023, 3:44pm

Hi @smuzzamil

Just seeing your response. Hopefully you solved this by now but just in case…

https://frappeframework.com/docs/user/en/bench/reference/backup

https://frappeframework.com/docs/user/en/bench/reference/restore

To fix the error, you could try setting up a new virtual environment…

Move your old python env folder to env-old (just a backup so you can return to the original if any issue)

cd /home/myuser/frappe-bench/

mv env env-old

Create new virtual env for python-3.10

pip install virtualenv

virtualenv --python python3.10 env

env/bin/pip install -U pip

Install Frappe/ERPNext apps into your new env

env/bin/pip install -e apps/frappe -e apps/erpnext

pip3 install frappe-bench

Update Bench

bench update

Kind regards,

smuzzamil · November 6, 2023, 6:28pm

thank you for your response. I am still trying to figure it out.

I get two issues while trying your suggested solution:

backing up gives me this error:

Database or site_config.json corrupted

bench update gives me this error:

frappe@stt-desk:~$ bench update
WARN: Command not being executed in bench directory
ERROR: No module named 'bench.patches.v3'
Traceback (most recent call last):
  File "/home/frappe/.local/bin/bench", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/home/frappe/.local/lib/python3.11/site-packages/bench/cli.py", line 132, in cli
    bench_command()
  File "/home/frappe/.local/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frappe/.local/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/frappe/.local/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frappe/.local/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frappe/.local/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frappe/.local/lib/python3.11/site-packages/bench/commands/update.py", line 59, in update
    update(
  File "/home/frappe/.local/lib/python3.11/site-packages/bench/utils/bench.py", line 414, in update
    patches.run(bench_path=bench_path)
  File "/home/frappe/.local/lib/python3.11/site-packages/bench/patches/__init__.py", line 26, in run
    module = importlib.import_module(patch.split()[0])
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1126, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1140, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'bench.patches.v3'

wale · November 8, 2023, 9:30am

You need to investigate this. What’s in your site_config.json ? Also do a search on the forum for this error. A few other people encountered it previously and one of their solutions might work for you

You didn’t run the command from your bench directory

Cheers

smuzzamil · November 13, 2023, 1:58pm

appreciate your help wale. I ended up reinstalling the whole server using old backup, as nothing else was working, I tried several solutions I found online, but kept getting errors.

I feel it might have to do something with the latest python3.11 (probably some misconfiguration from my end)

Thank you again for your response!

system · May 11, 2024, 1:59pm

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.