How to upgrade to python 3.10

Adam.Niescierowicz · August 2, 2022, 8:23pm

Python wheel demands g++ so you need to install apt-get install g++

Usman_Ali · August 3, 2022, 8:40am

nginx: [emerg] unknown log format “main” in /etc/nginx/conf.d/frappe-bench.conf:101
nginx: configuration file /etc/nginx/nginx.conf test failed
ERROR:
Traceback (most recent call last):
File “/usr/local/bin/bench”, line 8, in
sys.exit(cli())
File “/usr/local/lib/python3.10/dist-packages/bench/cli.py”, line 127, in cli
bench_command()
File “/usr/lib/python3/dist-packages/click/core.py”, line 1128, in call
return self.main(*args, **kwargs)
File “/usr/lib/python3/dist-packages/click/core.py”, line 1053, in main
rv = self.invoke(ctx)
File “/usr/lib/python3/dist-packages/click/core.py”, line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/usr/lib/python3/dist-packages/click/core.py”, line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/usr/lib/python3/dist-packages/click/core.py”, line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File “/usr/lib/python3/dist-packages/click/core.py”, line 754, in invoke
return __callback(*args, **kwargs)
File “/usr/local/lib/python3.10/dist-packages/bench/commands/setup.py”, line 89, in setup_production
setup_production(user=user, yes=yes)
File “/usr/local/lib/python3.10/dist-packages/bench/config/production_setup.py”, line 87, in setup_production
reload_nginx()
File “/usr/local/lib/python3.10/dist-packages/bench/config/production_setup.py”, line 205, in reload_nginx
exec_cmd(f"sudo {which(‘nginx’)} -t")
File “/usr/local/lib/python3.10/dist-packages/bench/utils/init.py”, line 154, in exec_cmd
raise CommandFailedError
bench.exceptions.CommandFailedError

Nathaniel_B · August 3, 2022, 11:32pm

Same error here!

smino · August 4, 2022, 1:08am

Hi,

What does sudo nginx -t return?

nomi-gee · August 4, 2022, 2:56pm

It’s better to upgrade to Ubuntu 22.04 as a python upgrade sometime creates more issues with pip and the default python version.

pradpreeonline · August 10, 2022, 1:08am

Hi,

I am getting the below error running this step

env/bin/pip install -e apps/frappe -e apps/erpnext

pxp@erp2-server:~/frappe-bench$ env/bin/pip3.10 install -e apps/frappe/ -e apps/erpnext
Traceback (most recent call last):
File “/home/pxp/frappe-bench/env/bin/pip3.10”, line 8, in
sys.exit(main())
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pip/_internal/cli/main.py”, line 73, in main
command = create_command(cmd_name, isolated=(“–isolated” in cmd_args))
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pip/_internal/commands/init.py”, line 96, in create_command
module = importlib.import_module(module_path)
File “/usr/lib/python3.10/importlib/init.py”, line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File “”, line 1050, in _gcd_import
File “”, line 1027, in _find_and_load
File “”, line 1006, in _find_and_load_unlocked
File “”, line 688, in _load_unlocked
File “”, line 883, in exec_module
File “”, line 241, in _call_with_frames_removed
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pip/_internal/commands/install.py”, line 24, in
from pip._internal.cli.req_command import RequirementCommand
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pip/_internal/cli/req_command.py”, line 15, in
from pip._internal.index.package_finder import PackageFinder
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pip/_internal/index/package_finder.py”, line 21, in
from pip._internal.index.collector import parse_links
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pip/_internal/index/collector.py”, line 12, in
from pip._vendor import html5lib, requests
ImportError: cannot import name ‘html5lib’ from ‘pip._vendor’ (/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pip/_vendor/init.py)

pxp@erp2-server:~/frappe-bench$ pip3 --version
pip 20.0.2 from /usr/lib/python3/dist-packages/pip (python 3.8)
pxp@erp2-server:~/frappe-bench$ env/bin/
easy_install pip pip3.10 python3.10 wheel-3.10
easy_install3 pip3 python wheel
easy_install-3.10 pip-3.10 python3 wheel3
pxp@erp2-server:~/frappe-bench$ env/bin/

This step was successful though:

virtualenv --python python3.10 env

created virtual environment CPython3.10.6.final.0-64 in 466ms
creator CPython3Posix(dest=/home/pradeep/frappe-bench/env, clear=False, global=False)
seeder FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, pkg_resources=latest, via=copy, app_data_dir=/home/pxp/.local/share/virtualenv/seed-app-data/v1.0.1.debian.1)
activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator

I think it might be to do with the pip version still showing 20.0.2 even after the lastest version was installed. Could that be the problem?

pxp@erp2-server:~/frappe-bench$ sudo -H pip3 install --upgrade pip
Collecting pip
Downloading pip-22.2.2-py3-none-any.whl (2.0 MB)
|████████████████████████████████| 2.0 MB 1.3 MB/s
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 20.0.2
Not uninstalling pip at /usr/lib/python3/dist-packages, outside environment /usr
Can’t uninstall ‘pip’. No files were found to uninstall.
Successfully installed pip-22.2.2
pxp@erp2-server:~/frappe-bench$ pip --version
pip 20.0.2 from /usr/lib/python3/dist-packages/pip (python 3.8)

smino · August 10, 2022, 3:44am

Hi ,

I don’t recall encountering that error. I do see that I upgraded pip via: sudo pip3 install pip --upgrade

pip3 -V reports:

pip 22.2.2 from /usr/local/lib/python3.10/dist-packages/pip (python 3.10)

rmeyer · August 10, 2022, 6:17am

Take care which pip you’re using. pip3 is your system’s pip, env/bin/pip is your bench’s pip. They’re entirely different. You can upgrade the latter by running env/bin/pip install -U pip.

trustedcomputer · August 10, 2022, 11:13am

Thanks for this guide!

After installing python3.10 with add-apt-repository and apt install, I was able to migrate the env with this bench command: bench migrate-env python3.10

Then I continued with upgrading pip and re-installing the apps.

pradpreeonline · August 10, 2022, 2:17pm

I got this error when I tried to upgrade bench’s pip.

pxp@erp2-server:~/frappe-bench$ env/bin/pip install -U pip
Traceback (most recent call last):
File “/home/pxp/frappe-bench/env/bin/pip”, line 8, in
sys.exit(main())
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pip/_internal/cli/main.py”, line 73, in main
command = create_command(cmd_name, isolated=(“–isolated” in cmd_args))
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pip/_internal/commands/init.py”, line 96, in create_command
module = importlib.import_module(module_path)
File “/usr/lib/python3.10/importlib/init.py”, line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File “”, line 1050, in _gcd_import
File “”, line 1027, in _find_and_load
File “”, line 1006, in _find_and_load_unlocked
File “”, line 688, in _load_unlocked
File “”, line 883, in exec_module
File “”, line 241, in _call_with_frames_removed
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pip/_internal/commands/install.py”, line 24, in
from pip._internal.cli.req_command import RequirementCommand
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pip/_internal/cli/req_command.py”, line 15, in
from pip._internal.index.package_finder import PackageFinder
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pip/_internal/index/package_finder.py”, line 21, in
from pip._internal.index.collector import parse_links
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pip/_internal/index/collector.py”, line 12, in
from pip._vendor import html5lib, requests
ImportError: cannot import name ‘html5lib’ from ‘pip._vendor’ (/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pip/_vendor/init.py)
pxp@erp2-server:~/frappe-bench$

I then ran this after doing a little research but the pip version still seems to be not updated:

pxp@erp2-server:~/frappe-bench$ curl -sS https://bootstrap.pypa.io/get-pip.py | python3.10
Defaulting to user installation because normal site-packages is not writeable
Collecting pip
Using cached pip-22.2.2-py3-none-any.whl (2.0 MB)
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 22.2.2
Uninstalling pip-22.2.2:
Successfully uninstalled pip-22.2.2
WARNING: The scripts pip, pip3 and pip3.10 are installed in ‘/home/pxp/.local/bin’ which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed pip-22.2.2

pxp@erp2-server:~/frappe-bench$ pip -V
pip 20.0.2 from /usr/lib/python3/dist-packages/pip (python 3.8)

pradpreeonline · August 12, 2022, 10:13pm

I found another way to upgrade version 13 to version 14 by installing version 13 on ubuntu 22.04 and then restoring the data from version 13 (installed originally on ubuntu 20.04). After the data restore was done I then followed the steps from this URL:

Erpnext upgraded from version 13 to version 14 with a few errors and from the size of the databackup file and the error I see while using the application that I cannot trust the data on the upgraded system.
Has anybody successfully upgraded ERPNext version 13 to 14 with a trustable data? If so, can you please share the instructions? Although version 14 was launched due to the issue in upgrade I am unable to implement and use version 14.

AshCoft · August 28, 2022, 4:33am

If you have some problem just follow this link https://cloudbytes.dev/snippets/upgrade-python-to-latest-version-on-ubuntu-linux

Sagar_Patil · November 11, 2022, 9:09am

rmeyer:

Recently, Frappe updated its dependencies for version-14 to python > 3.10.

If you’re on version-13, try to upgrade to python3.8 instead.

Here’s what I did to upgrade to python3.10 under Ubuntu 20.04.

Remember to take a backup of your server first!

Install custom PPA, so Ubuntu knows where to find python3.10:
sudo apt install software-properties-common -y
sudo add-apt-repository ppa:deadsnakes/ppa
Install required python packages:
sudo apt install python3.10 python3.10-dev python3.10-distutils
Switch to your bench directory:
cd /home/frappe/frappe-bench
Move your old python env folder to env-old. This way you make space for the new env while keeping a copy to restore in case something goes wrong.
mv env env-old
Create a new python environment with python3.8
virtualenv --python python3.10 env
In case the above command doesn’t work (I had problems on one machine) you can try this alternative:
# OPTIONAL: only if the above failed
python3.10 -m venv env
Upgrade pip to the newest version. The later commands will fail in case your pip version is too old.
env/bin/pip install -U pip
Now we can reinstall our apps and their dependencies. Here I only show frappe and erpnext. Please append all your custom apps to the command like this: -e apps/my-app
env/bin/pip install -e apps/frappe -e apps/erpnext
I like to run bench update at the end to see if everything worked:
bench update --reset
If everything went well, you can delete your old env (optional, use with care):
rm -rf env-old
Didn’t test this properly yet, but if you get errors from socketio in your browser console, this can possibly be fixed by:
bench setup supervisor
bench restart --supervisor

Installing 11 applications...
Installing frappe
$ /home/sagar_p/frappe-bench/env/bin/python -m pip install --quiet --upgrade -e /home/sagar_p/frappe-bench/apps/frappe 
$ yarn install
yarn install v1.22.17
warning ../../package.json: No license field
warning package-lock.json found. Your project contains lock files generated by tools other than Yarn. It is advised not to mix package managers in order to avoid resolution inconsistencies caused by unsynchronized lock files. To clear this warning, remove package-lock.json.
[1/5] Validating package.json...
[2/5] Resolving packages...
success Already up-to-date.
Done in 0.65s.
Installing air_datepicker
$ /home/sagar_p/frappe-bench/env/bin/python -m pip install --quiet --upgrade -e /home/sagar_p/frappe-bench/apps/air_datepicker 
ERROR: 'App' object has no attribute 'tag'
Traceback (most recent call last):
  File "/usr/local/bin/bench", line 8, in <module>
    sys.exit(cli())
  File "/home/sagar_p/.local/lib/python3.10/site-packages/bench/cli.py", line 127, in cli
    bench_command()
  File "/usr/lib/python3/dist-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3/dist-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python3/dist-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3/dist-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/sagar_p/.local/lib/python3.10/site-packages/bench/commands/update.py", line 59, in update
    update(
  File "/home/sagar_p/.local/lib/python3.10/site-packages/bench/utils/bench.py", line 425, in update
    bench.setup.requirements()
  File "/home/sagar_p/.local/lib/python3.10/site-packages/bench/utils/render.py", line 105, in wrapper_fn
    return fn(*args, **kwargs)
  File "/home/sagar_p/.local/lib/python3.10/site-packages/bench/bench.py", line 449, in requirements
    app = App(path_to_app, bench=self.bench, to_clone=False).install(
  File "/home/sagar_p/.local/lib/python3.10/site-packages/bench/utils/render.py", line 126, in wrapper_fn
    return fn(*args, **kwargs)
  File "/home/sagar_p/.local/lib/python3.10/site-packages/bench/app.py", line 244, in install
    tag=self.tag,
AttributeError: 'App' object has no attribute 'tag'
[quote="rmeyer, post:1, topic:92222, full:true"]
Recently, Frappe [updated its dependencies](https://discuss.frappe.io/t/dropping-python-3-9-support-for-next-version-of-frappe-erpnext/84803?u=rmeyer) for `version-14` to python > 3.10. 

> If you're on `version-13`, try to [upgrade to python3.8](https://discuss.frappe.io/t/how-to-upgrade-erpnext-from-python-3-5-to-3-8/68587/13?u=rmeyer) instead.

Here's what I did to upgrade to python3.10 under Ubuntu 20.04.

> :bangbang: Remember to take a backup of your server first! :bangbang:

Install custom PPA, so Ubuntu knows where to find python3.10:

```bash
sudo apt install software-properties-common -y
sudo add-apt-repository ppa:deadsnakes/ppa

Install required python packages:

sudo apt install python3.10 python3.10-dev python3.10-distutils

Switch to your bench directory:

cd /home/frappe/frappe-bench

Move your old python env folder to env-old. This way you make space for the new env while keeping a copy to restore in case something goes wrong.

mv env env-old

Create a new python environment with python3.8

virtualenv --python python3.10 env

In case the above command doesn’t work (I had problems on one machine) you can try this alternative:

# OPTIONAL: only if the above failed
python3.10 -m venv env

Upgrade pip to the newest version. The later commands will fail in case your pip version is too old.

env/bin/pip install -U pip

Now we can reinstall our apps and their dependencies. Here I only show frappe and erpnext. Please append all your custom apps to the command like this: -e apps/my-app

env/bin/pip install -e apps/frappe -e apps/erpnext

I like to run bench update at the end to see if everything worked:

bench update --reset

If everything went well, you can delete your old env (optional, use with care):

rm -rf env-old

Didn’t test this properly yet, but if you get errors from socketio in your browser console, this can possibly be fixed by:

bench setup supervisor
bench restart --supervisor

[/quote]
after => bench update --reset

ankush · November 29, 2022, 7:12am

If you have 3.x and 3.10 installed you can use bench migrate-env path-to-python-3.10 to migrate in one step.

Just fixed some issues I faced while migrating (didnt know this functionality was broken, cause I always migrate manually )

ref: fix: migrate env by ankush · Pull Request #1396 · frappe/bench · GitHub

charleslcso · June 26, 2023, 4:28am

Hi @ankush i am updating version-13 v49 to the latest version manually.

git pull —rebase works fine.

However, after reading tons of posts I am still not comfortable with it. The steps needed to perform is still not clear. And you know bad things will happen if we are not clear.

Since “bench update” will perform a number of tasks such as patching, updating database schema and etc, a manual PULL should update the codebase only.

What steps do I have to do after doing a PULL? And in what sequence?

PULL
bench update --patch
bench migrate

?

rmeyer · June 26, 2023, 10:32am

@charleslcso this thread is about updating the python version frappe runs on. Manually updating to the latest release is a slightly different topic. If it’s a minor upgrade, you can just do things in this order:

git pull (Pull the latest changes)
bench setup requirements (Install any new dependencies)
bench build (Build changed JS and CSS files)
bench migrate (Migrate database schema to the new version)
bench restart && bench clear-cache (Restart and clear cache to use new code)

For a major update, other things might need to change as well, for example the NodeJS, Python and MariaDB versions used.

charleslcso · June 26, 2023, 2:45pm

Thank you @rmeyer . Me bad. Highjacking this thread. Hope it helps someone.

matthkarl · June 4, 2024, 2:59pm

@pradpreeonline Did you find a solution to that pip upgrade error? I have exactly the same problem now, being stuck on this step.

aashishvashisht6 · July 9, 2024, 7:39pm

@matthkarl you can refer this documentation for pip or python upgrade