Upgrade Error going from v8 to v9 ERPNext

The past 2 major version changes have not worked at all with the production systems I had in place. This does not say much for the future of ERPNext. :disappointed:

BKM

When an issue like this occasionally happens to me it normally involves 1 or 2 files not 2000. I can only guess there is something up with your server. Maybe a DateTime issue? (Check your NTP settings). Always a good idea to update server OS and then do a soft reboot not a hard power off. Maybe server permissions? (sudo chown and chmod on git folders). It could be corruption of your github control files? If what I suggested in an earlier post doesn’t work (which it should do) you can

cd home/frappe/frappe-bench/apps/erpnext
git reset
git checkout .
git clean -fdx
cd home/frappe/frappe-bench/apps/frappe
git reset
git checkout .
git clean -fdx
cd home/frappe/frappe-bench/
bench update

This is effectively the same as deleting all your local git files and doing a fresh git clone.
In short you can delete all of your local git files (and then replace them) without damaging your DB in anyway.
Good Luck.

Is the error because now they have removed --upgrade option?

For what it is worth, I have seen this issue couple of times in the past. As others have noted, git status/diff listed all the files as updated due to timestamp changes. All my virtual machines were created in the same way, so I don’t think it was due to server setup issue. Since it is going to be difficult to find the root cause of this issue, I think we should make ‘bench update --reset’ work in those cases.

I agree this should work, @bkm did you try bench update --reset (it basically calls git reset)

Ok, I think I might have a clue how it got to this point. Just not sure how to get it fixed. I spent the past two days going over every step of the process I used to create the VM instances in the Google Cloud Platform. I didn’t “initially” see anything that could have caused the issue. Then, when I tried to run “bench update --reset” in the /home/frappe/frappe-bench/apps/frappe directory, I kept getting permission denied. The system also would not let me run that command using “sudo”. It simply stated the command should not be run as root, then exited back to the command line.

Going back through my notes on the initialization of the VM instances I think I see where this all went bad.

  • It appears that ERPNext / Frappe does not like to have a system where “frappe” is not a valid user.
  • When running on Google Cloud Platform, any Debian 8 servers you create are done so using your Google credentials as the login account.
  • Wanting to keep the system from prying eyes that might seek out the default frappe user account to cause trouble, I intentionally used the default Google account to run the git command to fetch the install.py to the default home directory.
  • I then also used the default account to run the install.py script and execute all of the install steps with “sudo python install.py --production”
  • When finished, there is a good working ERPNext system on the Google Cloud Platform.
  • In the case of the last install, I also had to restore a previous database.
  • I believe this is where everything starts to go off the rails. Because there was no valid frappe user account, I had to use “sudo” to run the bench restore command from the default GCP account. I believe this altered the user and owner status of all the files involved in the restore.
  • As long as I was still running a version 8 ERPNext system, nothing was wrong. Even the bench --site #### migrate command worked just fine because it was also necessary to run it as “sudo” since there was no frappe account login credentials.
  • At this point I believe a have a system that cannot be upgraded, because it was created without a valid frappe user account. Therefore I cannot run anything in the bench unless I start the command with “sudo”, and sudo is not allowed for the bench update --reset command.
  • So I believe I have created a sort of Catch 22.

Now this leads me to ask the question… Why is it that a system can be installed without using a valid frappe user account if it might cause so much trouble later?

Better question… Why is it not covered in the instructions we all have to use here:

  • Does anyone have an idea on how to resolve this revolving door error condition I seem to have created (if I am correct) all the way back at the initial install?
  • Will someone recommend a way to use the install.py script that would allow the use of an account other than the frappe user? With ERPNext and Frappe becoming so popular over the past year or two, I would really like to use an alternate account name and still have everything work properly.
  • Or is there a way to create the frappe user on the back end (as in my case now)? You never really have admin control of the GCP Debian server you create except by maybe the sudo command, but there is not “root” login.

I am open to ideas. Is my testing and analysis correct, or am I heading down another rabbit hole?

Thanks in advance.

BKM

python install.py --user {username}

For more information

python install.py --help

@Pawan

Ahh… Thank you. I did scan right past that because it had the “–develop” switch included and dropped straight to the bottom for the only entry that had the “–production” switch. That is my glaring error in oversight. I remember reading it and thinking that I am not logged in as “root” when on GCP but as a regular user with automatic credentials that I cannot change and that I am doing a production install. However, thinking it through a bit further I guess the only way to install on a GCP server is to force it to install into the default account. At least that way you have privileges enough to run bench without using “sudo”. :disappointed_relieved:


The entire entry looks like this:

  1. Run the install script

If you are on a fresh server and logged in as root, use --user flag to create a user and install using that user

python install.py --develop --user frappe

For developer setup:

sudo python install.py --develop

For production:

sudo python install.py --production


Thanks again. I will use this on my next round of install.

Unfortunately, it does not appear anyone has a possible exit for me in this current revolving door error. So I may be forced into setting up a new VM instance and running export and import to get the data moved. I cannot seem to create credentials for the frappe account in the Debian server while running on GCP. All of the commands I found through google searches have failed so far. And, if you try to use logout while in the SSH window of GCP, it closes the session. So no real way to change the user logged into the Debian server.

BKM

I had the same issues due to uncommitted changes in “frappe”. If you go to
/home/frappe/frappe-bench/apps/frappe and run “git status” it will tell you what the issue is. In my case, I was tweaking an issue with my Stripe integration that I hadn’t finished. You can run “git reset --hard” which will blow away those changes if you don’t care. Otherwise, you need to address the issues there and commit.

After that, I was able to run “bench update” from frappe-bench successfully without issue.

try : bench update --upgrade --reset

worked for me

regards

When I ran sudo bench update --reset, it appeared to work and I answered the supervisor question near the end as No, and it aborted as described in other posts.

When I then logged into the system again, I checked the versioning as reported in the Help drop down box, it still reports being on version erpnext of 8.11.2 and frappe version of 8.10.5

When I first tried his command it was with the --upgrade --reset switches and it never would work. When I went back and did it with only the --reset switch, it got this far but still no upgrade to v9.

It took quite a bit of time to do the update, but still did not seem to work.

BKM

Hello ,
How to find out what are the local changes interfering with the upgrade?
Git command ?

i have made a few minor changes in the code from time to time in my local because at the time it seemed like a waste to create a app for small changes.
I then created a separate branch on git and uploaded the changes there and then pulled the code of that branch on my live server(might have screwed it up here).

My about looks like this:
image
and angler is my branch

How can i find out what changes are there from master branch(my guess is that the upgrade is on master) so that i can back them up and create an app for them later or something ( dont want to loose tried and tested code).?

I had a similar issue today while upgrading a production-like setup with no changes. I checked all the apps, and none of them had any changed files. I tried ‘bench update --reset’ which also failed with similar error message. After searching for a while, I realised it could be due to .bench changes. The permissions were indeed updated in ~/.bench/ folder which I have no clue as why. The upgrade went fine just two days before and only change I did later was creating a new site. Not sure why that would mess up ~/.bench file permissions. In any case, I can confirm ‘–reset’ doesn’t work for changed files in ~/.bench/ folder.

The permissions changed to what? that might give us a clue.

Ok, i got it! completely forgot about github compare feature!

question- does custom script reports prevent upgrade?

My bad, I should have taken a screen shot of the permission changes. I will do that if I see the issue again.

I hit this issue again and managed to get some info. However not really sure it is of much use.

frappe@osboxes:~/frappe-bench$ bench update
INFO:bench.utils:updating bench
INFO:bench.utils:git pull
remote: Counting objects: 329, done.
remote: Compressing objects: 100% (149/149), done.
Receiving objects: 100% (329/329), 43.07 KiB | 1024 bytes/s, done.
remote: Total 329 (delta 170), reused 288 (delta 140), pack-reused 0
Resolving deltas: 100% (170/170), completed with 29 local objects.
From GitHub - frappe/bench: CLI to manage Multi-tenant deployments for Frappe apps
7a7c611…c97be6a master → origin/master
Updating 7a7c611…c97be6a
error: Your local changes to the following files would be overwritten by merge:
docs/releasing_frappe_erpext.md
playbooks/develop/includes/mariadb_centos.yml
playbooks/develop/includes/mariadb_debian.yml
playbooks/develop/includes/mariadb_ubuntu.yml
playbooks/develop/includes/setup_mariadb.yml
playbooks/develop/includes/wkhtmltopdf.yml
playbooks/develop/install.yml
playbooks/develop/macosx.yml
playbooks/develop/templates/simple_mariadb_config.cnf
playbooks/develop/ubuntu.yml
playbooks/production/includes/setup_bench_production.yml
playbooks/production/install.yml
playbooks/production/templates/default_nginx.j2
playbooks/production/templates/frappe_selinux.te
vm/ansible/templates/mariadb_config.cnf
Please commit your changes or stash them before you merge.
Aborting
Traceback (most recent call last):
File “/usr/local/bin/bench”, line 11, in
load_entry_point(‘bench’, ‘console_scripts’, ‘bench’)()
File “/home/frappe/.bench/bench/cli.py”, line 40, in cli
bench_command()
File “/usr/local/lib/python2.7/dist-packages/click/core.py”, line 722, in call
return self.main(*args, **kwargs)
File “/usr/local/lib/python2.7/dist-packages/click/core.py”, line 697, in main
rv = self.invoke(ctx)
File “/usr/local/lib/python2.7/dist-packages/click/core.py”, line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/usr/local/lib/python2.7/dist-packages/click/core.py”, line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File “/usr/local/lib/python2.7/dist-packages/click/core.py”, line 535, in invoke
return callback(*args, **kwargs)
File “/home/frappe/.bench/bench/commands/update.py”, line 35, in update
update_bench()
File “/home/frappe/.bench/bench/utils.py”, line 256, in update_bench
exec_cmd(“git pull”, cwd=cwd)
File “/home/frappe/.bench/bench/utils.py”, line 140, in exec_cmd
raise CommandFailedError(cmd)
bench.utils.CommandFailedError: git pull
frappe@osboxes:~/frappe-bench$
frappe@osboxes:~/frappe-bench$ ls -lrth ~/.bench/playbooks/
total 20K
-rwxr-xr-x 1 frappe frappe 12K Sep 3 10:03 install.py
-rwxr-xr-x 1 frappe frappe 0 Sep 3 10:03 init.py
drwxr-xr-x 4 frappe frappe 4.0K Sep 3 10:03 develop
drwxr-xr-x 4 frappe frappe 4.0K Sep 3 10:03 production
frappe@osboxes:~/frappe-bench$ ls -lrth ~/.bench/playbooks/production/
total 20K
-rwxr-xr-x 1 frappe frappe 427 Sep 3 10:03 install.yml
drwxr-xr-x 2 frappe frappe 4.0K Sep 3 10:03 includes
-rwxr-xr-x 1 frappe frappe 331 Sep 3 10:03 change_ssh_port.yml
drwxr-xr-x 2 frappe frappe 4.0K Sep 3 10:03 templates
-rwxr-xr-x 1 frappe frappe 1.2K Sep 3 10:03 setup_firewall.yml
frappe@osboxes:~/frappe-bench$ cd ~/.bench/
frappe@osboxes:~/.bench$ git status
On branch master
Your branch is behind ‘origin/master’ by 31 commits, and can be fast-forwarded.
(use “git pull” to update your local branch)
Changes not staged for commit:
(use “git add …” to update what will be committed)
(use “git checkout – …” to discard changes in working directory)

modified:   .gitignore
modified:   .travis.yml
modified:   LICENSE.md
modified:   MANIFEST.in
modified:   bench/__init__.py
modified:   bench/commands/config.py
modified:   bench/commands/git.py
modified:   bench/commands/utils.py
modified:   bench/config/__init__.py
modified:   bench/config/common_site_config.py
modified:   bench/config/nginx.py
modified:   bench/config/redis.py
modified:   bench/config/site_config.py
modified:   bench/config/supervisor.py
modified:   bench/config/templates/502.html
modified:   bench/config/templates/Procfile
modified:   bench/config/templates/frappe_sudoers
modified:   bench/config/templates/nginx.conf
modified:   bench/config/templates/nginx_default.conf
modified:   bench/config/templates/redis_cache.conf
modified:   bench/config/templates/redis_queue.conf
modified:   bench/config/templates/redis_socketio.conf
modified:   bench/config/templates/supervisor.conf
modified:   bench/package.json
modified:   bench/patches/__init__.py
modified:   bench/patches/patches.txt
modified:   bench/patches/v3/__init__.py
modified:   bench/patches/v3/celery_to_rq.py
modified:   bench/patches/v3/deprecate_old_config.py
modified:   bench/patches/v3/redis_bind_ip.py
modified:   bench/patches/v4/__init__.py
modified:   bench/patches/v4/update_node.py
modified:   bench/patches/v4/update_socketio.py
modified:   bench/tests/__init__.py
modified:   bench/tests/test_setup_production.py
modified:   completion.sh
modified:   docs/branch_details.md
modified:   docs/contribution_guidelines.md
modified:   docs/release_policy.md
modified:   docs/releasing_frappe_erpext.md
modified:   install_scripts/contrib-apps.json
modified:   install_scripts/erpnext-apps-master.json
modified:   install_scripts/erpnext-apps.json
modified:   playbooks/__init__.py
modified:   playbooks/develop/includes/mariadb_centos.yml
modified:   playbooks/develop/includes/mariadb_debian.yml
modified:   playbooks/develop/includes/mariadb_ubuntu.yml
modified:   playbooks/develop/includes/setup_bench.yml
modified:   playbooks/develop/includes/setup_dev_env.yml
modified:   playbooks/develop/includes/setup_erpnext.yml
modified:   playbooks/develop/includes/setup_mariadb.yml
modified:   playbooks/develop/includes/setup_selinux_policy.yml
modified:   playbooks/develop/includes/wkhtmltopdf.yml
modified:   playbooks/develop/install.yml
modified:   playbooks/develop/macosx.yml
modified:   playbooks/develop/templates/mariadb_centos.repo
modified:   playbooks/develop/templates/simple_mariadb_config.cnf
modified:   playbooks/develop/ubuntu.yml
modified:   playbooks/production/includes/setup_bench_production.yml
modified:   playbooks/production/install.yml
modified:   playbooks/production/templates/default_nginx.j2
modified:   playbooks/production/templates/frappe_selinux.te
modified:   requirements.txt
modified:   setup.py
modified:   vm/Readme.md
modified:   vm/ansible/base.yml
modified:   vm/ansible/library/bench.py
modified:   vm/ansible/roles/mariadb/README.md
modified:   vm/ansible/roles/mariadb/defaults/main.yml
modified:   vm/ansible/roles/mariadb/handlers/main.yml
modified:   vm/ansible/roles/mariadb/meta/main.yml
modified:   vm/ansible/roles/mariadb/tasks/centos.yml
modified:   vm/ansible/roles/mariadb/tasks/main.yml
modified:   vm/ansible/roles/mariadb/tasks/mysql_secure_installation.yml
modified:   vm/ansible/roles/mariadb/tasks/ubuntu.yml
modified:   vm/ansible/roles/mariadb/templates/mariadb_centos.repo.j2
modified:   vm/ansible/roles/mariadb/templates/mariadb_ubuntu.list.j2
modified:   vm/ansible/roles/mariadb/templates/my.cnf.j2
modified:   vm/ansible/roles/mariadb/vars/main.yml
modified:   vm/ansible/roles/nginx/.travis.yml
modified:   vm/ansible/roles/nginx/README.md
modified:   vm/ansible/roles/nginx/defaults/main.yml
modified:   vm/ansible/roles/nginx/handlers/main.yml
modified:   vm/ansible/roles/nginx/meta/main.yml
modified:   vm/ansible/roles/nginx/tasks/main.yml
modified:   vm/ansible/roles/nginx/tasks/setup-Debian.yml
modified:   vm/ansible/roles/nginx/tasks/setup-RedHat.yml
modified:   vm/ansible/roles/nginx/tasks/vhosts.yml
modified:   vm/ansible/roles/nginx/templates/nginx.conf.j2
modified:   vm/ansible/roles/nginx/templates/nginx.repo.j2
modified:   vm/ansible/roles/nginx/templates/vhosts.j2
modified:   vm/ansible/roles/nginx/tests/inventory
modified:   vm/ansible/roles/nginx/tests/test.yml
modified:   vm/ansible/roles/nginx/vars/Debian.yml
modified:   vm/ansible/roles/nginx/vars/RedHat.yml
modified:   vm/ansible/roles/swap/defaults/main.yml
modified:   vm/ansible/roles/swap/tasks/main.yml
modified:   vm/ansible/roles/wkhtmltopdf/defaults/main.yml
modified:   vm/ansible/roles/wkhtmltopdf/tasks/main.yml
modified:   vm/ansible/templates/mariadb_config.cnf
modified:   vm/ansible/templates/nginx.conf
modified:   vm/ansible/vars.yml
modified:   vm/ansible/vm.yml
modified:   vm/ansible/vm_vars/all.yml
modified:   vm/build.py
modified:   vm/scripts/install_ansible.sh
modified:   vm/scripts/set_message.sh
modified:   vm/scripts/zerofree.sh
modified:   vm/vm.json

no changes added to commit (use “git add” and/or “git commit -a”)

frappe@osboxes:~/.bench$ cd ~/frappe-bench/
frappe@osboxes:~/frappe-bench$ bench update --reset
INFO:bench.utils:updating bench
INFO:bench.utils:git pull
Updating 7a7c611…c97be6a
error: Your local changes to the following files would be overwritten by merge:
docs/releasing_frappe_erpext.md
playbooks/develop/includes/mariadb_centos.yml
playbooks/develop/includes/mariadb_debian.yml
playbooks/develop/includes/mariadb_ubuntu.yml
playbooks/develop/includes/setup_mariadb.yml
playbooks/develop/includes/wkhtmltopdf.yml
playbooks/develop/install.yml
playbooks/develop/macosx.yml
playbooks/develop/templates/simple_mariadb_config.cnf
playbooks/develop/ubuntu.yml
playbooks/production/includes/setup_bench_production.yml
playbooks/production/install.yml
playbooks/production/templates/default_nginx.j2
playbooks/production/templates/frappe_selinux.te
vm/ansible/templates/mariadb_config.cnf
Please commit your changes or stash them before you merge.
Aborting
Traceback (most recent call last):
File “/usr/local/bin/bench”, line 11, in
load_entry_point(‘bench’, ‘console_scripts’, ‘bench’)()
File “/home/frappe/.bench/bench/cli.py”, line 40, in cli

Following related issues like git status shows all files as modified · Issue #184 · microsoft/WSL · GitHub, I ran the following command on ~/.bench/ repository
$git config core.filemode false
and ‘bench update’ started proceeding with the update. So my suggestion is to include this command by default on one of the install scripts so that all erpnext repositories ignore the permission changes.

1 Like

Why does Custom script report also prevent update ?
can it be maintained like an app?

should i set it as standard or custom.?..
some reports don’t interfere with anything anyway , just some common columns …
why is it that git pull from app causes no problems but bench update does?