Switching from master to develop, and back to master

paramjitrohits · June 25, 2018, 6:10am

Hi all,
Yesterday, I set up a new ERPNext (Ubuntu 18.04 LTS) instance, and migrated my old database to this new one. As the develop branch was causing me lots of troubles, I switched to the master branch, and was able to run ERPNext perfectly for one time. After rebooting the instance this morning, I can’t seem to login with the console error message:

Traceback (most recent call last):
  File "/home/user/.erpnext/frappe-bench/apps/frappe/frappe/app.py", line 59, in application
    init_request(request)
  File "/home/user/.erpnext/frappe-bench/apps/frappe/frappe/app.py", line 121, in init_request
    frappe.local.http_request = frappe.auth.HTTPRequest()
  File "/home/user/.erpnext/frappe-bench/apps/frappe/frappe/auth.py", line 53, in __init__
    frappe.local.login_manager = LoginManager()
  File "/home/user/.erpnext/frappe-bench/apps/frappe/frappe/auth.py", line 107, in __init__
    if self.login()==False: return
  File "/home/user/.erpnext/frappe-bench/apps/frappe/frappe/auth.py", line 126, in login
    self.authenticate(user=user, pwd=pwd)
  File "/home/user/.erpnext/frappe-bench/apps/frappe/frappe/auth.py", line 206, in authenticate
    self.user = self.check_password(user, pwd)
  File "/home/user/.erpnext/frappe-bench/apps/frappe/frappe/auth.py", line 218, in check_password
    return check_password(user, pwd)
  File "/home/user/.erpnext/frappe-bench/apps/frappe/frappe/utils/password.py", line 35, in check_password
    )""",{ 'doctype': doctype, 'name': user, 'fieldname': fieldname, 'pwd': pwd }, as_dict=True)
  File "/home/user/.erpnext/frappe-bench/apps/frappe/frappe/database.py", line 166, in sql
    self._cursor.execute(query, values)
  File "/home/user/.erpnext/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/cursors.py", line 170, in execute
    result = self._query(query)
  File "/home/user/.erpnext/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/cursors.py", line 328, in _query
    conn.query(q)
  File "/home/user/.erpnext/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/connections.py", line 893, in query
    self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  File "/home/user/.erpnext/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/connections.py", line 1103, in _read_query_result
    result.read()
  File "/home/user/.erpnext/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/connections.py", line 1396, in read
    first_packet = self.connection._read_packet()
  File "/home/user/.erpnext/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/connections.py", line 1059, in _read_packet
    packet.check_error()
  File "/home/user/.erpnext/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/connections.py", line 384, in check_error
    err.raise_mysql_exception(self._data)
  File "/home/user/.erpnext/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/err.py", line 109, in raise_mysql_exception
    raise errorclass(errno, errval)
InternalError: (1054, u"Unknown column 'salt' in 'field list'")

The same error occurs in logs/frappe.log.

I saw some earlier queries regarding the same, one of which suggested to switch to develop branch. I would like to stay away from the develop branch for a while for stability. Another one suggested to run bench setup-admin-password password (with password appropriately changed), but I got the following error:

Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/home/user/.erpnext/frappe-bench/apps/frappe/frappe/utils/bench_helper.py", line 94, in <module>
    main()
  File "/home/user/.erpnext/frappe-bench/apps/frappe/frappe/utils/bench_helper.py", line 18, in main
    click.Group(commands=commands)(prog_name='bench')
  File "/home/user/.erpnext/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/user/.erpnext/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/user/.erpnext/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/user/.erpnext/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/user/.erpnext/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/user/.erpnext/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/user/.erpnext/frappe-bench/env/local/lib/python2.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/user/.erpnext/frappe-bench/apps/frappe/frappe/commands/__init__.py", line 24, in _func
    ret = f(frappe._dict(ctx.obj), *args, **kwargs)
  File "/home/user/.erpnext/frappe-bench/apps/frappe/frappe/commands/site.py", line 423, in set_admin_password
    update_password(user='Administrator', pwd=admin_password, logout_all_sessions=logout_all_sessions)
  File "/home/user/.erpnext/frappe-bench/apps/frappe/frappe/utils/password.py", line 66, in update_password
    { 'doctype': doctype, 'name': user, 'fieldname': fieldname, 'pwd': pwd, 'salt': salt })
  File "/home/user/.erpnext/frappe-bench/apps/frappe/frappe/database.py", line 166, in sql
    self._cursor.execute(query, values)
  File "/home/user/.erpnext/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/cursors.py", line 170, in execute
    result = self._query(query)
  File "/home/user/.erpnext/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/cursors.py", line 328, in _query
    conn.query(q)
  File "/home/user/.erpnext/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/connections.py", line 893, in query
    self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  File "/home/user/.erpnext/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/connections.py", line 1103, in _read_query_result
    result.read()
  File "/home/user/.erpnext/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/connections.py", line 1396, in read
    first_packet = self.connection._read_packet()
  File "/home/user/.erpnext/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/connections.py", line 1059, in _read_packet
    packet.check_error()
  File "/home/user/.erpnext/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/connections.py", line 384, in check_error
    err.raise_mysql_exception(self._data)
  File "/home/user/.erpnext/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/err.py", line 109, in raise_mysql_exception
    raise errorclass(errno, errval)
pymysql.err.InternalError: (1054, u"Unknown column 'salt' in 'field list'")

Can someone please guide me in resolving this? Seems like a bug in the current master branch.

rmehta · June 25, 2018, 7:13am

Try running bench update again. Don’t think this is a standard issue.

And don’t switch to develop if you are not really sure!

paramjitrohits · June 25, 2018, 7:23am

@rmehta
Oh, actually running bench update produces the error:

INFO:bench.utils:updating bench
INFO:bench.utils:git pull
fatal: not a git repository (or any of the parent directories): .git
Traceback (most recent call last):
  File "/usr/local/bin/bench", line 11, in <module>
    load_entry_point('bench==4.1.0', 'console_scripts', 'bench')()
  File "/usr/local/lib/python2.7/dist-packages/bench/cli.py", line 40, in cli
    bench_command()
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/bench/commands/update.py", line 36, in update
    update_bench()
  File "/usr/local/lib/python2.7/dist-packages/bench/utils.py", line 260, in update_bench
    exec_cmd("git pull", cwd=cwd)
  File "/usr/local/lib/python2.7/dist-packages/bench/utils.py", line 159, in exec_cmd
    raise CommandFailedError(cmd)
bench.utils.CommandFailedError: git pull

Its strange, because bench update --patch, bench update --pull work fine. (I installed bench following the guide The Hitchhiker’s Guide to Installing Frappé on Linux OS from GitHub).

Also, I tried switching to develop branch before posting, but got some errors (some fields cannot be null, etc), possibly due to incompatible database schemas. I would prefer to stick to master branch.

codingCoffee · June 25, 2018, 7:29am

develop is far ahead of the master branch, and has a lot more features in it. So while going from develop to master you’re effectively going back in time. One of the changes in develop is the password hashing technique, which is more secure. There are changes in the DB schema related to this which is causing the issue.

Switching back to develop would be your best bet I feel. Also report any issues you face on develop, so we can fix them.

rmehta · June 25, 2018, 7:42am

like @codingCoffee says, you can’t switch back directly,

if you want to switch, restore from the backup and then run bench update again. (see backups folder in your site folder. Unless you did --no-backup, bench will create a backup everytime you do bench update)

paramjitrohits · June 25, 2018, 8:00am

Thanks @codingCoffee for the insightful reply. I had become optimistic, because master seemed to work perfectly for once, albeit before the first reboot. If so is the case, I’m up for switching to the develop branch. Following @rmehta’s suggested outline (thanks a lot), I switched to develop, and then force restored from my old backup, and then ran bench update --patch and it seems to work fine, persistently after reboots. Earlier, just switching and bench update --patch led to errors.
Thanks a lot guys.

Although the error while running bench update without any flag, namely the following still remains, but it is not critical:

INFO:bench.utils:updating bench
INFO:bench.utils:git pull
fatal: not a git repository (or any of the parent directories): .git
Traceback (most recent call last):
  File "/usr/local/bin/bench", line 11, in <module>
    load_entry_point('bench==4.1.0', 'console_scripts', 'bench')()
  File "/usr/local/lib/python2.7/dist-packages/bench/cli.py", line 40, in cli
    bench_command()
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/bench/commands/update.py", line 36, in update
    update_bench()
  File "/usr/local/lib/python2.7/dist-packages/bench/utils.py", line 260, in update_bench
    exec_cmd("git pull", cwd=cwd)
  File "/usr/local/lib/python2.7/dist-packages/bench/utils.py", line 159, in exec_cmd
    raise CommandFailedError(cmd)
bench.utils.CommandFailedError: git pull

codingCoffee · June 25, 2018, 9:21am

I’m not entirely sure, but I think your bench installation is flawed.
There was a point when the The Hitchhiker’s Guide to Installing Frappé on Linux OS used to suggest using

sudo pip install frappe-bench

dor installing bench. However there were some issues with the same, and we switched to using

git clone https://github.com/frappe/bench
sudo pip install ./bench

for installation. I presume you followed the former which is causing the issue.

As for the solution to the problem, uninstall bench and reinstall using the new method (from your home directory), and then check if bench update works (from your frappe-bench) directory.

paramjitrohits · June 25, 2018, 10:24am

@codingCoffee
I had followed the new tutorial (I just installed yesterday). For sake of confirmation, I uninstalled bench using sudo pip uninstall bench, and then git cloned the current repo, doing a sudo pip install ./bench again. However, the problem still persists.

A thing worth mentioning maybe is that the bench folder is not in my home folder, but rather in /home/user/.erpnext. I don’t think this makes any difference though. So the layout is:

$ ls ~/.erpnext
/home/user/.erpnext/bench/
/home/user/.erpnext/frappe-bench/

codingCoffee · June 26, 2018, 6:58am

Unable to replicate. Can you do give some more details. Execute the following and paster the output

ls -la ~/.erpnext/bench/

Is there a .git directory in it
Can you execute git pull in the folder ~/.erpnext/bench/

paramjitrohits · June 26, 2018, 6:08pm

@codingCoffee
Yes, I can do git pull in that directory. Also here are the contents of bench folder:

total 112
drwxrwxr-x 9 user user  4096 Jun 25 15:51 .
drwxr-xr-x 4 user user  4096 Jun 25 15:48 ..
drwxrwxr-x 6 user user  4096 Jun 25 15:51 bench
-rw-rw-r-- 1 user user   650 Jun 25 15:51 completion.sh
drwxrwxr-x 2 user user  4096 Jun 25 15:51 docs
drwxrwxr-x 8 user user  4096 Jun 25 15:59 .git
drwxrwxr-x 3 user user  4096 Jun 25 15:51 .github
-rw-rw-r-- 1 user user  1581 Jun 25 15:51 .gitignore
drwxrwxr-x 2 user user  4096 Jun 25 15:51 install_scripts
-rw-rw-r-- 1 user user 35121 Jun 25 15:51 LICENSE.md
-rw-rw-r-- 1 user user   190 Jun 25 15:51 MANIFEST.in
-rw-rw-r-- 1 user user    38 Jun 25 15:51 patches.txt
drwxrwxr-x 3 user user  4096 Jun 25 15:51 playbooks
-rwxrwxr-x 1 user user  8967 Jun 25 15:51 README.md
-rw-rw-r-- 1 user user    91 Jun 25 15:51 requirements.txt
-rw-rw-r-- 1 user user   957 Jun 25 15:51 setup.py
-rw-rw-r-- 1 user user  1056 Jun 25 15:51 .travis.yml
drwxrwxr-x 4 user user  4096 Jun 25 15:51 vm

nobesnickr · August 13, 2018, 6:56pm

I had the exact same issue that I was fighting with for a disturbingly long time and after digging through the “frappe” python code, discovered the solution. The problem is that the column “salt” is not created through a migration (or it does but only exists in some other branch) only during a fresh install.

I was able to resolve the issue by manually altering the database and then resetting the Administrator password.

Modify the database for the site with:

alter table __Auth add column `salt` VARCHAR(140)

Reset the Administrator Password

bench --site [SITE_NAME] set-admin-password [NEW_ADMIN_PASSWORD]

In my case, I only have the admin user as we are in the early stages of adoption. I am not sure what would need to be done in order to fix it for other non-admin users other then a mass password reset…

I am not comfortable enough yet with writing a migration and associated unit/integration tests, but hopefully someone who is can do so and commit it to save others the hassle.