Error restoring backup from ErpNext 13.36 to ErpNext 14

[pradpreeonline]

Hi,

I recently installed ERPNext 14 on Ubuntu 22.04 server. Then I tried to restore the backup with below command with some errors. So I went ahead and ran a bench migrate but I ended up getting more errors during the migration as well. I know there are major changes made in version 14 and I would like to know the correct way to restore the backup from version 13 to 14. When I access the site, I got an Internal Server Error.

Please note that I have truncated the error log due to the text limit of the forum.

Here is the error that occurred during the restore:

pxp@erpserver:~/frappe-bench/sites$ bench --site erp.mysite.com restore erp.mysite.com/private/backups_to_restore/20220810_013340-erp_mysite_com-database.sql.gz
MySQL root password:
File “/usr/lib/python3.10/runpy.py”, line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File “/usr/lib/python3.10/runpy.py”, line 86, in _run_code
exec(code, run_globals)
File “/home/pxp/frappe-bench/apps/frappe/frappe/utils/bench_helper.py”, line 109, in
main()
File “/home/pxp/frappe-bench/apps/frappe/frappe/utils/bench_helper.py”, line 18, in main
click.Group(commands=commands)(prog_name=“bench”)
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/click/core.py”, line 829, in call
return self.main(*args, **kwargs)
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/click/core.py”, line 782, in main
rv = self.invoke(ctx)
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/click/core.py”, line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/click/core.py”, line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/click/core.py”, line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/click/core.py”, line 610, in invoke
return callback(*args, **kwargs)
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/click/decorators.py”, line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File “/home/pxp/frappe-bench/apps/frappe/frappe/commands/init.py”, line 29, in _func
ret = f(frappe._dict(ctx.obj), *args, **kwargs)
File “/home/pxp/frappe-bench/apps/frappe/frappe/commands/site.py”, line 208, in restore
_new_site(
File “/home/pxp/frappe-bench/apps/frappe/frappe/installer.py”, line 101, in _new_site
install_app(app, verbose=verbose, set_as_patched=not source_sql, force=False)
File “/home/pxp/frappe-bench/apps/frappe/frappe/installer.py”, line 256, in install_app
frappe.clear_cache()
File “/home/pxp/frappe-bench/apps/frappe/frappe/init.py”, line 884, in clear_cache
for fn in get_hooks(“clear_cache”):
File “/home/pxp/frappe-bench/apps/frappe/frappe/init.py”, line 1479, in get_hooks
hooks = _dict(cache().get_value(“app_hooks”, _load_app_hooks))
File “/home/pxp/frappe-bench/apps/frappe/frappe/utils/redis_wrapper.py”, line 81, in get_value
val = generator()
File “/home/pxp/frappe-bench/apps/frappe/frappe/utils/caching.py”, line 57, in wrapper
return_val = func(*args, **kwargs)
File “/home/pxp/frappe-bench/apps/frappe/frappe/init.py”, line 1448, in _load_app_hooks
app_hooks = get_module(f"{app}.hooks")
File “/home/pxp/frappe-bench/apps/frappe/frappe/init.py”, line 1326, in get_module
return importlib.import_module(modulename)
File “/usr/lib/python3.10/importlib/init.py”, line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File “”, line 1050, in _gcd_import
File “”, line 1027, in _find_and_load
File “”, line 1006, in _find_and_load_unlocked
File “”, line 688, in _load_unlocked
File “”, line 883, in exec_module
File “”, line 241, in _call_with_frames_removed
File “/home/pxp/frappe-bench/apps/erpnext/erpnext/hooks.py”, line 97, in
“defaults”: {“doctype”: “Sales Order”, “parents”: [{“label”: _(“Orders”), “route”: “orders”}]},
File “/home/pxp/frappe-bench/apps/frappe/frappe/init.py”, line 115, in _
translated_string = get_full_dict(lang).get(msg)
File “/home/pxp/frappe-bench/apps/frappe/frappe/translate.py”, line 275, in get_full_dict
user_translations = get_user_translations(lang)
File “/home/pxp/frappe-bench/apps/frappe/frappe/translate.py”, line 338, in get_user_translations
user_translations = frappe.get_all(
File “/home/pxp/frappe-bench/apps/frappe/frappe/init.py”, line 1905, in get_all
return get_list(doctype, *args, **kwargs)
File “/home/pxp/frappe-bench/apps/frappe/frappe/init.py”, line 1877, in get_list
return frappe.model.db_query.DatabaseQuery(doctype).execute(*args, **kwargs)
File “/home/pxp/frappe-bench/apps/frappe/frappe/model/db_query.py”, line 171, in execute
result = self.build_and_run()
File “/home/pxp/frappe-bench/apps/frappe/frappe/model/db_query.py”, line 186, in build_and_run
args = self.prepare_args()
File “/home/pxp/frappe-bench/apps/frappe/frappe/model/db_query.py”, line 225, in prepare_args
self.build_conditions()
File “/home/pxp/frappe-bench/apps/frappe/frappe/model/db_query.py”, line 522, in build_conditions
self.build_filter_conditions(self.filters, self.conditions)
File “/home/pxp/frappe-bench/apps/frappe/frappe/model/db_query.py”, line 543, in build_filter_conditions
conditions.append(self.prepare_filter_condition(f))
File “/home/pxp/frappe-bench/apps/frappe/frappe/model/db_query.py”, line 555, in prepare_filter_condition
f = get_filter(self.doctype, f, additional_filters_config)
File “/home/pxp/frappe-bench/apps/frappe/frappe/utils/data.py”, line 1769, in get_filter
meta = frappe.get_meta(f.doctype)
File “/home/pxp/frappe-bench/apps/frappe/frappe/init.py”, line 1210, in get_meta
return frappe.model.meta.get_meta(doctype, cached=cached)
File “/home/pxp/frappe-bench/apps/frappe/frappe/model/meta.py”, line 51, in get_meta
meta = Meta(doctype)
File “/home/pxp/frappe-bench/apps/frappe/frappe/model/meta.py”, line 113, in init
super().init(“DocType”, doctype)
File “/home/pxp/frappe-bench/apps/frappe/frappe/model/document.py”, line 105, in init
self.load_from_db()
File “/home/pxp/frappe-bench/apps/frappe/frappe/model/meta.py”, line 118, in load_from_db
super().load_from_db()
File “/home/pxp/frappe-bench/apps/frappe/frappe/model/document.py”, line 166, in load_from_db
frappe.db.get_values(
File “/home/pxp/frappe-bench/apps/frappe/frappe/database/database.py”, line 566, in get_values
out = self._get_values_from_table(
File “/home/pxp/frappe-bench/apps/frappe/frappe/database/database.py”, line 801, in _get_values_from_table
r = self.sql(query, as_dict=as_dict, debug=debug, update=update, run=run, pluck=pluck)
File “/home/pxp/frappe-bench/apps/frappe/frappe/database/database.py”, line 227, in sql
traceback.print_stack()
Error in query:
(“SELECT * FROM tabDocType State WHERE parent=‘Translation’ AND parenttype=‘DocType’ AND parentfield=‘states’ ORDER BY idx ASC”, None)
File “/usr/lib/python3.10/runpy.py”, line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File “/home/pxp/frappe-bench/apps/frappe/frappe/database/database.py”, line 566, in get_values
out = self._get_values_from_table(
File “/home/pxp/frappe-bench/apps/frappe/frappe/database/database.py”, line 801, in _get_values_from_table
r = self.sql(query, as_dict=as_dict, debug=debug, update=update, run=run, pluck=pluck)
File “/home/pxp/frappe-bench/apps/frappe/frappe/database/database.py”, line 227, in sql
traceback.print_stack()
Error in query:
(“SELECT * FROM tabDocType State WHERE parent=‘DocType Action’ AND parenttype=‘DocType’ AND parentfield=‘states’ ORDER BY idx ASC”, None)
App frappe already installed
File “/usr/lib/python3.10/runpy.py”, line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File “/usr/lib/python3.10/runpy.py”, line 86, in _run_code
exec(code, run_globals)
File “/home/pxp/frappe-bench/apps/frappe/frappe/database/database.py”, line 801, in _get_values_from_table
r = self.sql(query, as_dict=as_dict, debug=debug, update=update, run=run, pluck=pluck)
File “/home/pxp/frappe-bench/apps/frappe/frappe/database/database.py”, line 227, in sql
traceback.print_stack()
Error in query:
(“SELECT * FROM tabDocType State WHERE parent=‘System Settings’ AND parenttype=‘DocType’ AND parentfield=‘states’ ORDER BY idx ASC”, None)
*** Scheduler is enabled ***
Site erp.mysite.com has been restored
pxp@erpserver:~/frappe-bench/sites$

Then I ran the migrate and here is what I got:

pxp@erpserver:~/frappe-bench/sites$ bench --site erp.mysite.com migrate
Migrating erp.mysite.com
File “/usr/lib/python3.10/runpy.py”, line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File “/usr/lib/python3.10/runpy.py”, line 86, in _run_code
exec(code, run_globals)
File “/home/pxp/frappe-bench/apps/frappe/frappe/utils/bench_helper.py”, line 109, in
main()
File “/home/pxp/frappe-bench/apps/frappe/frappe/utils/bench_helper.py”, line 18, in main
click.Group(commands=commands)(prog_name=“bench”)
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/click/core.py”, line 829, in call
return self.main(*args, **kwargs)
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/click/core.py”, line 782, in main
rv = self.invoke(ctx)
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/click/core.py”, line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/click/core.py”, line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/click/core.py”, line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/click/core.py”, line 610, in invoke
return callback(*args, **kwargs)
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/click/decorators.py”, line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File “/home/pxp/frappe-bench/apps/frappe/frappe/commands/init.py”, line 29, in _func
ret = f(frappe._dict(ctx.obj), args, **kwargs)
File “/home/pxp/frappe-bench/apps/frappe/frappe/commands/site.py”, line 523, in migrate
SiteMigration(
File “/home/pxp/frappe-bench/apps/frappe/frappe/database/database.py”, line 566, in get_values
out = self._get_values_from_table(
File “/home/pxp/frappe-bench/apps/frappe/frappe/database/database.py”, line 801, in _get_values_from_table
r = self.sql(query, as_dict=as_dict, debug=debug, update=update, run=run, pluck=pluck)
File “/home/pxp/frappe-bench/apps/frappe/frappe/database/database.py”, line 227, in sql
traceback.print_stack()
Error in query:
(“SELECT * FROM tabDocType State WHERE parent=‘Patch Log’ AND parenttype=‘DocType’ AND parentfield=‘states’ ORDER BY idx ASC”, None)
File “/usr/lib/python3.10/runpy.py”, line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File “/usr/lib/python3.10/runpy.py”, line 86, in _run_code
exec(code, run_globals)
File “/home/pxp/frappe-bench/apps/frappe/frappe/database/database.py”, line 566, in get_values
out = self._get_values_from_table(
File “/home/pxp/frappe-bench/apps/frappe/frappe/database/database.py”, line 801, in _get_values_from_table
r = self.sql(query, as_dict=as_dict, debug=debug, update=update, run=run, pluck=pluck)
File “/home/pxp/frappe-bench/apps/frappe/frappe/database/database.py”, line 227, in sql
traceback.print_stack()
Error in query:
(“SELECT * FROM tabDocType State WHERE parent=‘Custom DocPerm’ AND parenttype=‘DocType’ AND parentfield=‘states’ ORDER BY idx ASC”, None)
File “/usr/lib/python3.10/runpy.py”, line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File “/usr/lib/python3.10/runpy.py”, line 86, in _run_code
exec(code, run_globals)
File “/home/pxp/frappe-bench/apps/frappe/frappe/database/database.py”, line 801, in _get_values_from_table
r = self.sql(query, as_dict=as_dict, debug=debug, update=update, run=run, pluck=pluck)
File “/home/pxp/frappe-bench/apps/frappe/frappe/database/database.py”, line 227, in sql
traceback.print_stack()
Error in query:
(“SELECT * FROM tabDocType State WHERE parent=‘System Settings’ AND parenttype=‘DocType’ AND parentfield=‘states’ ORDER BY idx ASC”, None)
File “/usr/lib/python3.10/runpy.py”, line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File “/usr/lib/python3.10/runpy.py”, line 86, in _run_code
exec(code, run_globals)
File “/home/pxp/frappe-bench/apps/frappe/frappe/database/database.py”, line 227, in sql
traceback.print_stack()
Error in query:
('SELECT DISTINCT COUNT() FROM tabFeedback’, None)
Updating Dashboard for frappe
Updating Dashboard for erpnext
Updating customizations for Address
Updating customizations for Contact
Building search index for erp.mysite.com
Retrieving Routes : [========================================] 100%
Building Index : [========================================] 100%
Traceback (most recent call last):
File “/usr/lib/python3.10/runpy.py”, line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File “/usr/lib/python3.10/runpy.py”, line 86, in _run_code
exec(code, run_globals)
File “/home/pxp/frappe-bench/apps/frappe/frappe/utils/bench_helper.py”, line 109, in
main()
File “/home/pxp/frappe-bench/apps/frappe/frappe/utils/bench_helper.py”, line 18, in main
click.Group(commands=commands)(prog_name=“bench”)
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/click/core.py”, line 829, in call
return self.main(*args, **kwargs)
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/click/core.py”, line 782, in main
rv = self.invoke(ctx)
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/click/core.py”, line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/click/core.py”, line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/click/core.py”, line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/click/core.py”, line 610, in invoke
return callback(*args, **kwargs)
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/click/decorators.py”, line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File “/home/pxp/frappe-bench/apps/frappe/frappe/commands/init.py”, line 29, in _func
ret = f(frappe._dict(ctx.obj), *args, **kwargs)
File “/home/pxp/frappe-bench/apps/frappe/frappe/commands/site.py”, line 523, in migrate
SiteMigration(
File “/home/pxp/frappe-bench/apps/frappe/frappe/migrate.py”, line 172, in run
self.run_schema_updates()
File “/home/pxp/frappe-bench/apps/frappe/frappe/migrate.py”, line 40, in wrapper
ret = method(*args, **kwargs)
File “/home/pxp/frappe-bench/apps/frappe/frappe/migrate.py”, line 109, in run_schema_updates
frappe.modules.patch_handler.run_all(
File “/home/pxp/frappe-bench/apps/frappe/frappe/modules/patch_handler.py”, line 75, in run_all
run_patch(patch)
File “/home/pxp/frappe-bench/apps/frappe/frappe/modules/patch_handler.py”, line 62, in run_patch
if not run_single(patchmodule=patch):
File “/home/pxp/frappe-bench/apps/frappe/frappe/modules/patch_handler.py”, line 150, in run_single
return execute_patch(patchmodule, method, methodargs)
File “/home/pxp/frappe-bench/apps/frappe/frappe/modules/patch_handler.py”, line 185, in execute_patch
_patch()
File “/home/pxp/frappe-bench/apps/frappe/frappe/patches/v14_0/setup_likes_from_feedback.py”, line 7, in execute
if frappe.db.count(“Feedback”) > 20000:
File “/home/pxp/frappe-bench/apps/frappe/frappe/database/database.py”, line 1086, in count
count = self.sql(query, debug=debug)[0][0]
File “/home/pxp/frappe-bench/apps/frappe/frappe/database/database.py”, line 209, in sql
self._cursor.execute(query, values)
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pymysql/cursors.py”, line 148, in execute
result = self._query(query)
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pymysql/cursors.py”, line 310, in _query
conn.query(q)
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pymysql/connections.py”, line 548, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pymysql/connections.py”, line 775, in _read_query_result
result.read()
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pymysql/connections.py”, line 1156, in read
first_packet = self.connection._read_packet()
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pymysql/connections.py”, line 725, in _read_packet
packet.raise_for_error()
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pymysql/protocol.py”, line 221, in raise_for_error
err.raise_mysql_exception(self._data)
File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pymysql/err.py”, line 143, in raise_mysql_exception
raise errorclass(errno, errval)
pymysql.err.ProgrammingError: (1146, “Table ‘_004c3ffaabf02568.tabFeedback’ doesn’t exist”)
pxp@erpserver:~/frappe-bench/sites$

[smino]

Hi,

It might be better to install v13 on the server, restore the backup , then switch to branch version-14 , update-patch and migrate. It might be possible to revert your setup to version-13 and restore, then upgrade to 14 as an option.

[pradpreeonline]

I have version 13 running on another instance. Since version 14 requires Python 3.10 and my instance has version 3.8 I was not able to upgrade and while upgrading pip to the latest version I was having issues and couldn’t take that route, so installed a fresh copy of version 14 anticipating the data backup can be restored without issues but I was wrong.

This is the doc I am following and I submitted my issues there as well under the doc to upgrade Python.

How I upgraded from v13 to v14 and Ubuntu 20.04 LTS to 22.04 LTS

First, I made sure my last nightly backup of the VM was successful. Then I backed up the site with bench --site *sitename* backup --with-files and copied the backups off-host. Then I made another snapshot of the VM. Being OCD with backups is helpful with this kind of upgrade! Next, I upgraded frappe-bench to the latest: pip3 install frappe-bench --upgrade At this point, I followed this guide to upgrade to Python 3.10 and pip to the latest version. Next, I ran bench restart and tested the site…

[smino]

A couple of things from my notes from getting 13 running on 22.04:

I issued: sudo pip3 install frappe-bench --upgrade

I too followed @rmeyer 's excellent instructions, although I went ahead and did the 3.10 steps and 13 is running fine on 22.04 , I plan to switch to 14 soon.

[pradpreeonline]

Did you encounter this problem while following the @rmeyer’s steps to upgrade pip?

How to upgrade to python 3.10

Hi, I am getting the below error running this step env/bin/pip install -e apps/frappe -e apps/erpnext pxp@erp2-server:~/frappe-bench$ env/bin/pip3.10 install -e apps/frappe/ -e apps/erpnext Traceback (most recent call last): File “/home/pxp/frappe-bench/env/bin/pip3.10”, line 8, in sys.exit(main()) File “/home/pxp/frappe-bench/env/lib/python3.10/site-packages/pip/_internal/cli/main.py”, line 73, in main command = create_command(cmd_name, isolated=(“–isolated” in cmd_args)) File “/ho…

[smino]

sorry, this reply posted in the wrong thread:

Hi ,

I don’t recall encountering that error. I do see that I upgraded pip via: sudo pip3 install pip --upgrade

pip3 -V reports:

pip 22.2.2 from /usr/local/lib/python3.10/dist-packages/pip (python 3.10)

[pradpreeonline]

No worries. I saw your comments there and I’m following up with the error I am encountering on that thread. Thanks!