Restoring from older backup

ERPNext
1

Hello Friends,

I am faced with a situation with restoring data from an older backup on a locally hosted server.

erpnext 14.27.1
frappe 14.38.3
hrms 15.0.0-dev
india_compliance 14.10.1
payments 0.0.1

The older backup (with files) is of Mar-2023 and I need to restore (with files) in a site created with current releases. When I restore data, I get some errors such as:

SETVAL(access_log_id_seq, 226, 0)
226
SETVAL(activity_log_id_seq, 1195, 0)
1195
SETVAL(console_log_id_seq, 1, 0)
1
SETVAL(crm_note_id_seq, 1, 0)
1
SETVAL(docshare_id_seq, 35, 0)
35
SETVAL(document_follow_id_seq, 1, 0)
1
SETVAL(email_queue_id_seq, 58, 0)
58
SETVAL(energy_point_log_id_seq, 1, 0)
1
SETVAL(error_log_id_seq, 24693, 0)
24693
SETVAL(event_sync_log_id_seq, 1, 0)
1
SETVAL(event_update_log_id_seq, 1, 0)
1
SETVAL(notification_log_id_seq, 11, 0)
11
SETVAL(prospect_opportunity_id_seq, 1, 0)
1
SETVAL(scheduled_job_log_id_seq, 65411, 0)
65411
SETVAL(tax_withheld_vouchers_id_seq, 1, 0)
1
SETVAL(version_id_seq, 6272, 0)
6272
SETVAL(view_log_id_seq, 1, 0)
1
SETVAL(web_form_list_column_id_seq, 1, 0)
1

Bench Start is usual with one message in Red:
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.

I am able to see the Website (and can see data populated in links (Quotations & Invoices etc.) under “My Account”, BUT on logging in to the Desk gives error screen:

Uncaught Server Exception
There was an error building this page
Error Code: 500

And, there is a traceback:

Traceback (most recent call last):
File “apps/frappe/frappe/www/app.py”, line 28, in get_context
boot = frappe.sessions.get()
File “apps/frappe/frappe/sessions.py”, line 166, in get
bootinfo = get_bootinfo()
File “apps/frappe/frappe/boot.py”, line 38, in get_bootinfo
get_user(bootinfo)
File “apps/frappe/frappe/boot.py”, line 279, in get_user
bootinfo.user = frappe.get_user().load_user()
File “apps/frappe/frappe/utils/user.py”, line 210, in load_user
d = frappe.db.get_value(
File “apps/frappe/frappe/database/database.py”, line 510, in get_value
result = self.get_values(
File “apps/frappe/frappe/database/database.py”, line 607, in get_values
out = self._get_values_from_table(
File “apps/frappe/frappe/database/database.py”, line 837, in _get_values_from_table
return query.run(as_dict=as_dict, debug=debug, update=update, run=run, pluck=pluck)
File “apps/frappe/frappe/query_builder/utils.py”, line 76, in execute_query
return frappe.db.sql(query, params, *args, **kwargs) # nosemgrep
File “apps/frappe/frappe/database/database.py”, line 221, in sql
self._cursor.execute(query, values)
File “env/lib/python3.10/site-packages/pymysql/cursors.py”, line 158, in execute
result = self._query(query)
File “env/lib/python3.10/site-packages/pymysql/cursors.py”, line 325, in _query
conn.query(q)
File “env/lib/python3.10/site-packages/pymysql/connections.py”, line 549, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File “env/lib/python3.10/site-packages/pymysql/connections.py”, line 779, in _read_query_result
result.read()
File “env/lib/python3.10/site-packages/pymysql/connections.py”, line 1157, in read
first_packet = self.connection._read_packet()
File “env/lib/python3.10/site-packages/pymysql/connections.py”, line 729, in _read_packet
packet.raise_for_error()
File “env/lib/python3.10/site-packages/pymysql/protocol.py”, line 221, in raise_for_error
err.raise_mysql_exception(self._data)
File “env/lib/python3.10/site-packages/pymysql/err.py”, line 143, in raise_mysql_exception
raise errorclass(errno, errval)
pymysql.err.OperationalError: (1054, “Unknown column ‘onboarding_status’ in ‘field list’”)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File “apps/frappe/frappe/website/serve.py”, line 18, in get_response
response = renderer_instance.render()
File “apps/frappe/frappe/website/page_renderers/template_page.py”, line 84, in render
html = self.get_html()
File “apps/frappe/frappe/website/utils.py”, line 510, in cache_html_decorator
html = func(*args, **kwargs)
File “apps/frappe/frappe/website/page_renderers/template_page.py”, line 95, in get_html
self.update_context()
File “apps/frappe/frappe/website/page_renderers/template_page.py”, line 163, in update_context
data = self.run_pymodule_method(“get_context”)
File “apps/frappe/frappe/website/page_renderers/template_page.py”, line 225, in run_pymodule_method
return method(self.context)
File “apps/frappe/frappe/www/app.py”, line 30, in get_context
raise frappe.SessionBootFailed from e
frappe.exceptions.SessionBootFailed

Can I be please guided to set this right ?

Thanks in advance
Ratninder

2

Hi @ratninder,

If old data is not suitable for restore then does not work.
did you apply the command after restoring the database?

bench update
bench --site [sitename] migrate
bench --site [sitename] clear-cache

If not applied then please apply it and check it.

Thank You!

1 Like
3

Hi @NCP

Thanks for responding to my request.

I had not tried any of the 3 things you mentioned. I have now tried all three:

  1. Bench Migrate: did lot of processing, there were no serious error messages except for one ERROR (for example):
  • [new tag] v11.1.13 → v11.1.13
  • [new tag] v11.1.14 → v11.1.14
    .
    .
  • [new tag] v14.8.0 → v14.8.0
  • [new tag] v14.9.0 → v14.9.0
    .
    .
    yarn install v1.22.19
    [1/5] Validating package.json…
    [2/5] Resolving packages…
    success Already up-to-date.
    $ npm run snyk-protect

snyk-protect
snyk-protect

Applied Snyk patches.
Done in 3.92s.
Installing hrms
$ /home/erpnext/frappe-bench/env/bin/python -m pip install --quiet --upgrade -e /home/erpnext/frappe-bench/apps/hrms
ERROR: [Errno 39] Directory not empty: ‘/home/erpnext/frappe-bench/apps/india-compliance’ → ‘/home/erpnext/frappe-bench/apps/india_compliance’
Traceback (most recent call last):
File “/usr/local/bin/bench”, line 8, in
sys.exit(cli())
.
.
And, then bench migrate gave lots & lots of error messages:
Migrating aecoindia.com

Traceback with variables (most recent call last):
File “/usr/lib/python3.10/runpy.py”, line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
mod_name = ‘frappe.utils.bench_helper’
alter_argv = True
mod_spec = ModuleSpec(name=‘frappe.utils.bench_helper’, loader=<_frozen_importlib_external.SourceFileLoader object at 0x7febe1e56ef0>, origin=‘/home/erpnext/frappe-bench/apps/frappe/frappe/utils/bench_helper.py’)
code = <code object at 0x7febdfb63c00, file “/home/erpnext/frappe-bench/apps/frappe/frappe/utils/bench_helper.py”, line 1>
main_globals = {‘name’: ‘main’, ‘doc’: None, ‘package’: ‘frappe.utils’, ‘loader’: <_frozen_importlib_external.SourceFileLoader object at 0x7febe1e56ef0>, ‘spec’: ModuleSpec(name=‘frappe.utils.bench_helper’, loader=<_frozen_importlib_external.SourceFileLoader object at 0x7febe1e56ef0>, origin=‘/home/erpnext/frappe-bench/apps/frappe/frappe/utils/bench_helper.py’), ‘annotations’: {}, ‘builtins’: <module ‘builtins’ (built-in)>, ‘file’: ‘/home/erpnext/frappe-bench/apps/frappe/frappe/utils/bench_helper.py’, ‘cached’: ‘/home/erpnext/frappe-bench/apps/frappe/frappe/utils/pycache/bench_helper.cpython-310.pyc’, ‘importlib’: <module ‘importlib’ from ‘/usr/lib/python3.10/importlib/init.py’>, ‘json’: <module ‘json’ from ‘/usr/lib/python3.10/json/init.py’>, ‘os’: <module ‘os’ from ‘/usr/lib/python3.10/os.py’>, ‘traceback’: <module ‘traceback’ from ‘/usr/lib/python3.10/traceback.py’>, ‘warnings’: <module ‘warnings’ from ‘/usr/lib/python3.10/warnings.py’>, ‘click’:…
File “/usr/lib/python3.10/runpy.py”, line 86, in _run_code
exec(code, run_globals)
.
.
The setup at present is in production mode.

If you advise, I can do a fresh install of the Ubuntu Server and Frappe and then try your suggestions ?

Having done the above, the site is now totally stopped: Internal Server Error

Thanks & Regards
Ratninder

4

Ohh.

what did I understand like you have an old v14 database and restore on fresh v14.
so your case is too different.

If you site/database in v11. then first you should to migrate version 11 → 12, 12-> 13 and 13 → 14
If you have an old database in v14, then create a fresh site and restore it.

Thank You!

5

@NCP

Sorry if you had any confusion about the Bench Version. This is why I had mentioned details in the beginning of my message:

erpnext 14.27.1
frappe 14.38.3
hrms 15.0.0-dev
india_compliance 14.10.1
payments 0.0.1

The backup is of Mar-2023 and after that bench underwent may release updates and bad on my part that a recent backup was not taken.

Anyways, I would welcome appreciate any suggestions that you or any other friend may have for this scenario.

Thanks & Regards

6

Hello.

Traceback (most recent call last):
  File "apps/frappe/frappe/www/app.py", line 28, in get_context
    boot = frappe.sessions.get()
  File "apps/frappe/frappe/sessions.py", line 166, in get
    bootinfo = get_bootinfo()
  File "apps/frappe/frappe/boot.py", line 38, in get_bootinfo
    get_user(bootinfo)
  File "apps/frappe/frappe/boot.py", line 279, in get_user
    bootinfo.user = frappe.get_user().load_user()
  File "apps/frappe/frappe/utils/user.py", line 210, in load_user
    d = frappe.db.get_value(
  File "apps/frappe/frappe/database/database.py", line 510, in get_value
    result = self.get_values(
  File "apps/frappe/frappe/database/database.py", line 607, in get_values
    out = self._get_values_from_table(
  File "apps/frappe/frappe/database/database.py", line 837, in _get_values_from_table
    return query.run(as_dict=as_dict, debug=debug, update=update, run=run, pluck=pluck)
  File "apps/frappe/frappe/query_builder/utils.py", line 76, in execute_query
    return frappe.db.sql(query, params, *args, **kwargs)  # nosemgrep
  File "apps/frappe/frappe/database/database.py", line 221, in sql
    self._cursor.execute(query, values)
  File "env/lib/python3.10/site-packages/pymysql/cursors.py", line 158, in execute
    result = self._query(query)
  File "env/lib/python3.10/site-packages/pymysql/cursors.py", line 325, in _query
    conn.query(q)
  File "env/lib/python3.10/site-packages/pymysql/connections.py", line 549, in query
    self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  File "env/lib/python3.10/site-packages/pymysql/connections.py", line 779, in _read_query_result
    result.read()
  File "env/lib/python3.10/site-packages/pymysql/connections.py", line 1157, in read
    first_packet = self.connection._read_packet()
  File "env/lib/python3.10/site-packages/pymysql/connections.py", line 729, in _read_packet
    packet.raise_for_error()
  File "env/lib/python3.10/site-packages/pymysql/protocol.py", line 221, in raise_for_error
    err.raise_mysql_exception(self._data)
  File "env/lib/python3.10/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
    raise errorclass(errno, errval)
pymysql.err.OperationalError: (1054, "Unknown column 'onboarding_status' in 'field list'")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "apps/frappe/frappe/website/serve.py", line 18, in get_response
    response = renderer_instance.render()
  File "apps/frappe/frappe/website/page_renderers/template_page.py", line 84, in render
    html = self.get_html()
  File "apps/frappe/frappe/website/utils.py", line 510, in cache_html_decorator
    html = func(*args, **kwargs)
  File "apps/frappe/frappe/website/page_renderers/template_page.py", line 95, in get_html
    self.update_context()
  File "apps/frappe/frappe/website/page_renderers/template_page.py", line 163, in update_context
    data = self.run_pymodule_method("get_context")
  File "apps/frappe/frappe/website/page_renderers/template_page.py", line 225, in run_pymodule_method
    return method(self.context)
  File "apps/frappe/frappe/www/app.py", line 30, in get_context
    raise frappe.SessionBootFailed from e
frappe.exceptions.SessionBootFailed

I have this problem when trying to restore a v14 backup to a new v14 install.

I don’t know what might have happened…? is it scheduler disabled after restore problem? during the new install, should I installed the same apps version or I don’t install any app?

7

were u able to figure it out? u on docker?

8

@l4cky

I have not been able to restore so far… really lost.

Data restores but then the Desk disappears.

Thanks for reaching out with concern.

9

you’re welcome, Do you have some doctypes edited in dev mode and you tried to bench update ?

10

@l4cky

My notes tell me that I had made following changes to doc types:

Request For Quotation - added display of field “variant_of” in item table
Work order - added display of “has_variant” in item table

Changes were made some where in Nov-22 and thereafter site was then put in production mode. Many bench updates happened after that and the Last Backup with me is of Mar-2023.

Thanks & Regards
Ratninder

11

I think u have to use the custom form.

can you undo the changes u did on ur doctype, reverse it back to the original, to all doctypes, and then try to update to see if error still persists?

12

@l4cky

The thing is that I have re-installed the entire server recently and this is why I am trying to restore data to the site, which is vanilla at this point of time.

So, if I understand it correctly from your conversation, the doc types in the backup data and the current structure have to be the same.

In my case, ERPNext is a fresh installation whereas the Backup consists of Modified Doc Types.

How would you look at this scenario please ?

13

I think to be able to use your data, you need to reinstall the same erp and frappé versions of ur backup data.

Then redit the modified doctype to be back to the original settings.

And backup the above after doctype back to original settings.

Use that new back up to restore on the new server.

14

Thanks for the tip @l4cky.

In my hand, I have just the backup tar file and json file. I can’t figure out a way to know what version of ERPNext the data was backed up.

If there is a way, I’ll be very grateful to know & try your suggestion.

Thanks

15

You can unzip the sql.gz file and open with notepad, or visual code and look for the lines.

in visual code, in line 1, you have frappe version

-- Backup generated by Frappe 14.28.2 on branch version-14

in line 16187

INSERT INTO `tabInstalled Application` VALUES ('1b0e0cffb5','2023-03-20 16:57:30.564952','2023-03-20 16:57:30.564952','Administrator','Administrator',0,2,'erpnext','14.18.3','version-14','Installed Applications','installed_applications'

But your lines will vary.

Maybe try some keywords search…

Since you know it’s version 14 , you can ctrl+F and search those exact words and character: ‘erpnext’,'14.

make sure it’s the erpnext app version and not the other apps

16

That’s great @l4cky . I’ll give it a try.

Thanks

17

@l4cky

So, I followed your suggestions and found:
– Backup generated by Frappe 14.27.0 on Branch Version-14

Search revealed: ‘erpnext’, ‘14.18.2’, version 14

On my present installation, Bench version returns:
frappe 14.39.0
erpnext 14.27.2
hrms 15.0.0-dev
india_compliance 14.10.1
payments 0.0.1

Please guide me through the steps to take.

Thanks & Regards
Ratninder

18

u have to reinstall the same version of at least erpnext and frappe, and also the apps if u can. if u have ur old git frape-bench folder, u can reinstall, else u have to manually look for the version and pull them

19

Unfortunately I don’t have old git folder so will look for the versions as suggested.

@l4cky Thanks & Regards

20

@l4cky

I have been trying to get hold of Frappe 14.27.0 Branch Version-14 BUT no success. Git does not recognize 14.27.0 as a release number ?!?

Can you please guide ?

Thanks & Regards