[Frappe v14] Certain apps not persisting after restoring via database, but the apps are not deleted from the app folder

PhantomKunai · August 18, 2022, 3:59am

Description of the issue

Restoring a database either uninstalls certain apps or causes them to be set as not installed. Bench version still shows all apps.

Context information (for bug reports)

Context : Restoring of database from v13 to v14

Output of bench version

abs_india 0.6.18
erpnext 14.0.0
frappe 14.1.0
hrms 0.0.1
india_compliance 0.0.1
payments 0.0.1

Bench version still shows all apps as installed though.

Steps to reproduce the issue

Install apps in v14 that were not present in v13
Restore the database and files
Check apps which are installed and the apps not present in v13 are deleted.

Observed result

Checking installed apps before db restore

erp@erp14:~/frappe-bench$ bench console
Apps in this namespace:
frappe, payments, erpnext, india_compliance, hrms, abs_india

In [1]: frappe.get_installed_apps()
Out[1]: ['frappe', 'payments', 'erpnext', 'india_compliance', 'hrms', 'abs_india']

Restoring the db shows this stack trace but seems like stuff pulled through anyway

File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/erp/frappe-bench/apps/frappe/frappe/utils/bench_helper.py", line 109, in <module>
    main()
  File "/home/erp/frappe-bench/apps/frappe/frappe/utils/bench_helper.py", line 18, in main
    click.Group(commands=commands)(prog_name="bench")
  File "/home/erp/frappe-bench/env/lib/python3.10/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/erp/frappe-bench/env/lib/python3.10/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/erp/frappe-bench/env/lib/python3.10/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/erp/frappe-bench/env/lib/python3.10/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/erp/frappe-bench/env/lib/python3.10/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/erp/frappe-bench/env/lib/python3.10/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/erp/frappe-bench/env/lib/python3.10/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/erp/frappe-bench/apps/frappe/frappe/commands/__init__.py", line 29, in _func
    ret = f(frappe._dict(ctx.obj), *args, **kwargs)
  File "/home/erp/frappe-bench/apps/frappe/frappe/commands/site.py", line 208, in restore
    _new_site(
  File "/home/erp/frappe-bench/apps/frappe/frappe/installer.py", line 101, in _new_site
    install_app(app, verbose=verbose, set_as_patched=not source_sql, force=False)
  File "/home/erp/frappe-bench/apps/frappe/frappe/installer.py", line 256, in install_app
    frappe.clear_cache()
  File "/home/erp/frappe-bench/apps/frappe/frappe/__init__.py", line 884, in clear_cache
    for fn in get_hooks("clear_cache"):
  File "/home/erp/frappe-bench/apps/frappe/frappe/__init__.py", line 1489, in get_hooks
    hooks = _dict(cache().get_value("app_hooks", _load_app_hooks))
  File "/home/erp/frappe-bench/apps/frappe/frappe/utils/redis_wrapper.py", line 81, in get_value
    val = generator()
  File "/home/erp/frappe-bench/apps/frappe/frappe/utils/caching.py", line 57, in wrapper
    return_val = func(*args, **kwargs)
  File "/home/erp/frappe-bench/apps/frappe/frappe/__init__.py", line 1458, in _load_app_hooks
    app_hooks = get_module(f"{app}.hooks")
  File "/home/erp/frappe-bench/apps/frappe/frappe/__init__.py", line 1336, in get_module
    return importlib.import_module(modulename)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/erp/frappe-bench/apps/erpnext/erpnext/hooks.py", line 97, in <module>
    "defaults": {"doctype": "Sales Order", "parents": [{"label": _("Orders"), "route": "orders"}]},
  File "/home/erp/frappe-bench/apps/frappe/frappe/__init__.py", line 115, in _
    translated_string = get_full_dict(lang).get(msg)
  File "/home/erp/frappe-bench/apps/frappe/frappe/translate.py", line 285, in get_full_dict
    user_translations = get_user_translations(lang)
  File "/home/erp/frappe-bench/apps/frappe/frappe/translate.py", line 348, in get_user_translations
    user_translations = frappe.get_all(
  File "/home/erp/frappe-bench/apps/frappe/frappe/__init__.py", line 1915, in get_all
    return get_list(doctype, *args, **kwargs)
  File "/home/erp/frappe-bench/apps/frappe/frappe/__init__.py", line 1887, in get_list
    return frappe.model.db_query.DatabaseQuery(doctype).execute(*args, **kwargs)
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/db_query.py", line 172, in execute
    result = self.build_and_run()
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/db_query.py", line 187, in build_and_run
    args = self.prepare_args()
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/db_query.py", line 226, in prepare_args
    self.build_conditions()
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/db_query.py", line 523, in build_conditions
    self.build_filter_conditions(self.filters, self.conditions)
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/db_query.py", line 544, in build_filter_conditions
    conditions.append(self.prepare_filter_condition(f))
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/db_query.py", line 556, in prepare_filter_condition
    f = get_filter(self.doctype, f, additional_filters_config)
  File "/home/erp/frappe-bench/apps/frappe/frappe/utils/data.py", line 1769, in get_filter
    meta = frappe.get_meta(f.doctype)
  File "/home/erp/frappe-bench/apps/frappe/frappe/__init__.py", line 1220, in get_meta
    return frappe.model.meta.get_meta(doctype, cached=cached)
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/meta.py", line 51, in get_meta
    meta = Meta(doctype)
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/meta.py", line 113, in __init__
    super().__init__("DocType", doctype)
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/document.py", line 105, in __init__
    self.load_from_db()
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/meta.py", line 118, in load_from_db
    super().load_from_db()
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/document.py", line 166, in load_from_db
    frappe.db.get_values(
  File "/home/erp/frappe-bench/apps/frappe/frappe/database/database.py", line 562, in get_values
    out = self._get_values_from_table(
  File "/home/erp/frappe-bench/apps/frappe/frappe/database/database.py", line 792, in _get_values_from_table
    return self.sql(query, as_dict=as_dict, debug=debug, update=update, run=run, pluck=pluck)
  File "/home/erp/frappe-bench/apps/frappe/frappe/database/database.py", line 227, in sql
    traceback.print_stack()
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/erp/frappe-bench/apps/frappe/frappe/utils/bench_helper.py", line 109, in <module>
    main()
  File "/home/erp/frappe-bench/apps/frappe/frappe/utils/bench_helper.py", line 18, in main
    click.Group(commands=commands)(prog_name="bench")
  File "/home/erp/frappe-bench/env/lib/python3.10/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/erp/frappe-bench/env/lib/python3.10/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/erp/frappe-bench/env/lib/python3.10/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/erp/frappe-bench/env/lib/python3.10/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/erp/frappe-bench/env/lib/python3.10/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/erp/frappe-bench/env/lib/python3.10/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/erp/frappe-bench/env/lib/python3.10/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/erp/frappe-bench/apps/frappe/frappe/commands/__init__.py", line 29, in _func
    ret = f(frappe._dict(ctx.obj), *args, **kwargs)
  File "/home/erp/frappe-bench/apps/frappe/frappe/commands/site.py", line 208, in restore
    _new_site(
  File "/home/erp/frappe-bench/apps/frappe/frappe/installer.py", line 101, in _new_site
    install_app(app, verbose=verbose, set_as_patched=not source_sql, force=False)
  File "/home/erp/frappe-bench/apps/frappe/frappe/installer.py", line 256, in install_app
    frappe.clear_cache()
  File "/home/erp/frappe-bench/apps/frappe/frappe/__init__.py", line 884, in clear_cache
    for fn in get_hooks("clear_cache"):
  File "/home/erp/frappe-bench/apps/frappe/frappe/__init__.py", line 1489, in get_hooks
    hooks = _dict(cache().get_value("app_hooks", _load_app_hooks))
  File "/home/erp/frappe-bench/apps/frappe/frappe/utils/redis_wrapper.py", line 81, in get_value
    val = generator()
  File "/home/erp/frappe-bench/apps/frappe/frappe/utils/caching.py", line 57, in wrapper
    return_val = func(*args, **kwargs)
  File "/home/erp/frappe-bench/apps/frappe/frappe/__init__.py", line 1458, in _load_app_hooks
    app_hooks = get_module(f"{app}.hooks")
  File "/home/erp/frappe-bench/apps/frappe/frappe/__init__.py", line 1336, in get_module
    return importlib.import_module(modulename)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/erp/frappe-bench/apps/erpnext/erpnext/hooks.py", line 97, in <module>
    "defaults": {"doctype": "Sales Order", "parents": [{"label": _("Orders"), "route": "orders"}]},
  File "/home/erp/frappe-bench/apps/frappe/frappe/__init__.py", line 115, in _
    translated_string = get_full_dict(lang).get(msg)
  File "/home/erp/frappe-bench/apps/frappe/frappe/translate.py", line 285, in get_full_dict
    user_translations = get_user_translations(lang)
  File "/home/erp/frappe-bench/apps/frappe/frappe/translate.py", line 348, in get_user_translations
    user_translations = frappe.get_all(
  File "/home/erp/frappe-bench/apps/frappe/frappe/__init__.py", line 1915, in get_all
    return get_list(doctype, *args, **kwargs)
  File "/home/erp/frappe-bench/apps/frappe/frappe/__init__.py", line 1887, in get_list
    return frappe.model.db_query.DatabaseQuery(doctype).execute(*args, **kwargs)
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/db_query.py", line 172, in execute
    result = self.build_and_run()
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/db_query.py", line 187, in build_and_run
    args = self.prepare_args()
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/db_query.py", line 226, in prepare_args
    self.build_conditions()
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/db_query.py", line 523, in build_conditions
    self.build_filter_conditions(self.filters, self.conditions)
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/db_query.py", line 544, in build_filter_conditions
    conditions.append(self.prepare_filter_condition(f))
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/db_query.py", line 556, in prepare_filter_condition
    f = get_filter(self.doctype, f, additional_filters_config)
  File "/home/erp/frappe-bench/apps/frappe/frappe/utils/data.py", line 1769, in get_filter
    meta = frappe.get_meta(f.doctype)
  File "/home/erp/frappe-bench/apps/frappe/frappe/__init__.py", line 1220, in get_meta
    return frappe.model.meta.get_meta(doctype, cached=cached)
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/meta.py", line 51, in get_meta
    meta = Meta(doctype)
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/meta.py", line 114, in __init__
    self.process()
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/meta.py", line 136, in process
    self.add_custom_links_and_actions()
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/meta.py", line 429, in add_custom_links_and_actions
    for d in frappe.get_all(
  File "/home/erp/frappe-bench/apps/frappe/frappe/__init__.py", line 1915, in get_all
    return get_list(doctype, *args, **kwargs)
  File "/home/erp/frappe-bench/apps/frappe/frappe/__init__.py", line 1887, in get_list
    return frappe.model.db_query.DatabaseQuery(doctype).execute(*args, **kwargs)
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/db_query.py", line 172, in execute
    result = self.build_and_run()
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/db_query.py", line 187, in build_and_run
    args = self.prepare_args()
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/db_query.py", line 226, in prepare_args
    self.build_conditions()
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/db_query.py", line 523, in build_conditions
    self.build_filter_conditions(self.filters, self.conditions)
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/db_query.py", line 544, in build_filter_conditions
    conditions.append(self.prepare_filter_condition(f))
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/db_query.py", line 556, in prepare_filter_condition
    f = get_filter(self.doctype, f, additional_filters_config)
  File "/home/erp/frappe-bench/apps/frappe/frappe/utils/data.py", line 1779, in get_filter
    df = frappe.get_meta(f.doctype).get_field(f.fieldname)
  File "/home/erp/frappe-bench/apps/frappe/frappe/__init__.py", line 1220, in get_meta
    return frappe.model.meta.get_meta(doctype, cached=cached)
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/meta.py", line 51, in get_meta
    meta = Meta(doctype)
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/meta.py", line 113, in __init__
    super().__init__("DocType", doctype)
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/document.py", line 105, in __init__
    self.load_from_db()
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/meta.py", line 118, in load_from_db
    super().load_from_db()
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/document.py", line 166, in load_from_db
    frappe.db.get_values(
  File "/home/erp/frappe-bench/apps/frappe/frappe/database/database.py", line 562, in get_values
    out = self._get_values_from_table(
  File "/home/erp/frappe-bench/apps/frappe/frappe/database/database.py", line 792, in _get_values_from_table
    return self.sql(query, as_dict=as_dict, debug=debug, update=update, run=run, pluck=pluck)
  File "/home/erp/frappe-bench/apps/frappe/frappe/database/database.py", line 227, in sql
    traceback.print_stack()
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/erp/frappe-bench/apps/frappe/frappe/utils/bench_helper.py", line 109, in <module>
    main()
  File "/home/erp/frappe-bench/apps/frappe/frappe/utils/bench_helper.py", line 18, in main
    click.Group(commands=commands)(prog_name="bench")
  File "/home/erp/frappe-bench/env/lib/python3.10/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/erp/frappe-bench/env/lib/python3.10/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/erp/frappe-bench/env/lib/python3.10/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/erp/frappe-bench/env/lib/python3.10/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/erp/frappe-bench/env/lib/python3.10/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/erp/frappe-bench/env/lib/python3.10/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/erp/frappe-bench/env/lib/python3.10/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/erp/frappe-bench/apps/frappe/frappe/commands/__init__.py", line 29, in _func
    ret = f(frappe._dict(ctx.obj), *args, **kwargs)
  File "/home/erp/frappe-bench/apps/frappe/frappe/commands/site.py", line 208, in restore
    _new_site(
  File "/home/erp/frappe-bench/apps/frappe/frappe/installer.py", line 101, in _new_site
    install_app(app, verbose=verbose, set_as_patched=not source_sql, force=False)
  File "/home/erp/frappe-bench/apps/frappe/frappe/installer.py", line 256, in install_app
    frappe.clear_cache()
  File "/home/erp/frappe-bench/apps/frappe/frappe/__init__.py", line 884, in clear_cache
    for fn in get_hooks("clear_cache"):
  File "/home/erp/frappe-bench/apps/frappe/frappe/__init__.py", line 1489, in get_hooks
    hooks = _dict(cache().get_value("app_hooks", _load_app_hooks))
  File "/home/erp/frappe-bench/apps/frappe/frappe/utils/redis_wrapper.py", line 81, in get_value
    val = generator()
  File "/home/erp/frappe-bench/apps/frappe/frappe/utils/caching.py", line 57, in wrapper
    return_val = func(*args, **kwargs)
  File "/home/erp/frappe-bench/apps/frappe/frappe/__init__.py", line 1458, in _load_app_hooks
    app_hooks = get_module(f"{app}.hooks")
  File "/home/erp/frappe-bench/apps/frappe/frappe/__init__.py", line 1336, in get_module
    return importlib.import_module(modulename)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/erp/frappe-bench/apps/erpnext/erpnext/hooks.py", line 97, in <module>
    "defaults": {"doctype": "Sales Order", "parents": [{"label": _("Orders"), "route": "orders"}]},
  File "/home/erp/frappe-bench/apps/frappe/frappe/__init__.py", line 115, in _
    translated_string = get_full_dict(lang).get(msg)
  File "/home/erp/frappe-bench/apps/frappe/frappe/translate.py", line 285, in get_full_dict
    user_translations = get_user_translations(lang)
  File "/home/erp/frappe-bench/apps/frappe/frappe/translate.py", line 348, in get_user_translations
    user_translations = frappe.get_all(
  File "/home/erp/frappe-bench/apps/frappe/frappe/__init__.py", line 1915, in get_all
    return get_list(doctype, *args, **kwargs)
  File "/home/erp/frappe-bench/apps/frappe/frappe/__init__.py", line 1887, in get_list
    return frappe.model.db_query.DatabaseQuery(doctype).execute(*args, **kwargs)
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/db_query.py", line 172, in execute
    result = self.build_and_run()
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/db_query.py", line 187, in build_and_run
    args = self.prepare_args()
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/db_query.py", line 226, in prepare_args
    self.build_conditions()
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/db_query.py", line 523, in build_conditions
    self.build_filter_conditions(self.filters, self.conditions)
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/db_query.py", line 544, in build_filter_conditions
    conditions.append(self.prepare_filter_condition(f))
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/db_query.py", line 556, in prepare_filter_condition
    f = get_filter(self.doctype, f, additional_filters_config)
  File "/home/erp/frappe-bench/apps/frappe/frappe/utils/data.py", line 1769, in get_filter
    meta = frappe.get_meta(f.doctype)
  File "/home/erp/frappe-bench/apps/frappe/frappe/__init__.py", line 1220, in get_meta
    return frappe.model.meta.get_meta(doctype, cached=cached)
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/meta.py", line 51, in get_meta
    meta = Meta(doctype)
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/meta.py", line 114, in __init__
    self.process()
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/meta.py", line 136, in process
    self.add_custom_links_and_actions()
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/meta.py", line 429, in add_custom_links_and_actions
    for d in frappe.get_all(
  File "/home/erp/frappe-bench/apps/frappe/frappe/__init__.py", line 1915, in get_all
    return get_list(doctype, *args, **kwargs)
  File "/home/erp/frappe-bench/apps/frappe/frappe/__init__.py", line 1887, in get_list
    return frappe.model.db_query.DatabaseQuery(doctype).execute(*args, **kwargs)
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/db_query.py", line 172, in execute
    result = self.build_and_run()
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/db_query.py", line 187, in build_and_run
    args = self.prepare_args()
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/db_query.py", line 226, in prepare_args
    self.build_conditions()
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/db_query.py", line 523, in build_conditions
    self.build_filter_conditions(self.filters, self.conditions)
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/db_query.py", line 544, in build_filter_conditions
    conditions.append(self.prepare_filter_condition(f))
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/db_query.py", line 556, in prepare_filter_condition
    f = get_filter(self.doctype, f, additional_filters_config)
  File "/home/erp/frappe-bench/apps/frappe/frappe/utils/data.py", line 1779, in get_filter
    df = frappe.get_meta(f.doctype).get_field(f.fieldname)
  File "/home/erp/frappe-bench/apps/frappe/frappe/__init__.py", line 1220, in get_meta
    return frappe.model.meta.get_meta(doctype, cached=cached)
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/meta.py", line 51, in get_meta
    meta = Meta(doctype)
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/meta.py", line 113, in __init__
    super().__init__("DocType", doctype)
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/document.py", line 105, in __init__
    self.load_from_db()
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/meta.py", line 118, in load_from_db
    super().load_from_db()
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/document.py", line 166, in load_from_db
    frappe.db.get_values(
  File "/home/erp/frappe-bench/apps/frappe/frappe/database/database.py", line 562, in get_values
    out = self._get_values_from_table(
  File "/home/erp/frappe-bench/apps/frappe/frappe/database/database.py", line 792, in _get_values_from_table
    return self.sql(query, as_dict=as_dict, debug=debug, update=update, run=run, pluck=pluck)
  File "/home/erp/frappe-bench/apps/frappe/frappe/database/database.py", line 227, in sql
    traceback.print_stack()
Error in query:
("SELECT * FROM `tabDocType State` WHERE `parent`='Translation' AND `parenttype`='DocType' AND `parentfield`='states' ORDER BY `idx` ASC", None)
Error in query:
("SELECT * FROM `tabDocType State` WHERE `parent`='DocType Link' AND `parenttype`='DocType' AND `parentfield`='states' ORDER BY `idx` ASC", None)
Error in query:
("SELECT * FROM `tabDocType State` WHERE `parent`='DocType Action' AND `parenttype`='DocType' AND `parentfield`='states' ORDER BY `idx` ASC", None)
App frappe already installed
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/erp/frappe-bench/apps/frappe/frappe/utils/bench_helper.py", line 109, in <module>
    main()
  File "/home/erp/frappe-bench/apps/frappe/frappe/utils/bench_helper.py", line 18, in main
    click.Group(commands=commands)(prog_name="bench")
  File "/home/erp/frappe-bench/env/lib/python3.10/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/erp/frappe-bench/env/lib/python3.10/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/erp/frappe-bench/env/lib/python3.10/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/erp/frappe-bench/env/lib/python3.10/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/erp/frappe-bench/env/lib/python3.10/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/erp/frappe-bench/env/lib/python3.10/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/erp/frappe-bench/env/lib/python3.10/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/erp/frappe-bench/apps/frappe/frappe/commands/__init__.py", line 29, in _func
    ret = f(frappe._dict(ctx.obj), *args, **kwargs)
  File "/home/erp/frappe-bench/apps/frappe/frappe/commands/site.py", line 208, in restore
    _new_site(
  File "/home/erp/frappe-bench/apps/frappe/frappe/installer.py", line 105, in _new_site
    scheduler.toggle_scheduler(enable_scheduler)
  File "/home/erp/frappe-bench/apps/frappe/frappe/utils/scheduler.py", line 133, in toggle_scheduler
    frappe.db.set_value("System Settings", None, "enable_scheduler", 1 if enable else 0)
  File "/home/erp/frappe-bench/apps/frappe/frappe/database/database.py", line 858, in set_value
    modified = modified or now()
  File "/home/erp/frappe-bench/apps/frappe/frappe/utils/data.py", line 357, in now
    return now_datetime().strftime(DATETIME_FORMAT)
  File "/home/erp/frappe-bench/apps/frappe/frappe/utils/data.py", line 304, in now_datetime
    dt = convert_utc_to_user_timezone(datetime.datetime.utcnow())
  File "/home/erp/frappe-bench/apps/frappe/frappe/utils/data.py", line 344, in convert_utc_to_user_timezone
    time_zone = get_time_zone()
  File "/home/erp/frappe-bench/apps/frappe/frappe/utils/data.py", line 325, in get_time_zone
    return frappe.cache().get_value("time_zone", _get_time_zone)
  File "/home/erp/frappe-bench/apps/frappe/frappe/utils/redis_wrapper.py", line 81, in get_value
    val = generator()
  File "/home/erp/frappe-bench/apps/frappe/frappe/utils/data.py", line 318, in _get_time_zone
    return frappe.db.get_system_setting("time_zone") or "Asia/Kolkata"  # Default to India ?!
  File "/home/erp/frappe-bench/apps/frappe/frappe/database/database.py", line 1171, in get_system_setting
    return frappe.get_system_settings(key)
  File "/home/erp/frappe-bench/apps/frappe/frappe/__init__.py", line 2294, in get_system_settings
    local.system_settings = db.get_singles_dict("System Settings", cast=True)
  File "/home/erp/frappe-bench/apps/frappe/frappe/database/database.py", line 676, in get_singles_dict
    meta = frappe.get_meta(doctype)
  File "/home/erp/frappe-bench/apps/frappe/frappe/__init__.py", line 1220, in get_meta
    return frappe.model.meta.get_meta(doctype, cached=cached)
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/meta.py", line 51, in get_meta
    meta = Meta(doctype)
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/meta.py", line 113, in __init__
    super().__init__("DocType", doctype)
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/document.py", line 105, in __init__
    self.load_from_db()
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/meta.py", line 118, in load_from_db
    super().load_from_db()
  File "/home/erp/frappe-bench/apps/frappe/frappe/model/document.py", line 166, in load_from_db
    frappe.db.get_values(
  File "/home/erp/frappe-bench/apps/frappe/frappe/database/database.py", line 562, in get_values
    out = self._get_values_from_table(
  File "/home/erp/frappe-bench/apps/frappe/frappe/database/database.py", line 792, in _get_values_from_table
    return self.sql(query, as_dict=as_dict, debug=debug, update=update, run=run, pluck=pluck)
  File "/home/erp/frappe-bench/apps/frappe/frappe/database/database.py", line 227, in sql
    traceback.print_stack()
Error in query:
("SELECT * FROM `tabDocType State` WHERE `parent`='System Settings' AND `parenttype`='DocType' AND `parentfield`='states' ORDER BY `idx` ASC", None)
*** Scheduler is enabled ***
Site site1.local has been restored with files

Checking the installed apps again to see only the apps that were present in the v13 instance

erp@erp14:~/frappe-bench$ bench console
Apps in this namespace:
frappe, erpnext, abs_india

In [1]: frappe.get_installed_apps()
Out[1]: ['frappe', 'erpnext', 'abs_india']

Expected result

All apps must be present, as this list is used in the hrms migration patch which then deletes all doctype definitions based on this installed app list

Additional information

OS : Ubuntu 22.04
Install Method : Manual

I have even attempted to manually add the app list in the tabInstalled Apps table in the database, but no luck.

centaur · August 18, 2022, 4:19am

hi

Your v13 only had frappe, erpnext and abs_india. so then V13 database will only have those.

First thing you should have done was for V14 was only to install frappe, erpnext and abs_india. Then restore the db and once all the migration and update are completed , then proceed to install payments , then hrms and then india_compliance.

PhantomKunai · August 18, 2022, 4:51am

But the issue I think is that my custom app abs_india is dependant on certain features in the other apps and it also adds fixtures which involve customisations in those dependant apps.

So if I were to install my custom app, the installation would fail because it requires hrms.

snv · August 18, 2022, 5:05am

Start with a v13 restored DB. A restored DB will only have those apps present which were initially backed up.

Remove conflicts temporarily from custom app (or uninstall it).
Update frappe and erpnext (migrate if already on v14)
Install payments, hrms, india_compliance
Reinstall your app (if uninstalled)
Ensure all apps are installed: bench execute frappe.get_installed_apps
Execute following code in bench console to keep sequence proper (payments comes first):

frappe.db.set_global("installed_apps", json.dumps(["frappe", "payments", "erpnext", "hrms", "india_compliance", "abs_india"]))
frappe.db.commit()

Updating the sequence is optional - it’s a failsafe measure. Everything may still work even if the sequence is not okay. If you need your custom app’s frontend code to override code in other apps, you may also need to update sequence in sites/apps.txt

PhantomKunai · August 18, 2022, 5:10am

Okay, I will try this and get back with info.

Just for my information,

Where in the db does this global config sit? Can I modify it via mysql?
If I have to do this exercise on a Frappe Cloud instance, it means then that I have to get ssh access and do this, is it not?
Would this be a one time exercise?

snv · August 18, 2022, 5:15am

tabDefaultValue with parent as __global

If you need to, DB can be manipulated from System Console for FC. AFAIK, FC provides SSH access for installations with custom apps.

I think so, yes - atleast for v14

snv · August 18, 2022, 5:18am

I updated my first reply - please re-review.

PhantomKunai · August 18, 2022, 7:29am

I went through with these steps and came across a bizarre error that I didnt face in previous experiments.

   File "apps/frappe/frappe/www/app.py", line 26, in get_context
13:03:28 web.1            |     boot = frappe.sessions.get()
13:03:28 web.1            |   File "apps/frappe/frappe/sessions.py", line 161, in get
13:03:28 web.1            |     bootinfo = get_bootinfo()
13:03:28 web.1            |   File "apps/frappe/frappe/boot.py", line 36, in get_bootinfo
13:03:28 web.1            |     get_user(bootinfo)
13:03:28 web.1            |   File "apps/frappe/frappe/boot.py", line 268, in get_user
13:03:28 web.1            |     bootinfo.user = frappe.get_user().load_user()
13:03:28 web.1            |   File "apps/frappe/frappe/utils/user.py", line 255, in load_user
13:03:28 web.1            |     d.all_reports = self.get_all_reports()
13:03:28 web.1            |   File "apps/frappe/frappe/utils/user.py", line 259, in get_all_reports
13:03:28 web.1            |     return get_allowed_reports()
13:03:28 web.1            |   File "apps/frappe/frappe/boot.py", line 140, in get_allowed_reports
13:03:28 web.1            |     return get_user_pages_or_reports("Report", cache=cache)
13:03:28 web.1            |   File "apps/frappe/frappe/boot.py", line 243, in get_user_pages_or_reports
13:03:28 web.1            |     has_role[report.name]["report_type"] = report.report_type
13:03:28 web.1            | KeyError: 'e-Invoice Summary'

snv · August 18, 2022, 11:09am

e-Invoice Summary is standard report in India Compliance. For some reason, it’s not available in the has_role dictionary, can you try printing it?

PhantomKunai · August 30, 2022, 9:35am

Hello,

I apologies for the late reply. For some reason I was never able to replicate this scenario and this dictionary error never occurred.

Regarding the actual issue, the solution you posted worked perfectly.

Thank you very much.