Migration from v12 to v13 error on Opportunity

Frappe Framework Bug
,
1

I try to migrate locally (into docker dev env) from v12.22.0 to 13.6.0

bench init --skip-redis-config-generation --frappe-branch version-12 frappe-bench
bench new-site erpnext13.test --mariadb-root-password 123 --admin-password admin --no-mariadb-socket --db-name erpnext13
bench get-app --branch version-12 erpnext GitHub - frappe/erpnext: Free and Open Source Enterprise Resource Planning (ERP)
bench --site erpnext13.test install-app erpnext
bench get-app --branch master_fix erpnext_france GitHub - FHenry/erpnext_france: ERPNext Localization App
bench get-app --branch master customerapp url_customerapp
bench --site erpnext13.test set-config developer_mode 1
bench --site erpnext13.test install-app erpnext
bench --site erpnext13.test clear-cache

into my customerapp I have a fixtures Property Setters on Opportunty that change default value for Opportunity-sales_stage-default and Opportunity-opportunity_type-default

I import backup from my production site

bench --site erpnext13.test restore --mariadb-root-username root --mariadb-root-password 123 ./sites/erpnext13.test/private/backups/XXX.sql

bench start

From here it ok, I can go on local application in v12

bench switch-to-branch version-13 frappe erpnext --upgrade
bench setup requirements
bench migrate

I face a problem on Opportunity doctype migration

Migrating erpnext13.test
Executing erpnext.patches.v13_0.rename_issue_doctype_fields in erpnext13.test (erpnext13)
rename_field: first_response_time not found in Opportunity
Traceback (most recent call last):
File “/usr/lib/python3.7/runpy.py”, line 193, in _run_module_as_main
“main”, mod_spec)
File “/usr/lib/python3.7/runpy.py”, line 85, in _run_code
exec(code, run_globals)
File “/workspace/development/frappe-bench/apps/frappe/frappe/utils/bench_helper.py”, line 104, in
main()
File “/workspace/development/frappe-bench/apps/frappe/frappe/utils/bench_helper.py”, line 19, in main
click.Group(commands=commands)(prog_name=‘bench’)
File “/workspace/development/frappe-bench/env/lib/python3.7/site-packages/click/core.py”, line 829, in call
return self.main(*args, **kwargs)
File “/workspace/development/frappe-bench/env/lib/python3.7/site-packages/click/core.py”, line 782, in main
rv = self.invoke(ctx)
File “/workspace/development/frappe-bench/env/lib/python3.7/site-packages/click/core.py”, line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/workspace/development/frappe-bench/env/lib/python3.7/site-packages/click/core.py”, line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/workspace/development/frappe-bench/env/lib/python3.7/site-packages/click/core.py”, line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File “/workspace/development/frappe-bench/env/lib/python3.7/site-packages/click/core.py”, line 610, in invoke
return callback(*args, **kwargs)
File “/workspace/development/frappe-bench/env/lib/python3.7/site-packages/click/decorators.py”, line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File “/workspace/development/frappe-bench/apps/frappe/frappe/commands/init.py”, line 27, in _func
ret = f(frappe._dict(ctx.obj), *args, **kwargs)
File “/workspace/development/frappe-bench/apps/frappe/frappe/commands/site.py”, line 309, in migrate
skip_search_index=skip_search_index
File “/workspace/development/frappe-bench/apps/frappe/frappe/migrate.py”, line 67, in migrate
frappe.modules.patch_handler.run_all(skip_failing)
File “/workspace/development/frappe-bench/apps/frappe/frappe/modules/patch_handler.py”, line 41, in run_all
run_patch(patch)
File “/workspace/development/frappe-bench/apps/frappe/frappe/modules/patch_handler.py”, line 30, in run_patch
if not run_single(patchmodule = patch):
File “/workspace/development/frappe-bench/apps/frappe/frappe/modules/patch_handler.py”, line 71, in run_single
return execute_patch(patchmodule, method, methodargs)
File “/workspace/development/frappe-bench/apps/frappe/frappe/modules/patch_handler.py”, line 91, in execute_patch
frappe.get_attr(patchmodule.split()[0] + “.execute”)()
File “/workspace/development/frappe-bench/apps/erpnext/erpnext/patches/v13_0/rename_issue_doctype_fields.py”, line 47, in execute
frappe.db.set_value(‘Opportunity’, entry.name, ‘first_response_time’, mins_to_first_response, update_modified=False)
File “/workspace/development/frappe-bench/apps/frappe/frappe/database/database.py”, line 667, in set_value
values, debug=debug)
File “/workspace/development/frappe-bench/apps/frappe/frappe/database/database.py”, line 146, in sql
self._cursor.execute(query, values)
File “/workspace/development/frappe-bench/env/lib/python3.7/site-packages/pymysql/cursors.py”, line 148, in execute
result = self._query(query)
File “/workspace/development/frappe-bench/env/lib/python3.7/site-packages/pymysql/cursors.py”, line 310, in _query
conn.query(q)
File “/workspace/development/frappe-bench/env/lib/python3.7/site-packages/pymysql/connections.py”, line 548, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File “/workspace/development/frappe-bench/env/lib/python3.7/site-packages/pymysql/connections.py”, line 775, in _read_query_result
result.read()
File “/workspace/development/frappe-bench/env/lib/python3.7/site-packages/pymysql/connections.py”, line 1156, in read
first_packet = self.connection._read_packet()
File “/workspace/development/frappe-bench/env/lib/python3.7/site-packages/pymysql/connections.py”, line 725, in _read_packet
packet.raise_for_error()
File “/workspace/development/frappe-bench/env/lib/python3.7/site-packages/pymysql/protocol.py”, line 221, in raise_for_error
err.raise_mysql_exception(self._data)
File “/workspace/development/frappe-bench/env/lib/python3.7/site-packages/pymysql/err.py”, line 143, in raise_mysql_exception
raise errorclass(errno, errval)
pymysql.err.OperationalError: (1054, “Unknown column ‘first_response_time’ in ‘field list’”)

If I check tabOpportunity the field mins_to_first_response is still here and not replace by first_response_time as it should be done by frappe.reload_doc(‘crm’, ‘doctype’, ‘opportunity’) in
erpnext/patches/v13_0/rename_issue_doctype_fields.py
And into tabDocField table there is also still mins_to_first_response instead of first_response_time

If anyone have a clue ?

2

I’ve just try a fresh new install on v12 without any customization,login, create an opporuntity.

set logging : 2 into common_site_confing.js

bench switch-to-branch version-13 frappe erpnext --upgrade

<<<< query’
‘INSERT INTO __UserSettings(user, doctype, data)\n\t\t\t\tVALUES ('Administrator', 'Company', '{\“updated_on\”: \“Sun Jul 04 2021 14:25:57 GMT+0200\”, \“List\”: {\“filters\”: , \“sort_by\”: \“modified\”, \“sort_order\”: \“desc\”}, \“last_view\”: \“List\”}')\n\t\t\t\tON DUPLICATE key UPDATE data='{\“updated_on\”: \“Sun Jul 04 2021 14:25:57 GMT+0200\”, \“List\”: {\“filters\”: , \“sort_by\”: \“modified\”, \“sort_order\”: \“desc\”}, \“last_view\”: \“List\”}'’
‘>>>>’
‘<<<< query’
“INSERT INTO __UserSettings(user, doctype, data)\n\t\t\t\tVALUES (‘Administrator’, ‘Opportunity’, ‘{}’)\n\t\t\t\tON DUPLICATE key UPDATE data=‘{}’”
‘>>>>’
‘<<<< query’
‘INSERT INTO __UserSettings(user, doctype, data)\n\t\t\t\tVALUES ('Administrator', 'Price List', '{\“updated_on\”: \“Sun Jul 04 2021 14:26:25 GMT+0200\”, \“last_view\”: \“List\”, \“List\”: {\“filters\”: [[\“Price List\”, \“currency\”, \“=\”, \“EUR\”]], \“sort_by\”: \“modified\”, \“sort_order\”: \“desc\”}}')\n\t\t\t\tON DUPLICATE key UPDATE data='{\“updated_on\”: \“Sun Jul 04 2021 14:26:25 GMT+0200\”, \“last_view\”: \“List\”, \“List\”: {\“filters\”: [[\“Price List\”, \“currency\”, \“=\”, \“EUR\”]], \“sort_by\”: \“modified\”, \“sort_order\”: \“desc\”}}'’
‘>>>>’
‘<<<< query’
‘INSERT INTO __UserSettings(user, doctype, data)\n\t\t\t\tVALUES ('Administrator', 'Quotation', '{\“updated_on\”: \“Sun Jul 04 2021 14:25:35 GMT+0200\”, \“last_view\”: \“List\”, \“List\”: {\“filters\”: , \“sort_by\”: \“modified\”, \“sort_order\”: \“desc\”}}')\n\t\t\t\tON DUPLICATE key UPDATE data='{\“updated_on\”: \“Sun Jul 04 2021 14:25:35 GMT+0200\”, \“last_view\”: \“List\”, \“List\”: {\“filters\”: , \“sort_by\”: \“modified\”, \“sort_order\”: \“desc\”}}'’
‘>>>>’
‘<<<< query’
’ select user, doctype, data from __UserSettings\n\t\twhere doctype='Issue' and data like “%'mins_to_first_response'%”’
‘>>>>’
‘<<<< query’
"select name from tabDocType where name = ‘Opportunity’ order by modified desc "
‘>>>>’
‘<<<< query’
"\n\t\t\t\tselect column_name\n\t\t\t\tfrom information_schema.columns\n\t\t\t\twhere table_name = ‘tabOpportunity’ "
‘>>>>’
‘<<<< query’
‘select name, mins_to_first_response\n\t\t\tfrom tabOpportunity\n\t\t\t\n\t\t\t\n\t\t\t order by creation desc\n\t\t\t’
‘>>>>’
‘<<<< query’
"select modified from tabDocType where name = ‘Opportunity’ order by modified desc "
‘>>>>’
‘<<<< query’
"select * from tabDocType where name = ‘Opportunity’ order by modified desc "
‘>>>>’
‘<<<< query’
"select * from tabDocField where parent = ‘Opportunity’ and parenttype = ‘DocType’ and parentfield = ‘fields’ order by idx asc "
‘>>>>’
‘<<<< query’
"select * from tabDocPerm where parent = ‘Opportunity’ and parenttype = ‘DocType’ and parentfield = ‘permissions’ order by idx asc "
‘>>>>’
‘<<<< query’
"select * from tabDocType Action where parent = ‘Opportunity’ and parenttype = ‘DocType’ and parentfield = ‘actions’ order by idx asc "
‘>>>>’
‘<<<< query’
"select * from tabDocType Link where parent = ‘Opportunity’ and parenttype = ‘DocType’ and parentfield = ‘links’ order by idx asc "
‘>>>>’
‘<<<< query’
“\n\t\t\tSELECT * FROM tabCustom Field\n\t\t\tWHERE dt = ‘Opportunity’ AND docstatus < 2\n\t\t”
‘>>>>’
‘<<<< query’
“select * from tabProperty Setter where\n\t\t\tdoc_type=‘Opportunity’”
‘>>>>’
‘<<<< query’
“select *\n\t\t\tfrom tabDocType Link\n\t\t\twhere tabDocType Link.parent = ‘Opportunity’ and tabDocType Link.custom = 1.0\n\t\t\t\n\t\t\t order by tabDocType Link.modified DESC\n\t\t\t”
‘>>>>’
‘<<<< query’
“select *\n\t\t\tfrom tabDocType Action\n\t\t\twhere tabDocType Action.parent = ‘Opportunity’ and tabDocType Action.custom = 1.0\n\t\t\t\n\t\t\t order by tabDocType Action.modified DESC\n\t\t\t”
‘>>>>’
rename_field: first_response_time not found in Opportunity
‘<<<< query’
“select name from tabOpportunity where name = ‘CRM-OPP-2021-00001’ order by modified desc for update”
‘>>>>’
‘<<<< query’
“update tabOpportunity\n\t\t\t\t\tset first_response_time=0.0e0 where name=‘CRM-OPP-2021-00001’”
‘>>>>’
‘<<<< query’
‘rollback’
‘>>>>’
‘<<<< query’
‘START TRANSACTION’
‘>>>>’
Traceback (most recent call last):
File “/usr/lib/python3.7/runpy.py”, line 193, in _run_module_as_main
“main”, mod_spec)
File “/usr/lib/python3.7/runpy.py”, line 85, in _run_code
exec(code, run_globals)
File “/workspace/development/frappe-benchtest/apps/frappe/frappe/utils/bench_helper.py”, line 104, in
main()
File “/workspace/development/frappe-benchtest/apps/frappe/frappe/utils/bench_helper.py”, line 19, in main
click.Group(commands=commands)(prog_name=‘bench’)
File “/workspace/development/frappe-benchtest/env/lib/python3.7/site-packages/click/core.py”, line 829, in call
return self.main(*args, **kwargs)
File “/workspace/development/frappe-benchtest/env/lib/python3.7/site-packages/click/core.py”, line 782, in main
rv = self.invoke(ctx)
File “/workspace/development/frappe-benchtest/env/lib/python3.7/site-packages/click/core.py”, line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/workspace/development/frappe-benchtest/env/lib/python3.7/site-packages/click/core.py”, line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/workspace/development/frappe-benchtest/env/lib/python3.7/site-packages/click/core.py”, line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File “/workspace/development/frappe-benchtest/env/lib/python3.7/site-packages/click/core.py”, line 610, in invoke
return callback(*args, **kwargs)
File “/workspace/development/frappe-benchtest/env/lib/python3.7/site-packages/click/decorators.py”, line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File “/workspace/development/frappe-benchtest/apps/frappe/frappe/commands/init.py”, line 27, in _func
ret = f(frappe._dict(ctx.obj), *args, **kwargs)
File “/workspace/development/frappe-benchtest/apps/frappe/frappe/commands/site.py”, line 309, in migrate
skip_search_index=skip_search_index
File “/workspace/development/frappe-benchtest/apps/frappe/frappe/migrate.py”, line 67, in migrate
frappe.modules.patch_handler.run_all(skip_failing)
File “/workspace/development/frappe-benchtest/apps/frappe/frappe/modules/patch_handler.py”, line 41, in run_all
run_patch(patch)
File “/workspace/development/frappe-benchtest/apps/frappe/frappe/modules/patch_handler.py”, line 30, in run_patch
if not run_single(patchmodule = patch):
File “/workspace/development/frappe-benchtest/apps/frappe/frappe/modules/patch_handler.py”, line 71, in run_single
return execute_patch(patchmodule, method, methodargs)
File “/workspace/development/frappe-benchtest/apps/frappe/frappe/modules/patch_handler.py”, line 91, in execute_patch
frappe.get_attr(patchmodule.split()[0] + “.execute”)()
File “/workspace/development/frappe-benchtest/apps/erpnext/erpnext/patches/v13_0/rename_issue_doctype_fields.py”, line 47, in execute
frappe.db.set_value(‘Opportunity’, entry.name, ‘first_response_time’, mins_to_first_response, update_modified=False)
File “/workspace/development/frappe-benchtest/apps/frappe/frappe/database/database.py”, line 667, in set_value
values, debug=debug)
File “/workspace/development/frappe-benchtest/apps/frappe/frappe/database/database.py”, line 146, in sql
self._cursor.execute(query, values)
File “/workspace/development/frappe-benchtest/env/lib/python3.7/site-packages/pymysql/cursors.py”, line 148, in execute
result = self._query(query)
File “/workspace/development/frappe-benchtest/env/lib/python3.7/site-packages/pymysql/cursors.py”, line 310, in _query
conn.query(q)
File “/workspace/development/frappe-benchtest/env/lib/python3.7/site-packages/pymysql/connections.py”, line 548, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File “/workspace/development/frappe-benchtest/env/lib/python3.7/site-packages/pymysql/connections.py”, line 775, in _read_query_result
result.read()
File “/workspace/development/frappe-benchtest/env/lib/python3.7/site-packages/pymysql/connections.py”, line 1156, in read
first_packet = self.connection._read_packet()
File “/workspace/development/frappe-benchtest/env/lib/python3.7/site-packages/pymysql/connections.py”, line 725, in _read_packet
packet.raise_for_error()
File “/workspace/development/frappe-benchtest/env/lib/python3.7/site-packages/pymysql/protocol.py”, line 221, in raise_for_error
err.raise_mysql_exception(self._data)
File “/workspace/development/frappe-benchtest/env/lib/python3.7/site-packages/pymysql/err.py”, line 143, in raise_mysql_exception
raise errorclass(errno, errval)
pymysql.err.OperationalError: (1054, “Unknown column ‘first_response_time’ in ‘field list’”)

I’ve created an issue on github BUG: Migration Error from v12 to v13 (opportunity Unknown column 'first_response_time') · Issue #26321 · frappe/erpnext · GitHub

3

Ok, found a solution after deep debug.
I lear a lot about how Frappe deal with database model change.

But anyway, please Frappe/ERPNext guys, check what your doing and merge in previous version… Normal user cannot deal with that, and Quality process into code base continuous integration is a must have

4

Attempting to update from 12 to 13, and after any bench update , with --patch – reset or a bench migrate without skip-failing the error:

pymysql.err.OperationalError: (1054, “Unknown column ‘first_response_time’ in ‘field list’”)

Can someone provide the correct bench console code to fix? ie:

frappe.reload_doc(…?..)

tried:

frappe.reload_doc(‘first_response_time’, “doctype”,‘Opportuntity’)

with result:

KeyError: ‘first_response_time’

and

frappe.db.get_list(‘Opportunity’, ‘first_response_time’)

results in:

OperationalError: (1054, “Unknown column ‘first_response_time’ in ‘field list’”

TIA

5

Befrore upgrade
UPDATE tabDocType SET modified=“2020-09-08 11:10:35.318872” WHERE name=“Opportunity”

6

Thank you @FHenry. It worked, although I applied the code in bench --site mariadb after the upgrade. There were other issues that were overcome but this one was rather more vexing. I did have to change the ’ to " in order for mariadb to execute it.

UPDATE tabDocType SET modified=“2020-09-08 11:10:35.318872” WHERE name=“Opportunity”;

7

I am facing similar issue when restoring version-14 db in a version-14 setup which is completely strange to me.

I had to comment out this line from patch list to migrate successfully.