Sharing how I easy upgrade using docker and postgres

Hello everyone!
I’m new to this Forum, and I come with a gift :gift: :tada:
I want to share with the community a novel and easy upgrade process that is working for me. I’ll add a few open questions, if anyone know an answer, please share.
As an example it’s from v14 to v15, but could be used for v16 I guess.

I started with a frappe v14 docker production stack I made and changed the db to postgres.
It runs only frappe and erpnext and doesn’t have any customization or other apps.

To upgrade it to v15, there are multiple roads, but one that I find easy is to simply use docker volumes:
You must choose a docker stack name different from the v14 stack name, in my case they are 2 stacks: prodv14 and prodv15.

On the host (the machine running docker):
Copy and tweak pwd.yml for the v15 image version.
Stop the v14 stack (to properly shutdown the db and just in case something breaks but it might work without stopping it). Anybody tried postgres container?
cp -R /var/lib/docker/volumes/prodv14_sites /var/lib/docker/volumes/prodv15_sites
cp -R /var/lib/docker/volumes/prodv14_db-data /var/lib/docker/volumes/prodv15_db-data
This way you work on a backup of your db, apps and sites and you keep everything v14 ready to run again.

Now, to run your v15 stack with the right name, simply pass it as argument:
docker-compose -f pwd_v15.yml -p prodv15 up -d
Since prodv15 is the name of the stack, it will USE (not create) the volumes named prodv15_sites and prodv15_db-data.

Since it’s postgres, the new server doesn’t have any issue running existing data. This “volume-forwarding” didn’t work with mariadb. User and database got dumped. Any idea why?

In the backend container, you justbench --site frontend migrate --skip-failing
You(ll get some errors though (see below).
But now it runs frappe and erpnext v15 well:
image

I noticed a few errors during migration:

'Error in query:\ncolumn "name" is in a primary key\n'
        TypeError: argument of type 'object' is not iterable
psycopg2.errors.InvalidTableDefinition: column "name" is in a primary key
...
'Error in query:\ninvalid input syntax for integer: "k9o0ge273k"\nLINE 2:      VALUES (\'k9o0ge273k\', \'Administrator\', \'2025-01-07 13:3...\n                     ^\n'
psycopg2.errors.InvalidTextRepresentation: invalid input syntax for integer: "k9o0ge273k"
'Syntax error in query:\nUPDATE "tabAsset" JOIN (SELECT SUM("tabGL Entry"."debit") "value","tabAsset"."name" "asset_name" FROM "tabGL Entry" JOIN "tabAsset" ON "tabGL Entry"."against_voucher"="tabAsset"."name" JOIN "tabAsset Category Account" ON "tabAsset Category Account"."parent"="tabAsset"."asset_category" AND "tabAsset Category Account"."company_name"="tabAsset"."company" JOIN "tabCompany" ON "tabCompany"."name"="tabAsset"."company" WHERE "tabGL Entry"."account"=coalesce("tabAsset Category Account"."depreciation_expense_account","tabCompany"."depreciation_expense_account") AND "tabGL Entry"."debit"<> \'0\' AND "tabGL Entry"."is_cancelled"= \'0\' AND "tabAsset"."docstatus"= \'1\' AND "tabAsset"."calculate_depreciation"= \'0\' GROUP BY "tabAsset"."name") "sq0" ON "sq0"."asset_name"="tabAsset"."name" SET "value_after_depreciation"="tabAsset"."value_after_depreciation"-"sq0"."value" WHERE "tabAsset"."docstatus"= \'1\' AND "tabAsset"."calculate_depreciation"= \'0\' '
psycopg2.errors.SyntaxError: syntax error at or near "JOIN"
...
'Syntax error in query:\nUPDATE "tabAsset Depreciation Schedule"\n        JOIN "tabAsset"\n        ON "tabAsset Depreciation Schedule"."asset"="tabAsset"."name"\n        SET\n            "tabAsset Depreciation Schedule"."gross_purchase_amount"="tabAsset"."gross_purchase_amount",\n            "tabAsset Depreciation Schedule"."opening_number_of_booked_depreciations"="tabAsset"."opening_number_of_booked_depreciations"\n        WHERE\n        (\n            "tabAsset Depreciation Schedule"."gross_purchase_amount"<>"tabAsset"."gross_purchase_amount"\n            OR\n            "tabAsset Depreciation Schedule"."opening_number_of_booked_depreciations"<>"tabAsset"."opening_number_of_booked_depreciations"\n        )\n        AND "tabAsset Depreciation Schedule"."docstatus"< \'2\' '
        TypeError: argument of type 'object' is not iterable
psycopg2.errors.SyntaxError: syntax error at or near "JOIN"
'Error in query:\ninvalid input syntax for integer: "38sg4eftqc"\nLINE 2:      VALUES (\'38sg4eftqc\', \'Administrator\', \'2025-01-07 13:3...\n                     ^\n'
    frappe.log_error("Failed to remove Exotel Integration.")
  File "apps/frappe/frappe/utils/error.py", line 76, in log_error
    return error_log.insert(ignore_permissions=True)
      error_log = <ErrorLog: 38sg4eftqc>
      self = <ErrorLog: 38sg4eftqc>
      self = <ErrorLog: 38sg4eftqc>
      d = {'name': '38sg4eftqc', 'owner': 'Administrator', 'creation': '2025-01-07 13:35:59.566522', 'modified': '2025-01-07 13:35:59.566522', 'modified_by': 'Administrator', 'docstatus': 0, 'idx': 0, 'seen': 0, 'reference_doctype': None, 'reference_name': None, 'method': 'Failed to remove Exotel Integration.', 'error': 'Traceback with variables (most recent call last):\n  File "apps/erpnext/erpnext/patches/v15_0/remove_exotel_integration.py", line 15, in execute\n    exotel = frappe.get_doc(SETTINGS_DOCTYPE)\n  File "apps/frappe/frappe/__init__.py", line 1291, in get_doc\n    doc = frappe.model.document.get_doc(*args, **kwargs)\n      args = (\'Exotel Settings\',)\n      kwargs = {}\n      frappe = <module \'frappe\' from \'apps/frappe/frappe/__init__.py\'>\n  File "apps/frappe/frappe/model/document.py", line 83, in get_doc\n    controller = get_controller(doctype)\n      args = (\'Exotel Settings\',)\n      kwargs = {}\n      doctype = \'Exotel Settings\'\n  File "apps/frappe/frappe/model/base...
      columns = ['name', 'owner', 'creation', 'modified', 'modified_by', 'docstatus', 'idx', 'seen', 'reference_doctype', 'reference_name', 'method', 'error', 'trace_id']
      query = 'INSERT INTO `tabError Log` (`name`, `owner`, `creation`, `modified`, `modified_by`, `docstatus`, `idx`, `seen`, `reference_doctype`, `reference_name`, `method`, `error`, `trace_id`)\n\t\t\t\t\tVALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) '
      query = 'INSERT INTO "tabError Log" ("name", "owner", "creation", "modified", "modified_by", "docstatus", "idx", "seen", "reference_doctype", "reference_name", "method", "error", "trace_id")\n\t\t\t\t\tVALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)'
psycopg2.errors.InvalidTextRepresentation: invalid input syntax for integer: "38sg4eftqc"
...
'Error in query:\nfor SELECT DISTINCT, ORDER BY expressions must appear in select list\nLINE 5:     order by "tabRepost Allowed Types"."modified" DESC\n                     ^\n'
        TypeError: argument of type 'object' is not iterable
psycopg2.errors.InvalidColumnReference: for SELECT DISTINCT, ORDER BY expressions must appear in select list
'Error in query:\nfunction if(boolean, numeric, numeric) does not exist\nLINE 3:             debit_in_transaction_currency = IF(transaction_c...\n                                                    ^\nHINT:  No function matches the given name and argument types. You might need to add explicit type casts.\n'
...
File "/home/frappe/frappe-bench/apps/frappe/frappe/database/database.py", line 252, in sql
    traceback.print_stack()
        TypeError: argument of type 'object' is not iterable
psycopg2.errors.UndefinedFunction: function if(boolean, numeric, numeric) does not exist
'Error in query:\ncolumn "%\'purchase_receipt_amount\'%" does not exist\nLINE 2:   where doctype=\'Asset\' and data like "%\'purchase_receipt_am...\n                                              ^\n'
psycopg2.errors.UndefinedColumn: column "%'purchase_receipt_amount'%" does not exist
'Error in query:\ncolumn "%\'number_of_depreciations_booked\'%" does not exist\nLINE 2:   where doctype=\'Asset\' and data like "%\'number_of_depreciat...\n                                              ^\n'
psycopg2.errors.UndefinedColumn: column "%'number_of_depreciations_booked'%" does not exist
'Syntax error in query:\nUPDATE "tabAsset Repair Consumed Item" ar_item\n\t\t\tJOIN "tabAsset Repair" ar\n\t\t\tON ar.name = ar_item.parent\n\t\t\tSET ar_item.warehouse = ar.warehouse\n\t\t\tWHERE coalesce(ar.warehouse, \'\') != \'\' '
        TypeError: argument of type 'object' is not iterable
psycopg2.errors.SyntaxError: syntax error at or near "JOIN"
'Syntax error in query:\nUPDATE "tabStock Entry" JOIN "tabAsset Repair" ON "tabStock Entry"."name"="tabAsset Repair"."stock_entry" SET "asset_repair"="tabAsset Repair"."name" '
...
File "/home/frappe/frappe-bench/apps/frappe/frappe/database/database.py", line 252, in sql
    traceback.print_stack()
psycopg2.errors.SyntaxError: syntax error at or near "JOIN"
'Syntax error in query:\nUPDATE\n\t\t"tabGL Entry" as gle\n\t\tINNER JOIN "tabSales Invoice" as si\n\t\tON gle.voucher_type = \'Sales Invoice\' AND gle.voucher_no = si.name AND gle.remarks = %(remarks)s\n\t\tSET\n\t\tgle.remarks = concat(\'Against Customer Order \', si.po_no)\n\t\tWHERE si.po_no <> \'\' AND si.docstatus = %(docstatus)s and si.remarks = %(remarks)s {\'remarks\': \'No Remarks\', \'docstatus\': \'1\'}'
psycopg2.errors.SyntaxError: syntax error at or near "INNER"
'Syntax error in query:\nALTER TABLE "tabStock Ledger Entry" DROP INDEX "posting_datetime_creation_index" '
'Error in query:\ncurrent transaction is aborted, commands ignored until end of transaction block\n'
    frappe.log_error("Failed to drop index")
  File "apps/frappe/frappe/utils/error.py", line 61, in log_error
    error_log = frappe.get_doc(
      kwargs = {'doctype': 'Error Log', 'error': 'Traceback with variables (most recent call last):\n  File "apps/erpnext/erpnext/patches/v15_0/drop_index_posting_datetime_from_sle.py", line 13, in execute\n    frappe.db.sql_ddl(f"ALTER TABLE `{table}` DROP INDEX `{index}`")\n      table = \'tabStock Ledger Entry\'\n      index = \'posting_datetime_creation_index\'\n  File "apps/frappe/frappe/database/database.py", line 413, in sql_ddl\n    self.sql(query, debug=debug)\n      self = <frappe.database.postgres.database.PostgresDatabase object at 0x7bb711799cd0>\n      query = \'ALTER TABLE `tabStock Ledger Entry` DROP INDEX `posting_datetime_creation_index`\'\n      debug = False\n  File "apps/frappe/frappe/database/postgres/database.py", line 220, in sql\n    return super().sql(modify_query(query), modify_values(values), *args, **kwargs)\n      self = <frappe.database.postgres.database.PostgresDatabase object at 0x7bb711799cd0>\n      query = \'ALTER TABLE `tabStock Ledger Entry` DROP INDEX `posting_d...
      kwargs = {'doctype': 'Error Log', 'error': 'Traceback with variables (most recent call last):\n  File "apps/erpnext/erpnext/patches/v15_0/drop_index_posting_datetime_from_sle.py", line 13, in execute\n    frappe.db.sql_ddl(f"ALTER TABLE `{table}` DROP INDEX `{index}`")\n      table = \'tabStock Ledger Entry\'\n      index = \'posting_datetime_creation_index\'\n  File "apps/frappe/frappe/database/database.py", line 413, in sql_ddl\n    self.sql(query, debug=debug)\n      self = <frappe.database.postgres.database.PostgresDatabase object at 0x7bb711799cd0>\n      query = \'ALTER TABLE `tabStock Ledger Entry` DROP INDEX `posting_datetime_creation_index`\'\n      debug = False\n  File "apps/frappe/frappe/database/postgres/database.py", line 220, in sql\n    return super().sql(modify_query(query), modify_values(values), *args, **kwargs)\n      self = <frappe.database.postgres.database.PostgresDatabase object at 0x7bb711799cd0>\n      query = \'ALTER TABLE `tabStock Ledger Entry` DROP INDEX `posting_d...
...
filters = 'Error Log'
      query = SELECT * FROM "tabDocType" WHERE "name"='Error Log' ORDER BY "modified" DESC LIMIT 1
      params = {'param1': 'Error Log'}
      values = {'param1': 'Error Log'}
      values = {'param1': 'Error Log'}
psycopg2.errors.InFailedSqlTransaction: current transaction is aborted, commands ignored until end of transaction block
...
'Syntax error in query:\nUPDATE "tabAsset Maintenance Log" JOIN "tabAsset Maintenance Task" ON "tabAsset Maintenance Log"."task"="tabAsset Maintenance Task"."name" SET "task_assignee_email"="tabAsset Maintenance Task"."assign_to" '
'Error in query:\ncurrent transaction is aborted, commands ignored until end of transaction block\n'
    frappe.log_error("Failed to update Task Assignee Email Field.")
  File "apps/frappe/frappe/utils/error.py", line 61, in log_error
    error_log = frappe.get_doc(
      kwargs = {'doctype': 'Error Log', 'error': 'Traceback with variables (most recent call last):\n  File "apps/erpnext/erpnext/patches/v15_0/update_task_assignee_email_field_in_asset_maintenance_log.py", line 15, in execute\n    .run()\n      asset_maintenance_log = Table(\'tabAsset Maintenance Log\')\n      asset_maintenance_task = Table(\'tabAsset Maintenance Task\')\n  File "apps/frappe/frappe/query_builder/utils.py", line 87, in execute_query\n    result = frappe.db.sql(query, params, *args, **kwargs)  # nosemgrep\n      query = \'UPDATE "tabAsset Maintenance Log" JOIN "tabAsset Maintenance Task" ON "tabAsset Maintenance Log"."task"="tabAsset Maintenance Task"."name" SET "task_assignee_email"="tabAsset Maintenance Task"."assign_to"\'\n      args = ()\n      kwargs = {}\n      child_queries = []\n      params = {}\n      execute_child_queries = <function patch_query_execute.<locals>.execute_child_queries at 0x7bb71119e3e0>\n      prepare_query = <function patch_query_execute.<locals>.prepare_qu...
      kwargs = {'doctype': 'Error Log', 'error': 'Traceback with variables (most recent call last):\n  File "apps/erpnext/erpnext/patches/v15_0/update_task_assignee_email_field_in_asset_maintenance_log.py", line 15, in execute\n    .run()\n      asset_maintenance_log = Table(\'tabAsset Maintenance Log\')\n      asset_maintenance_task = Table(\'tabAsset Maintenance Task\')\n  File "apps/frappe/frappe/query_builder/utils.py", line 87, in execute_query\n    result = frappe.db.sql(query, params, *args, **kwargs)  # nosemgrep\n      query = \'UPDATE "tabAsset Maintenance Log" JOIN "tabAsset Maintenance Task" ON "tabAsset Maintenance Log"."task"="tabAsset Maintenance Task"."name" SET "task_assignee_email"="tabAsset Maintenance Task"."assign_to"\'\n      args = ()\n      kwargs = {}\n      child_queries = []\n      params = {}\n      execute_child_queries = <function patch_query_execute.<locals>.execute_child_queries at 0x7bb71119e3e0>\n      prepare_query = <function patch_query_execute.<locals>.prepare_qu...
...
File "/home/frappe/frappe-bench/apps/frappe/frappe/database/database.py", line 230, in sql
    self._cursor.execute(query, values)
psycopg2.errors.InvalidTextRepresentation: invalid input syntax for integer: "8051jgfh7r"
LINE 2:      VALUES ('8051jgfh7r', 'Administrator', '2025-01-07 13:3...
                     ^

'Error in query:\ninvalid input syntax for integer: "8051jgfh7r"\nLINE 2:      VALUES (\'8051jgfh7r\', \'Administrator\', \'2025-01-07 13:3...\n                     ^\n'

Do you know if it may break frappe/erpnext from working?

Hope it helps you!

1 Like