Upgrade from v13 to v14

I tried to upgrade from ERPNext: v13.34.2 / Frappe Framework: v13.34.0 to v14 on my docker installation.

After the upgrade i can’t access the frontend any more (500 Internal server error).

I’m getting exceptions after doing this in my docker-based setup (with a lot of followup exceptions):

/home/frappe/frappe-bench/apps/frappe/frappe/database/query.py:197: UserWarning: The 'filters_config' hook used to add custom operators is not yet implemented in frappe.db.query engine. Use db_query (frappe.get_list) instead.
warn(
[2022-08-03 11:10:09 +0000] [9] [ERROR] Error handling request /
Traceback (most recent call last):
File "/home/frappe/frappe-bench/apps/frappe/frappe/website/serve.py", line 18, in get_response
response = renderer_instance.render() ...

Also duringbench migrate several exceptions occure, but continuing and reporting success at the end. However the UI keeps unaccesible with the same error.

Any suggestion how to solve?

3 Likes

Any help or infor would be appreciated.

can you describe how did you do upgrade(commands)? I’m also having difficulties to upgrade, but I have errors with sql query.

Try commands from list, one after another:

  • bench migrate
  • bench switch-to-branch version-14 --upgrade
  • update --reset
  • bench restart

Did you install aditional apps like payments, hrms non_profit?

1 Like

Thank you for your respond.
In ma docker installation i opened a console of the backend container and used the following commands:

bench --site testsite.lan set-config -p maintenance_mode 1
bench --site testsite.lan set-config -p pause_scheduler 1
bench --site testsite.lan migrate
bench --site testsite.lan set-config -p pause_scheduler 0
bench --site testsite.lan set-config -p maintenance_mode 0

The migrate command printed a punch of SQL error messages which i’m not sure if they are relevant or just because of not found tables or content.

The site in browser remains unaccesible with internal server errors (because of logged python errors on the backend container)
If i remove the asset volume and recreate this volume with the next start of v14 I can spin of the installation and until now seems to work well with all data available.

For the time being I managed to (potential) sucessful upgrade by

  • Ignoring the SQL error messages
  • removing the asset volume and let the next docker compuse up manage to recreate this volume

I’m not sure if ignoring and deleting/recreate asset volume is acceptible or if it will bring issues.

I did not run the switch-to-branch command - I will give it a try, if this prevents of the step to remove the asset volume

Try commands from list, one after another:

  • bench migrate
  • bench switch-to-branch version-14 --upgrade
  • update --reset
  • bench restart

Those commands are not applicable with a docker based installation

bench migrate wants a parameter --site set

All the next commands are not available in a docker based installation:
bench switch-to-branch version-14 --upgrade
update --reset
bench restart

1 Like

also interessted in a solution to upgrade the docker containers

Hey,
I faced the same issue and decided to do my migration “by hand”

I installed a new container with frappe v14 on a side, backuped my v13 database through bench backup.

I restored the database with command psql mytable < mybackup

V14 comes with new tables, so I created it with this script:

-- Table: public."tabDocType State"

-- DROP TABLE public."tabDocType State";

CREATE TABLE public."tabDocType State"
(
    name character varying(140) COLLATE pg_catalog."default" NOT NULL,
    creation timestamp(6) without time zone,
    modified timestamp(6) without time zone,
    modified_by character varying(140) COLLATE pg_catalog."default",
    owner character varying(140) COLLATE pg_catalog."default",
    docstatus smallint NOT NULL DEFAULT '0'::smallint,
    idx bigint NOT NULL DEFAULT '0'::bigint,
    title character varying(140) COLLATE pg_catalog."default",
    color character varying(140) COLLATE pg_catalog."default" DEFAULT 'Blue'::character varying,
    custom smallint NOT NULL DEFAULT 0,
    parent character varying(140) COLLATE pg_catalog."default",
    parentfield character varying(140) COLLATE pg_catalog."default",
    parenttype character varying(140) COLLATE pg_catalog."default",
    CONSTRAINT "tabDocType State_pkey" PRIMARY KEY (name)
)

TABLESPACE pg_default;

ALTER TABLE public."tabDocType State"
    OWNER to _2de17b05f9ed9da2;
	
-- Table: public."tabData Import Log"

-- DROP TABLE public."tabData Import Log";

CREATE TABLE public."tabData Import Log"
(
    name character varying(140) COLLATE pg_catalog."default" NOT NULL,
    creation timestamp(6) without time zone,
    modified timestamp(6) without time zone,
    modified_by character varying(140) COLLATE pg_catalog."default",
    owner character varying(140) COLLATE pg_catalog."default",
    docstatus smallint NOT NULL DEFAULT '0'::smallint,
    idx bigint NOT NULL DEFAULT '0'::bigint,
    data_import character varying(140) COLLATE pg_catalog."default",
    row_indexes text COLLATE pg_catalog."default",
    success smallint NOT NULL DEFAULT 0,
    docname character varying(140) COLLATE pg_catalog."default",
    messages text COLLATE pg_catalog."default",
    exception text COLLATE pg_catalog."default",
    log_index bigint NOT NULL DEFAULT 0,
    _user_tags text COLLATE pg_catalog."default",
    _comments text COLLATE pg_catalog."default",
    _assign text COLLATE pg_catalog."default",
    _liked_by text COLLATE pg_catalog."default",
    CONSTRAINT "tabData Import Log_pkey" PRIMARY KEY (name)
)

TABLESPACE pg_default;

ALTER TABLE public."tabData Import Log"
    OWNER to _2de17b05f9ed9da2;
	
	
-- Table: public."tabDocument Share Key"

-- DROP TABLE public."tabDocument Share Key";

CREATE TABLE public."tabDocument Share Key"
(
    name character varying(140) COLLATE pg_catalog."default" NOT NULL,
    creation timestamp(6) without time zone,
    modified timestamp(6) without time zone,
    modified_by character varying(140) COLLATE pg_catalog."default",
    owner character varying(140) COLLATE pg_catalog."default",
    docstatus smallint NOT NULL DEFAULT '0'::smallint,
    idx bigint NOT NULL DEFAULT '0'::bigint,
    reference_doctype character varying(140) COLLATE pg_catalog."default",
    reference_docname character varying(140) COLLATE pg_catalog."default",
    key character varying(140) COLLATE pg_catalog."default",
    expires_on date,
    _user_tags text COLLATE pg_catalog."default",
    _comments text COLLATE pg_catalog."default",
    _assign text COLLATE pg_catalog."default",
    _liked_by text COLLATE pg_catalog."default",
    CONSTRAINT "tabDocument Share Key_pkey" PRIMARY KEY (name)
)

TABLESPACE pg_default;

ALTER TABLE public."tabDocument Share Key"
    OWNER to _2de17b05f9ed9da2;

-- Index: reference_docname

-- DROP INDEX public.reference_docname;

CREATE INDEX reference_docname
    ON public."tabDocument Share Key" USING btree
    (reference_docname COLLATE pg_catalog."default")
    TABLESPACE pg_default;

-- Table: public."tabIMAP Folder"

-- DROP TABLE public."tabIMAP Folder";

CREATE TABLE public."tabIMAP Folder"
(
    name character varying(140) COLLATE pg_catalog."default" NOT NULL,
    creation timestamp(6) without time zone,
    modified timestamp(6) without time zone,
    modified_by character varying(140) COLLATE pg_catalog."default",
    owner character varying(140) COLLATE pg_catalog."default",
    docstatus smallint NOT NULL DEFAULT '0'::smallint,
    idx bigint NOT NULL DEFAULT '0'::bigint,
    folder_name character varying(140) COLLATE pg_catalog."default",
    append_to character varying(140) COLLATE pg_catalog."default",
    uidvalidity character varying(140) COLLATE pg_catalog."default",
    uidnext character varying(140) COLLATE pg_catalog."default",
    parent character varying(140) COLLATE pg_catalog."default",
    parentfield character varying(140) COLLATE pg_catalog."default",
    parenttype character varying(140) COLLATE pg_catalog."default",
    CONSTRAINT "tabIMAP Folder_pkey" PRIMARY KEY (name)
)

TABLESPACE pg_default;

ALTER TABLE public."tabIMAP Folder"
    OWNER to _2de17b05f9ed9da2;

-- Table: public."tabLogs To Clear"

-- DROP TABLE public."tabLogs To Clear";

CREATE TABLE public."tabLogs To Clear"
(
    name character varying(140) COLLATE pg_catalog."default" NOT NULL,
    creation timestamp(6) without time zone,
    modified timestamp(6) without time zone,
    modified_by character varying(140) COLLATE pg_catalog."default",
    owner character varying(140) COLLATE pg_catalog."default",
    docstatus smallint NOT NULL DEFAULT '0'::smallint,
    idx bigint NOT NULL DEFAULT '0'::bigint,
    ref_doctype character varying(140) COLLATE pg_catalog."default",
    days bigint NOT NULL DEFAULT 30,
    parent character varying(140) COLLATE pg_catalog."default",
    parentfield character varying(140) COLLATE pg_catalog."default",
    parenttype character varying(140) COLLATE pg_catalog."default",
    CONSTRAINT "tabLogs To Clear_pkey" PRIMARY KEY (name)
)

TABLESPACE pg_default;

ALTER TABLE public."tabLogs To Clear"
    OWNER to _2de17b05f9ed9da2;
	
-- Table: public."tabNewsletter Attachment"

-- DROP TABLE public."tabNewsletter Attachment";

CREATE TABLE public."tabNewsletter Attachment"
(
    name character varying(140) COLLATE pg_catalog."default" NOT NULL,
    creation timestamp(6) without time zone,
    modified timestamp(6) without time zone,
    modified_by character varying(140) COLLATE pg_catalog."default",
    owner character varying(140) COLLATE pg_catalog."default",
    docstatus smallint NOT NULL DEFAULT '0'::smallint,
    idx bigint NOT NULL DEFAULT '0'::bigint,
    attachment text COLLATE pg_catalog."default",
    parent character varying(140) COLLATE pg_catalog."default",
    parentfield character varying(140) COLLATE pg_catalog."default",
    parenttype character varying(140) COLLATE pg_catalog."default",
    CONSTRAINT "tabNewsletter Attachment_pkey" PRIMARY KEY (name)
)

TABLESPACE pg_default;

ALTER TABLE public."tabNewsletter Attachment"
    OWNER to _2de17b05f9ed9da2;


-- Table: public."tabPackage"

-- DROP TABLE public."tabPackage";

CREATE TABLE public."tabPackage"
(
    name character varying(140) COLLATE pg_catalog."default" NOT NULL,
    creation timestamp(6) without time zone,
    modified timestamp(6) without time zone,
    modified_by character varying(140) COLLATE pg_catalog."default",
    owner character varying(140) COLLATE pg_catalog."default",
    docstatus smallint NOT NULL DEFAULT '0'::smallint,
    idx bigint NOT NULL DEFAULT '0'::bigint,
    package_name character varying(140) COLLATE pg_catalog."default",
    readme text COLLATE pg_catalog."default",
    license_type character varying(140) COLLATE pg_catalog."default",
    license text COLLATE pg_catalog."default",
    _user_tags text COLLATE pg_catalog."default",
    _comments text COLLATE pg_catalog."default",
    _assign text COLLATE pg_catalog."default",
    _liked_by text COLLATE pg_catalog."default",
    CONSTRAINT "tabPackage_pkey" PRIMARY KEY (name)
)

TABLESPACE pg_default;

ALTER TABLE public."tabPackage"
    OWNER to _2de17b05f9ed9da2;


-- Table: public."tabPackage Import"

-- DROP TABLE public."tabPackage Import";

CREATE TABLE public."tabPackage Import"
(
    name character varying(140) COLLATE pg_catalog."default" NOT NULL,
    creation timestamp(6) without time zone,
    modified timestamp(6) without time zone,
    modified_by character varying(140) COLLATE pg_catalog."default",
    owner character varying(140) COLLATE pg_catalog."default",
    docstatus smallint NOT NULL DEFAULT '0'::smallint,
    idx bigint NOT NULL DEFAULT '0'::bigint,
    attach_package text COLLATE pg_catalog."default",
    activate smallint NOT NULL DEFAULT 0,
    force smallint NOT NULL DEFAULT 0,
    log text COLLATE pg_catalog."default",
    _user_tags text COLLATE pg_catalog."default",
    _comments text COLLATE pg_catalog."default",
    _assign text COLLATE pg_catalog."default",
    _liked_by text COLLATE pg_catalog."default",
    CONSTRAINT "tabPackage Import_pkey" PRIMARY KEY (name)
)

TABLESPACE pg_default;

ALTER TABLE public."tabPackage Import"
    OWNER to _2de17b05f9ed9da2;

-- Table: public."tabPackage Release"

-- DROP TABLE public."tabPackage Release";

CREATE TABLE public."tabPackage Release"
(
    name character varying(140) COLLATE pg_catalog."default" NOT NULL,
    creation timestamp(6) without time zone,
    modified timestamp(6) without time zone,
    modified_by character varying(140) COLLATE pg_catalog."default",
    owner character varying(140) COLLATE pg_catalog."default",
    docstatus smallint NOT NULL DEFAULT '0'::smallint,
    idx bigint NOT NULL DEFAULT '0'::bigint,
    "package" character varying(140) COLLATE pg_catalog."default",
    publish smallint NOT NULL DEFAULT 0,
    path text COLLATE pg_catalog."default",
    major bigint NOT NULL DEFAULT 0,
    minor bigint NOT NULL DEFAULT 0,
    patch bigint NOT NULL DEFAULT 0,
    release_notes text COLLATE pg_catalog."default",
    _user_tags text COLLATE pg_catalog."default",
    _comments text COLLATE pg_catalog."default",
    _assign text COLLATE pg_catalog."default",
    _liked_by text COLLATE pg_catalog."default",
    CONSTRAINT "tabPackage Release_pkey" PRIMARY KEY (name)
)

TABLESPACE pg_default;

ALTER TABLE public."tabPackage Release"
    OWNER to _2de17b05f9ed9da2;

-- Table: public."tabPrint Format Field Template"

-- DROP TABLE public."tabPrint Format Field Template";

CREATE TABLE public."tabPrint Format Field Template"
(
    name character varying(140) COLLATE pg_catalog."default" NOT NULL,
    creation timestamp(6) without time zone,
    modified timestamp(6) without time zone,
    modified_by character varying(140) COLLATE pg_catalog."default",
    owner character varying(140) COLLATE pg_catalog."default",
    docstatus smallint NOT NULL DEFAULT '0'::smallint,
    idx bigint NOT NULL DEFAULT '0'::bigint,
    document_type character varying(140) COLLATE pg_catalog."default",
    field character varying(140) COLLATE pg_catalog."default",
    template_file character varying(140) COLLATE pg_catalog."default",
    module character varying(140) COLLATE pg_catalog."default",
    standard smallint NOT NULL DEFAULT 0,
    template text COLLATE pg_catalog."default",
    _user_tags text COLLATE pg_catalog."default",
    _comments text COLLATE pg_catalog."default",
    _assign text COLLATE pg_catalog."default",
    _liked_by text COLLATE pg_catalog."default",
    CONSTRAINT "tabPrint Format Field Template_pkey" PRIMARY KEY (name)
)

TABLESPACE pg_default;

ALTER TABLE public."tabPrint Format Field Template"
    OWNER to _2de17b05f9ed9da2;

-- Table: public."tabWeb Form List Column"

-- DROP TABLE public."tabWeb Form List Column";

CREATE TABLE public."tabWeb Form List Column"
(
    name bigint NOT NULL,
    creation timestamp(6) without time zone,
    modified timestamp(6) without time zone,
    modified_by character varying(140) COLLATE pg_catalog."default",
    owner character varying(140) COLLATE pg_catalog."default",
    docstatus smallint NOT NULL DEFAULT '0'::smallint,
    idx bigint NOT NULL DEFAULT '0'::bigint,
    fieldname character varying(140) COLLATE pg_catalog."default",
    fieldtype character varying(140) COLLATE pg_catalog."default",
    label character varying(140) COLLATE pg_catalog."default",
    parent character varying(140) COLLATE pg_catalog."default",
    parentfield character varying(140) COLLATE pg_catalog."default",
    parenttype character varying(140) COLLATE pg_catalog."default",
    CONSTRAINT "tabWeb Form List Column_pkey" PRIMARY KEY (name)
)

TABLESPACE pg_default;

ALTER TABLE public."tabWeb Form List Column"
    OWNER to _2de17b05f9ed9da2;

-- Table: public."tabWorkflow Action Permitted Role"

-- DROP TABLE public."tabWorkflow Action Permitted Role";

CREATE TABLE public."tabWorkflow Action Permitted Role"
(
    name character varying(140) COLLATE pg_catalog."default" NOT NULL,
    creation timestamp(6) without time zone,
    modified timestamp(6) without time zone,
    modified_by character varying(140) COLLATE pg_catalog."default",
    owner character varying(140) COLLATE pg_catalog."default",
    docstatus smallint NOT NULL DEFAULT '0'::smallint,
    idx bigint NOT NULL DEFAULT '0'::bigint,
    role character varying(140) COLLATE pg_catalog."default",
    parent character varying(140) COLLATE pg_catalog."default",
    parentfield character varying(140) COLLATE pg_catalog."default",
    parenttype character varying(140) COLLATE pg_catalog."default",
    CONSTRAINT "tabWorkflow Action Permitted Role_pkey" PRIMARY KEY (name)
)

TABLESPACE pg_default;

ALTER TABLE public."tabWorkflow Action Permitted Role"
    OWNER to _2de17b05f9ed9da2;

-- Table: public."tabWorkspace Quick List"

-- DROP TABLE public."tabWorkspace Quick List";

CREATE TABLE public."tabWorkspace Quick List"
(
    name character varying(140) COLLATE pg_catalog."default" NOT NULL,
    creation timestamp(6) without time zone,
    modified timestamp(6) without time zone,
    modified_by character varying(140) COLLATE pg_catalog."default",
    owner character varying(140) COLLATE pg_catalog."default",
    docstatus smallint NOT NULL DEFAULT '0'::smallint,
    idx bigint NOT NULL DEFAULT '0'::bigint,
    document_type character varying(140) COLLATE pg_catalog."default",
    label character varying(140) COLLATE pg_catalog."default",
    quick_list_filter text COLLATE pg_catalog."default",
    parent character varying(140) COLLATE pg_catalog."default",
    parentfield character varying(140) COLLATE pg_catalog."default",
    parenttype character varying(140) COLLATE pg_catalog."default",
    CONSTRAINT "tabWorkspace Quick List_pkey" PRIMARY KEY (name)
)

TABLESPACE pg_default;

ALTER TABLE public."tabWorkspace Quick List"
    OWNER to _2de17b05f9ed9da2;


Awesome!
I will try that tomorrrow and give Feedback but that Looks promising

Actually, it doesn’t work yet… I guess this is a necessary step, but I’m still facing issues with missing columns in the existing tables …

A migration script out of bench from the developper team would have been appreciated !

2 Likes

After attempting to upgrade the dockerized system from 13.42.0 to 14.7.0 (both ERPNext and Frappe, same versions), I’m getting the same error in a different file:

/usr/local/lib/python3.10/functools.py:981: UserWarning: The 'filters_config' hook used to add custom operators is not yet implemented in frappe.db.query engine. Use db_query (frappe.get_list) instead.
  val = self.func(instance)
[2022-11-21 06:22:00 +0000] [7] [ERROR] Error handling request /
Traceback (most recent call last):
  File "/home/frappe/frappe-bench/apps/frappe/frappe/website/serve.py", line 18, in get_response
    response = renderer_instance.render()

Still no solution???

For the record, this is what I did:

mariabackup ...
vim ~/gitops/erpnext-one.env
   FRAPPE_VERSION=v14.7.0
   ERPNEXT_VERSION=v14.7.0
docker compose --project-name erpnext-one   --env-file ~/gitops/erpnext-one.env   -f compose.yaml   -f overrides/compose.erpnext.yaml   -f overrides/compose.redis.yaml   -f docs/compose/compose.multi-bench.yaml config > ~/gitops/erpnext-one.yaml
docker compose --project-name erpnext-one -f ~/gitops/erpnext-one.yaml down
docker compose --project-name erpnext-one -f ~/gitops/erpnext-one.yaml up -d
docker compose --project-name erpnext-one exec backend bench --site SITE migrate

Any news regarding this upgrade?
I have 3 sties running v13 in Docker Swarm / Portainer and want to migrate them to v14.

Somehow I’m using v14 that was migrated from v13.

You will need to try migration locally to identify list of apps that you will need when you move to v14, e.g. India Compliance, HRMS, payments, Education etc.

I knew I only depended on India Compliance. So I didn’t even check local migrations, I just made a custom image with erpnext + payments + india_compliance.

Later removed payments when someone found out it was not needed Do not have the ERPNext components after docker deployment · Issue #1060 · frappe/frappe_docker · GitHub

Official migration documentation

ERPNext: Migration Guide to ERPNext version 14 · frappe/erpnext Wiki · GitHub
Frappe Framework: Migrating to Version 14 · frappe/frappe Wiki · GitHub

1 Like

Hi All

I have installed ERPNEXT v13 on Docker Swarm and Portainer following exactly the instructions made by @revant_one as per URL:

I have 4 sites up and running in Production (v13.52.7)

I do not have any custom image. Just couple of DocTypes added to 3 sites.

Unfortunately I can’t find any “step by step” migration guide to help in a smooth migration from v13 to v14 over Docker Swarm and Portainer.

I understand that the steps are:

1- “If required” Build custom images with required apps (in my case I need erpnext + hrms + ecommerce_integrations
I skipped this step.

2- I added the following lines to the “migration” service in the erpnext.yml used to deploy the erpnext stack:
as per URL

    bench --site all set-config -p maintenance_mode 1
    bench --site all set-config -p pause_scheduler 1
    # Start adding commands:
    bench --site all install-app erpnext
    bench --site all install-app hrms
    bench --site all install-app ecommerce_integrations
    # End of adding commands:
    bench --site all migrate
    bench --site all set-config -p maintenance_mode 0
    bench --site all set-config -p pause_scheduler 0

3- Change the environment variable “VERSION” to be : v14.35.0

Result:

  • The version was upgraded to v14.35.0 however, I lost the HR, Payroll and E-Commerce Apps .

Any ideas on how to add these apps ?

PS: in the migration service log I noticed that some apps were removed:

e.g.
Executing erpnext.patches.v14_0.remove_hr_and_payroll_modules # 20-07-2022 in nourerp.misrtech.com (_038dec5e1ffd9f28)

Please Help:

  • Any idea how to add the required apps (hrms and ecommerce_integrations) to this installation on Docker Swarm + Portainer ?

  • Do I also have to add the “payments” App

Thanks in advance

Dears
Any detailed instructions regarding how to safely upgrade Frappe/ERPNEXT + HRMS From V13 to v14 without loosing the Payroll data in Dockerswarm + Portainer environment ?
Thanks to all the community. #docker

  1. Build custom image with list of apps you need. e.g. hrms, education, etc.
  2. Use your custom image and do bench --site site.name.com migrate. (Change site.name.com to your site name)

Migration guides:

Thanks @revant_one

Then when I want to upgrade to a higher v14; will I just change the VERSION env variable in the erpnext stack (Portainer) or I will switch to CLI ?

Can I build the image, then pull it in the stack?

I didn’t know how to build v14 image with hrms.

Thanks,
Yasser Sedky

You will need to enter the container and install hrms app.

To build image check documentation. I’ve helped other multiple times on forum to build hrms image, you can search.

Thanks @revant_one
I went through

I will follow the instructions and will post my results.

One question please, after building the custom image (ERPNEXT + HRMS) and updating the apps.json (in the backend container already running v13)
Is there anyway to use the custom image that I created in the erpnext stack to do the magic from within Portainer ?

PS: I m using your compose/erpnext.yml as described in your documentation custom_containers/docs/docker-swarm.md at main · castlecraft/custom_containers · GitHub

In other words, can I replace the line:
image: frappe/erpnext:${VERSION:?No image version set}

by
x-customizable-image: &customizable_image
image: url-to-the-built-custom-image:version-no

And can I do this upgrade (using custom image in the Stack deployment) and adding some commands in the “migration” service section to get the app and install it?

No need of this line. It’s not in yaml?

Rest everything remains same and it’ll do migration on update