Migrating from older docker custom images to 2023 frappe_docker custom image

@revant_one could you please help us on this issue?

In 2022, we have built custom docker images to our app as explained in GitHub - castlecraft/custom_containers

We recently checked that erpnext-worker and frappe-worker images in Docker Hub are no longer updated. Since Jan/2023 a new build process was available: frappe_docker/custom-apps.md at main · frappe/frappe_docker · GitHub

We have built new images using 2023 frappe_docker, but we are struggling to migrate data from old containers to new ones…

We tried to “reuse” same volumes from old containers in new one, but isn’t working.

We have a lot of sites already created in the older container structure, is there any way to do a “bulk” migration of all sites? What’s the best strategy to convert the old structure to this new one?

Thanks

@marco.aurelio Did you already read this document?

Main difference now is there’s no need for assets volume- they are built into the image. So if you are still using old assets volumes in docker-compose.yml, that will definitely cause a problem- you won’t get the new assets in your containers.

In general, you should be able to just use the new process (which you linked to) to create new images that have your custom apps “baked in” to the images. Then generate a new docker-compose.yml. Customize what you need to. Back your data first, but then you should be able to spin up your containers with the old ‘sites’ and ‘db’ volumes and have a working system.

I’m sure @revant_one might have something to add, but this was what worked for me, anyway.

1 Like

If you are not having too many sites in a particular project, then build custom image with the current version of GitHub - castlecraft/custom_containers.
Create a new project with newly created custom image(you need to update the image with your custom image in the compose.yaml something like “image: ghcr.io/lmnaslimited/lensdocker/lenshxm:${ERPNEXT_VERSION:?No ERPNext version set}”) and restore sites from old project to new project.

1 Like

I’ve created a docker-compose.yaml based on the new process, so these changes are naturally incorporated in my docker-compose.yaml . The problem is that new containers don’t work with previous volumes. I tried to “reuse” volumes from previous version with the new image, but I got a HTTP 500 error.

The problem is that I already have lots of sites created, so the task of backing up and restoring all sites is huge. My question is if there is any automated way to do this backup and restore task or if there is some way to “reuse” old volumes with the new image.

Are the source and target erpnext versions identical? Did you run bench migrate?

500 would be some kind of permissions error, wouldn’t it? If you examine any traceback and/or nginx logs as you attempt to access the pages, the solution might become quite obvious. Or if not, if you post them someone else might see what’s going on. But I don’t think it’s a problem inherent to the process.

I have started creating a CLI to do this. But that is only half baked. I started of with Bash script, then now doing with oclif.
It is now semi automated. Wherever not automated at least i am generating the command then executing them manually.

For example moving the backup files from the VM or local storage to the container, you can generate the command and then run in the vm manually

Primarily written for GCP.

We found that is unsafe to reuse old volumes, because of mariadb processes. The safest way seems to be doing backup and restoring all sites.

Good, we are on GCP, so this CLI would be useful. Please let us know when you have it ready.

old volumes should work as the mount location is same.

make sure you’ve executed bench --site all migrate (mentioned by trustedcomputer)

read more about skip failing patches with bench migrate --help

what did you find out? I was able to upgrade all my servers without backup → restore. I did take extra backup as precaution, didn’t use it to restore.

I’ve run bench --site all migrate , but it still on 500 error.

We had some issues with mariadb, it didn’t recognize correctly the old volume, causing database corruption, so we decided to go with backup and restore. We tested in some sites and it works.