Help! Moving docker instalation from one server to another

[revant_one]

@nikkilocke

I started with a plain erpnext install. Then we realised we needed the notforprofit app. So I rebuilt to add that, and it didn’t work. That’s because the rebuild created a new volume with the new apps.json in it, but docker-compose up ignored that, and used the old volume, with the old apps.json in it.

apps.txt is important, ignore sites/apps.json

frappe/frappe_docker/blob/ce272a9290742befa8fa4ae1b6c73162fb23c266/pwd.yml#L23-L23

Above command updates the apps.txt with apps from image.

apps.json has git details about apps. In container images apps are not git repos. .git dir is removed

frappe/frappe_docker/blob/ce272a9290742befa8fa4ae1b6c73162fb23c266/images/custom/Containerfile#L122-L122

For updating static assets do not specify assets under volumes in your compose.yaml, refer pwd.yml

1. Stop containers
2. Remove stopped containers
3. remove hash / unnamed docker volume (docker volume ls|rm) if your sites is in bind mount on host directory then you can just do docker volume prune and it’ll remove volumes not attached to containers
4. Restart containers with new image, it’ll create new unnamed/hash volumes for new static assets.

To add or remove apps, run command to update apps.txt and bench install-app uninstall-app. Sequence:

To add app

1. Update images using above steps
2. update apps.txt
3. bench install-app

To remove

1. bench uninstall-app
2. Update image
3. update apps.txt

Other notes:

create-site is run once, it’s there in PWD for demo.
You need configurator and migration, refer custom_containers/compose/erpnext.yml at main · castlecraft/custom_containers · GitHub for migration. Refer PWD for configurator. Configurator needs to run when you change db, redis hosts or any common config or you need to update apps.txt

You can remove them after their job is done. Or the docker-compose exec command can be used to avoid adding run once services

[nikkilocke]

Thanks @revant_one - that’s really helpful.

I hadn’t got the rebuild of apps.txt line in my docker-compose - I guess I must have made mine more than 4 months ago!

Will it do any harm to rerun the build container on every restart? Or do I have to take it out of the docker-compose, or put some test in it so it does nothing for an existing site?

Any offers of wisdom on this question? How to restore a backup from one site name to another - #5 by nikkilocke

[revant_one]

@nikkilocke

Will it do any harm to rerun the build container on every restart? Or do I have to take it out of the docker-compose, or put some test in it so it does nothing for an existing site?

What do we mean by “build”

• if build is the “configurator” and “migration” service ideally keep it separate so even on server restarts it doesn’t trigger them. If you are okay with them running over and over again. Or you managed to script it in for when to run, then keep it in compose yaml
• if build means image build service, then ideally build your image with a name and tag, it will be visible locally, execute docker images to list local images. Then you can use the image in your compose, if the {name}:{tag} exists locally it’ll not complain or try to pull from remote and use the local image. This way you don’t trigger builds on server restart.

[nikkilocke]

I should have been more clear, sorry - I mean the containers in the pwd.yml called configurator and create-site. Will it do any harm to run them on an existing, working setup? e.g. when the server is rebooted, or the site is taken down for some reason and then restarted?

[revant_one]

create site will fail with “site already exists”.

configurator will run and update files, unnecessary io on disk.

migration, will do bench migrate, site will be unavailable for some time.

[nikkilocke]

OK, so it won’t actually break anything. I will leave them in, because without them people have to know how to (and remember to) do these steps manually sometimes.

[revant_one]

the configurator should complete successfully and nothing else should try to read/write to same json.

it causes error if multiple processes try to update json

Service "configurator" didn't completed successfully: exit 1

configurator should be run once. make sure your common_site_config.json or site_config.json are correct and valid json

[nikkilocke]

@revant_one

the configurator should complete successfully and nothing else should try to read/write to same json.

I’m not quite sure what that means. I am running a docker-compose file based very closely on the pwd.yml one you provide.

Is that wrong in some way?

[revant_one]

@nikkilocke

Is that wrong in some way?

PWD is for demo. It includes all services that need to run to have the site running. There are no restarts expected in this environment.

[nikkilocke]

Sorry, @revant_one , I’m still totally unclear what should be in my docker-compose file, then.

What I am looking for is a single docker-compose file that I can start up, and be sure that I will end up with a working system. It should still function if I:

• Rebuild the container, to get the latest code
• Restore a backup from another instance, possibly running a different version
• Build from scratch with empty volumes
• Start or build with existing volumes

I’m trying to make this foolproof (and failing miserably!). Perhaps because I am an inventive fool

[revant_one]

Safest alternative is to remove run once services from compose and run them as single task using docker run

frappe/frappe_docker/blob/main/docs/single-compose-setup.md#run-once-configuration

Refer container basics for docker run command

If you use immutable image tags then once configured the system can restart without any change.

Build image should be separate. May be create a compose for it separately.

If you don’t want immutable image tags, prefer use of branches instead of tags, no plan to scale bench across servers then struggle so much over native bench and it can do everything with single command bench update