I think it says there the best way to deploy is something called a swarm.
Don’t have such a thing!
We are just deploying using docker-compose (the version which comes with our Debian - not the new one you all seem to use).
We don’t have the option of deploying the latest docker, or a swarm (whatever that is).
It would be really helpful to understand why there are 3 volumes created in your container (sites, sites/assets and logs), and why it is configured in such a way as to cause this problem.
If you have 3 volumes you can remove the assets volume. logs is optional.
You need to go through this to understand the change in volumes frappe_docker/docs/migrate-from-multi-image-setup.md at main · frappe/frappe_docker · GitHub
pwd.yml explained frappe_docker/docs/single-compose-setup.md at main · frappe/frappe_docker · GitHub
pwd.yml frappe_docker/pwd.yml at main · frappe/frappe_docker · GitHub
once your volumes for assets are not bind mounted or specified in containers, It’ll auto-create volumes and mount with new created containers.
I think docker-compose up re-creates new containers.
Debian Bookworm was released recently. Check if it gives you newer version of software.
If you’re moving to new server, move to updated debian stable.
I have looked at all those, and am none the wiser. I guess I’m thicker than I think I am!
In the Containerfile for custom (for that is what I must start with - I want to install apps like notforprofit), there is the following code:
VOLUME [ \
"/home/frappe/frappe-bench/sites", \
"/home/frappe/frappe-bench/sites/assets", \
"/home/frappe/frappe-bench/logs" \
]
If I understand correctly, that will create 3 persistent docker volumes the first time the container is built, and the data will remain there for subsequent builds.
Is there a reason for the assets volume?
Also, I decided, as an experiment, to try to use the standard frappe-custom image mentioned in the repo - only to find it no longer exists
pull access denied for frappe-custom, repository does not exist or may require 'docker login': denied: requested access to the resource is denied
There was no such image built
If you specify the mount then it does that.
If you don’t specify any type of mount then it’ll create new unnamed volume whenever the container is recreated.
So how do I follow the instructions (e.g. frappe_docker/docs/list-of-containers.md at main · frappe/frappe_docker · GitHub and Best way to deploy new versions of custom app in self hosted docker setup ) to use it, if it doesn’t exist?
In which file do I “specify the mount”?
In the container build file (Dockerfile, in my version)?
In the docker-compose?
In both?
Sorry to be so ignorant about how docker works. I have read pages and pages of documentation, but seem to have learned nothing - I have no model in my mind of how it works, so most of it is just magic to me!
On a related note, is the frontend container necessary? We have nginx running on the host - if we adapted the nginx.conf file from the frontend container into a nginx conf file on the host, could we not bother with the frontend container? It seems inefficient to have 2 versions of nginx running, the host proxying to the one in frontend, which presumably proxies to the one in backend and/or websocket.
I didn’t get what you’re trying to follow
If you’re trying to build image with custom apps then follow frappe_docker/docs/custom-apps.md at main · frappe/frappe_docker · GitHub
In compose yaml, under volumes section. Don’t specify anything for assets.
If your nginx can serve static assets from some running container, reverse proxy gunicorn and websocket, serve public and private files per site. Then try it without frontend containers.
So I have to build a container using plain docker? I am used to just using docker-compose build.
The instructions say I have to push it to docker hub - but I find (by experiment) that I don’t, it will use the local image,
I don’t understand why ERPNext is so difficult to deploy. Is it just that there haven’t been enough resources to make it easier, or is there some fundamental problem with making a docker image you can just pull and run?
But my experiment seems to have failed - I built the new container, and apps.json still only has frappe in it.
Could that be a left-over from having an existing sites volume? Should I delete the sites volume before building? Or before bringing the container up for the first time?
By the way, and very importantly, thank you for all your help.
Whatever needs to be there for building images, pushing them or not pushing them, FAQ, docker compose, docs is out there in public.
There were few people who were able to build and use custom images.
I’m out of ideas to help you, may be someone who has successfully setup erpnext with your constraints might help.
So most people couldn’t get it to work?
I haven’t yet found an overview of how docker and docker-compose work - I find manuals, and read them, but without a better understanding of how and why, it is difficult.
Will do some more experiments on this tomorrow.
Seems like it. Only I use containers. No one else answers.
There are countless working site running on containers.
More than 100 that I can remember I’ve setup. I stopped keeping count. No one will answer here. If no one answers here means no one uses containers. Or containers are so reliable that no one has any problem!
Kubernetes/Helm chart users are separate. There are many setups running on Kubernetes. Many I’ve helped upgrade when they ask on frappe/helm repo. One one will answer here.
In past few days someone faced problem building image with payments, it was reported and fixed, user forked payments repo and used it. Countless people use it and don’t even report back.
I don’t think you’ll find the following posts useful. Or I guess they are not comprehensive enough for users to understand anything.
I really can’t help you much.
I’m one of those (few?) people who got erpnext working with containers on regular docker-compose. =)
I think lots of people have problems when they think it should be super easy, then find out it isn’t. Docker (and projects that promote using docker to deploy) tend to get that advertising, “It’s so easy, why doesn’t everyone just use docker?” etc.
Real life, that’s not really true. Every kind of technology needs to be learned. That’s true for networking, coding (even “easy” languages), virtualization, containers, you name it. There are no shortcuts. If you want to utilize a technology, you have to put in the time to learn it. Sorry. It’s true.
I started with ERPNext on an Ubuntu VM since that’s what most people use, because I knew already how to deploy virtual machines, and because that’s what the documentation was mostly written for. Then I got bitten by the “I want to learn Docker” bug. I tried deploying erpnext that way, but I had problems since I was shorcutting the process of learning Docker slowly and properly.
Eventually I gave into the concept that I would actually have to put in the time learning how Docker works and experimenting with it before expecting to do something useful with it. Docker/containers are complicated and there’s a lot to learn: images, custom builds, compose files, networking, volumes, etc.
I’m still not an expert, but I know enough to get what I want working now. That “Container Basics” link @revant_one posted would be a great first stop. Then try to deploy a vanilla erpnext install from the “single server example” in the frappe_docker github without customizing anything. Just follow the steps and see if the login page shows up. Like anything worth doing, this takes time!
I can build and install plain erpnext using docker-compose at least one time in two (I have found a problem where the assets.json file gets out of step with the randomly generated asset names, usually every other build). Especially now I have realised you need to migrate if you are using an existing database, because the code might have changed (I added that into the create-site container in my docker-compose.yml, and it helped a lot).
But I keep hitting issues caused by my not understanding docker properly (especially volumes).
I started with a plain erpnext install. Then we realised we needed the notforprofit app. So I rebuilt to add that, and it didn’t work. That’s because the rebuild created a new volume with the new apps.json in it, but docker-compose up ignored that, and used the old volume, with the old apps.json in it.
In a plug-and-play dockerised app, there would be some code that recognised the new build was connecting with an old volume, and set about upgrading anything in the old volume that needed it.
So I just deleted my sites volume, and re-deployed. Now the new frappe can’t connect to the database, because it has created a new site_config.json, with a new random database password, but it hasn’t granted access to the frappe user with that password.
All very fine if you fully understand frappe, and are prepared to patch files in your volume after running it, but not exactly plug and play.
So I edited the database password in the sites volume to be the same as the old one. Now it connects, but there is still no sign of the apps in the site. Apps.json is correct.
Looking at the create-site logs, I find ERPNext can only be installed on a fresh site where the setup wizard is not completed.. So it looks as if I have to trash my database volume too. But how would you go about installing a new app to an existing site?
I’m writing this down both to explain my problems, and to enlighten anyone else who comes along later and experiences any of these issues.
apps.txt is important, ignore sites/apps.json
Above command updates the apps.txt with apps from image.
apps.json has git details about apps. In container images apps are not git repos. .git dir is removed
For updating static assets do not specify assets under volumes in your compose.yaml, refer pwd.yml
docker volume ls|rm) if your sites is in bind mount on host directory then you can just do docker volume prune and it’ll remove volumes not attached to containersTo add or remove apps, run command to update apps.txt and bench install-app uninstall-app. Sequence:
To add app
To remove
Other notes:
create-site is run once, it’s there in PWD for demo.
You need configurator and migration, refer custom_containers/compose/erpnext.yml at main · castlecraft/custom_containers · GitHub for migration. Refer PWD for configurator. Configurator needs to run when you change db, redis hosts or any common config or you need to update apps.txt
You can remove them after their job is done. Or the docker-compose exec command can be used to avoid adding run once services
Thanks @revant_one - that’s really helpful.
I hadn’t got the rebuild of apps.txt line in my docker-compose - I guess I must have made mine more than 4 months ago!
Will it do any harm to rerun the build container on every restart? Or do I have to take it out of the docker-compose, or put some test in it so it does nothing for an existing site?
Any offers of wisdom on this question? How to restore a backup from one site name to another - #5 by nikkilocke
What do we mean by “build”
docker images to list local images. Then you can use the image in your compose, if the {name}:{tag} exists locally it’ll not complain or try to pull from remote and use the local image. This way you don’t trigger builds on server restart.I should have been more clear, sorry - I mean the containers in the pwd.yml called configurator and create-site. Will it do any harm to run them on an existing, working setup? e.g. when the server is rebooted, or the site is taken down for some reason and then restarted?
create site will fail with “site already exists”.
configurator will run and update files, unnecessary io on disk.
migration, will do bench migrate, site will be unavailable for some time.