Not sure why the linked thread has been closed when I’m sure that the community can contribute to their experience with Docker based ERPNext deployment.
I’ve myself used the raman/erpnext docker image which seems to be more recently updated and is based on the davidgu/erpnext image you mentioned.
Indeed it does not follow good practices of linking to external mysql/redis containers, but at least it recommends (like the davidgu image and a number of other Docker images out there, to store the /var/lib/mysql folder in a Docker data container.
Today I’m upgrading a 6.20.0 image to the latest master version (therefore I’m building the raman/erpnext docker image manually (as the last published image is from 12 days ago).
I expect it to just work, but I’m worried about whether database migrations are going to run and prepare myself to have to go inside the container and run a bench update or something.
Maybe of use for me to share how my updating experience went with the raman/erpnext image? I’ll also comment on the image here Docker
So despite the data container being linked, rerunning the build of the Docker image created new mysql passwords (in fact it makes me wonder whether everyone using the docker image without building it is actually using the same passwords… ). I had to bash into the container to fix this. Then I was able to run the bench update --patch migrations.
So my thoughts to make this image more useful would be to follow best practices. a Dockerfile which I think does this well is this Open Project docker repo GitHub - progtologist/docker-openproject The main COMMAND of the container is a start_application.sh script which checks if it’s the first time the image is ran (by checking a .set-complete file in the container). When an updated image (rebuilt or downloaded from the docker hub) is ran, it runs a setup.sh script which is idempotent and runs migrations as well as recompile assets, seeds the database - this is also idempotent and doesn’t change anything if the database already has data.
It would also work for the erpnext container’s password generation which would execute the password generation as part of the container’s first time running (so that it’s not in the image file) and wouldn’t create a new one on updates.
Hope this is useful to others and that the community will pick it up!
@rmehta@jmatsushita I did some experimenting on this and practically came to a similar solution (replacing FROM debian to FROM ubuntu:14.04).
I think we need some docker expertise (which I am lacking) to actually get something really good. I’d guess about 5 containers are required ro seperate all required services
I could imagine @David_Arnold & @mquandalle might have the docker wisdom needed. Would you be able to help?
i completely agree with your concerns regarding passwords and migrations but using bash script to start services isn’t compelling enough, services often breaks and we need to restart them time and again. using supervisor does the job better. using bash script to start supervisor could be a solution.
furthermore i have been using docker for quite some time and i find putting all applications in a single image/container and data as volume or in separate container to be more practical than using separate containers for every service unless you are dealing with large enough loads that could not be handled on a single server.
I am a complete novice with docker but what “they” recommend it’s to separate services and only run 1 process per container. Naturally such a single-service architecture is much less complex, but well I am not sure (without much knowledge of the matter as mentioned) whether it pays off to simplify things maybe on the wrong end.
To start a discussion …
can you draw a line up to which load the one-in-all image architecture you prefer is still fine?
if in the end we came to the conclusion that the single-service setup is most desirable … would you have the knowledge (and would be willing) to build such a complex structure?
@frahergal@alex_melkoff@ArtemBraga@mikhail.s sorry, to launching a blunt attack on you … I have noted that you @semilimes have started remarkable coding activities for the ERPNext universe recently (which I think has been welcomed highly in the Community).
Now the selfish part … I’d like to cannibalize on your expertize (which I lack) for a project I am a little stuck with. Feel free to just say ‘no’ if your priorities are different ones.
I am trying to get a docker micro service environment# going for ERPNext since a while but don’t see myself getting anywhere in the foreseeable future due to lack of expertize and time to gather such. Just a thought … would you be able and willing to assign some resources on this effort? I am sure the ERPNExt ecosystem would greatly benefit from it and maybe for someone who knows his docker it’s not even such a big thing to create.
#with `micro service environment’ I mean separate containers serving only one service bundled together with docker compose (or maybe Vagrant which seems to be popular here) to serve ERPNext to the user. I think those would be MariaDB, Nginx, redis, python 2.7, ERPNext, … plus some datacontainers probably
Hello @vrms! Thank you for your kind words about Semilimes activity. I would like to share my opinion about docker and microservices architecture.
As a developer I absolutely love concept of microservice architecture as it allows to build complex, stable and easily maintainable applications. Docker is a great tool that helps to manage such applications and provides clean, top notch solution for deployment.
I would be happy to see docker-based microservice solution for ERPNext and Frappe but unfortunately I think that it’s currently imposible to achieve. Microservice architecture is software development pattern, not just a feature of docker. In order to be be packaged to separated docker containers application’s architecture should be designed in such a way that allows that separation. And I think that it’s currently not true for ERPNext and Frappe. Maybe I’m wrong, I have only basic understanding of Frappe’s architecture so far.
It would be nice to see some comments from Frappe’s developers about that. They are the only one who can break down the whole stack into microservices.
thanks @alex_melkoff for your feedback. I am not sure whether you are right about ERPNext being the monolithic block you suspect it is.
If you look at the manual install instructions you can at least clearly see which components it is build from (you are probably aware if those). I would assume it should be possible to have those function together as separate services should be possible.
In this topic ERPNext in 2014 and in 2016 - #9 by chlarsen such separation (not docker specific though) is being discussed (and at least partially solved) I think. @chlarsen@anand where the main contributors to that issue. Can you two maybe share some insight on the separation of services here?
Actually, existence of this topic about redis quite vividly illustrates my point.
Configs are all over the place, sometimes hardcoded here and there, a lot of undocumented behavior under the hood… This partially achieved solution you were talking about looks more like a hack and proves that Frappe’s architecture was not intended to be used like that. And it’s just redis: we also have SocketIO and Celery workers…
We set default values for various ports, but you are free to change them via common_site_config.json
Even before this, only the socketio port was hardcoded. The rest were configurable.
Frappe/ERPNext’s installer was designed for a novice user to get started quickly. But we neglected the advance user. So, if you can start a single thread about manual install, I can pitch in and then the community can build a readme about manual install?
@vrms well, readme about manual install would be very helpful, that’s for sure. However I still think that solution based on microservices is a question of system architecture and it’s up to developers to provide that. Maybe I’ll give it a try some time, when I’ll be more familiar with Frappe/ERPNext. For now it seems to me that it will require some deeply invasive actions.
Basically, there is a file called “Procfile” in bench directory and it contains list of processes to be run on “bench start”. For multiservice solution we should be able to safely configure and run each process (+ nginx as reverse proxy) on different machines. I would also prefer to have RESTful API as a separated microservice.
@anand I found an article about high availability for ERPNext. There is a section called “The Ideal Architecture”. Do you think it’s possible to achieve that setup with current version of Frappe/ERPNext?
@alex_melkoff thanks so far for engaging in this discussion (also thanks @anand)
Even though the technicalities are beyond my level understanding it seems this is a little more tricky then imagined.
How about taking an Agile approach? I mean looking at which services are easy to separate right now, separating those and leaving the rest of the pack in a remaining ERPNext image rather then putting efforts to halt until the ‘big solution’ is possible?
something (just from the top of my head) like …
MariaDB
Nginx
ERPNext (incl. all the rest needed)
wouldn’t that be a way to make progress right away even if the ideal case can’t be achieved yet?
Naturally the simple approach of the ERPNext setup makes sense on some level and therefore shouldn’t be abandoned. Still I think that aiming at being able to separate services (and putting a dockerized environment in place) might be something to pursue (and with such also maybe start at opening new possibilities for deployment scenarios.
). I had to bash into the container to fix this. Then I was able to run the bench update --patch migrations.