@rmehta@jmatsushita I did some experimenting on this and practically came to a similar solution (replacing FROM debian to FROM ubuntu:14.04).
I think we need some docker expertise (which I am lacking) to actually get something really good. I’d guess about 5 containers are required ro seperate all required services
I could imagine @David_Arnold & @mquandalle might have the docker wisdom needed. Would you be able to help?
i completely agree with your concerns regarding passwords and migrations but using bash script to start services isn’t compelling enough, services often breaks and we need to restart them time and again. using supervisor does the job better. using bash script to start supervisor could be a solution.
furthermore i have been using docker for quite some time and i find putting all applications in a single image/container and data as volume or in separate container to be more practical than using separate containers for every service unless you are dealing with large enough loads that could not be handled on a single server.
I am a complete novice with docker but what “they” recommend it’s to separate services and only run 1 process per container. Naturally such a single-service architecture is much less complex, but well I am not sure (without much knowledge of the matter as mentioned) whether it pays off to simplify things maybe on the wrong end.
To start a discussion …
can you draw a line up to which load the one-in-all image architecture you prefer is still fine?
if in the end we came to the conclusion that the single-service setup is most desirable … would you have the knowledge (and would be willing) to build such a complex structure?
@frahergal@alex_melkoff@ArtemBraga@mikhail.s sorry, to launching a blunt attack on you … I have noted that you @semilimes have started remarkable coding activities for the ERPNext universe recently (which I think has been welcomed highly in the Community).
Now the selfish part … I’d like to cannibalize on your expertize (which I lack) for a project I am a little stuck with. Feel free to just say ‘no’ if your priorities are different ones.
I am trying to get a docker micro service environment# going for ERPNext since a while but don’t see myself getting anywhere in the foreseeable future due to lack of expertize and time to gather such. Just a thought … would you be able and willing to assign some resources on this effort? I am sure the ERPNExt ecosystem would greatly benefit from it and maybe for someone who knows his docker it’s not even such a big thing to create.
#with `micro service environment’ I mean separate containers serving only one service bundled together with docker compose (or maybe Vagrant which seems to be popular here) to serve ERPNext to the user. I think those would be MariaDB, Nginx, redis, python 2.7, ERPNext, … plus some datacontainers probably
Hello @vrms! Thank you for your kind words about Semilimes activity. I would like to share my opinion about docker and microservices architecture.
As a developer I absolutely love concept of microservice architecture as it allows to build complex, stable and easily maintainable applications. Docker is a great tool that helps to manage such applications and provides clean, top notch solution for deployment.
I would be happy to see docker-based microservice solution for ERPNext and Frappe but unfortunately I think that it’s currently imposible to achieve. Microservice architecture is software development pattern, not just a feature of docker. In order to be be packaged to separated docker containers application’s architecture should be designed in such a way that allows that separation. And I think that it’s currently not true for ERPNext and Frappe. Maybe I’m wrong, I have only basic understanding of Frappe’s architecture so far.
It would be nice to see some comments from Frappe’s developers about that. They are the only one who can break down the whole stack into microservices.
thanks @alex_melkoff for your feedback. I am not sure whether you are right about ERPNext being the monolithic block you suspect it is.
If you look at the manual install instructions you can at least clearly see which components it is build from (you are probably aware if those). I would assume it should be possible to have those function together as separate services should be possible.
In this topic ERPNext in 2014 and in 2016 - #9 by chlarsen such separation (not docker specific though) is being discussed (and at least partially solved) I think. @chlarsen@anand where the main contributors to that issue. Can you two maybe share some insight on the separation of services here?
Actually, existence of this topic about redis quite vividly illustrates my point.
Configs are all over the place, sometimes hardcoded here and there, a lot of undocumented behavior under the hood… This partially achieved solution you were talking about looks more like a hack and proves that Frappe’s architecture was not intended to be used like that. And it’s just redis: we also have SocketIO and Celery workers…
We set default values for various ports, but you are free to change them via common_site_config.json
Even before this, only the socketio port was hardcoded. The rest were configurable.
Frappe/ERPNext’s installer was designed for a novice user to get started quickly. But we neglected the advance user. So, if you can start a single thread about manual install, I can pitch in and then the community can build a readme about manual install?
@vrms well, readme about manual install would be very helpful, that’s for sure. However I still think that solution based on microservices is a question of system architecture and it’s up to developers to provide that. Maybe I’ll give it a try some time, when I’ll be more familiar with Frappe/ERPNext. For now it seems to me that it will require some deeply invasive actions.
Basically, there is a file called “Procfile” in bench directory and it contains list of processes to be run on “bench start”. For multiservice solution we should be able to safely configure and run each process (+ nginx as reverse proxy) on different machines. I would also prefer to have RESTful API as a separated microservice.
@anand I found an article about high availability for ERPNext. There is a section called “The Ideal Architecture”. Do you think it’s possible to achieve that setup with current version of Frappe/ERPNext?
@alex_melkoff thanks so far for engaging in this discussion (also thanks @anand)
Even though the technicalities are beyond my level understanding it seems this is a little more tricky then imagined.
How about taking an Agile approach? I mean looking at which services are easy to separate right now, separating those and leaving the rest of the pack in a remaining ERPNext image rather then putting efforts to halt until the ‘big solution’ is possible?
something (just from the top of my head) like …
MariaDB
Nginx
ERPNext (incl. all the rest needed)
wouldn’t that be a way to make progress right away even if the ideal case can’t be achieved yet?
Naturally the simple approach of the ERPNext setup makes sense on some level and therefore shouldn’t be abandoned. Still I think that aiming at being able to separate services (and putting a dockerized environment in place) might be something to pursue (and with such also maybe start at opening new possibilities for deployment scenarios.
I have just tried this with success after a lot of wrangling. The images haven’t been updated in a couple of month so the latest version is still version 7 in beta, but I was able to make it work with each service in a different container (with the multi-container images based on alpine).
I’m having a problem to update the image right now because bench update --ugrade expects to have access to the redis-server which isn’t the case in a multi-container setup.
