Deployment on stateless containers

Install / Update Deployment
, , ,
1

End Goal

To have a scalable(and hence stateless) frappe setup with custom apps that works along with managed database, redis and s3

Context

We are trying to setup infrastructure for frappe framework based application where we have an app(krp). Following the frappe_docker/images/custom/Containerfile at main · frappe/frappe_docker · GitHub, we are able to generate a custom image which has our app(available at GitHub - yash0212/krp-base). We build the assets in the build stage in Dockerfile and store it in image for use at runtime. For first time setup, we run the first-run.sh script which connects to database and creates a new site.

Problems/Doubts:

  1. When a new app is added to the apps.json, it’ll be included in the custom docker image at build time and to run it’s migrations, we have migrate.sh script which initially did bench migrate only but we noticed apps were not getting installed, so we updated it to have bench install-apps command to install the apps to the site. The problem arise when we remove an app from apps.json, the migrate script which runs bench migrate fails as it checks the db for installed apps on the site and finds an app whose migration has to be checked but doesn’t exist anymore in the image. If we go with volume based containers as per frappe_docker/pwd.yml at main · frappe/frappe_docker · GitHub, we’ll still face the same problem when an app is uninstalled

  2. Since we have stateless containers, will multiple backend containers be able to manage the lock? as the server maintains file based lock

  3. Since we have stateless containers, does frappe refer to database.log and frappe.log to identify delta changes in the database? this might break as multiple containers will have different contents of database.log and frappe.log

Not preferred solution

Setting up frappe for deployment in stateful(volume backed) containers

If we go with volume based containers as per frappe_docker/pwd.yml at main · frappe/frappe_docker · GitHub and we have multiple containers running for backend.

Q: How is the database migration and switch to new image planned as if there is a new app added or change in existing app, the currently request serving containers wouldn’t have the updated logic as they’ll be based on old image. If containers are upgraded to new images then request serving might fail as the new changes are not yet migrated in the database