Ok Thanks. I was able to successfully set it up. We intend to use an AWS Auto scaling group for our EC2 instances running Frappe so we needed to have a central server for Redis (ElastiCache), MariaDB (RDS) and file storage (S3) so that each replica in the auto scaling group can be in sync when running concurrently.
We have an architecture where we create new sites for different clients. We usually create new VMs (i.e EC2 instances) and install Frappe, ERPNext and our custom apps. To avoid a single point of failure we limited the number of sites that can be created on a single bench.
To have as many sites as possible on a single VM and allow auto-scaling to handle traffic and CPU spikes, we decided to run our servers behind a load balancer on an auto-scaling group. This would require building an image that has been installed with Frappe, ERPNext and our custom apps. The image when started by an auto-scaling group would mount a network filesystem (AWS EFS) shared by other VMs in the same auto-scaling group and all VMs would be connected to the same database (AWS RDS) database and Redis server (AWS ElastiCache ). This allows scaling to be seamless on handling traffic and CPU fluctuations and elastic disk space provided by the NFS volumes.
For the image, does it include the existing sites?
How if there is a new site created?
Is it included to the new auto-scaling created server or to the old one?
What type of load balancer are you using?
Yes. In terms of CI/CD, you can either make the apps part of the image which means every change pushed to the pipeline would build a new image using AWS CodeBuild or you can make apps part of the volume if you prefer to update code using AWS CodeDeploy tool.