Redundancy in deployment

Hi everyone, I currently have two bare-metal servers which I plan to put 18.04 LTS on and would like them to mirror each other, so that there is redundancy when we go to production (and possibly add a third server as well). What is the best way to achieve this? If we have one server die again I’d like the other one to be able to take over, but don’t know how to configure the ERPNext software to allow this. The reason I ask is that I had set up a test server a year ago and had to attend to other things, in the meantime the computer died and I had to re-start from scratch. If this had been the actual production server we would have lost data, beyond the issue of downtime. What are current best-practice methods to prevent this from happening?

@authum Follow this guide by @bkm.

Poor Man’s Guide to Backup System

Set up two servers and configure the backup system to replicate on the backup server. If the primary server ever goes down it will only take a few minutes to restore the most recent backup and redirect to your backup server to production. You could set the backup to automatically restore to the backup server but that is not advisable because if your issue is data corruption it may mess up the backup server as well so you would want to give yourself a few minutes to query the backup data before restoring to your backup server.

I followed the guide to setup mine and I rest easy knowing that it’s almost impossible for both servers to crash at the same time. To be on a safe side, I set up extra parameters for s3 hourly backups for safekeeping to ensure that if in the off chance both servers crash at the same time (which is highly unlikely as they are hosted by different providers), I can restore to at worst one hour before the crash if I have to start from scratch.

2 Likes

I’ll have to look into that, right now I have two old workstations running 18.04 here and haven’t really looked into renting baremetal server space. In any case, thank you for all that! I’ll be looking for that guide once I get the first one up and running again, I am trying to use the install script but it has failed to complete the install for me and I’m not sure what’s going on, as per my other topic. Thank you for the informative reply!

Welcome @authum to the ERPNext Discussion board and the community at large!!

You will find a great deal of information using the search functions of the forum. Please take time to read all that you can. You will see many other users like yourself helping each other.

Sometimes it is just finding documentation, and sometimes it is trying to understand a “use case” that has not been discussed before. Whatever it may be, there is “usually” someone here that can at least point you in the right direction. As you learn more in your own ERPNext discovery process please also give back to the community when you see a question that you can answer. You never know… you may be the only one that has experienced a process before and your answer then can help everyone.

Thank you for joining us as we all learn more…

BKM :sunglasses:

1 Like

Bare metal servers are great for keeping your system secure internally to your company or group.

However, learning to install erpnext is sometimes tougher than you first expect. If you are trying to learn this on a bare metal server and you have an issue (or make a mistake) then stripping the server to start over is quite time consuming. I did this at first and became quickly discouraged.

Consider getting yourself a free trial at Google Cloud Platform and use that as your installation learning tool. You can delete a server instance in seconds and spin up a fresh Ubuntu serer in less than 2 minutes if you want to start over. This will quickly make you more comfortable with trying different paths through the install process because recovery is so simple.

When you have learned the best way to install your system , then you use that knowledge to install on your bare metal servers. You will save tons of time in the learning curve and you won’t be wanting to bounce a server down the steps after a bad day. :wink:

If GCP is not offering the free trial in your location ,then try looking at lowendbox.com for a cheap KVM type VPS server so you can do the same thing. There are some servers there that you can get for less than $10 US for a month and get all of your learning over faster.

Hope this helps…

BKM

3 Likes

For redundancy you should look at the load balancing features of nginx. I don’t know enough to advise you how to do it, but one of the projects I work on has a few servers, with nginx as a load balancer, and that serves to redirect traffic to another inline site if one of the servers are not available.

For mysql, a master slave architecture will work.

1 Like

I think the MySQL master-slave setup will be the critical part. If I have to manually do something in order to make sure that the site comes back up, that’s fine provided that the data itself is protected. I guess having an externally hosted server would make it easy to access the data from multiple locations, which would be nice; and it certainly makes more sense than to invest in new hardware. I’m just using old workstations that we have lying around for now and if I run into resource constraints with the hardware I’ll probably look into that hosting. That website is awesome!

You should then distinguish between redudancy and a good backup strategy, There are some resources in this channel for backup scripts to google, dropbox. etc.