How to deploy a database from development to production?

Install / Update Deployment
21

For a moment, I’m going to set aside things like containers.

Fundamentally, every ERPNext environment consists of a few things:

  • A database server, with 1 SQL database per Site.
  • Python and JS code, stored (mostly) in plain text files. These files are located within '../frappe-bench/apps/<app_name>/'
  • Some configuration data (e.g. 'common_site_config.json', ‘site_config.json’)
  • Some public and private files (usually attachments like .pdf, .doc, .xlsx, .txt)
  • The prerequisite software (MariaDB, Python, Node.js, Redis, Nginx, wkhtmltopdf, and more)

How to copy a Database from one environment to another:

  1. Create a database backup file on the source device. (e.g. using the 'mysqldump' tool)
  2. Copy that backup file to your target device.
  3. Restore the backup file on your target device’s database.
  4. Run the command 'bench migrate' to synchronize the newly-restored database with your target system’s schema requirements.

NOTE: The SQL database contains -mostly- business data. But there are important exceptions:

  • It might contain configurations that are specific to an environment (e.g. 3rd Party integration API keys)
  • It might have rows that contain code scripts (Python, JavaScript), that might only be compatible with certain environments.

Because of the above, you may require extra steps after restoring your database.

For example, one of my clients has a custom Mailchimp integration. There are different DocType values (configurations) for Production versus non-Production. I don’t want a non-Production environment to accidentally transmit emails to customers or suppliers! :grimacing:

So after restoring a database (but before starting ERPNext) it may be important to review certain configurations or scripts, and alter them. Frappe has some features that help with this (Fixtures). You can also write your own scripts to help automate this, as part of your database deployment process.

How to copy Code from one environment to another:

ERPNext “code” can exist in 2 places:

  • In a plain-text file, on disk.
  • Inside a SQL table, as mentioned in the section above.

To copy the former, the most common method is by using 'git' commands. Every ERPNext ‘App’ has its own git repository. When we make code changes in an environment, we ‘commit’ and ‘push’ those changes to GitHub/GitLab. On another device, we can ‘pull’ those changes downward.

The Frappe and ERPNext “code” is really nothing more than plain-text Python and JavaScript. While you’d have to be extremely careful and diligent, an experienced administrator “could” just copy from one environment to another. Or use a tool like ‘rsync’ to help.

There are very important considerations. Each App has it’s own 3rd party package dependencies (Python requires PyPi packages, JavaScript requires NPM packages). These must be correct per environment.

However, most people use a combination of ‘git’ and ‘Bench’ commands to simplify deployment.

How to copy Configuration or Files from one environment to another:

These are ordinary files. So you can just copy and paste. Use a tool like ‘rsync’ to handle the copying. Whatever you want.

If you don’t actually need the latest files and attachments in an environment, then skip this step.

How I Do It

I write a bunch of Bash, Python, and SQL scripts to help me automate this. For example:

  • Pulling the latest code into Production.
  • Moving Production data into my Staging environment.
  • Clean up cache files
  • Rebuild certain tables from scratch.

Whether you’re on bare metal, a cloud VPS, or inside a Docker container, the fundamentals are all the same: text files and SQL data. There aren’t (yet) any compiled languages in the workflow.

1 Like
22

@adelalmajed if you are talking about migrating existed customizations you can actually do that by creating the app then , exporting all customizations to this app using (bench export-fixtures --site sitename) , you have to write some code in hooks , and the changes must be throught customizations not editing doctypes , new doctypes should be changed to the new app module. just keep learning about apps and you will find out that all you need could be done .

23

Thank you @brian_pond , your reply seems full of details that I will need later, and you have provided me with a hint to start learning about a tool called ‘rsync’ which seems will help me somehow. Therefore I bookmarked this page.

24

Thank you @bahaou for your help.

I don’t have any existing customization nor a real active instance running ERPnext yet.
I’m just planning and preparing to start implementing ERPnext for a start-up business and trying to cover all expected issues that I might get into before officially working with ERPnext. And I feel that I’m almost there.
There is still something I don’t understand, and I can’t ask about anything before learning and applying the method you suggest.

Therefore, I can’t say anything now, but many thanks for your help.