Local backups of dockerized install

francois072 · September 19, 2021, 11:07am

I have ERPNext installed on KDE Neon via Docker. I have an external HDD which I use for backups via Vorta. How should I configure my backups in such a way that I can easily restore the ERPNext settings and the database on another system?

revant_one · September 19, 2021, 12:18pm

check this if it helps GitHub - frappe/frappe_docker: Docker images for production and development setups of the Frappe framework and ERPNext

check this for site operations related to backup and restore frappe_docker/site-operations.md at main · frappe/frappe_docker · GitHub

francois072 · September 19, 2021, 1:04pm

These links have what I want, but I have no idea what to do with the information. It’s my first time using Docker, ERPNext and Frappe.

Are there any GUI options to create local backups? I found Portainer and thought that might work.

revant_one · September 20, 2021, 12:07am

portainer just gives you gui to manage docker swarm. You will still use the same Yaml files that are used without portainer. You can’t escape learning docker if you’re planning to use it.

sharing again : GitHub - frappe/frappe_docker: Docker images for production and development setups of the Frappe framework and ERPNext

This link contains stacks to setup cronjob on docker swarm. Add the stack using portainer ui or docker cli.

francois072 · September 20, 2021, 6:26am

Thanks I’ll look into it further.

What do I actually need to backup? I read up on the difference between containers, volumes and images, but I’m not sure if I need to backup the volumes or containers.

revant_one · September 20, 2021, 7:02am

For frappe/erpnext you need to backup

database dump
private files, public files and site_config.json files (the sites volume, sites-vol)

No need to backup containers. Only backup volumes.

francois072 · September 20, 2021, 5:57pm

This looks like it may be helpful.

The guide seems easy enough, but I can’t figure out how to ssh into my ERPNext installation at mysite.localhost.

If I can get this to work I might be able to get a script somewhere to perform the actions automatically.

revant_one · September 21, 2021, 12:05am

The guide is not related to dockerized setup.

to enter any container use

docker exec -it container-name bash

replace container name with erpnext-python container to exec into it and use bench_helper commands.

With docker, script should be written like the cron jobs examples shared before. scripts are additional containers that run and complete.

francois072 · September 21, 2021, 6:52am

After executing docker exec -it container-name bash and bench --site <sitename> backup --with-files I got:

frappe@e2f0cfa4c092:~/frappe-bench/sites$ bench --site mysite.localhost backup --with-files
Backup Summary for mysite.localhost at 2021-09-21 08:36:52.947017
Config : ./mysite.localhost/private/backups/20210921_083638-mysite_localhost-site_config_backup.json 266.0B
Database: ./mysite.localhost/private/backups/20210921_083638-mysite_localhost-database.sql.gz 2.2MiB
Public : ./mysite.localhost/private/backups/20210921_083638-mysite_localhost-files.tar 10.0KiB
Private : ./mysite.localhost/private/backups/20210921_083638-mysite_localhost-private-files.tar 56.4MiB
Backup for Site mysite.localhost has been successfully completed with files

revant_one · September 21, 2021, 7:03am

the backups are in sites-vol.

they are under {{site_name}}/private/backups for each site.

francois072 · September 21, 2021, 7:05am

frappe@e2f0cfa4c092:~/frappe-bench/sites/mysite.localhost/private/backups$ ls
20210920_161910-mysite_localhost-database.sql.gz 20210920_161910-mysite_localhost-site_config_backup.json 20210921_083638-mysite_localhost-private-files.tar
20210920_161910-mysite_localhost-files.tar 20210921_083638-mysite_localhost-database.sql.gz 20210921_083638-mysite_localhost-site_config_backup.json
20210920_161910-mysite_localhost-private-files.tar 20210921_083638-mysite_localhost-files.tar

francois072 · September 21, 2021, 7:09am

I just didn’t know what to do with these, but now it made sense when it was reformatted.

francois072 · September 21, 2021, 9:14am

It worked and 0 */6 * * * cd /home/francois && /usr/local/bin/bench --site all backup >> /home/francois/logs/backup.log 2>&1 should backup to the {{site_name}}/private/backups directory.

I used bench restore to find the restore commands. Although I couldn’t find the command to use for restoring site_config.json

frappe@e2f0cfa4c092:~/frappe-bench/sites$ bench --site mysite.localhost restore ./mysite.localhost/private/backups/20210921_104636-mysite_localhost-database.sql.gz --with-public-files ./mysite.localhost/private/backups/20210921_104636-mysite_localhost-files.tar --with-private-files ./mysite.localhost/private/backups/20210921_104636-mysite_localhost-private-files.tar
MySQL root password:
*** Scheduler is enabled ***
Site mysite.localhost has been restored with files

revant_one · September 21, 2021, 11:38am

if you need containerized cronjob check following link

There is no way to restore site_config. It is backed up so you don’t lose the secrets stored in the file. like the encryption_key. You’ll have to manually add your secrets to the site_config

francois072 · September 21, 2021, 1:15pm

This seems to be what I need. I just don’t have the expertise to modify it so that it backs up to my local disk.

francois072 · September 21, 2021, 1:25pm

I know now how to do the process manually. I think that configuring a regular cron-job to do the same actions automatically would be more than enough.

How long would those backups under {{site_name}}/private/backups remain there? I know that ERPNext has an option to configure the number of backups. Do those settings apply to backups that weren’t made by ERPNext?

francois072 · September 21, 2021, 1:33pm

Something like this although this doesn’t work for some reason.

/usr/bin/docker exec -it e2f0cfa4c092 bash && bench --site mysite.localhost backup --with-files && docker cp e2f0cfa4c092:./mysite.localhost/priva
te/backups/ /home/francois/backups

revant_one · September 21, 2021, 4:22pm

you can do lot of things with that docker-compose

mount a local directory as volume in container
expand the command: [...] section to run multiple commands that take backup first and then copy them to the mounted vol/directory

francois072 · September 21, 2021, 5:40pm

version: “3.7”
services:
minio:
image: minio/minio
command: [“server”, “/data”]
environment:
- MINIO_ACCESS_KEY=RANDOMACCESSKEY
- MINIO_SECRET_KEY=RANDOMSECRETKEY
volumes:
- “minio-vol:/data”
networks:
- erpnext-network
# Do not enable, check how to secure minio, out of scope of this project.
#labels:
# - “traefik.enable=true”
# - “traefik.http.routers.minio.rule=Host(backup.example.com)”
# - “traefik.http.routers.minio.entrypoints=websecure”
# - “traefik.http.routers.minio.tls.certresolver=myresolver”
# - “traefik.http.services.minio.loadbalancer.server.port=9000”

networks:
erpnext-network:
external: true
name: <your_frappe_docker_project_name>_default

volumes:
minio-vol:

I created a stack with the above content.
All I changed was the random keys and the <your_frappe_docker_project_name> to the name that I used for the ERPNext stack.

francois072 · September 21, 2021, 5:42pm

version: "3.7"
services:
  push-backup:
    image: frappe/erpnext-worker:v13
    entrypoint: ["bash", "-c"]
    command: ["docker-entrypoint.sh backup; docker-entrypoint.sh push-backup"]
    environment:
      - WITH_FILES=1
      - BUCKET_NAME=erpnext
      - REGION=us-east-1
      - ACCESS_KEY_ID=RANDOMACCESSKEY
      - SECRET_ACCESS_KEY=RANDOMSECRETKEY
      - ENDPOINT_URL=http://minio:9000
      - BUCKET_DIR=backups
      - BACKUP_LIMIT=8
    volumes:
      - "sites-vol:/home/frappe/frappe-bench/sites"
    networks:
      - erpnext-network

networks:
  erpnext-network:
    external: true
    name: <your_frappe_docker_project_name>_default

volumes:
  sites-vol:
    external: true
    name: <your_frappe_docker_project_name>_sites-vol

Where should I save the backup-job.yml file?