Automated continuous master/slave streaming backup

ERPNext Customize ERPNext
41

To date I have had no feedback from anyone. :face_with_raised_eyebrow:

Please be aware that I will happily sort out any difficulties you may encounter.

It is quite likely that I have allowed dependencies on my own installations to creep into the toolkit’s functionality. In order to solve those issues I need to see the results of running the scripts in other environments, so I would be grateful to anyone who is willing to collaborate a bit to clear those up.

1 Like
42

@MartinHBramwell I did promise to help test and provide feedback but I have been out of commission due to other commitments. I will try and test it out this weekend and provide feedback.

Thank you very much for your efforts.

43

Thank you for letting me know.

Appreciated!

44

I just now uploaded some changes.

The main improvement is protection of the slave site’s Social Login credentials.

A normal database backup and restore expects the restored site to be the same as the backed up site. Since replication necessarily requires the slave site to have a different domain name, while Social Logins require an exact amtch to the domain name, I provided a restore tool that replaces references to the master site URL with references to the slave site URL.

That tool did not include updating the Social Login credentials. Now it does.

You merely need to have a file named socials_google.json in the same directory as your site_config.json

admin@denh:~$ cat ~/frappe-bench-DENH/sites/dev.erpnext.host/socials_google.json 
{
    "name": "Google",
    "enable_social_login": 1,
    "social_login_provider": "Google",
    "client_id": "{{ClientID}}",
    "provider_name": "Google",
    "client_secret": "{{ClientSecret}}",
    "base_url": "https://www.googleapis.com",
    "authorize_url": "https://accounts.google.com/o/oauth2/auth",
    "access_token_url": "https://accounts.google.com/o/oauth2/token",
    "redirect_url": "/api/method/frappe.www.login.login_via_google",
    "api_endpoint": "oauth2/v2/userinfo",
    "auth_url_data": "{\"scope\": \"https://www.googleapis.com/auth/userinfo.profile https://www.googleapis.com/auth/userinfo.email\", \"response_type\": \"code\"}",
    "doctype": "Social Login Key"
}
admin@denh:~$

(Albeit with the correct replacements for the variables {{ClientID}} & {{ClientSecret}} )

@TurkerTunali I do hope you and those close to you are safe and secure.

45

V14 supported now, as you requested

6 Likes
46

We are OK. Thank you. It is really hard to do stuff. I hope next week will be better.

47

Will attempt and update, thank you

1 Like
48

This past week, my VPS hosting service had some sort of maintenance hiccup. The slave server stopped pulling changes from the master, and showed it was missing 4 hours of data changes.

I rebooted the slave, logged back into ERPNext and found that the missing deltas had already all been applied and the two databases were synchronised again.

I gotta say, after trying initially to see if there would be real interest in this toolkit, getting many positive responses and, consequently, doing all work, it is very disappointing that still no one has even tried it.