SymmetricDS Replication Integrated into Frappe V13

Hi All

Ive been using SymmetricDS Database Replication for the past few months now to create a bidirectional replication from our cloud servers to our on premises servers and with the release of v13 and the Virtual Doctypes feature I have created a Integration Module to manage it. I would like to donate it back to the community for anyone who might be interested or just need an example of using Virtual Doctypes.

Current Features

  1. Create and Link Engines to a Frappe Site
  2. Basic Frappe Bench Commands to Setup Replication
  3. Basic GUI for Managing Symmetric DS Based on Virtual Doctypes
  4. Sending Bench Commands to a Node (Still in its Infancy)
  5. Only Full Bidirectional Replication Supported at the moment

For now I have only configured it to Replicate the Entire Database however in future I plan on replicating on a per company basis.

Planned Features

  1. Full Setup of Replication Through Frappe Instalation
    • Standardise Settings per Site (Engine Name, URL`s, etc)
    • Generate Engine Template on App Instalation
    • Create Tables on App Instalation
    • Fix Permisions on App Instalation
    • Create Reload Requests for only Modified Tables on any Schema Changes
    • Set all nodes to maintenance mode during any Schema Changes
  2. Create Standard Channels and Triggers based on Frappe`s Structure (Esspecialy for ERPNext)
    • Triggers need to be created correctly on a per document basis to filter data PER Company. (Child Doctypes are currently a problem)
    • Create Initial Load Scripts to Only Load Specific Company`s Data on Registration or Reload
  3. Allow Frappe Framework to know of Replication and monitor and make use of it
    • Integrate into Frappe the Current NodeID, whether or not its master or client, is replication service running or not
    • Integrate into SymmetricDS the means of allowing Triggers, Jobs, Monitors, Notifications, Data, etc to access the frappe framework
  4. Implement File Syncronisation to Sync Site Files as well as Apps
    • Currently SymmetricDS File Sync is not implemented but is available.
    • Create Triggers for Syncing Files and Apps

I would love to hear your thoughts on it and would really appreciate any form of help with reaching the end goal of this project.

6 Likes

Did you stop working on this great project?