Importing large numbers of complex DocTypes (Journal Entries, GL, Sales Orders, Invoices)

ERPNext
, ,
1

Hello,

I am in the process of implementing an ERPNext instance and I am currently working on importing historical data from our old ERP system. GL records total over 3.5M, Sales order are 500k +, etc. Using the xlsx/csv import feature really isn’t feasible here so I decided to write a custom integration that sends records to the REST API and enters them that way. What I have noticed for DocTypes like this that require either an approval process or trigger other functions within the system (ie Journal Entry => GL Records or Sales Order => Sales Invoice => etc…) when you send a record to the API it triggers these downstream processes. Here are my questions:

  • What are best practices in this case?
  • Is there a way to insert the record without triggering these processes so that I can do the same thing for each related/child record and tie them together manually?
  • Are there betters ways of doing this when importing large amounts of data outside of the CLI import file option?
  • What are the risks and potential down stream impacts of writing to the DB directly?

Any help or guidance would be much appreciated!

Thanks,
Ryan

2

If you are just concerned about financial data then maybe just load summary opening balances and opening invoices for debtors and creditors.

If your goal is to load transaction wise balances then you may have to break down your data Fiscal year wise and load.

Also, it is in general not a good idea to bypass doc validations and do direct DB inserts unless you are very familiar with the DB tables.