E-commerce Data Synchronisation

Hi guys,

We are developing a module to connect an e-commerce platform to ERPNext.
The most difficult problem is to synchronise Data.

For example with Item Data, since the e-commerce has ID to identify each Item, the SKU (Item_code equivalent) can be changed. In ERPNext, this is not the case (I supposed so because the Item Code is used as ID here).

The problem is not only applied for the Item but also Categories (or Item Group)…

When synchronising, this could be a problem since there is virtually no real stable ID to match between the 2 dbs. With deeper understanding of the ERPNext DB’s structure, do you have any suggestion ?
Is there a safe way to change Document’s Name and cascade to linked Documents that I don’t know of ?

@nathan_dole we will help you on this, if you share your plugin !

Sharing makes us all happy and productive :smile:

Sure, still on conceptualise phase. Trying to connect opencart and ERPNext actually.
Give me a few days to consolidate then I will share.

Hi rmehta,

Published the basic code for the ERPNext side here: GitHub - nathando/erpnext-opencart-api: ERPNext API to connect with opencart
Please feel free to comment or create PR.
I need to get through the problem with the ID before can actually continue with opencart side. Currently stuck because from Opencart’s POV, there’s gotta be something unique (numeric) to map these Categories or Items.

Will update an expected work flow between the 2 for you guys to comment. I believe this problem shouldn’t limit to opencart, it’s universal for other ecom’s platform.

2 Likes

Hi.

Here is the flow intended (basic, still not cover all cases).
https://drive.google.com/file/d/0BxAM1sgeemT1SHV4bXA1cHp3cUk/view?usp=sharing

I need some feedback on the model of syncing data. Should I do periodic synchronisation of data or only incremental (even from ERPNext).
As I explained above, the biggest challenge seems to be the ID. For example, if in Opencart I change the SKU (Item Code) or the Item Name, it will be pretty hard to find out with Item I am referring to in ERPNext

1 Like

@nathan_dole

Diagram was great for initial understanding.

Why manual syncing? It could be real time.

You should maintain Items on in ERPNext - and sync to OpenCart via hooks

And push order info via REST into ERPNext

Btw, why would you change SKU?

Why manual syncing? It could be real time.

I imagine there will be a scheduler to periodically pull data from ERPNext, so the Manual Syncing is an illustration of each run. Real time means: ERPNext push to Opencart as well ?

You should maintain Items on in ERPNext - and sync to OpenCart via hooks

Oh I see, so you suggest: If I add/update item in ERPNext, the hook should call to Opencart side and update info ?
(Basically no more bulk update but incremental, this is actually safer)
Will there be any concurrent problem then because both side can modify the other’s DB ?

Btw, why would you change SKU?

Not all Customer have every products’ SKU fixed. Sometimes, they will change it when they follow a new series for example.
Ok but for that matter, even if I don’t change SKU but Item Name, I will still not have anything to refer to in ERPNext. Or should I store the Opencart Item’s ID in ERPNext too ?

I think all master data should be updated from ERPNext only. You should disable creation / editing in Open Cart. There should be one way data flow.

If you cannot stop in open cart, let ERPnext overwrite whenever. You can also implement two way syncing (will leave that to you)

  • In Open Cart Plugin,
    • push order data into ERPNext
  • in ERPNext Plugin,
    • push item data in Open Cart
    • push delivery updates (shipping status) in Open Cart
    • push invoice PDF (? - I am not sure how Open Cart handles this)
    • sync inventory info into open cart (periodically)

Ok agreed that the Item Master should be pushed from ERPNext (for normal mode).
And periodically update should only update Quantity info (Inventory Info)

I think I gotta save the IDs from Opencart whenever adding Item data.
Just like save the Opencart’s Order Number in ERPNext’s Sales Order.

Yeah. Doubly link both ERPNext and OpenCart with each other’s IDs.

Hi rmehta,

I have a minor question but it will make development very convenient.
So developing a plugin, apart from the code there are other things like add Roles/Permission/Additional Fields to ERPNext’s own DocTypes.

This can be done manually but it will be troublesome when I actually update the ERPNext. Then I will need to remerge these changes every time. Is there a patches or some sort to handle this (cos I saw it when doing update but not sure how to start) ?

If you are in developer_mode, the .json fiels for each doctype are automatically updated.

When you update using --latest, these changes are updated in the site schema too!

(Permissions do not get updated because the user may have changed them). To update permissions, add a new patch. You can also add

execute:frappe.permissions.reset_perms("[docype]") in the patches.txt of your app.

Hi rmeta,

Do you have any code example of a patch that do the followings:

  • Add a Custom Field to a DocType
  • Add a Custom Script to a DocType
    I need to add certain fields into Item DocType but doing it manually will defeat the purpose of creating an App…
    Also Custom Script is important to change the UI behaviours.

You can use fixtures for that. Check Not Found

1 Like

Thank you, I was so blur … didn’t think of that.

Another question :smiley:
Do you have any util to do POST/GET request (on server directly not using frappe-client) ?
Should I call the worker, and how to do that :frowning: ?

I currently use httplib to POST to the Opencart but it seems quite hard to debug… no request can actually be captured

1 Like

The postman app for chrome is my favourite REST client for debugging.

2 Likes

Hi,

I currently need to calculate the Item Quantity that is currently available for the E-commerce. This will be checked before every confirmation of order.
But the tricky part is I need to deduct the Quantity in submitted Sales Order. If not then there will be inconsistency when one Order confirmed but the qty not yet reduced and ppl will be able to buy still.

So I wonder between 2 ways:

  • Calculate Stock Balance using the same way the Stock Balance report did (going through all transactions ? or is there a faster way?). Then deduct the Qty from (not completely delivered) Sales Order. I realize we can map the Sales Order Item with the Delivery Note Item by field prevdoc_detail_name just in case it delivers partially.
  • Calculate by querying all transactions from Sales Order + Delivery Note. And calculate from there.
    Any advise ?

Hi nathan_dole

How far did you get with your synchronisation project? Did you manage to come up with a solution to the stock problem mentioned on your last post?

I am interested to know whether you got this working as we are considering a similar approach - using ERP Next and an ecommerce platform (namely Sylius).

Thanks

Hi frappeval,

Sorry, I’ve been so busy that there is no time to do this during last 2 months. Will restart in 2 weeks time.
So I use a plugin called Opencart API to handle the Opencart part. So far, the things i’ve done are:

  • Synchonize Items between 2 platforms
  • Synchonize Categories between 2 platforms
  • Checking and make orders in ERPNext when there is order in OC

Hi @frappeval,
We have implemented e-Commerce Integration with ERPNext for one of our client. If you are interested then do connect on skype kanhaiya.kale or send me email on kanhaiya@indictranstech.com

1 Like