Integrate ERPNext with company's pre-existing web apps

Integration
1

Hi everyone,

I am new to ERPNext, currently analyzing the system to see if it fits the needs of my company. In a first phase, I have gather the requirements and done a cross-check to see how well the tool/system covers the requirements, and what parts will need configuration and/or extension to meet our requirements more accurately. So far, so good :slight_smile:

Now I am tackling questions regarding integration with pre-existing subsystems. The biggest restriction we have to select an ERP system for the company is that it has to co-exist with part of our current subsystems (I guess this is a typical scenario), at least in the mid-term, to avoid drastic changes, all at once, for the employees. These subsystems have been developed by us internally using Drupal. My question here would be if someone has experience with such situation, and if so, what steps would be recommended to follow for an as-much-as-possible-seamless integration.

To provide a bit more context: the existing subsystem is basically a customer portal for our products. Thus, this web app contains information regarding customers, licensing and products, which are key data for the ERP when it comes to company’s management processes, so it is crucial to keep the two subsystems (pre-existing subsystem and ERPNext) synced. The first step would be either to sync the data between the two subsystems in the background or use the same database for both, and that shouldn’t be a problem (I haven’t checked yet, so not sure which option is better in this case, but that’s another topic). But what I do not see clearly is how to make the different subsystems work seamlessly. End users will for sure complain if they have to work with the same or similar data in two completely different systems.

The UI of both subsystems are completely different, and the users will have to get used to the two of them, and jump continuously from one to another. We can, of course, adapt the UI of both sides to look alike as much as possible (that’s already under discussion), but I am still not sure about how seamless this integration would be for the user. I’ve found references in the forum to iframe for embedding external websites, but I don’t think it is a solution in this case. Any better suggestion?

Maybe I am over-complicating the topic in my head, and just a similar UI (with, for example, same header, footer and fonts/icons, to keep consistency) and links from one subsystem to another (and the other way around) is enough. But it would be nice to know the experience of other people with a similar use case.

I hope the use case is clear. Do not hesitate to request more information/context if needed.

2

Hi @aferte,

It’s great you’ve put a lot of thoughts into this! :slight_smile:
There is no simple answer to your questions, but I’ll give some advice.

1. Data Synchronization

ERPNext must use its own database. It has very specific schema requirements. You cannot point it at another database, like whatever is behind Drupal.

A bi-directional synchronization is “possible” in theory. But it will be a lot of development effort. What I suggest you do is write a detailed Requirements document. What exactly must be synchronized?

  • Customers?
  • Sales Orders?
  • Addresses?
  • Orders?
  • Inventory

You need to determine the Scope of your sync. Not only will this help you make decisions. But if you approach any consultants/developers for an Estimate, they will need to know this. Do you need to sync 5 tables, or 50, or 500?

From there, it’s a matter of writing the “sync tool”. Whether that’s done by ERPNext itself, or another App, is a matter for analysis and debate. ERPNext -could- accomplish this. It’s extremely flexible. It can do almost anything, if you put in the effort.

2. UI Synchronization

This is much more difficult, imo. ERPNext comes with a UI that is opinionated. All the screens share a common design. You have -some- flexibility in customizing them easily. You can hide fields, move them around, add buttons.

But there are limits:

  • A grid has 11 columns at maximum.
  • You cannot link 2 tables together on the same page, unless 1 is a Parent and the other is a Child.
  • The “created date” of a record is not visible on a Document page.
  • Empty fields are hidden automatically once a Document is Submitted.
  • (hundreds more)

If you were extremely skilled, and extremely familiar with ERPNext, it’s possible to reskin it. ERPNext’s web pages are no different than any others on the planet: HTML, CSS, and JavaScript. They can be modified.

But it is not easy. And if you did accomplish a major re-skin, you would find Upgrades extremely challenging. Because every time you wanted to move from version 13 to 14, or 14 to 15, you would have to completely reconcile your custom changes, with the latest, standard code.

I doubt many people have attempted a large redesign of ERPNext’s UI. But maybe they will reply to your thread!

2 Likes
3

Wow, thanks for the fast and detailed answer. The points you mention are more or less what I expected, but I wanted to ask anyway to make sure I was not missing anything.

If I had full authority on the decision-making, I would go for merging into ERPNext - from day 1, and not in 6 months or 1 year - all the functionality we have right now in that customer portal, but unfortunately it is not something I can do so easily (too many parties involved), so I was hoping I could find a solution to make both worlds coexist for a while.

I completely agree that changes in the UI would be too cumbersome, so the only feasible changes I see are header, footer and…not much more. Changing the UI of the other subsystem would be slightly easier, but also problematic.

Regarding the database, yes, I am aware that touching the database structure of ERPNext would be a no go, but I was thinking about something like creating views to make it easier to access to the information in the way the other system expects. But creating a sync tool using the API of one or the other system is definitely a safer path to follow.

Thanks a lot for the feedback, it helps me a lot to organize my thoughts about the proposal I am working on :slight_smile:

5

Creating SQL Views is a great idea, and would certainly help Drupal see data in a way it’s accustomed to. :100:

6

We’ve been working with a client on almost exactly this problem for the last couple of months. I’ll try to generalise our thinking and hopefully that is helpful to you.

Data Synchronisation
As @brian_pond said above - this is a big issue. The first stage is mapping the fields in all of the apps to ERPNext fields. Then deciding what type of sync you want to use.

From our side we decided that we only ever wanted one “source of truth”, so we are not looking at sync in both directions. One system will manage the processes and it’s data will be synced across. Then at an appropriate point in development ERPNext will take over the management and the sync and previous system will be retired.

In terms of how to sync the data, our current planning is external script(s) which will connect to the existing database and export the data across as required. While that sentence makes it seem quite easy, this has taken up a large part of our planning time in working out how to actually do this and maintain the sync in a sensible way.

Backend UI
We are not even attempting to change the actual UI beyond making forms fit the workflows required. What we are doing is introducing small, incremental changes to the end users workflow to allow them to get familiar with the new system over time.

Frontend Design
This is actually the easiest bit of the whole project, re-templating the frontend website part of ERPNext is very easy to achieve, and with a little thought even implementing template compatible blocks to the website editor isn’t hard.

We’ve gone exactly the opposite way with this - our decision was based on the idea that if the changes are small enough then users have little to get used to. It also means we can pick up one part of the business process at a time and really make sure it works before adding another part into the mix. Our current timeline suggest 2 years to get every part of all external systems migrated.

I should say that the decision to go that way has significantly added to the planning around data migration and integrity - so it’s not without it’s costs and downsides.

4 Likes
7

@Kieran Thanks a lot for your input! Very much appreciated!

We’ve gone exactly the opposite way with this - our decision was based on the idea that if the changes are small enough then users have little to get used to. It also means we can pick up one part of the business process at a time and really make sure it works before adding another part into the mix.

Maybe I should have mentioned that these pre-existing web apps we have are fairly new, so not that well established among the users as others. That’s why I said I would get rid of them from the beginning. But thinking twice about it, I think the best approach is doing it iteratively, incorporating little by little functionality throughout the time, as you mention. Definitely the safest way to go to keep functionality and guarantee stability of the whole system.

I knew already that making several subsystems coexist would be a tough task, but…that’s life! Important is to make a good planning and consider all the risks on the way…and be patient :slightly_smiling_face: I will work on a deployment plan taking into consideration your input @Kieran and @brian_pond.

PS: It is really cool to see that there is an active community behind all this that is willing to help. That’s what made me choose ERPNext above others in the end.

1 Like
9

One word of caution… @aferte @Kieran

The average target for major version releases has been approx. 14 months when looking backward into the life of ERPNext.

The current version (v13) was released in January of 2021 making it 15 months old this week.

If you started working right now on an incremental plan to ease your users into the change to ERPNext, then by the time you reached your 2yr goal, you would likely be 2 versions behind in ERPNext.

Or you could set yourself on a constant development path in this ease-of-use concept that would never really reach it’s goal.

I may be wrong here about how you planned this out, but any time someone seems to indicate they are going to go 2 years on the same version, I want to warn them about the average release schedule.

One last thing…
I have just completed a transition from v10 to v13 that was NOT simple. Please read the forums by searching on “upgrade” or “update” to see the stories of difficulties in the history of achieving even sometimes simple point updates.

In my case, I chose to setup v13 sandbox servers for the client to train themselves on for about 8 months, then I did a hard switch to v13. The problems were greatly reduced and in 3 days there were no longer any major issues to resolve. All users were content.

Plan carefully.

BKM

6 Likes
10

@bkm you raise a extremely good point, one we have been considering in the planning. Given the likely lifetime of this implementation, several version upgrades may be required.

Even once “finished”, I’ve found these systems never are. They are a constant cycle of maintenance, improvements and changes as users work out who else to leverage the power of the data they now have.

The upgrade cycle is being factored into that - It’s probably one of the single biggest concerns. Upgrade of these type of systems is never trivial. And the exact requirement to upgrade from one version to another can’t always be predicted in advance. Part of our DevOps planning has been to have several layers of development and testing machines before changes to production will be made.

11

Thanks a lot for pointing this out, @bkm. To be honest, when I got the task to analyze different ERP solutions, my biggest concern was (and is) how long the whole process will take until the chosen ERP is up and running, and fully integrated in the company. It is not a trivial task at all, we are not talking about a couple of months, not even a year, and things change so quickly (both in sw development and in my company :slight_smile:) that it causes a bit of anxiety on me just to think about it :fearful:

Reading about your experiences is extremely helpful to identify the risks and know which points will require special attention. I will take a deep look at the threads related to version upgrades to get a better idea of what is to expect when the time comes.

Thanks again to everyone!

1 Like
12

Hello all

I completely agree with all of the above sentiment.

We have been presented with similar situations, and our subsequent approach was as follows.

1 Make the effort to map your business processes. It’s crucial. Understand which systems and users form part of the processes. We use a BPMN compliant tool to do so. This effort will reward you with an overview of the as-is and to-be inter-system and user-system complexity.

2 Keep existing systems in-tact for as long as possible, provided they have an API so that they can be accessed bi-directionally. Obviously this does not apply if they are no longer fit for purpose.

3 Gradually introduce Frappe / ERPNext as a new master / hub system, eg initially only the CRM or Accounting module.

4 Use the various APIs in Frappe to interact with the existing systems and to gradually bring their functionality into Frappe / ERPNext, even at the expense of redundancy.

5 If your processes are very simple, use the Workflow (actually just state transitioning on a specific DocType’s document) capabilities of Frappe. If they are more complex (and invariably they are) or you need inter-DocType workflow then use some lightweight process engine such as n8n to orchestrate the interactions. We found that this is the easiest way of keeping inter-system communication “documented” and in fact in operation. It also allows us to sweat the investments in existing systems for as long as possible.

The success of any new system depends largely on user (in-house and public) acceptance, and this should not be underestimated. Frappe / ERPNext is a complex beast, and in constant flux, and by removing the business processes out of Frappe / ERPNext into an overarching process engine gave us the flexibility to accommodate this flux more readily, although still not easy.

1 Like