I doubt the permissions would be much of an issue, relative to the other programming involved.
IMHO the largest issue with the way workflows currently function is you can only have a single, monolithic workflow for each doctype. If you want diverging paths (say, for different order types, customers, or amount tiers), you need to build multiple conditions upon multiple conditions, resulting in a large cumbersome table–if it’s even feasible at all.
Ideally you could have multiple workflows enabled, where a document can enter and leave a particular workflow, and possibly enter a new one, even while a previous workflow is still active. It would require a 1:many instead of a 1:1 relationship, but definitely doable.
What if it works similar to how the Project templates work?
The Workflow could be a template that defines the Workflow Steps/Actions, and a separate set of Documents could be linked to the specific documents.
For instance:
Workflow - Stores the data about the workflow
Steps/Actions - Stores the individual steps that make up the Workflow
Document Actions - A set of actions to be completed that are linked to the document to be executed against/referenced. You could pause flows for specific documents, skip steps in the flow, or delete all upcoming actions for that document entirely. You could also easily view the completed and upcoming actions for individual documents.
Then the cron can loop through each of the ‘Document Actions’ and evaluate each one to see if the action should be executed. Since the database could grow quite large this way, perhaps completed actions should be purged from the database after a certain amount of time.
This sounds like a great suggestion. I think much of the community could benefit from a more robust automation tool, especially the non-programmers (which is probably the majority).
And done properly, eventually you might be able to consolidate a few similar but more rigid doctypes—notifications, workflows, assignment rules, etc.–into a single, much more flexible automation feature.
For reference, how Mautic handles this is with the following tables
campaigns defines basic info about the campaign campaign_events is the list of events in a particular campaign (the actions/steps) campaign_leads is the junction table - campaign to leads campaign_leadlist_xref is the junction table - campaigns to leadlist lead_list is a list of leads (Contacts) lead_list_leads is the junction table - lead_lists_leads campaign_lead_event_log is basically the “assignment” of an event to a lead
@ankush - can you please advise if you are ware whether the ‘no-code workflows’ feat is on the Frappe roadmap anywhere, and if so, is the community able to provide some input on its development?
Another current feature that could be consolidated and proved using an automation system could be the Milestone Tracker. An automation action type could be added to create a milestone, which would give more flexibility than the current Milestone Tracker (for instance, you could create a milestone only for certain status changes using automation conditions, vs. all status changes, etc).
I agree that it’s great if your team only needs to use one software. (but it would never happen because of the complexity of today’s work). However, I don’t believe your team uses automation software on a daily basis because that software operates in the background, handling necessary tasks. Setting up these workflows isn’t a continuous job, and I think it requires a different skill set, which may involve different team members.
My second consideration is how you should structure your stack and business processes. I believe ERPNext is excellent as a centralized source of truth, especially from the selling phase with inbuild or with the new Frappe CRM. However, if you’re primarily engaged in marketing activities, you might not need all that data in ERPNext, bc you want to keep your transaction data clean. Even for analysis purposes, you can integrate data from various sources using BI tools, for example.
In my opinion, it’s not a good idea to implement everything, especially when there are already excellent, ready-made solutions available. It’s better to focus on Frappe/ERPNext and leverage existing software solutions.
I’m a core contributor on an open source python-based workflow orchestration tool called SpiffWorfklow. Several people have reached out to us recently about Frappe Integrations and I’m here doing a little research. I really don’t know if we would be a good fit for an integration or how we might compare to a tool like Mautic, but I do think we meet many of the characteristics described in this thread: low-code / citizen developer, purely python, general purpose, visual (BPMN) diagramming based. If there is interest in working together to integrate SpiffWorkflow into the Frappe Framework, I’d be down talking to folks about it.
I certainly have no objections to a SpiffWorkflow integration (though I’ve never used it). That doesn’t really solve the problem of: it’s another system that the business has to maintain.
If it would be feasible to include automation workflows in ERPNext/Frappe, I think that would be huge.
There is a core SpiffWorkflow library - GitHub - sartography/SpiffWorkflow: A powerful workflow engine implemented in pure Python - pure python with only one dependency on lxml, that could be integrated into anything and can execute BPMN workflow diagrams. - You would need our BPMN IO modeler extensions added to the frappe UI so users could draw out their workflows. - you would need to re-implement all the storage, file management, testing framework, reporting that comprise our complete application, but the pieces are there if you wanted to tightly integrate the library into Frappe.
I’m in agreement with @oguruma. There’s already a Frappe graphical workflow builder, milestone tracker, notification module, etc. Expanding and/or eventually replacing these with a more flexible and comprehensive automation toolkit that works natively with the Frappe framework doctype model would seem a preferable strategy vs. trying to integrate with 3rd party apps (over whose strategy/direction the Frappe development team wouldn’t have any control).
I would love to see a BPMN compliant workflow engine incorporated into Frappe.
BPMN is an open standard (https://www.omg.org/bpmn/) and the de facto foundation for most enterprise strength process automation tools.
This idea has been discussed many times on this forum:
My organisation started off using Camunda, then migrated all elementary workflows to N8N, and recently transferred most of the remaining Camunda processes to Spiff. Spiff is not as yet a complete replacement for Camunda, but suffice in most scenarios. We use the open source bpmn.io modeller and then execute the models on Spiff.
It would be a god-send to incorporate the bpmn.io modeller into Frappe / ERPNext and bundle the Spiff engine into Frappe.
The biggest “drawback” with regard to the current Workflow capability in Frappe is that it is in essence a State Transitioning tool, not a fully fledged process automation tool.
In other words, I can influence how a DocType’s status changes but I cannot model a complete business process at a very high level in an organisation. I refer to this level of abstraction as “top ware”. It should exist without assigning any specific software or human participant to the process’ constituent tasks. This level of abstraction then allows you to choose your systems dependent on the process, and not the other way around.
There are many other aspects which comes with a BPMN capable solution such as human participant delegation and escalation, which is not possible with the current Workflow. Many of these considerations were discussed in the above referenced topic.
I (actually we) am not a developer. We are best described as hackers who can cobble together a few scripts, but to participate in developing such a feature is completely beyond our capabilities.
Having said that, we can certainly assist with insights and our current work-around we use to incorporate BPMN.
I think @danfunk is the most critical resource we could hope for. He’s the one who will be able to determine the nature of the 2 main components and their integration into Frappe (at least as seen from my non-technical perspective):
1 The BPMN modeller. Spiff uses a customised version of bpmn.io so that the models are embellished with sufficient metadata in order to execute on Spiff. bpmn.io is a JS library which can be rendered in any Desk view just as Frappe incorporates the Ace editor to edit HTML / JS / Python etc.
2 The Spiff engine itself is a Python module / library with a single dependency namely lxml. So it should be no effort at all to bundle the engine with Frappe.
The biggest amount of effort will be to amend the Frappe database structures and the ORM abstraction layer to accommodate this feature. The modeller would have to have access to the ORM layer so users can select an object on the modelling canvas. And likewise the engine would have to update the site’s DB over and above maintaining it’s own data structures. Please advise whether I’m smoking my socks…
If you don’t mind me asking, what is the actual process that BPMN uses to execute the programmed actions?
I started toying around with a custom app using a similar approach to Mautic (only because that’s the closest application to this that I am familiar with).
Mautic basically creates a row in a table and then uses the cron to loop through that execute each action. The system admin has to manually configure the cron jobs (it doesn’t have a scheduler like Frappe), and Mautic recommends having the cron run every 5-10 mins.
Since every workflow action has to be looped through once that cron runs, it can create some scalability issues if there are a lot of records. Unfortunately, I don’t have clients big enough for that to be a concern, though
