I use an open source software called Mautic to handle newsletters and drip campaigns. The software works well, but it would be much better if that was implemented into ERPNext directly.
I’d like to build a module that’s similar to Email Campaigns, except that you can schedule any action, not just sending of emails.
For instance, suppose you have the DocType called “Cadence” and another linked DocType called “Cadence Touch.”
You can build a 7-day follow-up Cadence like:
Cadence: 7-day Follow-Up
-Cadence Touch 1: Name:“Make first follow-up call”, Type: “Phone Call” (create a ToDo), Assigned To: “User@user.com”, Days: 1, Reference DocType: Lead, Reference Field: “status”, Reference Operator: “not equals”, Reference Value: “Opportunity”
That would create a ToDo for the specified Lead 1 day after they are added to the Cadence. Then, Cadence Touch 2 would send an email 2 days after that (but only if Cadence Touch 1 is fulfilled).
Agreed that some type of automation rules in ERPNext (or better yet, Frappe) would be great. Most other ERP systems have these.
It could combine some of the features of Workflows and Assignment Rules. Basically you’d want triggers, which would fire on certain events and conditions, that would then execute one or more actions, which might be sending an email, updating some fields, creating a transaction, etc.
It would reduce a lot of the custom scripting that’s currently done, and help make Frappe even more of a low code / no code framework.
I would be interested in helping to sponsor and/or develop something like this.
These are not Frappe-specific. However, you could certainly use them to call Frappe code. They handle all your requirements, such as "If Task 1 succeeds, run Task 2. Otherwise email me the error messages from Task 1."
If you’re looking for something Frappe/ERPNext specific? The closest thing I know about is my own open source App: Background Tasks Unleashed. I wrote this 2 years ago, and use it actively today. It enables you to run Tasks on difference cadences, and logs the results. After installation, everything is configured via the web UI.
(Warning: It does -not- meet all your requirements. For example, each Task currently exists in isolation: you cannot chain them together.)
You’re welcome to use it, modify, examine the code, etc. I do not “think” anyone besides me is actually using it in a Production environment (I don’t advertise or blog about it). So you may run into issues. If so, let me know in GitHub, or send me an email.)
Thanks for the suggestions. I’m not sure about the OP’s view, but I was thinking of something tied in more closely to Frappe/ERPNext. It looks like the products you recommend are more like orchestrators and batch schedulers, but they would still require a programmer to write Python code for the jobs.
My thoughts were more of a low-code/no-code module that linked to existing Frappe doctypes, administered through the standard UI, with the option to execute the actions immediately using triggers, or on a schedule based on time periods (similar to Netsuite and Acumatica Workflows, MS Dynamics Power Automate, Odoo Automation Rules, etc).
ERPNext does have Workflows, but my understanding is the only “triggers” are menu-driven actions, and it has very limited action & update options, so its automation capability is pretty limited.
While I agree a no-code action editor could be made for Frappe (drag-and-drop UI for Server and Client Scripts), it’s a huge amount of work, for a small amount of users (in my opinion) : people that know what to code, but can’t write code (or just lazy developers efficient developers like many of us ).
I agree!
Moving back to the topic: I’m lightly working on a no-code workflow block editor for Frappe in my free time for fun.
However, design is hard. I have a prototype UI but it’s the easy part. What’s hard is making it simple to use. I have ideas (but no code) for the back-end.
This work is inspired by Apple’s Shortcuts application. I’m thinking of just using a pre-made UI for this, but it might be difficult to find something that would be flexible enough for our needs.
In terms of design, what I have is:
Design Ideas (click to open)
Blocks, Block Types with Tags and Slots
Example: the WorkflowRoot block has a triggers slot that accepts blocks of types tagged with #trigger (like a DocumentTrigger or #no-op like a Comment)
Example: You can’t put a SendMail action inside the triggers slot, it’s not tagged correctly!
Using safe_exec once for every block (or possibly workflow-level code synthesis)
Providing inputs as arguments to a execute function that is provided by a Block Type document.
Possibly code rewriting for variable substitution (writing a custom Python script in a RawPython block containing Out[5] (the output of the 6th block in the UI) would get rewritten to Out["block_p4Ag5uB"] on-the-fly to have absolute block references).
Universal scripts (workflows/blocks that can be both JS and Python)
Example: computing a sum when fields change, so that the UI refreshes, and that the server recomputes on save, as a security measure
I appreciate the input. My current solution for this, called Mautic, does pretty much everything I need it to do. The problems is that it is still a standalone piece of software with a completely different interface.
Some kind of automation system being in ERPNext would be greatly beneficial, since it would leave companies with at least one less piece of software to worry about, and give the users a single interface.
I think Mautic has done a good job in creating a no-code/low-code interface for automation flows.
In essence, with Mautic, you create a “flow chart” via the UI by dragging/dropping various elements. With each “step” in the campaign flow, you can:
Perform checks: I.e. “does this Contact have an email address?”
Carry out actions: send an email, send a text message, call a webhook
Specify times/days: for example, Send this email only between 12:00-4:00PM.
I will say that Mautic does have a couple limitations:
It can only carry out campaigns on “Contacts” which are designed to represent people.
It doesn’t have a way to iterate a certain action more than once without something of a workaround. For example, suppose you want to send a reminder email to a Customer every week if an invoice isn’t paid. There’s no real (good) way to do that in Mautic.
ERPNext’s Email Campaigns is the closest thing (from what I can tell) inside ERPNext. If you look at the Email Campaign, it does have “Send after days,” so you can send an email on day 1, wait 3 days, then send another email etc. But a couple things that make it unuseable for me are:
It seems all emails send at midnight. This is, if I’m being honest, kind of terrible. For one thing, many people get audio notifications on their smartphones when they get an email, and they probably won’t appreciate getting emails at midnight. It also makes it kind of obvious that the email wasn’t written by a person if the emails arrive at exactly the same time every day.
there’s no way to perform some validation before sending the email, or to remove them automatically from the campaign based on some criteria.’
It can only send emails. It can’t send SMS,update documents, or call webhooks.
While I would love to see a automation system built into erpnext I’m not 100% sure that it is needed. We use mautic and fluent crm for these task and they do it extremely well.
You can connect any of these software together using N8N (Open Source), Active Peaces (Open Source), Zapier, Make, Pabbley etc. Really anything that support webhooks.
Here is an example integration for checking to see if a contact in ERPNext is in Mautic.
I agree that Mautic, and other solutions, likely work very well.
Here’s the problem: Companies don’t want to have to maintain two pieces (or 3, or 4) of software, or train their employees on how to use 2 different pieces of software, plus all of the other issues that go along with that (each employee having another set of credentials, etc.).
^ That is one reason I decided against 3rd party software, and just wrote a Frappe-only solution. I wanted employees who are already familiar with ERPNext to manage tasks themselves. Without help from IT department or developers. If the task needs to run at 8:00 AM instead of 10:00 AM? Any authorized Frappe/ERPNext User can make the change in their web browser. The only time a developer is required is to write new Python functions (if they do not already exist)
Mautic looks interesting, but 2 things make me hesitate:
First, it is clearly targeting Marketing. Not something I’m interested in right now. Can it do other, non-marketing automation? I’m assuming the answer is ‘yes’. Still, I’m reluctant to adopt something where I’m not the target audience/user.
Written in PHP. Inevitably, I’ll want to customize it. But I’m just not interested in learning yet another programming language right now. And if I was, PHP wouldn’t have been my choice.
I am by no means encouraging anybody to use Mautic, though I have found it to be a very powerful piece of software. I bring up Mautic in this thread really just to reference the kind of capabilities I am looking for.
I use Mautic for my own marketing, and as a service I sell to clients. Other than that, I have no affiliation with them, so I hope people don’t get the impression I am trying to shill for them.
I have the same problem with regards to Python vs PHP, except in the opposite direction than yourself. I work a lot with Wordpress, and web development in general, so I am at least semi-skilled at PHP. Not so much with Python. That said, Frappe Framework is compelling enough for me to learn Python.
Can Mautic do “non-marketing” automation. Yes, but the thing to understand is that the automation flows (called “Campaigns”) are really based around the Contact (meaning, the actual “person” you’re trying to convert). You could, for example, send an email on Day 1, then, if they don’t open that email within 3 days, send an text message via Twilio, then if they don’t respond to that, you could use a third-party API (Postgrid, for example) to send them a physical letter/postcard via snail mail. Since the Campaigns support webhooks, you can integrate it with virtually anything with a REST API. For example, you could call an ERPNext method to create a ToDo for a Sales rep to call the Lead at some point during the campaign.
The campaigns are built using a relatively easy-to-use drag and drop flowchart builder type of interface.
A couple more things that Mautic does (again, I’m not trying to convince anybody to use it) are
creation of web forms via GUI, which can be inserted on any website via HTML/Javascript
allow contacts to manage which newsletters they’re subscribed to
uses cookies to track users of your website, and can track known and anonymous users of your website
uses a pixel to verify whether an email was opened (very useful, and something I wish ERPNext supported)
I’m wondering if a UI could be developed around the if/then/else paradigm using the same library as the new Workflow Builder in v15, which uses Vue Flow. This would be the most logical path forward IMO, and have the best chance for inclusion into Frappe upstream at some point.
Beyond the projects already mentioned, inspiration (or alternative frameworks, as a last resort) could be gathered from these similar projects:
Thanks for the input. I think I might start working on something over the weekend.
Any input as far as how to structure the data/schema for this?
I’m thinking of having the following DocTypes:
Automation Flow: The actual automation eg “7-day follow-up cadence”
Automation Step: The various steps/actions/components within the flow. eg “Send email template”
The Automation Flow will define various settings for the entire flow such as the DocTypes it can be executed against
The Automation Step will include:
The conditions required for step to be executed (eg, ReferenceDocument.Status != ‘Do Not Contact’)
How long to wait after the last step executed (similar to how Email Campaigns work, but relative to
the last step’s execution instead of being relative to the start of the campaign
The day/time window to execute the action (eg, Monday-Friday, 0900-1700)
Parent Step: The preceeding step that this step will follow
One thing I’m struggling with is the best way to link the Automation Steps to the referenced DocTypes…
I’m not a very good frontend developer, so unless somebody is capable of building a drag/drop UI, I think the MVP would probably just use a table UI to build out the Automation.
This is mainly just me spitballing here, so feel free to provide any input or criticism.
I think modeling something on the Workflow Builder makes sense.
In fact, it could be that the simplest and easiest-to-implement solution could be to expand the existing workflow doctype to be more flexible. Currently the workflow seems to be primarily for manually handling document approvals, but it could do much more.
Instead of menus being the only triggers, other events could start a workflow, including scheduled events. And more updates and actions could be executed at multiple stages within the flow.
The system would also need to handle multiple active workflows at one time, something it can’t currently handle.
Not sure if the existing workflow could be refactored to handle all this, or if it would need to be an entirely new module.
I agree completely with the concept of expanding the Workflow Builder. The only potential issue (and it may not even be an issue) might be the handling of permissions. For example, suppose you want to let your Sales Supervisor role make approval Workflows, but only a System User can create “automation” Workflows.
I doubt the permissions would be much of an issue, relative to the other programming involved.
IMHO the largest issue with the way workflows currently function is you can only have a single, monolithic workflow for each doctype. If you want diverging paths (say, for different order types, customers, or amount tiers), you need to build multiple conditions upon multiple conditions, resulting in a large cumbersome table–if it’s even feasible at all.
Ideally you could have multiple workflows enabled, where a document can enter and leave a particular workflow, and possibly enter a new one, even while a previous workflow is still active. It would require a 1:many instead of a 1:1 relationship, but definitely doable.
What if it works similar to how the Project templates work?
The Workflow could be a template that defines the Workflow Steps/Actions, and a separate set of Documents could be linked to the specific documents.
For instance:
Workflow - Stores the data about the workflow
Steps/Actions - Stores the individual steps that make up the Workflow
Document Actions - A set of actions to be completed that are linked to the document to be executed against/referenced. You could pause flows for specific documents, skip steps in the flow, or delete all upcoming actions for that document entirely. You could also easily view the completed and upcoming actions for individual documents.
Then the cron can loop through each of the ‘Document Actions’ and evaluate each one to see if the action should be executed. Since the database could grow quite large this way, perhaps completed actions should be purged from the database after a certain amount of time.
This sounds like a great suggestion. I think much of the community could benefit from a more robust automation tool, especially the non-programmers (which is probably the majority).
And done properly, eventually you might be able to consolidate a few similar but more rigid doctypes—notifications, workflows, assignment rules, etc.–into a single, much more flexible automation feature.
For reference, how Mautic handles this is with the following tables
campaigns defines basic info about the campaign campaign_events is the list of events in a particular campaign (the actions/steps) campaign_leads is the junction table - campaign to leads campaign_leadlist_xref is the junction table - campaigns to leadlist lead_list is a list of leads (Contacts) lead_list_leads is the junction table - lead_lists_leads campaign_lead_event_log is basically the “assignment” of an event to a lead
).
