Are there any good reasons for the artificial limitations imposed on the Notification Recipients?
1. Why must you always send it to ‘owner’ to be able to also send it to a static email address (i.e. CC field?)
Currently you cannot save it unless you also set an email field from the doctype, example ‘owner’
2. Why are Virtual fields not supported in the notification?
For example: One Client → Many Quotes
It is straight-up silly to store the client’s email address inside EACH quote
What if they change their email? You now have to scour the entire database to update each doc with the new email - was there ever a time when this made sense to anyone?
Notifications are the perfect use case for virtual fields, to ensure that they will always 100% reliably and accurately go to the client’s most up-to-date email address and avoid cluttering the database with unnecessary, repetitive and redundant data.
Additionally, another excellent use-case for virtual fields in combination with notifications is the following:
Per-Client Notification settings:
For example, a specific Client may be interested to:
Automatically receive emails about new Quotes Automatically receive emails with weekly Project Timesheets
These 2 preferences (along with the email) should be added as virtual fields in the Quote & Timesheet doctypes, set to always fetch the current values from the Client Doctype.
This way they are now available as conditions (and recipients) in the (Email) Notifications for when to send a specific notification to a specific client depending on their preferences.
The worst part is that most of this functionality is already built-in, but then purposely handicapped for no reason that I can make sense of.
How do you currently cope with scenarios like this?
Do you just keep stuffing your database with redundant data that then requires additional maintenance?
Many email clients require a to: field to be populated even if CC field is set.
You can enter any email address in the to field, it doesn’t have to be owner.
I’m not following “Virtual Field”. Can you define virtual field?
I recorded a video to show the process step by step:
What we’re doing:
Sending an email Notification to the Client, when a new ‘Custom Order’ is created.
Problems:
You cannot send a notification ONLY to a static email address (CC or BCC)
Frappe does NOT let you save the Notification unless you select a field from the doc (such as ‘owner’)
Owner = myself
I do not want to email myself, I want to email the Client
So we need to add a new field ‘client_email’ on the ‘Custom Order’ doctype.
Problems:
‘Fetch From’ fields (example client_email) are stored again in the (‘Custom Order’) table (redundant and duplicate data)
They are not just symbolic links to the original field
They duplicate the parent field data
‘Fetch From’ fields in Frappe only work once
They will never re-fetch values again when the ‘linked parent’ value changes
If the Client email changes, the email in the ‘Custom Order’ will never update, therefore it is now incorrect
In order to see the email field in the Notifications, we must add Options = ‘Email’ to the field.
However, it is still incorrect, because the Client email has since changed, yet the ‘Fetch from’ field does NOT reflect the change.
We should use a virtual field instead
The fetched value will always be automatically updated
It is not saved in the database (no redundant data, no maintenance)
Problems:
You cannot choose a virtual field in the recipients list!!!
Because the Virtual field requires ‘Options’ to take the command/instruction, you cannot add ‘Email’, therefore the ‘Notification’ cannot see or use that field, despite that field showing EXACTLY the value we need to work with.
I think you need to step back and decide if ERPNext is the right software for you. I sense a tone of disappointment and discontention in your posts. Please realize ERPNext is an Open Source software based on Frappe Framework. It is offered for free without warranty, yet full rights to modify as you see fit.
With that out of the way, in my experience… when frustration sets in and you’re wondering why in the world would a developer make such a decision, consider it could be user error, lack of documentation, unrealistic expectations, or perhaps the developer made a poor decision (in that order). I’ve found myself in your situation countless times, more often than not I was using the software wrong, missing documentation, or had unrealistic expectations. Occasionally, you’ll be on the sorry end of a bad decision (perhaps made ten years ago).
Now onto my advice. Take a look at how core doctypes handle this sort of workflow.
Sales invoices don’t store email addresses, yet I can automate a communication.
Please take the time to evaluate the behavior. Create an automated email when a new sales invoice is created. Later, change the customer’s email address and let us know if the new sales invoice is sent to the wrong/earlier address.
The Notification Recipient receiver_by_document_field is a blank select field that gets populated at runtime. As you’ve identified, there is a conflict between email addresses and virtual docfields over the use of Options. The result is that the Notification form can’t identify virtual docfields with inline logic that are also email addresses.
There are at least three options available to you:
Set the field manually using the console
Change the fieldtype from Select to Data and type in whatever you want
Come up with better way to do it and submit a pull request.
Thank you both for your thoughtful and respectful responses, despite my initial tone.
@volkswagner, your advice is sound, and I appreciate it. I’ve walked a similar path, moving from ERPNext to working directly with the framework.
My intention isn’t to criticize Frappe - I realize my 2 posts have questioned core functionalities that have been around for years.
Rather, I hope to start a constructive dialogue that either reaffirms the current design and educates newcomers that come with different experiences/perspectives, or explores potential improvements for everyone’s benefit.
I’ve noticed a similar tone in some GitHub discussions, even toward new contributors. Assuming this was acceptable, I mistakenly mirrored it.
@peterg, your suggestion to change the fieldtype to Data is intriguing. I took a more complicated route to achieve the same goal by modifying the code filtering receiver_by_document_field, specifically removing the restriction to Email fields which seemed to work at the time, hence my (albeit, very strong) choice of words - handicapping existing functionality:
frappe.notification = {
setup_fieldname_select: function (frm) {
[...]
frappe.model.with_doctype(frm.doc.document_type, function () {
[...]
let receiver_fields = [];
if (frm.doc.channel === "Email") {
receiver_fields = $.map(fields, function (d) {
// Add User and Email fields from child into select dropdown
if (frappe.model.table_fields.includes(d.fieldtype)) {
let child_fields = frappe.get_doc("DocType", d.options).fields;
return $.map(child_fields, function (df) {
>>>>>>>> return df.options == "Email" ||
(df.options == "User" && df.fieldtype == "Link")
? get_select_options(df, d.fieldname)
: null;
});
// Add User and Email fields from parent into select dropdown
} else {
return d.options == "Email" ||
(d.options == "User" && d.fieldtype == "Link")
? get_select_options(d)
: null;
}
});
It all seemed to work exactly as expected for a while, and I was eager to share it with the rest of the community. Until a few days later, when it simply stopped working.
My best guess is that it was due to ghost fields left behind, and when pushed to a staging environment, frappe was unable to compute the actual value of the (calculated) virtual field, so the recipient was always blank.
Therefore the issue still stands, and the only reliable way I found was to manually recreate the entire functionality of the Notification system using hardcoded backend functionality, just because virtual fields seem to be a taboo topic and they are not used/implemented very often (if at all).
A more thoughtful choice of words and tone for my original post:
I believe that the mandatory requirement to select a docfield as the receiver_by_document_fieldto be a minor bug
A more inclusive and better reflecting validation rule (of the interface and choices presented to the user) would be to: check if receiver_by_document_field is null ANDcc is null ANDbcc is null
As opposed to the current implementation which only checks if receiver_by_document_field is null, and disregards the CC and BCC fields entirely
I think the use-cases for virtual fields are very strong (and frappe does not deny this since virtual fields are part of the core framework), and I think everyone would benefit from adopting them more frequently, where suitable
Perhaps I am implementing them incorrectly in my Notification use case, and I am hoping that there’s something obvious that I’m missing
I also hope that I haven’t put everyone off from chiming in with my initial tone.
I really admire the fact that you’ve shifted your approach here. It’s easy for things to stay negative on the internet. Thanks for that.
If the notification worked and then stopped working, it might be worth trying to figure out why that is. Virtual Fields definitely have some limitations (see below), but the issue might also be issue the content of the Notification document. Many/most fieldtype distinctions in ERPNext are purely front end, and I don’t think there’s any difference in how the server or the database treat Data and Select fields.
I definitely agree there’s a huge amount of potential for virtual doctypes and fields, but there are also some important constraints on their functionality.
As I understand it, virtual fields are implemented at the document-controller level, which means that they work with methods that instantiate document objects (like frappe.get_doc) but not with API methods that poll the database directly (like frappe.get_list). This is a significant limitation for filtering or report building, but there’s not a simple and performant way to get around it.
There’s a further issue with virtual fields with logic defined in “Options” (as opposed to logic defined in the doctype_name.py controller). I’ve always found these to be very inconsistent. They tend to work correctly on the client side (and, strangely, in API calls), but they don’t always show up in server-side calls. Something weird is happening with caching (see here for some inconclusive discussion). That may be the issue you’re seeing.
It seems unintiutive to me to send emails without a TO recipient, only CC or BCC. I just checked on the two email systems I use and one allows it and the other doesn’t. I have no idea what the spec says, but it seems like something that different email platforms implement differently. You could probably un-mandatory the field with a Property Setter, but I’m not sure how the frappe sendmail implementation would respond. It would need a bit of testing.
As far as other functionality goes, you can always extend the doctype with a custom controller class. That way, you could add your own functionality while building on the bulk of the Notification doctype’s existing structures: https://docs.frappe.io/framework/v14/user/en/python-api/hooks#override-doctype-class
This would also allow for things like communication preferences to be implemented, of the sort you mentioned in your original post.
Likewise, I appreciate you giving this interaction a second chance after the rocky start
I attempted to investigate it, but I quickly hit roadblocks.
One of the (fairly common) challenges I face is the often inexplicable differences between Frappe’s ‘development’ and ‘production’ environments. Features that work flawlessly in development can behave differently or fail outright in production, even when the setup seems identical.
This inconsistency makes it hard to trust development findings, so I now prioritize testing directly in production:
(meaning staging). If a clever solution doesn’t work there, I often opt for a simpler, less pretty, more hardcoded and static approach.
Admittedly, my inexperience with Frappe is showing here, in case it wasn’t obvious so far
Regarding virtual fields, my database-oriented background makes me a big advocate for their use.
For instance, Frappe stores a user’s full_name as a static field, requiring manual updates, when it could (and should) be a virtual field that dynamically concatenates first_name, middle_name, and last_name.
Virtual fields offer numerous benefits: always accurate data, reduced developer effort, cleaner databases, less storage usage, better scalability, and improved performance. Most major database engines, including MySQL/MariaDB, support them, so not leveraging them feels like a missed opportunity.
I understand that Frappe’s virtual fields aren’t true database virtual fields and are only accessible within the frappe.get_doc context. While this might be a necessary compromise to achieve a database-agnostic environment, my findings so far share your exact sentiment: they’re often inconsistent and unreliable. It’s like having a Philips screwdriver, but we’re mostly dealing with flathead screws.
For now, I’ve created a global Python function to handle notifications, including dynamic recipient calculations, HTML template rendering and communication entries. While it works, it feels like a hacky workaround - reinventing the wheel and adding maintenance overhead; when simply using a virtual field would have solved it (also since Notifications are sent from within the frappe.get_doc context, so we’re already 90% there).
Generally speaking, I’m comfortable enough to customize, extend or override core functionality to suit my needs, and since this is a very generous open-source software and many people more knowledgeable and experienced than me use and contribute to it, I’m very intrigued to see whether I’m using it incorrectly, or whether other people hit similar roadblocks and I can share my findings for the greater good of all of us.
I recognize that implementing virtual fields consistently across Frappe may require significant effort, but if enough of us see their value, I believe that we can make it happen.
I’m happy to contribute however I can, though I doubt I can drive this alone or bring it up myself as a discussion topic to the greater minds behind this framework.
I agree it’s unintuitive.
It’s a widely accepted practice that’s less about strict logic and more about meeting user expectations.
I noticed other people on this forum who expressed similar expectations - being able to use CC/BCC without a mandatory TO recipient.
This aligns with how major email providers like Gmail and Outlook operate, allowing emails to be sent with only CC or BCC recipients.
On the other hand, when talking about what Frappe brings to the table in terms of customizability, flexibility and power to create and handle complex scenarios with ease, I believe that revisiting this restriction continues to align with Frappe’s philosophies and practices.
Realistically, I’m fairly certain that this validation rule was put in place before the CC and BCC fields were added, and the framework has simply evolved since.
To my mind, there’s definitely place in the Frappe framework for two kinds of virtual fields:
Python controller @property-based, with no limits on scope but unavailable to database queries
SQL GENERATED column-based, with less functional flexibility but fully visible to the API.
The second would be great for table joins and simple combinations (first_name + last_name-type stuff). The first, meanwhile, would have a lot more power but also some performance-motivated constraints.
I can’t speak for the maintainers, but in principle at least I think there’s clear interest to seeing this functionality in the framework. At the same time, I understand why they’re very cautious about accepting pull requests that very large, very complicated, and very broad.
Things tend to have an easier time getting accepted if they can be broken down into small, incremental parts. I wonder if that’s possible here.
I can’t think why that would be. Production handles caching very differently (and possibly some things related to routing/serving/workers), but other than that I’m not sure where variations would come from.
If the differences you’re seeing involve inline-defined virtual fields, I still think it’s a caching problem. Inline virtual fields should probably be understood as client-side/api only.
I don’t have much of an opinion about whether “to-less” emails should be allowed, but it’s certainly something you could pitch to the repository maintainers. The virtual field issue is a different question.
If it were me, I’d definitely look into using the class override hook to extend the existing Notification doctype rather than making something from scratch. I don’t think you’d need much to do what you want.
Automatically receive emails about new Quotes
Automatically receive emails with weekly Project Timesheets
