[New Feature Proposal] Doctype Variant: more user friendly scenario specific user interface (form view) and permission control for complex process

Background
Currently except Single and Report, each new doctype will create one dedicated database table, for business transactions cover different business scenario(type) like Payment Entry to handle both Receive and Pay, per current design it is to be modelled in one doctype with control field: payment type, user can use the same form by selecting different payment type and input other depending fields to do his/her Receive or Pay payment.

Problem

  1. Covering all different business scenarios in one doctype/form by using control field to generate dynamic form is a great feature, but it often makes the single complicated doctype hard to maintain and evolve for new business scenario, and on user side, the actual form (user interface) for specific business scenario especially those simple one displayed a lot of irrelevant fields.

  2. As business scenario(Payment type) is Select other than Link field, there is no easy way to restrict user to create/change only one specific business type(Receive) while displaying other(or all) scenarios.

  3. Even though there is permission level at field level to control field’s availability per assigned role for each user, but doctype wise(database table) field level permission rule applies to all business scenarios on the same doctype. no easy way to adapt field level control per business scenario( records level).

Solution

multi doctypes (doctype variant) per database table,

use existing standard doctype as base doctype which defines all available fields in database table(the model), at the same time base doctype defines the default form(user interface, the view), other new doctypes (doctype variant) which has base doctype assigned can base on or add extra fields to base doctype’s database table, define other alternative form(view) for specific business scenario. in other word, there will be only one data source stored in database table, with different presentation( form ) per different business requirement.
in MVC( Model, View, Control) architecture, it is one model, one controller with multiple views. in SAP, it is called transaction variant, in Odoo, it is multi (form) views per model(defined in py file).

How it works

frappe.core.doctype.doctype.doctype

add 2 fields: base_doctype and filter to DocType doctype

  • base_doctype is the doctype which already mapped to database table, no base_doctype assigned to it or base_doctype is the same of the doctype name

  • for any new doctype with assigned to base_doctype,no new database table will be created, instead the corresponding base_doctype’s database table will be altered accordingly. the py controller class will have the base_doctype’s class instead of the generic Document as parent class.

  • Filter field defines the SQL where condition which will be applied to the get_list call to this doctype, this is to be acted as the permission control mechanism.

model.document.py / model.base_document.py/ model.db_query.py/database.database.py

each db.sql call with doctype as paramter , replace it with base_doctype as necessary

model.db_query
append the doctype.filter as where condition

desk.form.load
getdoc: if user has no permission for the base doctype, auto switch to the doctype variant with read.

other use cases

  1. Provide 2 versions of sales order form to different users, standard version with all information to sales order user and sales manager, simplified version with limited fields, i.e hide the sensitive price and amount info for other users

  2. Service item (base doctype Item) with service item relevant fields and filtered by item group equals to service,

  3. Material Receipt doctype (base doctype Stock Entry) with Stock Entry Type Material Receipt as default, hidden, assign this doctype to Material Receipt Role, assign only this role to warehouse keeper responsible for material receipt.

I have done the development on my local instance and tested by myself till now, it is kind of moderate change to the core, per discussion with core team, only if there is real customer use case, and the customers strongly support this new feature by put some money to it, and someone is going to use and test it in PRODUCTION, then there is chance it can be accepted into the core.

So I PIN this post on top for sometime to get the community feedback, please show you support by lIKE it or PM me.

PR is here
https://github.com/frappe/frappe/pull/7946

This PR is submitted for interested potential customer to see the code changes needed for the feature, adopt it into their own local system, test it and release to the production system to verfiy it works as expected, then there is chance for the core team to start review and consider merging into the core.

Many thanks.

15 Likes

Interesting, why not create a new Doctype named Form View, where different views of the doctype can be saved.

Role Permission Manager by default will select standard Form View (All the fields in Doctype) but it can be changed as per requirement.

We could also end up building Form Builder like Print Format builder.

good suggestion,then we can totally separate model(doctype) and view(form), I am also thinking whether we should consider supporting some sort of form inheritance, I. e allow form inherit from another parent form but overwrite some selected field attributes to reuse the form design.

I am skeptical whether Form inheritance will be useful or not, since one can always Duplicate a form and reuse it.

duplicate does not support parent form change automatically cascades to forms inherited from it ,
which will easily cause inconsistency after a while. odoo form is powerful on inheritance but it used customized xpath based XML grammar which make it more consultant other than end user oriented

Well in that use case it surely is nice feature to have. But still personally I would try not to use inheritance

After thinking a while, introduce a new doctype Form View will be a fundamental change to the core, because currently almost all important public API functions such as get_doc, get_list, get_all, db.set_value etc use doctype as parameter, for the new Form View doctype to be integrated, those public API need to be reviewed and changed accordingly. this will be a huge refactor work.

@szufisher, I think this is a good concept but wonder if this should not be implemented as below:

  • All doctype fields should be defined in the main doctype so that there is a central place to go and find all fields related to a doctype
  • Let us be able to define a new doctype of type variant that references the main doctype
  • The doctype variant defined above should have specifications that allow fields in the main doctype to be either hidden, read-only, editable or mandatory and maybe to customize layout. New fields should not be able to be defined in variant doctypes
  • Both main doctype and variant doctype behave like standard doctypes with respect to authorizations
  • However, only main doctypes should be reportable since they contain all the data and the variant doctypes are essentially views

This way, the presentation of doctypes can be varied depending on the specific business process and role accesses

Regards,
Chude

2 Likes

@Chude_Osiegbu
I agree with you, we can keep the main/base doctype as a central place for database table definition and business logic, other doctype variant is purely for different presentaion/view and authorization control(filter on record level, hide, read only , mandatory on field level) .
As I promised, if there is enough interest and feedback from the community member, I can submit the PR soon.

1 Like

I’m interested :slightly_smiling_face:. A great side effect of something like this is that we move away from having to write a lot of code in controllers to change presentation of a document based on specific field states and move towards achieving this same behaviour with configuration of variant doctypes. Less code. More value.

1 Like

Hi @szufisher

This would be very beneficial to all… hope to see it contributed soon

Cheers!

Any other community member like this proposed feature? or has any other comments?

I would like to see at least 5 LIKES before start the PR:smiley:.

3 Likes

Still one more LIKE needed to start the PR, :grin:

7 Likes

Are you going to implement it in some existing doctype too? Because that would be nice.

this feature is for creating existing doctype’s variant.

So, there is actually no doctype that will use this “as default”? Like, there is no “doctype’s variants” in the core of ERPNext?

Yes, this feature only enables end user to base on standard doctype creating his/her own doctype variant for specific business scenario, there will be no out of the box standard doctype variant after installation.

@szufisher

I do can offer an real user case!
I do have some customers that have Roles involved into the Sales Order

Sales User
Sales Manager
Accounts User

The Sales User is allowed to create the Order, by is not allowed to apply Order Discounts, but can apply discounts into the Items.

The Sales Manager, only is allowed to review the Order and Apply discounts on the grand total, on cases where it’s needed.
For Review, the Sales Manager only needs see (Customer Info, Address Info, Totals and Discounts)

The accounts User, needs also the same information of the Sales Manager, with the difference that him will attach the commissions for the 2 past users involved into the transaction, so Accounts User, can see Sales Team also, that is not allowed by the Sales Users and Sales Manager.

It’s a simple case, but I think It’s enough to offer one real Business Case.

I have been looking at this feature for long, especially in Payment/Receive and Stock Entry. It is difficult to make some users understand this One Doctype for many activities concept when some doctypes are used for operationally very different functions by the business