[New Feature] Background Reports

We’ve finally merged Background Reports (actual PR: Background Reports by pratu16x7 · Pull Request #5826 · frappe/frappe · GitHub), which allow you to set the ‘Prepared Report’ property in long running reports, so that they are henceforth generated in background.

Flow

The reports that have the ‘Prepared Report’ property checked, have a generate new report button. Reports are generated as CSV files and notified on completion. On the next viewing, the latest prepared report is fetched and displayed.
(relatively short report, so both initiation and completion alerts appear almost simultaneously).

ezgif com-optimize

Old prepared reports viewable,

Latest shown by default
(if the user is fine with the generated ones, he won’t need to create a new one).

ezgif com-optimize

Do test it out and let us know your thoughts. We’d appreciate your feedback.

13 Likes

Lets add an HTML template for the filters instead of showing raw JSON !

4 Likes

Will this fix the time-out issue with Analytics Reports ?

https://github.com/frappe/erpnext/issues/13664
https://github.com/frappe/erpnext/issues/14226

@rmehta Done: [prepared-report] show tabular filters by pratu16x7 · Pull Request #5851 · frappe/frappe · GitHub

@dhananjay The objective here is different from a complete rewrite as mentioned in the above issues; however this will enable the reports to be generated without breaking in the time they need.

The prepared reports is a good feature but has a lot of bugs and also there is no way to control if you don’t want a report NOT TO BE MADE prepared report.

The bugs are as below which are listed in the support tickets as well:

  1. Suppose user 1 creates a prepared report with MANDATORY filter ABC as value in the mandatory filter then if user 2 logs in to the system and enters the filter value (mandatory) XYZ then user 2 is shown the latest report generated by user 1 which is very confusing for user2 since he entered XYZ in filter value but got the report of ABC filtered values.

  2. Another major issue with prepared reports is that they don’t follow PERMISSIONS manager. I have just got the shocker when I realised that system automatically changed the Accounts Receivable report to PREPARED report without any user making it a prepared report. Now user1 logs in enters the filter for customer ABC and prepares the report and leaves then user2 logs into the report and enters the filter for customer as XYZ and is shown the values for customer ABC whom he is NOT AUTHORIZED to view, so if in this case the permissions are not followed I am presuming they would not be followed elsewhere as well.

  3. This one is quite annoying, I haven’t still been able to figure out the logic the system is taking to convert almost every other report to the prepared report. I am presuming that if a report is accessed many times the system automatically changes that to a prepared report now the idea behind prepared report is good but since the filtering of the data is creating a big problem the prepared are nothing but a load on the HDD space on the server. In the last 12 days since we have migrated to the v11 in our production server we have prepared report DB for 1374 reports with all the CSV.

My suggestions:

  1. There should be a way to NOT ALLOW a report to become PREPARED report
  2. Also there is no point in having ALL the reports in the DB with their CSV files, I think this is obsolete data and the system should automatically delete them, better still have a section for prepared reports in the System Settings and define the values for how long to keep prepared reports and delete them after that many days.
  3. Since the permissions are getting bypassed in the prepared report, either the user reports should not be shared or the user should not see the data from the csv.
3 Likes

Just got to know that the the system automatically makes a Standard report into prepared report if the total time taken is more than 10seconds for the report to generate.

My suggestion is that since the time out is set to 120seconds (i guess) then in that case the reports which take more than 90 seconds should be automatically converted to prepared reports the 10sec limit is ridiculously small and invariably it converts many reports into prepared reports.

1 Like

It’s very important to give the user ability to manage how these reports are generated and managed

For example, the user should be able to set the threshold where the reports get converted and also how many previously generated reports should be retained so that it doesn’t clutter the system

Kind regards,

It appears the permission issue with Prepared Reports is still there! This is quite shocking considering that many of the major reports in V11 are now Prepared Reports. I’ve opened a Github issue:

https://github.com/frappe/erpnext/issues/17648

Kind regards,

1 Like

The issue is still persistent in V15