Max File Size allowed exceeded (413 Error)

1

When there is a relatively large doctype and trying to save the record, on frontend system throws error Max File Size allowed exceeded. But in Network tab of Developer tools, it shows 413 error. So initially tried with higher file size in System Setting. But whatever value I set, it gives error with this value. So even if I set 100 Mb, 200 Mb, it gives error with this value.

I tried setting this in site_config.json, but this also gives same error.

Third option I tried by changing nginx config by setting higher client_max_body_size. I tried with 50, 75,100,200,500 MB. But the system still gives error for file size set in System Setting or site config.

I am getting same error when I added some 20 fields in Standard Doctype through customize Doctype. When clicked on Update button on Customize form, it gives error Max File Size allowed exceeded.

I checked the content length being sent through frontend. It is not more than 1 Mb (As reported in Network Tab). I even checked size of utf encoded data which is also not more than 1 Mb. Below is the header of one such post request.

POST /api/method/frappe.desk.form.save.savedocs HTTP/1.1
Accept: application/json
Accept-Encoding: gzip, deflate, br, zstd
Accept-Language: en-IN,en-GB;q=0.9,en-US;q=0.8,en;q=0.7,mr;q=0.6,hi;q=0.5
Connection: keep-alive
Content-Length: 557424

3

Can someone please help with this? Because everything is stuck at the moment. I also checked for the solution on other platform. But everything leads up to client_max_body_size in nginx conf. is it something related to frappe?
Any help would be highly appreciated.

4

I checked all logs including web logs, nginx logs but nowhere 413 error is logged. So basically request is not reaching at nginx layer because on Dev tool it shows error generated from request.js code. Even for content length of 171 bytes system is throwing error. Is it a werkzeug error? If there is ready solution available, it would be really helpful.

5

If you have setup Production, add this line in supervisor.conf

–limit-request-line 0

  • command=/filepath/gunicorn -b 127.0.0.1:8001 -w 9 -t 120 --limit-request-line 0 frappe.app:application --preload

  • Save and exit

supervisorctl reload

6

Assuming you’ve already adjusted client_max_body_size you may want to set client_body_timeout. Its usually entire chain you need to figure out. So basically when you make a request it goes from: Nginx → Supervisor → Gunicorn. You need to check setting both at Nginx and Frappe level.

7

Thank you for your reply. I already tried with setting --limit-request-line 0 in supervisor.conf. But no solution.

8

Yes that too set in nginx configuration.

9

Resolved the issue. The issue is not particularly regarding Frappe Framework or NGINX or Gunicorn configuration. This particular error is occurring from Wekzeug which is a WSGI tool used in production setup.

Here I will try to explain how error got triggered, where it got triggered from, and how I solved it.
Below are the flow of events which triggered the error:

We have developed a custom doctype which has around 20 child tables. These child table may contain large number of rows. When I tried to save the record, then on site I got the dialogue box with message “File size exceeded the maximum allowed size of 50 MB“. Second, on browser console it was showing error coming from request.js file. This code is available at frappe/frappe/public/js/frappe/request.js at develop · frappe/frappe · GitHub

image
image666×212 5 KB

Above screenshot is from request.js file. So on frappe site it shows above dialogue box, in browser console tab it shows error coming from request.js and in network tab, it shows 413 error from NGINX. So first I tried with setting higher value for max file size in system settings, but this value is get picked while showing in the dialogue box. Second I tried with NGINX max_client_body_size. Third, I tried with higher configuration for gunicorn workers. But request was successfully processed by NGINX as there was no error log in NGINX. And my client body size was well below 1 MB.

As there was no error in bench, web, NGINX error logs, it confirmed that error is coming from somewhere else. As dialogue box was triggering from JS code, I thought that some python code is triggering the error and sending it to JS code which in turn popping up the dialogue box.

So it was the time to escape from the loop of suggested solutions from Google, every possible community out there and try some different approach. First I went through Frappe Documentation regarding production setup. There I saw werkzeug tool being setup in production environment. I gave it a try to find out werkzeug package in frappe setup. The werkzeug tool uses request.py to process the data coming in the request body. This file is located at bench/env/lib/python3.11/site-packages/werkzeug/wrappers.
This request.py file enforces various limitations on request body coming from client side.

image
image787×311 8.77 KB

The above screenshot is from request.py file where various limits are enforced. In latest werkzeug versions, they (not frappe) are enforcing these limits. In earlier versions there were no such limitations. One such limitation enforced with following variable:
max_form_memory_size: int | None = 500_000
This 500_000 converts into 500Kb. So when my client body size was exceeding 500Kb, werkzeug was triggering RequestEntityTooLarge error and sending it to request.js which then popping up the dialogue box with max file size and 413 NGINX error.

In earlier versions of werkzeug variable looked like below:
max_form_memory_size: int | None = None

So I had three options to solve this issue:

  1. Downgrade the Werkzeug version. This wasn’t feasible as there were other dependencies.

2.To directly the request.py file and remove 500Kb limitation. Though possible but it wasn’t the cleanest way to achieve it because making changes in the main file sometimes break the things up.
3.To override the variable value using _init_.py file in my custom file. This was the solution I adopted which ultimately resolved the issue. I added below lines in the _init_.py file of my custom app:
from werkzeug.wrappers import Request
Request.max_form_memory_size = None
So i

1 Like