Bug: Frappe API encoding case "TypeError: the JSON object must be str"

Frappe Framework Bug
, , , ,
1

Hello everyone , I found a bug in api.py but I’m not sure is it bug or not.

I have created API with PHP which is connected with Frappe/ERPNEXT ,I tried to insert new lead but I got error

Traceback (most recent call last):\n File "/home/frappe/frappe-bench/apps/frappe/frappe/app.py", line 66, in application\n response = frappe.api.handle()\n File "/home/frappe/frappe-bench/apps/frappe/frappe/api.py", line 117, in handle\n data = json.loads(frappe.local.form_dict.data)\n File "/usr/lib/python3.5/json/init.py", line 312, in loads\n s.class.name))\nTypeError: the JSON object must be str, not ‘bytes’\n

Firstly I thought it’s about my request, I checked with different post fields but i had always same error.

what i tried with curl :

curl_setopt($ch, CURLOPT_POSTFIELDS, array('data' => json_encode($params['data'])));
curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($params['data']));
curl_setopt($ch, CURLOPT_POSTFIELDS, '{"lead_name":"Sercan Tarakçı","email_id":"Sercan Tarakçı","source":"Campaign"}');

but it didnt work. Then i think that maybe it’s in api.py , I opened line 117

if frappe.local.request.method=="POST":
                                        data = json.loads(frappe.local.form_dict.data)
                                        data.update({
                                                "doctype": doctype
                                        })
                                        frappe.local.response.update({
                                                "data": frappe.get_doc(data).insert().as_dict()
                                        })
                                        frappe.db.commit()

I changed this line
data = json.loads(frappe.local.form_dict.data)
with this simple trick:
data = json.loads(frappe.local.form_dict.data.decode('utf-8'))

all my request works really fine now.Can someone explain is problem in my curl request or in python code?

3 Likes
2

I got a same error too but from Stock Projected Qty report page.

Traceback (most recent call last):
  File "/home/frappe/bench/apps/frappe/frappe/app.py", line 61, in application
	response = frappe.handler.handle()
  File "/home/frappe/bench/apps/frappe/frappe/handler.py", line 21, in handle
	data = execute_cmd(cmd)
  File "/home/frappe/bench/apps/frappe/frappe/handler.py", line 56, in execute_cmd
	return frappe.call(method, **frappe.form_dict)
  File "/home/frappe/bench/apps/frappe/frappe/__init__.py", line 1007, in call
	return fn(*args, **newargs)
  File "/home/frappe/bench/apps/frappe/frappe/__init__.py", line 489, in wrapper_fn
	retval = fn(*args, **get_newargs(fn, kwargs))
  File "/home/frappe/bench/apps/frappe/frappe/desk/query_report.py", line 177, in run
	result = get_prepared_report_result(report, filters, dn, user)
  File "/home/frappe/bench/apps/frappe/frappe/desk/query_report.py", line 203, in get_prepared_report_result
	data = json.loads(uncompressed_content)
  File "/usr/lib/python3.5/json/__init__.py", line 312, in loads
	s.__class__.__name__))
TypeError: the JSON object must be str, not 'bytes'

After adding .decode('utf-8') to json.loads(uncompressed_content) it work.

I’m using Python 3 with
ERPNext: v11.0.3-beta.31 () (staging)
Frappe Framework: v11.0.3-beta.40 () (staging)

Anyone knows what might be the problem here?

3

So I was working on something like this recently and the problem with the JSON encoding was the there were no new lines ("\n") after each comma - it has to be valid JSON and the new line is part of that. Encoded:

image

And the header:

image
image1152×196 6.37 KB

I’m not sure if this was your specific issue or not, but it was what got me over the hump.

1 Like
4

The problem is Python 3.x

The JSON section of this article (near the bottom) gets into better detail, but this same fix worked for me.

Before this can be an accepted fix, some testing has to be done that it doesn’t break on 2.7.

1 Like
5

I too encountered this error. It appears when i run query report having no data. i.e. on fresh instance, but once it has some data which appears as part of report. the said error goes away

6

Hi, do you know if someone is looking into this?
I can do Post requests, but not Put requests (v12 and Python v3)

7

i having the same issue with frappe v12.0.20. do anyone have proper solution?

8

Don’t use Ubuntu 16. Instead use Ubuntu 18

9

is this the solution?
I have added .decode(‘utf-8’) at the end of line 120 and 122 to solve this issue in frappe/api.py

10

You need to do this if you are on python 3.5. You don’t do this in python 3.6.
This is why for Erpnext, if you use Ubuntu 16, this error is going to bite you because it uses python 3.5.
If you use Ubuntu 18, you will not have this problem because it uses python 3.6.
The official Erpnext repository is tested in travis using python 3.6.

11

Thanks for help, I will try this soon and let people know here.