Encoding Error in python, how to resolve?

Frappe Framework Customization
1

Hi,

I have a customization in item code wherein the description is automatically generated. The problem is that I have started to get this encoding error after updating to v11, would need to know how could I resolve this error:

	

Traceback (most recent call last):
  File "/home/frappe/frappe-bench/apps/frappe/frappe/desk/form/save.py", line 22, in savedocs
    doc.save()
  File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 260, in save
    return self._save(*args, **kwargs)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 283, in _save
    self.insert()
  File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 222, in insert
    self.run_before_save_methods()
  File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 876, in run_before_save_methods
    self.run_method("validate")
  File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 772, in run_method
    out = Document.hook(fn)(self, *args, **kwargs)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 1048, in composer
    return composed(self, method, *args, **kwargs)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 1033, in runner
    add_to_return_value(self, f(self, method, *args, **kwargs))
  File "/home/frappe/frappe-bench/apps/rigpl_erpnext/rigpl_erpnext/rigpl_erpnext/item.py", line 22, in validate
    generate_description(doc,method)
  File "/home/frappe/frappe-bench/apps/rigpl_erpnext/rigpl_erpnext/rigpl_erpnext/item.py", line 380, in generate_description
    concat = str(prefix[0][0][1:-1]) + str('{0:g}'.format(d.attribute_value))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xd8' in position 1: ordinal not in range(128)

Traceback (most recent call last):
  File "/home/frappe/frappe-bench/apps/frappe/frappe/app.py", line 61, in application
    response = frappe.handler.handle()
  File "/home/frappe/frappe-bench/apps/frappe/frappe/handler.py", line 21, in handle
    data = execute_cmd(cmd)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/handler.py", line 56, in execute_cmd
    return frappe.call(method, **frappe.form_dict)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/__init__.py", line 1007, in call
    return fn(*args, **newargs)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/desk/form/save.py", line 22, in savedocs
    doc.save()
  File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 260, in save
    return self._save(*args, **kwargs)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 283, in _save
    self.insert()
  File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 222, in insert
    self.run_before_save_methods()
  File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 876, in run_before_save_methods
    self.run_method("validate")
  File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 772, in run_method
    out = Document.hook(fn)(self, *args, **kwargs)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 1048, in composer
    return composed(self, method, *args, **kwargs)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 1033, in runner
    add_to_return_value(self, f(self, method, *args, **kwargs))
  File "/home/frappe/frappe-bench/apps/rigpl_erpnext/rigpl_erpnext/rigpl_erpnext/item.py", line 22, in validate
    generate_description(doc,method)
  File "/home/frappe/frappe-bench/apps/rigpl_erpnext/rigpl_erpnext/rigpl_erpnext/item.py", line 380, in generate_description
    concat = str(prefix[0][0][1:-1]) + str('{0:g}'.format(d.attribute_value))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xd8' in position 1: ordinal not in range(128)

The code in question here is:

Now if you see the code there is already a reference to utf-8 here:

but still I am getting an error since the prefix which is getting encoding has a character Ø which is giving this encoding error, is there a way I can remove this error.

2

Is this value coming via some kind of data import or copying it from a spreadsheet ? You can try stripping those characters via regular expressions. Like this:

self.attribute_value = re.sub('[^\x00-\x7f]', r'', self.attribute_value)

More like cleanup. You can do so for others. There are more solutions mentioned on Stack Overflow like this one:

also what does :g do ?

3

Hi @root13F,

Thanks for the prompt reply, basically I cannot strip the special character as it needs to be a part of the description and the hence the concat variable.

Basically I have a prefix custom field in my item master table for the attributes child-table and where we define the prefix for any attribute value and which gets shown in the description for the customers.

I have tried to use ‘Ø’ and also I have tried ‘&#216’ but the surprising part is that both the cases don’t seem to work on my production server but the more surprising thing is that the code it seems is working perfectly fine on my local test environment.

Now I am kind of confused as to how to make sure that code runs exactly the same way it does on my test account.

4

Oh then you should try the solution mentioned in the Stack Overflow link I mentioned. It recommends usage of encode over str.