Error Log in email account every 4 minutes

Frappe Framework Bug
1

email_account.receive

Traceback (most recent call last):
  File "/home/frappe/frappe-bench/apps/frappe/frappe/email/doctype/email_account/email_account.py", line 280, in receive
    communication = self.insert_communication(msg, args=args)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/email/doctype/email_account/email_account.py", line 337, in insert_communication
    email = Email(raw)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py", line 372, in __init__
    self.parse()
  File "/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py", line 393, in parse
    self.process_part(part)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py", line 445, in process_part
    self.text_content += self.get_payload(part)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py", line 489, in get_payload
    charset = self.get_charset(part)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py", line 484, in get_charset
    charset = chardet.detect(frappe.safe_encode(part))['encoding']
  File "/home/frappe/frappe-bench/env/lib/python3.5/site-packages/chardet/__init__.py", line 34, in detect
    '{0}'.format(type(byte_str)))
TypeError: Expected object of type bytes or bytearray, got: <class 'email.message.Message'>

pull_from_email_account

{'kwargs': {'email_account': 'NFDHR ERP System'}, 'method_name': 'pull_from_email_account', 'event': 'all', 'log': <function log at 0x7ffbcc9ce488>, 'method': <function pull_from_email_account at 0x7ffbcc8f8ae8>, 'site': 'erptest.nfdhr.info', 'user': 'Administrator', 'job_name': 'pull_from_email_account|NFDHR ERP System', 'retry': 0, 'is_async': True}
Traceback (most recent call last):
  File "/home/frappe/frappe-bench/apps/frappe/frappe/utils/background_jobs.py", line 99, in execute_job
    method(**kwargs)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/email/doctype/email_account/email_account.py", line 724, in pull_from_email_account
    email_account.receive()
  File "/home/frappe/frappe-bench/apps/frappe/frappe/email/doctype/email_account/email_account.py", line 303, in receive
    raise Exception(frappe.as_json(exceptions))
Exception: [
 "Traceback (most recent call last):\n  File \"/home/frappe/frappe-bench/apps/frappe/frappe/email/doctype/email_account/email_account.py\", line 280, in receive\n    communication = self.insert_communication(msg, args=args)\n  File \"/home/frappe/frappe-bench/apps/frappe/frappe/email/doctype/email_account/email_account.py\", line 337, in insert_communication\n    email = Email(raw)\n  File \"/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py\", line 372, in __init__\n    self.parse()\n  File \"/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py\", line 393, in parse\n    self.process_part(part)\n  File \"/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py\", line 445, in process_part\n    self.text_content += self.get_payload(part)\n  File \"/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py\", line 489, in get_payload\n    charset = self.get_charset(part)\n  File \"/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py\", line 484, in get_charset\n    charset = chardet.detect(frappe.safe_encode(part))['encoding']\n  File \"/home/frappe/frappe-bench/env/lib/python3.5/site-packages/chardet/__init__.py\", line 34, in detect\n    '{0}'.format(type(byte_str)))\nTypeError: Expected object of type bytes or bytearray, got: <class 'email.message.Message'>\n",
 "Traceback (most recent call last):\n  File \"/home/frappe/frappe-bench/apps/frappe/frappe/email/doctype/email_account/email_account.py\", line 280, in receive\n    communication = self.insert_communication(msg, args=args)\n  File \"/home/frappe/frappe-bench/apps/frappe/frappe/email/doctype/email_account/email_account.py\", line 337, in insert_communication\n    email = Email(raw)\n  File \"/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py\", line 372, in __init__\n    self.parse()\n  File \"/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py\", line 393, in parse\n    self.process_part(part)\n  File \"/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py\", line 445, in process_part\n    self.text_content += self.get_payload(part)\n  File \"/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py\", line 489, in get_payload\n    charset = self.get_charset(part)\n  File \"/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py\", line 484, in get_charset\n    charset = chardet.detect(frappe.safe_encode(part))['encoding']\n  File \"/home/frappe/frappe-bench/env/lib/python3.5/site-packages/chardet/__init__.py\", line 34, in detect\n    '{0}'.format(type(byte_str)))\nTypeError: Expected object of type bytes or bytearray, got: <class 'email.message.Message'>\n"
2

Capture
Capture1161×682 38 KB

3

Thank you @Akram_Mutaher
Someone finally take a look of the Error log:

4

Note you both run on Python 3.5 where 3.6 is suggested instead - for example on code checkin to travis the automated tests run 3.6 and so on.

Other woes with 3.5 include Can't generate stock balance report - #6 by Joseph_Marie_Alba

1 Like
5

Thank you to pointing out.

Can you help how to upgrade to pyton 3.6?

In forum there is no indication.

6

Assuming you are on Ubuntu 16

7

@Joseph_Marie_Alba @clarkej
Thank you for your help.

It seem the following update procedure is not precise:

sudo apt-get install python3-dev
bench migrate-env python3
bench restart
bench update

it should exactly indicate the python version:
bench migrate-env python3.6

In this case the system is on python 3.6 instead of 3.5.

Now let see how it works.

8

please tell us if work with you

9

Depending on your (Ubuntu?) distro you may need to explicitly specify 3.6, for example:

sudo apt-get install python3.6

That creates a symbolic like so:

clarkej@ubuntu1804lts:~$ ls -al /usr/bin/python3
lrwxrwxrwx 1 root root 9 Oct 25  2018 /usr/bin/python3 -> python3.6
1 Like
10

@Akram_Mutaher @clarkej

Unfortunatelly it is not the case.

Same situation on python 3.6 (UBUNTU 16.04) :

pull_from_email_account
Error
{'method_name': 'pull_from_email_account', 'log': <function log at 0x7f5f18240378>, 'retry': 0, 'is_async': True, 'user': 'Administrator', 'kwargs': {'email_account': 'Sales'}, 'job_name': 'pull_from_email_account|Sales', 'event': 'all', 'method': <function pull_from_email_account at 0x7f5f180cabf8>, 'site': 'site1.local'}
Traceback (most recent call last):
  File "/home/frappe/frappe-bench/apps/frappe/frappe/utils/background_jobs.py", line 99, in execute_job
    method(**kwargs)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/email/doctype/email_account/email_account.py", line 724, in pull_from_email_account
    email_account.receive()
  File "/home/frappe/frappe-bench/apps/frappe/frappe/email/doctype/email_account/email_account.py", line 303, in receive
    raise Exception(frappe.as_json(exceptions))
Exception: [
 "Traceback (most recent call last):\n  File \"/home/frappe/frappe-bench/apps/frappe/frappe/email/doctype/email_account/email_account.py\", line 280, in receive\n    communication = self.insert_communication(msg, args=args)\n  File \"/home/frappe/frappe-bench/apps/frappe/frappe/email/doctype/email_account/email_account.py\", line 337, in insert_communication\n    email = Email(raw)\n  File \"/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py\", line 372, in __init__\n    self.parse()\n  File \"/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py\", line 393, in parse\n    self.process_part(part)\n  File \"/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py\", line 445, in process_part\n    self.text_content += self.get_payload(part)\n  File \"/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py\", line 489, in get_payload\n    charset = self.get_charset(part)\n  File \"/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py\", line 484, in get_charset\n    charset = chardet.detect(frappe.safe_encode(part))['encoding']\n  File \"/home/frappe/frappe-bench/env/lib/python3.6/site-packages/chardet/__init__.py\", line 34, in detect\n    '{0}'.format(type(byte_str)))\nTypeError: Expected object of type bytes or bytearray, got: <class 'email.message.Message'>\n"
]  

email_account.receive
ERROR
Traceback (most recent call last):
  File "/home/frappe/frappe-bench/apps/frappe/frappe/email/doctype/email_account/email_account.py", line 280, in receive
    communication = self.insert_communication(msg, args=args)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/email/doctype/email_account/email_account.py", line 337, in insert_communication
    email = Email(raw)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py", line 372, in __init__
    self.parse()
  File "/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py", line 393, in parse
    self.process_part(part)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py", line 445, in process_part
    self.text_content += self.get_payload(part)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py", line 489, in get_payload
    charset = self.get_charset(part)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py", line 484, in get_charset
    charset = chardet.detect(frappe.safe_encode(part))['encoding']
  File "/home/frappe/frappe-bench/env/lib/python3.6/site-packages/chardet/__init__.py", line 34, in detect
    '{0}'.format(type(byte_str)))
TypeError: Expected object of type bytes or bytearray, got: <class 'email.message.Message'>

Installed Apps
ERPNext: v12.1.6 (version-12)

Frappe Framework: v12.0.16 (version-12)

11

ok you want to identify an example email that causes the problem. Once you have that, inspect that email’s source text - the goal is to get an example string that chardet fails on. The detect problem may relate to language locale. As well, the software that originated the email may contribute to the problem, for example Microsoft products.

This PR shows an example case fix(email): Fixed py3 chardet error - Expected object of type bytes or bytearray, got: <class 'str'> by ashish-greycube · Pull Request #8061 · frappe/frappe · GitHub

12

Interesting and indeed every email gets this message.
Every incoming .

I do receive email correctly but the log is filled with errors.

Are you @Akram_Mutaher facings the same issue?

13

I could not receive any email

but now I think my problem is solved after change Email Sync Option field value from ALL to UNSEEN in Email Account doctype

and error logs stoped now @krnkris please check if this work for you

14

@clarkej

I can ask many of our custmers, but many people are working in field, they don’t know about computer technical settings at all & the government offices will not care …

Any other solution, because on ERPNext 11v + python 2.7 there was no problem at all.
We can1t switch back ,because the python 2.7 will be not supported any more.

15

Yes these confirm that’s the case

Probably your best option is to advise users to switch to utf-8

16

@clarkej

Thank you , make my today happy.

Finally updated my system to UBUNTU 18.04,
tough it helps.
Nothing changed.

Dig it to charset in mysql database.

I had to allow in mariadb.cnf the # out lines and let set the charset to

[mysqld]

charser-set-server=utf8mb4 (it was utf8)
collation-server=utf8mb4_unicode_ci (it was general_ci)

Now it works!
No more error in list.

Please close this subject.

17

I also have logs flooded with these errors but a bit skeptical about making changes to mariadb.cnf as I don’t want that leading to any other issues somewhere down the line…

Shouldn’t this be fixed in the core?

Kind regards,