V12.1 EMAIL pull from server not working properly

Frappe Framework Bug
, ,
1

Only I facing this issue?
A lot of ERROR message get with my incoming emails.

pull_from_email_account
Error
{‘job_name’: ‘pull_from_email_account|Replies’, ‘method_name’: ‘pull_from_email_account’, ‘method’: <function pull_from_email_account at 0x7fa4089db488>, ‘is_async’: True, ‘kwargs’: {‘email_account’: ‘Replies’}, ‘log’: <function log at 0x7fa408aabe18>, ‘retry’: 0, ‘site’: ‘site1.local’, ‘event’: ‘all’, ‘user’: ‘Administrator’}
Traceback (most recent call last):
File “/home/frappe/frappe-bench/apps/frappe/frappe/utils/background_jobs.py”, line 99, in execute_job
method(**kwargs)
File “/home/frappe/frappe-bench/apps/frappe/frappe/email/doctype/email_account/email_account.py”, line 723, in pull_from_email_account
email_account.receive()
File “/home/frappe/frappe-bench/apps/frappe/frappe/email/doctype/email_account/email_account.py”, line 302, in receive
raise Exception(frappe.as_json(exceptions))
Exception: [
“Traceback (most recent call last):\n File "/home/frappe/frappe-bench/apps/frappe/frappe/email/doctype/email_account/email_account.py", line 279, in receive\n communication = self.insert_communication(msg, args=args)\n File "/home/frappe/frappe-bench/apps/frappe/frappe/email/doctype/email_account/email_account.py", line 336, in insert_communication\n email = Email(raw)\n File "/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py", line 372, in init\n self.parse()\n File "/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py", line 393, in parse\n self.process_part(part)\n File "/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py", line 445, in process_part\n self.text_content += self.get_payload(part)\n File "/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py", line 489, in get_payload\n charset = self.get_charset(part)\n File "/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py", line 484, in get_charset\n charset = chardet.detect(frappe.safe_encode(part))[‘encoding’]\n File "/home/frappe/frappe-bench/env/lib/python3.5/site-packages/chardet/init.py", line 34, in detect\n ‘{0}’.format(type(byte_str)))\nTypeError: Expected object of type bytes or bytearray, got: <class ‘email.message.Message’>\n”
]

email_account.receive
Error
Traceback (most recent call last):
File “/home/frappe/frappe-bench/apps/frappe/frappe/email/doctype/email_account/email_account.py”, line 279, in receive
communication = self.insert_communication(msg, args=args)
File “/home/frappe/frappe-bench/apps/frappe/frappe/email/doctype/email_account/email_account.py”, line 336, in insert_communication
email = Email(raw)
File “/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py”, line 372, in init
self.parse()
File “/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py”, line 393, in parse
self.process_part(part)
File “/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py”, line 445, in process_part
self.text_content += self.get_payload(part)
File “/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py”, line 489, in get_payload
charset = self.get_charset(part)
File “/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py”, line 484, in get_charset
charset = chardet.detect(frappe.safe_encode(part))[‘encoding’]
File “/home/frappe/frappe-bench/env/lib/python3.5/site-packages/chardet/init.py”, line 34, in detect
‘{0}’.format(type(byte_str)))
TypeError: Expected object of type bytes or bytearray, got: <class ‘email.message.Message’>

Installed Apps
ERPNext: v12.1.2 (version-12)

Frappe Framework: v12.0.12 (version-12)

On github:

https://github.com/frappe/frappe/issues/8361

2

I still receive these email issues:
As above + this:

email_account.receive

Error

Traceback (most recent call last):
File “/home/frappe/frappe-bench/apps/frappe/frappe/email/doctype/email_account/email_account.py”, line 280, in receive
communication = self.insert_communication(msg, args=args)
File “/home/frappe/frappe-bench/apps/frappe/frappe/email/doctype/email_account/email_account.py”, line 395, in insert_communication
communication._attachments = email.save_attachments_in_doc(communication)
File “/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py”, line 535, in save_attachments_in_doc
_file.save()
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 271, in save
return self._save(*args, **kwargs)
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 294, in _save
self.insert()
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 229, in insert
self._validate()
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py”, line 461, in _validate
self._validate_length()
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/base_document.py”, line 595, in _validate_length
self.throw_length_exceeded_error(df, max_length, value)
File “/home/frappe/frappe-bench/apps/frappe/frappe/model/base_document.py”, line 611, in throw_length_exceeded_error
.format(reference, (df.label), max_length, value), frappe.CharacterLengthExceededError, title=(‘Value too big’))
File “/home/frappe/frappe-bench/apps/frappe/frappe/init.py”, line 360, in throw
msgprint(msg, raise_exception=exc, title=title, indicator=‘red’)
File “/home/frappe/frappe-bench/apps/frappe/frappe/init.py”, line 346, in msgprint
_raise_exception()
File “/home/frappe/frappe-bench/apps/frappe/frappe/init.py”, line 315, in _raise_exception
raise raise_exception(msg)
frappe.exceptions.CharacterLengthExceededError: File 8111439148: ‘File Name’ (IBA-6522_v2_EN 10204 3_1_Szxxxxxxéxi_mixxxxx_bixxxxy_és_bexxxxxxlítxx_exxexx_Kxxxxxxx9022_20xxxx90x6_300p_ELSO_RxxxxxxxS_Kxxxxxxft_avdhxxA8-dxxxx9e83-44b2-837xxxxxxed535.pdf) will get truncated, as max characters allowed is 140

3

It seems in the firs case if the email has a name with á or é letter in it we receive the error.

On second case if the subject is more than 140 character then we receive the other error.

4

Now after reboot I have one more ERROR:

Cím
pull_from_email_account

Error

{‘user’: ‘Administrator’, ‘job_name’: ‘pull_from_email_account|Sales’, ‘is_async’: True, ‘method_name’: ‘pull_from_email_account’, ‘site’: ‘site1.local’, ‘method’: <function pull_from_email_account at 0x7f838bb43620>, ‘log’: <function log at 0x7f838bc1b048>, ‘retry’: 0, ‘event’: ‘all’, ‘kwargs’: {‘email_account’: ‘Sales’}}
Traceback (most recent call last):
File “/home/frappe/frappe-bench/apps/frappe/frappe/utils/background_jobs.py”, line 99, in execute_job
method(**kwargs)
File “/home/frappe/frappe-bench/apps/frappe/frappe/email/doctype/email_account/email_account.py”, line 724, in pull_from_email_account
email_account.receive()
File “/home/frappe/frappe-bench/apps/frappe/frappe/email/doctype/email_account/email_account.py”, line 303, in receive
raise Exception(frappe.as_json(exceptions))
Exception: [
“Traceback (most recent call last):\n File "/home/frappe/frappe-bench/apps/frappe/frappe/email/doctype/email_account/email_account.py", line 280, in receive\n communication = self.insert_communication(msg, args=args)\n File "/home/frappe/frappe-bench/apps/frappe/frappe/email/doctype/email_account/email_account.py", line 392, in insert_communication\n communication.insert(ignore_permissions=True)\n File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 228, in insert\n self.run_before_save_methods()\n File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 886, in run_before_save_methods\n self.run_method("validate")\n File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 786, in run_method\n out = Document.hook(fn)(self, *args, **kwargs)\n File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 1056, in composer\n return composed(self, method, *args, **kwargs)\n File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 1039, in runner\n add_to_return_value(self, fn(self, *args, **kwargs))\n File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 780, in \n fn = lambda self, *args, **kwargs: getattr(self, method)(*args, **kwargs)\n File "/home/frappe/frappe-bench/apps/frappe/frappe/core/doctype/communication/communication.py", line 65, in validate\n self.set_timeline_links()\n File "/home/frappe/frappe-bench/apps/frappe/frappe/core/doctype/communication/communication.py", line 261, in set_timeline_links\n contacts = get_contacts([self.sender, self.recipients, self.cc, self.bcc])\n File "/home/frappe/frappe-bench/apps/frappe/frappe/core/doctype/communication/communication.py", line 351, in get_contacts\n email = get_email_without_link(email)\n File "/home/frappe/frappe-bench/apps/frappe/frappe/core/doctype/communication/communication.py", line 404, in get_email_without_link\n email_host = email.split("@")[1]\nIndexError: list index out of range\n”
]

5

I checked the problematic emails subjects are:

  • [@Mail] Érdeklődés
  • Újítsd meg hirdetésed!
  • [@Mail] Térköves szerszàm

The subjects not cusing problems:

  • PgBouncer 1.12.0 released
  • pgAdmin 4 v4.14 released

It seems the Hungarian letters fails on chardet.

Can someone help on this?

6

Thank you , make my today happy.

Finally updated my system to UBUNTU 18.04,
tough it helps.
Nothing changed.

Dig it to charset in mysql database.

I had to allow in mariadb.cnf the # out lines and let set the charset to

[mysqld]

charser-set-server=utf8mb4 (it was utf8)
collation-server=utf8mb4_unicode_ci (it was general_ci)

Now it works!
No more error in list.

Please close this subject.