Unable to configure IMAP

Sebastian_Lopez · April 25, 2020, 5:39pm

Hi,
I am attempting to configure our corporate email domain, and ran into an issue configuring the inbound server. Basically I cannot configure the IMAP server - same hostname, port and credentials work fine on different email clients we tried.

The only way I was able to configure inbound email is by using POP without SSL, and in this scenario the received emails are removed from the email server, which is not acceptable for us and I do not see any means of changing this behavior from the ERP.

I am using:
ERPNext: v12.6.0 (version-12)
Frappe Framework: v12.4.1 (version-12)

The error shown in the logs when attempting IMAP wSSL is the following:

> [ERROR] 2020-04-25 16:31:11,882 | /home/frappe/frappe-bench/apps/frappe/frappe/app.py:
> Site: erp.mydomain.com
> Form Dict: {
>  "action": "Save",
>  "cmd": "frappe.desk.form.save.savedocs",
>  "doc": "{\"name\":\"mydomain.com\",\"owner\":\"Administrator\",\"creation\":\"2020-04-10 13:06:15.743939\",\"modified\":\"2020-04-24 20:37:46.451213\",\"modified_by\":\"Administrator\",\"idx\":0,\"docstatus\":0,\"domain_name\":\"mydomain.com\",\"email_id\":\"erp@mydomain.com\",\"email_server\":\"server.emailprovider.com\",\"use_imap\":1,\"use_ssl\":1,\"incoming_port\":\"\",\"attachment_limit\":10,\"smtp_server\":\"server.emailprovider.com\",\"use_tls\":1,\"smtp_port\":\"587\",\"doctype\":\"Email Domain\",\"__last_sync_on\":\"2020-04-25T16:11:24.452Z\",\"__unsaved\":1}"
> }
> Request Error
> Traceback (most recent call last):
>   File "/home/frappe/frappe-bench/apps/frappe/frappe/email/doctype/email_domain/email_domain.py", line 76, in on_update
>     email_account.save()
>   File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 273, in save
>     return self._save(*args, **kwargs)
>   File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 309, in _save
>     self.run_before_save_methods()
>   File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 895, in run_before_save_methods
>     self.run_method("validate")
>   File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 794, in run_method
>     out = Document.hook(fn)(self, *args, **kwargs)
>   File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 1065, in composer
>     return composed(self, method, *args, **kwargs)
>   File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 1048, in runner
>     add_to_return_value(self, fn(self, *args, **kwargs))
>   File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 788, in <lambda>
>     fn = lambda self, *args, **kwargs: getattr(self, method)(*args, **kwargs)
>   File "/home/frappe/frappe-bench/apps/frappe/frappe/email/doctype/email_account/email_account.py", line 68, in validate
>     self.get_incoming_server()
>   File "/home/frappe/frappe-bench/apps/frappe/frappe/email/doctype/email_account/email_account.py", line 168, in get_incoming_server
>     email_server.connect()
>   File "/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py", line 43, in connect
>     return self.connect_imap()
>   File "/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py", line 51, in connect_imap
>     self.imap = Timed_IMAP4_SSL(self.settings.host, self.settings.incoming_port, timeout=frappe.conf.get("pop_timeout"))
>   File "/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py", line 564, in __init__
>     self._super.__init__(self, *args, **kwargs)
>   File "/usr/lib/python3.6/imaplib.py", line 1288, in __init__
>     IMAP4.__init__(self, host, port)
>   File "/usr/lib/python3.6/imaplib.py", line 198, in __init__
>     self.open(host, port)
>   File "/usr/lib/python3.6/imaplib.py", line 1301, in open
>     IMAP4.open(self, host, port)
>   File "/usr/lib/python3.6/imaplib.py", line 299, in open
>     self.sock = self._create_socket()
>   File "/usr/lib/python3.6/imaplib.py", line 1293, in _create_socket
>     server_hostname=self.host)
>   File "/usr/lib/python3.6/ssl.py", line 407, in wrap_socket
>     _context=self, _session=session)
>   File "/usr/lib/python3.6/ssl.py", line 817, in __init__
>     self.do_handshake()
>   File "/usr/lib/python3.6/ssl.py", line 1077, in do_handshake
>     self._sslobj.do_handshake()
>   File "/usr/lib/python3.6/ssl.py", line 689, in do_handshake
>     self._sslobj.do_handshake()
> ssl.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:852)

smino · April 25, 2020, 7:27pm

Can you say the operating environment? Is it self hosted?
Can you bench console then
import ssl and
print(ssl.OPENSSL_VERSION)`

Sebastian_Lopez · April 25, 2020, 9:26pm

hi @smino,
yes, this is self hosted on an Ubuntu 18,
I seem to be on SSL version 1.1.1:

~/frappe-bench$ bench console
In [2]: import ssl
In [3]: print(ssl.OPENSSL_VERSION)
OpenSSL 1.1.1 11 Sep 2018

smino · April 25, 2020, 11:09pm

If IMAP is working on email clients on the same machine, not just another computer with the same settings, it might be something else in the domain settings or something other than what the error message suggests. You can send email with the same provider?

Sebastian_Lopez · April 26, 2020, 10:21pm

hi,
yes, I am able to send email from the ERP (albeit with some delay) with secure SMTP. And I was able to configure the same provider IMAP server on different clients (Thunderbird, mobile email app).
Attempting IMAP connections from the same machine seems to work too:

server = imaplib.IMAP4_SSL(imap_ssl_host, imap_ssl_port)
server.login(username, password)
(‘OK’, [b’Logged in’])
server.select(‘INBOX’)
(‘OK’, [b’1’])

Now, I tried removing the user I was trying to configure, and I was able to configure the IMAP server in the domain settings, but then I go an configure the user again, and once I click on ‘Enable incoming’ the error appears. The domain settings are exactly the same as configured elsewhere, so it points to a problem with ERPNext. Any clues?

smino · April 27, 2020, 1:44am

Hi @Sebastian_Lopez
Can you get a screenshot Settings Email Domain and Account that you are trying?

Sebastian_Lopez · April 27, 2020, 11:09am

sure, shared below.
One detail that is not visible there, the Email Server name (both SMTP and IMAP) does not match with my domain (they are under the provider’s domain). Only thing ‘unusual’ I can think of in the setup.

smino · April 27, 2020, 2:14pm

Try adding the domain as seen here: sdf is a public shell service. A test case. See if it will save without error.

Sebastian_Lopez · April 27, 2020, 4:13pm

hi @smino,
actually it turns out the problem seemed to be the server mismatch. I tried with an alternate domain where the fqdn of the server matched with the domain (eg mail.myserver.com for erp@myserver.com) and I did not get any error.
Is this an expected behavior? sounds like a bug.

smino · April 27, 2020, 5:27pm

I think that the domains in the server and example should match, I don’t know that erp or example or another name in the example should matter. It doesn’t seem to here.

Sebastian_Lopez · April 27, 2020, 5:39pm

yeah, not the name of the user, but failing because the domain of the server does not match the email domain seems like a limitation. I mean, it is not an issue in other email clients I tried. Not sure if it should be reported as a bug though or whether it is just a documented limitation (documentation on email setup is very scarce tbh).

Anyway, thanks a lot for your help!

clarkej · April 27, 2020, 7:34pm

To debug that your SSL certificate is not the problem, connect to a ‘public’ service like imap.gmail.com and be sure to check your logs for clues Email error for inbox

edit: Here are more basic troubleshooting tips to try too Can't set up email in local environment

Be sure to followup with what you conclude and learn

Manu5118 · October 28, 2020, 8:45am

Hi, i don’t know if you have still the problem or want to know what the issue was/could be, but we had a similar or the same issue with the ssl_handshake.

The imap_port from the doctype email_account is hidden if you select a saved email_domain. This makes it difficult to see where the “bug” is.

In our case, the imap port was not updated and was still 143, not 993. The update method from email_domain.py did not include the IMAP port and this was the problem.

We created a issue with the needed pull requests on GitHub and it got merged on 18th October in develop and version-12-hotfix.
https://github.com/frappe/frappe/issues/11556