Email Account Configuration OAuth from Microsoft Office 365

[Jecintha]

Hi everyone,

I try to configured email account authentication method as ‘OAuth’ and its shown below error message please guide anyone how to configure email account based on OAuth.

Traceback with variables (most recent call last):
File “apps/frappe/frappe/email/doctype/email_account/email_account.py”, line 494, in get_inbound_mails
email_server = self.get_incoming_server(in_receive=True, email_sync_rule=email_sync_rule)
process_mail = <function EmailAccount.get_inbound_mails..process_mail at 0x7f1051b06050>
email_sync_rule = ‘UNSEEN’
mails = []
self = <EmailAccount: Vijayk>
File “apps/frappe/frappe/email/doctype/email_account/email_account.py”, line 208, in get_incoming_server
self.check_email_server_connection(email_server, in_receive)
self = <EmailAccount: Vijayk>
in_receive = True
email_sync_rule = ‘UNSEEN’
oauth_token = Traceback (most recent call last):
File “env/lib/python3.10/site-packages/traceback_with_variables/core.py”, line 222, in to_cropped_str
raw = print(obj)
TypeError: _get_sanitizer...() takes 0 positional arguments but 1 was given

  args = {'email_account_name': 'Vijayk', 'email_account': 'Vijayk', 'host': 'outlook.office365.com', 'use_ssl': 1, 'use_starttls': 0, 'username': 'vijayk@online.com', 'use_imap': 1, 'email_sync_rule': 'UNSEEN', 'incoming_port': 993, 'initial_sync_count': '250', 'use_oauth': True, 'access_token': 'eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6Ii1LSTNROW5OUjdiUm9meG1lWm9YcWJIWkdldyIsImtpZCI6Ii1LSTNROW5OUjdiUm9meG1lWm9YcWJIWkdldyJ9.eyJhdWQiOiIwMDAwMDAwMi0wMDAwLTAwMDAtYzAwMC0wMDAwMDAwMDAwMDAiLCJpc3MiOiJodHRwczovL3N0cy53aW5kb3dzLm5ldC8wNDA1ZGJiOC1kYmIyLTQ2YTgtOGQyOC1lOTYwNTU4MjQ5OTUvIiwiaWF0IjoxNjc1NDA4Mjg4LCJuYmYiOjE2NzU0MDgyODgsImV4cCI6MTY3NTQxMzA2NiwiYWNyIjoiMSIsImFpbyI6IkFWUUFxLzhUQUFBQXZnRDNGQWJ6dFo0dTMwLzN3K0E4ZjhJSFJ1ZkViTCsybWQraXhwR0RWT0FxSytzM0tKdWdQZFRBRjlsR0ttbElrSmowQnZpNDJZbGJzY2JVRU5FYTI1OGZpNlNrL2Y5b3Q4VkxRZzhNTVZBPSIsImFtciI6WyJwd2QiLCJtZmEiXSwiYXBwaWQiOiJlYmQyZjhmOS0yZmQzLTRhMTAtYTJhMC1mNmE3ZTAxOWI5ZWYiLCJhcHBpZGFjciI6IjEiLCJmYW1pbHlfbmFtZSI6IksiLCJnaXZlbl9uYW1lIjoiVmlqYXlhIEt1bW...
  email_server = <frappe.email.receive.EmailServer object at 0x7f10505f0370>

File “apps/frappe/frappe/email/doctype/email_account/email_account.py”, line 246, in check_email_server_connection
frappe.throw(cstr(e))
self = <EmailAccount: Vijayk>
email_server = <frappe.email.receive.EmailServer object at 0x7f10505f0370>
in_receive = True
auth_error_codes = [‘authenticationfailed’, ‘loginfailed’]
other_error_codes = [‘err[auth]’, ‘errtemporaryerror’, ‘loginviayourwebbrowser’]
all_error_codes = [‘authenticationfailed’, ‘loginfailed’, ‘err[auth]’, ‘errtemporaryerror’, ‘loginviayourwebbrowser’]
message = ‘authenticatefailed.’
File “apps/frappe/frappe/init.py”, line 525, in throw
msgprint(
msg = ‘AUTHENTICATE failed.’
exc = <class ‘frappe.exceptions.ValidationError’>
title = None
is_minimizable = False
wide = False
as_list = False
File “apps/frappe/frappe/init.py”, line 493, in msgprint
_raise_exception()
title = None
as_table = False
as_list = False
indicator = ‘red’
alert = False
primary_action = None
is_minimizable = False
wide = False
sys = <module ‘sys’ (built-in)>
out = {‘message’: ‘AUTHENTICATE failed.’, ‘title’: ‘Message’, ‘indicator’: ‘red’, ‘raise_exception’: 1}
_raise_exception = <function msgprint.._raise_exception at 0x7f105258a050>
_strip_html_tags = <functools._lru_cache_wrapper object at 0x7f105014c9e0>
inspect = <module ‘inspect’ from ‘/usr/lib/python3.10/inspect.py’>
msg = ‘AUTHENTICATE failed.’
raise_exception = <class ‘frappe.exceptions.ValidationError’>
strip_html_tags = <function strip_html_tags at 0x7f105320aa70>
File “apps/frappe/frappe/init.py”, line 442, in _raise_exception
raise raise_exception(msg)
inspect = <module ‘inspect’ from ‘/usr/lib/python3.10/inspect.py’>
msg = ‘AUTHENTICATE failed.’
raise_exception = <class ‘frappe.exceptions.ValidationError’>
frappe.exceptions.ValidationError: AUTHENTICATE failed.

[amjithlal]

did you get any solution?

[avc]

Frappe v14.23 and above supports IMAP Microsoft 365 accounts using OAuth.
You need to configure a connected app on Frappe with your registration app on your Azure AD side.

Hope this helps.

[Jiri_Sir]

Hello @Jecintha, did you solve the problem, please? I have configured Frappe app and Azure app and then connect them together as @avc mentioned. SMTP works ok, but IMAP (incoming mails) works about one hour and then fails with throw as you mentioned. I have to press button “Authorize API Access” for connect. It fails in hour again😞.

[avc]

Hi:

Default behavior is definable on Azure AD platform.

Configurable token lifetimes - Microsoft Entra

Hope this helps.

[Jiri_Sir]

Hello @avc,
thanks for quick answer. But unfortunately I cannot figure out, how to change the token lifetime value in Azure app I have read about offline_access scope:
" When a user approves the offline_access scope, your app can receive refresh tokens from the Microsoft identity platform token endpoint. Refresh tokens are long-lived. Your app can get new access tokens as older ones expire"
I have configured Frappe App with offline scopes (picture attached), but unfortunately it not help.

[revant_one]

offline_access is separate entry in scopes child table.
don’t prefix offline_access[space] on other scopes

everything is lower case in offline_access

[Jiri_Sir]

@revant_one, thank you very much for the advice. But unfortunately, it is not still working. Configuration is attached. There are still two throws in one time, the first one - mentioned by @Jecintha and the second one is below.
In token cache the refresh token is given, I think the frappe configuration is working, so. The Azure app configuration (API permission section) is attached.
Microsoft give back the basic authorization, please. I will sleep well, then

Traceback with variables (most recent call last):
  File "apps/frappe/frappe/email/oauth.py", line 44, in connect
    self._connect_imap()
      self = <frappe.email.oauth.Oauth object at 0x7f2b50af10c0>
  File "apps/frappe/frappe/email/oauth.py", line 72, in _connect_imap
    self._conn.authenticate(self._mechanism, lambda x: self._auth_string)
      self = <frappe.email.oauth.Oauth object at 0x7f2b50af10c0>
  File "/usr/lib/python3.10/imaplib.py", line 444, in authenticate
    raise self.error(dat[-1].decode('utf-8', 'replace'))
      self = <frappe.email.receive.Timed_IMAP4_SSL object at 0x7f2b4d5bbeb0>
      mechanism = 'XOAUTH2'
      authobject = <function Oauth._connect_imap.<locals>.<lambda> at 0x7f2b4d5e6560>
      mech = 'XOAUTH2'
      typ = 'NO'
      dat = [b'AUTHENTICATE failed.']
imaplib.error: AUTHENTICATE failed.

[Jiri_Sir]

Any advice @revant_one or @avc, please?

[Jecintha]

@amjithlal @Jiri_Sir

I found the solution. Please follow the steps.

1. below-listed API permissions
   

   

2. Enter the scopes listed below.
   

   

3. Use the OAuth v2.0 authorization URL
   

   

4. Verify the Office 365 user has had modern authentication enabled (ex: MFA).

5. Add owners in app registration whose email you have configured in your email account in ERP.
   

[Jiri_Sir]

Hello @Jecintha ,
thank you very much for the answer, I appriciate it very much.
I have “copied” your configurations:

1. API permission without POP rows (we use just IMAP)
2. Scopes as you mentioned.
3. Endpoints too.
4. Admin enable MFA for my account (login with authenticator app in the mobile)
5. Owner is just me, for testing.

In frappe in my email account I have authorized the API, but the connection is still the same, it fails after token timeout .

What about the:
a) “offline_access” scope:

or b) email domain configuration:

or c) authetication setting in azure app (access tokens and ID tokens):

Can you share your settings, please?
Thank you very much in advance.

[Jecintha]

@Jiri_Sir

@Jiri_Sir

access tokens and ID tokens

Those access tokens and ID tokens do not need to be configured.

Can you please share your API permission list in Office 365?

To access email in Office 365, you must first verify the API list below.

1. Check that the API is selected from Microsoft Graph Application/Delegated.

Scopes are auto-configured from the API after the access token is generated.

Verify the below Office 365 app registration roles and assigned users.

1. Create app roles and select the “allowed members” options as “both.”
   

   

2. Next, navigate to enterprise applications, select your application, and then click on the properties button.
   Select the Assignment option.
   

   

3. Add the user to the list of users and groups.
   

   

[Jiri_Sir]

Hello @Jecintha,
thanks for you help.
First of all, during this night I have found two “working” gaps in error logs (emails arrived to linked Issue doctype):

Is it standard @avc @revant_one , please?

API permissions from Azure app are here (POP is missing):

But some scopes are missing in my auto-generated token (after creating the app roles, enabling assignment required and checking the users (As the owner I have the Default Access)):

Now I am waiting what will happen after one hour => still the same

Thank you.

[Patrick.St]

Hi together,
I have right now the same issue @Jiri_Sir , my token expire after 60-90mins.

@Jecintha

Scopes are auto-configured from the API after the access token is generated.

Which scopes have you added within the Connected App itself?
I have set it up as suggested here: Feat: use Connected App for OAuth based Email Account - #2 by revant_one

Do you have a “whitespace” after “offline_accesss” added to the row, as suggested Email Account Configuration OAuth from Microsoft Office 365 - #7 by revant_one here?

What is also a difference for me (and within @Jiri_Sir screenshot from the token), that we have the outlook.com prefix in comparison to you @Jecintha

I am not sure, why this is, yet

[revant_one]

try deleting the token cache and fetching token again.

may be with correct offline_access scope in connected app it’ll generate new token with offline access. i.e. it’ll get a refresh_token field populated in token cache

[Jecintha]

@Jiri_Sir
One more issue I am facing is that ERP-connected app users need email in Office 365.

When saving the email account configuration, it will verify the connected user’s email ID. So I’m using the alternative email option method.

Simply keep the email address and alternate email ID the same.

That option continues to work for me. No token has expired.

[Jiri_Sir]

Hello @revant_one ,
thanks for the advice. But I always delete the current token, and reauthorize API in annonymous browser window for “new” log in proccess when I change something.

I can see 8 scopes in @Jecintha printscreen (token) but in Frappe app config I can see just 3 scopes. Why?

Now I have these confs:

Is this expected behavior?

[Jiri_Sir]

Thanks @Jecintha ,
I will try that.
J.

[revant_one]

token cache needs to be present, it is the bearer token with refresh token. It will be used to regenerate new access token/bearer token when existing one expires.

if you delete token cache, then offline access or any email access is revoked immediately!

for missing scope,
My guess is, if you’ve created connected app with less scopes, generated a token with less scope and then added more scope to connected app? then token cache will keep refreshing existing scopes which are stored on token cache.

[Jiri_Sir]

Thanks @revant_one for answer.

I delete token cache just in moments when I change some scopes or confs in Azure app, after authorize API I am waiting for result, 60 - 90 minutes w/o deleting token cache ;-).

About scopes, I have created a new app with all scopes (IMAP.AccessAsUser.All, SMTP.Send, openid, offline_access and profile) but the response is token just with 3 scopes (IMAP.AccessAsUser.All, SMTP.Send and User.Read).

But now if I set Token URI in endpoints of connected app to v1: “…oauth2/token” instead of “oauth2/v2.0/token” than I have 7 scopes in token cache:

I will see the result up to 90 minutes

Errors again