Email Account Configuration OAuth from Microsoft Office 365

[Jiri_Sir]

Any advice @revant_one or @avc, please?

[Jecintha]

@amjithlal @Jiri_Sir

I found the solution. Please follow the steps.

1. below-listed API permissions
   

   

2. Enter the scopes listed below.
   

   

3. Use the OAuth v2.0 authorization URL
   

   

4. Verify the Office 365 user has had modern authentication enabled (ex: MFA).

5. Add owners in app registration whose email you have configured in your email account in ERP.
   

[Jiri_Sir]

Hello @Jecintha ,
thank you very much for the answer, I appriciate it very much.
I have “copied” your configurations:

1. API permission without POP rows (we use just IMAP)
2. Scopes as you mentioned.
3. Endpoints too.
4. Admin enable MFA for my account (login with authenticator app in the mobile)
5. Owner is just me, for testing.

In frappe in my email account I have authorized the API, but the connection is still the same, it fails after token timeout .

What about the:
a) “offline_access” scope:

or b) email domain configuration:

or c) authetication setting in azure app (access tokens and ID tokens):

Can you share your settings, please?
Thank you very much in advance.

[Jecintha]

@Jiri_Sir

@Jiri_Sir

access tokens and ID tokens

Those access tokens and ID tokens do not need to be configured.

Can you please share your API permission list in Office 365?

To access email in Office 365, you must first verify the API list below.

1. Check that the API is selected from Microsoft Graph Application/Delegated.

Scopes are auto-configured from the API after the access token is generated.

Verify the below Office 365 app registration roles and assigned users.

1. Create app roles and select the “allowed members” options as “both.”
   

   

2. Next, navigate to enterprise applications, select your application, and then click on the properties button.
   Select the Assignment option.
   

   

3. Add the user to the list of users and groups.
   

   

[Jiri_Sir]

Hello @Jecintha,
thanks for you help.
First of all, during this night I have found two “working” gaps in error logs (emails arrived to linked Issue doctype):

Is it standard @avc @revant_one , please?

API permissions from Azure app are here (POP is missing):

But some scopes are missing in my auto-generated token (after creating the app roles, enabling assignment required and checking the users (As the owner I have the Default Access)):

Now I am waiting what will happen after one hour => still the same

Thank you.

[Patrick.St]

Hi together,
I have right now the same issue @Jiri_Sir , my token expire after 60-90mins.

@Jecintha

Scopes are auto-configured from the API after the access token is generated.

Which scopes have you added within the Connected App itself?
I have set it up as suggested here: Feat: use Connected App for OAuth based Email Account - #2 by revant_one

Do you have a “whitespace” after “offline_accesss” added to the row, as suggested Email Account Configuration OAuth from Microsoft Office 365 - #7 by revant_one here?

What is also a difference for me (and within @Jiri_Sir screenshot from the token), that we have the outlook.com prefix in comparison to you @Jecintha

I am not sure, why this is, yet

[revant_one]

try deleting the token cache and fetching token again.

may be with correct offline_access scope in connected app it’ll generate new token with offline access. i.e. it’ll get a refresh_token field populated in token cache

[Jecintha]

@Jiri_Sir
One more issue I am facing is that ERP-connected app users need email in Office 365.

When saving the email account configuration, it will verify the connected user’s email ID. So I’m using the alternative email option method.

Simply keep the email address and alternate email ID the same.

That option continues to work for me. No token has expired.

[Jiri_Sir]

Hello @revant_one ,
thanks for the advice. But I always delete the current token, and reauthorize API in annonymous browser window for “new” log in proccess when I change something.

I can see 8 scopes in @Jecintha printscreen (token) but in Frappe app config I can see just 3 scopes. Why?

Now I have these confs:

Is this expected behavior?

[Jiri_Sir]

Thanks @Jecintha ,
I will try that.
J.

[revant_one]

token cache needs to be present, it is the bearer token with refresh token. It will be used to regenerate new access token/bearer token when existing one expires.

if you delete token cache, then offline access or any email access is revoked immediately!

for missing scope,
My guess is, if you’ve created connected app with less scopes, generated a token with less scope and then added more scope to connected app? then token cache will keep refreshing existing scopes which are stored on token cache.

[Jiri_Sir]

Thanks @revant_one for answer.

I delete token cache just in moments when I change some scopes or confs in Azure app, after authorize API I am waiting for result, 60 - 90 minutes w/o deleting token cache ;-).

About scopes, I have created a new app with all scopes (IMAP.AccessAsUser.All, SMTP.Send, openid, offline_access and profile) but the response is token just with 3 scopes (IMAP.AccessAsUser.All, SMTP.Send and User.Read).

But now if I set Token URI in endpoints of connected app to v1: “…oauth2/token” instead of “oauth2/v2.0/token” than I have 7 scopes in token cache:

I will see the result up to 90 minutes

Errors again

[revant_one]

@Jiri_Sir

I will see the result up to 90 minutes

the logic doesn’t poll.

whenever get_active_token() is called,

• it’ll fetch token cache
• check if access token is expired
  • if access token is expired, it’ll request new bearer token using refresh token, new bearer token updates the token cache and expiry of token cache
  • if valid returns access token from cache

frappe/frappe/blob/3406b6d7527be04dc170d6fd0572ef786a5be144/frappe/integrations/doctype/connected_app/connected_app.py#L107-L124

[Jecintha]

@Jiri_Sir
Can you please share the full error log

[Jiri_Sir]

Of course @Jecintha,
here are two errors in one time, reapeting every 4th minute. But exist time gaps without errors.

First error:

  File "apps/frappe/frappe/email/oauth.py", line 44, in connect
    self._connect_imap()
      self = <frappe.email.oauth.Oauth object at 0x7efd8a6f6710>
  File "apps/frappe/frappe/email/oauth.py", line 72, in _connect_imap
    self._conn.authenticate(self._mechanism, lambda x: self._auth_string)
      self = <frappe.email.oauth.Oauth object at 0x7efd8a6f6710>
  File "/usr/lib/python3.10/imaplib.py", line 444, in authenticate
    raise self.error(dat[-1].decode('utf-8', 'replace'))
      self = <frappe.email.receive.Timed_IMAP4_SSL object at 0x7efd8a6f5c90>
      mechanism = 'XOAUTH2'
      authobject = <function Oauth._connect_imap.<locals>.<lambda> at 0x7efd8b717760>
      mech = 'XOAUTH2'
      typ = 'NO'
      dat = [b'AUTHENTICATE failed.']
imaplib.error: AUTHENTICATE failed.

And second one:

  File "apps/frappe/frappe/email/doctype/email_account/email_account.py", line 494, in get_inbound_mails
    email_server = self.get_incoming_server(in_receive=True, email_sync_rule=email_sync_rule)
      process_mail = <function EmailAccount.get_inbound_mails.<locals>.process_mail at 0x7efd909edf30>
      email_sync_rule = 'UNSEEN'
      mails = []
      self = <EmailAccount: Jiri Sir>
  File "apps/frappe/frappe/email/doctype/email_account/email_account.py", line 208, in get_incoming_server
    self.check_email_server_connection(email_server, in_receive)
      self = <EmailAccount: Jiri Sir>
      in_receive = True
      email_sync_rule = 'UNSEEN'
      oauth_token = <exception while printing> Traceback (most recent call last):
          File "env/lib/python3.10/site-packages/traceback_with_variables/core.py", line 222, in _to_cropped_str
            raw = print_(obj)
        TypeError: _get_traceback_sanitizer.<locals>.<listcomp>.<lambda>() takes 0 positional arguments but 1 was given
        
      args = {'email_account_name': 'Jiri Sir', 'email_account': 'Jiri Sir', 'host': 'outlook.office365.com', 'use_ssl': 1, 'use_starttls': 0, 'username': 'jiri.sir@beeinside.com', 'use_imap': 1, 'email_sync_rule': 'UNSEEN', 'incoming_port': 993, 'initial_sync_count': '250', 'use_oauth': True, 'access_token': 'eyJ0eXAiOiJKV1QiLCJub25jZSI6IjYyd29Wc0dkYmo1WFd6R2ljTGs3aUxuVldMUVJkREh3clpBa3k2U1pMV1EiLCJhbGciOiJSUzI1NiIsIng1dCI6Ii1LSTNROW5OUjdiUm9meG1lWm9YcWJIWkdldyIsImtpZCI6Ii1LSTNROW5OUjdiUm9meG1lWm9YcWJIWkdldyJ9.eyJhdWQiOiJodHRwczovL291dGxvb2sub2ZmaWNlLmNvbSIsImlzcyI6Imh0dHBzOi8vc3RzLndpbmRvd3MubmV0Lzc4NmJjMGZmLTlmODUtNDc1Mi1hOTM3LTBiZjY3Mzc2MDY2NS8iLCJpYXQiOjE2NzcxMjA5MTksIm5iZiI6MTY3NzEyMDkxOSwiZXhwIjoxNjc3MTI1MjYwLCJhY2N0IjowLCJhY3IiOiIxIiwiYWlvIjoiQVZRQXEvOFRBQUFBZ3FDeFMzRjlJQVJubzNMUlNRL1ZaUzV1bHQwUis0RFpQNjhqSzg2aGVxV096Q3hKZjdEUzhjRXRTSE4vb3d4ZGtaRDM3ZGFBczhHRUNWS0gxcm05ZnZKSWtiWGRBVXEyNFFoN2dRb3grYWs9IiwiYW1yIjpbInB3ZCIsIm1mYSJdLCJhcHBfZGlzcGxheW5hbWUiOiJlcnBuZXh0X2pzIiwiYXBwaWQiOiJjMjZkM2Yw...
      email_server = <frappe.email.receive.EmailServer object at 0x7efd8aa55300>
  File "apps/frappe/frappe/email/doctype/email_account/email_account.py", line 246, in check_email_server_connection
    frappe.throw(cstr(e))
      self = <EmailAccount: Jiri Sir>
      email_server = <frappe.email.receive.EmailServer object at 0x7efd8aa55300>
      in_receive = True
      auth_error_codes = ['authenticationfailed', 'loginfailed']
      other_error_codes = ['err[auth]', 'errtemporaryerror', 'loginviayourwebbrowser']
      all_error_codes = ['authenticationfailed', 'loginfailed', 'err[auth]', 'errtemporaryerror', 'loginviayourwebbrowser']
      message = 'authenticatefailed.'
  File "apps/frappe/frappe/__init__.py", line 525, in throw
    msgprint(
      msg = 'AUTHENTICATE failed.'
      exc = <class 'frappe.exceptions.ValidationError'>
      title = None
      is_minimizable = False
      wide = False
      as_list = False
  File "apps/frappe/frappe/__init__.py", line 493, in msgprint
    _raise_exception()
      title = None
      as_table = False
      as_list = False
      indicator = 'red'
      alert = False
      primary_action = None
      is_minimizable = False
      wide = False
      sys = <module 'sys' (built-in)>
      out = {'message': 'AUTHENTICATE failed.', 'title': 'Message', 'indicator': 'red', 'raise_exception': 1}
      _raise_exception = <function msgprint.<locals>._raise_exception at 0x7efd8a5e9870>
      _strip_html_tags = <functools._lru_cache_wrapper object at 0x7efd8a498460>
      inspect = <module 'inspect' from '/usr/lib/python3.10/inspect.py'>
      msg = 'AUTHENTICATE failed.'
      raise_exception = <class 'frappe.exceptions.ValidationError'>
      strip_html_tags = <function strip_html_tags at 0x7efd920a3370>
  File "apps/frappe/frappe/__init__.py", line 442, in _raise_exception
    raise raise_exception(msg)
      inspect = <module 'inspect' from '/usr/lib/python3.10/inspect.py'>
      msg = 'AUTHENTICATE failed.'
      raise_exception = <class 'frappe.exceptions.ValidationError'>
frappe.exceptions.ValidationError: AUTHENTICATE failed.

[Jecintha]

@Jiri_Sir
We had the same problem before we fixed it.
Can you please share the below screen shots?

1. Setup of email accounts
2. On your app registration, users and groups in the enterprise application.

[Jiri_Sir]

Here you are @Jecintha:
1.

2. Users and groups in the enterprise application:
   
   

Thank you very much.

[Jecintha]

@Jiri_Sir

1. Change the password on your Office 365 email account.
2. Remove your token cache.
3. After 15 minutes, click API-authorize to create the token.

[Jiri_Sir]

@Jecintha, thank you.
I will do that and I will let you know.

Unfortunately, errors are here again. What type of the Token URI do you use, please? v1 or v2?

[Jecintha]

@Jiri_Sir
We are used to OAuth 2.0 token endpoint (v1) token URI