Email Account Configuration OAuth from Microsoft Office 365

@Jiri_Sir
One more issue I am facing is that ERP-connected app users need email in Office 365.

When saving the email account configuration, it will verify the connected user’s email ID. So I’m using the alternative email option method.

Simply keep the email address and alternate email ID the same.

That option continues to work for me. No token has expired.

Hello @revant_one ,
thanks for the advice. But I always delete the current token, and reauthorize API in annonymous browser window for “new” log in proccess when I change something.

I can see 8 scopes in @Jecintha printscreen (token) but in Frappe app config I can see just 3 scopes. Why?

Now I have these confs:

Is this expected behavior?

Thanks @Jecintha ,
I will try that.
J.

token cache needs to be present, it is the bearer token with refresh token. It will be used to regenerate new access token/bearer token when existing one expires.

if you delete token cache, then offline access or any email access is revoked immediately!

for missing scope,
My guess is, if you’ve created connected app with less scopes, generated a token with less scope and then added more scope to connected app? then token cache will keep refreshing existing scopes which are stored on token cache.

Thanks @revant_one for answer.

I delete token cache just in moments when I change some scopes or confs in Azure app, after authorize API I am waiting for result, 60 - 90 minutes w/o deleting token cache ;-).

About scopes, I have created a new app with all scopes (IMAP.AccessAsUser.All, SMTP.Send, openid, offline_access and profile) but the response is token just with 3 scopes (IMAP.AccessAsUser.All, SMTP.Send and User.Read).

But now if I set Token URI in endpoints of connected app to v1: “…oauth2/token” instead of “oauth2/v2.0/token” than I have 7 scopes in token cache:

I will see the result up to 90 minutes :wink:

Errors again :disappointed:

the logic doesn’t poll.

whenever get_active_token() is called,

  • it’ll fetch token cache
  • check if access token is expired
    • if access token is expired, it’ll request new bearer token using refresh token, new bearer token updates the token cache and expiry of token cache
    • if valid returns access token from cache

@Jiri_Sir
Can you please share the full error log

Of course @Jecintha,
here are two errors in one time, reapeting every 4th minute. But exist time gaps without errors.

First error:

  File "apps/frappe/frappe/email/oauth.py", line 44, in connect
    self._connect_imap()
      self = <frappe.email.oauth.Oauth object at 0x7efd8a6f6710>
  File "apps/frappe/frappe/email/oauth.py", line 72, in _connect_imap
    self._conn.authenticate(self._mechanism, lambda x: self._auth_string)
      self = <frappe.email.oauth.Oauth object at 0x7efd8a6f6710>
  File "/usr/lib/python3.10/imaplib.py", line 444, in authenticate
    raise self.error(dat[-1].decode('utf-8', 'replace'))
      self = <frappe.email.receive.Timed_IMAP4_SSL object at 0x7efd8a6f5c90>
      mechanism = 'XOAUTH2'
      authobject = <function Oauth._connect_imap.<locals>.<lambda> at 0x7efd8b717760>
      mech = 'XOAUTH2'
      typ = 'NO'
      dat = [b'AUTHENTICATE failed.']
imaplib.error: AUTHENTICATE failed.

And second one:

  File "apps/frappe/frappe/email/doctype/email_account/email_account.py", line 494, in get_inbound_mails
    email_server = self.get_incoming_server(in_receive=True, email_sync_rule=email_sync_rule)
      process_mail = <function EmailAccount.get_inbound_mails.<locals>.process_mail at 0x7efd909edf30>
      email_sync_rule = 'UNSEEN'
      mails = []
      self = <EmailAccount: Jiri Sir>
  File "apps/frappe/frappe/email/doctype/email_account/email_account.py", line 208, in get_incoming_server
    self.check_email_server_connection(email_server, in_receive)
      self = <EmailAccount: Jiri Sir>
      in_receive = True
      email_sync_rule = 'UNSEEN'
      oauth_token = <exception while printing> Traceback (most recent call last):
          File "env/lib/python3.10/site-packages/traceback_with_variables/core.py", line 222, in _to_cropped_str
            raw = print_(obj)
        TypeError: _get_traceback_sanitizer.<locals>.<listcomp>.<lambda>() takes 0 positional arguments but 1 was given
        
      args = {'email_account_name': 'Jiri Sir', 'email_account': 'Jiri Sir', 'host': 'outlook.office365.com', 'use_ssl': 1, 'use_starttls': 0, 'username': 'jiri.sir@beeinside.com', 'use_imap': 1, 'email_sync_rule': 'UNSEEN', 'incoming_port': 993, 'initial_sync_count': '250', 'use_oauth': True, 'access_token': 'eyJ0eXAiOiJKV1QiLCJub25jZSI6IjYyd29Wc0dkYmo1WFd6R2ljTGs3aUxuVldMUVJkREh3clpBa3k2U1pMV1EiLCJhbGciOiJSUzI1NiIsIng1dCI6Ii1LSTNROW5OUjdiUm9meG1lWm9YcWJIWkdldyIsImtpZCI6Ii1LSTNROW5OUjdiUm9meG1lWm9YcWJIWkdldyJ9.eyJhdWQiOiJodHRwczovL291dGxvb2sub2ZmaWNlLmNvbSIsImlzcyI6Imh0dHBzOi8vc3RzLndpbmRvd3MubmV0Lzc4NmJjMGZmLTlmODUtNDc1Mi1hOTM3LTBiZjY3Mzc2MDY2NS8iLCJpYXQiOjE2NzcxMjA5MTksIm5iZiI6MTY3NzEyMDkxOSwiZXhwIjoxNjc3MTI1MjYwLCJhY2N0IjowLCJhY3IiOiIxIiwiYWlvIjoiQVZRQXEvOFRBQUFBZ3FDeFMzRjlJQVJubzNMUlNRL1ZaUzV1bHQwUis0RFpQNjhqSzg2aGVxV096Q3hKZjdEUzhjRXRTSE4vb3d4ZGtaRDM3ZGFBczhHRUNWS0gxcm05ZnZKSWtiWGRBVXEyNFFoN2dRb3grYWs9IiwiYW1yIjpbInB3ZCIsIm1mYSJdLCJhcHBfZGlzcGxheW5hbWUiOiJlcnBuZXh0X2pzIiwiYXBwaWQiOiJjMjZkM2Yw...
      email_server = <frappe.email.receive.EmailServer object at 0x7efd8aa55300>
  File "apps/frappe/frappe/email/doctype/email_account/email_account.py", line 246, in check_email_server_connection
    frappe.throw(cstr(e))
      self = <EmailAccount: Jiri Sir>
      email_server = <frappe.email.receive.EmailServer object at 0x7efd8aa55300>
      in_receive = True
      auth_error_codes = ['authenticationfailed', 'loginfailed']
      other_error_codes = ['err[auth]', 'errtemporaryerror', 'loginviayourwebbrowser']
      all_error_codes = ['authenticationfailed', 'loginfailed', 'err[auth]', 'errtemporaryerror', 'loginviayourwebbrowser']
      message = 'authenticatefailed.'
  File "apps/frappe/frappe/__init__.py", line 525, in throw
    msgprint(
      msg = 'AUTHENTICATE failed.'
      exc = <class 'frappe.exceptions.ValidationError'>
      title = None
      is_minimizable = False
      wide = False
      as_list = False
  File "apps/frappe/frappe/__init__.py", line 493, in msgprint
    _raise_exception()
      title = None
      as_table = False
      as_list = False
      indicator = 'red'
      alert = False
      primary_action = None
      is_minimizable = False
      wide = False
      sys = <module 'sys' (built-in)>
      out = {'message': 'AUTHENTICATE failed.', 'title': 'Message', 'indicator': 'red', 'raise_exception': 1}
      _raise_exception = <function msgprint.<locals>._raise_exception at 0x7efd8a5e9870>
      _strip_html_tags = <functools._lru_cache_wrapper object at 0x7efd8a498460>
      inspect = <module 'inspect' from '/usr/lib/python3.10/inspect.py'>
      msg = 'AUTHENTICATE failed.'
      raise_exception = <class 'frappe.exceptions.ValidationError'>
      strip_html_tags = <function strip_html_tags at 0x7efd920a3370>
  File "apps/frappe/frappe/__init__.py", line 442, in _raise_exception
    raise raise_exception(msg)
      inspect = <module 'inspect' from '/usr/lib/python3.10/inspect.py'>
      msg = 'AUTHENTICATE failed.'
      raise_exception = <class 'frappe.exceptions.ValidationError'>
frappe.exceptions.ValidationError: AUTHENTICATE failed.

@Jiri_Sir
We had the same problem before we fixed it.
Can you please share the below screen shots?

  1. Setup of email accounts
  2. On your app registration, users and groups in the enterprise application.

Here you are @Jecintha:
1.

  1. Users and groups in the enterprise application:

Thank you very much.

@Jiri_Sir

  1. Change the password on your Office 365 email account.
  2. Remove your token cache.
  3. After 15 minutes, click API-authorize to create the token.

@Jecintha, thank you.
I will do that and I will let you know.

Unfortunately, errors are here again. What type of the Token URI do you use, please? v1 or v2?

@Jiri_Sir
We are used to OAuth 2.0 token endpoint (v1) token URI

Hi @Jecintha ,
I guess you are also using v1 for authorization URI? Have you set anything at “Revocation URI” and “Introspection URI”?

Hi @revant_one ,
thanks for the code preview, it seems clear.
What do you thing about no error time gaps, please?
Here error gaps with ca. 50 minutes duration and no error gaps with ca. 1,5 hours duration:

50 minutes - errors
1,5 hours - no errors
50 minutes - errors
1,5 hours - no errors
50 minutes - errors
1,5 hours - no errors

etc…

Thanks for your opinion in advance.

Thanks @Jecintha, I will try it.

Can you share screen shot from Azure → Users → “your user” → Assigned roles, please?

introspection endpoint depends on provider.

I think Microsoft doesn’t have introspection, they recommend to verify and use id_token.

1 Like

Another thing that came to my mind: As I understand the offline_access (token refresh) is working well for @revant_one and @Jecintha , right? In this discussion it seem that @Jiri_Sir and we have issues with the setup.

What kind of Azure AD license do you have? You can see that at Microsoft Azure
We are using right now “Azure AD Free” tier. Based on the documentation, that should be ok (10 apps are included). But who knows, what in-detail difference there might be in addition…

Hello all @Patrick.St , @avc, @Jecintha, @revant_one, @amjithlal,
I have found temporary solution for my issue.
I have added “small” server side script, that is hour scheduled. It get new tokens for all emails in array without taking care about expiration time of the previous tokens.

email_users = ['jiri.sir@bee.com','xyz@bee.com','abc@bee.com','xyt@bee.com'];
for email_user in email_users:
    connected_app = frappe.get_doc('Connected App', 'f28440895f');
    oauth_session = connected_app.get_oauth2_session(email_user);
    token_cache = connected_app.get_token_cache(email_user)
    token = oauth_session.refresh_token(body=f"redirect_uri={connected_app.redirect_uri}",token_url=connected_app.token_uri,);
    token_cache.update_data(token);

It is inspired by get_active_token function in connected_app.py in @revant_one comment: Email Account Configuration OAuth from Microsoft Office 365 - #21 by revant_one

I think there is a bug in frappe/token_cache.py at develop · frappe/frappe · GitHub
in:

	def get_expires_in(self):
		modified = frappe.utils.get_datetime(self.modified)
		expiry_utc = modified.astimezone(pytz.utc) + timedelta(seconds=self.expires_in)
		now_utc = datetime.utcnow().replace(tzinfo=pytz.utc)
		return cint((expiry_utc - now_utc).total_seconds())

	def is_expired(self):
		return self.get_expires_in() < 0

My actual tokens has been created in 8:48:29 and has expires in 4578 second, it means it will expire at 10:04:47. Now is ca. 10:05:00 but "is_expired() function gives me still “false” and the get_expires_in function gives me 3600 seconds, that is next one hour. But real token is expired.
This is the reason why exist time gaps, they took 52 minutes in error logs + plus 2 x 4 minutes (internal timer) and it is exactly 1 hour.

Is it problem of the time zones? My timezone is UTC + 1, Czech Republic.

Sorry! I have this thread silenced until now (i don’t know why :face_with_peeking_eye: …)
Connected app is working for us without issues. In this tenant there is a free AzureAD suscription (associated to M365 plan …). We are using v2 OAUTH endpoint …

I remember some kind of fight with scopes.
Connected app scopes:

Token scopes (automatic setted)

Hope this helps.