Google email via OAuth not possible?

avc · June 12, 2023, 9:06am

Sorry, first approach was wrong Forget about SMTP settings

peterg · June 12, 2023, 9:15am

Thanks @avc, once again you come through!

I was copying the details here and didn’t realize I needed a scope named https://mail.google.com. It’s working splendidly. Thank you so much! I owe you.

avc · June 12, 2023, 10:06am

Hi:
I’m glad to been helpful … actually I owe you much more

peterg · June 13, 2023, 5:26am

That’s kind.

I have one more question in case you happen to know: do you have an idea if it is possible to authorize an email address that isn’t matched to a Frappe User? For example, if I want to set up a shared email account like “contact@example.com”, do I also have to create a user with the email address “contact@example.com”?

avc · June 13, 2023, 8:22am

Hi:

I can’t try this, but maybe you can achieve it with “send as” feature on Gmail

In Frappe side try to configure this account using “Use different Email ID” check in email account config.

This way one authenticated gmail user can send mail from different accounts.
Anyway, I am not an GMail experienced user …

revant_one · June 13, 2023, 9:24am

Token for combination of connected app + frappe user is created when frappe user clicks the Connect to {app} button on connected app form view.

This means multiple frappe users can create a token using same account from oauth2 provider.

E.g. you’ve Google workspace account newsletter@example.com, then multiple frappe users can create tokens by login using newsletter@example.com.
Not recommended as every user needs to know credentials and 2fa for newsletter@example.com

It just the flexibility available.

Recommended to create token for frappe users by login using their respective Google workspace accounts. No password or any 2fa sharing is necessary in this case.

peterg · June 15, 2023, 2:15pm

Thanks to you both. This makes a lot of sense. So long as app-specific passwords keep working for google, that seems like a better way to go for shared inboxes. I really appreciate the work and explanations.

rmeyer · January 26, 2024, 12:26pm

Google says:

Scope: https://mail.google.com/
Description: Full access to the account’s mailboxes, including permanent deletion of threads and messages This scope should only be requested if your application needs to immediately and permanently delete threads and messages, bypassing Trash; all other actions can be performed with less permissive scopes.

Any ideas why we are forced to use this excessive scope?

I’ve tried less permissive scopes, like https://www.googleapis.com/auth/gmail.insert, but they lead to this error message when trying to send an email:

Traceback with variables (most recent call last):
  File "apps/frappe/frappe/email/oauth.py", line 48, in connect
    self._connect_smtp()
      self = <frappe.email.oauth.Oauth object at 0x7f21adcc6ef0>
  File "apps/frappe/frappe/email/oauth.py", line 75, in _connect_smtp
    self._conn.auth(self._mechanism, lambda x: self._auth_string, initial_response_ok=False)
      self = <frappe.email.oauth.Oauth object at 0x7f21adcc6ef0>
  File "/usr/lib/python3.10/smtplib.py", line 662, in auth
    raise SMTPAuthenticationError(code, resp)
      self = <smtplib.SMTP object at 0x7f21a6d83a90>
      mechanism = 'XOAUTH2'
      authobject = <function Oauth._connect_smtp.<locals>.<lambda> at 0x7f21adcdac20>
      initial_response_ok = False
      initial_response = None
      response = 'REDACTED__SOME_LONG_TOKEN'
      code = 535
      resp = b'5.7.8 Username and Password not accepted. For more information, go to\n5.7.8  https://support.google.com/mail/?p=BadCredentials j32-20020a05600c1c2000b0040ecd453e8esm5687929wms.34 - gsmtp'
      challenge = b'{"status":"400","schemes":"Bearer","scope":"https://mail.google.com/"}'
smtplib.SMTPAuthenticationError: (535, b'5.7.8 Username and Password not accepted. For more information, go to\n5.7.8  https://support.google.com/mail/?p=BadCredentials j32-20020a05600c1c2000b0040ecd453e8esm5687929wms.34 - gsmtp')

revant_one · January 27, 2024, 4:50am

Try multiple scopes, combination might work?

Example: send, compose, insert, readonly

Found more scopes here: Choisir les champs d'application de l'API Gmail | Google for Developers

rmeyer · February 6, 2024, 3:54pm

I tried a couple of combinations but didn’t find any working one.

On a different note, I’ve just update the docs to make it more clear how to correctly configure the Connected App, using Gmail as an example:

https://frappeframework.com/docs/user/en/guides/app-development/connected-app

VIMAL_PATEL · December 2, 2024, 4:31pm

@rmeyer, thank you for creating/updating the amazing document on configuring the Connected App. It helped me a lot. I have now managed to use the Connected App with an Email Account successfully with the following additional steps:

Steps:

Setup connected app as described in this doc - Set up a new Connected App
Once the Connected App is saved, we should see a Connect to GMail button. Clicking on it creates a Token.
Now open the Email Account you wish to set up.
Select OAuth from the Method dropdown.
Untick the “Authenticate as Service Principle” if it is ticked.
Select the Connected App from the dropdown.
Select the Connected User. IMPORTANT: 1. The email account used to set up the Connected App must belong to this user, and 2. You must be logged in as the same user otherwise on save, you will get this error OAuth Error: Please Authorize OAuth for Email Account Notifications
Once saved, the ** Authorised API Access ** button will appear under the ** Method ** dropdown. Click on it. NOTE: It may happen that the button does not appear. In that case, try to untick the Enable Outgoing and Enable Incoming options and save it.
Once Authorised API Access succeded, it is all set.

volkswagner · March 29, 2025, 12:51pm

@VIMAL_PATEL would you mind sharing your knowledge on how(if possible) the email account can be used as the default outgoing email account?

Can the email account be added to Frappe user child table as an authorized email account (without this user setting up Oauth (logging into Google to authenticate)?

VIMAL_PATEL · March 31, 2025, 9:03am

@volkswagner Sure, could you please give me more details so I can point you in the right direction?

volkswagner · March 31, 2025, 10:27am

I’m not sure how to rephrase the question.

Can a single email account using Google Oauth be used by all users on the system? If so, do we just set it as the default account in email settings? After setting it as the default account, do we still need to add the email account to each system user’s settings as an authorized email account?

VIMAL_PATEL · March 31, 2025, 1:46pm

Aah, I understand now. Yes, in our case, we used a single Google Email Account for all the outgoing email notifications and set it as the Default Outgoing account.

I used the step mentioned in my original answers to set that up.
I hope this helps.

antgel · March 31, 2025, 3:23pm

Thanks @revant_one for pointing me here from Connected App for outgoing email via OAuth.

We’re currently sending outgoing mail with Google Workspace SMTP Relay with authentication off (limited by IP addresses). But I’m wondering now if what I want to do will even work as Google Workspace seems to use JWT, not OAuth for service accounts / domain-wide delegation. Is there a way to get ERPNext to work with Google Workspace for sending outgoing mail without linking to any specific user? (I also have admin on Google Workspace / Cloud.)

(We don’t want to create accounts for non-human users, they should be aliases, so we want to start with my account and use the “Use different Email ID” option.

I then tried setting OAuth up anyway with my personal company email account, just to see if that would work. I was able to set up the OAuth Connected App and authenticate with “Connect to…”.

But when I went back to Email Account, enabled SMTP authentication, and chose OAuth with this connected app, I got the following message:

Please check your email login credentials. smtplib.SMTPAuthenticationError: (535, b'5.7.8 Username and Password not accepted. For more information, go to\n5.7.8 https://support.google.com/mail/?p=BadCredentials 98e67ed59e1d1-3039f8b28acsm500967a91.10 - gsmtp')

What might I be doing wrong? ERPNext shouldn’t be sending a username and password.

(I did change the Email Domain server from smtp.gmail.com to smtp-relay.gmail.com.)

volkswagner · March 31, 2025, 3:31pm

@antgel have you tried using basic auth with a fictitious uername and password?

This may work with your ip authentication.

You may need the correct username with a fake password??

antgel · March 31, 2025, 3:57pm

The IP “authentication” (SMTP relay, lack of Basic / OAuth2 authentication) does work. But I’d rather be able to configure OAuth2 with a service account that isn’t related to a “real” user, or at least is connected to my user and using the “Use different email ID” option.

revant_one · April 1, 2025, 3:17am

Code for backend app token / client_credentials grant is there:

github.com

frappe/frappe/blob/d73c716fe0bba7a49ec38429d0ccdf677e4b2431/frappe/integrations/doctype/connected_app/connected_app.py#L151-L177


      
          	def get_backend_app_token(self, include_client_id=None):
          		"""Get an Access Token for the Cloud-Registered Service Principal"""
          		# There is no User assigned to the app, so we give it an empty string,
          		# otherwise it will assign the logged in user.
          		token_cache = self.get_token_cache("")
          		if token_cache is None:
          			token_cache = frappe.new_doc("Token Cache")
          			token_cache.connected_app = self.name
          		elif not token_cache.is_expired():
          			return token_cache
          
          		# Get a new Access token for the App
          		client = BackendApplicationClient(client_id=self.client_id, scope=self.get_scopes())
          		oauth_session = OAuth2Session(client=client)
          
          		token = oauth_session.fetch_token(
          			self.token_uri,
          			client_secret=self.get_password("client_secret"),
          			include_client_id=include_client_id,
          		)

This file has been truncated. show original

Does calling this works? “System Console” or bench --site ... console

c = frappe.get_doc("Connected App", "abcdefgh01")
c.get_backend_app_token(c.client_id)

If it shows the token in token cache we’ll figure out a way to use it.

the get_backend_app_token also needs to change. no need for passing it a client_id. it should be picked up from self/cls in stead of passing it as arg.

antgel · April 2, 2025, 4:40pm

Thanks so much for jumping in to explain that. I’m travelling next couple of days but will definitely take a look ASAP! Cheers