Google Calendar Integration Bug

Chaibi_Alaa · November 7, 2018, 8:39pm

Hello,

I followed Google Calendar Integration in order to link google calendar to my erpnext system and all went fine, I authorized the app and everything says all is good.

In the same time, once activated, I get nothing syncronized in my google calendar, and a bench of errors gets into the Error Log :

Traceback (most recent call last):
File “/home/frappe/frappe-bench/apps/frappe/frappe/data_migration/doctype/data_migration_connector/connectors/calendar_connector.py”, line 42, in _create_calendar
created_calendar = self.gcalendar.calendars().insert(body=calendar).execute()
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/googleapiclient/_helpers.py”, line 130, in positional_wrapper
return wrapped(*args, **kwargs)
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/googleapiclient/http.py”, line 837, in execute
method=str(self.method), body=self.body, headers=self.headers)
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/googleapiclient/http.py”, line 162, in _retry_request
resp, content = http.request(uri, method, *args, **kwargs)
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/google_auth_httplib2.py”, line 187, in request
self._request, method, uri, request_headers)
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/google/auth/credentials.py”, line 122, in before_request
self.refresh(request)
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/google/oauth2/credentials.py”, line 129, in refresh
'The credentials do not contain the necessary fields need to ’
RefreshError: The credentials do not contain the necessary fields need to refresh the access token. You must specify refresh_token, token_uri, client_id, and client_secret.

Traceback (most recent call last):
File “/home/frappe/frappe-bench/apps/frappe/frappe/data_migration/doctype/data_migration_connector/connectors/calendar_connector.py”, line 142, in insert_events
remote_event = self.gcalendar.events().insert(calendarId=self.account.gcalendar_id, body=event).execute()
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/googleapiclient/discovery.py”, line 730, in method
raise TypeError(‘Missing required parameter “%s”’ % name)
TypeError: Missing required parameter “calendarId”

{‘event’: None, ‘retry’: 0, ‘log’: <function log at 0x7fedd14a3758>, ‘site’: u’manager’, ‘job_name’: u’frappe.utils.background_jobs.run_doc_method’, ‘method_name’: u’frappe.utils.background_jobs.run_doc_method’, ‘method’: <function run_doc_method at 0x7fedd33182a8>, ‘user’: u’Administrator’, ‘kwargs’: {‘doc_method’: u’run_current_mapping’, ‘name’: u’6f2c6d8f7b’, ‘doctype’: u’Data Migration Run’}, ‘is_async’: True}
Traceback (most recent call last):
File “/home/frappe/frappe-bench/apps/frappe/frappe/utils/background_jobs.py”, line 101, in execute_job
method(**kwargs)
File “/home/frappe/frappe-bench/apps/frappe/frappe/utils/background_jobs.py”, line 80, in run_doc_method
getattr(frappe.get_doc(doctype, name), doc_method)(**kwargs)
File “/home/frappe/frappe-bench/apps/frappe/frappe/data_migration/doctype/data_migration_run/data_migration_run.py”, line 73, in run_current_mapping
raise e
TypeError: Missing required parameter “calendarId”

chdecultot · November 7, 2018, 10:37pm

Hi @Chaibi_Alaa,

It seems that Frappe is not able to fetch your calendar ID.
Have you created a GCalendar account after the initial setup in GCalendar settings?
Does it have a GCalendar ID (automatically added upon validation of the account) ?

Chaibi_Alaa · November 7, 2018, 10:48pm

Hi and thanks for trying to help,

My google calendar account is created since years now I’m using as my calendar since a while, so no, it’s already created (we are talking about normal google calendar right ?) and I didn’t create anything in google calendar after setup in erpnext.

After the initial setup, nothing happened while erpnext says all is good, I still see the button saying authorize the app in erpnext.

FYI, the calendar I’m trying to link has an Id, here it is uj7bd1953pivkgfeokap3ll7u4

chdecultot · November 7, 2018, 10:53pm

Actually with the current design, you will need to create a new calendar directly from ERPNext.

You need to go to GCalendar Account and create a new account.
You can unfortunately not import your current calendar (could be a future enhancement )

Once the account created it will start syncing the events linked to this calendar ID.

In GCalendar you will be able to see your old calendar and this new calendar in the same view.

Chaibi_Alaa · November 7, 2018, 10:56pm

Excuse me I don’t get it, how can I create a new calendar from erpnext ?

chdecultot · November 7, 2018, 10:58pm

Just create a new GCalendar Account and authorize the access to GCalendar by clicking on the button “authorize access to GCalendar”

Chaibi_Alaa · November 7, 2018, 11:11pm

Okey so just did this (following what you requested) :

Created a new GCalendar Calendar
The Account and Sync went successfully in ERPNext (Same thing now, Authorize goes well and nothing changes later apart from the page that shows on the callback)
Named the Calendar same in Google

Same errors … may be I don’t get some point

chdecultot · November 7, 2018, 11:20pm

I’m sorry it’s not easy to help without seeing your whole configuration.

You shouldn’t have to rename anything in Google Calendar… Normally there are only two steps:

You create an app in Google Cloud and configure GCalendar Settings in ERPNext (one time config)
You create a GCalendar Account in ERPNext, authorize it and wait three-four minutes for the first synchronization to happen (config for each user)

I guess you’ll need to debug to find out what’s wrong…

Chaibi_Alaa · November 7, 2018, 11:34pm

I know and I’m thankfull for you trying to help. I don’t know how I can share with you my whole configuration.

So for the steps :

App is created, validated the domain, otherwise I wouldn’t be able to Authorize the App (means Callback URI is good, the domain as well)
Settings configured (Secret and ID)
While waiting for the synchronization, I get each 2 minutes the same 4 errors above.

Chaibi_Alaa · November 9, 2018, 12:03pm

Any other suggestion that can help please ?

bradbendy · December 11, 2018, 3:54pm

How are you guys getting the GCalendar part to show up in ERPNext? I don’t see that anywhere. From what I have read it just shows up but I do not see it anywhere. I do see something on GitHub but shows as depreciated and also fails when try to install. Am I missing something here?

Thanks

clarkej · December 11, 2018, 5:03pm

If you give pointers to your research, a screenshot detail or other handholds, folks can benefit and explore for themselves too?

bradbendy · December 11, 2018, 5:07pm

This article here Google Calendar Integration states it’s all done from ERPnext (what I can tell). Google searches show this GitHub page and this is the one it shows as deprecated. This is about all I can find it. https://github.com/DOKOS-IO/gcalendar

chdecultot · December 11, 2018, 5:13pm

Hi @bradbendy,

Which version of ERPNext are you using ? GCalendar integration is available from v11 onwards only.

System19 · December 12, 2018, 2:29am

Don’t use the DOKOS-IO repo. It is legacy third party and several forum users never managed to get it to work.
Just use the built in Google Services settings available in the Integration Module.

bradbendy · December 13, 2018, 11:14pm

It appears we had v10 installed when I was told v11. Trying to get it upgraded now. Never trust and always verify what you’re told is the moral to this story Hopefully we are good then.

Thanks

joshreeder · February 13, 2019, 8:56pm

Same issue here. I unhid the fields and it looks like the o-auth works but no calendar is created and linked. No calendar in google calendars either. I’m on v11. Anybody get this to work?

On the auth process. it authorizes, asks for permission, and then sends you to a frappe confirmation screen with return to desk button. No calendar is created.

How does it know if you want to send an event to which calendar?

Im so confused. You get o-auth working for google calendar, you let erpnext authorize and make a calendar for the user. 1 or more??? then you make an event and how do you know what calendars it will end up on? just yours? a company calendar? Participants do not include “Users” so how do you get an event on someones calendar? When the calendars are specifically linked to a user?

These questions answers are not dealt with in the instruction manual. Happy to update the instructions if i could understand the expected behaviors.

Please someone explain how to fix or how to make this work or what is wrong.

I thought maybe it didnt like me logged in with the oauth login. so i tried being logged in via google and via email. Same bug.

For the record i have 3 different google o-auth2 credentials set up: social login, gsuite, gcalendar. Yes, i removed the account and tried again. Yes i made calendar account after google calendar settings was setup properly. Yes calendar name in account is unique and not in google calendar already.

chdecultot · February 20, 2019, 2:51pm

Hi @joshreeder,

I’m using it everyday in production and it is working as expected (v11/Python3.6)…
You seem to have configured everything correctly, it’s strange that it doesn’t work.

On the auth process. it authorizes, asks for permission, and then sends you to a frappe confirmation screen with return to desk button. No calendar is created.

How does it know if you want to send an event to which calendar?

The calendar is created during the first synchronization. Currently the system creates a new calendar that is added to your Google Calendar list.
It would be nice to give the possibility to use an existing calendar in the future though.

Im so confused. You get o-auth working for google calendar, you let erpnext authorize and make a calendar for the user. 1 or more??? then you make an event and how do you know what calendars it will end up on? just yours? a company calendar? Participants do not include “Users” so how do you get an event on someones calendar? When the calendars are specifically linked to a user?

You only need 1 calendar per user. ERPNext will check if the user has the permission to see an event and will synchronize all events for which the user has permissions. Therefore all private, but owned by the user, and public events will be synchronized to match ERPNext’s view.

Participants are not synchronized at this stage, since they don’t match any specific email address like it would be expected by Google Calendar. It would be a nice enhancement.

The synchronization uses Frappe’s data migration tool.

Every 3-4 minutes Frappe creates a new data migration run for each gcalendar account.
You can check the status of the run in “Data Migration Run”.
If an error is thrown, it can be seen in the standard “Error Log” doctype.

Can you please check if a data migration run is correctly generated ? If not, it means that your scheduler is disabled or that you have an issue with it.

If you have a data migration run, can you check its status and if you have any related error in error logs ?

joshreeder · February 20, 2019, 4:40pm

@chdecultot can you confirm that on your install and current version you can setup a new calendar for a new/existing user? Maybe you set up yours on a different version and it is now broken???

migration connector is created.
migration run is not created.
no immediate errors, but several errors from last time i tried it. error just has my email address. Thats helpful!

@tmatteson

Also, THANK YOU FOR RESPONDING on here!!! You are amazing.

As soon as I can get this working I will be adding part of your response to the documentation.

chdecultot · February 20, 2019, 4:50pm

You’re welcome!

I can confirm that I just retested on a test server with v11.1.3 (Python2.7) and I have been able to setup a new calendar for a user.

If the connector is created and your user’s calendar account has all the necessary information (refresh_token, access_token, …) the problem comes probably from the background job.

Can you do a bench clear-cache and disable/enable the scheduler ?

Can you then check your failed background jobs to see if there is any error trace at this level ?

The error with your email address is not useful at all I agree, it should be cleaned up…