JobTimeoutException: Job exceeded maximum timeout value (300 seconds)

Frappe Framework Bug
21

Hi

thanks for the updates @clarkej John, this helped me to fix my same problem I received today with no email being sent or received.

I have not had any errors show up in normal usage or issues with scheduled processes in the web broswer.

But when i ran ‘show variables like ‘innodb_lock_wait_timeout’;’ in my server mariasql prompt, I got a result as so.

Database changed
MariaDB [mysql]> show variables like ‘innodb_lock_wait_timeout’;
±-------------------------±------+
| Variable_name | Value |
±-------------------------±------+
| innodb_lock_wait_timeout | 50 |
±-------------------------±------+
1 row in set (0.11 sec)

Obviously there is a lock in Mariasql which is stopping emails from being sent or received as per the forum post. I do not use Gmail, I have my own Exim server setup sensibly in the UK. But obviously something caused the innodb_lock still unknown, maybe spam .

I have now done:-

bench set-config scheduler_interval 600 instead of the default value of 300

to increase the timeout period, as I had another very similar problem about 3 months ago. At the time, I wasn’t clear what the issue was, but now I’m sure this was the same issue stopping email.

How should I go about calculating or estimating what a sensible scheduler interval may be?

I have a support contract with Frappe as this relates to my own server and asked if there’s anything they can do at all to help here, as this issue has occured for a few of us recently, and it’s not an easy one to debug and not all obvious what is stopping email. In the end just restarting mariadbsql fixed it for me, but that bit is easy .

22

Hi Julian,

“How should I go about calculating or estimating what a sensible scheduler interval may be?”

The provider decides on performance etc so I would say trial and error.

Here is the scoop with for example gmail

23

Hi John

Thanks for this. Just so I’m clear, you are saying basically that the email polling interval is affecting how the email scheduler via mysql in ERPNext works and thus if you have an email ie spam perhaps that is causing a timeout that this affects everything further down the chain ?

The mail server is wholly under my control as it uses Symbiosis https://symbiosis.bytemark.co.uk/docs/ch-mailreference.html so I can make changes in teh config if this will help.

Any thoughts or help you can offer are very gratefully accepted :wink:

24

scheduler_interval dictates how often the scheduler polls, to pull or push emails waiting in the queue at either client and server end.

So if the scheduler polls say every 60 seconds and the database has a lock timeout of say 30 or 50 seconds, then yes the ‘slow’ database might cause a bottleneck so that emails with say a large attachment do not transit within the lock timeout interval. You might need to add RAM or more cpu power or limit the size of emails.

Does this help answer your question?

My response to kirthi’s post # 19 did not relate to email poll frequency but instead to lock wait timeout…

1 Like
25

Hey, I’m struggling with the similar problem (unable to pull emails from gmail to erpnext) and mine is actually an instance created from an image. Do I need to change anything if this is the case?

Versions:
Frappe Framework: v8.0.61 (master)
ERPNext: v8.0.42 (master)

Error log list:
Title: pull_from_email_account
Error:
{‘retry’: 0, ‘log’: <function log at 0x7f0ba1a43140>, ‘site’: u’site1.local’, ‘event’: u’all’, ‘method_name’: u’pull_from_email_account’, ‘method’: <function pull_from_email_account at 0x7f0ba1202f50>, ‘user’: u’Administrator’, ‘kwargs’: {‘email_account’: u’Vishesh’}, ‘async’: True, ‘job_name’: u’pull_from_email_account|Vishesh’}
Traceback (most recent call last):
File “/home/frappe/frappe-bench/apps/frappe/frappe/utils/background_jobs.py”, line 65, in execute_job
method(**kwargs)
File “/home/frappe/frappe-bench/apps/frappe/frappe/email/doctype/email_account/email_account.py”, line 698, in pull_from_email_account
email_account.receive()
File “/home/frappe/frappe-bench/apps/frappe/frappe/email/doctype/email_account/email_account.py”, line 253, in receive
emails = email_server.get_messages()
File “/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py”, line 136, in get_messages
self.retrieve_message(message_meta, i+1)
File “/home/frappe/frappe-bench/apps/frappe/frappe/email/receive.py”, line 271, in retrieve_message
self.imap.uid(‘STORE’, message_meta, ‘+FLAGS’, ‘(\SEEN)’)
File “/usr/lib/python2.7/imaplib.py”, line 772, in uid
typ, dat = self._simple_command(name, command, *args)
File “/usr/lib/python2.7/imaplib.py”, line 1087, in _simple_command
return self._command_complete(name, self._command(name, *args))
File “/usr/lib/python2.7/imaplib.py”, line 909, in _command_complete
typ, data = self._get_tagged_response(tag)
File “/usr/lib/python2.7/imaplib.py”, line 1016, in _get_tagged_response
self._get_response()
File “/usr/lib/python2.7/imaplib.py”, line 928, in _get_response
resp = self._get_line()
File “/usr/lib/python2.7/imaplib.py”, line 1026, in _get_line
line = self.readline()
File “/usr/lib/python2.7/imaplib.py”, line 1188, in readline
return self.file.readline()
File “/usr/lib/python2.7/socket.py”, line 447, in readline
data = self._sock.recv(self._rbufsize)
File “/usr/lib/python2.7/ssl.py”, line 341, in recv
return self.read(buflen)
File “/usr/lib/python2.7/ssl.py”, line 260, in read
return self._sslobj.read(len)
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/rq/timeouts.py”, line 51, in handle_death_penalty
‘value ({0} seconds)’.format(self._timeout))
JobTimeoutException: Job exceeded maximum timeout value (300 seconds)

26

“Do I need to change anything if this is the case?”

Yes, the default 300 is too frequent to poll and pull from gmail - try 900 as noted here JobTimeoutException: Job exceeded maximum timeout value (300 seconds) - #18 by clarkej

27

Hi,

I changed the scheduler_interval to 1800 using below command
bench set-config scheduler_interval 1800

Still I am facing same issue “Task exceeded maximum timeout value (300 seconds)”. My scheduled job is marked as Complete without any error because of this.

Can someone please help.

28

@GouthamM As frappe.enqueue method has some parameters which you can use for your job.
You can do it in two ways:-

  1. You can define custom parameter queue=‘long’

frappe.enqueue(method, queue=‘long’, **kwargs)

2.Or You can just pass a parameter timeout=1800

frappe.enqueue(method, timeout=1800, **kwargs)

29

Seems below code can change default timeout

try to set below code in your common_site_config

"workers": {
    "default": {
        "timeout": 900
    },
    "short": {
        "timeout": 900
    }
 }

Then your background job default timeout will be change to 900 seconds

30

get error: “unsupported operand type(s) for //: ‘str’ and ‘int’”
solution: “unsupported operand type(s) for //: ‘str’ and ‘int’”

31

This worked for me,

sudo nano site_config.json

Then add “scheduler_interval”: 600, without quote for value.