Hi @peterschmidler i have the same problem as you and here how i solved it.
I don’t know much about server administration but I own a VPS at digitalOcean and i tryed out this command
sudo service mysql restart
It solved the problem but technically speaking don’t know why.
Hope we could find a solutions so when we have our application in a production environment it doesn’t crash.
Thanks 
In my case, the “Internal Server Error” was the result of MariaDB crashing due to lack of memory. Restarting MariaDB corrected the immediate problem, but the permanent solution was to reduce the InnoDB memory pool size. See Mysql (mariadb) crashing on DO Centos7 (SOLVED)
Fwiw, I have a 512MB droplet on Digital Ocean, and have experienced “cannot allocate memory” errors doing a “yum upgrade” (rebooting the server solved the problem the last time it happened, but shutting MariaDB down before yum upgrade would probably also work and be more robust).
@pdvyas @peterschmidler
I have same problem, restart the droplet will fix the problem, it is definitely the MariaDB problem because the database is failed. My web.error log file showing that the database is not able to connect. If I run service mysqld start it will show me the database is failed to start with error code.
Is it due to memory not enough problem? How should I fix the problem?
[2017-03-24 05:04:17 +0000] [712] [ERROR] Error handling request / Traceback (most recent call last): File "/home/john/frappe-bench/env/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 135, in handle self.handle_request(listener, req, client, addr) File "/home/john/frappe-bench/env/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 176, in handle_request respiter = self.wsgi(environ, resp.start_response) File "/home/john/frappe-bench/env/lib/python2.7/site-packages/werkzeug/local.py", line 228, in application return ClosingIterator(app(environ, start_response), self.cleanup) File "/home/john/frappe-bench/env/lib/python2.7/site-packages/werkzeug/wrappers.py", line 291, in application return f(*args[:-2] + (request,))(*args[-2:]) File "/home/john/frappe-bench/apps/frappe/frappe/app.py", line 88, in application frappe.db.rollback() File "/home/john/frappe-bench/apps/frappe/frappe/database.py", line 732, in rollback self.sql("rollback") File "/home/john/frappe-bench/apps/frappe/frappe/database.py", line 106, in sql self.connect() File "/home/john/frappe-bench/apps/frappe/frappe/database.py", line 52, in connect use_unicode=True, charset='utf8') File "/home/john/frappe-bench/env/lib/python2.7/site-packages/MySQLdb/__init__.py", line 81, in Connect return Connection(*args, **kwargs) File "/home/john/frappe-bench/env/lib/python2.7/site-packages/MySQLdb/connections.py", line 193, in __init__ super(Connection, self).__init__(*args, **kwargs2) OperationalError: (2002, 'Can\'t connect to local MySQL server through socket \'/var/lib/mysql/mysql.sock\' (111 "Connection refused")') [2017-03-24 05:04:18 +0000] [712] [ERROR] Error handling request / Traceback (most recent call last): File "/home/john/frappe-bench/env/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 135, in handle self.handle_request(listener, req, client, addr) File "/home/john/frappe-bench/env/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 176, in handle_request respiter = self.wsgi(environ, resp.start_response) File "/home/john/frappe-bench/env/lib/python2.7/site-packages/werkzeug/local.py", line 228, in application return ClosingIterator(app(environ, start_response), self.cleanup) File "/home/john/frappe-bench/env/lib/python2.7/site-packages/werkzeug/wrappers.py", line 291, in application return f(*args[:-2] + (request,))(*args[-2:]) File "/home/john/frappe-bench/apps/frappe/frappe/app.py", line 88, in application frappe.db.rollback() File "/home/john/frappe-bench/apps/frappe/frappe/database.py", line 732, in rollback self.sql("rollback") File "/home/john/frappe-bench/apps/frappe/frappe/database.py", line 106, in sql self.connect() File "/home/john/frappe-bench/apps/frappe/frappe/database.py", line 52, in connect use_unicode=True, charset='utf8') File "/home/john/frappe-bench/env/lib/python2.7/site-packages/MySQLdb/__init__.py", line 81, in Connect return Connection(*args, **kwargs) File "/home/john/frappe-bench/env/lib/python2.7/site-packages/MySQLdb/connections.py", line 193, in __init__ super(Connection, self).__init__(*args, **kwargs2) OperationalError: (2002, 'Can\'t connect to local MySQL server through socket \'/var/lib/mysql/mysql.sock\' (111 "Connection refused")')
I try to set my innodb_buffer_pool_size to 88M in /etc/my.cnf, but I don’t know why it becomes 512M, but now it seems to working well now.
And previously My system got memory allocation error when generate PDF. Now the problem gone already.
Now it seems the problem will still come back (on digitalocean VM), will anybody else encounter similar problem? now only restart server and database service will solve the problem. change the innodb_buffer_pool_size seems will not solve the problem at all. My innodb_buffer_pool parameters are showing as follow, what should we do to prevent the problem? It makes me worrying everyday about whether my system will stop working or not.
It will be really appreciated it this problem can be solved, this could be a major issue for ERPNEXT System.
-------------------------------------±---------------+
| Variable_name | Value |
±------------------------------------±---------------+
| innodb_buffer_pool_dump_at_shutdown | OFF |
| innodb_buffer_pool_dump_now | OFF |
| innodb_buffer_pool_dump_pct | 100 |
| innodb_buffer_pool_filename | ib_buffer_pool |
| innodb_buffer_pool_instances | 8 |
| innodb_buffer_pool_load_abort | OFF |
| innodb_buffer_pool_load_at_startup | OFF |
| innodb_buffer_pool_load_now | OFF |
| innodb_buffer_pool_populate | OFF |
| innodb_buffer_pool_size | 521142272
Hi All,
I have similar issue. Here is my error log. So how should I resolve this issue? Thank your kindly help.
Traceback (most recent call last):
File “/usr/lib/python2.7/runpy.py”, line 162, in _run_module_as_main
“main”, fname, loader, pkg_name)
File “/usr/lib/python2.7/runpy.py”, line 72, in _run_code
exec code in run_globals
File “/home/frappe/frappe-bench/apps/frappe/frappe/utils/bench_helper.py”, line 79, in
main()
File “/home/frappe/frappe-bench/apps/frappe/frappe/utils/bench_helper.py”, line 16, in main
click.Group(commands=commands)(prog_name=‘bench’)
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py”, line 722, in call
return self.main(*args, **kwargs)
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py”, line 697, in main
rv = self.invoke(ctx)
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py”, line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py”, line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py”, line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py”, line 535, in invoke
return callback(*args, **kwargs)
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/click/decorators.py”, line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File “/home/frappe/frappe-bench/apps/frappe/frappe/commands/init.py”, line 24, in _func
ret = f(frappe._dict(ctx.obj), *args, **kwargs)
File “/home/frappe/frappe-bench/apps/frappe/frappe/commands/site.py”, line 210, in migrate
migrate(context.verbose, rebuild_website=rebuild_website)
File “/home/frappe/frappe-bench/apps/frappe/frappe/migrate.py”, line 31, in migrate
frappe.modules.patch_handler.run_all()
File “/home/frappe/frappe-bench/apps/frappe/frappe/modules/patch_handler.py”, line 24, in run_all
executed = [p[0] for p in frappe.db.sql(“”“select patch from tabPatch Log”“”)]
File “/home/frappe/frappe-bench/apps/frappe/frappe/database.py”, line 107, in sql
self.connect()
File “/home/frappe/frappe-bench/apps/frappe/frappe/database.py”, line 53, in connect
use_unicode=True, charset=‘utf8mb4’)
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/MySQLdb/init.py”, line 81, in Connect
return Connection(*args, **kwargs)
File “/home/frappe/frappe-bench/env/local/lib/python2.7/site-packages/MySQLdb/connections.py”, line 193, in init
super(Connection, self).init(*args, **kwargs2)
_mysql_exceptions.OperationalError: (2002, ‘Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111 “Connection refused”)’)
Hi Hellorachel,
I believe now I got the solution. If you are using cloud server with RAM 1GB or less, it will easily got this issue. The whole system memory usage will easily goes over 1GB and makes the mysqld process crashed.
Previously I just reboot the droplet and start mysqld it will resume, but problem will come back again within a few days, the more user, the faster it comes back again.
So the suggestion is either increase your droplet/VM RAM size to 2GB or even more, or use the SWAP to make use of some of the harddrive to become your memory, but it is not recommended by digitalOcean as they are using SSD. You may refer to the link: How To Add Swap on CentOS 7 | DigitalOcean
I will create a new topic to let all people to check if this is the solution of common internal server error solution, I haven’t encounter problem for more than a week already. Previously the problem is quite frequently. As you can see my SWAP is starting to use already:
total used free shared buff/cache available
Mem: 993 629 106 56 256 168
Swap: 4095 240 3855
Hopefully this can help other people on this problem as I was in this trouble for quite a long time. If anybody got any better solution please let me know as my system is in use for my company, it needs to be stable and reliable.
Thanks & Best Regards,
John Yao
Your mysql service is not running, please try to restart the sql server using following command
sudo service mysql restart
Hi John_Yao,
Really appreciated your replay. I alsp agree your opinion because I check my server RAM almost was exhausted by redis service. So I upgrade my AWS ec2 from t2.micro to t2.small, so far it didnt crash again.
Thanks, I try but it show me restart fail. I upgrade my ec2 level then it work now. Thanks your kindly help.
You are welcome, I hope we can help each others to solve problems