Internal Server Error on Power Failure

I am not very familiar with Linux :grimacing: I might stumble at basics.
Very interested in OpenSource ERP system; I decided to give it a shot and I am determined to make this work.

I downloaded the production image from erpnext.org fwdslash download and booted it on VirtualBox.
Everything looked great and I liked ERPNext. I had to shutdown and restart my system “Internal Server Error” shows up.

After searching around a bit, following running commands:

sudo mysqld --tc-heuristic-recover=ROLLBACK
sudo service mysql start

…everything was good again. A sudden power failure few hours later pushed me back to “Internal Server Error”. Trying the above rollback statements did not work.

systemctl status mariadb.service
results in:

A little help shall assist me long way.
A peek at what went wrong and how to troubleshoot will be of great help too. :slight_smile:

Update:
Part of /var/lib/mysql/mysql-error.log :


Hoping above log makes sense to someone here :neutral_face:

Desperate times desperate measures.

Here’s what I did to resolve my situation. I am still not sure how and what impact the following did on ERPNext server. Someone explaining me what happened will help in better understanding what happened:

  • Start the VBox server

  • In terminal run command: sudo vi /etc/mysql/my.cnf <<opens my.cnf a mariadb config file in vi editor

  • scrolled to the section where statement starts with “innodb_…”

  • Pressed “i” to enter insert mode

  • Added new line:
    innodb_log_checksums = ON

  • Press “Esc” key

  • Type “:wq” <<this saves and quits the vi editor

  • In terminal run command: rm /var/lib/mysql/ib_logfile*

  • In terminal run command: sudo reboot

Voila!
localhost:8080 is accessible again!

Today I learned:
linux is cool
vi is text editor
command line is not that scary
DO NOT PANIC if something goes wrong. Make a copy of disk and be brave :stuck_out_tongue:

I am sorry if this is too basic to be here on ERPNext forum. But I figured someone else might stumble upon same issue some day.

3 Likes

Oh also “Internal Server Error” does not point to one specific error.

It would show up if server fails to start ERPNext successfully if something went wrong.

Hi welcome to ERPNext

Well done - yes a web search on ‘Missing MLOG_CHEKPOINT’ turns up this mysql - InnoDB: Ignoring the redo log due to missing MLOG_CHECKPOINT - Database Administrators Stack Exchange

Here’s another case resolved by deleting those ib_logfile artifacts [solved] MariaDB 10.1 to MariaDB 10.2 upgrade crash

I think VB lets you take a snapshot copy of your disk image.

No in fact quite the opposite, your pointers will most certainly help many.

Possibly even yourself one day? If you can’t recall what you did now you have notes!

Hey thanks for the warm welcome. I’ll keep posting more about upcoming ERPNext adventures :smiley: