Wizard won't finish

1

I have spent days tracking this down. On a suggestion I rearranged my installation automation to follow the manual install rather than use install.py. After doing all that … the Wizard STILL won’t complete.

Right now, whether I hit retry when it times out, <ctrl><alt>R the browser and reenter the data or completely rebuild the server from scratch and reenter the data … the failure is the same:


[2020-02-16 11:11:31 -0500] [1544] [CRITICAL] WORKER TIMEOUT (pid:1918)
[2020-02-16 11:11:31 -0500] [1918] [ERROR] Error handling request /api/method/frappe.desk.page.setup_wizard.setup_wizard.setup_complete
Traceback (most recent call last):
  File "/home/erpnext/frappe-bench/apps/frappe/frappe/app.py", line 60, in application
    response = frappe.api.handle()
  File "/home/erpnext/frappe-bench/apps/frappe/frappe/api.py", line 55, in handle
    return frappe.handler.handle()
  File "/home/erpnext/frappe-bench/apps/frappe/frappe/handler.py", line 22, in handle
    data = execute_cmd(cmd)
  File "/home/erpnext/frappe-bench/apps/frappe/frappe/handler.py", line 61, in execute_cmd
    return frappe.call(method, **frappe.form_dict)
  File "/home/erpnext/frappe-bench/apps/frappe/frappe/__init__.py", line 1042, in call
    return fn(*args, **newargs)
  File "/home/erpnext/frappe-bench/apps/frappe/frappe/desk/page/setup_wizard/setup_wizard.py", line 71, in setup_complete
    task.get('fn')(task.get('args'))
  File "/home/erpnext/frappe-bench/apps/erpnext/erpnext/setup/setup_wizard/setup_wizard.py", line 92, in stage_fixtures
    fixtures.install(args.get('country'))
  File "/home/erpnext/frappe-bench/apps/erpnext/erpnext/setup/setup_wizard/operations/install_fixtures.py", line 276, in install
    set_more_defaults()
  File "/home/erpnext/frappe-bench/apps/erpnext/erpnext/setup/setup_wizard/operations/install_fixtures.py", line 296, in set_more_defaults
    add_uom_data()
  File "/home/erpnext/frappe-bench/apps/erpnext/erpnext/setup/setup_wizard/operations/install_fixtures.py", line 327, in add_uom_data
    }).insert(ignore_permissions=True)
  File "/home/erpnext/frappe-bench/apps/frappe/frappe/model/document.py", line 259, in insert
    self.run_post_save_methods()
  File "/home/erpnext/frappe-bench/apps/frappe/frappe/model/document.py", line 929, in run_post_save_methods
    self.run_method('on_change')
  File "/home/erpnext/frappe-bench/apps/frappe/frappe/model/document.py", line 787, in run_method
    out = Document.hook(fn)(self, *args, **kwargs)
  File "/home/erpnext/frappe-bench/apps/frappe/frappe/model/document.py", line 1058, in composer
    return composed(self, method, *args, **kwargs)
  File "/home/erpnext/frappe-bench/apps/frappe/frappe/model/document.py", line 1043, in runner
    add_to_return_value(self, f(self, method, *args, **kwargs))
  File "/home/erpnext/frappe-bench/apps/frappe/frappe/social/doctype/energy_point_rule/energy_point_rule.py", line 90, in process_energy_points
    if not is_energy_point_enabled():
  File "/home/erpnext/frappe-bench/apps/frappe/frappe/social/doctype/energy_point_settings/energy_point_settings.py", line 15, in is_energy_point_enabled
    return frappe.get_cached_value('Energy Point Settings', None, 'enabled')
  File "/home/erpnext/frappe-bench/apps/frappe/frappe/__init__.py", line 712, in get_cached_value
    doc = get_cached_doc(doctype, name)
  File "/home/erpnext/frappe-bench/apps/frappe/frappe/__init__.py", line 697, in get_cached_doc
    doc = get_doc(*args, **kwargs)
  File "/home/erpnext/frappe-bench/apps/frappe/frappe/__init__.py", line 740, in get_doc
    doc = frappe.model.document.get_doc(*args, **kwargs)
  File "/home/erpnext/frappe-bench/apps/frappe/frappe/model/document.py", line 70, in get_doc
    return controller(*args, **kwargs)
  File "/home/erpnext/frappe-bench/apps/frappe/frappe/model/document.py", line 105, in __init__
    self.load_from_db()
  File "/home/erpnext/frappe-bench/apps/frappe/frappe/model/document.py", line 135, in load_from_db
    single_doc = frappe.db.get_singles_dict(self.doctype)
  File "/home/erpnext/frappe-bench/apps/frappe/frappe/database/database.py", line 519, in get_singles_dict
    """, doctype)
  File "/home/erpnext/frappe-bench/apps/frappe/frappe/database/database.py", line 156, in sql
    self._cursor.execute(query, values)
  File "/home/erpnext/frappe-bench/env/lib/python3.6/site-packages/pymysql/cursors.py", line 170, in execute
    result = self._query(query)
  File "/home/erpnext/frappe-bench/env/lib/python3.6/site-packages/pymysql/cursors.py", line 328, in _query
    conn.query(q)
  File "/home/erpnext/frappe-bench/env/lib/python3.6/site-packages/pymysql/connections.py", line 517, in query
    self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  File "/home/erpnext/frappe-bench/env/lib/python3.6/site-packages/pymysql/connections.py", line 732, in _read_query_result
    result.read()
  File "/home/erpnext/frappe-bench/env/lib/python3.6/site-packages/pymysql/connections.py", line 1082, in read
    self._read_result_packet(first_packet)
  File "/home/erpnext/frappe-bench/env/lib/python3.6/site-packages/pymysql/connections.py", line 1152, in _read_result_packet
    self._read_rowdata_packet()
  File "/home/erpnext/frappe-bench/env/lib/python3.6/site-packages/pymysql/connections.py", line 1186, in _read_rowdata_packet
    packet = self.connection._read_packet()
  File "/home/erpnext/frappe-bench/env/lib/python3.6/site-packages/pymysql/connections.py", line 674, in _read_packet
    recv_data = self._read_bytes(bytes_to_read)
  File "/home/erpnext/frappe-bench/env/lib/python3.6/site-packages/pymysql/connections.py", line 691, in _read_bytes
    data = self._rfile.read(num_bytes)
  File "/home/erpnext/frappe-bench/env/lib/python3.6/site-packages/gunicorn/workers/base.py", line 201, in handle_abort
    sys.exit(1)
SystemExit: 1

The key line seems to be …

  File "/home/erpnext/frappe-bench/apps/erpnext/erpnext/setup/setup_wizard/operations/install_fixtures.py", line 327, in add_uom_data
    }).insert(ignore_permissions=True)

… but I could be wrong.

Comparing before and after database snapshots it is obvious that the middle tier is able to write to MariaDb.

I was advancing well with ErpNext until a week ago. Now I am completely stonewalled.

2

Do you try any solutions? What have you tried?

3

Martin,

This issue has existed for at least 3 years. Below is a link to my thread from March of 2019. In that post, I mention that I’d seen this problem in 2018 also.

I had this Wizard error happen while demonstrating ERPNext to a prospect. I was very embarrassed. :frowning:

Below is a link to the (still) Open Issue on GitHub, from 2016/2017.

The community is well-aware of this issue. But either:

  • People don’t have time to solve…
  • …or people are not capable of solving…
  • … or people don’t care enough to solve.

My Suspicions

  • The error is inconsistent and intermittent. Even with the exact same code base. I’ve had it happen during installation 3 times in a row. And then succeed the next 2 times immediately later.

  • I strongly suspect the root cause is a Race Condition, related to the database. I believe the database is still populating tables with rows, but the Wizard doesn’t wait for that to complete.

  • UOM seems a very likely candidate. So does Time Zone.

When I first encountered these issues, I couldn’t solve, because I didn’t know enough about Python and Frappe. I’m now probably capable. But over the past 3 years, I’ve become entirely fed-up with installation, Bench, and the entire setup and configuration process. So I permanently forked Bench. And have committed to writing a replacement installation tool.

In the meantime, the Setup Wizard remains unsolved. It happens to many people. They just try, try, try again. Until the stupid thing finally works.

I’m doing installations this week. Very likely I’ll encounter the same issue. If I solve it, I’ll share my code and solution somehow.

1 Like
4

Brian

I’m not sure we’re both referring to the same issue.

In my case, the Wizard does not freeze, in the sense of having to reload the page, I get
to the last page consistently. I hit “start installation” and it begins the initialization process. The progress bar stops for a long time at “installing presets” and then a Timed Out alert pops up.

Screenshot_2020-02-17_14-56-51

The retry button takes me back to the last wizard page where I can hit the button and run it again… But it then fails at the same place.

I am travelling right now and probably won’t be looking at this again 'til Thursday morning.

Thank you for the detailed response.

Martin

5

We’re talking about the same issue, yes.

6

Is there a way to do the same tasks using the API?

Is there a setting in the database that says the Wizard completed?

7

i have a similar problem when i choose south africa it freezes
this was reported but looks like the developers not really interested in fixing

8

Yes. In the SQL table “tabDefaultValue”, there is a setting named ‘desktop:home_page’

  • Before the wizard completes, the value is “setup-wizard”
  • After the wizard completes, the value is “desktop”

ERPNext examines the value, to determine whether the Wizard or Desktop should be displayed, when you navigate to the site.

Here’s the SQL query for that.

SELECT defvalue, defkey, parentfield
FROM <your_database_name>.tabDefaultValue
WHERE
defkey = 'desktop:home_page';

-- defvalue is either 'setup-wizard' or 'desktop'

There should be an API for the wizard. You should be able to install ERPNext, and transmit JSON data containing answers to all the wizard’s questions.

No idea if that is actually implemented, though.

9

Thanks for the SQL.

Are there operations performed by the Wizard that cannot be done by Administrator or using Bench?

Also, do you know if the ErpNext pay by the month service, permits SQL access? API access?

10

Bench definitely cannot do it (yet). If there are Administrator tasks for such a thing, I’ve never seen them.

I have no knowledge of Frappe’s hosting services, and what they allow clients to do.

11

This issue has been a problem for long. I think it’s time to fix it once and for all.

There are a few small patterns that we have identified. However they cannot be attributed to the failure as a whole yet.

@MartinHBramwell can you tell me the specifications of the machine you’re installing ERPNext on? Also the versions you’re installing

12

I recall revant_one suggested this workaround awhile back [Mar-2017] Showing there were error during setup ERPNext - #19 by revant_one

13

@clarkej - Thanks, John…that appears to be a good workaround. With a small effort, should be able to make a Bench CLI command that does that.

@scmmishra - I’ve seen nothing that indicates OS or version makes any difference. Bug has existed since at least version 7. That being said. I know for a fact it’s happened on a freshly-installed Ubuntu 18.04 with ERPNext version 12. You can start there, if you want to experiment.

14

Hey Clarke,

That link is hugely helpful.

Ultimately I’m going to go with this genius recommendation from that thread:

Meanwhile this is kind of a “WTF” for me:

Huh?

I would like to use the setup_complete(args) RPC that @revant_one suggested, but I would need all those different data points to be documented. Is that available somewhere?

15

FYI, I completed a clean install this morning. Ubuntu 18.10, ERPNext version 12. No errors during installation. No errors upon bench_start

Logged into http://localhost:8000, and completed the Setup Wizard.

First Attempt: Wizard failed for no apparent reason. Gave me option to “Retry”. So I clicked that.

Second Attempt: Wizard appeared to be working. The progress bar never moved. But my CPU was being utilized. So I patiently waited. Eventually, I was told the installation completed. However…my new user didn’t have access to DocTypes, and my browser refused to show any pages. Refreshing did not work. Browser page totally frozen.

Third Attempt: Closed my browser completely, and re-opening to http://localhost:8000. Was allowed to login as Administrator.

Conclusion:
Setup Wizard is still unreliable, broken, and only succeeds intermittently. Needs Rework.

16

Another day. Another ERPNext installation. This time for one of my clients. Ubuntu 18.04, ERPNext v12.3. Browser used was Opera.

No errors this time. The Wizard took a long time to complete. 4+ minutes. During this time the progress bar never moved a micron. I thought about giving up, but just kept starting at the screen, and the terminal where Bench was running. Then it finally refreshed, and took me straight to Desk.

Why better today than yesterday? Who knows.

Done testing Bench installations for a while; have other ERPNext issues to solve first.

17

To be honest i am wondering why you guys are facing this wizard issue. I have tested on cloud VPS, bare metal local pc and virtual pc (virtualbox, vmware and Hyper-V) and in all cases, wizard completed without issues after fixing the do not panic error after easy install. I couldn’t reproduce this wizard error.

18

Yesterday I spent the early morning with an ErpNext 14-days trial account, and all the rest of the day with a virtual machine and the Bitnami installer.

I was trying to find any other way to get the Wizard to work once, JUST ONCE, so I could back up the database and have it available to restore where I want it.

The trial account uses a different Wizard (is anyone surprised). It asked most, but not all of the questions of the publicly available Wizard. When I ran the backup I discovered that for the trial version the database archive is not included in the bundle (is anyone surprised).

So that was yet another dead end. On to Bitnami…

After figuring out how to workaround the defects in their installer (accepting that it set up mail services corrupts site-config.json) I found that it filled in some, but not all, of the Wizard stuff automatically, and worked ok, but did not set it up with what I wanted.

I ran bench drop-site then bench new-site and subsequent commands, and got to a clean Wizard.

After that, my experience was then the same as @brian_pond experienced yesterday: Wizard won't finish - #15 by brian_pond

… but then my second attempt, was like his third…

So I do now, at last, have a way to get a prepared database to work from.

I cannot explain why I had no problems with this 'til 10 days ago, then always problems with it since then. Nor can I explain why some people see it and not others.

The good news is I got some personal assistance from the founder @rmehta!! Yes!

He corrected the title of one of my posts to read “Is there any way at all to configure ERPNext without using the Wizard?” instead of “Is there any way at all to configure ErpNext without using the Wizard?”

It’s nice to see that he’s concerned about 10 days of my life down the toilet on a 3 year old unresolved defect.

3 Likes
19

@MartinHBramwell Lol this is funny but a good spin. I hope you find a way to get around the issue you’re facing because i have tried on different hosting platforms to replicate your issue but each time my wizard completes without any errors (maybe the wizard recognizes my noobiness and decided to conform :man_shrugging:).

1 Like
20

This difficulty of replicating the issue is why I tried the Bitnami route.

The only similarity to my previous attempts is in the host, an Ubuntu 18.04 LTS VM running in Xen running in Apache Cloudstack.

If someone wanted to give it a try, I’d spin up a copy and give them the root password.