Sporking Bench: New tools for installing ERPNext

Install / Update
1

ERPNext Community,
I will skip all introduction, and get straight to the point:

The current Installation process is awful, embarrassing, and a barrier to ERPNext growth and success.

My words should not surprise you. Many of you personally experienced the pain of ERPNext Installation. Last Friday, I quickly examined the 7 prior days of forum threads. I found 28 separate threads where people struggled with Installation issues. Those were only the easily-found posts. How many if I’d really spent time digging around?

I first started learning ERPNext at the beginning of 2017. Installation was bad then. It’s no better today.

TL;DR : I’m sporking Bench. And leading an open source project to write new, companion tools for ERPNext Installation.

(the famous spork is a hybrid of both fork and spoon. Since I’m not technically ‘forking’ bench, I use the word ‘spork’)

Contributors & Visitors welcome, and appreciated! It will take a Team Effort to accomplish this.

  • GitLab repository for project Weavlo here, with introduction README here.
  • Telegram channel here.
  • There will be a dedicated project Wiki. I’ll share the URL once it’s online. It’s been a long weekend, friends.
  • My Telegram handle is @BrianPond. You can ping me with questions, send funny cat photos, etc.
  • List of other Maintainers is TBD (once they volunteer).

Project Goals:

  • Make ERPNext more accessible for everyone.
  • Create an Installation Process that is easy and bulletproof.
  • Help the community generate Long Term Support environments.

~Brian

Frequently Anticipated Questions:

Q: Wow Brian…is this really that big of a deal?
A : Yes! We are losing valuable community members and comrades. Not because ERPNext isn’t excellent. But some people are unable even try using ERPNext. Because they’re stuck debugging the {expletive-deleted} installation scripts.

Q: Fair enough. So why not simply submit PR’s against Bench?
A: Good question! Two reasons for that:

  1. I’ve maintained my own private fork of Bench for half a year. It helped a little. But I don’t think any quantity of PR’s can save the tool, and make installation bulletproof. We need major, significant change. Bench tries to be One Tool for Doing Anything. Unfortunately it ends up Mediocre At Everything.

    No, we need to have Small Tools. Where each tool Does One Thing, And Does it Well.

  2. Revamping Bench code means (inevitably & unfortunately) introducing errors downstream. Breaking the Bench repository will likely harm ERPNext and the Community. We definitely don’t want that! So it’s safer (and easier) to build new tools independently and in isolation. Nothing breaks this way.

Q: Does that mean no modifications to Frappe Framework or ERPNext code?
A: We would love to submit PRs against the Frappe framework code, and make it more-compatible with these new Weavlo companion tools. We’ll see what happens! :slightly_smiling_face: But in the short-term? We’re going to start with the existing v12 code base.

Q: So…I will just download this new installer tool. And it will install ERPNext, without any fuss, and launch it?
A: Yes, that is The Plan.

A: Isn’t that what the Easy Install script does?
Q: Supposedly. Yet experience has demonstrated that a more industrial-strength installation tool is required.

Q: What about Bench?
A: Not sure yet; that is a topic for discussion. Personally, I’ve never been a fan. But we’ll discuss as a team, and see where it goes. At the minimum, it seems like a “mini-Bench” should be included, to support those people who enjoy using certain commands. Our goal is to improve ERPNext usage; not subtract functionality. Still, these Weavlo tools are designed to replace a lot of Bench. We won’t be improving on it.

Q: When will these new ERPNext Installation tools be available?! I could really use them right now!
A: That’s difficult to predict at this moment. It will depends a lot on contributions. Stay tuned!

Q: Brian, why is ERPNext Installation so hard in the first place?
A: Uggh. Sometimes cool things are complex . ERPNext glues lots of ideas together. Here’s a quick rundown:

  1. ERPNext can be installed on many OS distributions. And versions of each. That alone introduces much chaos.
  2. It has lots of pre-requisite software, which must be installed and configured correctly.
  3. Bench, Frappe, and ERPNext code on GitHub change over time. Even the stable versions are patched. And sometimes humans can make honest mistakes.
  4. ERPNext depends on Python & Node packages downloaded from the internet. Sometimes those upstream packages introduce errors in ERPNext.
  5. Python2 vs. Python3 didn’t help matters.
  6. Because Python and JavaScript are interpreted programming languages.
  7. Because computers.

Q: Why use GitLab, instead of GitHub where all the other code is at?
A: This post is already too long; read on that here: weavlo / weavlo_intro · GitLab

19 Likes
2

+1, I did some work on installation tutorial for ubuntu wsl, [Tutorial] How To install ERPNext V12 on windows 10 home / Pro WSL ubuntu 18.04LTS - #3 by nmami

1 Like
3

I would like to collaborate and contribute, here in China I can not use Telegram, but only skype possible.

4

FYI, GitHub now has built-in CI/CD (GitHub Actions), so having an integrated CI/CD is no longer an issue.

The main reason why I’d suggest GitHub (I like Gitlab more actually), is simply due to project visibility.

5

Awesome @brian_pond - I’ve wanted to something similar but on the testing front - Putting my hand up to help out also - I do have a fair bit of experience with ansible so can help out there if needed.

Thanks

6

One way to simplify scope in the new tools is potentially the following

  1. Choose to only support CentOS 8 and Ubuntu 18.04 rather than a variety of OS’s
  2. Choose to only support installation of the v10.x, v 11.x and current v12 branches

That knocks things down to 6 possible configurations, and should cover almost all the community’s use-cases, according to what i’ve seen on the forum.

I’m not sure about the importance of maintaining py2 in the installation process moving forward, but if that’s important, then that adds complexity.

2 Likes
7

I don’t know much about Ansible; your knowledge would be very welcome! :slightly_smiling_face:

8

I agree @felix: we should limit support to only certain combinations. Definitely, I don’t believe we should support ERPNext versions older than v10.

I’m using Ubuntu 18.04 for daily use. Debian 10 for Production. I don’t know anything about CentOS. But I see people mentioning it on the forums, so I believe it’s important.

To start, we can take an “80/20” approach. Solve 80% of the problems for 80% of the installations. Then begin solving other 20%. Do we know anyone using Fedora? I have not seen it since 2005, or so.

I don’t want to support Python2. But we may need to. That is one of many “Questions To Be Answered” in the project.

I’ve given myself a deadline of Friday to release the Wiki Website.

Thank you all very much for the support and feedback! I look forward to working with you. :sunglasses:

9

Remembering how I, being a total Linux newbie, had to manually setup around 1/4 of the stuff from the install script and edit both, the yaml playbook and Python scripts themselves, I wholeheartedly support this and would definetely like to help, but I don’t know how exactly yet.

10

Igrekus, as I continue getting organized, I’ll create a list of Tasks and Ideas. Definitely you could help with Testing. Create an empty environment, download the Weavlo Installer Tool, and run it. If it doesn’t work or there are bugs, report them. That would be incredibly helpful. Of course, first we need to build something for everyone to test. :slight_smile:

Also, any ideas are welcome. Do you know about existing tools/packages that can validate Software PreRequisites? Manage software configurations using JSON/TOML/YAML? Create Install Wizards or manage binary executables?

There’s no right/wrong way to complete this project, as long as it meets the Goals. Even a small suggestion may lead to big improvements later.

1 Like
11

Hi @brian_pond

I would love to be a tester, been trying to install erpnext for the last 2 weeks many times and still no luck, so would be all too happy to support something that will eventually work well.

Please count me in…

Kind regards
Hemant

1 Like
13

That’s the first thing they tell me when I tell that I used ERPNext at my office.

1 Like
14

I don’t think that there is a big effort required.
Just go step by step the whole process, migrate to Python3, eliminate old repos and introduce new versions of products.
Get rid of the locale check which should have no relation whatsoever.
Get rid of the memory check and watch out for the redis server installation.

I guess that is mostly this. I hacked the code in every step and managed to install under different circumstances.

15

I’ll definitely participate.

I have some experience building Windows binary distributions, but, alas, not Linux ones. Also, me and Linux have a difficult relationship =)

Unfortunately, nothing more than fiddling with Bench’s YAML playbook and Frappe’s Webpack JSON. But I’m willing to learn.

16

Why not consider developing /supporting a SNAP package for ERPNext? Would solve installation issues, allow background updates, keep the unified vision, and perhaps free up resources from the current VM, script, docker installs. There have been some posts requesting this…

1 Like
17

All the best @brian_pond for starting this. Would love to see some code when its ready :slight_smile: . I hope it does not become another bench.

An analysis of these issues might help to identify this. The easy install script broke last week as the Ansible version was not pinned. Maybe a general pinning of all versions would be a good start to fix that problem. But then we have to add the overhead of regularly updating all dependencies!

Either ways, installation is a hard problem. Maybe a docker-first approach might help!

18

Good luck. And happy to help. I have just started looking into the bench code base to add:

  • command line flag for CORS configurations (via site_config.json)
  • support for faster, partial updates (essentially, bench update app_name)

I will volunteer for specifics once the Wiki is up. I can help with Python/Bash scripts, Python code and testing.

I do have the same comment as above on Docker - last I checked the image was broken. However, I have seen a few commits since then. If that is fixed, then I would recommend something like CapRover + Docker as an ideal combination to get a UI for managing + reverse proxy (with Lets Encrypt).

19

Why are the following lines ever needed during easy install?

  • name: Check current locale
    shell: localectl
    register: locale_test
    when: ansible_distribution == ‘Centos’ or ansible_distribution == ‘Ubuntu’

  • name: Set Locale
    command: “localectl set-locale LANG={{ locale_lang }}”
    when: (ansible_distribution == ‘Centos’ or ansible_distribution == ‘Ubuntu’) and locale_test.stdout.find(‘LANG=locale_lang’) == -1

  • name: Set keymap
    command: “localectl set-keymap {{ locale_keymap }}”
    when: “(ansible_distribution == ‘Centos’ or ansible_distribution == ‘Ubuntu’) and locale_test.stdout.find(‘Keymap:locale_keymap’) == -1”

  • name: Set Locale as en_US
    lineinfile: dest=/etc/environment backup=yes line=“{{ item }}”
    with_items:

    • “LC_ALL=en_US.UTF-8”
    • “LC_CTYPE=en_US.UTF-8”
    • “LANG=en_US.UTF-8”
20

Not sure, Tufan. I have not examined Easy Install since 2017. If they are truly needed (perhaps not), the answer is probably somewhere in Ansible scripts? I have briefly examined the Ansible scripts. But not in detail. I prefer to use my own instructions for installing prerequisites and dependencies.

21

Hi @szufisher : There will be an editable Wiki, plus the project on GitLab. Telegram is convenient for instant chat. But not mandatory, and nothing official will be written there. Most of the collaboration and contribution work will happen outside of Telegram.

I chose Telegram only because it’s already-used for ERPNext Dev channel, ERPNext Help, and other channels.