Wow! this may well be the best thing for erpnext installation! I pray this goes well. @brian_pond this is awesome. I’m a witness after the fact that installation and upgrades are tricky. this is a dream, we pray it becomes a reality. More oil to your gears @brian_pond
1 Like
pip-tools can really help here with pinning all versions, then you can even do things like pip-sync, pip-compile, and all dependancies auto upgraded to test etc.
1 Like
@brian_pond I’d like to play devil’s advocate here and suggest that instead of creating a new installer, erpnext would be better served if we packaged it for Debian and Fedora. IOW, make it as simple as apt install erpnext.
The process of packaging it would shake out all the hardcoded dependency issues, IMO[1]. The packaging reviewers are really good and will help with it. Getting it into Debian means all downstream distros would get it (including Ubuntu) and getting it into Fedora would mean it would enter CentOS and RHEL.
Longer term, this would help ERPNext developers to think consciously about breaking ABI and staging features so that they can keep rolling out updates to distros (think security and bugfixes) while bigger ABI-breaking features will be prepped for the next LTS. The latest development nightly could also be made available through Ubuntu PPAs for testers.
The configuration of a site, OTOH, could then be handled separately - either through ansible, juju charms or the like.
Just my 0.02. But this problem definitely needs addressing.
[1] An example is the dependency on mysql 10.2. I don’t think erpnext uses any cutting-edge features of mysql that should require that particular version. I’ve seen folks go through great lengths to install that version from dubious repositories by turning off basic package signing checks. Who knows what happens on an upgrade.
4 Likes
@rmehta, you might want to check what @revant_one is building on Kubernetes: Decoupled Frappe/ERPNext docker images. @revant_one thought up and built the current version, while discussing the problems with @brian_pond on separate Telegram group …
I like this idea.
1 Like
thanks Brian, there’s no arguing with your response - good luck and thanks in advance!.. the thing that amuses me, on brief interludes from the angry screams of a couple of years on and off of trying to install this great looking product, i sthat someone out there obviously has installed it… i would love to see a console log from this wizard at work!!
Many of us have successfully installed, manually. The Hitchhiker’s Guide describes how this can be accomplished. I maintain my own, personal set of instructions. I’m sure many other people do the same.
I started installing before the Hitchhiker’s Guide was written. It took me weeks before I could repeat my installation without errors. Yet it was months before I mastered it, and truly understood what exactly was happening. Certainly some people probably learned faster than I did. Either because they already had a background in Python web frameworks, or they received help from a friend. Others probably learned slower than I did.
Because when errors happen (and they do). Troubleshooting can be very hard. To successfully troubleshoot, you must understand exactly what’s happening. Is this a SQL error? A Redis error? Is it an error with a PyPi package, or a Node package? How do I change the version of a PyPi or Node package? What’s the SQL username and password that ERPNext is using? Etc.
We should not ask every ERPNext user, company, or enthusiast to build this deep set of knowledge and skills. Why should they be troubleshooting the Installer? It’s a waste of their time. Installation and configure should just work.
People could be spending time where it matters:
- Learning the Framework.
- Learning the Modules.
- Learning how to edit Print Formats, Reports.
- Learn to build new Apps and DocType, integrate with other software.
Instead of figuring out how to install an enterprise software platform.
7 Likes
Is there a benefit to having separate individual containers for the different dependencies? Especially for end users such as us (as compared to ERPNext hosting providers)? Docker also requires entire infrastructure to be installed which consumes system resources. I was recommending a SNAP package or apt install as it allows you to install to a standalone, VM, linux container, etc…while preserving the current structure.
i would kill for the chance to consider an upgrade!
one thing that i was interested in peoples thoughts on was the pre-built erpnext appliance that can be downloaded and installed on virtual-box… i ended up downloading this and “porting” it for use on qemu virtual host (proxmox)… this certainly worked in as much as it gave me a running system… however this made me nervous because i wanted something configured for production use and i dont understand the layers of this infrastructure enough to appreciate how hackable/configurable the pre-built image is… if this is a route that has milage then i will dig out and share my notes on this!
LXD is developing the ability to run .vmdk images in linux containers - per their release notes. This will hopefully negate the need for any overhead related to Virtual Box, etc. and associated learning of all related commands. Thats why I was asking about benefits of Docker - wondering if all that is needed for end users…
Virtualbox is OK for testing and getting a feel for erpnext. I doubt anybody would want to use it in production on account of the overhead. Your notes might still be useful for newcomers.
1 Like
I’ve previously thought about this. It would solve some problems. Other problems would remain unsolved.
- Anyone without those Linux distributions would be excluded from the community. MacOS, Arch, etc.
- We should not automatically bundle certain pre-requisite software.
- MariaDB: No need to install, if you already have your own instance.
- Redis: No need to install, if you already have your own instance.
Even with rpm’s and dpkg’s, we still need an Installer Tool that asks Questions. Instead of making Assumptions.
We should consider the various types of ERPNext users; For people just taking a “test drive”? They’d be fine with a Virtual-Box or containers. For small businesses? Sure, maybe RPMs and DPKGs would work (assuming we still ask questions during installation). For larger organizations? They need more precision control.
The biggest challenge with ERPNext is that our target audience is GIGANTIC. We have users with nearly zero Linux skills, who don’t know what a container is, who’ve never seen Python, Bash, or JavaScript. But also, we have web framework and programming gurus. And everyone in-between. On every OS+version we can imagine. On every Cloud platform we can imagine.
This would be awesome. But currently there is no LTS program for ERPNext. We’ve talked about it. But there’s no Plan. I haven’t seen any work on that since @vrms created a repo for it. ERPNext developers = anyone who submits a PR. Some may give thought to downstream consequences. Others probably not.
Who will maintain the Ubuntu and RedHat distros, update the nightly builds, think about dependencies, run the checks? If it’s a ton of ongoing work, we should probably pay them for this effort.
Don’t get me wrong, @idlethread. I am not dismissing your suggestion. It’s a good one. And this project may adopt some of that tech. But it wouldn’t solve everything. And if not done correctly? Could cause additional lost time/effort.
the notes show you how to NOT USE virtual box!
A few clarifications are in order:
- I’m suggesting separation of installation and configuration.
- We will still need to achieve most of the goals of Weavlo, wrt to configuration as you’ve described
- Having worked at Canonical before, I feel that all distros stick to fairly similar version numbers for their core packages (modulo minor versions and configuration choices). So while maintaining cross-distro packaging is some work, it shouldn’t be terrible.
Applying the 80/20 rule, supporting debian/fedora/ubuntu/centos/rhel probably reaches your 80% users, at least in the beginning. The others can use Debian VMs/docker inside Arch/MacOS/Windows servers until the community adds support for brew, windows, Arch, etc.
Every distribution already handles this well. e.g. In case of multiple applications needing mysql, the first one installs it, the others simply create the tables they need. e.g. using dbconfig
This is the separation of installation and configuration I was talking about. Enterprises would need something like a juju charm or ansible/chef/puppet recipes to provision multi-machine setups. But that still requires installation dependencies to be solved for the OS packages at some level.
I’m sure someone could be found (and even sponsored) to maintain the packages though a vast majority of Debian packages are maintained by kickass volunteers.
Packaging, if done correctly, will ensure we receive security updates for most of the code from the distributions. It will also force us to rethink how data is stored in an erpnext installation to allow for easier upgrades, backups, etc. It might even help erpnext develop a 2 year cadence with respect to LTS versions of distros. Then it truly becomes an enterprise product. 
Thanks for indulging me. I’m currently just a user who’s starting to use erpnext and am very worried about how this stack will move forward once I put it in production. I might look into the packaging issue as I find time.
1 Like
Good point!
has this project gone anywhere? I’ve just had a quick look and it appears not.
Work is very slow. I have not given up. I continue to work on small components, time permitting.
- I have a forked Bench that installs ERPNext 95% of the time.
- I have a set of manual installation instructions that succeed 99% of the time.
Frappe also recently pinned package requirements (Python, Node) for the first time. Last I checked, that work was in hotfix. Maybe it was eventually promoted to version-12.
All of this is helpful. But they’re just bug-fixes. Fundamentally, Bench is a flawed tool with an identity crisis.
I wish I could name an Expected Completion Date. But this is a very large project, with a rather massive learning curve. And client or paid work always takes priority. I have to keep paying my bills. 
In the meantime, if you’re struggling with specific installation issues, let me know.
2 Likes
nice one. Not a critism, I was genuinely interested to know if work was continuing, as I only discovered this thread yesterday.
I have tried every docker version of erpnext installation that I can find. None of them, not one, work, nor are the instructions for any of these very clear. It’s honestly been the worst experience I’ve encountered in the open source software community so far, so I applaud you for trying to make that issue better.
After messing around with others versions, and getting a sense for what others are doing, I’ve decided to start from scratch and build my own scripts, rather than attempt to correct where others are failing. Either path is a source of great frustration but at least walking through the process from the start might help me get clearer on all that is involved.
I haven’t really worked with bench. I mean I’ve had it installed in the past, and had working instances of erpnext when i had my last business… I can’t understand why i’m experiencing so many issues this time around. I’ve even tried to pay the subscription to use the cloud instance but there is some tom-fuckery going on at their end with my email address so I can’t even complete the sign up process.
Its a real shame, I really preferred erpnext to odoo… but it seems to not quite be there on so many levels.