A general comment on versions, design and architecture changes

All great ideas, all it now needs is someone to take charge and deliver :wink:

4 Likes

Really? That is very surprising to me. Do you know what the problem was?

Out of curiosity, I just tested it now, and I was able to get a new site running on an arbitrary release with no difficulty at all. It’s just a matter of telling your local git repo for both apps which commit you want to checkout (using a command like git checkout tags/v10.1.80, or whatever release you’re looking for).

Happy to elaborate more in a new thread if it’d be helpful.

1 Like

Can you say more about what kind of community support you’re looking for on this, and more specifically how the relationship would work?

This has been thrown around multiple times, will add it again: Contribution Guidelines · frappe/erpnext Wiki · GitHub

I’ve seen that page. It describes how to organize a pull request. I was under the impression that you were looking for more sustained organizational support in some form or another, but perhaps I misunderstood.

1 Like

From what I was told…

There was a significant change in “bench” sometime shortly after 10.1.14 and getting something to install prior to the ‘significant change’ creates a system that breaks when you restore your database to it.

Evidently there was something about one of the sources that was depricated and no longer supported, but the change to some replacement source created the incompatibility because the old sources were no longer available.

This is nothing new. It happens during the course of developing a project. The only way to make it work is to fork the sources when you are running a project so they do not disappear on you.

This is the Achilles Heel of the ERPNext project. It depends on its sources to remain stable. If something changes, you would no longer be able to install your version. The install.py and the playbook go to the components sources every time you do an install. So, if one changes, your installs will no longer work. It is a constant maintenance effort to keep the installation working.

This is why there is no easy way to currently create stable installation points for old versions. Nobody is paying attention to the changes in the sources that might affect the older versions. The longer the time after a version is closed, the less likely you will be to get a successful install.

I believe this is why the idea of using docker images was put forth as a method of keeping older versions available in some stable format. Once the image is created, there is no longer a need to go back to the original sources.

Anyway… that is the understanding of the situation that I was given for the low, low price of a bunch of paid developer hours.

It is also why I am so passionate about the LTS version thing. Been there, spent the money, and no result.

BKM

4 Likes

^ Exactly this.

The technical term is Nondeterministic. For the same inputs (ERPNext Version/Build), we are getting different results. For a production/live environment, you never want this to happen.

To avoid this, you must ensure that your environment conditions are 100% repeatable. Never install/clone software using “latest” or “stable”. Use precise version numbers. For example:

  • Assume that @BKM and myself have exact copies of ERPNext v10 + Database.
  • He installs his today, February 18th. Bench fetches the latest, stable version of MariaDB, 10.3.11 Stable.
  • I install mine, February 24th. Bench fetches a newer, stable version of MariaDB, 10.3.14 Stable
  • It is possible that our 2 ERPNext could behave differently! One of us could experience bugs.

Why? Yes, we’re both using “stable” MariaDB. But we have slightly different versions. In a production environment, you never want to download a random, latest stable version of software.

Docker containers are 1 tool that can help solve this. Because each container stands alone, independent of the host. My MariaDB Docker container 10.3.12 will be the same today, tomorrow, or 5 years from now. (just make sure your Dockerfile isn’t written to download nondeterministic versions of its own dependencies ! )

So. What needs to change from us? When we release ERPNext Version #?. It should be released for a specific Python, MariaDB, Redis, Nginx, Node.js, etc.

Otherwise, you will never get the same results. Which is why there are hundreds/thousands of posts on these forums, with people having Installation problems. The installation of ERPNext is currently NonDeterministic.

6 Likes

Just to be clear. I am not saying that all ERPNext users must deploy identical environments. I love the flexibility of ERPNext, and that we can run it in dozens or hundreds of different environments.

But regarding LTS, then yes. You must target specific software/environment builds and versions. Otherwise it won’t really be stable.

1 Like

This may also explain why there are some people on the forum that report problems that get critical ‘snubs’ from other ‘experts’ because the expert cannot replicate the issue simply because their sources may be slightly different from the newbie’s installation even though the erpnext and frappe version numbers match.

It is the one factor that has annoyed me to no end about how some error reports are treated here.

/rant off :grin:

BKM

1 Like

so, would it be ideal then that the installation descriptions (incl. the install.py) would refer to exact version rather then ‘stable’? This (once the mindset is put on this matter to be of importance, which may be simple or difficult) should technically be doable easily.

And if that would solve the problem you and BKM are describing here, should be made a goal achieving.

Exactly. They should refer to specific versions. Which versions to use? The same versions used during LTS stability and regression testing…

Of course, this now leads to a new set of questions:

  • How is LTS approval testing being performed? What are the steps?
  • By who?

The first question deserves a formal document. To describe the processes & procedures that allow us to formally label a version as LTS.

Time for us to open our favorite word processor, and get typing.

As for the second question. It would be easy to just say “the maintainers are responsible”. But I don’t think that’s fair. Instead, there should be team of volunteers. The ERPNext LTS Group, who are responsible for performing all the testing, and “signing off” on an LTS version. They would choose the environment/versions used during the process.

3 Likes

Let’s use this for developing such. I’d say once we’re done it’ll be put into the core or maybe somewhere under the Foundations wings

2 Likes

This just caught my attention on Twitter. Excellent timing. This article is extremely relevant to our discussion, so thought I’d share with you all.

https://pasztor.at/blog/reproducability

1 Like

being that I’ve been doing a lot of work on the official docker image, I think making package installs stable will be the next thing I work on after getting docker stack to run successfully.

UPDATE: changed my mind, it’s the next thing on the list!

1 Like

you mean GitHub - frappe/frappe_docker: Docker images for production and development setups of the Frappe framework and ERPNext?

Um, yes.

I make a PR there every so often with some new features or fixes.

From frappe/erpnext/wiki/Supported-Versions page ,

As a general principle, from Version 10 onwards, we plan to support each major version for at least 3 years from date of release. The reason for this is that companies that have significant customization on top of ERPNext should be not be forced to move to new releases immediately.

For contributors, this means that separate pull requests for bug fixes and security fixes must be sent for each of the supported versions.

Here is the support plan for major versions that are being supported now:

Version Branch EOL
10 v10.x.x. End of 2020
11 master End of 2021
12 develop Not Yet Released

Note: Fixes to Version 10 will not automatically be merged in Version 11, and so on.

Thanks @salman

I was not aware this information was listed anywhere yet.

I was hoping for something a little longer support time. However, at this point, I am happy to have some level of version stability.

Most people that want to have a stable version will be installing the last v10 knowing that it will not change while they are using it. This make it much easier to avoid v11 and all of the constant changes that are happening there. Even though it has been called ‘stable’ the reality is that bug fixes for one item sometimes tend to inject new behavour in another part of the module.

For example… with all the current reported buginess in the new permissions functions, if you come up with a work-around for your particular permission problem, any fixes to how permissions work (to fix the root cause of the bug) could very well render your work-around broken on the next bench update.

There will be many cases like this as all the fixes get implemented into v11 over then next year or so. Trying to use v11 as a stable platform is not yet a reality when you take these kinds of fixes into account. Certainly there will be no ‘new’ features added to v11, but the bug fixes alone will be enough to keep a lot of operators away from it until it much further along.

My only 2 ventures into using v11 as a possible launching platform for 2 new clients, both blew up in my face due to the permissions bugs keeping me from properly setting up lower level users to perform important functions. So I had to make all users “System Managers” in order to get most stuff working efficiently for the 2 week demo trials, only to have the constant nag screens asking any “System Manager” and Administrator to perform updates. By the time I found someone that could give me a way to stop the nag screens I had already lost enough credibility with the clients that they ask me to terminate the trials early and not bother with quotations for business.

So, lesson learned! I am only going to use the stable final releases for my production work. Now that there are published EOL dates, I can better plan new installations almost like any other production software.

I like this. I am happy about this.

BKM

1 Like

even though we are happy if you are happy this should not be the end of it yet.
Our mission is not yet accomplished

Point taken. Thank you. I back your ideals for making a more formalized (and documented) approach to the process.

BKM