Best strategy to avoid failed patches from new releases?

[MartinHBramwell]

So I have customizations to Sales Invoice.

I am working with Beta 13 and trying to stay reasonably up-to-date with the latest versions.

I try to put my patches as close as possible to the beginning of the file, but still they get clobbered by later code modifications.

Where can I read up on how to avoid this?

What I have figured out so far:

[Solved] What's the right way to version control my customizations?

Right, well I finally figured this out. There’s a vitally important yet scantly documented command : bench --site mySite export-fixtures I’ll explain and demonstrate how it actually works. I’ll use a virtual machine with snapshotting so I can do some work, then revert back to a snapshot, and continue. So I’ll : Create a site Take a snapshot Create an app, add it to version control. Install app in site. Customize the site Export the customizations Add the exported customizations to app usi…

[brian_pond]

Hi Martin,

I don’t update frequently, but I understand your situation, and the challenges of frequent merging. Probably some of the Frappe maintainers could offer Tips & Tricks in this area, since they’re doing this at a large scale every week.

One thing I try to do is wrap my customizations with comments.

# ---- BPO: Start some Modification
...
print(f"Good morning, '{foo}'")
...
# ---- BPO: End Modification

This helps me while doing visual merges, so I don’t accidentally break-apart my code blocks.

If I’m confident, I will run a run a git pull. And hope that mostly I’ll see fast-forwards, with minimal manual merges.

Sometimes I’ll do a git difftool ahead of time against the upstream branch. To preview what’s going to happen. And perhaps manually merge code changes, before I run git pull.

Definitely appears to be both art and science. I too would love to learn more from experts on how this is being smoothly accomplished.

I have a ton of customizations in my ERPNext environments. But I deliberately avoid updating them from upstream, except during scheduled maintenance windows.

[MartinHBramwell]

Hi Brian,

I am always grateful when you step in with your depth of experience.

I’m starting my first installation, so it makes sense to me try to be compatible with V13 from the get go.

The main point of my question was whether I had missed documentation of some critical technique, but it seems you have the same issues and handle them the same way.

I wish that the ERPNext team would make allowances for end-user developer customizations. I mean how hard would it be if every file had an import command for customizations. Eg;

erpdev@loso:~/frappe-bench-DELS$ ll apps/erpnext/erpnext/accounts/doctype/sales_invoice
total 484
drwxr-xr-x   5 erpdev erpdev  4096 Mar  4 18:31 ./
drwxr-xr-x 149 erpdev erpdev 12288 Mar  2 00:00 ../
-rw-r--r--   1 erpdev erpdev    40 Mar  1 23:58 __init__.py
drwxr-xr-x   2 erpdev erpdev  4096 Mar  4 18:32 __pycache__/
   :      :      :      :      
-rw-r--r--   1 erpdev erpdev 33968 Mar  4 16:47 sales_invoice.js
-rw-r--r--   1 erpdev erpdev   222 Mar  4 16:46 sales_invoice_dc.js
   :      :      :      :      
-rw-r--r--   1 erpdev erpdev 75707 Mar  4 18:31 sales_invoice.py
-rw-r--r--   1 erpdev erpdev   321 Mar  2 00:28 sales_invoice_dc.py
   :      :      :      :      

So, sales_invoice_dc.js and sales_invoice_dc.py would be null files, always imported by sales_invoice.js and sales_invoice.py respectively.

Like:

# -------------------------- User protected zone -------------------->>>
from . import sales_invoice_dc  # developer customization

# -------------------------- User protected zone --------------------<<<

[brian_pond]

That is a nice idea, Martin. I would definitely use something like that. Today, I’m either writing code to hooks.py (which calls various other functions). Or I just edit the original code, and deal with comparisons. It depends on what I need done.

Fun Fact: Did you know that hooks.py is being processed every time a worker job is executed? To see what I mean, add this line anywhere in your hooks.py (make sure it’s outside of a function)

print("This is hooks.py.  Something just imported or called me.")

Then start ERPNext with bench start. Keep an eye on your console logs. Every 5 minutes, the worker_short job is re-running all the code inside hooks.py.

Feels like a bug. Though I’m afraid to get rid of it. Because I don’t see where hooks.py is otherwise being called and loaded on startup.

One of many little mysteries. It’s inefficient (loading the same functions over and over into the namespace). But not dangerous, so long as my hooks.py code stays inside of Classes and Functions, and doesn’t “do anything” by itself.

[revant_one]

This applies for vanilla ERPNext only, and I DO NOT do any code changes unless it is a PR.
If anything fails in vanilla it will be fixed (eventually). No custom app to complicate things.

Every day there is a frappe_docker cron job that does following:

• Installs latest version of ERPNext (currently v12)
• Creates a site, (also tests basic backups, restores, etc)
• Migrate v12 site to develop branch.
• Once in a while we see patch failures.
• I report on issues and tag patch writers as soon as I see the patch failing.

When v13 will get released I’ll update the cron job.

Anyone can keep a watch and report, it will benefit everyone in the future during production migration.

Yesterday’s unsuccessful run - Travis CI - Test and Deploy with Confidence
Reported issue on 2021-02-16 - Error: Executing frappe.patches.v13_0.rename_list_view_setting_to_list_view_settings · Issue #12416 · frappe/frappe · GitHub

[MartinHBramwell]

Hi, @revant_one

It took me a minute to understand why you posted this, but I think I get it.

That CI/CD job exists to flag daily any regression in compatibility between code base and database of V12 vs V13. Correct?

I’m guessing you intended that in reply to my remark, “I’m starting my first installation, so it makes sense to me try to be compatible with V13 from the get go.” Should I understand it as a suggestion to stick with V12 until V13 is released, then migrate once?

[revant_one]

@MartinHBramwell

That CI/CD job exists to flag daily any regression in compatibility between code base and database of V12 vs V13. Correct?

Yes.

Also you can refer the job and set your own job that does basic tests for your apps.

[MartinHBramwell]

Sorry, what does “refer the job” mean? Clone it?

[revant_one]

Check this stage:

frappe/frappe_docker/blob/5121e2710653c8df513e953a21e8d1ee8d3c2bde/.travis.yml#L153-L166

and it runs this shell script. frappe_docker/docker-test.sh at e2e103e181b8f0fa0a2ccfce7fd34effdd7a8153 · frappe/frappe_docker · GitHub

The only challenge will be, either use docker or translate it into script of bench commands.

Objective is have some automated green ticks before doing production upgrade.

[brian_pond]

@revant_one

…and I DO NOT do any code changes unless it is a PR.

This is a very important sentence, Martin.

revant_one’s CI/CD process appears quite good. But he’s not maintaining a separate fork of Frappe/ERPNext. Any changes he makes to code, he also creates a Pull Request to the official source code. Therefore, when his CI/CD does a synchronization, almost all differences will be handled by git ‘fast forward’. Which requires no human intervention.

This would not work for my environments. I usually have 2 additional Apps. But more importantly, my Frappe and ERPNext code are forks of the official versions. I have made opinionated modifications and customizations, which are not in the official versions.

However, I am not performing a daily synchronization with official code. Maybe every 1-2 months? So just handle the upgrade manually, using git commands from the shell, and GUI tools like meld.

[max_morais_dmm]

I always go on top of custom apps first.

Make changes on ERPNext code, is the thing I always try to avoid, because maintain a fork ia harder.

When I found a bug in ERPNext this is the case where ill make a PR, or If the feature is too generic to benefit others.

Also, custom apps give you an nice centralization of changes, preventing that one instance receive changes that arent supposed to.

On top of custom apps, you will have the same control you are supposed to have, with direct changes in ERPNext.