Need advice on how to do merge and keep your changes

Frappe Framework
1

Hi guys

so i need advice on how to merge changes like say if an erpnext is now on 13.0.1 while im still at 13.0.0 i want to be able to update my fork to 13.0.1 while retaining my changes to the erpnext code.

example

  1. Fork Erpnext 13.0.0
  2. Update my forked erpnext by adding lines deleting lines and modifying a line of code etc.
  3. Main Erpnext gets updated to 13.0.1
  4. I want to be able to update my fork to 13.0.1 and retain any changes that i made in step 2

Thanks

1 Like
2

Hereā€™s how I handle this:

  • I began with regular Frappe and ERPNext.
  • I created new git branches for the Frappe and ERPNext apps.
  • My custom branches are always the active ones.

Whenever I need to, I make my modifications. I try to add inline documentation, to easily identify them. For example:

#  Brian Modification Begins Here:

Consistency matters. So I can do a global search, and identify what I previously edited. If I forget, itā€™s okay. I can always do a git diff. Adding comments is just an optional step I find helpful.

Fast forward into the future: I want to update to a newer Frappe/ERPNext version.

  1. I activate the official git branches.
  2. I update them using git fetch and git pull. This should always succeed. Because Iā€™ve never edited the official branches.
  3. I change back to my custom branches.
  4. For each App, I merge the official branches into my custom branches:
    git merge upstream/master

99% of the upstream changes are handled by fast forwards. The other 1% must be manually merged.

I open the files that require manual merging. And edit the code, so that my changes are still valid.

If I ever get confused or lost? I use git difftool, and visually compare My Code versus Official Code. Iā€™m using Meld, but there are many other GUI tools for comparing with git.

  1. When Iā€™m done merging the changes to my custom branches, I git commit and git push.

And thatā€™s it.

(Note: Some of what I described can be done with Bench commands. But I prefer to operate without it most of the time.)

2 Likes
3

@brian_pond

so can i ask if i got this right

  1. you forked entire frappe/erpnext project. lets call the official project A then the forked project project B
  2. git branch project B
  3. go into your custom branch and start modifying the code then add comments to mark where you made changes(git diff also has the same effect)
  4. if a new update is released on project A then I go to forked project B then use git fetch and git pull
  5. go back to the branch you made earlier with your changes then for each app use the command git merge upstream/master
  6. check and see if my update were reflected after doing the merge if not then change accordingly
  7. git commit then git push to push my branch to the project B

Did i understand it correctly?

I have one more question

lets say project A is at 13.0.1 but i want to get 13.0.0 what would be the best way to fork this specific release? like what command should i use.
The reason why i ask this is because i want to practice git merge and by doing this i wont need to wait for a long time for an update to come

4

Hi Sir,

May I know what kind of changes youā€™ve made in original Frappe & ERPNext ?
Did you also change standard Frappe/ERPNext doctype ?

Regards,
Subhajit

5

Thatā€™s pretty close, yes. When you perform a git merge, it automatically warns you about files that cannot be automatically resolved. So, you immediately know which files must be manually cleaned up. Also, git adds special tags to your files, to point you at the places that need work:
https://git-scm.com/docs/git-merge#_how_conflicts_are_presented

ERPNext is complex, because it has 2-3+ Git repositories to think about:

  • Frappe app
  • ERPNext app
  • The Bench application (most people donā€™t modify this, but you certainly could)
  • Any custom Apps you create.

If you want to practice git workflows, Iā€™d suggest starting with a smaller software project. Find one online, or create your own sandbox. Once youā€™re comfortable with all the different commands, and common tasks, ERPNext will make a lot more sense.

Answer to your 2nd question: What if Frappe/ERPNext are on version 13.0.1, but you want to go backwards to 13.0.0?

You donā€™t need a fork to accomplish that. You just need to checkout an older commit of Frappe and ERPNextā€¦

ā€¦howeverā€¦

ā€¦going backwards is not that simple.

  1. Because of the MariaDB database. When you use git to change branches or commits, youā€™re just changing files on your machine. But the MariaDB SQL database is unaffected. To synchronize your MariaDB schema (to make it match the JSON schema files on disk) you use the Bench command: bench migrate.

    However, this may not work correctly. ERPNext has patch files that let you move forward in versions. But not backward. The are no tools that will (safely) move you backwards in versions. You will have to manually cleanup your database, write your own migration scripts, etc. Itā€™s not easy, and you can absolutely place ERPNext in a broken state.

  2. Because specific versions of ERPNext also target specific Python and JavaScript dependencies. When you run bench update to move forward, this is taken care of.

    But when you begin ā€œdoing your own thingā€ (forking ERPNext, moving backwards in time)? Then you must handle dependencies yourself. For example, ERPNext 13.0.1 might require versions of Python and JavaScript packages that will not work correctly on ERPNext 13.0.0. So if you go backwards, you must be prepared to rollback to earlier versions of dependencies. Which is a whole other topic.

Anything is possible. But not easy. There are many great articles about Git online.
[Git Tricks for Maintaining a Long-Lived Fork]

But this can become extremely tech-heavy work.

1 Like
6

@SubhajitDey All kinds. Way too many to list here online. Hereā€™s just a few examples:

  • Allow Email Domains/Accounts to save records, even when validation fails.
  • Some extra validation to the REST APIs, to prevent strange situations.
  • Replacing os.path with pathlib.Path, and doing more path validation and error handling.
  • Added support for SQL CTE queries.
  • A safer way of updating DocField values.
  • Adding user message levels (DEBUG, INFO, WARNING, etc.) to frappe.msgprint()
  • A huge fork and redesign of Bench; Iā€™ve probably touched 25% of the original code.
  • Lots more.

Yes, Iā€™ve edited the standard DocTypes too. Adding new DocFields, editing functions, etc.

Usually, I prefer not to perform Customization, guerilla patches, or ā€˜hooksā€™. The consequence is I frequently fork and edit DocTypes. This is probably ā€œweirdā€ for an ERPNext developer; most people will not recommend this. I come from a different background, and just have different preferences.

7

According to GitHub, there are several ways to integrate changes from one branch into another:

  • Merge branches

  • Rebase branches

  • Apply separate commits from one branch to another (cherry-pick)

  • Apply separate changes from a commit

  • Apply specific file to a branch

Suppose you have created a feature branch to work on a specific task, and want to integrate the results of your work into the main code base after you have completed and tested your feature.

8

Thanks for sharing your experience @brian_pond

May I ask another one ? :slight_smile:

  • Have you already upgrade major version release, e.g. from v12 to v13 ? I assume you also make core changes in ERPNext (?)

  • How did you merge doctype changes that are in JSON format ?

  • How did you perform testing after upgrade ?

P.S. I still stuck at Frappe 12.5.2 & ERPNext 12.8.0 and have customization on core/standard and custom apps as well

Regards,
Subhajit

9
  • Yes, Iā€™ve done some major release upgrades. A few of them required hopping multiple major versions.
  • I handle the JSON documents the same exact same way as .py and .js files. Perform a comparison and merge using git or GUI tools. This sometimes takes longer than I would like. But I donā€™t upgrade very often, so thatā€™s okay.

Regarding testingā€¦

Brianā€™s Test Methodology:

ā†’

testing_code

Now, to answer your question seriously. For certain features, I write Unit Tests. I would like to learn to write Integration Tests; but havenā€™t learned how to do that for ERPNext yet.

For the rest, I perform functional regression testing.

  • Can I still write and post my Purchase Orders?
  • Can I still write an post my Sales Orders?
  • Can I still count my inventory, and perform Stock adjustments?
  • Etc.

In my experience, Testing is 1 of 3* things we usually skip during projects. Not because we donā€™t want to. But because we run out of time, and have deadlines to meet. So the testing gets pushed aside. Which is really unfortunate, because itā€™s much more important than we think.

Hopefully this information helps you. Upgrades are very-often a challenging project.

* the 2 other components that usually get skipped are Reporting, and Documentation
1 Like
10

Thank you so much for sharing Brian !!!