GIT Experience Share: How to proper setup for Contributing feature to the Core

Frappe Framework
1

So, ppl of Frappe/ERPNext, I am seeking to optimize our teams setup for contribution back to core. Please share experience on your setup.

We are trying to have stable Production server and at the same time contributing back.

Scenario:
We are using ERPNext with a small team of devs (2-3 persons). We have a feature we want to contribute back to core. This feature is essential for our end users and we need it to our system asap but in the same time is generic for everyone in the community. We first need to test it before pushing to code.

Note: Discuss about the feature is not part of this. We as seeking only git repo setup to optimize the process. Also git fetch and git pull are consider known steps.

Current Setup

Production Server :

  • frappe: Official Repo remote in develop branch
  • erpnext: Official Repo remote in develop branch
  • customapp: Repo remote in production branch

Request is made for FeatureA that involve to change core fields / methods.

Initial Steps:

  1. Team Leader forks the Official ERPnext repo, I will call it “TL-ERPNext-fork”
  2. Each Dev forks TL-ERPNext-fork I will call it “DEV-ERPnext-fork”
  3. Team Leader create a new branch to “TL-ERPNext-fork” named after the feature example “feature-A”
  4. Each team member add to their local the TL-ERPNext-fork remote : git remote add TL-ERPNext-fork <git url>
  5. Each team member add to their local the “DEV-ERPnext-fork” remote : git remote add "DEV-ERPnext-fork" <git url>

So at this stage a dev for ERPNext repo will have 3 remotes:

  1. Official ERPNext
  2. DEV-ERPnext-fork
  3. TL-ERPNext-fork

Developing Feature:

  1. Dev member: `git checkout branch TL-ERPNext-fork/feature-A
  2. Make the changes
  3. Push to his/her own repo: git push DEV-ERPnext-fork feature-A
  4. Ask your PR from TEAM Leader between branches DEV-ERPnext-fork/feature-A versus TL-ERPNext-fork/feature-A
  5. this circle is continuing until the feature is done

Here we start will the problems,

Testing the Feature in real Production Server case (show case to end users). Of course there can be a Staging Server for Testing but for now we assume as equal for simplicity.

  1. Team Leader add to Production server the new repo: git add remote TL-ERPNext-fork <git url>
  2. Team Leader switch to branch with the feature: git checkout TL-ERPNext-fork/feature-A
  3. Team Leader manually updates: bench migrate or maybe need css/js also bench build

At this stage
we have The Problem 1 , the bench update.
The Production server ERPNext is with remote to TL-ERPNext-fork but in the same time core team fix a bug that we need our system. If we bench update we get the conflict … You aren’t in develop branch (because in our setup is the default).
In order to pass this one, we need big workarourd.

  1. Team Leader: Must push to TL-ERPNext-fork the new fetch/pull Official ERPNext
  2. Team Leader: Must rebase the feature, git chekcout feature-A, git rebase develop
  3. Next, Go to Production server and manual update will: fetch/pull and bench migrate

OK, we can survive … we can do it manually

next the PR to the core… the feature is ready

  1. Team Leader creates a PR to Official ERPNext versus TL-ERPNext-fork/feature-A
  2. Write a commit message etc and get first review

we have the The Problem 2, the Code Review
The response for review is change the name of a field from fieldA to fieldB. (Team leader and dev members are having a break down, went for a beer, smashed an employee computer because it was asked to fix the printer).

  1. Team Leader: Make the changes and push updated updated PR
  2. Team Leader: Goes to production server updates again MANUALLY the server instance to play with the new field

The days are passed …
The Team Leader remember the days were a bench update has enough.

PR is now merged we are in The Problem 3 ,
Our Production Server has the PR as branch … and is merged to the core !!!
Where is the data? Our users have used the feature and added data!!!
If we switch to Official ERPNext repo and bench update the data? will be merged? NO!!!
The dark side of the data.

Please share experience… :slight_smile:

1 Like
2

Too complex!

More than repos, maintain branches for your own production. Maybe just keep a eso-develop and eso-production branch in addition to the standard develop, staging and hotfix

If you are okay waiting for a 2-3 weeks and getting good quality, let everything go via the official branches. Not only will you get better quality check and code quality, but it will be very easy to workflow.

If you can’t wait at-all, then you will have to maintain separate merges. Which is when you get into this spaghetti… When you think about it, 2-3 weeks wait is not a bad deal. In a couple of months hopefully a couple of you would graduate to reviewers and maintainers, then you can speed up the workflow.

3

Of course is complex. We have a team of devs and company that relies of the ERP. We must have s stable system 24/7 without bugs but the same time the flexibility for developing and contributing without any side affects to production.

Sorry but this out of the scope of a enterprise setup . Production should be clean as possible. We are running in most security / not break / not even touch scenario not because we are crazy but because our business relay of this stable state. The clear path for this should be Production Server should have a clear production branch that only tested code is pushed but can not happen at the moment.

So, summarising,
From your comment I take that for you the setup should be :

  1. Keep multi branches in production server, all work to same repo without MR or PR or each dev fork, all as masters?
  2. Do not merge or switch to develop branch until your feature is merged to core ? (2-3 week , I totally understand the timelag for this)

PS. I am using our company and our business not as ESO but in general term like I own a company.