Hi All,
we have 20GB Data on production and we have a custom app also,
after every production upgrade bench migrate processes took nearly 2 hrs to just migrate the one site. So I want to optimize it. not sure why its taking time to migrate even Redis-cache open.
Please provide me a suggestion on above topic
Maybe you can check which patches are taking time to run and optimize the SQL(s) or code behind them.
@Pawan As per Observed in frappe codebase, I found below Observations
While bench migrate we migrate all fixtures .json which are present in the respective directory of app,
In that process, bench migrate re-insert all fields which are not updated as well as the new one/updated and also in simple bench migrate process it does the same. Due to this, we are doing unnecessary Alteration in Database if there is no change in other fields like custom_field.json,property_setter.json,role.json etc … and bench migrate takes too much of time just to syncing.
Another Major Point. if you have multiple Apps in System, fixtures of the last app erased by fixtures of another app (Next app - last migrated app) if both fixtures are not in sync and we end with issues/bug fixes and developer goes into a panic situation 
.
Solution:
I have Added one condition, which checks whether field already exists or not in Database with the same modified date. if it is not modified then it exits from the loop and if it is a new field or modified then it will insert into the custom field.
For Multiple App it will not override another apps fixtures in Database. Will update only those are modified or updated.
Due to this Fix, we improved bench migrate process check below Screenshot for analysis on the server-side.
If it is a good one then can I send Pull Request in frappe?
OR Please Correct me if it is wrong.
How would it work with Dec machine date time differences?
Also we see a need to order the fixtures when exported so that the diff shows easily. fix: Order the fields in fixtures based on the order specified in hooks.py · Issue #10909 · frappe/frappe · GitHub
Dear @aakvatech yes your point is valid but have done changes in migrate.py for sync_fixtures which unnecessarily re-inserting field even it is not modified.
