I have an installation with “System settings > Number of backups” set to 20.
Every night,
bench backup --with-files
is executed.
But instead of keeping 20 last files, only 2 files are kept. This seems to be unrelated to the value entered into the “Number of backups”.
Observed in
ERPNext: v8.5.2 (master)
Frappe Framework: v8.5.5 (master)
Can someone confirm this as a bug or am I missing something?
Same issue on VM instance.
Please advise…
Would you mind creating a github issue for this. Seems to be replicating in test account as well.
Thanks for your cooperation.
If you do a
bench update --upgrade
it reverts all backups to the last two (2) instance.
Please close this issue.
An update on this: this problem is still present in Frappe 8.10.
It cannot be reproduce when manually running
bench backup
in this scenario it works just fine. But when using the automated backup (nightly), it only keeps 2 nights… To reproduce, set number of backups to any value > 2. Then wait three days and check number of backups. It will be 2. Should be as many as defined.
What is the difference between the nightly backup (crontab, see above) and manual bench backup? Are the --with-files posing an issue?
in a prod scenario you would backup those to a separated backup server anyway (instead ofkeeping them on the same machine). Once you do this the problem is not as severe any longer (even though the automated backup should keep as many copies as you tell it to )
There are different ways to look at this even with external backup storage
The nice thing about this backup feature is that the end-customer can download them easily. Therefore, I still see this as a bug.
This issue still prevails in Frappe v13, System is not keeping more than 5 backups even if I set the number higher
Hi. Found same issue on V11 and V12. Not more than 5 backups are stored. No idea why it doesn’t go higher as per settings.
Anybody managed to make it work to go higher than 5 backups?
To my understanding, the System Settings > Number of backups is for auto backup. It will keep daily backups. So for 3 backups (the standard) it will do auto backup every 8 hours (24÷3).
While the bench backup will do the backup of current state. So only 1 set of backup (4 files if with files: db, files, private-files, site_config.json).
cmiiw
Here is my findings for anyone still dealing with auto backup issues
Auto backup function is triggered for every 6 hours by a cron job present in crontab of [frappe_user]. The backup period is determined by the cron expression of cronjob, For eg: */6 * * *
0 */6 * * * cd /home/frappe/frappe-bench && /usr/local/bin/bench --verbose --site all backup >> /home/frappe/frappe-bench/logs/backup.log 2>&1 # bench auto backups set for every 6 hours
The number of backups to be kept is determined by two different variables. backup_limit field in System Settings limits the maximum number of backups, this field is only used to delete the extra backups when you are refreshing the Download Backups page. keep_backups_for_hours from site_config.json determines how much time a backup is to be kept and if undefined, it will default to 24. This condition is checked every time a bench backup is performed so the reason for auto backups maxing out at 5 is the default value of keep_backups_for_hours
To solve this inconsistency, You can ignore the backup_limit of System Settings by assigning it a very large number(Eg: 1000) and use keep_backups_for_hours for limiting backups.
CC: @aakvatech @Muzzy @rahy
Reference: fix(backups): ensure `delete_temp_backups` always respects config by sagarvora · Pull Request #12807 · frappe/frappe · GitHub
frappe/backups.py at ab3be339fdc6c6aef314f46e6bcb73e0f1cb8a6b · frappe/frappe · GitHub
frappe/backups.py at ab3be339fdc6c6aef314f46e6bcb73e0f1cb8a6b · frappe/frappe · GitHub
Thank you @manasan for the explanation.
But I found these info a bit confusing:
Which one is the default? 24 hours or 5 hours?
Or 5 is the number of backups acitivty per day (24 hrs) = every 4.8hrs?
And the keep_backups_for_hours setting in site_config.json should be manually created?
Because I plan to make the field in System Settings to read only and/or hidden so user can’t change it (affecting the storage capacity planning).
If we do this, I think it will be even more unpredictable.
Example scenario:
backup_limit = 1000
keep_backups_for_hours = 6 hrs
Now, if in that 6 hrs we do bench backup 50 times there will be more than 50 backups are available. This can be too much storage.
keep_backups_for_hours default value is 24, which translates to 24 hours. To change the keep_backups_for_hours, you need to manually define it in the site_config.json.
5 is the number of backups preserved, if default value of 24 hours is unchanged. The default backup interval is 6 hours so 24 hours/6 hours will result in 4 intervals and the number of backups will be 5 because of arithmetic(4 gaps and 5 poles)
If you plan to hide the backup_limit in System Settings make sure its default value is set to some higher number.
I edit my comment above when your last answer posted.
Can you please comment on my last comment also?
Thank you
In this case these 50 backups will get deleted on next bench backup after 6 hours ageing from the file creation time. backup_limit will not help you here because it deletes the extra backups only when you are refreshing the Download Backups page. So until you visit the backups page in your desk, you will have the high storage issue.
So in short, backup_limit is pretty useless
Can these be used in common_sites_config?