For smaller backups <10GB ea. I run a 3 phased approach
- rsync to a local folder /srv/backup/
- rsync that to a remote nas
- rclone that to a b2 bucket
These scripts run on the cron service and I log this info out to a file using --log-file option for rsync/rclone so I can do spot checks of the results
This way I have access to the data locally if the network is down, remotely on a different networked machine for any other device that can browse it, and finally an offsite cloud backup.
Doing this setup manually through rsync/rclone has been important to get the domain knowledge to think about the overall process; scheduling multiple backups at different times overnight to not overload the drive and network, ensuring versioning is stored for files that might require it and ensuring I am not using too many api calls for B2.
For large media backups >200GB I only use the rclone script and set it to run for 3hrs every night after all the more important backups are finished. Its not important I get it done asap but a steady drip of any changes up to b2 matters more.
My next steps is to maybe figure out a process to email the backup logs every so often or look into a full application to take over with better error catching capabilities.
For any service/process that has a backup this way I try and document a spot testing process to confirmed it works every 6months:
- For my important documents I will add an entry to my keepass db, run the backup, navigate to the cloud service and download the new version of the db and confirm the recently added entry is present.
- For an application I will run through a restore process and confirm certain config or data is present in the newly deployed app. This also forces me to have a fast restore script I can follow for any app if I need to do this every 6months.
Extortion is when corporations have to pay for the services they consume