Trying (and failing) to rethink storage

I happen to like collecting DVDs of movies, mostly because I've grown jaded at streaming services constant cost and inconsistent access. Instead of using the disks, I prefer to make a digital backup of the films and view that. This keeps wear on the disks down, and it's much more convenient for me. 

Initially, I used an SFTP backup service for this. That worked, but the transfer speeds weren't great. It helped that I already had the backup service for my websites as well. It felt appropriate to use that for everything.

Two or so years ago, I was gifted a small 4-bay Network Attached Storage (NAS) system. While it didn't come with any drives, I did eventually find some at a local computer recycler and stood up the system for my film collection. This worked amazingly, although I continued to backup my collection to the SFTP backup service just in case.

The drives I got weren't great, but they were working well enough for me. Since then, the 3TB NAS is now nearly completely full of backed-up DVDs. All this time I was still mirroring to the SFTP provider, and the costs jumped high enough for me to be concerned. 

This creates a few problems:

• I would like to eliminate the additional monthly cost from my budget
• My website backups rely on the SFTP provider
• My DVD backups only have the SFTP backup in the case the NAS fails

The NAS is set up as a RAID 5, which, while redundant, can only handle a single drive failure at a time before full data loss comes into effect. The only solution to back that up would be a larger, single drive that can be used as a fall back. This would also be necessary in the case that I would ever want to upgrade the NAS to bigger drives.

After some conversations on Fediverse, it seems the most sensible thing to do would be to target 4 4TB drives in a RAID6 configuration, resulting in a total storage size of 8TB. Thus, if I wanted to have a local backup of the NAS, I'd need one 8TB external drive. 

While in the grand scheme of things, an 8TB external drive isn't that expensive, it's certainly more-so than a impulse purchase. And even if I did have it, what would I do with it afterward? Leave it hooked up to the NAS for a daily backup? 

And then due to recent...events...I start to worry about physical security. In the event of a house fire, societal collapse, fleeing the country, etc., there would be total data loss. This is something the SFTP service accounted for. The backups are elsewhere, not even on the same physical premises. 

To account for that, two 8TB drives would be needed. They would be cycled out every 6 months, the spare being put in a fire resistant box. Given that, only one drive need-be purchased now. Once at least one drive is purchased, it becomes easier to consider buying new drives to expand the cluster.

The website backups still pose a particular difficulty -- and opportunity. If I wanted to completely eliminate the SFTP service expense, I would need to do one of two things:

• Expose an on-premises SFTP server to the public internet to receive pushed backups
• Devise a new backup system for my site which pulls backups instead

My existing backup system backs up the websites on the same cluster on which they're hosted. Once done, they're pushed to any number of remotes, including SFTP. The problem with this is you need that SFTP server to be available and publicly routable. Ideally, I'd put this behind a proxy server such as to protect my home IP address. This becomes a natural initial experiment which may allow me to eliminate some external web hosting entirely.

Alternatively, I could rewrite the backup system to pull backups from the hosting cluster. That too would work, but I'd have to think about how to safely do that. It would require more thought. 

Apparently, my SFTP hosting service charged me on the 1st this month. This leaves me the entire month to consider what to do and purchase any additional hardware. With the holidays approaching, there may be sales which can lower the barrier to entry for this entire project. 

For now, I guess I'm waiting.