Suddenly I find myself trapped in my own home due to COVID-19. It’s not really a big deal because I work from home normally. But it has changed my weekend plans drastically. So it’s time to start tackling one of my 2020 goals:
Continue reading →Organizing all of our digital files and having a consistent back-up strategy.
Organization
At our house important files mostly fall into two realms: Photos and Music. Many people have at one point decided to keep all of their photos or music with a 3rd party service like Flickr or iTunes. That’s fine, but you just have to be prepared to commit to them like a marriage – except they decide all of the terms – it’s not a mutual agreement. Personally, no matter if I upload my photos to a 3rd party service, I always keep the originals on a disk in my home so I can own my own content.
But this becomes an issue when everyone in your family isn’t on the same page. I have some photos on a USB drive and on my computer, and my wife has done the same on a different USB drive and computer. We need a way to have photos accessible in a consistent place, and make sure they’re getting backed-up.
When it comes to shopping for a big hard drive to store all of this stuff, I try to find whatever is the best space-per-cost within my budget. At one point I had devised a “2×100” rule: I get the biggest drive I can for $100 and consider buying two of them (for redundancy). As disk sizes increase, once a drive is available that is 2x the size of my current one for $100, I replace them. Currently you can get up to 4TB drives for $100.
I bought a 4TB drive to use as network attached storage to act as our central storage place. I attached it to a Raspberry Pi using a USB/SATA hard drive dock and share it read-only over the network using Samba. It shows up as a Windows-style network share on all computers and other devices like AppleTV & FireTV (using VLC). For read-write access, I use SSHFS so authenticated peeps can write to the disk and move files around.
We’re still working on how to consistently name our photo albums, but having them in one place that gets backed up off-site is a big win.
Backups
I used to be an amateur network administrator. In my back-office (laundry room) I had a tower server that had 6 disk drives. A 3-disk RAID5 for backups, a 2-disk RAID0 for temporary storage (for MythTV recordings), all in swappable drive trays. Then there was one more disk hard-mounted for operating system boot-up. It worked for a while, but when I was going to have to replace the 1000-Watt power supply for the second time, I thought maybe there’s a better way. One thing I hadn’t considered before is: the more disk drives you have, the higher chance of having a drive fail – an increased chance of the click of death was always in the back of my mind. Using one big disk with a Raspberry Pi is much simpler, smaller, and uses much less power.
With all disks on-site, there is still the chance of everything getting destroyed by fire. To handle off-site backup I had used CrashPlan, but that didn’t pan out – time to look for something different. Since we have quite the collection of different computers here – Linux, MacOS & Windows, I looked for a consistent backup solution that is operating system agnostic. I wound up going with Duplicati using Backblaze’s B2 for off-site storage.
All of our computers use Duplicati independently to backup to B2. The Raspberry Pi does as well to backup our photos and music.
The Setup
I’m still working out (and documenting) the details of our setup, but I’ll list the specifics of our setup here as I get to them:
If you have any tips or strategies you could offer, please let me know in the comments!
