Suddenly I find myself trapped in my own home due to COVID-19. It’s not really a big deal because I work from home normally. But it has changed my weekend plans drastically. So it’s time to start tackling one of my 2020 goals:
Organizing all of our digital files and having a consistent back-up strategy.
At our house important files mostly fall into two realms: Photos and Music. Many people have at one point decided to keep all of their photos or music with a 3rd party service like Flickr or iTunes. That’s fine, but you just have to be prepared to commit to them like a marriage – except they decide all of the terms – it’s not a mutual agreement. Personally, no matter if I upload my photos to a 3rd party service, I always keep the originals on a disk in my home so I can own my own content.
But this becomes an issue when everyone in your family isn’t on the same page. I have some photos on a USB drive and on my computer, and my wife has done the same on a different USB drive and computer. We need a way to have photos accessible in a consistent place, and make sure they’re getting backed-up.
When it comes to shopping for a big hard drive to store all of this stuff, I try to find whatever is the best space-per-cost within my budget. At one point I had devised a “2×100” rule: I get the biggest drive I can for $100 and consider buying two of them (for redundancy). As disk sizes increase, once a drive is available that is 2x the size of my current one for $100, I replace them. Currently you can get up to 4TB drives for $100.
I bought a 4TB drive to use as network attached storage to act as our central storage place. I attached it to a Raspberry Pi using a USB/SATA hard drive dock and share it read-only over the network using Samba. It shows up as a Windows-style network share on all computers and other devices like AppleTV & FireTV (using VLC). For read-write access, I use SSHFS so authenticated peeps can write to the disk and move files around.
We’re still working on how to consistently name our photo albums, but having them in one place that gets backed up off-site is a big win.
I used to be an amateur network administrator. In my back-office (laundry room) I had a tower server that had 6 disk drives. A 3-disk RAID5 for backups, a 2-disk RAID0 for temporary storage (for MythTV recordings), all in swappable drive trays. Then there was one more disk hard-mounted for operating system boot-up. It worked for a while, but when I was going to have to replace the 1000-Watt power supply for the second time, I thought maybe there’s a better way. One thing I hadn’t considered before is: the more disk drives you have, the higher chance of having a drive fail – an increased chance of the click of death was always in the back of my mind. Using one big disk with a Raspberry Pi is much simpler, smaller, and uses much less power.
With all disks on-site, there is still the chance of everything getting destroyed by fire. To handle off-site backup I had used CrashPlan, but that didn’t pan out – time to look for something different. Since we have quite the collection of different computers here – Linux, MacOS & Windows, I looked for a consistent backup solution that is operating system agnostic. I wound up going with Duplicati using Backblaze’s B2 for off-site storage.
All of our computers use Duplicati independently to backup to B2. The Raspberry Pi does as well to backup our photos and music.
I’m still working out (and documenting) the details of our setup, but I’ll list the specifics of our setup here as I get to them:
If you have any tips or strategies you could offer, please let me know in the comments!
Great post- thanks Justin. What do you do for remote access?
Thanks Gerry – in the past (with my tower server) I had opened up a non-standard SSH port on my router because Comcast, my ISP, was filtering port 22. I redirected that traffic to my home server. I used nsupdate combined with a self-hosted nameserver https://www.foell.org/justin/diy-dynamic-dns-with-openwrt-bind/ to make a “home” domain name so I didn’t have to remember the IP address.
Honestly, I let it go because I’ve been doing fewer and fewer client visits, but it is worth revisiting – especially if I want to listen to some of my music while I’m away 🎧. I’m thinking of dropping my server so I’d have to rethink the domain name thing.
Really cool- I’m a hobbyist who also does a bit of network/switch configuration in my job, so I understand enough of what you’re saying. I simply had no idea how you could DIY such a thing. What does the user interface look like- are you accessing through a browser or something? The reason I ask is because your comments about the proprietary storage platform really resonates with me. I pay for Dropbox, and I think it’s a good value, actually, and really like the way it works in ‘simple mode’, but it’s just a matter of time, I suppose, before the take the big evil step. Last update, they made the default mode ‘no local storage’ which is the main reason I liked it. No doubt they want to make it difficult to leave 🙂
Oh yeah, also, remote access is critical for me- I can find myself working _anywhere_ almost, so getting that working is key.
Gerry – I did find this: https://www.nsupdate.info/about/ which is a free dynamic DNS service that supports nsupdate. I’m going to look into this – wait for a post about it 👍
Pingback: RaspberryPi + IncrediblePBX = Phone/Fax/NAS - Justin Foell
Pingback: Duplicati setup tips for Linux, RaspberryPi & MacOS - Justin Foell
Pingback: Dynamic DNS with OpenWrt 19.07 and nsupdate.info - Justin Foell