Before you install Duplicati, there’s one question you need to ask:
Does this computer have multiple user logins (that you want to back-up), or is it primarily used by one user?
This is important because you can install Duplicati in one of two ways: to run as a single-user, or to run as a system service that is available to all users. By default Duplicati is installed for a single user, but if there is more than one user on the computer you want to be able to back-up, you’ll want to install it as a service.
If you don’t install it as a service, when Duplicati tries to read files that belong to the other users, it will get a permission denied error and won’t be able to back those files up. You don’t want to find out that your files weren’t backed up when it’s too late.
I’ve provided some MacOS specific configurations in that section for setting it up as a service.
RaspberryPi
Install Duplicati on a RasperryPi
To get Duplicati successfully installed on a RaspberryPi running Raspbian, you’ll likely have to also install some of the mono dependencies to avoid installation errors.
Once that’s done you can (re)install Duplicati, hopefully without errors.
MacOS
For my MacOS installation I needed to install it as a service. It’s my wife’s Macbook. I also use it for Fusion360, and I like my scroll settings different than she does. So it has two user accounts, both with files we want to back-up. Installing Duplicati as a service in this case is a must.
Both of the plist configuration files can be installed as any user as they’ll be going in a system-wide location which will run Duplicati as a system service.
Service LaunchDaemon
$ sudo vi /Library/LaunchDaemons/com.duplicati.server.plist
Once those two files are in place, the next time you reboot and log in, any and all users should have a Duplicati icon in the system tray. You can then configure a backup job to read all users’ files without issue.
Useful Duplicati Options (all platforms)
There are two settings to reduce Duplicati’s resource usage:
use-background-io-priority
thread-priority
On my brand-new 2019 XPS13 developer system Duplicati was using 280% CPU.
In other words, three of the 8 available cores were dedicated to Duplicati, plus disk I/O. I recommend setting use-background-io-priority to true and thread-priority to something below “normal” to adjust the CPU usage to an acceptable level. You can find these under “Advanced options” on the last page of the Add/Edit Configuration page of a backup.
Test
As always, test your configuration. I like to do a daily backup with “Smart backup retention” so my backup cloud disk usage doesn’t grow indeterminately.
Make sure your RaspberryPi backups are completing. It will take a while if you have a lot of data. For a 1TB backup from my RaspberryPi, it took a full week to complete because it is s-l-o-w. Once a full backup is made, the incremental updates shouldn’t take as long, but they may still take longer than a day to complete.
Suddenly I find myself trapped in my own home due to COVID-19. It’s not really a big deal because I work from home normally. But it has changed my weekend plans drastically. So it’s time to start tackling one of my 2020 goals:
Organizing all of our digital files and having a consistent back-up strategy.
At our house important files mostly fall into two realms: Photos and Music. Many people have at one point decided to keep all of their photos or music with a 3rd party service like Flickr or iTunes. That’s fine, but you just have to be prepared to commit to them like a marriage – except they decide all of the terms – it’s not a mutual agreement. Personally, no matter if I upload my photos to a 3rd party service, I always keep the originals on a disk in my home so I can own my own content.
But this becomes an issue when everyone in your family isn’t on the same page. I have some photos on a USB drive and on my computer, and my wife has done the same on a different USB drive and computer. We need a way to have photos accessible in a consistent place, and make sure they’re getting backed-up.
When it comes to shopping for a big hard drive to store all of this stuff, I try to find whatever is the best space-per-cost within my budget. At one point I had devised a “2×100” rule: I get the biggest drive I can for $100 and consider buying two of them (for redundancy). As disk sizes increase, once a drive is available that is 2x the size of my current one for $100, I replace them. Currently you can get up to 4TB drives for $100.
I bought a 4TB drive to use as network attached storage to act as our central storage place. I attached it to a Raspberry Pi using a USB/SATA hard drive dock and share it read-only over the network using Samba. It shows up as a Windows-style network share on all computers and other devices like AppleTV & FireTV (using VLC). For read-write access, I use SSHFS so authenticated peeps can write to the disk and move files around.
We’re still working on how to consistently name our photo albums, but having them in one place that gets backed up off-site is a big win.
Backups
I used to be an amateur network administrator. In my back-office (laundry room) I had a tower server that had 6 disk drives. A 3-disk RAID5 for backups, a 2-disk RAID0 for temporary storage (for MythTV recordings), all in swappable drive trays. Then there was one more disk hard-mounted for operating system boot-up. It worked for a while, but when I was going to have to replace the 1000-Watt power supply for the second time, I thought maybe there’s a better way. One thing I hadn’t considered before is: the more disk drives you have, the higher chance of having a drive fail – an increased chance of the click of death was always in the back of my mind. Using one big disk with a Raspberry Pi is much simpler, smaller, and uses much less power.
With all disks on-site, there is still the chance of everything getting destroyed by fire. To handle off-site backup I had used CrashPlan, but that didn’t pan out – time to look for something different. Since we have quite the collection of different computers here – Linux, MacOS & Windows, I looked for a consistent backup solution that is operating system agnostic. I wound up going with Duplicati using Backblaze’s B2 for off-site storage.
All of our computers use Duplicati independently to backup to B2. The Raspberry Pi does as well to backup our photos and music.
The Setup
I’m still working out (and documenting) the details of our setup, but I’ll list the specifics of our setup here as I get to them:
