Four Years Later: Symlinks and Dropbox for Backup and Restore

Note: This post is about is a homegrown solution to backups/restores I developed four years ago, not the newly-announced Dropbox Backup.  

I originally developed this method for four reasons:

• restoring from backups was never a seamless or pain-free process
• settings and preferences I adjusted over time were inevitably lost and hard to set back up in the exact same way
• duplicating my preferences and development environment across more than one computer always ended up with some level of delta
• I wanted access to all my files from my phone, not just what I put in my Dropbox folder

Dropbox fits this bill nicely, particularly for numbers one and four above.  For a Restore of my files on a new compuer, I wanted to the entire process to flow be just this:

• run my script (now written in Ansible)
• install Dropbox
• sign in (to Dropbox)

That's it.  No picking where to put files nor selecting a date to restore from.  And when the files are done downloading from the cloud, it's exactly as I left it.  Resuming work is easy because I don't have to spend a day or week setting things back up just the way I like.  My files just start appearing in the place they should go.

No Deltas

Even if I had been making changes on another Mac running this setup, those settings/files would be there when I did this restore (since my files are constantly syncing via Dropbox), so there's no manual work to fill in the gap while one of my Macs was out of comission.

Backing Up/Syncing Preferences

Since Apple saves most of their preferences in .plist files, it's quite trivial to back them up since they are just a regular old xml file.

What Makes It Work?

1. Dropbox (for syncing changes to/from the cloud)
2. Hard and soft links
3. Apple's File System Domains
4. Apple's Home Folder Structure

1. Dropbox

Dropbox is quite efficient at downloading and uploading files.  It doesn't even struggle with .git folders, which have all sorts of files in them (though they do caution against syncing these).  

So even if I make a small change, like changing a preference (thus writing to a .plist file), that change is detected by Dropbox and synced to the cloud.  

2. Hard and Soft Links

Modeled after Apple's Time Machine itself, this puts links to good use by keeping Apple's Home Folder structure but replacing them with symlinks.  I organize all my files into their main folders, plus some uncreated ones they don't really include any more ( ~/Applications and ~/Sites).  

3. Apple's Home Folder Structure

Like I mentioned, keeping files in the spots Apple hints at: documents in Documents/, photos and pictures in Pictures/, etc work in your favor here because this hasn't changed much over many years.  It's a nice logical structure, and many parts of the OS integrate with these locations.

4. Apple's File Resource Domains

This part is not commonly known, but Apple organizes it's files into Resource Domains:

There are four file-system domains:

• User. The user domain contains resources specific to the user who is logged in to the system. This domain is defined by the user’s home directory, which can either be on the boot volume (/Users) or on a network volume. The user has complete control of what goes into this domain.
• Local. The local domain contains resources such as applications and documents that are shared among all users of a particular system but are not needed for the system to run. The local domain does not correspond to a single physical directory, but instead consists of several directories on the local boot (and root) volume. Users with system administrator privileges can add, remove, and modify items in this domain.
• Network. The network domain contains resources such as applications and documents that are shared among all users of a local area network. Items in this domain are typically located on network file servers and are under the control of a network administrator.
• System. The system domain contains the system software installed by Apple. The resources in the system domain are required by the system to run. Items in this domain are located on the local boot (and root) volume. Users cannot add, remove, or alter items in this domain.

So by limiting the files that get synced to the User and Local domains, it's easy to script and keep things the Apple way.

The Script

I converted the bash script I used to use into and Ansible script because it's a little easier to maintain.  Essentially, it does the following:

• forcefully removes ~/* (the default folders in your home directory)
• create symlinks for those same folder names that point to my Dropbox folder of the same name
• create hard links for things like my dotfiles ( .bash_profile, .vimrc, etc.) and other files that may be in the Local resource domain
• runs some chflags and a few other commands to tidy things up a bit on the Finder end of things

That Magical Moment

I've only experienced this feeling one other time, but when I install Dropbox and run my script, all my files magically appear without any other intervention.  That's my favorite thing about this whole setup.

Caveats

Just one small one that has never caused me any problems.

Summary

Your backups are only useful if they are easy to restore.  

Using Dropbox is is great for this because when you install it and sign in, files immediately start downloading without any further intervention.  For that reason, I continue to use this method to this day (plus I already pay for Dropbox, so there's no need to pay for another backup solution).