on-the-fly encrypted backups

I was wondering how easy it was to encrypt files before rsyncing them away to the backup machine.

A quick search turned up the suggestion to use encfs by the user Thor on ServerFault.

That looks like a decent solution. Let’s figure out if it meets our needs.

The idea is that we do this:

# mount read-only encrypted virtual copy of unencrypted local data:
encfs --reverse -o ro ~/data/ ~/.tmp_encrypted_data/

# at this point, you can rsync your data to the backup location.

# unmount like this:
fusermount -u ~/.tmp_encrypted_data/

So, what does that look like? I tested it on my Documents directory, and it looks like this:

$ mkdir ~/Documents-enc
$ encfs --reverse -o ro ~/Documents ~/Documents-enc
Creating new encrypted volume.
Please choose from one of the following options:
 enter "x" for expert configuration mode,
 enter "p" for pre-configured paranoia mode,
 anything else, or an empty line will select standard mode.

For now, I went with standard, which produced a Documents/.encfs6.xml file with these parameters:

    <name>ssl/aes</name>
...
  <keySize>192</keySize>
  <blockSize>1024</blockSize>

At this point, you have two directories, one if which has read-only encrypted files:

$ du -sh Documents Documents-enc
319M  Documents
319M  Documents-enc

$ ls -ltr Documents | head -n4; ls -ltr Documents-enc | head -n4
total 192140
-rw-r--r-- 1 me me 1608 mrt 15  2010 customer_dev.oo.db.odb
-rw-r--r-- 1 me me 7810 mrt 15  2010 foo.ods
-rw-r--r-- 1 me me 8390 mrt 16  2010 salesliters.france.ods
total 192148
-rw-r--r-- 1 me me 1608 mrt 15  2010 R7YZkAKRlOLVJYK1YnzVm6rpZW6glWIJvDpBjb4uLVjFh,
-rw-r--r-- 1 me me 7810 mrt 15  2010 gCalD6peJm8GllS-PtaWvN14
-rw-r--r-- 1 me me 8390 mrt 16  2010 OIBZny,CQlxoHX7ZdhyZEvTB68c6cJ7tvPSkgUtHdDDyg1

File sizes (*), file ownership and time stamps are unaltered. File names and file contents are encrypted.

(*) The fact that the file sizes are identical, means there is no per-file initialization vector, despite that the 1.7.4-2.4ubuntu2 manual says that they are enabled by default. The uniqueIV option is not available for --reverse mounted filesystems, because of the nature of the filesystem: it has no place to store the randomly chosen IV; and you don’t want it constantly changing.

The block based cipher keeps changes inside a block. I edited one of the first and one of the last bytes in a file, and got this:

$ cmp  Documents-enc/qbEp9gilGP8PbjxNYZh6YhjdfebfTuxqM3cBDg0Fw0,6I- Telecommwet1.txt  -bl
   465 141 a    170 x
   466  16 ^N   344 M-d
   467 320 M-P  135 ]
...
  1022 326 M-V  370 M-x
  1023 122 R    220 M-^P
  1024 366 M-v  370 M-x
430529 141 a    273 M-;
430530   0 ^@    60 0
430531 375 M-}  262 M-2
...
430606 260 M-0  270 M-8
430607  14 ^L   110 H
430608 271 M-9    6 ^F

That allows rsync and incremental backups to operate more efficiently when files are only appended to or changed inline without inserting bytes. (Depending on the common type of changes you make to files, you could add the --whole-file parameter to rsync.)

Decrypting?

To decrypt the data again, you will need the configuration file. Make sure you have that backed up as well.

A quick command to decrypt a single file:

$ ENCFS6_CONFIG=/home/walter/Documents/.encfs6.xml encfsctl decode path/to/encrypted/file \
    qbEp9gilGP8PbjxNYZh6YhjdfebfTuxqM3cBDg0Fw0,6I-
EncFS Password:
original.rtf

$ ENCFS6_CONFIG=/home/walter/Documents/.encfs6.xml encfsctl cat path/to/encrypted/file \
    qbEp9gilGP8PbjxNYZh6YhjdfebfTuxqM3cBDg0Fw0,6I- > original.rtf
EncFS Password:

$ file original.rtf
original.rtf: Rich Text Format data, version 1, ANSI

But you will probably mount the entire filesystem instead — unless you chose to not encrypt the file names.

$ mkdir ~/Documents-enc-dec
$ echo my-password | ENCFS6_CONFIG=/home/walter/Documents/.encfs6.xml encfs \
    --stdinpass ~/Documents-enc ~/Documents-enc-dec

$ tthsum Documents{,-enc-dec}/salesliters.france.ods
ET73GNCTLCLTNURHZMK2AFY4BI7RN2SQQLWXR2Q  Documents/salesliters.france.ods
ET73GNCTLCLTNURHZMK2AFY4BI7RN2SQQLWXR2Q  Documents-enc-dec/salesliters.france.ods

What’s next?

• Right now we backup customer files by initiating the rsync from the backup server. We can still do that, but we need to:

  • Pre-mount the directories we want to back up. The encfs mount tool provides several useful options for this (--ondemand, --extpass).

  • Back up the ENCFS6_CONFIG config file itself. If you want it backed up on the backup host, encrypt it separately with PGP and place it next to the encrypted dirs. Securing the PGP keys and the filesystem password is your own responsibility.

  • Limit access to the backup provider. You’ll probably need to mount the files as root since you want to back up sensitive files. The backup user gets NOPASSWD sudo power to rsync:

    backuppc ALL=NOPASSWD: /usr/bin/rsync --server --sender *
    backuppc ALL=NOPASSWD: /usr/bin/ionice -c2 -n7 /usr/bin/rsync --server --sender *
    

    Limiting access to the encrypted directories only should be done by a separate tool, such as AppArmor. Or you could push the backups to the backup host instead of having them polled. But that may not be an option.

  • I did not check whether encryption is always done, or just on-demand. If the latter is the case, we suffer no performance loss while we’re not doing a backup run.

• If you prefer comfort over security, you can choose to use null-encryption on the file names. This may be a valid trade-off if your backup host is decently secure.

• The lack of a per-file IV means that it’s not a secure as it could be. Again we have the security-usability trade-off.

Please do read the encfs manpage for more info. It’s nice and verbose.

#!/bin/sh

echo -n "Password: "
read password

input=`mktemp -d`
output=`mktemp -d`

printf 'x\n1\n256\n4096\n2\n%s\n' "$password" |
    encfs --stdinpass --reverse "$input" "$output" >/dev/null
fusermount -u "$output"

key=`grep -A1 '<encodedKeyData>' "$input/.encfs6.xml" | tail -n1`
salt=`grep -A1 '<saltData>' "$input/.encfs6.xml" | tail -n1`
iters=`sed -ne 's/.*<kdfIterations>\([^<]*\)<.*/\1/p' "$input/.encfs6.xml"`

echo "key = $key"
echo "salt = $salt"
echo "iters = $iters"

rm -rf "$input" "$output"