Proper Aegir backups with Backupninja and duplicity under Debian 7

A client using Aegir had a unique requirement for backups of their hosted Aegir  sites.

In short, the client wanted a complete backups of their Aegir sites.  They had the following demands;

  • Full Aegir-compatible backup files.  They didn’t want to use incremental backups, for ease and restore speed
  • Backups must be created via a drush command for completeness
  • Backups should be secure and integrity maintained.
  • Backup system should not be complicated.  It should be easy to add and remove sites from the backup

As it turns out, using duplicity, backupninja and some bash scripting, I was able to come up with a solution.

Installation

First, we install requirements

sudo apt-get install backupninja duplicity gnupg

Generate GPG Keys

Now we will need to generate gpg keys for encryption and signing.

sudo gpg --gen-key

Which results in something like the following

kurth@aegir:~# sudo gpg --gen-key
gpg (GnuPG) 1.4.10; Copyright (C) 2008 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

gpg: directory `/root/.gnupg' created
gpg: new configuration file `/root/.gnupg/gpg.conf' created
gpg: WARNING: options in `/root/.gnupg/gpg.conf' are not yet active during this run
gpg: keyring `/root/.gnupg/secring.gpg' created
gpg: keyring `/root/.gnupg/pubring.gpg' created
Please select what kind of key you want:
   (1) RSA and RSA (default)
   (2) DSA and Elgamal
   (3) DSA (sign only)
   (4) RSA (sign only)
Your selection? 1
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 4096
Requested keysize is 4096 bits
Please specify how long the key should be valid.
         0 = key does not expire
      <n>  = key expires in n days
      <n>w = key expires in n weeks
      <n>m = key expires in n months
      <n>y = key expires in n years
Key is valid for? (0)
Key does not expire at all
Is this correct? (y/N) Y

You need a user ID to identify your key; the software constructs the user ID
from the Real Name, Comment and Email Address in this form:
    "Heinrich Heine (Der Dichter) <heinrichh@duesseldorf.de>"

Real name: Duplicity Encryption and Signing Key
Email address: sysadmin@aegir.example.com
Comment:
You selected this USER-ID:
    "Duplicity Encryption and Signing Key <sysadmin@aegir.example.com>"

Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? O
You need a Passphrase to protect your secret key.

We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.

......+++++
.........+++++

gpg: /root/.gnupg/trustdb.gpg: trustdb created
gpg: key CC593F68 marked as ultimately trusted
public and secret key created and signed.

gpg: checking the trustdb
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: depth: 0  valid:   1  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 1u
pub   4096R/CC593F68 2015-02-07
      Key fingerprint = 5858 077B D718 1C05 89EB  7913 0031 F545 CC59 3F68
uid                  Duplicity Encryption and Signing Key <sysadmin@aegir.example.com>
sub   4096R/C8D9E041 2015-02-07

kurth@aegir:~#

Pay attention to the highlighted lines

  • Line 16
    Select 1 (RSA and RSA)
  • Line 18
    Choose 4096 bits here for maximum data security.
  • Line 26
    Choose 0 here to indicate the key should never expire.
  • Line 34/Line 35/Line 36
    Enter required information,like your name and admin email address
  • Line 41
    Enter a passphrase for the key.  You will need this and the key id later for the backupninja job below.
  • Line 52
    Notice the CC593F68? This is the gpg key id, used to identify the key in the backupninja job below

Confirm your keys have been generated and are in place by issuing the command sudo gpg -k. Your output should resemble the following;

kurth@aegir:~# sudo gpg -k
/root/.gnupg/pubring.gpg
------------------------
pub   4096R/CC593F68 2015-02-07
uid                  Duplicity Encryption and Signing Key <sysadmin@aegir.example.com>
sub   4096R/C8D9E041 2015-02-07

kurth@aegir:~# 

Daily Backupninja Script

Go ahead and create the file /etc/backup.d/20-aegir-daily.sh containing the following;

#!/bin/sh
when = everyday at 03:30

DUPLICITY=/usr/bin/duplicity
BACKUP_DIR="/var/aegir/backups/tmp"

DUP_ENCRYPTKEY='CC593F68'
DUP_SIGNKEY='CC593F68'
export PASSPHRASE='<PASSWD>'
export FTP_PASSWORD='<PASSWD>'

DUP_FTP_TARGET='ftp://<FTP_USER>@<REMOTE BACKUP SERVER>/daily'

SITES=`cat /srv/etc/aegir-backup-sites.txt`

umask 007

for SITE in $SITES; do
        info "Backing Up $SITE"

        rm $BACKUP_DIR/$SITE/*.*

        BACKUP_FILE=$SITE-daily-`date +%m-%d-%y`.tar.gz

        sudo -u aegir mkdir -p $BACKUP_DIR/$SITE
        sudo -u aegir drush -q @${SITE} archive-backup --overwrite --destination=$BACKUP_DIR/$SITE/$BACKUP_FILE

        info "Uploading $SITE ($BACKUP_FILE)"
        $DUPLICITY --encrypt-key $DUP_ENCRYPTKEY --sign-key $DUP_SIGNKEY $BACKUP_DIR/$SITE $DUP_FTP_TARGET/$SITE

        info "Cleaning Up"
        rm $BACKUP_DIR/$SITE/*
        $DUPLICITY remove-all-but-n-full 7 --force $DUP_FTP_TARGET/$SITE
done

Pay attention to the highlighted lines

  • Line 2
    Change this to the time you want the job to run at.
  • Line 5
    This is the temporary location of your drush backups.  This volume should be large enough to contain a full copy of your largest site.
  • Line 7/Line 8
    The gpg key id of the key(s) to be used for encrypting and signing backup volumes
  • Line 9/Line 10
    The passphrase of the above specified keys.
  • Line 12
    Set your FTP information username and host here.
  • Line 14
    This is a simple list of hosted hostnames.  This file is used as input to the script as to which sites to backup.  This file consists of a single hostname on a single line.
  • Line 29
    This line preforms the duplicity backup. You may need to change options or portions of this command to match your environment.
  • Line 33
    Here we remove backups that are older then seven days. Adjust to taste.

Once you have the above in place, you will need to chmod the script to be executable. Something like the following will work;
chmod 600 /etc/backup.d/20-aegir-daily.sh

Testing Everything Out

With the gpg keys and backup script in place, it’s time to test Backupninja to ensure the job will execute as intended.

Use the following command

backupninja --now -d --run /etc/backup.d/20-aegir-daily.sh

Assuming everything is correct and working, you should see a line similar to the following upon completion of the backup job

Info: FINISHED: 1 actions run. 0 fatal. 0 error. 0 warning.

Leave a Reply

Your email address will not be published. Required fields are marked *