Hetzner - DokuWiki

(small stylistic changes to rest of article)
(Automate backups with Cron)
Zeile 92: Zeile 92:
First, create a script which will execute the backups. This could look like the following script (comments currently in German) and be under '/usr/local/bin/backup.sh'.
First, create a script which will execute the backups. This could look like the following script (comments currently in German) and be under '/usr/local/bin/backup.sh'.
     #!/usr/bin/env bash
     #!/usr/bin/env bash
Zeile 162: Zeile 163:
     echo "###### Backup beendet: $(date) ######"
     echo "###### Backup beendet: $(date) ######"
Now test the script before you create the cronjob.
Now test the script before you create the cronjob.

Version vom 4. Dezember 2017, 08:54 Uhr

BorgBackup (short: Borg) is a deduplicating backup program. Compression and authenticated encryption are also supported as options.

Borg's main goal is to provide an efficient and secure backup solution. Thanks to deduplication, the backup process with Borg is very fast and makes Borg very interesting for daily backups. You may notice that Borg is significantly quicker than some other methods, depending on the amount of data and the number of changes you need to back up. With Borg, all data is already encrypted on the client side, which makes Borg a good choice for hosted systems.

More information about BorgBackup can be found at https://borgbackup.readthedocs.io/



There are three ways to install Borg.

  • Distribution package
  • Standalone binary
  • From source

In the Borg Documentation you will find very detailed descriptions of the different methods. That's why we do not go into detail here.

> For compatibility reasons, please use a current version of Borg! (> = 1.0.9)

Workflow with Borg

0. Activate Borg and configure your storage box

For Borg to be enabled on your storage box, you must first enable the service on the Robot webinterface. To do this, go to the settings page of your storage box on the Robot webinterface and click "activate" on "Borg support".

Since SSH is not available on the storage box, you must use SFTP or something similar for this step.

For Borg, you can use password authentication, but authentication via the public key is recommended. This is especially recommended if you want to automate the backups with cronjobs.

Create the folder '.ssh' in your storage box and store the file 'authorized_keys' in it. This must contain your public key:

   ssh-rsa AAAAB3NzaC1yc2EAAAA.......rdj7eitNUjlIV8ovvAH/6SAsKD6

Set the permissions for the '.ssh' folder to '0700' and for the 'authorized_keys' to '0600'.

Now you have to create the directory for the backup repository in the storage box. For example, create a folder 'backups', and below that, a folder 'server1'. The folder 'server1' will then be initialized as a Borg repository in the next step. Under 'backups' you could then create further directories for other servers you want to back up.


1. Initialize Borg repository

If you are using an SSH key, and this is not the default key, you have the option to specify the desired key using the environment variable BORG_RSH. You can specify the SSH command that Borg should use. The standard would be just 'ssh'.

   $ export BORG_RSH='ssh -i /home/userXY/.ssh/id_ed25519'

When initializing Borg, you will be prompted for a password for your repository. Only with this password can you access the repository in the future. It is therefore required for every read or write operation on the repository. You must be able to remember the password well because it cannot be restored! To avoid having to enter the password every time Borg calls, you can optionally set the environment variable BORG_PASSPHRASE.

   export BORG_PASSPHRASE="top_secret_passphrase"

First, you need to initialize the Borg repository. The repository is nothing more than a folder on your storage box that Borg provides with some basic structures. All backups are stored in this folder.

The following command initializes the / backups / server1 folder on your storage box.

   $ borg init --encryption=repokey

2. Create first backup

For example, use the following command to back up the src and transfer folders from your home directory to the repository on your storage box. You must give each backup a unique name. A timestamp is useful for creating unique names.

   $ borg create ssh://u123456@u123456.your-storagebox.de:23/./backups/server1::2017_11_11_initial ~/src ~/built

You can call Borg create using many other options. You can do this, for example, to view the progress of a backup while it is processing or to see statistics about the backup once it is finished. In addition, you can specify exclude patterns and other things.

For more information, please visit the Borg create documentation.

3. Following (incremental) backups

The follwing backups are identical to the first one. Thanks to deduplication, however, they are much faster and extremely memory-efficient, since they are only incremental.

You only need to adjust the name of the backup during the follow-up backup. Remember, you must use unique names as mentioned above.

Just use the '--stats' option on the next backup to see how efficient it is.

   $ borg create --stats ssh://u123456@u123456.your-storagebox.de:23/./backups/server1::2017_11_12 ~/src ~/built

4. More Borg commands including List archives, restore backups

The Borg documentation provides a very detailed description of all Borg commands.

It is best to start with a look at the quickstart section and then dive into the usage section to get into the details.

The documentation provides many examples of listing archives or restoring backups. It is also possible, for example, to display diffs between backups or to delete old backups to recover storage space.

Automate backups with Cron

Create a directory for the log file.

   $ mkdir -p /var/log/borg

First, create a script which will execute the backups. This could look like the following script (comments currently in German) and be under '/usr/local/bin/backup.sh'.

    #!/usr/bin/env bash

    ## Setzten von Umgebungsvariablen

    ## falls nicht der Standard SSH Key verwendet wird können
    ## Sie hier den Pfad zu Ihrem private Key angeben
    # export BORG_RSH="ssh -i /home/userXY/.ssh/id_ed25519"

    ## Damit das Passwort vom Repository nicht eingegeben werden muss
    ## kann es in der Umgepungsvariable gesetzt werden
    # export BORG_PASSPHRASE="top_secret_passphrase"

    ## Setzten von Variablen


    ## Hinweis: Für die Verwendung mit einem Backup-Account muss
    ## 'your-storagebox.de' in 'your-backup.de' geändert werden.


    ## Ausgabe in Logdatei schreiben

    exec > >(tee -i ${LOG})
    exec 2>&1

    echo "###### Backup gestartet: $(date) ######"

    ## An dieser Stelle können verschiedene Aufgaben vor der
    ## Synchronisierung der Dateien ausgeführt werden, wie z.B.
    ## - Liste der installierten Software erstellen
    ## - Datenbank Dump erstellen

    ## Dateien mit Repository synchronisieren
    ## Gesichert werden hier beispielsweise die Ordner root, etc,
    ## var/www und home
    ## Ausserdem finden Sie hier gleich noch eine Liste Excludes,
    ## die in kein Backup sollten uns somit per default ausgeschlossen
    ## werden.

    echo "Synchronisiere Dateien ..."
    borg create -v --stats                                  \
        $REPOSITORY::'{now:%Y-%m-%d_%H:%M}'                 \
        /root                                               \
        /etc                                                \
        /var/www                                            \
        /home                                               \
        --exclude /dev                                      \
        --exclude /proc                                     \
        --exclude /sys                                      \
        --exclude /var/run                                  \
        --exclude /run                                      \
        --exclude /lost+found                               \
        --exclude /mnt                                      \
        --exclude /var/lib/lxcfs

    echo "###### Backup beendet: $(date) ######"

Now test the script before you create the cronjob.

   $ chmod u+x /usr/local/bin/backup.sh
   $ /usr/local/bin/backup.sh

If everything works fine, you can now run the script as a cronjob. Open crontab as root:

   crontab -e

And add the following line to run a daily backup at 00:00.

0 0 * * * /usr/local/bin/backup.sh> / dev / null 2> & 1


Full system backup

If you want to backup the entire system on your Linux server, you should remember that not all files and folders belong in a backup. Some should be excluded by default.

For this, the create command has an '- exclude' option or you can specify an exclude file. The usage is described in detail in the Borg create documentation.

Here is an example call to 'borg create' for a backup of the complete system:

  borg create -v --stats                                  \
      $REPOSITORY::'{now:%Y-%m-%d_%H:%M}'                 \
      --exclude /dev                                      \
      --exclude /proc                                     \
      --exclude /sys                                      \
      --exclude /var/run                                  \
      --exclude /run                                      \
      --exclude /lost+found                               \
      --exclude /mnt                                      \
      --exclude /var/lib/lxcfs

Deduplication and reliability

Since BorgBackup uses duplication, you can make backups very quickly and without using much storage.

But you also have to be aware that each file is saved exactly once. Should a file be damaged by a disk failure, for example, this file will be corrupted in all following backups.

Therefore, it is best practice to store very important data in more than one repository!

© 2018. Hetzner Online GmbH. Alle Rechte vorbehalten.