Hetzner - DokuWiki

BorgBackup/en
(small stylistic changes to beginning of article)
Zeile 3: Zeile 3:
 
BorgBackup (short: Borg) is a deduplicating backup program. Compression and authenticated encryption are also supported as options.
 
BorgBackup (short: Borg) is a deduplicating backup program. Compression and authenticated encryption are also supported as options.
  
Borg's main goal is to provide an efficient and secure backup solution. Thanks to deduplication, the backup process with Borg is very fast and makes Borg very interesting for daily backups. Depending on the amount of data and the number of changes, significantly shorter time intervals for backups are also conceivable. All data is already encrypted on the client side, which makes Borg interesting for use on hosted systems.
+
Borg's main goal is to provide an efficient and secure backup solution. Thanks to deduplication, the backup process with Borg is very fast and makes Borg very interesting for daily backups. You may notice that Borg is significantly quicker than some other methods, depending on the amount of data and the number of changes you need to back up. With Borg, all data is already encrypted on the client side, which makes Borg a good choice for hosted systems.
  
 
More information about BorgBackup can be found at https://borgbackup.readthedocs.io/
 
More information about BorgBackup can be found at https://borgbackup.readthedocs.io/
Zeile 11: Zeile 11:
 
There are three ways to install Borg.
 
There are three ways to install Borg.
  
* Distribution Package
+
* Distribution package
* Standalone Binary
+
* Standalone binary
* From Source
+
* From source
  
In the [https://borgbackup.readthedocs.io/en/stable/installation.html Borg Documentation] you will find very detailed descriptions of the different possibilities. That's why we do not go into this at this point.
+
In the [https://borgbackup.readthedocs.io/en/stable/installation.html Borg Documentation] you will find very detailed descriptions of the different methods. That's why we do not go into detail here.
  
 
> For compatibility reasons, please use a current version of Borg! (> = 1.0.9)
 
> For compatibility reasons, please use a current version of Borg! (> = 1.0.9)
Zeile 21: Zeile 21:
 
== Workflow with Borg ==
 
== Workflow with Borg ==
  
=== 0. Activation of Borg and configuration of your storage box ===
+
=== 0. Activate Borg and configure your storage box ===
  
For Borg to be enabled on your storage box, you must first enable the service in the Robot webinterface. To do this, go to the settings page of your storage box in the Robot webinterface and click activate on "Borg support".
+
For Borg to be enabled on your storage box, you must first enable the service on the Robot webinterface. To do this, go to the settings page of your storage box on the Robot webinterface and click "activate" on "Borg support".
  
Since SSH is not available on the storage box, you must use for example SFTP for this step.
+
Since SSH is not available on the storage box, you must use SFTP or something similar for this step.
  
 
For Borg, you can use password authentication, but authentication via the public key is recommended. This is especially recommended if you want to automate the backups with cronjobs.
 
For Borg, you can use password authentication, but authentication via the public key is recommended. This is especially recommended if you want to automate the backups with cronjobs.
Zeile 35: Zeile 35:
 
Set the permissions for the '.ssh' folder to '0700' and for the 'authorized_keys' to '0600'.
 
Set the permissions for the '.ssh' folder to '0700' and for the 'authorized_keys' to '0600'.
  
Now you have to create the directory for the backup repository in the storage box. For example, create a folder 'backups' and below that a folder 'server1'. The folder 'server1' will then be initialized as a Borg repository in the next step. Under Backups you could then create further directories for other servers to be backed up.
+
Now you have to create the directory for the backup repository in the storage box. For example, create a folder 'backups', and below that, a folder 'server1'. The folder 'server1' will then be initialized as a Borg repository in the next step. Under "backups" you could then create further directories for other servers to be backed up.
  
 
     /backups/server1
 
     /backups/server1
Zeile 93: Zeile 93:
  
 
     #!/usr/bin/env bash
 
     #!/usr/bin/env bash
+
 
 
     ##
 
     ##
 
     ## Setzten von Umgebungsvariablen
 
     ## Setzten von Umgebungsvariablen
 
     ##
 
     ##
+
 
 
     ## falls nicht der Standard SSH Key verwendet wird können
 
     ## falls nicht der Standard SSH Key verwendet wird können
 
     ## Sie hier den Pfad zu Ihrem private Key angeben
 
     ## Sie hier den Pfad zu Ihrem private Key angeben
 
     # export BORG_RSH="ssh -i /home/userXY/.ssh/id_ed25519"
 
     # export BORG_RSH="ssh -i /home/userXY/.ssh/id_ed25519"
+
 
 
     ## Damit das Passwort vom Repository nicht eingegeben werden muss
 
     ## Damit das Passwort vom Repository nicht eingegeben werden muss
 
     ## kann es in der Umgepungsvariable gesetzt werden
 
     ## kann es in der Umgepungsvariable gesetzt werden
 
     # export BORG_PASSPHRASE="top_secret_passphrase"
 
     # export BORG_PASSPHRASE="top_secret_passphrase"
+
 
 
     ##
 
     ##
 
     ## Setzten von Variablen
 
     ## Setzten von Variablen
 
     ##
 
     ##
+
 
 
     LOG="/var/log/borg/backup.log"
 
     LOG="/var/log/borg/backup.log"
 
     BACKUP_USER="u602"
 
     BACKUP_USER="u602"
 
     REPOSITORY_DIR="server1"
 
     REPOSITORY_DIR="server1"
+
 
 
     ## Hinweis: Für die Verwendung mit einem Backup-Account muss
 
     ## Hinweis: Für die Verwendung mit einem Backup-Account muss
 
     ## 'your-storagebox.de' in 'your-backup.de' geändert werden.
 
     ## 'your-storagebox.de' in 'your-backup.de' geändert werden.
+
 
 
     REPOSITORY="ssh://${BACKUP_USER}@${BACKUP_USER}.your-storagebox.de:23/./backups/${REPOSITORY_DIR}"
 
     REPOSITORY="ssh://${BACKUP_USER}@${BACKUP_USER}.your-storagebox.de:23/./backups/${REPOSITORY_DIR}"
+
 
 
     ##
 
     ##
 
     ## Ausgabe in Logdatei schreiben
 
     ## Ausgabe in Logdatei schreiben
 
     ##
 
     ##
+
 
 
     exec > >(tee -i ${LOG})
 
     exec > >(tee -i ${LOG})
 
     exec 2>&1
 
     exec 2>&1
+
 
 
     echo "###### Backup gestartet: $(date) ######"
 
     echo "###### Backup gestartet: $(date) ######"
+
 
 
     ##
 
     ##
 
     ## An dieser Stelle können verschiedene Aufgaben vor der
 
     ## An dieser Stelle können verschiedene Aufgaben vor der
Zeile 135: Zeile 135:
 
     ## - Datenbank Dump erstellen
 
     ## - Datenbank Dump erstellen
 
     ##
 
     ##
+
 
 
     ##
 
     ##
 
     ## Dateien mit Repository synchronisieren
 
     ## Dateien mit Repository synchronisieren
Zeile 144: Zeile 144:
 
     ## werden.
 
     ## werden.
 
     ##
 
     ##
+
 
 
     echo "Synchronisiere Dateien ..."
 
     echo "Synchronisiere Dateien ..."
 
     borg create -v --stats                                  \
 
     borg create -v --stats                                  \
Zeile 160: Zeile 160:
 
         --exclude /mnt                                      \
 
         --exclude /mnt                                      \
 
         --exclude /var/lib/lxcfs
 
         --exclude /var/lib/lxcfs
+
 
 
     echo "###### Backup beendet: $(date) ######"
 
     echo "###### Backup beendet: $(date) ######"
  

Version vom 1. Dezember 2017, 14:42 Uhr

BorgBackup (short: Borg) is a deduplicating backup program. Compression and authenticated encryption are also supported as options.

Borg's main goal is to provide an efficient and secure backup solution. Thanks to deduplication, the backup process with Borg is very fast and makes Borg very interesting for daily backups. You may notice that Borg is significantly quicker than some other methods, depending on the amount of data and the number of changes you need to back up. With Borg, all data is already encrypted on the client side, which makes Borg a good choice for hosted systems.

More information about BorgBackup can be found at https://borgbackup.readthedocs.io/

Inhaltsverzeichnis

Installation

There are three ways to install Borg.

  • Distribution package
  • Standalone binary
  • From source

In the Borg Documentation you will find very detailed descriptions of the different methods. That's why we do not go into detail here.

> For compatibility reasons, please use a current version of Borg! (> = 1.0.9)

Workflow with Borg

0. Activate Borg and configure your storage box

For Borg to be enabled on your storage box, you must first enable the service on the Robot webinterface. To do this, go to the settings page of your storage box on the Robot webinterface and click "activate" on "Borg support".

Since SSH is not available on the storage box, you must use SFTP or something similar for this step.

For Borg, you can use password authentication, but authentication via the public key is recommended. This is especially recommended if you want to automate the backups with cronjobs.

Create the folder '.ssh' in your storage box and store the file 'authorized_keys' in it. This must contain your public key:

   ssh-rsa AAAAB3NzaC1yc2EAAAA.......rdj7eitNUjlIV8ovvAH/6SAsKD6

Set the permissions for the '.ssh' folder to '0700' and for the 'authorized_keys' to '0600'.

Now you have to create the directory for the backup repository in the storage box. For example, create a folder 'backups', and below that, a folder 'server1'. The folder 'server1' will then be initialized as a Borg repository in the next step. Under "backups" you could then create further directories for other servers to be backed up.

   /backups/server1

1. Initialize Borg repository

If you are using an SSH key, and this is not the default key, you have the option to specify the desired key using the environment variable BORG_RSH. You can specify the SSH command that Borg should use. The standard would be just 'ssh'.

   $ export BORG_RSH='ssh -i /home/userXY/.ssh/id_ed25519'

When initializing Borg, you will be prompted for a password for your repository. Only with this password can the repository be accessed in the future. It is therefore required for every read or write operation on the repository. The password should be remembered well, because it can not be restored! To avoid having to enter the password every time Borg calls, you can optionally set the environment variable BORG_PASSPHRASE.

   export BORG_PASSPHRASE="top_secret_passphrase"

First, the Borg repository needs to be initialized. The repository is nothing more than a folder on your storage box that Borg provides with some basic structures. Therein all backups are stored.

The following command initializes the / backups / server1 folder on your storage box.

   $ borg init --encryption=repokey
   ssh://u123456@u123456.your-storagebox.de:23/./backups/server1

2. Create first backup

For example, use the following command to back up the src and built folders from your homedirectory to the repository on your storage box. Each backup must be given a unique name. A timestamp offers for this.

   $ borg create ssh://u123456@u123456.your-storagebox.de:23/./backups/server1::2017_11_11_initial ~/src ~/built

Borg create can be called with many more options. For example, the progress during or a statistic can be displayed at the end of the backup. In addition, exclude patterns and much more can be specified.

For more information, please visit the Borg create documentation.

3. Following (incremental) Backups

The follwing backups are identical to the first one. Thanks to deduplication, however, they are much faster and extremely memory-efficient, since they are only incremental.

Only the name of the backup has to be adjusted during the follow-up backup, as this must be clear as mentioned above.

Just use the '--stats' option on the next backup to see how efficient it is.

   $ borg create --stats ssh://u123456@u123456.your-storagebox.de:23/./backups/server1::2017_11_12 ~/src ~/built

4. More Borg commands like: List archives, restore backups

The Borg documentation provides a very detailed description of all Borg commands.

It is best to start with a look in the quickstart section and then dive into the usage section to get into the details.

Here are many examples of listing archives or restoring backups. It is also possible, for example, to display diffs between backups or to delete old backups to recover storage space.

Automation of backups with Cron

Create a directory for the log file.

   $ mkdir -p /var/log/borg

First of all, a script has to be created, which executes the backups. This could look like the following script (comments currently in German) and be under '/usr/local/bin/backup.sh'.

   #!/usr/bin/env bash
   ##
   ## Setzten von Umgebungsvariablen
   ##
   ## falls nicht der Standard SSH Key verwendet wird können
   ## Sie hier den Pfad zu Ihrem private Key angeben
   # export BORG_RSH="ssh -i /home/userXY/.ssh/id_ed25519"
   ## Damit das Passwort vom Repository nicht eingegeben werden muss
   ## kann es in der Umgepungsvariable gesetzt werden
   # export BORG_PASSPHRASE="top_secret_passphrase"
   ##
   ## Setzten von Variablen
   ##
   LOG="/var/log/borg/backup.log"
   BACKUP_USER="u602"
   REPOSITORY_DIR="server1"
   ## Hinweis: Für die Verwendung mit einem Backup-Account muss
   ## 'your-storagebox.de' in 'your-backup.de' geändert werden.
   REPOSITORY="ssh://${BACKUP_USER}@${BACKUP_USER}.your-storagebox.de:23/./backups/${REPOSITORY_DIR}"
   ##
   ## Ausgabe in Logdatei schreiben
   ##
   exec > >(tee -i ${LOG})
   exec 2>&1
   echo "###### Backup gestartet: $(date) ######"
   ##
   ## An dieser Stelle können verschiedene Aufgaben vor der
   ## Synchronisierung der Dateien ausgeführt werden, wie z.B.
   ##
   ## - Liste der installierten Software erstellen
   ## - Datenbank Dump erstellen
   ##
   ##
   ## Dateien mit Repository synchronisieren
   ## Gesichert werden hier beispielsweise die Ordner root, etc,
   ## var/www und home
   ## Ausserdem finden Sie hier gleich noch eine Liste Excludes,
   ## die in kein Backup sollten uns somit per default ausgeschlossen
   ## werden.
   ##
   echo "Synchronisiere Dateien ..."
   borg create -v --stats                                  \
       $REPOSITORY::'{now:%Y-%m-%d_%H:%M}'                 \
       /root                                               \
       /etc                                                \
       /var/www                                            \
       /home                                               \
       --exclude /dev                                      \
       --exclude /proc                                     \
       --exclude /sys                                      \
       --exclude /var/run                                  \
       --exclude /run                                      \
       --exclude /lost+found                               \
       --exclude /mnt                                      \
       --exclude /var/lib/lxcfs
   echo "###### Backup beendet: $(date) ######"

Now the script should be tested before the cronjob is created.

   $ chmod u+x /usr/local/bin/backup.sh
   $ /usr/local/bin/backup.sh

If everything works fine, you can now run the script as a cronjob. Open crontab as root:

   crontab -e

And add the following line to run a daily backup at 00:00.

0 0 * * * /usr/local/bin/backup.sh> / dev / null 2> & 1

Hints

Full System Backup

If you want to backup the entire system on your Linux server, you should remember that not all files and folders belong in a backup. Some should be excluded by default.

For this, the create command has an '- exclude' option or an exclude file can be specified. The usage is described in detail in the Borg create documentation.

Here is an example call to 'borg create' for a backup of the complete system:

  borg create -v --stats                                  \
      $REPOSITORY::'{now:%Y-%m-%d_%H:%M}'                 \
      /
      --exclude /dev                                      \
      --exclude /proc                                     \
      --exclude /sys                                      \
      --exclude /var/run                                  \
      --exclude /run                                      \
      --exclude /lost+found                               \
      --exclude /mnt                                      \
      --exclude /var/lib/lxcfs

Deduplication and Reliability

Deduplication provides Borg Backups with very efficient memory usage and high speed.

But you also have to be aware that each file is saved exactly once. Should a file be e.g. damaged by a disk failure, this is corrupted in all backups.

Therefore, it is best practice to store very important data in more than one repository!



© 2018. Hetzner Online GmbH. Alle Rechte vorbehalten.