Hetzner - DokuWiki

BackupService/en

Inhaltsverzeichnis

About the Backup Service

Backup Service is a new feature for our storage products. It can be used with storage boxes and backup spaces. With the Backup Service it is easy to run automated file system backups. The service consists of 3 essential elements:

  • the web panel:
    The user interface to manage backup plans and access the stored data
  • the backup software:
    The application that executes the backups on the client servers. borgbackup
  • the backup agent:
    The backup agent is the broker between the backup software and the webpanel. The backup agent is installed on the client server and receives settings or commands from the web panel. The backup agent manages the implementation of the backup software and sends backup statistics to the web panel.

Disclaimer

Please note you are still responsible for your data. Hetzner Online provides no guarantees regarding possible data loss. The data is not mirrored to other servers. Please also see points 4.1 and 4.2 of our Terms and conditions: https://www.hetzner.com/rechtliches/agb

Features

  • Encryption:
    All backup data gets encrypted on the client server before it is transferred to the backup server. A password [BORG_PASSPHRASE] is required to decrypt the data on the backup server. This password won't be saved on the server!
  • Automation:
    The backups can be scheduled with flexible time plans. After you set them up, backups will run automatically.
  • Easy:
    Backup Service has an easy to use web interface. Its also easy to access your backup data and make it available on your storage device for direct access.

Compatibility

Supported operation systems:

  • Debian 9
  • Ubuntu 16.04

Python Versions:

  • >= Python 3.5

Configuration

Activation

You can activate the backup service via your Robot account. At the storage box / backup space configuration dialog, there is an option to de/activate the backup service. In order to perform backups you also have to activate the option SSH.

Access

The URL for the Backup Service web panel depends whether it is used for a storage box or a backup space.

Storage Box:

https://webpanel-<username>.your-storagebox.de

Backup Space:

https://webpanel-<username>.your-backup.de

To log in, you will need your storage box or backup space login details.

Create server

The backup service can be used to back up multiple servers. For every server, it will create a separate backup repository on the storage box or on the backup space. To create a server the following information is required:

  • Name:
    It is important to name the servers to get a relation between servers and backups
  • Backup password:
    The password used to encrypt the rbackup repository.
    CAUTION: WITHOUT THIS PASSWORD IT IS NOT POSSIBLE TO ACCESS THE BACKUP DATA
  • SSH_Key:
    The backups are transferred via SSH. To enable automatic backups, you need to store a public SSH key on the storage box or the backup space. It's important that this public key is not protected with a passphrase!

Whenever a server is created using the web panel, a unique server ID is assigned to the server. This server ID is required to setup the backup agent.

Backup agent setup

After the server is configured on the web panel, the backup agent needs to be configured on the client server.

Installation

The backup agent and its dependencies are available on a mirror (backup-packages-mirror.hetzner.de)

  • Enable the mirror:
wget https://backup-packages-mirror.hetzner.de/stretch.list -O /etc/apt/sources.list.d/backup-packages-mirror.list

or

wget https://backup-packages-mirror.hetzner.de/xenial.list -O /etc/apt/sources.list.d/backup-packages-mirror.list


  • Add GPG key to apt keychain:
wget -O - https://backup-packages-mirror.hetzner.de/backupservice.hetzner.gpg.key | apt-key add -
  • Update the APT repository and install the backup agent
apt-get update && apt-get install python3-hetznerbackupagent
  • If the installation fails with this error:
E: The method driver /usr/lib/apt/methods/https could not be found.

You need to install the transport-https libs:

apt-get install apt-transport-https

Configuration

After the installation, the backup agent can be started with backup_agent.

backup_agent --help
 usage: backup_agent [-h] [-r config_id] [-v] [-i] [-s [help-PARAM]] [-t] [-c]
  [-d]
 Hetzner Backup Agent
 optional arguments:
  -h, --help Shows this help message and exit
  -r config_id, --run config_id
  Triggers a backup with a config id.
  -v, --version Displays version information
  -i, --info Displays application information
  -s [help-PARAM], --setup [help-PARAM]
  Starts the configuration of the application. More
  information with "-setup help-all"
  -t, --test_ssh Checks if an SSH connection can be established.
  -c, --clean Removes all files associated with the backup agent,
  including conf-, log-, and systemd files
  -d, --debug Sets the loglevel to debug

The setup can be started with backup_agent --setup. The setup wizard will ask for the following information:

  • FQDN of the backup server:
    The same URL as is used for the web panel.
  • Username:
    The username for the storage box or backup space account.
  • Server ID:
    The server ID that was created by the web panel.
  • API Key:
    The key for authentication on the web panel. Will be provided by the web panel.
  • IP Address:
    The public ip address of your server.
  • Backup Passphrase:
    The passphrase that was used when the server was created on the web panel.
  • SSH key path:
    Path to the private SSH key. This path needs to lead to the counterpart key for the public key that was provided on the web panel.

Whenever a server is created on the web panel, all information needed to set up the backup agent will be displayed.

After the setup, the backup agent will be registered as a systemd unit and will then be started. Backups are triggered by systemd timers. The following systemd units will be created:

  • hetzner.backup_agent.service:
    Systemd unit for the backup agent process; ensures that the backup agent process is always running. This is necessary to send configuration parameters or commands from the web panel to the backup agent. When all backup plans are configured, it is possible to deactivate this unit.
  • hetzner.backup_timer_x.timer:
    For every backup plan, there is a timer. The timer schedules the backup's execution.
  • hetzner.backup_runner_x.service:
    This unit gets called by the associated timer and contains the command to execute a backup.

Further files created by the backup agent:

  • /etc/BackupAgent/agent_conf.json:
    The backup agent configuration file.
  • /var/lib/BackupAgent/backuptrack.json:
    Contains a list of all running backups with its process ID.
  • /var/log/hetzner.backup_agent.log:
    Log file for the backup agent process.
  • /var/log/hetzner.backup_agent_runner_x.log:
    Log file for every individual backup process

Tips for using the Backup agent

  • Usage behind a firewall: The Backup agent opens a port (default: 45111) on your system.

This port is used by the webpanel to send configuration data.
Please take care this port and the backupserver`s ip does not get blocked by your firewall (internal / Robot).

  • Start / Stop: The Backup agent is managed by systemd. So it gets started automatically at boot time and restarted after application failures. You can start or stop the systemd unit with the following commands:
    • systemctl start hetzner.backup_agent.service
    • systemctl stop hetzner.backup_agent.service
  • Manual starting: If you don't want systemd to manage your Backup agent you can disable it and start the Backup agent manually.
    backup_agent -d

Create Backupplan

Multiple backup plans can be configured for for every server. A backup plan describes the task for how to execute a backup. The following information is necessary for a backup plan:

  • Name:
    A unique name to distinguish multiple backup plans
  • File selection:
    • Include:
      Selection of all files and directories that should be included in the backup.
    • Exclude:
      Selection of all files and directories that should be excluded from the backup.
    • Hint:
      If the complete filesystem should be backed up, the following directories should be excluded:
      /dev, /proc, /sys, /var/run, /run, /lost+found, /mnt, /var/lib/lxcfs
  • Timeplan:
    Defines the time when a backup is executed. It is possible to select multiple weekdays, multiple hours, and minutes in 15 minute steps.

Whenever a backup plan is saved, it will be sent automatically to the backup agent.

Backup trigger

If the configuration is successful a backup can be triggered manually. The backup plan options dialogue has a trigger button to allow you to perform a manual trigger.

Backups

Backup details

Statistics will be saved for every backup, so it is easy to evaluate if backups are executed as expected.

  • Status:
    Indicates if a backup was successful.
  • Backup/Repository size:
    Shows the size of a backup or the size of the complete repository.
  • System usage:
    Shows the client servers system usage while a backup was executed. This is helpful to see any relation between system issues and backup executions.
  • Files count:
    Shows how many files were stored in the backup
  • Execution time:
    Duration and execution times of backups

Access backup data

To access the backup data, you need the repository passphrase. After you enter the password, all stored files and directories will be listed on the web panel. For large backups, this process may take a while. There 3 ways to get direct access to the files:

  • Extract to Storagebox / backup Space:
    The selected files or directories will be restored on the storage box or the backup space. In a "recovery" directory, the complete directory structure will be restored. If webdav is enabled, the files can be downloaded directly.
  • Extract to Storagebox / backup Space as archive:
    Similar to the previous option, except that only one single TAR-file will be stored on the storage box or the backup space.
  • View file:
    Text files and some image file formats can be opened in the web panel.
    • Text files:
      Syntax highlighting is available for many file formats.
    • Image files:
      Currently supported image formats: jpg, gif, png.

Operation modes

Daemon mode

By default the backup agent is executed in daemon mode. That means the application is permanently in execution. So the backup agent can receive messages from the webpanel all the time. That enables the user to update his backup plans or trigger backups at any time. However, the disadvantage of this operation mode is that the backup agent will keep a port open all the time. You should be aware that open ports can be a potential security risk.

On demand mode

The backup agent does not need be executed permanently for you to execute backups and see the backup results and statistics. After you have successfully configured the backup plans, you can stop the backup agent. At this time, systemd timers are already configured and will take care of the backup execution. To execute the backup agent in the on demand mode, all you need to do is to stop the backup agent after you have successfully configured your backup plans.

systemctl stop hetzner.backup_agent.service

When the agent is shut down, it won't listen anymore on a port and so it won't be possible to transmit new configurations or trigger backups manually from the webpanel. If you want to re-enable the agent, all you need to do is start the systemd unit.

systemctl start hetzner.backup_agent.service

CAUTION: After a system reboot, the backup agent will be started again in daemon mode.



© 2018. Hetzner Online GmbH. Alle Rechte vorbehalten.