Hetzner - DokuWiki

BackupService/en

Inhaltsverzeichnis

About the Backup Service

Backup Service is a new feature for our storagebox products. It can be used with storage boxes and backup spaces. With the Backup Service it is easy to run automated file system backups. The service consists of 3 essential elements:

  • the web panel:
    The user interface to manage backup plans and access the stored data
  • the backup software:
    The application that executes the backups on the client servers. borgbackup
  • the backup agent:
    The backup agent is the broker between the backup software and the webpanel. The backup agent is installed on the client server and receives settings or commands from the web panel. The backup agent manages the implementation of the backup software and sends backup statistics to the web panel.

Beta Stage

This new service is still in beta testing, which we tentitively plan to end on 31 July 2017. With this beta, we want to see if there is a demand for a managed backup service. We will continue to work on improving usability and will add new features as the beta continues, and if all goes well, we will keep doing this afterwards. We invite our customers to test this service and give us feedback on our discussion forum.

Features

  • Encryption:
    All backup data gets encrypted on the client server before it is transferred to the backup server. A password [BORG_PASSPHRASE] is required to decrypt the data on the backup server. This password won't be saved on the server!
  • Automation:
    The backups can be scheduled with flexible time plans. After you set them up, backups will run automatically.
  • Easy:
    Backup Service has an easy to use web interface. Its also easy to access your backup data and make it available on your storage device for direct access.

Compatibility

Supported operation systems:

  • Debian 9
  • Ubuntu 16.04

Python Versions:

  • >= Python 3.5

Configuration

Activation

At our beta start it is only possible to activate the backup service with our internal systems. If you want to participate our beta please send us a storage box request with a hint to activate the backup service.

Access

The URL for the Backup Service web panel depends whether it is used for a storage box or a backup space.

Storage Box:

https://webpanel-<username>.your-storagebox.de

Backup Space:

https://webpanel-<username>.your-backup.de

To log in, you will need your storage box or backup space login details.

Create server

The backup service can be used to back up multiple servers. For every server, it will create a separate backup repository on the storage box or on the backup space. To create a server the following information is required:

  • Name:
    It is important to name the servers to get a relation between servers and backups
  • Backup password:
    The password used to encrypt the rbackup repository.
    CAUTION: WITHOUT THIS PASSWORD IT IS NOT POSSIBLE TO ACCESS THE BACKUP DATA
  • SSH_Key:
    The backups are transferred via SSH. To enable automatic backups, you need to store a public SSH key on the storage box or the backup space. It's important that this public key is not protected with a passphrase!

Whenever a server is created using the web panel, a unique server ID is assigned to the server. This server ID is required to setup the backup agent.

Backup agent setup

After the server is configured on the web panel, the backup agent needs to be configured on the client server.

Installation

The backup agent and its dependencies are available on a mirror (backup-packages-mirror.hetzner.de)

  • Enable the mirror:
wget https://backup-packages-mirror.hetzner.de/stretch.list -O /etc/apt/sources.list.d/backup-packages-mirror.list

or

wget https://backup-packages-mirror.hetzner.de/xenial.list -O /etc/apt/sources.list.d/backup-packages-mirror.list


  • Add GPG key to apt keychain:
wget -O - https://backup-packages-mirror.hetzner.de/backupservice.hetzner.gpg.key | apt-key add -
  • Update the APT repository and install the backup agent
apt-get update && apt-get install python3-hetznerbackupagent
  • If the installation fails with this error:
E: The method driver /usr/lib/apt/methods/https could not be found.

You need to install the transport-https libs:

apt-get install apt-transport-https

Configuration

After the installation, the backup agent can be started with backup_agent.

backup_agent --help
 usage: backup_agent [-h] [-r config_id] [-v] [-i] [-s [help-PARAM]] [-t] [-c]
  [-d]
 Hetzner Backup Agent
 optional arguments:
  -h, --help Shows this help message and exit
  -r config_id, --run config_id
  Triggers a backup with a config id.
  -v, --version Displays version information
  -i, --info Displays application information
  -s [help-PARAM], --setup [help-PARAM]
  Starts the configuration of the application. More
  information with "-setup help-all"
  -t, --test_ssh Checks if an SSH connection can be established.
  -c, --clean Removes all files associated with the backup agent,
  including conf-, log-, and systemd files
  -d, --debug Sets the loglevel to debug

The setup can be started with backup_agent --setup. The setup wizzard will ask for the following information:

  • FQDN of the backup server:
    The same URL as is used for the web panel.
  • Username:
    The username for the storage box or backup space account.
  • Server ID:
    The server ID that was created by the web panel.
  • API Key:
    The key for authentication on the web panel. Will be provided by the web panel.
  • IP Address:
    The public ip address of your server.
  • Backup Passphrase:
    The passphrase that was used when the server was created on the web panel.
  • SSH key path:
    Path to the private SSH key. This path needs to lead to the counterpart key for the public key that was provided on the web panel.

Whenever a server is created on the web panel, all information needed to set up the backup agent will be displayed.

After the setup, the backup agent will be registered as a systemd unit and will then be started. Backups are triggered by systemd timers. The following systemd units will be created:

  • hetzner.backup_agent.service:
    Systemd unit for the backup agent process; ensures that the backup agent process is always running. This is necessary to send configuration parameters or commands from the web panel to the backup agent. When all backup plans are configured, it is possible to deactivate this unit.
  • hetzner.backup_timer_x.timer:
    For every backup plan, there is a timer. The timer schedules the backup's execution.
  • hetzner.backup_runner_x.service:
    This unit gets called by the associated timer and contains the command to execute a backup.

Further files created by the backup agent:

  • /etc/BackupAgent/agent_conf.json:
    The backup agent configuration file.
  • /var/lib/BackupAgent/backuptrack.json:
    Contains a list of all running backups with its process ID.
  • /var/log/hetzner.backup_agent.log:
    Log file for the backup agent process.
  • /var/log/hetzner.backup_agent_runner_x.log:
    Log file for every individual backup process

Tips for using the Backup agent

  • Usage behind a firewall: The Backup agent opens a port (default: 45111) on your system.

This port is used by the webpanel to send configuration data.
Please take care this port and the backupserver`s ip does not get blocked by your firewall (internal / Robot).

  • Start / Stop: The Backup agent is managed by systemd. So it gets started automatically at boot time and restarted after application failures. You can start or stop the systemd unit with the following commands:
    • systemctl start hetzner.backup_agent.service
    • systemctl stop hetzner.backup_agent.service
  • Manual starting: If you don't want systemd to manage your Backup agent you can disable it and start the Backup agent manually.
    backup_agent -d

Create Backupplan

Multiple backup plans can be configured for for every server. A backup plan describes the task for how to execute a backup. The following information is necessary for a backup plan:

  • Name:
    A unique name to distinguish multiple backup plans
  • File selection:
    • Include:
      Selection of all files and directories that should be included in the backup.
    • Exclude:
      Selection of all files and directories that should be excluded from the backup.
    • Hint:
      If the complete filesystem should be backed up, the following directories should be excluded:
      /dev, /proc, /sys, /var/run, /run, /lost+found, /mnt, /var/lib/lxcfs
  • Time plan:
    Defines the time when a backup is executed. It is possible to select multiple weekdays, multiple hours, and minutes in 15 minute steps.

Whenever a backup plan is saved, it will be sent automatically to the backup agent.

Backup trigger

If the configuration is successful a backup can be triggered manually. The backup plan options dialogue has a trigger button to allow you to perform a manual trigger.

Backups

Backup details

Statistics will be saved for every backup, so it is easy to evaluate if backups are executed as expected.

  • Status:
    Indicates if a backup was successful.
  • Storage:
    Shows the size of a backup or the size of the complete repository.
  • Systemusage:
    Shows the client servers system usage while a backup was executed. This is helpful to see any relation between system issues and backup executions.
  • Filescount:
    Shows how many files where stored in the backup
  • Times:
    Duration and execution times of backups

Access backup data

To access the backup data, you need the repository passphrase. After you enter the password, all stored files and directories will be listed on the web panel. For large backups, this process may take a while. There 3 ways to get direct access to the files:

  • Extract to storage box / backup Space:
    The selected files or directories will be restored on the storage box or the backup space. In a "Recovery" directory, the complete directory structure will be restored. If webdav is enabled, the files can be downloaded directly.
  • Extract to storage box / backup Space as archive:
    Similar to the previous option, except that only one single TAR-file will be stored on the storage box or the backup space.
  • Show files:
    Text files and some image file formats can be opened in the web panel.
    • Text files:
      Syntax highlighting is available for many file formats.
    • Image files:
      Currently supported image formats: jpg, gif, png.

Operation modes

Daemon mode

By default the backup agent is executed in daemon mode. That means the application is permanently in execution. So the backup agent can receive messages from the webpanel all the time. That enables the user to update his backup plans or trigger backups at any time. However, the disadvantage of this operation mode is that the backup agent will keep a port open all the time. You should be aware that open ports can be a potential security risk.

On demand mode

The backup agent does not need be executed permanently for you to execute backups and see the backup results and statistics. After you have successfully configured the backup plans, you can stop the backup agent. At this time, systemd timers are already configured and will take care of the backup execution. To execute the backup agent in the on demand mode, all you need to do is to stop the backup agent after you have successfully configured your backup plans.

systemctl stop hetzner.backup_agent.service

When the agent is shut down, it won't listen anymore on a port and so it won't be possible to transmit new configrations or trigger backups manually from the webpanel. If you want to re-enable the agent, all you need to do is start the systemd unit.

systemctl start hetzner.backup_agent.service

CAUTION: After a system reboot, the backup agent will be started again in daemon mode.



© 2018. Hetzner Online GmbH. Alle Rechte vorbehalten.