Hetzner - DokuWiki

Docker/en

Inhaltsverzeichnis

Introduction

This article was adapted and translated from the original German version in May 2017. For that reason, you may see some German in the images. Also some of the links in the original version are not included in the English version.

This article, which was written by a client, provides a detailed description for how you can configure a Hetzner Online server with Docker. This article is not an official set of instructions from Hetzner Online. Please note that Hetzner Online does not provide technical support for Docker, but should you need assistance, you could contact a Docker expert, for example, the author of this article: christian.mueller@vr-worlds.de.

Selecting a provider

I wanted a change from the currecnt Amazon configuration, and wanted to pick a solution that would allow me to have root control over the server. After reviewing several offers, I decided to try out Hetzner since it is based in Germany and it made a good impression on me. Some other providers like 1&1 had similar offers. However, the server auction at Hetzner Online allowed be to rent a "gently used" server: http://www.serverbidding.com/?country=OTHER

Hetznerserverboerse.png

Here, I found a server that met all my needs. My past experiences with Hetzner Online have been generally positive. However, I want to make it clear that I did not do an analysis of the complete market. The size of the market is simply too huge for me to do that, even if I just consider what's available in Germany.]

Architecture: considering base operating system

The next decision I had to make was which OS to use. I quickly eliminated Windows as an option, since the monthly license fees are so high. Plus, I have been comfortable using Unix (AIX) and Linux systems since 1992.

In the end, I chose CentOS 7.3 [1] based on the following factors:

  • supported by Hetzner servers
  • available free of cost - no licensing fees
  • highly stable
  • active community
  • good enterprise support (because it's so similar to Red Hat Linux [2]

The only serious alternative for me would have been Debian. [3]

Auswahl.png

Basic setup for Docker: considerations

I had some base functions in mind for my website. I wanted the following need to be availble as a service:

  • Ghost [4] as a blog service
  • ownCloud [5] for file storage, especially to allow me to share files with partners and customers
  • GITLab [6] to allow me to develop software together with partners. This also has a basic Wiki and task planning feature, which allows me to coordinate project development
  • Shipyard [7] this lets me remotely administer a Docker container via a user interface.

Since I wanted to be able to access these services via a port (80 for HTTP or 443 for HTTPS), I used NGINX as a proxy.

As a database for Ghost and ownCloud, I used MariaDB [8].

In the diagram, you can see this setup more clearly:

Schaubild1.png

To make sure that everything works together properly, make sure to set up the components for this system in the following order:

  • Do base CentOS configuration and Docker installation
  • Configure MariaDB
  • Configure Ghost as a blog service and connect it to MariaDB
  • Install ownCloud and connect it to MariaDB
  • Install GitLab
  • Install NGINX and set up proxy routings
  • Configure Shipyard and set up the connection in NGINX
  • Finally, set up firewall using Hetzner Online's firewall tool to minimize the number of open ports

Before you start doing any configuration, you need to make sure that you have the following tools if you are working with Windows. (If you are using Linux as a client, you do not need to install any additional tools, since SSH and SCP are already in place.

  • Putty as a command line tool [9]
  • WinScp to upload and download files [10]

Base configuration for CentOS und Docker installation

After installing CentOS, you should then perform an update:

yum -y update

Then comes the Docker installation:

yum -y install docker docker-registry

To make Docker run automatically when you boot, you need to enable Docker Service:

systemctl enable docker.service
systemctl start docker.service

Then you can check if everything has been configured properly by performing a status query:

root@CentOS-73-64-minimal ~]# systemctl status docker.service docker.service - Docker Application Container Engine

  Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
  Active: active (running) since Fri 2017-03-31 22:14:56 CEST; 1 months 0 days ago
    Docs: http://docs.docker.com
Main PID: 13307 (dockerd-current)
  Memory: 107.5M
  CGroup: /system.slice/docker.service

Configuring MariaDB and learning basic Docker commands

Since you want Ghost and ownCloud to have access to a central database, you first need to set this database up as the first container.

Even though Docker will save you a lot of work, you will still need to make a few decisions. (I recommend looking at the helpful Docker Hub documentation [11] )

  • Which name do you want MariaDB to use? I used „unfortunately not perfect ghost-mysql“. (original in German: "„nicht perfekt leider ghost-mysql")
  • Which port to do want to use? I decided to use the standard, port 3306.
  • Where exactly do I want to sore data?
  • What is the root password?

You could decide, for example, to store data in Home. You can create a data directory for this by entering:

mkdir /home/data
mkdir /home/data/MariaDB

After I thought about the above questions, I chose the following configuration:

  • Name: ghost-mysql
  • Data directory sent to: /home/data/MariaDB/ folders
  • SQL password: XXXXXXX
  • Docker image to use: MariaDB

docker run --name ghost-mysql -v /home/data/MariaDB/:/var/lib/mysql:z -e MYSQL_ROOT_PASSWORD=XXXXXXX -d -p 3306:3306 MariaDB

By running "docker ps", you can then see whether or not the data bank is working properly:

[root@CentOS-73-64-minimal ~]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES d952d2b55a4e MariaDB "docker-entrypoint.sh" 3 weeks ago Up 3 weeks 0.0.0.0:3306->3306/tcp ghost-mysql

The directory has the relevant data for MariaDB:

450px

If you want to remove a container that is running, you can do that by executing "docker rm". If the contain continues to run, you can do a force delete with "-f":

Dockerrm.png

The command "docker start" allows you to start an existing container. Use "restart" to close and restart a container. And, naturally, use "docker stop" to close a container.

Dockerrestart.png

If the Docker image does not start, you may be able to see the mistake by not using the detach option "-d":

Detach.png

In this case, the error is a "chown" error. The option ":z" was missing. To mount the directory correctly, it should instead look like this:

Mount.png

You can see a list of installed images (meaning the basis for each container) by entering "docker images":

Images.png

Using this configuration, it is then possible to use HeidiSQL (https://www.heidisql.com/download.php) to access the database.

However, in this case (and I would recommend only doing this temporarily), make sure to add one firewall rule to the server using Hetzner's firewall tool; the rule should limit access to only your IP address.

Firewall1.png

Then, it should be possible to connect to the mySQL/MariaDB database:

Mysqlconnect.png

You can use the command "docker logs ghost-mysql" to view the container log:

Dockerlogs.png

Configuring Ghost as a blog service

The next step is to install Ghost on your server.

Here are also a few factors to consider:

  • Which port do you want to use? I used the standard port 2368 for this, which is not accessible from the Internet. (For testing, you can open the port, just like in MariaDB above).
  • The name should simply be "blog".
  • The location for storage should naturally also be in a directory for the data. For this case, I used "/home/data/Ghost".
  • The mySQL database should be mapped/linked to ghost-mysql.

The start for Docker should then look like this:

mkdir /home/data/Ghost
docker run --name blog -p 2368:2368 -v /home/data/Ghost/:/var/lib/ghost:z -d --link ghost-mysql:mysql ghost

I will not go into a lot of detail on the overall configuration in this article, since it should really be its own article. I will limit myself here to only the bare minimum, meaning that Ghost must be adapted to use MariaDB instead of using the interal SQlite database.

For this, you need to change the "config.js" setting. The easiest way to do this is using WinSCP. You can also use "vi" on your server, if you don't feel comfortable using WinSCP: (https://scotch.io/tutorials/getting-started-with-vim-an-interactive-guide)

Configjs.png

This is where you then need to enter the database configuration. For the hostname, use "mysql" since this is also defined this way in Docker. I used "blog_user" for User and assigned a password: (WARNING: The configuration must be done in "development".)

Editconfigjs.png

You still need to put the database in MariaDB and set up the User.

Then you can creat the database via HeidiSQL:

Createdbblog.png

And then enter the following in this dialog:

Dbcreateheidi.png

Then you can authorize the User to access the database:

Adduser1.png

Assign a password for this user and all necessary authorizations:

Userrights2.png

Then you must restart the blog so that the new configurations are implemented by config.js:

Restart.png

If everything has worked so far, then you should be able to access the blog via "http" and there should be serveral tables that are availble to you in MariaDB: (Use F5 to refresh, so that everything is also viewable in HeidiSQL!)

Displaydbheidi.png

Setting up ownCloud and GitLab

In theory, the set up for both of these services is very similar, as it was for Ghost.

For ownCloud, you also need to setup a user in MariaDB, while this is more simple in GitLab. Therefore, I will just summarize the most important parts.

Create a relevant database in MariaDB.

Showdb.png

I used the following command in ownCloud:

docker run --name owncloud -v /home/data/owncloud:/var/www/html:z -p 8082:80 --link ghost-mysql:mysql -d owncloud:8.1

You can use the same configuration for ownCloud that you used for MariaDB:

Configown.png

Then you should be able to also see the relevant tables in MariaDB:

Dbtables.png

And for GitLab, I used the following command:

docker run --detach --name gitlab --hostname git.vr-worlds.de --sysctl net.core.somaxconn=1024 --ulimit sigpending=62793 --ulimit nproc=131072 --ulimit nofile=60000 --ulimit core=0 --publish 8443:443 --publish 8083:80 --publish 8022:22 --publish 8060:8060 --restart always --volume /home/data/gitlab/config:/etc/gitlab:z --volume /home/data/gitlab/logs:/var/log/gitlab:z --volume /home/data/gitlab/data:/var/opt/gitlab:z --volume /etc/localtime:/etc/localtime gitlab/gitlab-ce

Setting up NGINX

Now here comes the exciting part of the aricle. Your NGINX should be configured in such a way that requests to port 80 are forwarded to the relevant Docker container.

The specified sub-domain will enable this to happen:

  • www.vr-worlds.de -> forwarded to blog
  • cloud.vr-worlds.de -> forwarded to ownCloud
  • git.vr-worlds.de -> forwarded to GitLab

To do this, you will need to point all sub-domains to the same IP address on the server: (Unfortunately, for this step, I am still using Route53 (https://aws.amazon.com/route53) from AWS Amazon since GoDaddy isn't able to accept DE domain transfers (https://de.godaddy.com/help/about-de-domains-5825)).

Awsdns.png

It can take some time before the new addresses are available via DNS. You can occassionally check to see if they're ready by using "nslookup" from your own PC:

Nslookup.png

Think about the following questions and considerations for NGINX:

  • Which ports will outbound ports? Naturally, 80 (HTTP) and 443 (HTTPS) in this case.
  • We will need links to the other Docker containers: blog, GitLab, ownCloud.
  • The name should be "nginxserver".
  • Storage for sites-enabled, certs, and logs should be done in the /home/data/Nginx directory.

We could use the follow commando (but we will not use it):

docker run -v /home/data/Nginx/sites-enabled/:/etc/nginx/conf.d/ -v /home/data/Nginx/certs/:/etc/nginx/certs -v /home/data/Nginx/logs/:/var/log/nginx --name nginxserver -p 80:80 -p 443:443 -d --link blog:blog --link gitlab:gitlab --link owncloud:owncloud

Since you may need to change this configuration frequently, for example, when there are new hosts, I recommend saving the configuration for the Docker container in a file and using "docker-compose". (https://docs.docker.com/compose/install/)

You can use "curl" to then run the installation:

curl -L https://github.com/docker/compose/releases/download/1.12.0/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose

Docker Compose still does not have any execute permissions; you need to use chmod to add these:

Chmodax.png

Now we can see whether or not "docker-compose" is working properly:

Testdockercompose.png

To do this, we create a directory in the root home directory and then enter "doccker-compose.yml":

Compose2.png

The file should look like this:

version: '2'
services:
 nginxserver:
   container_name: nginxserver
   image: nginx
   network_mode: bridge
   external_links:
      - blog:blog
      - gitlab:gitlab
      - owncloud:owncloud
   ports:
      - "80:80"
      - "443:443"
   volumes:
      - /home/data/Nginx/sites-enabled/:/etc/nginx/conf.d:z
      - /home/data/Nginx/certs/:/etc/nginx/certs:z
      - /home/data/Nginx/logs/:/var/log/nginx:z

Then you can start the "nginx" using "docker-compose":

Startcompose.png

Once Nginx has started, you will need to create the configuration For each sub-domain, you will need to specify the transfer path to the relevant mapped Docker container:

Map.png

Once the configuration has been applied, you can restart the "nginx":

Restartnginx.png

If that does not work, it may be useful to use "docker logs" to look for a mistake:

Loghelp.png

Configuring Shipyard

The last service that I added was Shipyard, so that I could manager a Docker container remotely.

The installation is pretty simple. You don't have to consider too much in advance, but:

  • The DNS should be ship.vr-worlds.de. (You will need to adapt the Nginx configuration for this.)

Do not use "docker run" for the deployment. Instead, use curl:

curl -sSL https://shipyard-project.com/deploy | bash -s

Now you can access Shipyard via port 8080 and several containers have been installed:

[root@CentOS-73-64-minimal ~]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 48f5cd2ce123 shipyard/shipyard:latest "/bin/controller --de" 2 weeks ago Up 2 weeks 0.0.0.0:8080->8080/tcp shipyard-controller ec4955037d0f swarm:latest "/swarm j --addr 176." 2 weeks ago Up 2 weeks 2375/tcp shipyard-swarm-agent 48487fb7223c swarm:latest "/swarm m --replicati" 2 weeks ago Up 2 weeks 2375/tcp shipyard-swarm-manager fee6b7fcc71e shipyard/docker-proxy:latest "/usr/local/bin/run" 2 weeks ago Up 2 weeks 0.0.0.0:2375->2375/tcp shipyard-proxy 2058c074314b alpine "sh" 2 weeks ago Up 2 weeks shipyard-certs d710310dae40 microbox/etcd:latest "/bin/etcd -addr 176." 2 weeks ago Up 2 weeks 0.0.0.0:4001->4001/tcp, 0.0.0.0:7001->7001/tcp shipyard-discovery 0fe8eb95b8fb rethinkdb "rethinkdb --bind all" 2 weeks ago Up 2 weeks 8080/tcp, 28015/tcp, 29015/tcp shipyard-rethinkdb

Now you just need to integrate "Shipyard" into the nginx and to configure it. You need to add the following to the nginx configuration: (/home/data/Nginx/sites_enabled)

SiteEnabled.png

Now it really makes sense to have used "docker-compose" for the configuration for Nginx; it's really easy to add to it now:

Addshipyard.png

Then you can rebuild the container using "docker-compose up -d":

Compose3.png

Centrally managing the configuration via Docker-Compose

To make sure that the automation from Docker is as effective as possible, I will now bring together the complete configuration in one new Docker-Compose file.

Compose4.png

This file looks like this:

version: '2.1'
services:
 ghost-mysql:
   container_name: ghost-mysql
   image: mariadb
   network_mode: bridge
   ports:
      - "3306:3306"
   volumes:
      - /home/data/MariaDb/:/var/lib/mysql:z
   environment:
      - MYSQL_ROOT_PASSWORD=rosi2511
 blog:
   container_name: blog
   image: ghost
   network_mode: bridge
   links:
      - ghost-mysql:mysql
   ports:
      - "2368:2368"
   volumes:
      - /home/data/Ghost/:/var/lib/ghost:z
 owncloud:
   container_name: owncloud
   image: owncloud:8.1
   network_mode: bridge
   links:
      - ghost-mysql:mysql
   ports:
      - "8082:80"
   volumes:
      - /home/data/owncloud:/var/www/html:z
 gitlab:
   container_name: gitlab
   image: gitlab/gitlab-ce
   network_mode: bridge
   ports:
      - "8443:443"
      - "8083:80"
      - "8022:22"
      - "8060:8060"
   volumes:
      - /home/data/gitlab/config:/etc/gitlab:z
      - /home/data/gitlab/logs:/var/log/gitlab:z
      - /home/data/gitlab/data:/var/opt/gitlab:z
      - /etc/localtime:/etc/localtime
   sysctls:
      - net.core.somaxconn=1024
   ulimits:
      sigpending: 62793
      nproc: 131072
      nofile: 60000
      core: 0
 nginxserver:
   container_name: nginxserver
   image: nginx
   network_mode: bridge
   links:
      - blog:blog
      - gitlab:gitlab
      - owncloud:owncloud
   external_links:
      - shipyard-controller:shipyard-controller
   ports:
      - "80:80"
      - "443:443"
   volumes:
      - /home/data/Nginx/sites-enabled/:/etc/nginx/conf.d:z
      - /home/data/Nginx/certs/:/etc/nginx/certs:z

Now we can delete all of the containers and create new ones with "docker-compose":

Test2.png

Configuring the firewall with Hetzner's firewall tool

My firewall configuration at Hetzner Online looks like this:

Hetzner.png

Rule #1 lets ICMP packets pass, for example pings and requests that let you determine the network bandwidth (MTU size etc.) For this reason, I recommend keeping the setting here on "accept".

Rule #2 allows you to connect with the server via SSH. You do not need to always leave this port open. When you aren't doing any maintenance, you can change the firewall action here to "discard". Then it is no longer possible to connect via SSH; therefore, it is not possible for a "putty" or a "winscp" to connect to the server:

Firewall2.png

Rule #3 allows you to access the system via "http" (port 80) and "https" (port 443).

Rule #4 is really necessary to allow the Linux host to also respond.

Summary

In this article, I have demonstrated which things you need to do so that a configuration with a NGINX proxy and a few other containers can work effectively together. I hope that these tips are ones that you can actually make use of and that my description is clear enough.

If you need support with a Docker environment, please contact me at christian.mueller@vr-worlds.de.



© 2018. Hetzner Online GmbH. Alle Rechte vorbehalten.