Hetzner - DokuWiki

KVM mit Nutzung aller IPs aus Subnetz/en

Special KVM Setup on Debian Lenny.

KVM mit Nutzung aller IPs aus Subnetz.png

Inhaltsverzeichnis

Introduction

After reading these instructions, some people might say in an exasperated tone: "This is much too complicated for me".

IMPORTANT: To avoid frustration and disappointment please: Read through the introduction completely before you turn your server upside down - then at least you will know what to expect.

Further instructions on the subject of KVM can be found in Virtualisation.

As with all instructions found on the Internet, the same applies here: Use the instructions at your own risk.

Characteristics

These instructions are for those system administrators who wish to use KVM on Hetzner servers under the following constraints:

  • NAT does not take place
    • each virtual machine has at least one public IP address
  • the physical machine routes the traffic
    • Routing of the traffic between individual VMs
    • Routing of the traffic between VMs and the Internet
    • hence no Proxy ARP is required either
    • as we are routing and not switching, iptables can be used on the Host system
      • main firewall rules on the Host instead of in each individual Guest
      • Firewall rules for traffic between the VMs is possible
  • IP addresses do not get "lost"
    • the network and broadcast addresses for the additional single IP nets can be used
    • no IP address disappears as a result of an IP having to serve as gateway for the additional net
      • all IP addresses of the additional IP net really can be used
  • the use of libvirt for network configuration is waived
    • in these instructions, libvirt only manages the Guests, no nets
  • Debian Lenny is used as the operating system on the Host

Requirements

  • Hetzner Dedicated Server
  • Additional IP net from Hetzner
  • Minimal installation of Debian Lenny ideally with LVM
    • in addition, the following packages and their dependencies:
      • bridge-utils
      • dhcp3-server
      • iptables
      • kvm

Host System

Network Configuration

It is useful to make a note of the current server network configuration. The default gateway is especially important.

The default gateway can be taken from

ip route

(from packet iproute) or

route -n

IP Addresses

Let's assume you have noted the following details or received these from Hetzner:

  • Server
    • Main IP: xx.yy.99.146
    • Default Gateway: xx.yy.99.129
  • IP Net
    • IP: xx.yy.240.64
    • Mask: 255.255.255.240
    • Broadcast: xx.yy.240.79

To put these details in plain language:

Your server with the IP xx.yy.99.146 sends all packets which are addressed in the Internet to the gateway xx.yy.99.129. Hetzner routes the additional IP addresses xx.yy.240.64 - xx.yy.240.79 to the main IP xx.yy.99.146 (in the routing table of the Hetzner router it shows: the net xx.yy.240.64/28 is behind xx.yy.99.146).

/etc/network/interfaces

The file /etc/network/interfaces on your Host system should now look like this:

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet static
        address xx.yy.99.146
        netmask 255.255.255.255
        gateway xx.yy.99.129
        pointopoint xx.yy.99.129

We staticallyset the IP address of your server as well as the default gateway here.

The net mask 255.255.255.255 ensures that we still always address outward packets to the Hetzner default gateway - even if we wish to speak with a rack neighbour.

The line pointopoint (Attention: not pointtopoint) is necessary due to the restrictive net mask: So that we can reach the gateway at all now, we require a Host route. This line takes care of just that.


If we really want to use all additional IP addresses and ensure none get "lost", we cannot unfortunately simply route the IP net to a bridge completely. We need to create a single route for each individual IP address in the additional net whether we want to or not.

But even this isn't enough: As we want to route and not switch between individual VMs, the virtual machines with their network interfaces cannot be put in the same bridge (if they were, VM1 could bypass the firewall and access the Host system on VM2).

Conclusion

For each IP address from the additional IP net

  1. we create an own bridge
  2. we set an entry in the routing table
  3. give the bridge an IP address in the format 172.30.X.1/24 (more on this later: DHCP server for virtual machines)

Entry in /etc/network/interfaces for the first additional IP address xx.yy.240.64

auto br64
iface br64 inet static
        address 172.30.64.1
        netmask 255.255.255.0
        pre-up brctl addbr $IFACE
        post-up route add -host xx.yy.240.64 $IFACE
        post-down brctl delbr $IFACE

Explanation:

Bridge br64 gets IP address 172.30.240.64.1/24 (more on this later: DHCP servers for virtual machines), pre-up sets up the bridge, post-up sets the route (put simply: IP address xx.yy.240.64 is directly at interface br64) and post-down ensures, that the bridge is deleted, when the interface is turned off.

These blocks now repeat themselves in /etc/network/interfaces until the last address of the IP net:

auto br79
iface br79 inet static
        address 172.30.79.1
        netmask 255.255.255.0
        pre-up brctl addbr $IFACE
        post-up route add -host xx.yy.240.79 $IFACE
        post-down brctl delbr $IFACE

DHCP Server for Virtual Machines

As a lot of installers cannot cope with the Host route, we enable a DHCP server in the Host system and allocate private IP addresses to RFC1918 by DHCP to our VMs.

/etc/dhcp3/dhcpd.conf

authoritative;
default-lease-time              3600;
max-lease-time                  3600;
ddns-update-style               ad-hoc;
log-facility                    local7;
use-host-decl-names             on;

option subnet-mask              255.255.255.0;
option domain-name              "lan";
option domain-name-servers      213.133.100.100, 213.133.99.99, 213.133.98.98;

subnet 172.30.64.0 netmask 255.255.255.0 {
        option routers          172.30.64.1;
        range                   172.30.64.10 172.30.64.200;
}

subnet 172.30.65.0 netmask 255.255.255.0 {
        option routers          172.30.65.1;
        range                   172.30.65.10 172.30.65.200;
}

...

subnet 172.30.78.0 netmask 255.255.255.0 {
        option routers          172.30.78.1;
        range                   172.30.78.10 172.30.78.200;
}

subnet 172.30.79.0 netmask 255.255.255.0 {
        option routers          172.30.79.1;
        range                   172.30.79.10 172.30.79.200;
}

A DHCP subnet is declared for each bridge and IP net, which defines the gateway to be used for DHCP clients in this network.

In this way the virtual machine, for example, whose "network card" is added to the bridge br64, is allocated by the DHCP server one of the IP addresses from the area 172.30.64.10-200, the default gateway 172.30.64.1 and the three from Hetzner (213.133.100.100, 213.133.99.99 und 213.133.98.98) as DNS servers.

/etc/default/dhcp3-server

So that we can only listen and respond to DHCP requests on our bridge interfaces (especially not on eth0 - this goes to Hetzner), our file /etc/default/dhcp3-server looks like this:

INTERFACES="br64 br65 br66 br67 br68 br69 br70 br71 br72 br73 br74 br75 br76 br77 br78 br79"

Your DHCP server should now use

/etc/init.d/dhcp3-server restart

to start the new config.

Firewall

A rudimentary and incomplete firewall script (packet filter is a more appropriate name) would be as follows, for example:

#!/bin/sh
it="/sbin/iptables"

# Your additional IPs and additional subnets listed and separated by a space
#
# Example:
#
# MY_NET="192.0.2.8/29  192.0.2.44  192.0.2.128/29"
MY_NET="xx.yy.240.64/28"

# Your internal DHCP nets listed and separated by a space
MY_NET_DHCP="172.30.0.0/16"

# Main IP of the server
HAUPT_IP="xx.yy.99.146"


#
# INPUT
#


# DHCP Requests from the VMs
$it -A INPUT -p udp --dport 67 -i br+                    -j ACCEPT

# other rules for INPUT

# Discard rest
#$it -A INPUT -j DROP


#
# FORWARD
#


# Packets belonging to existing connections
$it -A FORWARD -m state --state RELATED,ESTABLISHED      -j ACCEPT

# VMs (Additional IP/Additional Net) --> Internet
for NET in $MY_NET
do
  $it -A FORWARD -i br+ -o eth0 -s $NET                  -j ACCEPT
done

# VMs (DHCP) --> Internet
for NET in $MY_NET_DHCP
do
  $it -A FORWARD -i br+ -o eth0 -s $NET                  -j ACCEPT
done

# VMs <--> VMs
# Traffic between VMs can be regulated here

# VMs --> VMs
for SOURCE_NET in $MY_NET $MY_NET_DHCP
do
  for DEST_NET in $MY_NET
  do
    $it -A FORWARD -i br+ -o br+ -s $SOURCE_NET -d $DEST_NET -j ACCEPT
  done
done

# Internet --> VMs
for NET in $MY_NET
do
  $it -A FORWARD -i eth0 -o br+ -d $NET                  -j ACCEPT
done

# Discard rest
$it -A FORWARD -j DROP


#
# POSTROUTING
#


# Masquerading
# Concealing private internal DHCP net behind the main IP
for NET in $MY_NET_DHCP
do
  $it -t nat -A POSTROUTING -o eth0 -s $NET -j SNAT --to-source $HAUPT_IP
done


#
# Routing
#


# Enable routing
echo 1 > /proc/sys/net/ipv4/ip_forward

The customized script can be set up as /usr/local/bin/firewall and started in /etc/rc.local .

Virtual Machines

Setting Up a Virtual Machine

KVM has tools for creating virtual machines (e.g. their XML file).

One such file for the virtual machine "cookie", /etc/libvirt/qemu/cookie.xml, could look like this:

<domain type='kvm'>
  <name>cookie</name>
  <uuid>00000000-2812-2009-1719-000000000067</uuid>
  <memory>2097152</memory>
  <vcpu>4</vcpu>
  <os>
    <type arch='i686' machine='pc'>hvm</type>
    <boot dev='hd'/>
    <boot dev='cdrom'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/bin/kvm</emulator>
    <disk type='file' device='cdrom'>
      <source file='/var/lib/libvirt/iso/grml-medium_2009.10.iso'/>
      <target dev='hda'/>
      <readonly/>
    </disk>
    <disk type='block' device='disk'>
      <source dev='/dev/vgdom/cookie'/>
      <target dev='vda' bus='virtio'/>
    </disk>
    <interface type='bridge'>
      <mac address='54:52:00:00:67:01'/>
      <source bridge='br67'/>
      <model type='virtio'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target port='0'/>
    </console>
    <input type='mouse' bus='ps2'/>
    <graphics type='vnc' port='5967' autoport='no' keymap='de'/>
  </devices>
</domain>

The only important thing here really is that the UUID, the MAC address and the name of the VM are clear. In this example, the virtual machine "cookie" is mounted with its network interface in bridge br67.

Network Configuration

By DHCP

Each one of the virtual machines, whose "virtual" network interface has been added to one of the bridges on the Host system (libvirt does this automatically on starting the VMs), accesses the Internet via DHCP.

Example: VM "cookie" with network interface in bridge br67.

The VM receives by DHCP

As soon as the VM sends packets to the Internet (from a Host point of view, these are packets which go out to eth0), these are rewritten to the main IP xx.yy.99.146 . As no port redirections are defined, virtual machine services (SSH, HTTP, ...) are not reachable from the Internet.

Network configuration per DHCP is also an excellent way for installing new virtual machines.

static

As soon as the installation is finished, you can switch to the static network configuration with public IP address.

Example: VM "cookie" with network interface in bridge br67.

File /etc/network/interfaces in the VM:

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet static
        address xx.yy.240.67
        netmask 255.255.255.255
        gateway xx.yy.99.146
        pointopoint xx.yy.99.146

Public IP address xx.yy.240.67 with net mask 255.255.255.255, gateway is the main IP xx.yy.99.146 of the server (not the default gateway xx.yy.99.129 of the Host system!) and as already known from /etc/network/interfaces a Host route to the gateway is required because of the restrictive net mask.

The VM and its services (SSH, HTTP, ...) are now accessible via the public IP address.

Obstacles and Pitfalls

Alternative Method for Firewall Script

I have used these instructions (see above.) as a basis for my own configuration. However, the additionally available single IPs are fragmented and therefore not in a common subnet. So, I have written the following script (uses Bash Features):

#!/usr/bin/env bash
IPT='echo iptables'
# In this example only 2 IPs are in the same subnet
# Format is:
# [Name of the bridge interfaces]:[real Target IP]
VMS="_otto:88.x.y.z+0 _hugo:88.x.y+1.z+0 _horst:88.x.y+2.z+3 _bernd:88.x.y.z+4"

# Permit existing connections
$IPT -A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT
# Between VMS and Internet
for i in $VMS; do
  # First extract names and IP
  VM_NAME=${i%%:*}
  VM_ADDR=${i##*:}
  # Now the Internet rules  <--> Target IP (both sides)
  $IPT -A FORWARD -i $VM_NAME -o eth0 -s $VM_ADDR -j ACCEPT
  $IPT -A FORWARD -i eth0 -o $VM_NAME -d $VM_ADDR -j ACCEPT
  for j in $VMS; do
    if [[ "$i" == "$j" ]]; then continue; fi # if identical, next round
    # Extract names and IP again
    VM_NAME2=${j%%:*}
    VM_ADDR2=${j##*:}
    # Allow any traffic between VMs
    $IPT -A FORWARD -i $VM_NAME -o $VM_NAME2 -s $VM_ADDR -d $VM_ADDR2 -j ACCEPT
  done
done
# Here the rest can be prohibited etc.
# ....

I have completely left out DHCP configuration, as I do not need it for my purposes. Therefore, the corresponding rules are not included either.

This script makes it a little easier to set the rules between the individual VMs.



© 2018. Hetzner Online GmbH. Alle Rechte vorbehalten.