IT From All Angles

Today we will be discussing the process of configuring VLANs for the KVM Guests to use to gain access to a wider variety of networks from a single host.  This is extremely helpful if you are able to take advantage of 10Gbps connections, as you can eliminate multiple 1Gbps connections for a single 10Gbps and still get a net gain in bandwidth.  If you do not use 10Gbps yet, but still take advantage of VLANs then this will still help you reduce the cabling for your hosts.

Now obviously you will need to have the network hardware and configurations to support VLANs before you try and implement it on your Virtualization stacks (assuming you would like connectivity outside of your host).  That said please make sure you switches, cards, etc support it.  Also I am assuming that you have a working

This procedure was documented for use on Ubuntu Server amd64 (10.10 and 11.04).  Due to similarities it will most likely work on other Ubuntus and Debians, however your mileage may vary.

Also I have found that this procedure can cause downtime for guests connected to bridges which can only be resolved with a reboot of the guest (and in some cases the host).  I think this occurs when you are essentially yanking the bridge away from guests (as a result of a restart of networking).  That said I recommend you plan this change for a quiet time that you will have some time to think.
Install Prerequisites

This procedure will require the following packages to be installed.

# apt-get install vlan ebtables

Configure Ebtables to Persist Rules

By default Ebtables will only store the rules you create until the ebtables service is restarted (or the host is rebooted).  Making the below changes to the configuration file at /etc/default/ebtables will modify it the service to persist the rules after a restart of the service.

# cat /etc/default/ebtables
EBTABLES_LOAD_ON_START="yes"
EBTABLES_SAVE_ON_STOP="yes"
EBTABLES_SAVE_ON_RESTART="yes"

Configure Ebtables to Allow Tagged Traffic

# ebtables -t broute -A BROUTING -i eth0 -p 802_1Q -j DROP

Configure Network Interfaces

Now in the config below we have one physical interface eth0, VLAN 20, and two bridges exposing VLAN 20 and the untagged interface.  Obviously you will want to adjust you VLAN numbers, IP addresses and interface names (especially the vlan_raw_device and bridge_ports parameters).  Another important note is that only br0 has a default gateway.  This will be the “management” interface, all of the other bridges will be for guest access only.  As such they will not be able to route traffic to other networks.  However from a guest which is connected to the bridge it will be able to route assuming it is configured with the correct default gateway.

# cat /etc/network/interfaces
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet manual

auto eth0.20
iface eth0.20 inet manual
vlan_raw_device eth0

auto br0
iface br0 inet static
address 192.168.0.15
netmask 255.255.255.0
gateway 192.168.0.1
bridge_ports eth0
bridge_stp off
bridge_maxwait 0
bridge_fd 0

auto br0.20
iface br0.20 inet static
address 192.168.20.15
netmask 255.255.255.0
bridge_ports eth0.20
bridge_stp off
bridge_maxwait 0
bridge_fd 0

Once your configuration is in place you can put it into place by restarting networking, or rebooting the machine.  Please keep in mind if you screwed up or for some reason it just doesn’t work in your environment you may need physical access to revert or repair your configuration.

This should give you a working VLAN bridge configuration.  Now all you need to do is connect the guest to the correct bridge interface and then perform the guest side networking configuration.

Related posts:

1. KVM Guests: Manipulating Libvirt XML for Guest Creation
2. Linux-KVM: Ubuntu 12.04 with Openvswitch
3. Hyper-V Guests: Linux Integration Components (v2) on Oracle Enterprise Linux
4. Linux-KVM Management: Offline Migration
5. Linux-KVM Management: Live Migration Without Shared Storage