vancluevertech.com

I’ve spent the last few days getting my bearings around Open vSwitch. It’s pretty amazing, and IMO if you are virtualizing under Linux these days, it’s pretty much a must.

So what is Open vSwitch? It’s basically the Open Source answer to other proprietary technologies, such as VMware’s distributed vSwitch (which you have probably used if you have ever used multiple servers within vSphere). It allows you to build a distributed layer-2 network entirely in software. It also performs very well under a single server setup, allowing you to build sophisticated switch fabrics under a single physical interface. It solves some frustrations surrounding network bridging under a KVM setup as well that you may have encountered, such as having a bridge that shares the same physical interface as your host’s management address.

Check out http://openvswitch.org/ for some documentation and tutorials.

I will be explaining some basic concepts regarding Open vSwitch here, namely how to set up a bridge, attach an interface to it, and also how to automate the process using ifcfg-* files under CentOS (and by proxy, probably RHEL and Fedora as well). Also, we will discuss how to set up a VM on the bridge using libvirt.

Some Terms

A bridge is a network fabric under Open vSwitch. For the purpose of this tutorial, a bridge represents a broadcast domain within the fabric at large, ie: a VLAN. Note that this does not have to be the case all the time, as it is possible to have a bridge that has ports on different VLANs, just like a physical switch.

A port is a virtual switch port within the bridge. These ports are attached to interfaces, such as physical ones, virtual machine interfaces, or other bridges.

Above is a very simple diagram that depicts the bridge br-ex, with ports connected to a VM’s eth0, and the host machine’s eth0.

Creating a Bridge

Run the following command to create a bridge:

ovs-vsctl add-br br-ex

This command would create a new empty bridge, br-ex. This bridge can then be addressed just like a regular interface on the system, but of course, would not do much at this point in time since we do not have any ports attached to it.

Adding a Port

ovs-vsctl add-port br-ex eth0

This would add eth0 to the bridge br-ex.

CentOS/RedHat/Fedora Interface Configuration Files

You can also have the OS set up bridges for you upon system startup – this is especially useful if you are binding IP addresses to a specific bridge. Note that any bridges that you create like this will get destroyed/re-created upon restart of the network (ie: system network restart or systemctl restart network.service).

Change your ifcfg-eth0 file to look something like this:

HWADDR="00:11:22:AA:BB:CC"
DEVICE="eth0"
ONBOOT="yes"
NM_CONTROLLED="no"
TYPE=OVSPort
DEVICETYPE=ovs
OVS_BRIDGE=br-ex

And create a ifcfg-br-ex interface configuration file:

DEVICE=br-ex
DEVICETYPE=ovs
TYPE=OVSBridge
BOOTPROTO=static
ONBOOT=yes
IPADDR="1.2.3.4"
NETMASK="255.255.255.0"
GATEWAY="1.2.3.1"
DNS1="1.2.3.2"
DNS2="1.2.3.3"

Sub in your values for MAC addresses, physical interface names, and IP addresses, obviously.

Another note here that is extremely important is that you need to make sure that you use the DEVICE directive instead of the NAME directive. The latter may be left over in your physical interface configuration file from installation, so make a note to change it. I will address the exact reason why in a different article.

Setting Up a Libvirt VM to use a Bridge

Now that you have set up the above, you can add a VM to the bridge with Libvirt. Edit your domain’s (VM’s) XML file and add a block like this for every NIC you want to create:

 <interface type='bridge'>
  <mac address='52:54:00:71:b1:b6'/>
  <source bridge='br-ex'/>
  <virtualport type='openvswitch'/>
  <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
 </interface>

This was taken directly from the Open vSwitch Libvrirt HOWTO.

Make sure of course that you assign a correct PCI ID. You may wish to create the domain first via other means, add the devices you will need, and just elect to not use network at first. Unfortunately, it does not seem that a lot of Libvirt admin tools have specific Open vSwitch support just yet (at least it does not seem that the version of virt-manager that comes with most distributions does, anyway).

Edit Feb 12 2015 – Slight correction to physical interface config file – duplicate device type.