flyship163@hotmail.com
flyship163

I am installing OVM Manager 3.2.1 and a Linux 5.8 x86-64 guest. I successfully run the ./createOracle.sh script. The install stops/freezes on step 5 – ‘Installing Oracle ADF Patch…’. Please advise.

Regards,
Gopal.

Thank you for an informative blog, how can I install phpMyAdmin in Solaris 11. will be most grateful – thanks

Gulaab

Ambu Ambooken

-matt

-matt

Maybe remove each server from the pool and re-add / rediscover it after an agent password change?

If I don’t require the external connectivity, I wouldn’t need to add the physical ethernet device to the bridge, right? What about the internal ovsbr0p1 port?

My final ideal configuration would be to have a GRE tunnel to another open vswitch host, and that the VMs on each of them be allowed to communicate with each other only, but not any other networks.

matthew.mattoon :Your XML is not complete, looks like libvirt is not adding the target dev and other port specific information. I would recommend following the article completely. And once you get it working as I had it working, then backtrack and undo the pieces you don’t want (external connectivity etc).

# Versions
ovs-vsctl (Open vSwitch) 1.4.0+build0
Compiled Feb 18 2013 13:13:22

libvirtd –version
libvirtd (libvirt) 1.0.2

# vsctl show – I don’t have the physical eth interface bridged because I don’t want my client OS to have network access to the physical network at this point
012603db-d204-4381-8cae-50e3f15c35cc
Bridge “br0″
Port “br0″
Interface “br0″
type: internal
Port “br0p1″
Interface “br0p1″
type: internal
ovs_version: “1.4.0+build0″

# XML config – I’ve snippet the other stuff for brevity.

I did change the XML to point the source bridge to the one I created in ovs-vsctl, as well as specifying the port type as openvswitch.

Is there a way to dump a trace of exactly what happened (is it somewhere in the libvirt logs, perhaps?) that could further help me understand the problem? I’m rather stumped.

There is no need for the bridge compatibility module, right? I’ve also tried turning that on but I seem to have a separate issue with the brcomp module not loading.

Calreth :

Following the same configuration, I’m getting the following error when I try to start my vm

error: Unable to add bridge ovsbr0 port vnet0: Operation not supported

Any ideas?

Shot in the dark, because you didn’t give me a lot of context. Looks like you are trying to use virt-manager instead of libvirt directly (as I did in my article). If this is the case refer to comment #26. Basically you need to fix your XML manually and then start your VMs however you want, virt-manager is trying to connect to a legacy bridge, which you don’t have by following this article.

-matt

It is possible yes, but with far too many caveats to qualify in a comment. Version requirements, hardware requirements, mobility requirements. Either way, outside the scope of this article.

-matt

Sascha :

Ok, let’s assume the following use case:

You have one physical server that you rent from a hosting company. You get exactly one IP address. That IP address needs to get assigned to the only physical ethernet port in the machine. Now you want to set up KVM and use OVS instead of the linux default bridge. Since you only have one real IP address (/32) you need to assign RFC1918 addresses to the virtual machines.

Now if you pull eth0 (the only physical interface) into OVS, ovsbr0 would have the /32 IP address. See where I am getting at? If your ovsbr0 interface has a /32 IP, what are you assigning to the VMs? If you assign RC1918 there is no routing between the VM and the host/outside_net.

Maybe I am just thinking wrong or I can’t bring it across in english. But my understanding is that in this usecase, you would have to give ovsbr0 an IP address with a large enough subnet mask that allows VMs to be in the same subnet, so that VMs can use the IP of ovsbr0 as their default gateway.

Basically what you are trying to accomplish really has very little to do with OVS. You are looking to multihome your machine, though this can be accomplished with a single eth adapter (IP aliasing, VLAN, et al). Your VMs are configured to use private address space, and communicate with their gateway which is also on private address space (ip alias) the router needs to then take over and route that outbound traffic out (and of course NAT it) but my point is that this doesn’t have anything to do with OVS. OVS handles the virtual “physical” connectivity. Something else needs to do the routing and NAT in order for something like this to work.

-matt

error: Unable to add bridge ovsbr0 port vnet0: Operation not supported

Any ideas?

You have one physical server that you rent from a hosting company. You get exactly one IP address. That IP address needs to get assigned to the only physical ethernet port in the machine. Now you want to set up KVM and use OVS instead of the linux default bridge. Since you only have one real IP address (/32) you need to assign RFC1918 addresses to the virtual machines.

Now if you pull eth0 (the only physical interface) into OVS, ovsbr0 would have the /32 IP address. See where I am getting at? If your ovsbr0 interface has a /32 IP, what are you assigning to the VMs? If you assign RC1918 there is no routing between the VM and the host/outside_net.

Maybe I am just thinking wrong or I can’t bring it across in english. But my understanding is that in this usecase, you would have to give ovsbr0 an IP address with a large enough subnet mask that allows VMs to be in the same subnet, so that VMs can use the IP of ovsbr0 as their default gateway.

Thanks for the clarification. I am still not sure of what your premise and ultimate goal here is. Are you asking if you can use OVS inside of a guest? As far as I can see from your notes, you are drawing connections between OVS and legacy bridges where are not there…

Case in point, when you configure a OVS (openvswitch – notice not a bridge) call it ovsbr0, then you add a couple of VMs VM1 and VM2, each of these are given a port on the openvswitch in the same way that you would connect up a physical nic to a physical switch port. In OVS this is displayed via the naming of the interface. ovsbr0p1, so VM1 would be connected to ovsbr0p1 and VM2 would be connected to ovsbr0p2 and so on. OVS provides a way to perform these physical connections and the switching of the packets. Now that said I see no reason why you could not have port level isolation via VLANs which you could then limit access to via the firewall/router (even if it is on the hypervisor).

All that to say that I think it is physically and technically possible though I really don’t understand a reasonable use-case for it.

-matt

thanks. My question re:NAT was more related to whether OVS needs a physical interface to operate on or whether I could just setup a dummy bridge interface (routing, NAT etc. would be done by the OS). Example (think KVM):

eth0: 10.10.10.1
ovsbr0: 192.168.1.1
–VM 1: 192.168.1.100 (bridged to ovsbr0)
–VM 2: 192.168.1.101 (bridged to ovsbr0)

Since ovsbr0 looks like just another interface to Linux, I could use standard utilities (route, iptables) to NAT outgoing traffic from VMs to eth0. e.g. inside VM default route = 192.168.1.1, default route for host = 10.10.10.254, use iptables to NAT 192.168.1.* to host’s IP 10.10.10.1).

I know I can do this with the default linux bridge and I was wondering if a similar setup is possible with OVS.

Hope this makes some sense….

I did some more digging on this tonight and it’s now working. Suspect working all along however a network config issue on the Pfsense LAN side (from a snapshot on earlier working install) was preventing DHCP delivery to the guests behind pfsense.

To recap for anyone else viewing these pages I have used Matthews excellent tutorial here on a Linux Mint 14 host running Virtualbox with Pfsense as DHCP/router/firewall – openvswitch – other VBox clients.

PFsense has 2 ports in Virtualbox, one bridged to wlan0 (WAN) and the other to lan0p1 (LAN)

As I’m using Virtualbox not KVM I omitted the following installation steps in Matthews tutorial

# apt-get install aptitude apt-show-versions ntp ntpdate vim kvm libvirt-bin vlan virtinst virt-manager virt-viewer openssh-server iperf pv openvswitch-brcompat

# virsh net-destroy default
# virsh net-autostart –disable default

Stop Libvirt and Qemu
This will prevent libvirt from bringing up the legacy bridge
# service libvirt-bin stop
# service qemu-kvm stop

The above prove to be unnecessary if using Virtualbox and not KVM

You will also not need

# apt-get purge ebtables

running this command gihves response ebtables not installed

Otherwise follow Mattews steps as he lays out and you’ll be fine.

On top of this I referred to this link which details how to create tuntap devices to act as ports on the switch.

http://golanzakai.blogspot.co.uk/2012/01/openvswitch-with-virtualbox.html

These are not persistent on reboot so you’re best to add to a script. Here’s mine which deletes any existing tuntap devices (lan0p1-lan0p16) then creates them again and binds them to ovsbr0

# vi vboxstart.sh

Add the following to the script and save

# delete existing lan0p interfaces
for tap in `seq 1 16`; do tunctl -d lan0p$tap; done;

# delete existing lan0p ports attached to ovsbr0
for tap in `seq 1 16`; do ovs-vsctl del-port ovsbr0 lan0p$tap; done;

# create tap devices on the host machine so we can bind them to the guests
for tap in `seq 1 16`; do tunctl -u tom -t lan0p$tap; done;

# Bring the tap interfaces up:
for tap in `seq 1 16`; do ip link set lan0p$tap up; done;

# Now we use ovs-vsctl to bind the tap devices to ovsbr0 switch
for tap in `seq 1 16`; do ovs-vsctl add-port ovsbr0 lan0p$tap; done;

Now edit file so it’s executable

# chmod +x vboxstart.sh

Hope this saves someone else some time!

I followed your Ubuntu 12.10 instructions for my Mint 14 laptop, not the 12.04 ones. So I think I’m ok there. I mention the 12.04 instructions as they worked for Mint 13 for me and the steps are not hugely different so struggling to work out where I’m going wrong.

I omitted installing the openvswitch-brcompat component as 12.10 doesn’t use that (I tried both with and without, made no difference),

I also omitted the following as I assume they are required for KVM and not Virtualbox (again I’ve tried with and without).

aptitude apt-show-versions ntp ntpdate vim kvm libvirt-bin vlan virtinst virt-manager virt-viewer openssh-server iperf pv

Instructions then followed to the letter (very clear and concise they are too) from this point on

aptitude purge ebtables

This includes amending the /etc/network/interfaces file to include

# The primary network interface
auto eth0
iface eth0 inet manual
up ifconfig $IFACE 0.0.0.0 up
down ifconfig $IFACE down

# hypervisor interface
auto ovsbr0p1
iface ovsbr0p1 inet dhcp

I “think” the openvswitch base install is ok and functioning its how to add ports to the switch (use tuntap devices per the other link in my earlier post or ovs-vsctl add-port command?) and then tie them to virtualbox guests.

Appreciate your instructions are KVM specific but hope you or another person finding this page can shed some light on where I’m going wrong.

Thanks again for this how to and the time you have taken to reply, it’s appreciated.

I suspect that perhaps the issue is with using the Ubuntu 12.04 instructions. Mint 14 is closer related to Ubuntu 12.10, and we had to do things differently with regards to the bridges themselves. In 12.10 we don’t do the brcompat stuff, since everything was upgraded enough to avoid it. Which means we don’t have to worry about the brcompat module not loading. That said once you have everything loading properly make sure that you have added the appropriate entries in the /etc/network/interfaces file.

When using Openvswitch, you no longer are using bridge-utils (brctl), so you do not need to add this sort of additional entry. You only need to add what is in the article.

As to your second question a device is either managed by ovs or it is not. So you can choose to not use eth0 with openvswitch, and instead use eth0 as a standard ethernet interface, but then have eth1 be managed by openvswitch.

To speak to NAT. I don’t believe that is a feature of openvswitch (and I doubt it will be added as that is not a switch function) that said, openvswitch could provision the connections while another device NAT’d the connections, but frankly I wouldn’t focus on that. If you need NAT I wouldn’t go to openvswitch.

I used your 12.04 instructions successfully in conjunction with this link http://golanzakai.blogspot.co.uk/2012/01/openvswitch-with-virtualbox.html

My set up is Virtualbox 4.2 on a Linux host with pfsense > openvswitch > multiple vbox clients and this worked perfectly under Ubuntu 12.04 and Mint 13.

I recently upgraded to Mint 14 so used your new instructions but the clients behind PFsense and Openvswitch can’t get out to the internet – they only get Apipa addresses.

I am missing some basic understanding of how the host, bridge and switch all bolt together I suspect.

My ovs-vsctl show output is

aeaa370c-2a19-4a0a-9f65-e1e8c2a34201
Bridge “ovsbr0″
Port “ovsbr0p1″
Interface “ovsbr0p1″
type: internal
Port “ovsbr0″
Interface “ovsbr0″
type: internal
ovs_version: “1.4.3″
Port “lan0p1″
Interface “lan0p1″
…..

Note there are multiple lan0p* entries I’ve just added the one for brevity.This is all on a laptop with wlan0 and eth0 – I’ve bridged ovsbr0 to eth0 and created the internal port, setup /etc/network/interfaces as required and then added tuntap devices to represent actual ports on the switch.

I’ve searched virtualbox forums, openvswitch mailing lists and numerous other places but can’t find the answer to what is likely a very simple problem.

Any pointers gratefully received.

thanks.

Couple of questions to get my head wrapped around this:

bridge-utils needed a stanza like “bridge_ports” in the interface configuration in /etc/network/interfaces, or you would have to call “brctl addbr xxx” somewhere else in your startup scripts to bring up a bridge during boot. I suppose this is not needed with OVS? If you do a manual “ovs-vsctl add-br” it is boot-persistant? If so, where does it store it’s config?

The other question I have: Could I keep my eth0 with a static IP and not add it to OVS but instead just create OVS bridge with IP address and then route traffic upstream? This would be helpful in a scenario where you have a hosted server with only one IP address. You could give your guests some RFC1918 addresses and just NAT if traffic needs to go out of the box… basically recreating libvirt’s virbr0 bridge… would that be possible?

Thanks!

# zlogin -C -e ‘#.’

Then just use #. to jump out of the console.

Cheers,
Toki

The benefit is having the Service Management Framework manage it. The overcomplications you are seeing are relating to maintaining facilities to help people get used to the new way. The SMF is really pretty simple and elegant. That said only time will tell if having DNS client configurations in the SMF is worth it.

-matt

After creating the VM through virt-manager, even ignoring the error (by manually adding the interface name), I was able to run “virsh edit vmnamehere” and fix the XML file according to your post.

After that, virt-manager starts the VM normaly using openvswitch anyway…

I’ll back to Ubuntu 12.04 + Ubuntu Cloud Archives now…

Thank you!

Regards,
Thiago

I just saw this:

“The virt-manager doesn’t yet support openvswitch. So there are currently no options available to start a VM using vswitch from virt-manager.”

Source: http://en.community.dell.com/techcenter/networking/w/wiki/3820.openvswitch-openflow-lets-get-started.aspx

I’m sorry to flood you blog… Feel free to remove my irrelevant posts…

Tks!
Thiago

85e2cda4-d955-4491-b69a-1dacada1301a
Bridge “ovsbr0″
Port “eth0″
Interface “eth0″
Port “ovsbr0p1″
Interface “ovsbr0p1″
type: internal
Port “ovsbr0″
Interface “ovsbr0″
type: internal
ovs_version: “1.4.3″

You think that it is okay?

Thanks!
Thiago

I’m using:

—
auto ovsbr0p1
iface ovsbr0p1 inet static
address X.X.X.X
netmask 24
—

Instead of it via DHCP… I think that it is okay (don’t have DHCP within my network)…

But, the error come from libvirt (libvirtError)… Since virt-manager uses libvirt.

Anyway, I’ll try via CLI in a couple minutes…

Best,
Thiago

Sorry for my delayed response I was out of town last week and unreachable.

I didn’t test any of my procedures with Virt-Manager. I would suspect that Virt-Manager hasn’t added to support yet for this different network connection. If you use the XML configuration method as described above you should not see any issues correct?

-matt

I’m facing this:

Traceback (most recent call last):
File “/usr/share/virt-manager/virtManager/asyncjob.py”, line 45, in cb_wrapper
callback(asyncjob, *args, **kwargs)
File “/usr/share/virt-manager/virtManager/create.py”, line 1909, in do_install
guest.start_install(False, meter=meter)
File “/usr/lib/python2.7/dist-packages/virtinst/Guest.py”, line 1236, in start_install
noboot)
File “/usr/lib/python2.7/dist-packages/virtinst/Guest.py”, line 1304, in _create_guest
dom = self.conn.createLinux(start_xml or final_xml, 0)
File “/usr/lib/python2.7/dist-packages/libvirt.py”, line 2501, in createLinux
if ret is None:raise libvirtError(‘virDomainCreateLinux() failed’, conn=self)
libvirtError: Unable to add bridge ovsbr0p1 port vnet0: Operation not supported

At virt-manager, I can’t select the listed bridges, they are “unselectable” (gray)..

Any tips?

libvirtd –version
libvirtd (libvirt) 0.9.13

Thanks!

I found what I think/hope will be a better workaround for the “Waiting for network configuration…” issue at boot up. I just told the networking upstart job to call the openvswitch init scripts before doing its thing by adding them to the pre-start script stanza like this:
—————
pre-start script
mkdir -p /run/network

#added to get openvswitch to start before networking – gets rid of delay waiting for networking at boot
/etc/init.d/openvswitch-controller start
/etc/init.d/openvswitch-switch start

ifup -a
end script
—————–

This way the switch is already available and “networking” can bring everything up like normal. I did not have to modify the failsafe.conf

141MB is really not that large. Even Damn Small Linux (which is the smallest semi-modern Linux I know of) still weighs in at ~50MB.

As far as how you can trim it down, there is only one way. Find something you don’t need and remove it. So start digging through the packages and seeing what you don’t need and removing it.

-matt

Thank you for the informative info.
I could configure OVS on Linux Mint 14 thanks to your great post !
It works well.

Shigehiro

I have been spending a huge amount of time lately trying to get to know OpenVswitch, but i have not honestly not succeeded that much. I am currently in Kathmandu where i am working witha small WISP and i as of now only have my laptop to run tests etc on. I think its important to be able to demonstate the different ideers and so forth, before implementing them etc. Because of the fact that i am running the most resent libvirt on my ubuntu 12.10 laptop, and the connections available is wireless, making a bridged interface is a bit of a problem.

Basically i would like to set up a enviroment where i could show them how NIDS works and could benefit there installations etc. If have read that putting up a bridged interface and attaching the vms directly would work as demonstration because they work in promiscuous mode. But again, this seems to not work that well with the connection being all wireless and so forth.
Lastly but not least, i have thought of using a ubiquiti loco or nanobridge, connecting it to the wired card and so forth…though it seems to be a over the top, amount of work.

I hope my post is not to confusing, and i hope you will one day get to the point where you will put up a guide on how to set up port mirroring on openVswitch