Oracle VM 3: Orphaned VMs
One of my biggest complaints about Oracle VM 3 is that everything is abstracted at every level. A VM name is simply meta data stored in a database (kind of) while everywhere that it matters the VM only exists as a UUID, lets look at some examples.
The Way It Is
From the hypervisor if we use native Xen commands to look at all of the instantiated VMs.
# xm list Name ID Mem VCPUs State Time(s) 0004fb000006000002a00feb1f52cae4 155 8192 2 -b---- 39176.1 0004fb00000600000b5b1f8673c451be 171 8192 2 -b---- 1007276.5 0004fb000006000026a40b6fb53fe2c3 135 16384 2 -b---- 169716.7 0004fb0000060000506f591d6877c284 194 3072 1 -b---- 16108.4 0004fb00000600007052ad7b040749ea 176 8192 2 -b---- 1726118.2 0004fb0000060000a07b66923b0f68a4 138 4096 2 -b---- 2092394.0 0004fb0000060000abfcf710ebd640df 101 2048 2 -b---- 540488.4 0004fb0000060000bf16256b99159fc6 143 16384 2 -b---- 3972112.7 0004fb0000060000e3f234f6c1d26448 173 8192 2 -b---- 852019.6 0004fb0000060000e602b0e0969dba07 92 2048 2 -b---- 234244.0 0004fb0000060000f9205e11159e5459 175 12288 2 r----- 4145289.3 Domain-0 0 2486 24 r----- 4455022.6
Now since under the covers we are using Xen that means that somewhere there is a vm.cfg file (could be named anything, but for now they are sticking with vm.cfg).
# cat vm.cfg vif = ['mac=00:21:f6:00:00:07,bridge=172.16.88.0'] OVM_simple_name = 'PRD-IM' guest_os_type = 'linux' disk = ['file:/OVS/Repositories/0004fb00000300005527ce584481a013/VirtualDisks/0004fb000012000052e7181e3458aba7.img,xvda,w'] vncunused = '1' uuid = '0004fb00-0006-0000-506f-591d6877c284' on_reboot = 'restart' boot = 'cn' cpu_weight = 27500 memory = 3072 cpu_cap = 0 maxvcpus = 2 OVM_high_availability = False vnclisten = '127.0.0.1' vnc = '1' OVM_description = 'Keste IM - Prod' on_poweroff = 'destroy' on_crash = 'restart' maxmem = 3072 name = '0004fb0000060000506f591d6877c284' builder = 'hvm' vcpus = 1 keymap = 'en-us' OVM_os_type = 'Oracle Linux 6' OVM_cpu_compat_group = '' OVM_domain_type = 'xen_hvm'
But finding it can be tricky. All repositories are stored in UUID form under /OVS/Respositories. Then inside of each repo you have multiple folders, Assemblies, ISOs, Templates, VirtualDisks, VirtualMachines. They are pretty self explanatory, but vm.cfgs go under VirtualMachines, they then refer to virtual disk images in the VirtualDisks folders. Here is an example for the vm.cfg we looked at above.
# pwd /OVS/Repositories/0004fb00000300005527ce584481a013/VirtualMachines/0004fb0000060000506f591d6877c284
This breaks down like this…
/OVS/Repositories/ [UUID of REPO] /VirtualMachines/ [UUID of VM] /vm.cfg
If we lose the manager, then we lose all ability to manage the environment without heavy navigation of the file system, or if we discover a hypervisor which was previously managed by the same OVM Manager UUID then we end up with a bunch of Orphaned VMs. You will notice that not only is there no meaningful names, there is also no meaningful information about the VMs, I assure that not all of my VMs have 256MB of RAM with a single CPU.
Figure 1-1: Orphaned VMs
Now the crux of this problem is that while we have discovered the Hypervisor we have not discovered the underlying repository.
Figure 1-2: Our Server Pool and Hypervisor
So first thing we need to do is right-click on the hypervisor and “Rescan Physical Disks” this should complete fairly quickly. This will allow OVM Manager to be aware of the Physical Disk on which we have our repository with our running VMs. Disregard the warning icon, that is due to having an older version of the hypervisor on that machine (3.1.1)
Figure 1-3: Rescan Physical Disks Job
Here we simply monitor the job until completion. After it is complete we right-click on the hypervisor and “Rediscover Server” this allows the OVM Manager to look at the physical disk that it just found out about to see if there is a repository on it.
Figure 1-4: Rediscover Server Job
Monitor until the job completes.
Figure 1-5: New Repository Visible
Now we see a UUID for a new repository under the Repositories tab. Right-click and Refresh.
Figure 1-6: Repository Refresh Job
Monitor until the job completes.
Figure 1-7: Fixed VMs
Now our VMs show the correct information.
A Few Notes
Any names that you assigned VirtualDisks are gone, they are back to UUID, sorry. Also if you had any machines that are stopped when you perform this action you will not see them on the hypervisor. They are in the Unassigned Virtual Machines folder. Click and drag to re-assign them. Or use the Migrate button.