Installing OpenIndiana in a Xen DomU
So I am on the ground in Dallas and trying to get my head around some of the technologies that we use in my new environment. One of those being the Xen kernel. I have never actually used Xen before, though I have used xenserver a bit. Anyways so my first foray into the Xen world is trying to get OpenIndiana (successor to OpenSolaris) running on it, and see what sort of performance issues it might have. We are hoping to be able to leverage its CIFS and vscan functionality to provide some security for a secured network file server.
I am by no means an expert in Xen, so you might notice some minor mistakes. These instructions worked in my environment and should work in your environment as well, and the performance was actually quite good. I am making the following assumptions.
- You are using a Linux Dom0, this might not matter, but for the purposes of this exercise we will pretend that it does.
- You are using LVM to back your DomU.
- You are going to run OpenIndiana, if you are trying to run something different (Solaris, Nexenta, SmartOS, etc) then there will be other nuances to get those particular distributions working.
- You are using XM configuration files directly and not using libvirt to try and simplify the configuration (I love libvirt, but could not get it working for these configurations).
- You do have libvirt installed, so that we can take advantage of virsh console.
Prepare the File System
Make directories for your Xen DomU kernels, and for the OpenIndiana cd mount.
# mkdir /etc/xen/kernels/oi151a; mkdir /mnt/oi151a
Create the Logical Volume for your OpenIndiana DomU.
# lvcreate -L40G yourvgnamehere -n vm_openindiana_boot
Mount the OpenIndiana ISO.
# mount -o loop,ro /mnt/scratch/oi151a.iso /mnt./oi151a
Copy the Kernel and Boot Archive out of the CD.
# cp /mnt/oi151a/platform/i86pc/amd64/boot_archive /etc/xen/kernels/oi151a/ # cp /mnt/oi151a/platform/i86xpv/kernel/amd64/unix /etc/xen/kernels/oi151a/
Unmount the OpenIndiana CD.
# umount /mnt/oi151a
Create the DomU Configurations
You actually need 2 configurations, you have to boot your LiveCD installer differently then the installed system.
The first is the build configuration, in other words the one that you can boot the LiveCD and run the installer for OpenIndiana. This configuration requires that the ISO image be the first disk, we also must use the pygrub boot loader and finally we configure the console and livemode with extra parameters.
# cat /etc/xen/vm/openindiana-build disk = [ 'file:/mnt/scratch/oi151a.iso,xvdc:cdrom,r', 'phy:yourvgnamehere/vm_openindiana_boot,xvda,w' ] vcpus = 1 memory = 1024 name = "openindiana" #kernel = "/etc/xen/kernels/oi151a/unix" #ramdisk = "/etc/xen/kernels/oi151a/boot_archive" #extra = "/platform/i86xpv/kernel/amd64/unix -B console=ttya,livemode=text" bootloader = "/usr/bin/pygrub" on_shutdown = "destroy" on_reboot = "restart" on_crash = "destroy" vif = [ 'mac=aa:00:0a:4c:00:31, bridge=br0' ]
The second is the running configuration, which will actually boot of the disk, and persist your changes across reboots.
This one no longer needs to pygrub bootloader, console and livemode, but instead we need to point to the rpool (system zfs filesystem) and the boot path, so that it can find the file systems.
# cat /etc/xen/vm/openindiana-run disk = [ 'phy:yourvgnamehere/vm_openindiana_boot,xvda,w' ] vcpus = 1 memory = 1024 name = "openindiana" kernel = "/etc/xen/kernels/oi151a/unix" ramdisk = "/etc/xen/kernels/oi151a/boot_archive" extra = "/platform/i86xpv/kernel/amd64/unix -B zfs-bootfs=rpool/ROOT/openindiana,bootpath='/xpvd/xdf@51712:a'" on_shutdown = "destroy" on_reboot = "restart" on_crash = "destroy" vif = [ 'mac=aa:00:0a:4c:00:31, bridge=br0' ]
Boot to OpenIndiana LiveCD
Here is where everything starts to get a bit crazy. So firstly we need to create the VM.
# xm create openindiana-build
Now we need to connect to the VM via console. Keep in mind that when you need to close the console, you can use ^] to close it. Default username is jack with a password of jack.
# virsh console openindiana OpenIndiana Build oi_151a 64-bit (illumos f342d051b376) SunOS Release 5.11 - Copyright 1983-2010 Oracle and/or its affiliates. All rights reserved. Use is subject to license terms. WARNING: No randomness provider enabled for /dev/random. Use cryptoadm(1M) to enable a provider. Hostname: openindiana Remounting root read/write Probing for device nodes ... Preparing live image for use Done mounting Live image USB keyboard 1. Albanian 25. Latin-American 2. Arabic 26. Lithuanian 3. Belarusian 27. Latvian 4. Belgian 28. Macedonian 5. Brazilian 29. Malta_UK 6. Bulgarian 30. Malta_US 7. Canadian-Bilingual 31. Norwegian 8. Croatian 32. Polish 9. Czech 33. Portuguese 10. Danish 34. Romanian 11. Dutch 35. Russian 12. Dvorak 36. Serbia-And-Montenegro 13. Estonian 37. Slovak 14. Finnish 38. Slovenian 15. French 39. Spanish 16. French-Canadian 40. Swedish 17. Hungarian 41. Swiss-French 18. German 42. Swiss-German 19. Greek 43. Traditional-Chinese 20. Icelandic 44. TurkishF 21. Italian 45. TurkishQ 22. Japanese-type6 46. UK-English 23. Japanese 47. US-English 24. Korean To select the keyboard layout, enter a number [default 47]: 1. Arabic 12. Hungarian 2. Catalan 13. Indonesian 3. Chinese - Simplified 14. Italian 4. Chinese - Traditional 15. Japanese 5. Czech 16. Korean 6. Dutch 17. Polish 7. English 18. Portuguese - Brazil 8. French 19. Russian 9. German 20. Slovak 10. Greek 21. Spanish 11. Hebrew 22. Swedish To select the language you wish to use, enter a number [default is 7]: User selected: English Configuring devices. openindiana console login: Jan 10 16:06:51 openindiana nwamd[66]: 10: start_dhcp: ipadm_create_addr failed for xnf0: Address object already exists Jan 10 16:06:54 openindiana gdm-simple-slave[2150]: CRITICAL: file gdm-session-direct.c: line 2167: assertion `value != NULL' failed Jan 10 16:06:56 openindiana gnome-keyring-daemon[2301]: couldn't allocate secure memory to keep passwords and or keys from being written to the disk openindiana console login: jack Password: Last login: Tue Jan 10 16:06:54 on console OpenIndiana (powered by illumos) SunOS 5.11 oi_151a September 2011
Configure VNC in the LiveCD Session
In my environment we do not have DHCP on this VLAN, so I have to reconfigure networking to use a static IP address. I have a separate article here. But we will need to configure VNC, so that we can launch the gui install program.
$ mkdir .vnc $ cp .Xclients .vnc/startup $ vncserver You will require a password to access your desktops. Password: Verify: xauth: creating new authority file /jack/.Xauthority New 'openindiana:1 ()' desktop is openindiana:1 Starting applications specified in /jack./.vnc/startup Log file is /jack/.vnc/openindiana:1.log $
Now simply use a VNC client on your workstation to connect to openindiana:1 via VNC. Depending on your local environment you might need to use the IP address instead of the name. Also please keep in mind that if you execute the VNC server as root (by doing a sudo -s) then you will be logged on as root. Which means you won’t have an icon from which to install OpenIndiana, you can still execute it manually… However at some point your screen could lock and you will be unable to unlock it. So long story short execute it as jack.
Install OpenIndiana
Now everything should be setup in such a way that you are able to actually perform the installation of OpenIndiana. So go ahead and do it.
Switch to the Running Configuration
So once we are done, just shutdown the machine. Once it is safely shutdown we can proceed.
Destroy the DomU which is configured for the LiveCD.
# xm destroy openindiana
Create the DomU which is configured to boot from the disk.
# xm create openindiana-run Using config file "./openindiana-run". Started domain openindiana
Now once it is up and running you can go about your business of configuring your DomU services.