August 6th, 2012 | Tags: , , , , ,

Solaris 11 for SPARC, as well as previous versions of Solaris, include a neat little technology called Logical Domains (which has been rebranded as Oracle VM for SPARC).  Logical Domains are implemented in the SPARC processor.

This series is going to focus on the basic administration of a Logical Domains environment using the basic command line tools of the Solaris 11 Operating System.  In Part One we will go over what it takes to configure a Solaris 11 installation to be a Logical Domains hypervisor.  In Part Two we will document defining and configuring Logical Domains.  In Part Three we will discuss connecting to the Logical Domain in order to perform an installation of the operating system.

A little bit about my environment.  I am using a Oracle Sun SPARC T4-1 which has a single socket eight-core 2.85GHz SPARC T4 Processor.  Each processor core on this model has 8 threads, giving us a total of 64 processor threads.  Lets take a look at our environment shall we?

Check Operating System Version

This is the released version of Solaris 11 for SPARC, which of course means it can only be used on the SPARC processor architecture.

root@t4:~# uname -a
SunOS t4 5.11 11.0 sun4v sparc sun4v

View Logical Domains Software Version

The release version of Solaris 11 for SPARC comes with Logical Domains 2.1.  If you have purchased support and configured access to the repositories then pkg update will give you access to Logical Domains 2.2.

root@t4:~# ldm -V

Logical Domains Manager (v
Hypervisor control protocol v 1.7
Using Hypervisor MD v 1.3

System PROM:
Hostconfig      v. 1.1.1        @(#)Hostconfig 1.1.1 2011/08/03 23:04
Hypervisor      v. 1.10.1.      @(#)Hypervisor 1.10.1.b 2011/09/12 09:56
OpenBoot        v. 4.33.1       @(#)OpenBoot 4.33.1 2011/08/03 10:34

View Existing Logical Domains

Here we see the specifics of our physical machine.  The primary domain has all of the resources and before we will be able to allocate resources to other Logical domains we will need to remove some from the primary domain.

root@t4:~# ldm list
primary          active     -n-c--  UART    64    32256M   0.0%  4d 23h 30m

Create Services for Control Domain

Here we are creating the Virtual Console Concentrator service.  This is what allows us to connect to the console of the domains for installation and reconfiguration.

root@t4:~# ldm add-vcc port-range=5000-5100 primary-vcc0 primary

This is the Virtual Disk Server service.  This is used to hand out disk devices to the domains.

root@t4:~# ldm add-vds primary-vds0 primary

This is the Virtual Switch service.  It provides networking to the domains.

root@t4:~# ldm add-vsw net-dev=net0 primary-vsw0 primary

Lets review our changes.

root@t4:~# ldm list-services primary
NAME             LDOM             PORT-RANGE
primary-vcc0     primary          5000-5100

NAME             LDOM             MAC               NET-DEV   ID   DEVICE     LINKPROP   DEFAULT-VLAN-ID PVID VID                  MTU   MODE   INTER-VNET-LINK
primary-vsw0     primary          00:14:4f:fb:bd:bd net0      0    switch@0              1               1                         1500         on

NAME             LDOM             VOLUME         OPTIONS          MPGROUP        DEVICE
primary-vds0     primary  

Configure the Control Domain

Look at the existing domains.  By default we have primary, which is essentially the host operating system.

root@t4:~# ldm list
primary          active     -n-c--  UART    64    32256M   0.0%  5d 53m

Reduce the number of vCPU available to the control domain.

root@t4:~# ldm set-vcpu 4 primary

Reduce the amount of RAM available to the control domain.

root@t4:~# ldm set-memory 4G primary

Reconfigure primary domain to reflect changes.

root@t4:~# ldm start-reconf primary
Initiating a delayed reconfiguration operation on the primary domain.
All configuration changes for other domains are disabled until the primary
domain reboots, at which time the new configuration for the primary domain
will also take effect.

Create alternate configuration file.

root@t4:~# ldm add-config primary-config
root@t4:~# ldm list-config
primary-config [current]

Enable the Virtual Network Terminal Server Daemon.

root@t4:~# svcadm enable vntsd
root@t4:~# svcs | grep vntsd
online         16:40:01 svc:/ldoms/vntsd:default


root@t4:~# reboot

After a reboot, the primary domain reflects the new memory and vCPU configuration.

root@t4:~# ldm list
primary          active     -n-cv-  UART    4     4G       0.4%  1h 4m

Create Install Media Repository

Create a ZFS file system to hold the iso files.  The -p will create the parent file system /rpool/ldoms as well.

root@t4:~# zfs create -p rpool/ldoms/iso

Add the Solaris 11 and Solaris 10 iso files to the Virtual Disk Server service.

root@t4:~# ldm add-vdsdev /rpool/ldoms/iso/oracle-solaris-11-11.11-sparc.iso solaris11_media@primary-vds0
root@t4:~# ldm add-vdsdev /rpool/ldoms/iso/oracle-solaris-10-8.11-sparc.iso solaris10_media@primary-vds0

Now we have a working environments to support the Logical Domains that we will be adding tomorrow in part two.

Comments Off
August 2nd, 2012 | Tags: , , , , , , , ,

In my laptop I have a solid state disk, and frankly I am way past addicted to solid state disks, but what you get in performance you lose in capacity.  As such I have had to be creative with how I can have the capacity and the performance that I need.  Lately the solution has been to have a spinning disk in the slot that would normally be occupied by my CD/DVD drive.  However the problem with this, revealed itself to be that my Fedora install would actually fail to boot when it tried and failed to mount the file systems if this device was not present.  So my first “solution” was to set noauto in the /etc/fstab for these secondary file systems.  Which presented a new problem…  I would forget to manually mount these devices before launching Virtualbox to start my VMs.  So I decided that I would need to have an auto-mount script which would check for the volume group to be present and then if it was mount the file systems contained within the /etc/fstab which live on the volume group.

Create the Service Script

Below you will see the script I created.  This script will test for the presence of a volume group, and if present on invocation (boot up or manual) then it will mount each logical volume defined in the /etc/fstab of the present volume group.

Name       :
Version   :  1.0.3
MD5        :  0b8cf3b0b5c9d3c16e4384fde88e8659
SHA256  :  877ef66ab464e71172986fa9cd5510f0ad6752f05a414af703878d7e058c845b
URL         :

# chkconfig: 345 85 15
# description: Will automatically mount a removable device if present.
#: Script Name    :
#: Version    : 1.0.3
#: Author    : Matthew Mattoon -
#: Date Created    : August 15, 2012
#: Date Updated    : February 20, 2013
#: Description    : Automount Removable Logical Volumes Script.
#: Examples    : ACTION
#:         : start

vgs=`vgs | grep $vgtest`

case "$1" in
if [ -n "$vgs" ]
echo "Logical Volume Group: $vgtest present."
mounts=`cat /etc/fstab | grep $vgtest | grep noauto | tr '\t' ' ' | tr -s ' ' | cut -d " " -f 2`
for mount in $mounts
if [ -z "`mount | grep $mount`" ]
echo "Mounting $mount file system..."
mount $mount
echo "File system $mount is already mounted..."
echo "Logical Volume Group: $vgtest not present."
exit 1
if [ -n "$vgs" ]
echo "Logical Volume Group: $vgtest present."
mounts=`cat /etc/fstab | grep $vgtest | grep noauto | tr '\t' ' ' | tr -s ' ' | cut -d " " -f 2`
for mount in $mounts
if [ -n "`mount | grep $mount`" ]
echo "Unmounting $mount file system..."
umount $mount
echo "File system $mount is already unmounted..."
echo "Logical Volume Group: $vgtest not present."
exit 1
echo "Usage: $0 start"
exit 1

Create the Service to Auto Start

Based on my /etc/init.d/custom_mount script it will start in run levels 3, 4, and 5.

# chkconfig --add custom_mount
Comments Off
August 1st, 2012 | Tags: , ,

I have finally got my hands on some SPARC gear, and I am making use of it by documenting the crap out of it.  So after I got the ILOM situation sorted out I noticed that the OS itself wasn’t starting up, and I was just ending up at a prompt that looks like this.

{0} ok

This is the OK Prompt, some systems will display ok> and this is how I will refer to it in subsequent examples.  So lets dig in.

What is OpenBoot PROM?

The OpenBoot PROM (OK Prompt) is a boot monitor which allows us to interact with the boot environment, we can use it to pass different boot parameters.  We can use it to select a boot device, boot into various modes (verbose, single user, reconfigure).  There is much more to it then this, look up Open Firmware for more information.

Why am I Stuck at the OK Prompt?

Now in my situation, the system was not even attempting booting up.  That is because on this particular system auto_boot was disabled.  This can be seen by printing the environment…

ok> printenv
Variable Name           Value                          Default Value

auto-boot?              false                          true

Now in my case that is fine.  I don’t want this to boot automatically, but I still need to boot it, but before we do lets check our boot order.

Configuring Boot Devices

ok> printenv boot-device
boot-device =           disk net

We can change this easily enough if it is incorrect.  For example if we wanted to add a cdrom, but after the disk.

ok> setenv boot-device disk cdrom net
ok> printenv boot-device
boot-device =           disk cdrom net

But I don’t want to have a cdrom as part of the boot order, so instead I would rather boot up the device as an override.

Manually Select a Boot Device

ok> boot cdrom

This can also be done against any other device which is bootable, for example another disk.  But first lets find out what exactly we have for disks.

View Disks

ok> show-disks
a) /pci@400/pci@2/pci@0/pci@f/pci@0/usb@0,2/hub@2/hub@3/storage@2/disk
b) /pci@400/pci@2/pci@0/pci@4/scsi@0/disk
c) /pci@400/pci@1/pci@0/pci@4/scsi@0/disk
d) /iscsi-hba/disk
Enter Selection, q to quit: q

So we can see here we have 2 disks:


Now with this information, we still haven’t made the connection between our boot-device which is “disk cdrom net” in order to define these names we need to look at devalias.

Just Call me Disk.  A Lesson in aliases.

Devalias allows us to define names for our devices, which can be more easily referred to.  Below is trimmed for brevity.

ok> devalias
net0                     /pci@400/pci@2/pci@0/pci@6/network@0
net                      /pci@400/pci@2/pci@0/pci@6/network@0
disk4                    /pci@400/pci@2/pci@0/pci@4/scsi@0/disk@p0
cdrom                    /pci@400/pci@2/pci@0/pci@4/scsi@0/disk@p6
disk0                    /pci@400/pci@1/pci@0/pci@4/scsi@0/disk@p0
disk                     /pci@400/pci@1/pci@0/pci@4/scsi@0/disk@p0

Now when we look at this output we notice a few things.

1) @p0 on disks – I believe this refers to partitions on the disk.  p0 being the whole disk.
2) Multiple aliases to the same underlying device, case in point disk and disk0 as well as net and net0.

So we can create aliases to denote a primary and a mirror disk on our two disks.

ok> devalias primary /pci@400/pci@1/pci@0/pci@4/scsi@0/disk@p0
ok> devalias mirror /pci@400/pci@2/pci@0/pci@4/scsi@0/disk@p0
ok> devalias
mirror                   /pci@400/pci@2/pci@0/pci@4/scsi@0/disk@p0
primary                  /pci@400/pci@1/pci@0/pci@4/scsi@0/disk@p0

Important to note that when you create an alias using devalias it doesn’t get committed, and will disappear after a reboot…  If you need a persistent alias, use nvalias.

ok> nvalias primary /pci@400/pci@1/pci@0/pci@4/scsi@0/disk@p0
ok> nvalias mirror /pci@400/pci@1/pci@0/pci@4/scsi@0/disk@p0

Then of course we can change our boot-device to reflect this.

ok> setenv boot-device primary mirror

Now at some point you might need to remove an alias.

ok> nvunalias mirror
nvunalias primary

To finalize the removal, we need to reset the machine.

ok> reset-all

There is much more to the OK prompt, but this ought to get you started.  Enjoy.

Comments Off
July 31st, 2012 | Tags: , , ,

In Solaris 11, a secondary user is created as part of the installation process, the text installation allows you to bypass this, but even if you do you will be unable to login as root to perform any configurations, even though you set the password as part of the setup.  Now if you went through this in the installation, you can salvage your installation by booting into single user mode.  But for those of us who just want to manage this box differently the steps are the same (minus the single user mode).

Configure SSH to Permit Root Logins

# vi /etc/ssh/sshd_config

Here we want to find “PermitRootLogin” and flip it to yes.

# svcadm restart ssh

Then restart SSH.  SSH will now accept root as a login account, however root is still set as a role.

# vi /etc/user_attr

Here we want to find the line which is for the user root, and remove the last portion of the line which says “type=role”.

That is it.  You can now login directly on the console or via SSH using the root account and password you set during installation.  Of course you can reset the password using passwd as well if need be.


Comments Off
June 11th, 2012 | Tags: , , , ,

In my new role I am using Oracle Linux more and more, so as an exercise I have been replicating my builds on Oracle Linux.  Here we are going to use Oracle Linux 6.2 x86_64 as a KVM hypervisor.  Keep in mind that KVM is not Oracle’s preferred hypervisor, however it is supported, but if you plan on taking advantage of the reduced software licensing cost by limiting your cores for Oracle Products you will need to use Oracle VM, which is built on Xen not KVM.  Read on to find out how to configure Oracle Linux as a KVM hypervisor.

Configure Oracle Linux Repositories

You can use Oracle’s Public Yum server.  This provides free software updates and a package source for Oracle Linux.  Keep in mind, these updates do not require a support contract and thus do not come with any support.  If you need support you can buy it from the Oracle Store.

# cd /etc/yum.repos.d/
# wget

Install Latest Updates

I installed all of the latest updates, it came out to be around 120 updates.  And it brought me up to the RHEL-Compatible Kernel 2.6.32-220.17.1.el6 and UEK 2.6.32-300.25.1.el6uek.

#yum update

Configure Ethernet Bridge

# cat ifcfg-br0
# cat ifcfg-eth1
# /etc/init.d/network restart
Shutting down interface eth1:  bridge br0 does not exist!
[  OK  ]
Shutting down loopback interface:                          [  OK  ]
Bringing up loopback interface:                            [  OK  ]
Bringing up interface eth1:                                [  OK  ]
Bringing up interface br0:
Determining IP information for br0... done.
[  OK  ]

Install Required Software

The only things we really have to install is kvm, and libvirt if you want to use libvirt to manage it.

# yum install kvm libvirt

I also include the following software in my KVM builds as they add a lot of flexibility (vim and pv are not even hypervisor related but I use them extensively to edit xml and move disk images).

# yum install python-virtinst virt-top virt-manager virt-v2v virt-viewer vim pv

Once you have performed the install everything required for KVM is ready to go.  Next we need to define a VM, this can be done with virt-manager.  Or by using one of my previously documented methods.

Comments Off
Page 10 of 27« First...89101112...20...Last »