August 8th, 2012 | Tags: , , , , ,

In our previous articles we built the foundation of our Logical Domains environment, which then enabled us to create a Logical Domain in part two.  Now in the final article in this series we will be connecting to and performing the installation of the operating system.

Solaris Virtualization: Using Logical Domains on Solaris 11 Part One
Solaris Virtualization: Using Logical Domains on Solaris 11 Part Two

Before we start the domain, we need to bind the associated resources to the domain.  This will also show us the port that the console is running on, so that we can connect to it.

root@t4:~# ldm list
primary          active     -n-cv-  UART    4     4G       0.8%  2h 44m
t4-g1            inactive   ------          8     4G
root@t4:~# ldm bind-domain t4-g1
root@t4:~# ldm list
primary          active     -n-cv-  UART    4     4G       3.0%  2h 45m
t4-g1            bound      ------  5000    8     4G    

In the above example we can see that once we “bind-domain” this is binding specific hardware resources to the domain.  In this case we can now see that we have a port for the console connections.  For this domain it is 5000.  So if we telnet to port 5000 on the localhost we will end up connected to the console.  I personally like to connect prior to starting the domain, because I prefer to watch the boot up process, so that I know everything is working properly, this means I will have a separate ssh session which I use for telnet.

root@t4:~# telnet localhost 5000
Trying ::1...
telnet: connect to address ::1: Connection refused
Connected to t4.
Escape character is '^]'.

Connecting to console "t4-g1" in group "t4-g1" ....
Press ~? for control options ..

Start the domain.

root@t4:~# ldm start-domain t4-g1
LDom t4-g1 started

Observe boot process.  During the below boot process we can see a couple of things.  Most importantly we see that the boot fails, with a “Can’t open boot device” this is expected as we are attempting to boot from disk0 which has not been installed yet.

SPARC T4-1, No Keyboard
Copyright (c) 1998, 2011, Oracle and/or its affiliates. All rights reserved.
OpenBoot 4.33.1, 4096 MB memory available, Serial #83496026.
Ethernet address 0:14:4f:fa:c:5a, Host ID: 84fa0c5a.

Boot device: disk0  File and args:
Bad magic number in disk label
ERROR: /virtual-devices@100/channel-devices@200/disk@0: Can't open disk label package

ERROR: boot-read fail


Can't open boot device

{0} ok

We will be dumped to the ok prompt for the domain, from here we can simply select to boot from the iso that you defined earlier in the domain setup.  In my case I named the device s11.iso, so if you do a devalias you would see s11.iso as an alias to a device.

{0} ok boot s11.iso

Just a quick note, the escape character for telnet is listed as ‘^]’ in the beginning of the telnet session, this is CTRL + ] once you have returned the the telnet> prompt you can type quit.

Now in my environment I went through the install of Solaris 11, since it was the disk I had readily available, you can additionally use Soalris 10, or OpenSolaris if you so desire.

August 7th, 2012 | Tags: , , , , ,

In our previous article we went through the process of preparing a Solaris 11 for SPARC machine as a Logical Domain hypervisor.  Now that we have laid out our foundation we are ready to create some Logical Domains.

Solaris Virtualization: Using Logical Domains on Solaris 11 Part One

Create the Logical Domain

root@t4:~# ldm add-domain t4-g1

Configure Devices

We will need to add devices to our newly created domain.  Here we will add virtual CPUs, memory, and a network device.

root@t4:~# ldm add-vcpu 8 t4-g1
root@t4:~# ldm add-memory 4G t4-g1
root@t4:~# ldm add-vnet vnet1 primary-vsw0 t4-g1

Configure Storage Devices

Add the installation media to our domain using an iso image which we added to the Virtual Disk Service service which we created in Part One.

root@t4:~# ldm add-vdisk s11.iso solaris11_media@primary-vds0 t4-g1

Create a couple of ZFS volumes for use by the domain, we are using the -p option to create the parent file system with the name of the logical domain for organization.

root@t4:~# zfs create -p -V 20G rpool/ldoms/t4-g1/disk0
root@t4:~# zfs create -p -V 20G rpool/ldoms/t4-g1/disk1

Add the ZFS volumes to the Virtual Disk Server service (primary-vds0).

root@t4:~# ldm add-vdsdev /dev/zvol/dsk/rpool/ldoms/t4-g1/disk0 t4-g1-disk0@primary-vds0
root@t4:~# ldm add-vdsdev /dev/zvol/dsk/rpool/ldoms/t4-g1/disk1 t4-g1-disk1@primary-vds0

Configure our domain with our disk storage devices (disk0 and disk1) from the Virtual Disk Server service.

root@t4:~# ldm add-vdisk disk0 t4-g1-disk0@primary-vds0 t4-g1
root@t4:~# ldm add-vdisk disk1 t4-g1-disk1@primary-vds0 t4-g1

Configure Boot Devices

We want our domain to auto-boot and use disk0 as the primary boot device.

root@t4:~# ldm set-var auto-boot\?=true t4-g1
root@t4:~# ldm set-var boot-device=disk0 t4-g1

Set the Host ID

root@t4:~# ldm set-domain hostid=85e92821 t4-g1

Additionally the hostid option can be passed in the add-domain command when you originally create the domain.

Tomorrow we will go through the process of connecting to the console of the Logical Domain and then install an operating system.


Comments Off
August 6th, 2012 | Tags: , , , , ,

Solaris 11 for SPARC, as well as previous versions of Solaris, include a neat little technology called Logical Domains (which has been rebranded as Oracle VM for SPARC).  Logical Domains are implemented in the SPARC processor.

This series is going to focus on the basic administration of a Logical Domains environment using the basic command line tools of the Solaris 11 Operating System.  In Part One we will go over what it takes to configure a Solaris 11 installation to be a Logical Domains hypervisor.  In Part Two we will document defining and configuring Logical Domains.  In Part Three we will discuss connecting to the Logical Domain in order to perform an installation of the operating system.

A little bit about my environment.  I am using a Oracle Sun SPARC T4-1 which has a single socket eight-core 2.85GHz SPARC T4 Processor.  Each processor core on this model has 8 threads, giving us a total of 64 processor threads.  Lets take a look at our environment shall we?

Check Operating System Version

This is the released version of Solaris 11 for SPARC, which of course means it can only be used on the SPARC processor architecture.

root@t4:~# uname -a
SunOS t4 5.11 11.0 sun4v sparc sun4v

View Logical Domains Software Version

The release version of Solaris 11 for SPARC comes with Logical Domains 2.1.  If you have purchased support and configured access to the repositories then pkg update will give you access to Logical Domains 2.2.

root@t4:~# ldm -V

Logical Domains Manager (v
Hypervisor control protocol v 1.7
Using Hypervisor MD v 1.3

System PROM:
Hostconfig      v. 1.1.1        @(#)Hostconfig 1.1.1 2011/08/03 23:04
Hypervisor      v. 1.10.1.      @(#)Hypervisor 1.10.1.b 2011/09/12 09:56
OpenBoot        v. 4.33.1       @(#)OpenBoot 4.33.1 2011/08/03 10:34

View Existing Logical Domains

Here we see the specifics of our physical machine.  The primary domain has all of the resources and before we will be able to allocate resources to other Logical domains we will need to remove some from the primary domain.

root@t4:~# ldm list
primary          active     -n-c--  UART    64    32256M   0.0%  4d 23h 30m

Create Services for Control Domain

Here we are creating the Virtual Console Concentrator service.  This is what allows us to connect to the console of the domains for installation and reconfiguration.

root@t4:~# ldm add-vcc port-range=5000-5100 primary-vcc0 primary

This is the Virtual Disk Server service.  This is used to hand out disk devices to the domains.

root@t4:~# ldm add-vds primary-vds0 primary

This is the Virtual Switch service.  It provides networking to the domains.

root@t4:~# ldm add-vsw net-dev=net0 primary-vsw0 primary

Lets review our changes.

root@t4:~# ldm list-services primary
NAME             LDOM             PORT-RANGE
primary-vcc0     primary          5000-5100

NAME             LDOM             MAC               NET-DEV   ID   DEVICE     LINKPROP   DEFAULT-VLAN-ID PVID VID                  MTU   MODE   INTER-VNET-LINK
primary-vsw0     primary          00:14:4f:fb:bd:bd net0      0    switch@0              1               1                         1500         on

NAME             LDOM             VOLUME         OPTIONS          MPGROUP        DEVICE
primary-vds0     primary  

Configure the Control Domain

Look at the existing domains.  By default we have primary, which is essentially the host operating system.

root@t4:~# ldm list
primary          active     -n-c--  UART    64    32256M   0.0%  5d 53m

Reduce the number of vCPU available to the control domain.

root@t4:~# ldm set-vcpu 4 primary

Reduce the amount of RAM available to the control domain.

root@t4:~# ldm set-memory 4G primary

Reconfigure primary domain to reflect changes.

root@t4:~# ldm start-reconf primary
Initiating a delayed reconfiguration operation on the primary domain.
All configuration changes for other domains are disabled until the primary
domain reboots, at which time the new configuration for the primary domain
will also take effect.

Create alternate configuration file.

root@t4:~# ldm add-config primary-config
root@t4:~# ldm list-config
primary-config [current]

Enable the Virtual Network Terminal Server Daemon.

root@t4:~# svcadm enable vntsd
root@t4:~# svcs | grep vntsd
online         16:40:01 svc:/ldoms/vntsd:default


root@t4:~# reboot

After a reboot, the primary domain reflects the new memory and vCPU configuration.

root@t4:~# ldm list
primary          active     -n-cv-  UART    4     4G       0.4%  1h 4m

Create Install Media Repository

Create a ZFS file system to hold the iso files.  The -p will create the parent file system /rpool/ldoms as well.

root@t4:~# zfs create -p rpool/ldoms/iso

Add the Solaris 11 and Solaris 10 iso files to the Virtual Disk Server service.

root@t4:~# ldm add-vdsdev /rpool/ldoms/iso/oracle-solaris-11-11.11-sparc.iso solaris11_media@primary-vds0
root@t4:~# ldm add-vdsdev /rpool/ldoms/iso/oracle-solaris-10-8.11-sparc.iso solaris10_media@primary-vds0

Now we have a working environments to support the Logical Domains that we will be adding tomorrow in part two.

Comments Off
August 2nd, 2012 | Tags: , , , , , , , ,

In my laptop I have a solid state disk, and frankly I am way past addicted to solid state disks, but what you get in performance you lose in capacity.  As such I have had to be creative with how I can have the capacity and the performance that I need.  Lately the solution has been to have a spinning disk in the slot that would normally be occupied by my CD/DVD drive.  However the problem with this, revealed itself to be that my Fedora install would actually fail to boot when it tried and failed to mount the file systems if this device was not present.  So my first “solution” was to set noauto in the /etc/fstab for these secondary file systems.  Which presented a new problem…  I would forget to manually mount these devices before launching Virtualbox to start my VMs.  So I decided that I would need to have an auto-mount script which would check for the volume group to be present and then if it was mount the file systems contained within the /etc/fstab which live on the volume group.

Create the Service Script

Below you will see the script I created.  This script will test for the presence of a volume group, and if present on invocation (boot up or manual) then it will mount each logical volume defined in the /etc/fstab of the present volume group.

Name       :
Version   :  1.0.3
MD5        :  0b8cf3b0b5c9d3c16e4384fde88e8659
SHA256  :  877ef66ab464e71172986fa9cd5510f0ad6752f05a414af703878d7e058c845b
URL         :

# chkconfig: 345 85 15
# description: Will automatically mount a removable device if present.
#: Script Name    :
#: Version    : 1.0.3
#: Author    : Matthew Mattoon -
#: Date Created    : August 15, 2012
#: Date Updated    : February 20, 2013
#: Description    : Automount Removable Logical Volumes Script.
#: Examples    : ACTION
#:         : start

vgs=`vgs | grep $vgtest`

case "$1" in
if [ -n "$vgs" ]
echo "Logical Volume Group: $vgtest present."
mounts=`cat /etc/fstab | grep $vgtest | grep noauto | tr '\t' ' ' | tr -s ' ' | cut -d " " -f 2`
for mount in $mounts
if [ -z "`mount | grep $mount`" ]
echo "Mounting $mount file system..."
mount $mount
echo "File system $mount is already mounted..."
echo "Logical Volume Group: $vgtest not present."
exit 1
if [ -n "$vgs" ]
echo "Logical Volume Group: $vgtest present."
mounts=`cat /etc/fstab | grep $vgtest | grep noauto | tr '\t' ' ' | tr -s ' ' | cut -d " " -f 2`
for mount in $mounts
if [ -n "`mount | grep $mount`" ]
echo "Unmounting $mount file system..."
umount $mount
echo "File system $mount is already unmounted..."
echo "Logical Volume Group: $vgtest not present."
exit 1
echo "Usage: $0 start"
exit 1

Create the Service to Auto Start

Based on my /etc/init.d/custom_mount script it will start in run levels 3, 4, and 5.

# chkconfig --add custom_mount
Comments Off
August 1st, 2012 | Tags: , ,

I have finally got my hands on some SPARC gear, and I am making use of it by documenting the crap out of it.  So after I got the ILOM situation sorted out I noticed that the OS itself wasn’t starting up, and I was just ending up at a prompt that looks like this.

{0} ok

This is the OK Prompt, some systems will display ok> and this is how I will refer to it in subsequent examples.  So lets dig in.

What is OpenBoot PROM?

The OpenBoot PROM (OK Prompt) is a boot monitor which allows us to interact with the boot environment, we can use it to pass different boot parameters.  We can use it to select a boot device, boot into various modes (verbose, single user, reconfigure).  There is much more to it then this, look up Open Firmware for more information.

Why am I Stuck at the OK Prompt?

Now in my situation, the system was not even attempting booting up.  That is because on this particular system auto_boot was disabled.  This can be seen by printing the environment…

ok> printenv
Variable Name           Value                          Default Value

auto-boot?              false                          true

Now in my case that is fine.  I don’t want this to boot automatically, but I still need to boot it, but before we do lets check our boot order.

Configuring Boot Devices

ok> printenv boot-device
boot-device =           disk net

We can change this easily enough if it is incorrect.  For example if we wanted to add a cdrom, but after the disk.

ok> setenv boot-device disk cdrom net
ok> printenv boot-device
boot-device =           disk cdrom net

But I don’t want to have a cdrom as part of the boot order, so instead I would rather boot up the device as an override.

Manually Select a Boot Device

ok> boot cdrom

This can also be done against any other device which is bootable, for example another disk.  But first lets find out what exactly we have for disks.

View Disks

ok> show-disks
a) /pci@400/pci@2/pci@0/pci@f/pci@0/usb@0,2/hub@2/hub@3/storage@2/disk
b) /pci@400/pci@2/pci@0/pci@4/scsi@0/disk
c) /pci@400/pci@1/pci@0/pci@4/scsi@0/disk
d) /iscsi-hba/disk
Enter Selection, q to quit: q

So we can see here we have 2 disks:


Now with this information, we still haven’t made the connection between our boot-device which is “disk cdrom net” in order to define these names we need to look at devalias.

Just Call me Disk.  A Lesson in aliases.

Devalias allows us to define names for our devices, which can be more easily referred to.  Below is trimmed for brevity.

ok> devalias
net0                     /pci@400/pci@2/pci@0/pci@6/network@0
net                      /pci@400/pci@2/pci@0/pci@6/network@0
disk4                    /pci@400/pci@2/pci@0/pci@4/scsi@0/disk@p0
cdrom                    /pci@400/pci@2/pci@0/pci@4/scsi@0/disk@p6
disk0                    /pci@400/pci@1/pci@0/pci@4/scsi@0/disk@p0
disk                     /pci@400/pci@1/pci@0/pci@4/scsi@0/disk@p0

Now when we look at this output we notice a few things.

1) @p0 on disks – I believe this refers to partitions on the disk.  p0 being the whole disk.
2) Multiple aliases to the same underlying device, case in point disk and disk0 as well as net and net0.

So we can create aliases to denote a primary and a mirror disk on our two disks.

ok> devalias primary /pci@400/pci@1/pci@0/pci@4/scsi@0/disk@p0
ok> devalias mirror /pci@400/pci@2/pci@0/pci@4/scsi@0/disk@p0
ok> devalias
mirror                   /pci@400/pci@2/pci@0/pci@4/scsi@0/disk@p0
primary                  /pci@400/pci@1/pci@0/pci@4/scsi@0/disk@p0

Important to note that when you create an alias using devalias it doesn’t get committed, and will disappear after a reboot…  If you need a persistent alias, use nvalias.

ok> nvalias primary /pci@400/pci@1/pci@0/pci@4/scsi@0/disk@p0
ok> nvalias mirror /pci@400/pci@1/pci@0/pci@4/scsi@0/disk@p0

Then of course we can change our boot-device to reflect this.

ok> setenv boot-device primary mirror

Now at some point you might need to remove an alias.

ok> nvunalias mirror
nvunalias primary

To finalize the removal, we need to reset the machine.

ok> reset-all

There is much more to the OK prompt, but this ought to get you started.  Enjoy.

Comments Off
Page 10 of 28« First...89101112...20...Last »