Solaris 11: Defining A Service Using Svcbundle and the SMF

Lately I have been going through an exercise of migrating all of my current services into Solaris 11 (more specifically a zone).  I have already been using similar technology on the Linux side, I use OpenVZ with a mix of Linux distributions.  However I would like to take advantage of ZFS and get a little more simplicity in management. One of the most important hurdles to overcome when making the move from Linux to Solaris is simply the execution of services or processes.  Now in Linux you would write a little init script and then set it to run at certain run levels by either using chkconfig or creating symbolic links into runlevel directories.  For an example of what these init scripts would look like you can see my article “Bash: Automatically Mount File Systems on a Volume Group if Present“.  Now I could simply port my script to run under Solaris (and use the existing rc.d symbolic linking which also exists), however this methodology only starts and stops services on a change of run level and it doesn’t actually do anything to ensure that these services stay up.

Enter the Service Management Facility (SMF) which is built into Solaris starting in Solaris 10.  Basically the idea is that starting and stopping services is similar enough from service to service, that why should we be writing/modifying scripts from service to service.  Why not simply tell it what command to run when starting and then the SMF does the heavy lifting.  The biggest benefit of this is that the SMF can work with other Solaris components to detect the failure of a service and then automatically start it again, making these services more resilient.  This is accomplished by simply acknowledging that services are not simply user processes and thus should not be treated as such.

Now as an example I have moved my mercurial repositories from Linux to a Solaris zone, mercurial has a functionality which can share repositories via HTTP.  This can be done ad-hoc by simply executing the following command.

Determine the Command to Execute

$ hg serve

This will share via HTTP on port 8000.  Now this isn’t entirely how I want it to operate so lets start by creating a configuration file to change some options.

root@source:~# more /etc/mercurial/hgpub.config
[paths]
hg/ = /rpool/repo/hg/pub/*

[web]
allow_archive = gz

So now to reference the above configuration file we need to modify the command we will execute.

$ hg serve --webdir-conf /etc/mercurial/hgpub.conf

Now we are getting closer.  Another thing I like to do is define the port number, even though I am using the default of 8000.

$ hg serve --webdir-conf /etc/mercurial/hgpub.conf -p 8000

Now we need to use the daemon mode flag to start this in the background and not the foreground.

$ hg serve --webdir-conf /etc/mercurial/hgpub.conf -p 8000 -d

A final test will reveal that we are ready to turn this process into a service.  Kill the service that you manually started.

Create a SMF Bundle

SMF Bundles are XML files which will tell the SMF what to do and when.  You can either copy one that is for another service and modify it to suit or you can use svcbundle to create one specific to your requirements.

# svcbundle -o hgpub.xml -s service-name=application/hgpub -s model=daemon -s start-method="/usr/bin/hg serve --webdir-conf /etc/mercurial/hgpub.config -p 8000 -d"

Inspect and Modify the SMF Bundle

# cat hgpub.xml
<?xml version="1.0" ?>
<!DOCTYPE service_bundle
SYSTEM '/usr/share/lib/xml/dtd/service_bundle.dtd.1'>
<!--
Manifest created by svcbundle (2013-Jan-04 21:10:40-0600)
-->
<service_bundle type="manifest" name="application/hgpub">
<service version="1" type="service" name="application/hgpub">
<!--
The following dependency keeps us from starting until the
multi-user milestone is reached.
-->
<dependency restart_on="none" type="service"
name="multi_user_dependency" grouping="require_all">
<service_fmri value="svc:/milestone/multi-user"/>
</dependency>
<exec_method timeout_seconds="60" type="method" name="start"
exec="/usr/bin/hg serve --webdir-conf /etc/mercurial/hgpub.config -p 8000 -d"
/>
<!--
The exec attribute below can be changed to a command that SMF
should execute to stop the service.  See smf_method(5) for more
details.
-->
<exec_method timeout_seconds="60" type="method" name="stop"
exec=":kill"/>
<!--
The exec attribute below can be changed to a command that SMF
should execute when the service is refreshed.  Services are
typically refreshed when their properties are changed in the
SMF repository.  See smf_method(5) for more details.  It is
common to retain the value of :true which means that SMF will
take no action when the service is refreshed.  Alternatively,
you may wish to provide a method to reread the SMF repository
and act on any configuration changes.
-->
<exec_method timeout_seconds="60" type="method" name="refresh"
exec=":true"/>
<!--
We do not need a duration property group, because contract is
the default.  Search for duration in svc.startd(1M).
-->
<instance enabled="true" name="default"/>
<template>
<common_name>
<loctext xml:lang="C">
<!--
Replace this comment with a short name for the
service.
-->
</loctext>
</common_name>
<description>
<loctext xml:lang="C">
<!--
Replace this comment with a brief description of
the service
-->
</loctext>
</description>
</template>
</service>
</service_bundle>

Here we can add a service common name and description to make our service a little easier to use.

1# cat hg-pub.xml
<?xml version="1.0" ?>
<!DOCTYPE service_bundle
SYSTEM '/usr/share/lib/xml/dtd/service_bundle.dtd.1'>
<!--
Manifest created by svcbundle (2013-Jan-11 11:43:52-0600)
-->
<service_bundle type="manifest" name="application/hgpub">
<service version="1" type="service" name="application/hgpub">
<!--
The following dependency keeps us from starting until the
multi-user milestone is reached.
-->
<dependency restart_on="none" type="service"
name="multi_user_dependency" grouping="require_all">
<service_fmri value="svc:/milestone/multi-user"/>
</dependency>
<exec_method timeout_seconds="60" type="method" name="start"
exec="/usr/bin/hg serve --webdir-conf /etc/mercurial/hgpub.config -p 8000 -d"
/>
<!--
The exec attribute below can be changed to a command that SMF
should execute to stop the service.  See smf_method(5) for more
details.
-->
<exec_method timeout_seconds="60" type="method" name="stop"
exec=":kill"/>
<!--
The exec attribute below can be changed to a command that SMF
should execute when the service is refreshed.  Services are
typically refreshed when their properties are changed in the
SMF repository.  See smf_method(5) for more details.  It is
common to retain the value of :true which means that SMF will
take no action when the service is refreshed.  Alternatively,
you may wish to provide a method to reread the SMF repository
and act on any configuration changes.
-->
<exec_method timeout_seconds="60" type="method" name="refresh"
exec=":true"/>
<!--
We do not need a duration property group, because contract is
the default.  Search for duration in svc.startd(1M).
-->
<instance enabled="true" name="default"/>
<template>
<common_name>
<loctext xml:lang="C">
hgpub
</loctext>
</common_name>
<description>
<loctext xml:lang="C">
Mercurial Web Server for Public Repositories
</loctext>
</description>
</template>
</service>
</service_bundle>

Import the Service Bundle

# mv hgpub.xml /lib/svc/manifest/site

Now just restart the manifest-import service to pick up the service.

# svcadm restart manifest-import

Now we can check for unhealthy services by using the below command.

# svcs -xv
svc:/system/manifest-import:default (service manifest import)
State: offline since January 11, 2013 12:02:15 PM CST
Reason: Start method is running.
See: http://support.oracle.com/msg/SMF-8000-C4
See: man -M /usr/share/man -s 5 smf_bootstrap
See: /var/svc/log/system-manifest-import:default.log
Impact: This service is not running.

Above shows you what you will see if you check the services to quickly, you will see the manifest-import service itself in a non-running state, this is due to the import.  Give it a couple seconds and try again and it should clear.

I also saw errors when I did the following.

  • Created Services with port conflicts
  • Forgot to put the service into daemon mode in the command (use & or nohup for utilities/servers which do not have a daemon mode parameter)
  • Created Services with duplicate names

Below is an example of the output that I got when I forgot to put the service into daemon mode.  The log file mentioned gives much more information on what is actually happening, in this case it is starting it, waiting, and then killing it, over and over again.

# svcs -xv
svc:/application/hgpub:default (hgpub)
State: offline* transitioning to online since January 11, 2013 11:58:49 AM CST
Reason: Start method is running.
See: http://support.oracle.com/msg/SMF-8000-C4
See: /var/svc/log/application-hgpub:default.log
Impact: This service is not running.

If you make a mistake and want to start over then you need to

  1. Disable the service (svcadm disable hgpub)
  2. Delete the service (svccfg delete hgpub)
  3. Delete the service bundle in /lib/svc/manifest/site/ (rm /lib/svc/manifest/site/hgpub.xml)
  4. Restart the manifest-import service (svcadm restart manifest-import) - I think a refresh would work here as well, but I have not tested that

REF: http://mercurial.selenic.com/wiki/hgserve

Comments are closed.