Friday, July 21, 2006

How small can you make Open Solaris - Part 5

In the latest post I have just done a quick update of the "Quick and Dirty Solaris Installer". Version 0.3 of the installer is now able to exclude Solaris clusters and packages from the installation. With this we are now able to start reducing the size and try to get close to what was achieved in the first couple of posts.

If you read through the code you can see that I have started with the SUNWCmreq metacluster, as it is the smallest defined on the Solaris installation media. I actually had to add the package SUNWmdr, as devfsadmd started complaining during the first reboot. In the code you will see a big list of packages defined in the "exclude array". Since this list includes device drivers, you may have to add and remove driver packages for your system. To do this you will need to read through the '.clustertoc' file in the Product directory of your installation media.

Using the configuration in this script, I was able to get a bootable multi-user system which only needed /etc/nodename and /etc/hosts configured. The disk usage is down to around 149MB (still a long way to go). Solaris core packages actually have a lot of files and command which I would not call 'core'. Later on I will look at creating new packages, and pulling files directly from a Open Solaris bfu archive. This may also require some files being pulled from "running" Solaris, as the bfu archives are not complete. If we can successfully use a bfu archive, then we could later see if changing some of the compiler flags in an Open Solaris build will shrink the binary size.

In the next post, I will start reducing installation further, by adding commands to the local_install.bash script which can be run just before the filesystem is un-mounted.


#!/bin/bash
#
# Quick and Dirty Solaris installer
# Version 0.3
#
PROD=/cdrom/sol_11_x86/Solaris_11/Product
#PROD=/a/Solaris_11/Product
TOK=${PROD}/.clustertoc
ORDER=${PROD}/.order

# Current metaclusters are
# SUNWCXall - Entire Distribution plus OEM support
# SUNWCall - Entire Distribution
# SUNWCprog - Developer System Support
# SUNWCuser - End User System Suppor
# SUNWCreq - Core System Support
# SUNWCrnet - Reduced Networking Core System Support
# SUNWCmreq - Minimal Core System Support
METACLUSTER=SUNWCmreq
export METACLUSTER

SWAP=/dev/dsk/c0d0s1
LOG=/tmp/install.log
SVCPROFILE=generic_limited_net.xml

#
# Change these to relect your system.
#
FS=zfs
[ "${FS}" != "zfs" -a "${FS}" != "ufs" ] && {
printf "Unknown filesystem ${FS}\n"
exit 1
}

[ "${FS}" = "zfs" ] && {
ROOTDEV=intdisk/snv43_zfs
RAWROOTDEV=-
GRUBFS=/dev/dsk/c0d0s0
ZFSBOOTARCHIVE=/grub/boot/boot_zfs
ZFSCOMPRESS=off
# ZFSCOMPRESS=on
MOB=yes
}

[ "${FS}" = "ufs" ] && {
ROOTDEV=/dev/dsk/c0d0s3
RAWROOTDEV=${ROOTDEV/dsk/rdsk}
MOB=no
}

typeset -a pkgs
# extrapkgs is an array of additional packages to be installed
typeset -a extrapkgs=( SUNWmdr )
# exclude is an array of packages or clusters which should not be installed
typeset -a exclude=( SUNWChbaapi SUNWCfcadb SUNWCfca SUNWCfcadb SUNWCfct
SUNWCfutil SUNWCiscsi SUNWCib SUNWCmpapi SUNWCtavor
CADP160 HPFC SK98sol SUNWaac SUNWadp SUNWadpu320 SUNWamr SUNWcadp SUNWced
SUNWchxge SUNWcqhpc SUNWlsimega SUNWmv88sx SUNWnge SUNWrge SUNWrmodr
SUNWrmodu SUNWrtls SUNWses SUNWsi3124 SUNWuksp SUNWuedg SUNWukspfw
SUNWuprl SUNWxge SYMhisl SKfp SUNWintgige SUNWwbsup
SUNWCpkgcmds SUNWjss SUNWidnl SUNWbzip
SUNWperl584core SUNWperl584usr )

#
# check_install tests whether $1 is in the list of packages to be excluded
#
function check_install() {
local i
[ "$#" != "1" ] && return 0

for i in ${exclude[@]} ; do
[ ${i} = "$1" ] && return 1
done
return 0
}

#
# add_extra_pkgs will add in the extrapkgs array to the pkgs array. It will
# void adding a package twice
#
function add_extra_pkgs() {
pkgcnt=${#pkgs[@]}
for i in ${extrapkgs[@]} ; do
found=0
for j in ${pkgs[@]} ; do
[ ${i} = ${j} ] && {
found=1
break;
}
done
# Add package if not already found
[ "${found}" = "0" ] && {
printf "Adding Package - %s [%d]\n" "${i}" ${pkgcnt}
pkgs[$(( pkgcnt++ ))]="$i"
}
done
}

#
# Solaris packages need to be installed in the correct order.
# The .order file contains all the packages in the correct
# installation order
#
function reorder_pkgs() {
typeset -a pkglist=( ${pkgs[@]} )
pkgcnt=0

while read order_pkg ; do
for i in ${pkglist[@]} ; do
[ "$order_pkg" = "${i%.i}" ] && {
pkgs[$(( pkgcnt++ ))]="${i}"
printf "."
}
done
done < ${ORDER}
}

#
# This function builds a list of packages in a cluster
# If there is a cluster within a cluster, it will call itself to
# resolve all the packages.
#
# Before calling make sure you initialize pkgcnt to 0
# Arg: $1 contains the cluster name
# Affected vars: pkgs, pkgcnt
#
function get_pkg_list() {
local IFS="="
local print_on=0
local cluster=$1

while read arg1 arg2
do
[ "${arg1}" = "END" -a "${print_on}" = "1" ] && break;
[ -z "${arg2}" ] && continue;

[ "${arg2}" = "${cluster}" ] && {
print_on=1
continue
}

[ "${print_on}" = "1" -a "${arg1}" = "SUNW_CSRMEMBER" ] && {
# Test to see if package/cluster is on the exclude list
check_install "${arg2}" || continue

ifcluster=`expr "${arg2}" : '\(SUNWC\)'`
if [ "${ifcluster}" = "SUNWC" ]; then
get_pkg_list ${arg2}
else
[ -d ${PROD}/${arg2} ] && {
pkgs[$(( pkgcnt++ ))]="${arg2}"
printf "."
continue;
}

arg2="${arg2}.i"
[ -d ${PROD}/${arg2} ] && pkgs[$(( pkgcnt++ ))]="${arg2}"
printf "."
fi
}
done < ${TOK}
}

#
# Check for the installation image before proceeding
#
[ ! -d ${PROD} ] && {
echo "Cannot find Solaris Installation"
exit 1
}

#
# Create a pkg admin file - see man admin(4)
#
sed 's/ask/nocheck/' /var/sadm/install/admin/default > /tmp/.admin.doit

#
# Build an ordered list of packages from the Solaris installation image
#
printf "Building a list of packages "
pkgcnt=0
get_pkg_list ${METACLUSTER}
echo
# Add extra packages before sorting
add_extra_pkgs
echo
printf "Sorting packages into the correct order for installation "
reorder_pkgs
echo
#
# Try to create the filesystem you define at the beginning
#
case ${FS} in
"zfs") zfs create ${ROOTDEV} || {
echo "Cannot create zfs filesystem!"
exit 1
}
zfs set mountpoint=legacy ${ROOTDEV}
zfs set compression=${ZFSCOMPRESS} ${ROOTDEV}
;;
"ufs") newfs ${ROOTDEV} || {
echo "Cannot create ufs filesystem!"
exit 1
} ;;
*) echo "Cannot create ufs filesystem!"
exit 1
;;
esac

mount -F ${FS} ${ROOTDEV} /mnt || {
echo "Cannot mount filesystem!"
exit 1
}

#
# Install packages from Solaris installation image
#
echo "Starting installation of packages"
echo
(
for i in ${pkgs[@]} ; do
pkgadd -n -a /tmp/.admin.doit -d ${PROD} -R /mnt $i
done
) > ${LOG}

#
# Update /etc/vfstab with swap and root partitions
#
(
printf "${SWAP}\t-\t-\tswap\t-\tno\t-\n"
printf "${ROOTDEV}\t${RAWROOTDEV}\t/\t${FS}\t1\t${MOB}\t-\n"
[ "$FS" = "zfs" ] && {
mkdir -m 0755 /mnt/grub
printf "${GRUBFS}\t${GRUBFS/dsk/rdsk}\t/grub\tufs\t3\tyes\t-\n"
}
) >> /mnt/etc/vfstab

#
# Copy links for disk partitions in /dev/dsk and /dev/rdsk
# This is needed so the system can find the root partion on boot
#
( cd /dev && find dsk rdsk -depth | cpio -pdm /mnt/dev 2>/dev/null )

#
# Configure system to initialize identity on first boot
# If there is a sysidcfg file in the current directory. This will
# be copied across.
#
PROFILEDIR=/mnt/var/svc/profile
[ -f ${PROFILEDIR}/${SVCPROFILE} ] && {
if [ -f ./sysidcfg ]; then
cp ./sysidcfg /mnt/etc
else
touch /mnt/etc/.UNCONFIGURED
fi
cp -p ${PROFILEDIR}/${SVCPROFILE} ${PROFILEDIR}/generic.xml
}

#
# set bootpath to root filesystem.
# Also set the console to text
#
(
[ "${FS}" = "ufs" ] && {
BOOTPATH=$( ls -l ${ROOTDEV} | nawk '{print $11}' |
sed -e 's#[./]*/devices/#/#' )

printf "setprop bootpath ${BOOTPATH}\n"
}
printf "setprop console 'text'\n"
) >> /mnt/boot/solaris/bootenv.rc

#
# If found execute local script before /mnt is unmounted
#
[ -x ./local_install.bash ] && ./local_install.bash

#
# Finish off installation
#
[ -f /etc/zfs/zpool.cache ] && {
cp -p /etc/zfs/zpool.cache /mnt/etc/zfs
echo "etc/zfs/zpool.cache" >> /mnt/boot/solaris/filelist.ramdisk
}

#
# Configure for a ZFS boot. At the moment you need a small UFS partition
# somewhere for grub.
#
[ "${FS}" = "zfs" ] && {
(
printf "rootfs:zfs\n"
printf "zfsroot:${ROOTDEV}\n"
) >> /mnt/etc/system
}

devfsadm -r /mnt
rm -f /mnt/reconfigure
bootadm update-archive -R /mnt

[ "${FS}" = "zfs" ] && {
cp /mnt/platform/i86pc/boot_archive ${ZFSBOOTARCHIVE}
cp -p /mnt/sbin/bootadm /mnt/sbin/bootadm.real
cat >/mnt/sbin/bootadm << EOM
#!/usr/bin/sh

/sbin/bootadm.real "\$@"
if [ "\$1" = "update-archive" -a -d /grub/boot/grub ]; then
/usr/bin/cp /platform/i86pc/boot_archive ${ZFSBOOTARCHIVE}
fi
exit 0
EOM
}

echo
echo "If you have not already, you will need to configure menu.lst to"
echo "boot this partition."

umount /mnt
# eject cdrom

Wednesday, July 19, 2006

How small can you make Open Solaris - Part 4

Below, I have posted the new version of the Quick and Dirty Solaris Installer. This version will install Open Solaris directly onto a zfs filesystem. It took me a little longer than expected as I was having problems with the create_ramdisk.ksh script which is used by bootadm. The problem is that create_ramdisk.ksh does a 'du' on every file it is copying to the ramdisk to calculate the space required. If the 'du' command is used on a compressed file, it will return the compressed size. The problem comes about when a UFS filesystem is used to create the boot_archive image. Files copied from a compress zfs volume, will soon fill the ramdisk.

Not being able to create a boot archive is not good. The current work around is to turn compression off for the root filesystem. In the next update, I will modify the script to start excluding packages, so we can see what is needed and what is not.


#!/bin/bash
#
# Quick and Dirty Solaris installer
# Version 0.2
#
PROD=/cdrom/sol_11_x86/Solaris_11/Product
# PROD=/a/Solaris_11/Product
TOK=${PROD}/.clustertoc
ORDER=${PROD}/.order

# Current metaclusters are
# SUNWCXall - Entire Distribution plus OEM support
# SUNWCall - Entire Distribution
# SUNWCprog - Developer System Support
# SUNWCuser - End User System Suppor
# SUNWCreq - Core System Support
# SUNWCrnet - Reduced Networking Core System Support
# SUNWCmreq - Minimal Core System Support
METACLUSTER=SUNWCrnet

SWAP=/dev/dsk/c0d0s1
LOG=/tmp/install.log
SVCPROFILE=generic_limited_net.xml

#
# Change these to relect your system.
#
FS=zfs
[ "${FS}" != "zfs" -a "${FS}" != "ufs" ] && {
printf "Unknown filesystem ${FS}\n"
exit 1
}

[ "${FS}" = "zfs" ] && {
ROOTDEV=intdisk/snv43_zfs
RAWROOTDEV=-
GRUBFS=/dev/dsk/c0d0s0
ZFSBOOTARCHIVE=/grub/boot/boot_zfs
ZFSCOMPRESS=off
# ZFSCOMPRESS=on
MOB=yes
}

[ "${FS}" = "ufs" ] && {
ROOTDEV=/dev/dsk/c0d0s3
RAWROOTDEV=${ROOTDEV/dsk/rdsk}
MOB=no
}

typeset -a pkgs

#
# Solaris packages need to be installed in the correct order.
# The .order file contains all the packages in the correct
# installation order
#
function reorder_pkgs() {
typeset -a pkglist=( ${pkgs[@]} )
pkgcnt=0

while read order_pkg ; do
for i in ${pkglist[@]} ; do
[ "$order_pkg" = "${i%.i}" ] && {
pkgs[$(( pkgcnt++ ))]="${i}"
printf "."
}
done
done < ${ORDER}
}

#
# This function builds a list of packages in a cluster
# If there is a cluster within a cluster, it will call itself to
# resolve all the packages.
#
# Before calling make sure you initialize pkgcnt to 0
# Arg: $1 contains the cluster name
# Affected vars: pkgs, pkgcnt
#
function get_pkg_list() {
local IFS="="
local print_on=0
local cluster=$1

while read arg1 arg2
do
[ "${arg1}" = "END" -a "${print_on}" = "1" ] && break;
[ -z "${arg2}" ] && continue;

[ "${arg2}" = "${cluster}" ] && {
print_on=1
continue
}

[ "${print_on}" = "1" -a "${arg1}" = "SUNW_CSRMEMBER" ] && {
ifcluster=`expr "${arg2}" : '\(SUNWC\)'`
if [ "${ifcluster}" = "SUNWC" ]; then
get_pkg_list ${arg2}
else
[ -d ${PROD}/${arg2} ] && {
pkgs[$(( pkgcnt++ ))]="${arg2}"
printf "."
continue;
}

arg2="${arg2}.i"
[ -d ${PROD}/${arg2} ] && pkgs[$(( pkgcnt++ ))]="${arg2}"
printf "."
fi
}
done < ${TOK}
}

#
# Check for the installation image before proceeding
#
[ ! -d ${PROD} ] && {
echo "Cannot find Solaris Installation"
exit 1
}

#
# Create a pkg admin file - see man admin(4)
#
sed 's/ask/nocheck/' /var/sadm/install/admin/default > /tmp/.admin.doit

#
# Build an ordered list of packages from the Solaris installation image
#
printf "Building a list of packages "
pkgcnt=0
get_pkg_list ${METACLUSTER}
echo
printf "Sorting packages into the correct order for installation "
reorder_pkgs
echo

#
# Try to create the filesystem you define at the beginning
#
case ${FS} in
"zfs") zfs create ${ROOTDEV} || {
echo "Cannot create zfs filesystem!"
exit 1
}
zfs set mountpoint=legacy ${ROOTDEV}
zfs set compression=${ZFSCOMPRESS} ${ROOTDEV}
;;
"ufs") newfs ${ROOTDEV} || {
echo "Cannot create ufs filesystem!"
exit 1
} ;;
*) echo "Cannot create ufs filesystem!"
exit 1
;;
esac

mount -F ${FS} ${ROOTDEV} /mnt || {
echo "Cannot mount filesystem!"
exit 1
}

#
# Install packages from Solaris installation image
#
echo "Starting installation of packages"
echo
(
for i in ${pkgs[@]} ; do
pkgadd -n -a /tmp/.admin.doit -d ${PROD} -R /mnt $i
done
) > ${LOG}

#
# Update /etc/vfstab with swap and root partitions
#
(
printf "${SWAP}\t-\t-\tswap\t-\tno\t-\n"
printf "${ROOTDEV}\t${RAWROOTDEV}\t/\t${FS}\t1\t${MOB}\t-\n"
[ "$FS" = "zfs" ] && {
mkdir -m 0755 /mnt/grub
printf "${GRUBFS}\t${GRUBFS/dsk/rdsk}\t/grub\tufs\t3\tyes\t-\n"
}
) >> /mnt/etc/vfstab

#
# Copy links for disk partitions in /dev/dsk and /dev/rdsk
# This is needed so the system can find the root partion on boot
#
( cd /dev && find dsk rdsk -depth | cpio -pdm /mnt/dev 2>/dev/null )

#
# Configure system to initialize identity on first boot
# If there is a sysidcfg file in the current directory. This will
# be copied across.
#
PROFILEDIR=/mnt/var/svc/profile
[ -f ${PROFILEDIR}/${SVCPROFILE} ] && {
if [ -f ./sysidcfg ]; then
cp ./sysidcfg /mnt/etc
else
touch /mnt/etc/.UNCONFIGURED
fi
cp -p ${PROFILEDIR}/${SVCPROFILE} ${PROFILEDIR}/generic.xml
}

#
# set bootpath to root filesystem.
# Also set the console to text
#
(
[ "${FS}" = "ufs" ] && {
BOOTPATH=$( ls -l ${ROOTDEV} | nawk '{print $11}' |
sed -e 's#[./]*/devices/#/#' )

printf "setprop bootpath ${BOOTPATH}\n"
}
printf "setprop console 'text'\n"
) >> /mnt/boot/solaris/bootenv.rc

#
# If found execute local script before /mnt is unmounted
#
[ -x ./local_install.bash ] && ./local_install.bash

#
# Finish off installation
#
[ -f /etc/zfs/zpool.cache ] && {
cp -p /etc/zfs/zpool.cache /mnt/etc/zfs
echo "etc/zfs/zpool.cache" >> /mnt/boot/solaris/filelist.ramdisk
}

#
# Configure for a ZFS boot. At the moment you need a small UFS partition
# somewhere for grub.
#
[ "${FS}" = "zfs" ] && {
(
printf "rootfs:zfs\n"
printf "zfsroot:${ROOTDEV}\n"
) >> /mnt/etc/system
}

devfsadm -r /mnt
rm -f /mnt/reconfigure
bootadm update-archive -R /mnt

[ "${FS}" = "zfs" ] && {
cp /mnt/platform/i86pc/boot_archive ${ZFSBOOTARCHIVE}
cp -p /mnt/sbin/bootadm /mnt/sbin/bootadm.real
cat >/mnt/sbin/bootadm << EOM
#!/usr/bin/sh

/sbin/bootadm.real "\$@"
if [ "\$1" = "update-archive" -a -d /grub/boot/grub ]; then
/usr/bin/cp /platform/i86pc/boot_archive ${ZFSBOOTARCHIVE}
fi
exit 0
EOM
}

echo
echo "If you have not already, you will need to configure menu.lst to"
echo "boot this partition."

umount /mnt
# eject cdrom

Sunday, July 16, 2006

How small can you make Open Solaris - Part 3

In this post I have decided to focus on installing Solaris directly without using the normal Solaris installer. If you are like me and forever installing the latest Solaris Nevada version, and would like to do this without rebooting and running the Solaris installer then the script below might be for you. If you are playing around to see what packages you can remove to shrink a Solaris installation, then a modified version of this script might be what you are looking for.

Solaris already has an excellent tool for upgrading your current system called live upgrade. The script below is different, as it does a totally fresh install. Hopefully the next version will also install directly onto a ZFS filesystem rather than a UFS partition.

To start off you need a Solaris X86 system (one which uses grub) with a free Solaris partition that has enough diskspace to hold the installation. Note, this script does no checking! It assumes you have configured the settings correctly, and have allocated enough space. You also have to have a DVD (or image) of Solaris (or maybe a jumpstart type installation directory).

Going through the script, at the beginning you will find some variables to may need to change to refect your setup. Since it uses the Solaris Meta Clusters you will need to define which one you want. The functions (which is most of the code), rattle through the cluster table of contents file and produces an order list of packages for the meta cluster you selected. Once it has a list a pkgadd is executed for each package.

After the packages have been installed, there are a few system configurations and device links that are required to get the system booted correctly. You can have the system automatically configure itself on reboot if you have a valid sysidcfg file in your current directory. Otherwise, the system will ask you many questions on reboot to configure hostnames, networking etc. If you are familiar with building zones, this will not be new.

Before you start, you should read through the script so you know exactly what it does. The is very little error/sanity checks in the script, and incorrect settings could be devastating as you will need to be root (or have the correct privs) to use it. When you have finish the installation you will need to modify grub to boot off the correct partition. Below the relevent section of my /boot/grub/menu.lst file.


title Solaris Test
root (hd0,0,d)
kernel /platform/i86pc/multiboot
module /platform/i86pc/boot_archive


Enjoy, and Good Luck!!!


#!/bin/bash
#
# Quick and Dirty Solaris installer
# Version 0.1
#
PROD=/cdrom/sol_11_x86/Solaris_11/Product
TOK=${PROD}/.clustertoc
ORDER=${PROD}/.order

# Current metaclusters are
# SUNWCXall - Entire Distribution plus OEM support
# SUNWCall - Entire Distribution
# SUNWCprog - Developer System Support
# SUNWCuser - End User System Suppor
# SUNWCreq - Core System Support
# SUNWCrnet - Reduced Networking Core System Support
# SUNWCmreq - Minimal Core System Support
METACLUSTER=SUNWCrnet

#
# Change these to relect your system. The next version should
# use ZFS rather that UFS.
#
ROOTDEV=/dev/dsk/c0d0s4
RAWROOTDEV=/dev/rdsk/c0d0s4

SWAP=/dev/dsk/c0d0s1
LOG=/tmp/install.log

SVCPROFILE=generic_limited_net.xml

typeset -a pkgs

#
# Solaris packages need to be installed in the correct order.
# The .order file contains all the packages in the correct
# installation order
#
function reorder_pkgs() {
typeset -a pkglist=( ${pkgs[@]} )
pkgcnt=0

while read order_pkg ; do
for i in ${pkglist[@]} ; do
[ "$order_pkg" = "${i%.i}" ] && {
pkgs[$(( pkgcnt++ ))]="${i}"
printf "."
}
done
done < ${ORDER}
}

#
# This function builds a list of packages in a cluster
# If there is a cluster within a cluster, it will call itself to
# resolve all the packages.
#
# Before calling make sure you initialize pkgcnt to 0
# Arg: $1 contains the cluster name
# Affected vars: pkgs, pkgcnt
#
function get_pkg_list() {
local IFS="="
local print_on=0
local cluster=$1

while read arg1 arg2
do
[ "${arg1}" = "END" -a "${print_on}" = "1" ] && break;
[ -z "${arg2}" ] && continue;

[ "${arg2}" = "${cluster}" ] && {
print_on=1
continue
}

[ "${print_on}" = "1" -a "${arg1}" = "SUNW_CSRMEMBER" ] && {
ifcluster=`expr "${arg2}" : '\(SUNWC\)'`
if [ "${ifcluster}" = "SUNWC" ]; then
get_pkg_list ${arg2}
else
[ -d ${PROD}/${arg2} ] && {
pkgs[$(( pkgcnt++ ))]="${arg2}"
printf "."
continue;
}

arg2="${arg2}.i"
[ -d ${PROD}/${arg2} ] && pkgs[$(( pkgcnt++ ))]="${arg2}"
printf "."
fi
}
done < ${TOK}
}

#
# Check for the installation image before proceeding
#
[ ! -d ${PROD} ] && {
echo "Cannot find Solaris Installation"
exit 1
}

#
# Create a pkg admin file - see man admin(4)
#
sed 's/ask/nocheck/' /var/sadm/install/admin/default > /tmp/.admin.doit

#
# Build an ordered list of packages from the Solaris installation image
#
printf "Building a list of packages "
pkgcnt=0
get_pkg_list ${METACLUSTER}
echo
printf "Sorting packages into the correct order for installation "
reorder_pkgs
echo

newfs ${ROOTDEV} || exit 1
mount ${ROOTDEV} /mnt

#
# Install packages from Solaris installation image
#
echo "Starting installation of packages"
echo
(
for i in ${pkgs[@]} ; do
pkgadd -n -a /tmp/.admin.doit -d ${PROD} -R /mnt $i
done
) > ${LOG}

#
# Update /etc/vfstab with swap and root partitions
#
(
printf "${SWAP}\t-\t-\tswap\t-\tno\t-\n"
printf "${ROOTDEV}\t${RAWROOTDEV}\t/\tufs\t1\tno\t-\n"
) >> /mnt/etc/vfstab

#
# Copy links for disk partitions in /dev/dsk and /dev/rdsk
# This is needed so the system can find the root partion on boot
#
( cd /dev && find dsk rdsk -depth | cpio -pdm /mnt/dev 2>/dev/null )

#
# Configure system to initialize identity on first boot
# If there is a sysidcfg file in the current directory. This will
# be copied across.
#
PROFILEDIR=/mnt/var/svc/profile
[ -f ${PROFILEDIR}/${SVCPROFILE} ] && {
if [ -f ./sysidcfg ]; then
cp ./sysidcfg /mnt/etc
else
touch /mnt/etc/.UNCONFIGURED
fi
cp -p ${PROFILEDIR}/${SVCPROFILE} ${PROFILEDIR}/generic.xml
}

#
# set bootpath to root filesystem.
# Also set the console to text
#
(
BOOTPATH=$( ls -l ${ROOTDEV} | nawk '{print $11}' |
sed -e 's#[./]*/devices/#/#' )

printf "setprop bootpath ${BOOTPATH}\n"
printf "setprop console 'text'\n"
) >> /mnt/boot/solaris/bootenv.rc

#
# If found execute local script before /mnt is unmounted
#
[ -x ./local_install.bash ] && ./local_install.bash

#
# Finish off installation
#
bootadm update-archive -R /mnt
echo "You will need to configure /boot/grub/menu.lst to boot this partition"

umount /mnt
# eject cdrom

Wednesday, July 12, 2006

ZFS saved my backside

Well, not totally, but it did save me a re-install!!!

Last night I decided to stay up and watch the webcast of product lunch of Sun's new servers (Hmmm, I want a Thumper...). In Thailand the festivities did not start till 12:30 at night. The laptop I am using is an Acer Ferrari 4005. Recently, Sun released a sound driver for it (I was using OSS before), but whenever I used it there was sound + static.

Now I had 1 hour before the webcast, so I decided to do the usual rounds of the Open Solaris site to fill in time. I found that the latest ON build had a patch for the sound driver. Hmmm, there is not enough time for a compile (even on a Ferrari), so I downloaded the bfu archive, and started the install. If you read my earlier articles, you will probably know, that I am running on a ZFS root. Since this is a loosely undocumented feature, it does complicate a 'bfu'. I have done a bfu update on a ZFS root once before, so I should not have a problem, "Right?". The download took almost 1 hour (20Kb on a 4Mb link, grrr), so I quickly put the original bootadm back before I 'bfu'd'. Atfer the 'bfu', the usual 'acr', and then I replaced bootadm with the zfs modified bootadm and updated the archive. Just in time for a reboot right on the bell.

As I was rebooting, I was thinking, "I should have done a zfs snapshot before I started". It was late, and I was fully aware that a failure would only affect me. Ok, reboot. . . . Ahhhhh!!! Almost as I rebooted, the ferrari reset and booted again. Grrrr. I started to get 'Rhymes with MISSED'. I quickly edited grub to add the "-kd" option to the kernel line, and booted. Great :(, the error message was that it could not find the root partition. Ok, failsafe it is then.....

In failsafe, everything looked ok. I could mount the ZFS root. /etc/zfs/zpool.cache existed, and looked ok (Jibberish. It is binary after all). I then decided to look at grub, and found that the file /boot/solaris/filelist.ramdisk did not contain zpool.cache. Ah we now have somebody else to blame!!! The 'bfu/acr' procedure updates this file without considering that I may have added to it.

Right, what to do now. I tried to update the boot archive from both failsafe, and from a spare UFS root, and kept on getting a "filesystem full" error on the ramdisk. Now I have 1.5GB of memory and plenty of swap space, and I had modified the create_ramdisk script to double the amount of memory allocated. I still got the same message. "Marvelleous"

This all prompted a late night re-think. "I wish I had done a snapshot before I started the bfu....". Ok it was looking like a re-install of the root partition was on the cards. This was something for after the morning coffee... Hang on, after I installed and created a ZFS root, I did a snapshot to create a clone. That snapshot was still there! "zfs rollback intdisk/snv42_root@initial". Reboot. Hey, we are in business. I still have some driver/app installing and JDS update to do, but no Solaris install. Fantastic. So I quickly brought up firefox, and connected to the webcast, just to hear the last sentence. Oh, well I will fix the rest up in the morning. Now if I had only done a snapshot again before the 'bfu'!!!!

ZFS Rocks!!!
P.S. The time to do a ZFS snapshot is less time then thinking about it.

Saturday, July 08, 2006

How small can you make Open Solaris - Part 2

In the first part of the article I decided to use the failsafe miniroot as a base for a minimal Solaris. I also described how to mount and unmount a copy of the miniroot image which I termed microroot. You should be very comfortable with doing this before tackling this article. I will first start with a quick analysis of the miniroot before we start jump in and start removing files. It is really important to find out what is required by the operating system before you weild the 'rm' axe.

From playing around in the last article you would have realised that failsafe stops at the single user milestone. So it would be really nice to see what file SMF executes to get to the shell prompt. After you get the shell prompt we should look at what processes are running, and what dynamic libraries are in use. This will give us an idea of what needs to be kept.

Lets start - In normal Solaris as the root user create a directory called 'work' in your root partition. This directory will later be used to transfer files between the microroot and normal solaris. In that directory create a script "report.sh" from the following listing. Other than listing services, and processes. The horrible munge of shell scripting goes through the process table listing share libraries and objects used be each process. It then sorts and does a 'unique', to give you a nice listing of libraries and objects that will be required to keep.

mkdir /work
vi /work/report.sh

#!/sbin/sh

echo "Shared libraries in use"
echo "-----------------------"
(
/usr/bin/ps -eo pid | /usr/bin/awk '
$1 != "PID" {
printf "/usr/bin/pmap -x %s\n",$1
}' | /sbin/sh 2>/dev/null
) | /usr/bin/awk '{ print $7 }' | /usr/bin/grep '[.]so' | /usr/bin/sort -u

echo
echo "Services Status"
echo "---------------"
/usr/bin/svcs

echo
echo "Process Table"
echo "-------------"
/usr/bin/ps -eo user,pid,comm


Now we are ready to reboot and run the report. At the grub prompt you can either select the failsafe or the microroot option from part 1 as they should be identical at this point. Instead of pressing ENTER to boot press 'e' for edit. This should bring up a screen with the entries from your menu.lst file in the grub directory. Using the arrow keys move down to the line starting with 'kernel' and press 'e' again. At the end of the line add the '-m debug' option. This option will send 'smf' into debug mode filling your screen with information when you boot. Don't worry any changes made here as not save back to the menu.1st file. Press ENTER and then 'b' to boot.

During the boot smf should be printing out a huge amount of information. Don't panic if you can not speed read, the information you need is at the end. You will see that the last thing smf does is related to install-discovery.xml file. Later if you look at this file you will find it runs the script /sbin/install-discovery. This is a file of great interest and we will definitely attack it.

At this point you are being asked to mount your normal Solaris partition on /a. Answer 'y' for yes. Now run the report and put the results back into your work directory.

cd /
/sbin/sh /a/work/report.sh > /a/work/report.txt

This should not take long to run. When it has finished reboot in to normal Solaris and review the report.

In the report you should see around 36 libraries/objects listed. The process table contains a small number of processes running. They seem perfectly reasonable, so we will leave them alone. Now mount the microroot onto /mnt and run a the du command on it. Note: I have truncated the output.

du -ks * .??* | sort -nr
89502 usr
24961 kernel
10857 lib
2230 platform
1317 etc
1215 sbin
1156 boot
227 .tmp_proto

As you may expect, from the ouput of the du command you will see that /usr, /kernel, /lib, /platform are the biggest disk space users. Since /kernel and /platform are the Solaris Kernel we will leave these alone. At the end if you want a smaller image, there is plenty of scope for removing unused driver modules. As most people would expect, it will be /usr and /lib, that will first come under the knife. You might find it strange that /tmp is not empty, and there is a .tmp_proto directory. This is because the Solaris image you are building unlike Solaris on your system is not writeable. The microroot will make use of tmpfs filesystem for files which need to be written to. You can see this by doing a 'ls -l /mnt/etc/vfstab'. If you want to know more, just look through the install-discovery script.

Now the first time I cutting sections out, I just went in to a directory, had a look and if I did not think the file or directory was relevent I attacked it with the 'rm' command. Then I rebooted to test the changes. After doing this a few times, I thought it would be better to copy the commands into an editor so I could easily repeat the whole process from scratch if needed.

The next step is to copy /work/report.txt to /work/libs. Edit this file and remove everything except for the libraries and share objects. You will notice that some libraries end with .so and others end with '.so.1' etc. This is the version number of the shared library. With the editor remove the version from the end of the lines. Now all the lines should now end with '.so'. We will use the file later with fgrep to identify the libraries and links we want to keep.

It is now time to get to the fun part. Choose your weapon, 'as were going Hunting', er sorry - 'deleting'. I have listed the command so you can place them into a script at your leisure.

Lets start of with a no brainer and remove some obvious directories. No real space is gained yet but it may make you feel better...

cd /mnt
rm -rf /mnt/boot /mnt/cdrom /mnt/opt
Remove all locale's except for 'C' from /usr/lib/locale.
cd /mnt/usr/lib/locale && rm -rf [a-z]* POSIX
Now take the axe to anything that should not be required. Though make sure you leave anything related to devfsadm, sysevent, and booting the system alone. 'ls' does not need networking so while your at it take out anything network related. This is a script right! You can fix it later. ( The next part is not intended to look like uuencoding or base64. It just came out that way!)

cd /mnt/usr/lib && rm -rf zones zfs vplot term tabset t[0-9]* sunw,rcp spell
cd /mnt/usr/lib && rm -rf rcm print patch nss_nisplus.so.1 install ldap krb5
cd /mnt/usr/lib && rm -rf iconv inet crypto cron diff3prog diffh dns expreserve
cd /mnt/usr/lib && rm -rf netsvc newsyslog nfs nis nscd_nischeck nss_compat.so.1
cd /mnt/usr/lib && rm -rf nss_ldap.so.1 nss_nis.so.1 passwdutil.so.1 calprog
cd /mnt/usr/lib && rm -rf flash fp getoptcvt gmsgfmt help intrd lddstub libc
cd /mnt/usr/lib && rm -rf lp* lvm localedef lwp makekey more.help mps pt_chmod
cd /mnt/usr/lib && rm -rf fs/pcfs fs/fd fs/cachefs fs/nfs security/pam_krb5* drv
cd /mnt/usr/lib && rm -rf pam_dial* pam_ldap* pam_sample* platexec embedded_su
cd /mnt/usr/lib && rm -rf mdb smartcard sasl rsh kssladm ll* gss exrecover
cd /mnt/usr/lib && rm -rf nss_dns.so.1 abi adb class link_audit saf utmp* lib.b

Ok we need to copy the libraries we want to keep, and move them back once we have been brutal with 'rm'. It is now time to use the /work/libs file for input to fgrep. The logic of this script could be re-done, but it does the job for now.

cd /mnt/usr/lib
mkdir .bak
for i in `ls *[.]so*` ; do echo $i |fgrep -f /work/libs | cpio -pdm .bak 2>/dev/null ; done
rm -f *[.]so*
mv .bak/* .
rm -rf .bak

Move up one level and take the broad axe to /usr. I will leave /usr/bin and /usr/sbin to your discretion.

cd /mnt/usr && rm -rf X X11 adm ccs dict dt java kernel kvm mail net
cd /mnt/usr && rm -rf platform preserve pub sadm sfw share snadm spool
cd /mnt/usr && rm -rf news old openwin perl5 src xpg4 proc

For the second time use the /works/libs file, but this time on /lib.

cd /mnt/lib
mkdir .bak
for i in `ls *[.]so*` ; do echo $i |fgrep -f /work/libs | cpio -pdm .bak 2>/dev/null ; done
rm -f *[.]so*
mv .bak/* .
rm -rf .bak

The final directory we will attack before we build the image and reboot will be /sbin

cd /mnt/sbin && rm -rf rc* install* ifconfig ifparse getpart getmemory zpool
cd /mnt/sbin && rm -rf rc* biosdev sysid* meta* p* d* e* route* suninstall jsh
cd /mnt/sbin && rm -rf bootadm bpgetfile cleanup_hosts getInstallLangs swap*
cd /mnt/sbin && rm -rf getbootargs grepInstalledLocales hostconfig in.mpathd
cd /mnt/sbin && rm -rf mkmenu mountall netstrategy setup* selection siwrapper
cd /mnt/sbin && rm -rf soconfig umountall zfs zonename getconsole get_netmask

Now create 2 files. We need bootadm to just exit. The install-discovery file is just the original stripped down (remove the installation code) and compressed to fit in the blog.
vi /mnt/sbin/bootadm

#!/sbin/sh
exit 0

vi /mnt/sbin/install-discover

#!/sbin/sh
# Copyright 2005 Sun Microsystems, Inc. All rights reserved
# Use is subject to license terms.
SHELL=/sbin/sh;export SHELL
PATH=/sbin:/usr/bin:${PATH};export PATH
PLATFORM=`/sbin/uname -p`;export PLATFORM
_INIT_RECONFIG=set; export _INIT_RECONFIG #Dont know what this does
exec /dev/console 2>&1
/sbin/mount -F tmpfs swap /tmp
if [ $? -ne 0 ]; then
echo "tmpfs mount failed."
/sbin/sh
fi
( cd /.tmp_proto; find . -print -depth | cpio -pdm /tmp 2>/tmp/cpio.out )
echo "Memory free after tmpfs initialization: `/sbin/mem`"
echo "swap - /tmp tmpfs - no -" >> /etc/vfstab
echo "/proc - /proc proc - no -" >> /etc/vfstab
find dev -depth -print | cpio -pdum /tmp >/dev/null 2>&1
ln -sf /devices /tmp/devices
/sbin/mount -F lofs -O /tmp/dev /dev
mkdir -p /tmp/etc
mkdir -p /tmp/etc/sysevent&amp;amp;amp;amp;amp;&/usr/lib/sysevent/syseventd -r /tmp
/usr/lib/devfsadm/devfsadmd -r /tmp -p /tmp/root/etc/path_to_inst
eval `/sbin/get_root -t Roottype -b Rootfs /`
echo "${Rootfs} - / ${Roottype} - no ro" >> /etc/vfstab
echo
echo "Welcome to Super Small Solaris"
echo
echo "Dont expect your normal list of command."
echo "Just try 'ls /sbin' and you will find all you want."
echo
exec /sbin/sh


After just have made the scripts executable you should unmount /mnt and commit the changes to the image on /grub. The instructions below will backup the filesystem, and restore it onto a smaller UFS image. If you do not do this you will most likely find that your image has actually grown, as the UFS partition is the same size.

chmod 755 /mnt/sbin/bootadm /mnt/sbin/install-discovery
cd /
du -ks /mnt
# Take note of the size returned and add 15%
sync
ufsdump 0f /work/x86.microroot.dmp /mnt
umount /mnt
lofiadm -d /tmp/x86.microroot
mkfile k /tmp/x86.microroot
DEV="`lofiadm -a /tmp/x86.microroot`"
newfs -m 0 $DEV
mount -F ufs $DEV /mnt
cd /mnt
ufsrestore -rf /work/x86.microroot.dmp
rm restoresymtable
cd /
umount /mnt
lofiadm -d /tmp/x86.microroot
gzip -c /tmp/x86.microroot > /boot/x86.microroot

The last step we should do is to create an iso image to try out on a CD

cd /tmp
mkdir iso
cp -pr /boot /tmp/iso
rm /tmp/iso/boot/x86.miniroot-safe
rm /tmp/iso/boot/boot_archive

cat > /tmp/iso/boot/grub/menu.lst << EOM
default 0
timeout 10
splashimage /boot/grub/splash.xpm.gz
title Solaris Microroot
kernel /boot/multiboot kernel/unix -s
module /boot/x86.microroot
EOM

mkisofs -R -b boot/grub/stage2_eltorito -no-emul-boot boot-load-size 4 -boot-info-table -o /work/microroot.iso iso

At the end of all of this you should now have a 25 megabyte iso image in /work. For the moment I will stop here. Before I have gone further and been able to reduce the iso to 18 megabytes which still includes the full 32bit kernel from the latest Open Solaris builds. The next part of the article we will see if we can do the same thing using scripts, getting the files straight from an Open Solaris build. This involves just a little more work as we have to tackle smf and also build our own /devices and /dev directories.

Good Luck. Let me know how you go!

Friday, July 07, 2006

How small can you make Open Solaris - Part 1

Solaris started its life as operating system for workstations and then progressed to servers. It has always been an operating system dominated by features, showing Sun's R&D capability. This is great if you are installing a server or a desktop, but has far too many features for building an appliance. Luckily the installing comes with some reduced installation clusters which tries to bring the installation down to the bare minimum. Unfortunately the last time I looked the smallest install was still several hundred megabytes. Linux on the other hand has had a project going for while now called "Damn Small Linux", which strips Linux down to around 50 megabytes. This is a perfect base to start building an appliance, build your own distro, or strip the kernel down further for an embedded device.

Can Solaris become as small as "Damn Small Linux". The answer is a resounding yes (and probably smaller). Lets investigate how this can be done. The first thing to do is to state the goal, which is to be able to successfully boot into a shell and execute a simple command such as 'ls'. The logical place to start is with the smallest running version of Solaris supplied by Sun. If you have a x86 grub version of Solaris you will find a 52 megabyte file in your /boot directory called x86.miniroot-safe. This file is a gzipped UFS image that is booted when you select "Solaris failsafe" from the grub menu. Using it to boot to single user mode will mount the root filesystem and give you a root shell. It also contains the code to start a Solaris installation.

Now we have found an ideal candidate, lets start ripping it apart. The first step is to copy it (as you may need it if you break something), and setup new menu option in grub.

cd /boot
cp x86.miniroot-safe x86.microroot
cd /boot/grub

Edit the file menu.lst, copy the failsafe section and modify it to look something like this -

title Solaris Micro Root
kernel /boot/multiboot kernel/unix -s
module /boot/x86.microroot

If you want you now can reboot and select "Solaris Micro Root" when the grub menu comes up. It should boot into your copy of failsafe. After you have finished testing, reboot into multiuser mode and mount this image so you can change it.

The file '/boot/x86.microroot' is actually a gzipped UFS filesystem image, which with a couple of commands can mounted and change. The following is the an example of the procedure to make changes. I would suggest you create mount and unmount scripts to automate the process. (Note: You will need superuser privs for the following steps, the root user or atleast sys_mount, file_dac_read, file_dac_write)

Important - Make a backup before making changes, and document your changes

cp /boot/x86.microroot /boot/x86.microroot.bak

Unzip image to /tmp

gzcat /boot/x86.microroot > /tmp/microroot.img

Create a loopback device for this file. The environment variable 'dev' catches the device name for later use.

dev="`lofiadm -a /tmp/microroot.img`"

Finally mount the image using the loopback device.

mount -F ufs ${dev} /mnt

At this point your image is mounted and you can cd to /mnt and make your changes. Note: Be VERY VERY CAREFUL that you are changing or removing the file in /mnt and NOT in the root filesystem. It could get very ugly if you make this mistake. Take your time and be very careful. Once you have made the changes DONT REBOOT. You will need to follow the next steps and commit the changes before you reboot. Also rather than deleting a file or a directory, it is a better practice to move them to a backup directory and then test the changes. If the changes were good then you can delete the backup directory later. If the changes caused problems you can simply move the files and directories back in your next editting session.

Umount the image and delete the loopback device.

cd /
umount /mnt
lofiadm -d /tmp/microroot.img

Copy/gzip the changes back to the /boot directory

gzip -c /tmp/microroot.img > /boot/x86.microroot

You can now reboot and test your changes. If the system hangs, just reset the system and undo what you did and try something else. If the system reboots too fast for you to read the kernel messages, a handy tip is to add the "-kd" options in the grub menu (combined with or after the '-s'). This will put the kernel straight to debug mode. To continue the boot type ':c' at the prompt. If the kernel panics it should print a message and then wait for you to press a key before rebooting.

Using this method I reduced the entire image to around 42 megabytes (uncompressed) without touching the 32bit kernel. I then was able to create a 18 megabyte bootable Solaris. The next part I will list the areas you should remove or modify. Hopefully, in Part 3, I will have finished a script which takes a Open Solaris build from the prototype directory and builds the microroot for you.