ZFS Root on Solaris Part 2

The first post was just a quick post to show people the commands to use to create a ZFS root filesystem, with a small UFS grub boot partition. This post is an update using a fresh solaris install with comments of what is happening. Most of the steps follow Tabriz's blog on blogs.sun.com.


First step is to do a clean Solaris install which I will leave out all the details except my setup used the following partitioning -

    c0d0s0   /grub    150MB         Location of the UFS grub boot code
    c0d0s1   swap     1GB           Standard Swap/Dump partition
    c0d0s3   /        4GB           Install/Upgrade UFS partition.
    c0d0s7   ZFS      all the rest  later configured as "intdisk" pool, which contains
                                    ZFS root & clones plus home directories etc

Once you have installed all of your standard software and configuration, now the fun begins.

Create a zfs pool, and turn off mounting filesystems by default

    zpool create -f -m none intdisk c0d0s7

Create a zfs partition for the root filesystem. Being on a laptop, I am saving space by turning on compression. It may also give a performance boost at the same time. Note that I am using a symbolic name for the partition name. Later I am going to clone the filesystem and create a test environment.

    zfs create intdisk/snv42_root
    zfs set mountpoint=legacy intdisk/snv42_root
    zfs set compression=on intdisk/snv42_root

Create a mountpoint for the zfs root and use ufsdump/ufsrestore to copy all of the UFS root filesystem. You could use cpio or tar, but you also want the data underneath /devices.

    mkdir -m 0755 /zfsroot
    echo "intdisk/snv42_root - /zfsroot zfs - yes -" >> /etc/vfstab
    mount /zfsroot
    cd /zfsroot
    ufsdump 0f - / | ufsrestore -rf -

Configure /etc/system on zfs root to use the correct zfs filesystem. This configuation actually gets used by the kernel from within the boot archive, and gets copied each time you update the archive. Also make sure the zpool configuration gets added to the boot archive, and use a classic onliner to up the vfstab on the zfs root.

    echo "rootfs:zfs" >> /zfsroot/etc/system
    echo "zfsroot:intdisk/snv42_root" >> /zfsroot/etc/system
    echo "etc/zfs/zpool.cache" >> /zfsroot/boot/solaris/filelist.ramdisk
    grep -v 'intdisk/snv42_root' /etc/vfstab | awk '$3 == "/" { printf "intdisk/snv42_root\t-\t/\tzfs\t-\tno\t-\n" } ; $3 != "/" { print $0 }' > /zfsroot/etc/vfstab

Create a modified hack from Tabriz's blog to fix bootadm.

    mv /zfsroot/sbin/bootadm /zfsroot/sbin/bootadm.real
    cat - > /zfsroot/sbin/bootadm << EOM
    #!/usr/bin/sh

    /sbin/bootadm.real "\$@"
    if [ "\$1" = "update-archive" -a -d /grub/boot/grub ]; then
        /usr/bin/cp /platform/i86pc/boot_archive /boot/boot_archive
    fi
    exit 0
    EOM

    chmod +x /zfsroot/sbin/bootadm

Now we are ready to update the boot archive and configure grub. The "root (hd0,0,a)" should point to the grub partition.

    /usr/sbin/bootadm update-archive -R /zfsroot
    cp -pr /zfsroot/boot /grub
    cp /zfsroot/platform/i86pc/boot_archive /grub/boot/boot_archive
    (
        echo "title Solaris ZFS"
        echo "root (hd0,0,a)"
        echo "kernel /boot/multiboot"
        echo "module /boot/boot_archive"
    ) >> /grub/boot/grub/menu.lst

    cd /grub/boot/grub
    installgrub stage1 stage2 /dev/rdsk/c0d0s0

Now you can reboot, and if all goes well you should be able to use the new entry in the grub menu to boot into the zfs partition. Ok, if it all looks good, lets try to configure another zfs root partition using a clone. To create copy we simply do a snapshot and a clone of the current zfs root partition. I am naming this partition "snv42_test". Note the changes...

    zfs snapshot intdisk/snv42_root@initial
    zfs clone intdisk/snv42_root@initial intdisk/snv42_test
    zfs set mountpoint=legacy intdisk/snv42_test
    zfs set compression=on intdisk/snv42_test
    echo "intdisk/snv42_test - /zfsroot zfs - yes -" >> /etc/vfstab
    mount /zfsroot

Follow similar steps you used last time. Again note the changes I have done to the boot archive name and filesystem name.

    sed -e "s/snv42_root/snv42_test/" /etc/system > /zfsroot/etc/system
    grep -v 'intdisk/snv42_test' /etc/vfstab | awk '$3 == "/" { printf "intdisk/snv42_test\t-\t/\tzfs\t-\tno\t-\n" } ; $3 != "/" { print $0 }' > /zfsroot/etc/vfstab
    
    cat - > /zfsroot/sbin/bootadm << EOM
    #!/usr/bin/sh

    /sbin/bootadm.real "\$@"
    if [ "\$1" = "update-archive" -a -d /grub/boot/grub ]; then
        /usr/bin/cp /platform/i86pc/boot_archive /boot/boot_archive.test
    fi
    exit 0
    EOM

    chmod +x /zfsroot/sbin/bootadm
    
    /usr/sbin/bootadm update-archive -R /zfsroot
    cp /zfsroot/platform/i86pc/boot_archive /grub/boot/boot_archive.test
    (
        echo "#"
        echo "title Solaris ZFS test"
        echo "root (hd0,0,a)"
        echo "kernel /boot/multiboot"
        echo "module /boot/boot_archive.test"
    ) >> /grub/boot/grub/menu.lst

Now you should be able to reboot, and you should now have a cloned test environment. Only changes from the original filesystem are added to the diskspace usage. The number of clones you can have is limited by the number of boot archives you can jam into the /grub partition, and to a lesser extent by the free space you have in the ZFS pool.

Have Fun!