ZFS boot/root - mirroring

Today we will see how mirroring can be used migrate an existing ZFS root pool to a new pair of disks.
This technique may be used to upgrade to larger drives.

Let's assume we have a ZFS root pool mirrored between c0t0d0 and c0t1d0.
We will go through the simple procedure of migrating to a pair new disks (c0t2d0 and c0t3d0)

First, verify the status of the pool.
# zpool status rpool
pool: rpool
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
c0t0d0s0 ONLINE 0 0 0
c0t1d0s0 ONLINE 0 0 0
errors: No known data errors



Run format and make sure the new disks have valid SMI labels

If there is an EFI label, it will need to be replaced with a standard SMI label; this can be done by running "format -e" and relabling the disk.

If the new and old disks are the same size, fmthard can be used to transfer the partition table.
# prtvtoc /dev/rdsk/c0t0d0s2 | fmthard -s - /dev/rdsk/c0t2d0s2
fmthard: New volume table of contents now in place.
# prtvtoc /dev/rdsk/c0t0d0s2 | fmthard -s - /dev/rdsk/c0t3d0s2
fmthard: New volume table of contents now in place.

If the new disks are not the same size as the old disks, the new disks must be manually partitioned using the format command.
Slice 0 on the new disks needs to be at least as big as slice 0 on the old disks.


You may with to view the partition table on one (or both) of the new drives
# echo verify | format c0t3d0
selecting c0t3d0
. . .
Volume name = < >
ascii name = <SUN72G cyl 14087 alt 2 hd 24 sec 424>
pcyl = 14089
ncyl = 14087
acyl = 2
nhead = 24
nsect = 424
Part Tag Flag Cylinders Size Blocks
0 root wm 0 - 14083 68.34GB (14084/0/0) 143318784
1 unassigned wu 0 0 (0/0/0) 0
2 backup wm 0 - 14086 68.35GB (14087/0/0) 143349312
3 unassigned wu 0 0 (0/0/0) 0
4 unassigned wu 0 0 (0/0/0) 0
5 unassigned wu 0 0 (0/0/0) 0
6 unassigned wu 0 0 (0/0/0) 0
7 unassigned wu 0 0 (0/0/0) 0


Add the new disks to the root pool. When specifying the source you can specify either c0t0d0s0 or c0t1d0s0.
# zpool attach rpool c0t0d0s0 c0t2d0s0
# zpool attach rpool c0t0d0s0 c0t3d0s0


Verify that the disks have been added.
# zpool status rpool
pool: rpool
state: ONLINE
status: One or more devices is currently being resilvered. The pool will
continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scrub: resilver in progress for 0h0m, 11.21% done, 0h0m to go
config:
NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
mirror ONLINE 0 0 0
c0t0d0s0 ONLINE 0 0 0
c0t1d0s0 ONLINE 0 0 0
c0t2d0s0 ONLINE 0 0 0
c0t3d0s0 ONLINE 0 0 0
errors: No known data errors
We now have a four way mirror!


After a few minutes, "zpool status" should show that mirroring (reslivering) has completed.
# zpool status rpool
pool: rpool
state: ONLINE
scrub: resilver completed after 0h1m with 0 errors on Fri Jul 17 15:56:38 2009
config:
NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
mirror ONLINE 0 0 0
c0t0d0s0 ONLINE 0 0 0
c0t1d0s0 ONLINE 0 0 0
c0t2d0s0 ONLINE 0 0 0
c0t3d0s0 ONLINE 0 0 0
errors: No known data errors


It is likely that future versions of ZFS will take care of installing the boot block (as is the case with SVM)), but for now it must be done manually.
For sparc systems run:
# installboot -F zfs /usr/platform/`uname -i`/lib/fs/zfs/bootblk /dev/rdsk/c0t2d0s0
# installboot -F zfs /usr/platform/`uname -i`/lib/fs/zfs/bootblk /dev/rdsk/c0t3d0s0
-
For x86 systems run:
# installgrub /boot/grub/stage1 /boot/grub/stage2 /dev/rdsk/c0t2d0s0
# installgrub /boot/grub/stage1 /boot/grub/stage2 /dev/rdsk/c0t3d0s0
If you forget to run "installboot" or "installgrub", your sever will not boot from the new disks!



Configure the server to use one of the new disks as the boot disk.
For sparc systems this can be done using luxadm(1M). For x86 systems it will likely require a change in the BIOS.
# luxadm set_boot_dev -y /dev/dsk/c0t2d0s0


On a sparc system if you want primary and secondary boot devices you can try this:
# luxadm set_boot_dev -y /dev/dsk/c0t2d0s0
# BOOTDISK1=`eeprom boot-device | sed s/^.*=//`
# luxadm set_boot_dev -y /dev/dsk/c0t3d0s0
# BOOTDISK2=`eeprom boot-device | sed s/^.*=//`
# eeprom boot-device="$BOOTDISK1 $BOOTDISK2"


Rebooting is not mandatory!
But… it is a good idea to try booting off of the new disks to make sure everything is working.
# reboot


Detach the old disks.
# zpool detach rpool c0t0d0s0
# zpool detach rpool c0t1d0s0


# zpool status rpool
pool: rpool
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
mirror ONLINE 0 0 0
c0t2d0s0 ONLINE 0 0 0
c0t3d0s0 ONLINE 0 0 0
errors: No known data errors
Notice the old disks are gone from the pool.

If the new disk had been larger than the old disk, the pool would have instantly grown after detaching of the last of the old disks.

Please note that there is currently no way to salvage any data from a device that has been detached from a ZFS pool.

This means that "splitting mirrors" does not provide a rollback patch when patching. But, ZFS does give us clones and snapshots which I will discuss in a future ZFS tip.

Readers who read this page, also read: