UsMan's WoRkSpAce

Tuesday, December 08, 2009

Safely replacing a failed mirrored SVM boot disk

Below procedure can be adopted to replace a failed mirrored boot disk under SVM control. It is assumed that disk at c1t0d0 path has failed and needs replacement.

* Verify the device is the boot disk
prtconf -vp |grep bootpath

* Verify the names of SVM meta devices (d14, d13, d11, d10 and d16 in this example) on the failed drive
metastat -p

* Detach the failed drive meta devices
metadetach d4 d14
metadetach d3 d13
metadetach d1 d11
metadetach d0 d10
metadetach d6 d16

* Delete the meta devices on the failed drive
metaclear d13 d14 d11 d10 d16

* Delete the meta state database on the failed drive
metadb -d c1t0d0s7

* Unconfigure the failed drive
cfgadm -c unconfigure c1::dsk/c1t0d0

It did not work as veritas did not allow the disk to be disabled or unconfigured. Errors below:

Dec 8 00:08:39 mail1 rcm_daemon[5440]: [ID 546518 daemon.error] rcm script es_rcm.pl: VxVM vxdmpadm ERROR V-5-1-10894 Attempt to disable path failed. Last path to the disk can not be disabled. use option -f otherwise.
Dec 8 00:08:39 mail1 pseudo: [ID 129642 kern.info] pseudo-device: fcsm0
Dec 8 00:08:39 mail1 genunix: [ID 936769 kern.info] fcsm0 is /pseudo/fcsm@0
Dec 8 00:09:03 mail1 vxdmp: [ID 917986 kern.notice] NOTICE: VxVM vxdmp V-5-0-112 disabled path 32/0x8 belonging to the dmpnode 233/0x0
Dec 8 00:09:03 mail1 vxdmp: [ID 824220 kern.notice] NOTICE: VxVM vxdmp V-5-0-111 disabled dmpnode 233/0x0

* Force unconfigure the failed boot drive
cfgadm -f -c unconfigure c1::dsk/c1t0d0

* Physically replace the failed drive with a new disk

* Configure the new drive
cfgadm -c configure c1::sd1

* Format the new drive similar to the other mirrored disk
prtvtoc /dev/rdsk/c1t1d0s2 | fmthard -s - /dev/rdsk/c1t0d0s2

* Install boot blocks on the new drive
/usr/sbin/installboot bootblk /dev/rdsk/c1t0d0s0

* Install meta state database on the new drive
metadb -afc3 c1t0d0s7

* Create the meta devices
metainit -f d14 1 1 c1t0d0s4
metainit -f d13 1 1 c1t0d0s3
metainit -f d11 1 1 c1t0d0s1
metainit -f d10 1 1 c1t0d0s0
metainit -f d16 1 1 c1t0d0s6

* Attach the meta devices to begin syncing
metattach d4 d14
metattach d3 d13
metattach d1 d11
metattach d0 d10
metattach d6 d16

* Update the meta state database with the device ID of the new disk
metadevadm -u c1t0d0

* Verify new disk status, meta database and meta device status using cfgadm -al, metadb -i and metastat commands respectively.