To replace a disk in a coordinator diskgroup when the cluster is online

1. Make sure system-to-system communication is functioning properly.
2. Make sure that the cluster is online.

   # vxfenadm -d

   I/O Fencing Cluster Information:
   ================================
   Fencing Protocol Version: 201
   Fencing Mode: SCSI3
   Fencing SCSI3 Disk Policy: dmp
   Cluster Members:
   		* 0 (galaxy)
   		1 (nebula)
   RFSM State Information:
   		node 0 in state 8 (running)
   		node 1 in state 8 (running)

3. Import the coordinator disk group.

   The file /etc/vxfendg includes the name of the disk group (typically, vxfencoorddg) that contains the coordinator disks, so use the command:

   # vxdg -tfC import 'cat /etc/vxfendg'

   where:

   -t specifies that the disk group is imported only until the node restarts.

   -f specifies that the import is to be done forcibly, which is necessary if one or more disks is not accessible.

   -C specifies that any import locks are removed.

4. If your setup uses VRTSvxvm version 5.1 RP2 (or later) or 5.1 SP1 (or later), then skip to step 5. You need not set coordinator=off to add or remove disks. For other VxVM versions, perform this step:

   Turn off the coordinator attribute value for the coordinator disk group.

   # vxdg -g vxfencoorddg set coordinator=off

5. To remove disks from the coordinator disk group, use the VxVM disk administrator utility vxdiskadm.

6. Perform the following steps to add new disks to the coordinator disk group:

   • Add new disks to the node.

   • Initialize the new disks as VxVM disks.

   • Check the disks for I/O fencing compliance.

   • Add the new disks to the coordinator disk group and set the coordinator attribute value as "on" for the coordinator disk group.

   See the Veritas Storage Foundation for Oracle RAC Installation and Configuration Guide for detailed instructions.

   Note that though the disk group content changes, the I/O fencing remains in the same state.

7. From one node, start the vxfenswap utility. You must specify the diskgroup to the utility.

   The utility performs the following tasks:

   • Backs up the existing /etc/vxfentab file.

   • Creates a test file /etc/vxfentab.test for the diskgroup that is modified on each node.

   • Reads the diskgroup you specified in the vxfenswap command and adds the diskgroup to the /etc/vxfentab.test file on each node.

   • Verifies that the serial number of the new disks are identical on all the nodes. The script terminates if the check fails.

   • Verifies that the new disks can support I/O fencing on each node.

8. If the disk verification passes, the utility reports success and asks if you want to commit the new set of coordinator disks.

9. Confirm whether you want to clear the keys on the coordination points and proceed with the vxfenswap operation.

   Do you want to clear the keys on the coordination points 
   and proceed with the vxfenswap operation? [y/n] (default: n) y

10. Review the message that the utility displays and confirm that you want to commit the new set of coordinator disks. Else skip to step 11.

    Do you wish to commit this change? [y/n] (default: n) y

    If the utility successfully commits, the utility moves the /etc/vxfentab.test file to the /etc/vxfentab file.

11. If you do not want to commit the new set of coordinator disks, answer n.

    The vxfenswap utility rolls back the disk replacement operation.