Replacing I/O fencing coordinator disks when the cluster is online

Review the procedures to add, remove, or replace one or more coordinator disks in a cluster that is operational.

Caution   The cluster might panic if any node leaves the cluster membership before the vxfenswap script replaces the set of coordinator disks.

To replace a disk in an active coordinator diskgroup

  1. Make sure system-to-system communication is functioning properly.
  2. Make sure that the cluster is online.

    # vxfenadm -d

    I/O Fencing Cluster Information:

    ================================

    Fencing Protocol Version: 201

    Fencing Mode: SCSI3

    Fencing SCSI3 Disk Policy: dmp

    Cluster Members:

    * 0 (galaxy)

    1 (nebula)

    RFSM State Information:

    node 0 in state 8 (running)

    node 1 in state 8 (running)

  3. Import the coordinator disk group.

    The file /etc/vxfendg includes the name of the disk group (typically, vxfencoorddg) that contains the coordinator disks, so use the command:

    # vxdg -tfC import 'cat /etc/vxfendg'

    where:

    -t specifies that the disk group is imported only until the node restarts.

    -f specifies that the import is to be done forcibly, which is necessary if one or more disks is not accessible.

    -C specifies that any import locks are removed.

  4. Turn off the coordinator attribute value for the coordinator disk group.

    # vxdg -g vxfencoorddg set coordinator=off

  5. To remove disks from the disk group, use the VxVM disk administrator utility vxdiskadm.
  6. Perform the following steps to add new disks to the coordinator disk group:
  7. Make sure that the /etc/vxfenmode file is updated to specify the correct disk policy.

    See the Veritas Cluster Server Installation Guide for detailed instructions.

  8. From one node, start the vxfenswap utility. You must specify the diskgroup to the utility:

    Do one of the following:

  9. If the disk verification passes, the script reports success and asks if you want to commit the new set of coordinator disks.
  10. Review the message that the utility displays and confirm that you want to commit the new set of coordinator disks. Else skip to step 11.

    Do you wish to commit this change? [y/n] (default: n) y

    If the utility successfully commits, the script moves the /etc/vxfentab.test file to the /etc/vxfentab file.

  11. If you do not want to commit the new set of coordinator disks, answer n.

    The vxfenswap utility rolls back the disk replacement operation.

Replacing the coordinator diskgroup in a cluster that is online

You can also replace the coordinator diskgroup using the vxfenswap utility. The following example replaces the coordinator disk group vxfencoorddg with a new disk group vxfendg.

To replace the coordinator diskgroup

  1. Make sure system-to-system communication is functioning properly.
  2. Make sure that the cluster is online.

    # vxfenadm -d

    I/O Fencing Cluster Information:

    ================================

    Fencing Protocol Version: 201

    Fencing Mode: SCSI3

    Fencing SCSI3 Disk Policy: dmp

    Cluster Members:

    * 0 (galaxy)

    1 (nebula)

    RFSM State Information:

    node 0 in state 8 (running)

    node 1 in state 8 (running)

  3. Find the name of the current coordinator diskgroup (typically vxfencoorddg) that is in the /etc/vxfendg file.

    # cat /etc/vxfendg

    vxfencoorddg

  4. Find the alternative disk groups available to replace the current coordinator diskgroup.

    # vxdisk -o alldgs list

    DEVICE TYPE DISK GROUP STATUS

    rhdisk64 auto:cdsdisk - (vxfendg) online

    rhdisk65 auto:cdsdisk - (vxfendg) online

    rhdisk66 auto:cdsdisk - (vxfendg) online

    rhdisk75 auto:cdsdisk - (vxfencoorddg) online

    rhdisk76 auto:cdsdisk - (vxfencoorddg) online

    rhdisk77 auto:cdsdisk - (vxfencoorddg) online

    DEVICE TYPE DISK GROUP STATUS

    sda auto:cdsdisk - (vxfendg) online

    sdb auto:cdsdisk - (vxfendg) online

    sdc auto:cdsdisk - (vxfendg) online

    sdx auto:cdsdisk - (vxfencoorddg) online

    sdy auto:cdsdisk - (vxfencoorddg) online

    sdz auto:cdsdisk - (vxfencoorddg) online

  5. Validate the new disk group for I/O fencing compliance. Run the following command:

    # vxfentsthdw -c vxfendg

    vxfentsthdw -c vxfendg

    See Testing the coordinator disk group using vxfentsthdw -c

  6. If the new disk group is not already deported, run the following command to deport the disk group:

    # vxdg deport vxfendg

  7. Make sure that the /etc/vxfenmode file is updated to specify the correct disk policy.

    See the Veritas Cluster Server Installation Guide for detailed instructions.

  8. From any node, start the vxfenswap utility. For example, if vxfendg is the new diskgroup that you want to use as the coordinator diskgroup:

    # vxfenswap -g vxfendg [-n]

    The utility performs the following tasks:

  9. If the disk verification passes, the script reports success and asks if you want to replace the coordinator disk group.
  10. Review the message that the utility displays and confirm that you want to replace the diskgroup. Else skip to step 11.

    Do you wish to commit this change? [y/n] (default: n) y

    If the utility successfully commits, the script moves the /etc/vxfentab.test file to the /etc/vxfentab file.

    The utility also updates the /etc/vxfendg file with this new diskgroup.

  11. Set the coordinator attribute value as "on" for the new coordinator disk group.

    # vxdg -g vxfendg set coordinator=on

    Set the coordinator attribute value as "off" for the old disk group.

    # vxdg -g vxfencoorddg set coordinator=off

  12. Verify that the coordinator disk group has changed.

    # cat /etc/vxfendg

    vxfendg

  13. If you do not want to replace the diskgroup, answer n.

    The vxfenswap utility rolls back the disk replacement operation.