Removing preexisting keys

If you encountered a split brain condition, use the vxfenclearpre utility to remove SCSI-3 registrations and reservations on the coordinator disks as well as on the data disks in all shared disk groups.

You can also use this procedure to remove the registration and reservation keys created by another node from a disk.

To clear keys after split brain

  1. Stop VCS and I/O fencing on all nodes.

    # hastop -all

    Enter the following command on each node:

    # /etc/init.d/vxfen stop

    If you have any applications that run outside of VCS control that have access to the shared storage, then shut down all other nodes in the cluster that have access to the shared storage. This prevents data corruption.

  2. Start the script:

    # vxfenclearpre [ -t all | -t vxfen | -t cluster -c clusterid ]

    The -t option applies only to the coordinator disks. If you do not use the -t option, the script clears all the keys from coordinator disks and data disks. If you use the -t option, the script behavior is as follows:

    -t all 

    Clears all keys from the coordinator disks 

    -t vxfen 

    Clears all vxfen keys (with the VF prefix) from the coordinator disks 

    -t cluster -c clusterid 

    Clears only vxfen keys from the coordinator disks that matches the clusterid you have specified. 

    The clusterid value is the cluster id that you provided during VCS configuration and ranges between 0 and 65535. 

  3. Read the script's introduction and warning. Review the list of keys for the disks that the script will clear before you enter y at the prompt to proceed.

    Do you still want to continue: [y/n] (default : n) y

    Informational messages resembling the following may appear on the console of one of the nodes in the cluster when a node is ejected from a disk/LUN:

    <date> <system name> scsi: WARNING: /sbus@3,0/lpfs@0,0/sd@0,1(sd91): <date> <system name> Error for Command: <undecoded cmd 0x5f> Error Level: Informational <date> <system name> scsi: Requested Block: 0 Error Block 0 <date> <system name> scsi: Vendor: <vendor> Serial Number: 0400759B006E <date> <system name> scsi: Sense Key: Unit Attention <date> <system name> scsi: ASC: 0x2a (<vendor unique code 0x2a>), ASCQ: 0x4, FRU: 0x0

    These informational messages may be ignored.

    Cleaning up the coordinator disks...

    Cleaning up the data disks for all shared disk groups...

    Successfully removed SCSI-3 persistent registration and reservations from the coordinator disks as well as the shared data disks.

    You can retry starting fencing module. In order to restart the whole

    product, you might want to restart the system.

    #

  4. Restart all nodes in the cluster.