< Previous  | TOC | Index | Next  >

Checking shared disks for I/O fencing

The shared storage for SF Oracle RAC must support SCSI-3 persistent reservations to enable I/O fencing. SF Oracle RAC involves two types of shared storage:

See Setting up shared storage

Perform the following checks for I/O fencing disks:

• Identify three SCSI-3 PR compliant shared disks as coordinator disks.

  List the disks on each node and pick three disks as coordinator disks.

  For example, execute the following commands to list the disks:

  # ioscan -nfC disk

  # insf -e

• Test the shared disks using the vxfentsthdw script.

  See Testing the shared disks for SCSI-3

Testing the shared disks for SCSI-3

Use the vxfentsthdw utility to test the shared storage arrays support SCSI-3 persistent reservations and I/O fencing. Review the guidelines to run vxfentsthdw program, verify that the systems see the same disk, and proceed to test the disks. Make sure to test disks serving as coordinator disks.

See Setting up coordinator disk groups

The vxfentsthdw utility has additional options suitable for testing many disks. Review the options for testing disk groups (-g) and disks listed in a file (-f). You can also test disks without destroying data using the -r option.

See Verifying data storage arrays using the vxfentsthdw utility

Reviewing guidelines on vxfentsthdw

• Verify the connection of the shared storage for data to two of the nodes on which you installed SF Oracle RAC.
  Warning: The tests overwrite and destroy data on the disks unless you use the -r option.
  
• The two nodes must have ssh (default) or remsh communication. If you use remsh, launch the vxfentsthdw utility with the -n option.

  See Setting up inter-system communication

  After completing the testing process, remove permissions for communication and restore public network connections.

  See Removing permissions for communication

• To ensure both nodes are connected to the same disk during the testing, use the vxfenadm -i diskpath command to verify the disk serial number.

  See Verifying the nodes see the same disk

Verifying the nodes see the same disk

To confirm whether a disk (or LUN) supports SCSI-3 persistent reservations, two nodes must simultaneously have access to the same disks. Because a shared disk is likely to have a different name on each node, check the serial number to verify the identity of the disk. Use the vxfenadm command with the -i option to verify that the same serial number for the LUN is returned on all paths to the LUN.

For example, an EMC array is accessible by the /dev/rdsk/c2t13d0 path on node A and by the /dev/rdsk/c2t11d0 path on node B.

From node A, enter:

# vxfenadm -i /dev/rdsk/c2t13d0

Vendor id : EMC

Product id : SYMMETRIX

Revision : 5567

Serial Number : 42031000a

The same serial number information should appear when you enter the equivalent command on node B using the /dev/rdsk/c2t11d0 path.

On a disk from another manufacturer, Hitachi Data Systems, the output is different and may resemble:

# vxfenadm -i /dev/rdsk/c2t0d2

Vendor id : HITACHI

Product id : OPEN-3 -HP

Revision : 0117

Serial Number : 0401EB6F0002

Refer to the vxfenadm(1M) manual page.

Testing the disks using vxfentsthdw script

This procedure uses the /dev/rdsk/c2t13d0 disk in the steps.

If the utility does not show a message stating a disk is ready, verification has failed. Failure of verification can be the result of an improperly configured disk array. It can also be caused by a bad disk.

If the failure is due to a bad disk, remove and replace it. The vxfentsthdw utility indicates a disk can be used for I/O fencing with a message resembling:

The disk /dev/rdsk/c2t13d0 is ready to be configured for I/O Fencing on node galaxy

See Adding or removing coordinator disks

 To test disks using vxfentsthdw script

1. Make sure system-to-system communication is functioning properly.

   See Setting up inter-system communication

2. From one node, start the utility. Do one of the following:
   • If you use ssh for communication:

     # /opt/VRTSvcs/vxfen/bin/vxfentsthdw

   • If you use remsh for communication:

     # /opt/VRTSvcs/vxfen/bin/vxfentsthdw -n

3. After reviewing the overview and warning that the tests overwrite data on the disks, confirm to continue the process and enter the node names.

   ******** WARNING!!!!!!!! ********

   THIS UTILITY WILL DESTROY THE DATA ON THE DISK!!

   Do you still want to continue : [y/n] (default: n) y

   Enter the first node of the cluster: galaxy

   Enter the second node of the cluster: nebula

4. Enter the names of the disks you are checking. For each node, the disk may be known by the same name:

   Enter the disk name to be checked for SCSI-3 PGR on node galaxy in the format: /dev/rdsk/cxtxdx

   /dev/rdsk/c2t13d0

   Enter the disk name to be checked for SCSI-3 PGR on node nebula in the format: /dev/rdsk/cxtxdx

   Make sure it's the same disk as seen by nodes galaxy and nebula

   /dev/rdsk/c2t13d0

   If the disk names are not identical, then the test terminates.

5. Review the output as the utility performs the checks and report its activities.
6. If a disk is ready for I/O fencing on each node, the utility reports success:

   ALL tests on the disk /dev/rdsk/c2t13d0 have PASSED.

   The disk is now ready to be configured for I/O Fencing on node galaxy.

   ALL tests on the disk /dev/rdsk/c2t13d0 have PASSED.

   The disk is now ready to be configured for I/O Fencing on node nebula.

   Removing test keys and temporary files, if any ...

   .

   .

7. Run the vxfentsthdw utility for each disk you intend to verify.