< Previous  | TOC | Index | Next  >

Before you configure coordinator disks

I/O fencing requires coordinator disks to be configured in a disk group that each cluster system can access. The use of coordinator disks enables the vxfen driver to resolve potential split-brain conditions and prevent data corruption. A coordinator disk is not used for data storage, so it can be configured as the smallest LUN on a disk array to avoid wasting space.

Coordinator disks must meet the following requirements:

• There must be at least three coordinator disks and the total number of coordinator disks must be an odd number. This ensures a majority of disks can be achieved.
• Each of the coordinator disks must use a physically separate disk or LUN.
• Each of the coordinator disks should be on a different disk array, if possible.
• Coordinator disks in a disk array should use hardware-based mirroring.
• The coordinator disks must support SCSI-3 PR. Note that the use of the vxfentsthdw utility to test for SCSI-3 PR support requires that disks be 1MB or greater. Smaller disks can be tested manually.

  Contact Veritas support (http://support.veritas.com) for the procedure.

Setting up the disk group for coordinator disks

If you have already added and initialized disks you intend to use as coordinator disks, you can begin the following procedure at step 7.

 To set up the disk group for coordinator disks

1. Physically add the three disks you intend to use for coordinator disks. All cluster nodes should physically share them. Veritas recommends that you use the smallest size disks or LUNs, so that space for data is not wasted.
2. If necessary, use the vxdisk scandisks command to scan the disk drives and their attributes. This command updates the VxVM device list and reconfigures DMP with the new devices. For example:

   # vxdisk scandisks

3. Use the vxdmpadm getdmpnode command to determine the VxVM name by which a disk drive (or LUN) is known.
4. In the following example, a disk with the AIX device name /dev/rhdisk75 is identified by VxVM as EMC0_17:

   # vxdmpadm getdmpnode nodename=hdisk75

   NAME STATE ENCLR-TYPE PATHS ENBL DSBL ENCLR-NAME

   ============================================================

   EMC0_17 ENABLED EMC 1 1 0 EMC0

5. You can, as an option, run the command vxdisk list vxvm_device_name to see additional information about the disk, including the AIX device name. For example:

   # vxdisk list EMC0_17

6. Use the command vxdisksetup command to initialize a disk as a VxVM disk. The example command that follows specifies the CDS format:

   # vxdisksetup -i vxvm_device_name format=cdsdisk

   For example:

   # vxdisksetup -i EMC0_17 format=cdsdisk

   Repeat this command for each disk you intend to use as a coordinator disk.

7. From one node, create a disk group named vxfencoorddg. This group must contain an odd number of disks or LUNs and a minimum of three disks. Symantec recommends that you use only three coordinator disks, and that you use the smallest size disks or LUNs to conserve disk space.

   For example, assume the disks have the device names EMC0_12, EMC0_16, and EMC0_17.

8. On any node, create the disk group by specifying the device name of one of the disks.

   # vxdg -o coordinator=on init vxfencoorddg EMC0_12

9. Add the other two disks to the disk group.

   # vxdg -g vxfencoorddg adddisk EMC0_16

   # vxdg -g vxfencoorddg adddisk EMC0_17

See the Veritas Volume Manager Administrator's Guide.

Requirements for testing the coordinator disk group

Review the requirments for testing the coordinator disk group.

Run the vxfentsthdw utility

Review the following guidelines on testing support for SCSI-3:

• The utility requires that the coordinator disk group be accessible from two systems. For example, if you have a four-system cluster, select any two systems for the test.
• If you configured ssh (SSH client) for the cluster nodes, VXFENTSTHDW can be used as long as ssh commands between nodes can execute without password prompting and confirmation.

  If you did not configure ssh, enable each node to have remote rsh access to the other nodes during installation and disk verification. On each node, placing a "+" character in the first line of the /.rhosts file gives remote access to the system running the install program. You can limit the remote access to specific nodes. Refer to the manual page for the /.rhosts file for more information. Remove the remote rsh access permissions after the installation and disk verification process.

• ssh is used by default and rsh is only used if you do use the vxfentsthdw -n command.
• To ensure both nodes are connected to the same disk during the test, use the vxfenadm -i diskpath command to verify the disk serial number.
• The vxfentsthdw utility has additional options suitable for testing many disks. You can test disks without destroying data using the -r option. The options for testing disk groups (-g) and disks listed in a file (-f) are described in detail:

Testing the coordinator disk group

After Setting up the disk group for coordinator disks, test the coordinator disk group.

 To test the coordinator disk group

1. To start the utility from one node, type the following command:

   # /opt/VRTSvcs/vxfen/bin/vxfentsthdw

   Make sure system-to-system communication is functioning properly before performing this step.

   See the vxfentsthdw(1M) man page.

2. After reviewing the overview and warning about overwriting data on the disks, confirm to continue the process and enter the node names.
3. Enter the name of the disk you are checking.

   For example, assume you must check a shared device known by two systems as /dev/rhdisk76.

Creating the vxfendg file

After Testing the coordinator disk group, configure it for use.

 To create the vxfendg file

1. To deport the disk group, type the following command:

   # vxdg deport vxfencoorddg

2. Import the disk group with the -t option to avoid automatically importing it when the nodes restart:

   # vxdg -t import vxfencoorddg

3. Deport the disk group. This operation prevents the coordinator disks from serving other purposes:

   # vxdg deport vxfencoorddg

4. On all nodes, type:

   # echo "vxfencoorddg" > /etc/vxfendg

   Do no use spaces between the quotes in the "vxfencoorddg" text.

   This command creates the /etc/vxfendg file, which includes the name of the coordinator disk group. Based on the contents of the /etc/vxfendg file, the rc script creates the /etc/vxfentab file for use by the vxfen driver when the system starts. The rc script also invokes the vxfenconfig command, which configures the vxfen driver to start and use the coordinator disks listed in /etc/vxfentab. /etc/vxfentab is a generated file; do not modify this file.