Creating storage for OCR and voting disk manually

Use one of the following storage options to create the OCR and voting disk storage:

CVM raw volumes

Note:

This option is supported only for Oracle RAC 10g and Oracle RAC 11g Release 1

See “To create OCR and voting disk volumes on raw volumes”.

CFS

See “To create the storage for OCR and voting disks on CFS”.

Note:

Whether you create volumes or file system directories, you can add them to the VCS configuration to make them highly available.

To create OCR and voting disk volumes on raw volumes

  1. Log in as the root user.

  2. Determine the CVM master:

    # vxdctl -c mode
  3. On the master node, create a shared disk group:

    # vxdg -s init ocrvotedg Disk_2 Disk_3
  4. Create mirrored volumes in the shared group for OCR and voting disk:

    # vxassist -g ocrvotedg make ocrvol 300M nmirrors=2
    # vxassist -g ocrvotedg make votevol 300M nmirrors=2
  5. Set the ownership for the volumes:

    # vxedit -g ocrvotedg set group=oinstall user=oracle mode=660
    ocrvol
    # vxedit -g ocrvotedg set group=oinstall user=oracle mode=660
    votevol
  6. Start the volume:

    # vxvol -g ocrvotedg startall
  7. Add the storage resources to the VCS configuration to make them highly available.

    See Adding the storage resources to the VCS configuration.

To create the storage for OCR and voting disks on CFS

  1. Determine the CVM master:

    # vxdctl -c mode
  2. From the master node, create a shared VxVM disk group:

    # vxdg -s init ocrvotedg Disk_2 Disk_3
  3. From the CVM master, create a mirrored volume ocrvotevol:

    # vxassist -g ocrvotedg make ocrvotevol 640M nmirrors=2

  4. From the CVM master, create a file system with the volume ocrvotevol.

    # mkfs -F vxfs /dev/vx/rdsk/ocrvotedg/ocrvotevol

  5. On each system, create a directory /ocrvote:

    # mkdir /ocrvote
  6. On each system, mount the directory /ocrvote:

    # mount -F vxfs -o cluster /dev/vx/dsk/ocrvotedg/ocrvotevol
    /ocrvote
  7. Create files for OCR and voting disk at the root of the file system:

    # touch /ocrvote/ocr
    # touch /ocrvote/vote

    Note:

    If the files are placed under a directory on CFS, the configuration of the CSSD resource (after installing Oracle RAC) using the SF Oracle RAC installer fails.

  8. From any system, change permissions on /ocrvote:

    # chown -R oracle:oinstall /ocrvote
  9. Add the storage resources to the VCS configuration to make them highly available.

    See Adding the storage resources to the VCS configuration.

Adding the storage resources to the VCS configuration

The type of storage resource you add to the VCS configuration depends on whether you chose to create the OCR and voting disk storage on raw volumes or CFS. If you chose to create the storage on raw volumes, you need to add a CVMVolDg resource to the VCS configuration. If you chose to create the storage on CFS, you need to add the CVMVolDg and CFSMount resources to the VCS configuration.

Use one of the following ways to add the storage resources to the VCS configuration:

Note:

Set the attribute "Critical" to "0" for all the resources in the cvm service group. This ensures that critical CVM and CFS resources are always online.

To add the storage resources created on CFS to the VCS configuration using CLI

  1. Change the permissions on the VCS configuration file:

    # haconf -makerw
  2. Configure the CVM volumes under VCS:

    # hares -add ocrvote_voldg_ocrvotedg CVMVolDg cvm
    # hares -modify ocrvote_voldg_ocrvotedg Critical 0
    # hares -modify ocrvote_voldg_ocrvotedg CVMDiskGroup ocrvotedg
    # hares -modify ocrvote_voldg_ocrvotedg CVMVolume -add ocrvotevol
    # hares -modify ocrvote_voldg_ocrvotedg CVMActivation sw
  3. Set up the file system under VCS:

    # hares -add ocrvote_mnt_ocrvotedg CFSMount cvm
    # hares -modify ocrvote_mnt_ocrvotedg Critical 0
    # hares -modify ocrvote_mnt_ocrvotedg MountPoint "/ocrvote"
    # hares -modify ocrvote_mnt_ocrvotedg BlockDevice  \ 
    "/dev/vx/dsk/ocrvotedg/ocrvotevol"
    # hares -modify ocrvote_mnt_ocrvotedg MountOpt  mincache=direct
  4. Link the parent and child resources:

    # hares -link ocrvote_voldg_ocrvotedg cvm_clus
    # hares -link ocrvote_mnt_ocrvotedg ocrvote_voldg_ocrvotedg
    # hares -link ocrvote_mnt_ocrvotedg vxfsckd
  5. Enable the resources:

    # hares -modify ocrvote_voldg_ocrvotedg Enabled 1
    # hares -modify ocrvote_mnt_ocrvotedg Enabled 1
    # haconf -dump -makero
  6. Verify the configuration of the ocrvote_mnt_ocrvotedg and ocrvote_voldg_ocrvotedg resources in main.cf.

    CFSMount ocrvote_mnt_ocrvotedg (  
                   Critical = 0                 
                   MountPoint = "/ocrvote"                 
                   BlockDevice = "/dev/vx/dsk/ocrvotedg/ocrvotevol"                        
                   MountOpt= "mincache=direct"
                   )
    CVMVolDg ocrvote_voldg_ocrvotedg ( 
        				      Critical = 0 
                  CVMDiskGroup = ocrvotedg
                  CVMVolume = { ocrvotevol } 
                  CVMActivation = sw
                  )     
    ocrvote_mnt_ocrvotedg requires ocrvote_voldg_ocrvotedg       
    ocrvote_mnt_ocrvotedg requires vxfsckd   
    ocrvote_voldg_ocrvotedg requires cvm_clus

  7. Verify that the ocrvote_mnt_ocrvotedg and ocrvote_voldg_ocrvotedg resources are online on all systems in the cluster.

    # hares -state ocrvote_voldg_ocrvotedg
    # hares -state ocrvote_mnt_ocrvotedg

    Note:

    The ocrvote_voldg_ocrvotedg resource is reported offline though the underlying volume is online. You need to manually bring the resource online on each node.

    To bring the resource online manually:

    # hares -online ocrvote_voldg_ocrvotedg -sys galaxy
    # hares -online ocrvote_voldg_ocrvotedg -sys nebula

To add the storage resources created on raw volumes to the VCS configuration using CLI

  1. Change the permissions on the VCS configuration file:

    # haconf -makerw
  2. Configure the CVM volumes under VCS:

    # hares -add ocrvote_voldg_ocrvotedg CVMVolDg cvm
    # hares -modify ocrvote_voldg_ocrvotedg Critical 0
    # hares -modify ocrvote_voldg_ocrvotedg CVMDiskGroup ocrvotedg
    # hares -modify ocrvote_voldg_ocrvotedg CVMVolume -add ocrvol
    # hares -modify ocrvote_voldg_ocrvotedg CVMVolume -add votevol
    # hares -modify ocrvote_voldg_ocrvotedg CVMActivation sw
  3. Link the parent and child resources:

    # hares -link ocrvote_voldg_ocrvotedg cvm_clus
  4. Enable the resources:

    # hares -modify ocrvote_voldg_ocrvotedg Enabled 1
    # haconf -dump -makero
  5. Verify the configuration of the ocrvote_voldg_ocrvotedg resource in main.cf.

    CVMVolDg ocrvote_voldg_ocrvotedg ( 
        				      Critical = 0
    														CVMDiskGroup = ocrvotedg
                  CVMVolume = { ocrvol, votevol } 
                  CVMActivation = sw
                  )     
    ocrvote_voldg_ocrvotedg requires cvm_clus

  6. Verify that the ocrvote_voldg_ocrvotedg resource is online on all systems in the cluster.

    # hares -state ocrvote_voldg_ocrvotedg

    Note:

    The ocrvote_voldg_ocrvotedg resource is reported offline though the underlying volume is online. You need to manually bring the resource online on each node.

    To bring the resource online manually:

    # hares -online ocrvote_voldg_ocrvotedg -sys galaxy
    # hares -online ocrvote_voldg_ocrvotedg -sys nebula

To add the storage resources to the VCS configuration using the main.cf file

  1. Log in to one of the systems as the root user.

  2. Save your existing configuration:

    # haconf -dump -makero

    If your configuration is not writable, a warning appears: "Cluster not writable".

    You may ignore the warning.

  3. Stop the VCS engine on all systems and leave the resources available.

    # hastop -all -force
  4. Back up the /etc/VRTSvcs/conf/config/main.cf file:

    # cd /etc/VRTSvcs/conf/config
    # cp -p main.cf main.orig
  5. Modify the CVM service group.

    If you created the storage for OCR and voting disk on CVM raw volumes:

    CVMVolDg ocrvote_voldg_ocrvotedg ( 
        				      Critical = 0 
                  CVMDiskGroup = ocrvotedg
                  CVMVolume = { ocrvol, votevol } 
                  CVMActivation = sw
                  )     
    ocrvote_voldg_ocrvotedg requires cvm_clus

    If you created the storage for OCR and voting disk on CFS:

    CFSMount ocrvote_mnt_ocrvotedg (  
                   Critical = 0                 
                   MountPoint = "/ocrvote"                 
                   BlockDevice = "/dev/vx/dsk/ocrvotedg/ocrvotevol"                        
                   MountOpt= "mincache=direct"
                   )
    CVMVolDg ocrvote_voldg_ocrvotedg ( 
        				      Critical = 0 
                  CVMDiskGroup = ocrvotedg
                  CVMVolume = { ocrvotevol } 
                  CVMActivation = sw
                  )     
    ocrvote_mnt_ocrvotedg requires ocrvote_voldg_ocrvotedg       
    ocrvote_mnt_ocrvotedg requires vxfsckd   
    ocrvote_voldg_ocrvotedg requires cvm_clus
  6. Save and close the file.

  7. Verify the syntax of the main.cf file:

    # cd /etc/VRTSvcs/conf/config
    # hacf -verify .
  8. Start the VCS engine on one of the nodes:

    # hastart
  9. Verify the status of VCS:

    # hastatus
  10. When "LOCAL_BUILD" is listed in the message column, start VCS on the other system with the following command:

    # hastart
  11. Verify that the CVM group is online:

    # hagrp -state
  12. Verify that the ocrvote_mnt_ocrvotedg and ocrvote_voldg_ocrvotedg resources are online:

    # hares -state