Creating storage for OCR and voting disk manually


Creating storage for OCR and voting disk manually
Prev	Creating storage for OCR and voting disk	Next

Use one of the following storage options to create the OCR and voting disk storage:

CVM raw volumes

Note:

This option is supported only for Oracle RAC 10g and Oracle RAC 11g Release 1

See “To create OCR and voting disk volumes on raw volumes”.

CFS

See “To create the storage for OCR and voting disks on CFS”.

Note:

Whether you create volumes or file system directories, you can add them to the VCS configuration to make them highly available.

To create OCR and voting disk volumes on raw volumes

Log in as the root user.
Determine the CVM master:
```
# vxdctl -c mode
```
On the master node, create a shared disk group:
```
# vxdg -s init ocrvotedg Disk_2 Disk_3
```

Create mirrored volumes in the shared group for OCR and voting disk:

# vxassist -g ocrvotedg make ocrvol 300M nmirrors=2

# vxassist -g ocrvotedg make votevol 300M nmirrors=2

Set the ownership for the volumes:

# vxedit -g ocrvotedg set group=oinstall user=oracle mode=660
ocrvol

# vxedit -g ocrvotedg set group=oinstall user=oracle mode=660
votevol

Start the volume:
```
# vxvol -g ocrvotedg startall
```
Add the storage resources to the VCS configuration to make them highly available.

See Adding the storage resources to the VCS configuration.

To create the storage for OCR and voting disks on CFS

Determine the CVM master:
```
# vxdctl -c mode
```
From the master node, create a shared VxVM disk group:
```
# vxdg -s init ocrvotedg Disk_2 Disk_3
```

From the CVM master, create a mirrored volume ocrvotevol:

# vxassist -g ocrvotedg make ocrvotevol 640M nmirrors=2

From the CVM master, create a file system with the volume ocrvotevol.
```
# mkfs -F vxfs /dev/vx/rdsk/ocrvotedg/ocrvotevol
```
On each system, create a directory /ocrvote:
```
# mkdir /ocrvote
```

On each system, mount the directory /ocrvote:

# mount -F vxfs -o cluster /dev/vx/dsk/ocrvotedg/ocrvotevol
/ocrvote

Create files for OCR and voting disk at the root of the file system:
```
# touch /ocrvote/ocr
# touch /ocrvote/vote
```
Note:

If the files are placed under a directory on CFS, the configuration of the CSSD resource (after installing Oracle RAC) using the SF Oracle RAC installer fails.
From any system, change permissions on /ocrvote:
```
# chown -R oracle:oinstall /ocrvote
```
Add the storage resources to the VCS configuration to make them highly available.

See Adding the storage resources to the VCS configuration.

Adding the storage resources to the VCS configuration

The type of storage resource you add to the VCS configuration depends on whether you chose to create the OCR and voting disk storage on raw volumes or CFS. If you chose to create the storage on raw volumes, you need to add a CVMVolDg resource to the VCS configuration. If you chose to create the storage on CFS, you need to add the CVMVolDg and CFSMount resources to the VCS configuration.

Use one of the following ways to add the storage resources to the VCS configuration:

Use the command line interface

See “To add the storage resources created on CFS to the VCS configuration using CLI”.

See “To add the storage resources created on raw volumes to the VCS configuration using CLI”.

Edit main.cf

See “To add the storage resources to the VCS configuration using the main.cf file”.

This procedure requires all the nodes to be restarted.

Note:

Set the attribute "Critical" to "0" for all the resources in the cvm service group. This ensures that critical CVM and CFS resources are always online.

To add the storage resources created on CFS to the VCS configuration using CLI

Change the permissions on the VCS configuration file:
```
# haconf -makerw
```

Configure the CVM volumes under VCS:

# hares -add ocrvote_voldg_ocrvotedg CVMVolDg cvm

# hares -modify ocrvote_voldg_ocrvotedg Critical 0

# hares -modify ocrvote_voldg_ocrvotedg CVMDiskGroup ocrvotedg

# hares -modify ocrvote_voldg_ocrvotedg CVMVolume -add ocrvotevol

# hares -modify ocrvote_voldg_ocrvotedg CVMActivation sw

Set up the file system under VCS:

# hares -add ocrvote_mnt_ocrvotedg CFSMount cvm

# hares -modify ocrvote_mnt_ocrvotedg Critical 0

# hares -modify ocrvote_mnt_ocrvotedg MountPoint "/ocrvote"

# hares -modify ocrvote_mnt_ocrvotedg BlockDevice  \ 
"/dev/vx/dsk/ocrvotedg/ocrvotevol"

# hares -modify ocrvote_mnt_ocrvotedg MountOpt  mincache=direct

Link the parent and child resources:

# hares -link ocrvote_voldg_ocrvotedg cvm_clus

# hares -link ocrvote_mnt_ocrvotedg ocrvote_voldg_ocrvotedg

# hares -link ocrvote_mnt_ocrvotedg vxfsckd

Enable the resources:

# hares -modify ocrvote_voldg_ocrvotedg Enabled 1

# hares -modify ocrvote_mnt_ocrvotedg Enabled 1

# haconf -dump -makero

Verify the configuration of the ocrvote_mnt_ocrvotedg and ocrvote_voldg_ocrvotedg resources in main.cf.

CFSMount ocrvote_mnt_ocrvotedg (  
               Critical = 0                 
               MountPoint = "/ocrvote"                 
               BlockDevice = "/dev/vx/dsk/ocrvotedg/ocrvotevol"                        
               MountOpt= "mincache=direct"
               )
CVMVolDg ocrvote_voldg_ocrvotedg ( 
    				      Critical = 0 
              CVMDiskGroup = ocrvotedg
              CVMVolume = { ocrvotevol } 
              CVMActivation = sw
              )     
ocrvote_mnt_ocrvotedg requires ocrvote_voldg_ocrvotedg       
ocrvote_mnt_ocrvotedg requires vxfsckd   
ocrvote_voldg_ocrvotedg requires cvm_clus

Verify that the ocrvote_mnt_ocrvotedg and ocrvote_voldg_ocrvotedg resources are online on all systems in the cluster.
```
# hares -state ocrvote_voldg_ocrvotedg
```
```
# hares -state ocrvote_mnt_ocrvotedg
```
Note:

The ocrvote_voldg_ocrvotedg resource is reported offline though the underlying volume is online. You need to manually bring the resource online on each node.

To bring the resource online manually:
```
# hares -online ocrvote_voldg_ocrvotedg -sys galaxy
```
```
# hares -online ocrvote_voldg_ocrvotedg -sys nebula
```

To add the storage resources created on raw volumes to the VCS configuration using CLI

Change the permissions on the VCS configuration file:
```
# haconf -makerw
```

Configure the CVM volumes under VCS:

# hares -add ocrvote_voldg_ocrvotedg CVMVolDg cvm

# hares -modify ocrvote_voldg_ocrvotedg Critical 0

# hares -modify ocrvote_voldg_ocrvotedg CVMDiskGroup ocrvotedg

# hares -modify ocrvote_voldg_ocrvotedg CVMVolume -add ocrvol

# hares -modify ocrvote_voldg_ocrvotedg CVMVolume -add votevol

# hares -modify ocrvote_voldg_ocrvotedg CVMActivation sw

Link the parent and child resources:

# hares -link ocrvote_voldg_ocrvotedg cvm_clus

Enable the resources:

# hares -modify ocrvote_voldg_ocrvotedg Enabled 1

# haconf -dump -makero

Verify the configuration of the ocrvote_voldg_ocrvotedg resource in main.cf.

CVMVolDg ocrvote_voldg_ocrvotedg ( 
    				      Critical = 0
														CVMDiskGroup = ocrvotedg
              CVMVolume = { ocrvol, votevol } 
              CVMActivation = sw
              )     
ocrvote_voldg_ocrvotedg requires cvm_clus

Verify that the ocrvote_voldg_ocrvotedg resource is online on all systems in the cluster.
```
# hares -state ocrvote_voldg_ocrvotedg
```
Note:

The ocrvote_voldg_ocrvotedg resource is reported offline though the underlying volume is online. You need to manually bring the resource online on each node.

To bring the resource online manually:
```
# hares -online ocrvote_voldg_ocrvotedg -sys galaxy
```
```
# hares -online ocrvote_voldg_ocrvotedg -sys nebula
```

To add the storage resources to the VCS configuration using the main.cf file

Log in to one of the systems as the root user.
Save your existing configuration:
```
# haconf -dump -makero
```
If your configuration is not writable, a warning appears: "Cluster not writable".

You may ignore the warning.
Stop the VCS engine on all systems and leave the resources available.
```
# hastop -all -force
```

Back up the /etc/VRTSvcs/conf/config/main.cf file:

# cd /etc/VRTSvcs/conf/config

# cp -p main.cf main.orig

Modify the CVM service group.

If you created the storage for OCR and voting disk on CVM raw volumes:

CVMVolDg ocrvote_voldg_ocrvotedg ( 
    				      Critical = 0 
              CVMDiskGroup = ocrvotedg
              CVMVolume = { ocrvol, votevol } 
              CVMActivation = sw
              )     
ocrvote_voldg_ocrvotedg requires cvm_clus

If you created the storage for OCR and voting disk on CFS:

CFSMount ocrvote_mnt_ocrvotedg (  
               Critical = 0                 
               MountPoint = "/ocrvote"                 
               BlockDevice = "/dev/vx/dsk/ocrvotedg/ocrvotevol"                        
               MountOpt= "mincache=direct"
               )
CVMVolDg ocrvote_voldg_ocrvotedg ( 
    				      Critical = 0 
              CVMDiskGroup = ocrvotedg
              CVMVolume = { ocrvotevol } 
              CVMActivation = sw
              )     
ocrvote_mnt_ocrvotedg requires ocrvote_voldg_ocrvotedg       
ocrvote_mnt_ocrvotedg requires vxfsckd   
ocrvote_voldg_ocrvotedg requires cvm_clus

Save and close the file.

Verify the syntax of the main.cf file:

# cd /etc/VRTSvcs/conf/config

# hacf -verify .

Start the VCS engine on one of the nodes:
```
# hastart
```
Verify the status of VCS:
```
# hastatus
```
When "LOCAL_BUILD" is listed in the message column, start VCS on the other system with the following command:
```
# hastart
```
Verify that the CVM group is online:
```
# hagrp -state
```
Verify that the ocrvote_mnt_ocrvotedg and ocrvote_voldg_ocrvotedg resources are online:
```
# hares -state
```

Prev	Up	Next
Creating storage for OCR and voting disk using the installer	Home	Configuring private IP addresses for Oracle RAC