< Previous  | TOC | Index | Next  >

Upgrading to SF 5.0 Oracle RAC

The procedure in the following sections assumes that you have a working SF Oracle RAC cluster at 4.1.

Overview of upgrade tasks

You will have to perform the following tasks:

• Stopping Oracle and cluster resources
• Invoking the installer to upgrade to SF 5.0 Oracle RAC

  ---

    Note   When invoking the installer, use the -rsh option if you have not configured SSH communications between systems. SSH is the default.

  ---

• Reinstating Oracle and other cluster resources after upgrading

Stopping Oracle and the cluster resources

1. Log in as root user to one of the nodes, galaxy for example, in the cluster.
2. Make a backup of VCS configuration files, main.cf and vcsmm.conf.

   # cp /etc/VRTSvcs/conf/config/main.cf \ /etc/VRTSvcs/conf/config/main.cf.save

   # cp /kernel/drv/vcsmm.conf /kernel/drv/vcsmm.conf.save

3. On the one node, galaxy for example, stop all Oracle instances in the cluster.
   1. If Oracle RAC instances are under VCS control, offline the corresponding service group:

      # hagrp -offlne <ora_grp> -any

   2. If Oracle RAC instances are managed by CRS, stop them using the CRS commands.
4. Stop the Oracle listener process. If it is configured under VCS control, enter:

   # hares -offline <listener-res> -sys galaxy

   # hares -offline <listener-res> -sys nebula

5. Stop CRS on each node in the cluster. If it is configured under VCS control, and assuming the "cssd" resources exist under VCS configuration, enter:

   # hares -offline cssd -sys galaxy

   # hares -offline cssd -sys nebula

6. Take the ocr and vote volumes or file systems offline.
7. Take offline all other VCS groups that depend on VxFS file systems or VxVM disk groups, whether local or CFS.
8. Stop all applications that use VxFS or VxVM but that are not under VCS control. Unmount the corresponding file systems and deport the corresponding disk groups.
9. Set the AutoStart attribute to 0 for all VCS resources for Oracle RAC instances and the cssd resource. For example:

   # haconf -makerw

   # hares -modify cssd AutoStart 0

   # hares -modify <oracle_res> AutoStart 0

   # haconf -dump -makero

Invoking installsfrac to upgrade to SF Oracle 5.0

1. Mount the DVD-ROM software disc. Notice the following displayed information:

   If volume management software is running on your system, the software disc automatically mounts as /cdrom. If volume management software is not available to mount the DVD-ROM, you must mount it manually, enter:

   # mount -F hsfs -o ro /dev/dsk/c0t6d0s2 /cdrom

2. Start the SF Oracle RAC installer utility. Include the -rsh option if you have not set up SSH:

   # cd /cdrom/dvd1/storage_foundation_for_oracle_rac

   # ./installsfrac galaxy nebula

3. The installer displays information about the cluster, the systems, and the service groups and prompts you about upgrading. At the prompt to upgrade, enter "y":

   ...

   Do you want to upgrade to version 5.0 on these systems using the current configuration? [y,n,q,?] (y)

4. The installer verifies rsh accessibility and the existing licenses.

   ...

   Do you want to enter another license key for galaxy? [y,n,q](n)

5. If necessary, answer "y" and enter the license keys at the prompt.
6. When the installer prompts you to confirm whether to proceed with the upgrade. Answer "y" to proceed.
7. As the installer proceeds, it:
   1. Takes the cvm group and other dependent group offline.
   2. Updates LLT and GAB configurations as needed for SF 5.0 Oracle RAC.
8. When it is ready to stop the cluster to make updates to the configuration, it prompts for your OK. At the prompt, press Enter.

   ...

   Are you ready to begin the Veritas Storage Foundation for Oracle RAC upgrade at this time?

9. The installer backs up the configuration files, freezes service groups, stops processes, and uninstalls the 4.1 SF Oracle RAC filesets.

   Backing up types.cf and main.cf .................Done

   Freezing group cvm ..............................Done

   ...

   Stopping SFRAC: 100%

   Shutdown completed successfully on all systems

   Uninstalling SFRAC: 100%

   Uninstall completed successfully on all systems

   Installing SFRAC: 100%

   Installation completed successfully on all systems

10. The installer displays where logs for the upgrade can be found. It then indicates you must reboot each system.
11. Depending on whether you want to use the DMP configuration or the raw device configuration, use one of the following commands:
    • For DMP configuration (preferred):

      # cp /etc/vxfen.d/vxfenmode_scsi3_dmp /etc/vxfenmode

    • For raw device configuration:

      # cp /etc/vxfen.d/vxfenmode_scsi3_raw /etc/vxfenmode

12. Restore the vcsmm.conf configuration file.

    # cp /kernel/drv/vcsmm.conf.save /kernel/drv/vcsmm.conf

13. Reboot each system in the cluster.

    # /usr/sbin/shutdown -g0 -y -i6

Performing post-upgrade tasks for SF 5.0 Oracle RAC

1. When the systems come back up after the rebooting, GAB ports, a, b, d, h, and o are configured. The VCS service groups are frozen.
2. Invoke installsfrac once again, this time to enter configuration information necessary to upgrade to SF 5.0 Oracle RAC. Include the -rsh option if you have not set up SSH:

   # cd /opt/VRTS/install

   #./installsfrac -configure

3. Enter the system names when prompted:

   Enter the system names separated by spaces on which to configure SFRAC: galaxy nebula

4. The installer now verifies rsh accessibility between the systems and prompts you whether to use the same VCS/GAB/LLT configuration. Answer "y."

   Are you sure you want to reuse configuration of SFRAC 4.1 [y,n,q] (y)

5. Next, installer will bring up the SF Oracle RAC "Main Menu". Select the option to "Configure SFRAC" (2):

   1) Check systems for SFRAC ** INSTRUCTIONS ONLY **

   2) Configure SFRAC

   3) Prepare to install Oracle

   4) Install or relink Oracle

6. From the "Configure SFRAC menu, select "Configure VCS, CVM and CFS" (1):

   1) Configure VCS, CVM and CFS

   2) Configure I/O fencing ** INSTRUCTIONS ONLY **

   b) [Go to previous menu]

7. The installer now stops all SF Oracle RAC processes.

   Stopping SFRAC processes. Please wait...

   SFRAC processes are stopped

   Press [Return] to continue:

8. The installer next prompts you to change permissions to allow the current oracle user and group to access Veritas Storage Foundation for Oracle RAC tools. The questions are asked for each system in the cluster.

   ...

   The Veritas Storage Foundation for Oracle RAC utility for database administration requires permission changes to allow database administrators (DBAs) access to the tools. The default settings at installation time for the /opt/VRTSdbed directory allows only the root user access to the directory. You need to change permissions to allow a DBA or a group of DBAs access to the Veritas Storage Foundation for Oracle RAC tools

   Do you want to add single user access on galaxy [y,n,q,?] (Y)

   Answer "y"

   Enter login account name for DBA user: oracle

   Enter group name for DBA users: dba

9. For each system, the installer prompts you to verify the fully qualified hostnames. For example:

   ...

   Is the fully qualified hostname of system "galaxy" =

   "galaxy.example.com"? [y,n,q,b,?] (y) y

   Querying fully qualified domain name of host "galaxy" .. ok

   Is the fully qualified hostname of system "nebula" =

   "nebula.example.com"? [y,n,q,b] (y) y

10. When you are prompted about enabling Storage Foundation Management Sever, answer "n".

    ...

    Enable Storage Foundation Management Server Management? [y,n,q,b] (y) n

11. The installer now configures VEA, gridnode, actionagent, and Storageagent for all nodes in the cluster.

    Verify the install systems Fully Qualified Hostnames.

    Configuring VEA in STANDALONE mode on "galaxy" ..... ok

    Configuring gridnode on "galaxy" ................... ok

    Registering gridnode on "galaxy" ................... ok

    Configuring actionagent on "galaxy" ................ ok

    Registering actionagent on "galaxy" ................ ok

    Stopping StorageAgent on "galaxy" .................. ok

    Registering StorageAgent on "galaxy" ............... ok

    ....

12. When you are prompted to start SF Oracle RAC processes, accept the default "y" by pressing Enter.

    Do you want to start Veritas Storage Foundation for Oracle RAC processes now? [y,n,q,b] (y)

    Starting SFRAC:100%

    Installation completed successfully on all systems

    ...

13. Ignore the message about executing shutdown that displays after the CFS agents are configured.
14. At this point, all SF Oracle RAC components should be up and all service groups should be online. Verify that all 8 GAB ports are configured. Also, cssd and Oracle resources would also be online (still linked to the old 4.1 SF Oracle RAC libraries).
15. Import any VxVM disk groups not under VCS control. Also mount any VxFS file systems not under VCS control.

Relinking Oracle10g (Installer)

1. Invoke installsfrac once again:

   # cd /opt/VRTS/install

   #./installsfrac -configure

2. Enter the system names when prompted:

   Enter the system names separated by spaces on which to configure SFRAC: galaxy nebula

3. Navigate to the "Install and Relink Oracle" menu.
   1. Select the appropriate Oracle 10g version (3):

      1) Oracle 10gR1

      2) Oracle 10gR2

   2. Select "Relink Oracle" (3) from the menu:

      1) Install Oracle Clusterware (CRS)

      2) Install Oracle RDBMS server

      3) Relink Oracle

      b) [Go to previous menu]

   3. From the menu displayed, enter the required information. For example:

      Enter Oracle UNIX user name: [b] (oracle) oracle

      Enter Oracle UNIX group name: [b] (oinstall) oinstall

      Enter Oracle base directory: [b] /app/oracle

      Enter absolute path of CRS Home directory: [b]

      /app/crshome

      Enter absolute path of Database Home directory: [b]

      /app/oracle/orahome

      Enter Oracle Bits (64/32) [b] (64) 64

   4. Confirm you responses in the verification screen. The installer copies the SF 5.0 Oracle RAC libraries to /opt/ORCLcluster, where it expects libskgxn.

      Oracle environment information verification

      Oracle Unix User: oracle

      Oracle Unix Group: oinstall

      Oracle Clusterware (CRS) Home:

      /app/crshome

      Oracle Release: 10.2

      Oracle Bits: 64

      Oracle Base: /app/oracle

      Oracle Home: /app/oracle/orahome

      Is this information correct? [y,n,q] (y)

      galaxy
      Copying /opt/VRTSvcs/rac/lib/libskgxn2_64.so

      /opt/ORCLcluster/lib/libskgxn2.so ........... success
      
      nebula
      Copying /opt/VRTSvcs/rac/lib/libskgxn2_64.so

      /opt/ORCLcluster/lib/libskgxn2.so .............. success

      galaxy

      Copying /opt/VRTSvcs/rac/lib/libskgxp10_ver25_64.so to

      /app/oracle/orahome/lib/libskgxp10.so ........... success

      Removing /oracle/10g/lib/libodm10.so ............ success

      Linking /opt/VRTSodm/lib/amd64/libodm.so /app/oracle/orahome/lib/libodm10.so ... success

      Setting permissions oracle:oinstall /app/oracle/orahome/lib/libskgxp10.so ... success

      nebula

      Copying /opt/VRTSvcs/rac/lib/libskgxp10_ver25_64.so to

      /app/oracle/orahome/lib/libskgxp10.so ........... success

      Removing /oracle/10g/lib/libodm10.so ............ success

      Linking /opt/VRTSodm/lib/amd64/libodm.so /app/oracle/orahome/lib/libodm10.so ... success

      Setting permissions oracle:oinstall /app/oracle/orahome/lib/libskgxp10.so ... success

   5. Enter "q" at the next prompt to leave the installer now that CRS setup tasks are complete.

4. Bring the CSSD resource online. Enter:

   # hares -online cssd -sys galaxy

   # hares -online cssd -sys nebula

5. Confirm that CRS in online. Enter:

   $CRS_HOME/bin/crs_stat -t

6. Bring online the oracle resources configured under VCS. If they're directly controlled by CRS, you may run the CRS commands to start the instance.

Relinking Oracle 10g (Using the command line)

1. For 10gR1, enter one set of the following commands:
2. For 32bit oracle:

   # cp /opt/VRTSvcs/rac/lib/libskgxn2_32.so \

   /opt/ORCLcluster/rac/lib/libskgxn2.so

   $ cp /opt/VRTSvcs/rac/lib/libskgxp10_ver23_32.so \

   $ORACLE_HOME/lib/libskgxp.so

   $ ln -s /usr/lib/libodm.so libodm10.so

3. For 64bit oracle:

   # cp /opt/VRTSvcs/rac/lib/libskgxn2_64.so \

   /opt/ORCLcluster/rac/lib/libskgxn2.so

   $ cp /opt/VRTSvcs/rac/lib/libskgxp10_ver23_64.so \

   $ORACLE_HOME/lib/libskgxp.so

   $ ln -s /usr/lib/amd64/libodm.so libodm10.so

2. For 10gR2, enter one set of the following commands:
3. For 32bit oracle:

   # cp /opt/VRTSvcs/rac/lib/libskgxn2_32.so \

   /opt/ORCLcluster/rac/lib/libskgxn2.so

   $ cp /opt/VRTSvcs/rac/lib/libskgxp10_ver25_32.so \

   $ORACLE_HOME/lib/libskgxp.so

4. For 64bit oracle:

   # cp /opt/VRTSvcs/rac/lib/libskgxn2_64.so \

   /opt/ORCLcluster/rac/lib/libskgxn2.so

   $ cp /opt/VRTSvcs/rac/lib/libskgxp10_ver25_64.so \

   $ORACLE_HOME/lib/libskgxp.so

3. Bring the CSSD resource online. Enter:

   # hares -online cssd -sys galaxy

   # hares -online cssd -sys nebula

4. Confirm that CRS in online. Enter:

   $CRS_HOME/bin/crs_stat -t

5. Bring online the oracle resources configured under VCS. If they're directly controlled by CRS, you may run the CRS commands to start the instance.