Symantec logo

< Previous  | TOC | Index | Next  >  

Upgrading to SF 5.0 Oracle RAC

The procedure in the following sections assumes that you have a working SF Oracle RAC cluster at 4.1.

Overview of upgrade tasks

You will have to perform the following tasks:

Stopping Oracle and the cluster resources
  1. Log in as root user to one of the nodes, galaxy for example, in the cluster.
  2. Make a backup of VCS configuration files, main.cf and vcsmm.conf.

    # cp /etc/VRTSvcs/conf/config/main.cf \ /etc/VRTSvcs/conf/config/main.cf.save

    # cp /kernel/drv/vcsmm.conf /kernel/drv/vcsmm.conf.save

  3. On all nodes, if you are using Oracle 9iR2, stop any gsd processes that may be started. Log in as Oracle user and enter:

    # $ORACLE_HOME/bin/gsdctl stop

  4. On the one node, galaxy for example, stop all Oracle instances in the cluster.
    1. If Oracle RAC instances are under VCS control, offline the corresponding service group:

      # hagrp -offlne <ora_grp> -any

    2. If Oracle RAC instances are managed by CRS (Oracle 10g only), stop them using the CRS commands.
  5. Stop the Oracle listener process. If it is configured under VCS control, enter:

    # hares -offline <listener-res> -sys galaxy

    # hares -offline <listener-res> -sys nebula

  6. If you are using Oracle 10g R2, stop CRS on each node in the cluster. Assuming the "cssd" resources exist under VCS configuration, enter:

    # hares -offline cssd -sys galaxy

    # hares -offline cssd -sys nebula

  7. For Oracle 10g R1 or R2, take offline the ocr and vote volumes or file systems.
  8. Take offline all other VCS groups that depend on VxFS file systems or VxVM disk groups, whether local or CFS.
  9. Stop all applications that use VxFS or VxVM but that are not under VCS control. Unmount the corresponding file systems and deport the corresponding disk groups.
  10. Set the AutoStart attribute to 0 for all VCS resources for Oracle RAC instances and the cssd resource. For example:

    # haconf -makerw

    # hares -modify cssd AutoStart 0

    # hares -modify <oracle_res> AutoStart 0

    # haconf -dump -makero

  11. Make sure no service groups dependent on qlogckd. Enter:

    # /opt/VRTS/bin/hares -dep qlogckd

  12. Take the qlogckd resources offline. You must specify the -sys option with the system names in the cluster. For example:

    # /opt/VRTS/bin/hares -offline qlogckd -sys galaxy

    # /opt/VRTS/bin/hares -offline qlogckd -sys nebula

    ...

  13. Stop VCS. From the one node (galaxy, for example), enter:

    # /opt/VRTS/bin/hastop -all -force

  14. Remove qlogckd from the /etc/VRTSvcs/conf/config/main.cf file. For example:
    1. Remove the resource definition:

      CFSQlogckd qlogckd (

      Critical = 0

      )

    2. Remove its definition in any dependency statement. Change:

      qlogckd requires cvm_clus

      vxfsckd requires qlogckd

      To:

      vxfsckd requires cvm_clus

    3. Remove it from the dependency comment section. Change:

      // resource dependency tree

      //

      // group cvm

      // {

      // CFSfsckd vxfsckd

      // {

      // CFSQlogckd qlogckd

      // {

      // CVMCluster cvm_clus

      // {

      // CVMVxconfigd cvm_vxconfigd

      // }

      // }

      // }

      // }

      To:

      // resource dependency tree

      //

      // group cvm

      // {

      // CFSfsckd vxfsckd

      // {

      // CVMCluster cvm_clus

      // {

      // CVMVxconfigd cvm_vxconfigd

      // }

      // }

      // }

    4. Save and close the file. Verify the syntax of the main.cf.

      # cd /etc/VRTSvcs/conf/config

      # /opt/VRTS/bin/hacf -verify .

  15. Start VCS:

    # /opt/VRTS/bin/hastart

  16. Start VCS on other nodes:

    # /opt/VRTS/bin/hastart

Invoking installsfrac to upgrade to SF Oracle 5.0
  1. Mount the DVD-ROM software disc. Notice the following displayed information:

    If volume management software is running on your system, the software disc automatically mounts as /cdrom. If volume management software is not available to mount the DVD-ROM, you must mount it manually, enter:

    # mount -F hsfs -o ro /dev/dsk/c0t6d0s2 /cdrom

  2. Start the SF Oracle RAC installer utility. Include the -rsh option if you have not set up SSH:

    # cd /cdrom/dvd1/storage_foundation_for_oracle_rac

    # ./installsfrac galaxy nebula

  3. The installer displays information about the cluster, the systems, and the service groups and prompts you about upgrading. At the prompt to upgrade, enter "y":

    ...

    Do you want to upgrade to version 5.0 on these systems using the current configuration? [y,n,q,?] (y)

  4. The installer verifies rsh accessibility and the existing licenses.

    ...

    Do you want to enter another license key for galaxy? [y,n,q](n)

  5. If necessary, answer "y" and enter the license keys at the prompt.
  6. When the installer prompts you to confirm whether to proceed with the upgrade. Answer "y" to proceed.
  7. As the installer proceeds, it:
    1. Takes the cvm group and other dependent group offline.
    2. Updates LLT and GAB configurations as needed for SF 5.0 Oracle RAC.
  8. When it is ready to stop the cluster to make updates to the configuration, it prompts for your OK. At the prompt, press Enter.

    ...

    Are you ready to begin the Veritas Storage Foundation for Oracle RAC upgrade at this time?

  9. The installer backs up the configuration files, freezes service groups, stops processes, and uninstalls the 4.0 SF Oracle RAC filesets.

    Backing up types.cf and main.cf .................Done

    Freezing group cvm ..............................Done

    ...

    Stopping SFRAC: 100%

    Shutdown completed successfully on all systems

    Uninstalling SFRAC: 100%

    Uninstall completed successfully on all systems

    Installing SFRAC: 100%

    Installation completed successfully on all systems

  10. The installer displays where logs for the upgrade can be found. It then indicates you must reboot each system.
  11. Depending on whether you want to use the DMP configuration or the raw device configuration, use one of the following commands:
  12. Restore the vcsmm.conf configuration file.

    # cp /kernel/drv/vcsmm.conf.save /kernel/drv/vcsmm.conf

  13. Reboot each system in the cluster.

    # /usr/sbin/shutdown -g0 -y -i6

Performing post-upgrade tasks for SF 5.0 Oracle RAC
  1. When the systems come back up after the rebooting, GAB ports, a, b, d, h, and o are configured. The VCS service groups are frozen.
  2. Invoke installsfrac once again, this time to enter configuration information necessary to upgrade to SF 5.0 Oracle RAC. Include the -rsh option if you have not set up SSH:

    # cd /opt/VRTS/install

    #./installsfrac -configure

  3. Enter the system names when prompted:

    Enter the system names separated by spaces on which to configure SFRAC: galaxy nebula

  4. The installer now verifies rsh accessibility between the systems and prompts you whether to use the same VCS/GAB/LLT configuration. Answer "y."

    Are you sure you want to reuse configuration of SFRAC 4.1 [y,n,q] (y)

  5. Next, installer will bring up the SF Oracle RAC "Main Menu". Select the option to "Configure SFRAC" (2):

    1) Check systems for SFRAC ** INSTRUCTIONS ONLY **

    2) Configure SFRAC

    3) Prepare to install Oracle

    4) Install or relink Oracle

  6. From the "Configure SFRAC menu, select "Configure VCS, CVM and CFS" (1):

    1) Configure VCS, CVM and CFS

    2) Configure I/O fencing ** INSTRUCTIONS ONLY **

    b) [Go to previous menu]

  7. The installer now stops all SF Oracle RAC processes.

    Stopping SFRAC processes. Please wait...

    SFRAC processes are stopped

    Press [Return] to continue:

  8. The installer next prompts you to change permissions to allow the current oracle user and group to access Veritas Storage Foundation for Oracle RAC tools. The questions are asked for each system in the cluster.

    ...

    The Veritas Storage Foundation for Oracle RAC utility for database administration requires permission changes to allow database administrators (DBAs) access to the tools. The default settings at installation time for the /opt/VRTSdbed directory allows only the root user access to the directory. You need to change permissions to allow a DBA or a group of DBAs access to the Veritas Storage Foundation for Oracle RAC tools

    Do you want to add single user access on galaxy [y,n,q,?] (Y)

    Answer "y"

    Enter login account name for DBA user: oracle

    Enter group name for DBA users: dba

  9. For each system, the installer prompts you to verify the fully qualified hostnames. For example:

    ...

    Is the fully qualified hostname of system "galaxy" =

    "galaxy.example.com"? [y,n,q,b,?] (y) y

    Querying fully qualified domain name of host "galaxy" .. ok

    Is the fully qualified hostname of system "nebula" =

    "nebula.example.com"? [y,n,q,b] (y) y

  10. When you are prompted about enabling Storage Foundation Management Sever, answer "n".

    ...

    Enable Storage Foundation Management Server Management? [y,n,q,b] (y) n

  11. The installer now configures VEA, gridnode, actionagent, and Storageagent for all nodes in the cluster.

    Verify the install systems Fully Qualified Hostnames.

    Configuring VEA in STANDALONE mode on "galaxy" ..... ok

    Configuring gridnode on "galaxy" ................... ok

    Registering gridnode on "galaxy" ................... ok

    Configuring actionagent on "galaxy" ................ ok

    Registering actionagent on "galaxy" ................ ok

    Stopping StorageAgent on "galaxy" .................. ok

    Registering StorageAgent on "galaxy" ............... ok

    ....

  12. When you are prompted to start SF Oracle RAC processes, accept the default "y" by pressing Enter.

    Do you want to start Veritas Storage Foundation for Oracle RAC processes now? [y,n,q,b] (y)

    Starting SFRAC:100%

    Installation completed successfully on all systems

    ...

  13. Ignore the message about executing shutdown that displays after the CFS agents are configured.
  14. At this point, all SF Oracle RAC components should be up and all service groups should be online. All 8 gab ports must be configured (by whom, the users or the installer?). Also, cssd (this is 10g only?) and Oracle resources would also be online (still linked to the old 4.0SF Oracle RAC libraries).
  15. Import any VxVM disk groups not under VCS control. Also mount any VxFS file systems not under VCS control.
Relinking Oracle10g R1 or R2
  1. Invoke installsfrac once again:

    # cd /opt/VRTS/install

    #./installsfrac -configure

  2. Enter the system names when prompted:

    Enter the system names separated by spaces on which to configure SFRAC: galaxy nebula

  3. Navigate to the "Install and Relink Oracle" menu.
    1. Select the appropriate Oracle 10g version (3):

      1) Oracle 9iR2

      2) Oracle 10gR1

      3) Oracle 10gR2

    2. Select "Relink Oracle" (3) from the menu:

      1) Install Oracle Clusterware (CRS)

      2) Install Oracle RDBMS server

      3) Relink Oracle

      b) [Go to previous menu]

    3. From the menu displayed, enter the required information. For example:

      Enter Oracle UNIX user name: [b] (oracle) oracle

      Enter Oracle UNIX group name: [b] (oinstall) oinstall

      Enter Oracle base directory: [b] /app/oracle

      Enter absolute path of CRS Home directory: [b]

      /app/crshome

      Enter absolute path of Database Home directory: [b]

      /app/oracle/orahome

      Enter Oracle Bits (64/32) [b] (64) 64

    4. Confirm you responses in the verification screen. The installer copies the SF 5.0 Oracle RAC libraries to /opt/ORCLcluster, where it expects libskgxn.

      Oracle environment information verification

      Oracle Unix User: oracle

      Oracle Unix Group: oinstall

      Oracle Clusterware (CRS) Home:

      /app/crshome

      Oracle Release: 10.2

      Oracle Bits: 64

      Oracle Base: /app/oracle

      Oracle Home: /app/oracle/orahome

      Is this information correct? [y,n,q] (y)

      galaxy
      Copying /opt/VRTSvcs/rac/lib/libskgxn2_64.so

      /opt/ORCLcluster/lib/libskgxn2.so ........... success

      nebula
      Copying /opt/VRTSvcs/rac/lib/libskgxn2_64.so

      /opt/ORCLcluster/lib/libskgxn2.so .............. success


      galaxy
      Copying /opt/ORCLcluster/lib/libskgxn2.so to

      /app/oracle/orahome/lib/ ....................... success
      Copying /opt/VRTSvcs/rac/lib/ to

      /app/oracle/orahome/lib/libskgxp9.so ........... success
      Move /app/oracle/orahome/lib/libodm9.so

      /app/oracle/orahome/lib/libodm9.so.old ......... success
      Linking /usr/lib/sparcv9/libodm.so

      /app/oracle/orahome/lib/libodm9.so ............. success
      Setting permissions oracle:oinstall

      /app/oracle/orahome/lib/libskgxp9.so ............ success

      nebula
      Copying /opt/ORCLcluster/lib/libskgxn2.so to

      /app/oracle/orahome/lib/ ....................... success
      Copying /opt/VRTSvcs/rac/lib/ to

      /app/oracle/orahome/lib/libskgxp9.so ........... success
      Move /app/oracle/orahome/lib/libodm9.so

      /app/oracle/orahome/lib/libodm9.so.old ......... success
      Linking /usr/lib/sparcv9/libodm.so

      /app/oracle/orahome/lib/libodm9.so ............. success
      Setting permissions oracle:oinstall

      /app/oracle/orahome/lib/libskgxp9.so ........... success

    5. Enter "q" at the next prompt to leave the installer now that CRS setup tasks are complete.
  4. If you are running 10gR1, copy the following SKGXN libraries to $CRS_HOME/lib and $CRS_HOME/lib32. (This step is not required for 10gR2).

    #cp /opt/VRTSvcs/rac/lib/libskgxn2.a $CRS_HOME/lib32

    #cp /opt/VRTSvcs/rac/lib64/libskgxn2.a $CRS_HOME/lib

  5. Bring the CSSD resource online. Enter:

    # hares -online cssd -sys galaxy

    # hares -online cssd -sys nebula

  6. Confirm that CRS in online. Enter:

    $CRS_HOME/bin/crs_stat -t

  7. Bring online the oracle resources configured under VCS. If they're directly controlled by CRS, you may run the CRS commands to start the instance.
Relinking Oracle9i
  1. Invoke installsfrac once again:

    # cd /opt/VRTS/install

    #./installsfrac -configure

  2. Enter the system names when prompted:

    Enter the system names separated by spaces on which to configure SFRAC: galaxy nebula

  3. Navigate to the "Install and Relink Oracle" menu.
  4. Select the Oracle 9i version (1):

    1) Oracle 9iR2

    2) Oracle 10gR1

    3) Oracle 10gR2

  5. Select "Relink Oracle (2) from the menu:

    1) Install Oracle RDBMS server

    2) Relink Oracle

    b) [Go to previous menu]

    Enter Oracle UNIX user name: [b] (oracle) oracle

    Enter Oracle UNIX group name: [b] (oinstall) oinstall

    Enter Oracle base directory: [b] /app/oracle

    Enter absolute path of Database Home directory: [b] /app/oracle/orahome

    Enter Oracle Bits (64/32) [b] (64) 64

    Oracle environment information verification

    Oracle Unix User: oracle

    Oracle Unix Group: oinstall

    Oracle Release: 9.2

    Oracle Bits: 64

    Oracle Base: /app/oracle

    Oracle Home: /app/oracle/orahome

    Is this information correct? [y,n,q](y)

    galaxy

    Copying /opt/VRTSvcs/rac/lib/libskgxn2_64.so

    /opt/ORCLculster/lib/libskgxn2.so ................ success

    nebula

    Copying /opt/VRTSvcs/rac/lib/libskgxn2_64.so

    /opt/ORCLculster/lib/libskgxn2.so ................ success

    galaxy

    Copying /opt/ORCLcluster/lib/libskgxn2.so to

    /app/oracle/orahome/lib/ ............................ success
    Copying /opt/VRTSvcs/rac/lib/ to

    /app/oracle/orahome/lib/libskgxp9.so ................ success
    Move /app/oracle/orahome/lib/libodm9.so

    /app/oracle/orahome/lib/libodm9.so.old .............. success
    Linking /usr/lib/sparcv9/libodm.so.

    /app/oracle/orahome/lib/libodm9.so .................. success
    Setting permissions oracle:oinstall

    /app/oracle/orahome/lib/libskgxp9.so ............. success

    nebula

    Copying /opt/ORCLcluster/lib/libskgxn2.so to

    /app/oracle/orahome/lib/ ............................ success
    Copying /opt/VRTSvcs/rac/lib/ to

    /app/oracle/orahome/lib/libskgxp9.so ................ success
    Move /app/oracle/orahome/lib/libodm9.so

    /app/oracle/orahome/lib/libodm9.so.old .............. success
    Linking /usr/lib/sparcv9/libodm.so.

    /app/oracle/orahome/lib/libodm9.so .................. success
    Setting permissions oracle:oinstall

    /app/oracle/orahome/lib/libskgxp9.so ............. success

  6. Bring online the Oracle resources configured under VCS.
  7. As Oracle user, enter the command:

    $ $ORACLE_HOME/bin/gsdctl start


 ^ Return to Top

< Previous  | TOC | Index | Next  >