OS: Solaris OS Version: Solaris Sparc 5.8, 5.9, 5.10 Fixes Applied for Products: VRTSdbac - Veritas SF Oracle RAC by Symantec Additional Instructions: Please read the instructions below before installing the patch. PATCH VRTSdbac 4.1MP2HF3 for Veritas SF Oracle RAC 4.1MP2 =============================================================== Patch Date: October, 2009 This README provides information on: * BEFORE GETTING STARTED * CRC AND BYTE COUNT * FIXES AND ENHANCEMENTS INCLUDED IN THE PATCH * PACKAGES AFFECTED BY THE PATCH * INSTALLING THE PATCH * UNINSTALLING THE PATCH BEFORE GETTING STARTED: ---------------------- This patch only applies to: VRTSdbac 4.1MP2 running on Solaris Sparc 5.8, 5.9 or 5.10 Ensure that you are running the supported configurations before installing this patch. FIXES AND ENHANCEMENTS INCLUDED IN THE PATCH: -------------------------------------------- Etrack Incidents: 1848205 SDR's of Fixed Symantec Incidents: -------------------------------- Symantec Incident : 1848205 Symptom: The system panics with the panic string: "kernel heap corruption detected" in lmxinitbuf. Defect Description: When the request is removed from the work queue/done queue, if present, work queue/done queue tail pointer are never updated. When the last buffer is deleted, the request is freed, but work queue/done queue tail pointer are not updated to point to previous request and would still be pointing to the freed request. Next time when the request is added to the queue, the corresponding request is added to the work queue (which has work queue tail pointer pointing to the freed request) using the work queue tail pointer. Since work queue tail pointer is still pointing to the freed request, the next pointer of the earlier freed request is updated to point to the new request, thus the freed buffer gets updated. Resolution: Fix the manipulation of the work queue tail pointer/ done queue tail pointer whenever the request is removed. PACKAGES AFFECTED BY THE PATCH: ------------------------------- This patch updates the following SF Oracle RAC package(s) VRTSdbac from 4.1MP2 or higher to 4.1MP2HF3 INSTALLING THE PATCH: -------------------- The following steps should be run on all nodes in the VCS cluster: Stopping services in the cluster node: -------------------------------------- The following steps should be run on each node in the cluster, one at a time: 1. Shutdown Oracle instances on all nodes of the cluster. a) If the database instances are configured under VCS control, offline the corresponding VCS database resource. As superuser, enter: # /opt/VRTSvcs/bin/hagrp -offline [oracle-group] -sys [node_name] Make sure the [oracle-resource] is offline. From any one node of the cluster enter: # /opt/VRTSvcs/bin/hares -state [oracle-resource] b) If the database instances are not configured under VCS control, run the following on any one node in the cluster (as Oracle user) enter: $ $ORACLE_HOME/bin/srvctl stop database -d [database_name] 2. For Oracle 9i, stop gsd. As Oracle user, enter: $ $ORACLE_HOME/bin/gsdctl stop 3. For Oracle 10g or Oracle 11g, stop Oracle clusterware. As superuser, enter: # hares -offline [cssd-resource] -sys [node-name] 4. On each node of the cluster, unconfigure and unload VCSMM. Un-configure VCSMM: # /etc/init.d/vcsmm stop Verify that port 'o' has been closed: # /sbin/gabconfig -a The display should not have port 'o' listed. Unload VCSMM: # modinfo | grep vcsmm Take note of VCSMM module id from the output. # modunload -i [vcsmm_module_id] 5. On each node of the cluster, unconfigure and unload LMX. Un-configure LMX: # /etc/init.d/lmx stop Unload LMX: # modinfo | grep lmx Take note of LMX module id from the output. # modunload -i [lmx_module_id] Installing the Patch: -------------------- 1. Un-compress the downloaded patch from Symantec. Change directory to the unzipped patch location. Install the VRTSdbac 4.1MP2HF3 patch using the following command: # patchadd [patch-id] Where [patch-id] is either of the following depending on system configuration, For SunOS Release 5.8, [patch-id] is 142623-01. For SunOS Release 5.9, [patch-id] is 142624-01. For SunOS Release 5.10, [patch-id] is 142625-01. 2. Verify that the new patch has been installed: # showrev -p | grep [patch-id] You will find the following output on display with the patch installed properly: Patch: [patch-id] Obsoletes: Requires: [4.1MP2-patch-id] Incompatibles: Packages: VRTSdbac Re-starting services in the cluster node: ----------------------------------------- 1. Relink Oracle with the SF Oracle RAC 4.1MP2HF3 libraries: Refer section "Copying the IPC and VCSMM Libraries" in "VERITAS Storage Foundation 4.1 for Oracle RAC" Release Notes for Solaris 4.1 MP2 2. On each node of the cluster, start VCSMM. As a superuser, enter: # /etc/init.d/vcsmm start Verify that port 'o' is us: # /sbin/gabconfig -a The display should have port 'o' listed. 3. On each node of the cluster, start LMX. As a superuser, enter: # /etc/init.d/lmx start 4. For Oracle 9i, start gsd on all nodes of the cluster. As Oracle user, enter $ $ORACLE_HOME/bin/gsdctl start 5. For Oracle 10g or Oracle 11g, start Oracle clusterware. As superuser, enter: # hares -online [cssd-resource] -sys [node-name] 6. Start Oracle instances on all nodes of the cluster. a) If the database instances are configured under VCS control, online the corresponding VCS database resource. As superuser, enter: # /opt/VRTSvcs/bin/hagrp -online [oracle-group] -sys [node_name] Make sure the [oracle-resource] is online. From any one node of the cluster enter: # /opt/VRTSvcs/bin/hares -state [oracle-resource] b) If the database instances are not configured under VCS control, run the following on any one node in the cluster (as Oracle user): $ $ORACLE_HOME/bin/srvctl start database -d [database_name] UNINSTALLING THE PATCH: ----------------------- Follow the steps below on each cluster node to remove, the patch from the cluster: Steps to remove the Patch from a cluster node: --------------------------------------------- 1. Follow the steps provided under "Stopping services in the cluster node" section above, to stop the services on the node. 2. Remove the patch by the following command: # patchrm [patch-id] Where [patch-id] is either of the following depending on system configuration, For SunOS Release 5.8, [patch-id] is 142623-01. For SunOS Release 5.9, [patch-id] is 142624-01. For SunOS Release 5.10, [patch-id] is 142625-01. 3. Verify that the patch has been removed from the system: # showrev -p | grep [patch-id] There shall be no result on the output. 4. Relink Oracle with the SF Oracle RAC 4.1MP2 libraries: Refer section "Copying the IPC and VCSMM Libraries" in "VERITAS Storage Foundation 4.1 for Oracle RAC" Release Notes for Solaris 4.1 MP2 5. Restart the node following the steps under "Re-starting services in the cluster node" section above.