OS: HPUX OS Version: 11iv2 (11.23) Etrack Incidents: 1843835 Fixes Applied for Products: VRTSgab - Veritas Group Membership and Atomic Broadcast By Symantec Patch Name: UNOF_40444 Patch Description: s700_800 11.23 VRTS 5.0MP1RP2 HF1 VRTSgab Kernel Patch Creation Date: 09/09/21 Post Date: 09/09/23 Hardware Platforms - OS Releases: s700: 11.23 s800: 11.23 Products: VRTSgab 5.0 MP1RP2HF1 Filesets: VRTSgab.VRTSGAB-KRN, fr=5.0.0.1, fa=HP-UX_B.11.23_IA, v=HP VRTSgab.VRTSGAB-KRN, fr=5.0.0.0, fa=HP-UX_B.11.23_IA, v=Symantec VRTSgab.GABRUNTIME, fr=5.0.0.1, fa=HP-UX_B.11.23_IA, v=HP VRTSgab.GABRUNTIME, fr=5.0.0.0, fa=HP-UX_B.11.23_IA, v=Symantec VRTSgab.VRTSGAB-KRN, fr=5.0.0.1, fa=HP-UX_B.11.23_PA, v=HP VRTSgab.VRTSGAB-KRN, fr=5.0.0.0, fa=HP-UX_B.11.23_PA, v=Symantec VRTSgab.GABRUNTIME, fr=5.0.0.1, fa=HP-UX_B.11.23_PA, v=HP VRTSgab.GABRUNTIME, fr=5.0.0.0, fa=HP-UX_B.11.23_PA, v=Symantec VRTSgab.VRTSGAB-ENG-MAN, fr=5.0.0.1, fa=HP-UX_B.11.23_IA/PA, v=HP VRTSgab.VRTSGAB-ENG-MAN, fr=5.0.0.0, fa=HP-UX_B.11.23_IA/PA, v=Symantec Automatic Reboot?: Yes (Dynamically Loadable Kernel Modules in the patch may not always require a reboot) Status: Beta Release Critical: Yes UNOF_40444: PANIC VERITAS Incident Number: 1843835 PHNE_38738: PANIC PHNE_35783: PANIC VERITAS Incident Number: 596273 Category Tags: defect_repair trial_patch critical panic dlkm Path Name: /hp-ux_patches/s700_800/11.X/UNOF_40444 Symptoms: UNOF_40444: VERITAS Incident Number: 1843835 System panics with the following panic string: "Port w halting system due to depleted memory" PHNE_38738: (SR: QXCR1000787691) VERITAS Incident Number: 1060657 In a Serviceguard environment, when the GAB master leaves the cluster and another node, having a lower node ID than the master node, joins the cluster, other surviving node(s) in the cluster may fail to complete cluster reconfiguration and panic with the following message: "panic: GAB: Port d halting system due to network failure" (SR: QXCR1000787694) VERITAS Incident Number: 1084656 Running the 'gabdebug' utility in a loop may trigger a data page fault. The panic stack trace includes the following kernel routines: " gab_print_uio gab_print_msg gab_ioc_debug gabioctl gabhpioctl spec_ioctl vno_ioctl ioctl syscall " (SR: QXCR1000787703) VERITAS Incident Number: 862507 The system panics with the panic string "assert failed" during cluster reconfiguration. The stack trace shows the following routines: " gab_sf_restseq gab_sf_recv_start gab_recv_start gab_receive gab_receive_port_que gab_receive_que gab_lrecv llt_lrsrv_port llt_deliver llt_threadfunc " PHNE_35783: ( SR:8606468172 CR:JAGag23508 ) VERITAS Incident Number: 596273 Node Panics with message- GAB: Port f halting system due to network failure. Defect Description: UNOF_40444: VERITAS Incident Number: 1843835 This panic happens only when system does not have sufficient memory in spinlock safe cache. Resolution: Code updated to allocate memory from gab internal memory, when spinlock safe cache does not have sufficient memory. PHNE_38738: (SR: QXCR1000787691) VERITAS Incident Number: 1060657 GAB is not equipped to address a scenario wherein the GAB master node with the lowest GAB node id leaves the cluster and another node with a lower node ID than the master node, joins the cluster. As a result, the cluster reconfiguration process fails to complete on the surviving nodes. Eventually, GAB halts these nodes. Resolution: GAB is modified to complete the cluster reconfiguration when the master node leaves the cluster and another node with a lower node id joins the cluster. (SR: QXCR1000787694) VERITAS Incident Number: 1084656 The debug utility accesses certain GAB internal variables that are incorrectly typecast, leading to an invalid pointer dereference and eventually a system panic. Resolution: The typecasting issues with GAB's internal variables are now resolved. (SR: QXCR1000787703) VERITAS Incident Number: 862507 The assertion failure uncovers a race condition in GAB. Due to the race condition, GAB incorrectly set a flag indicating that a sequence number request was sent during a previous cluster reconfiguration. This incorrect setting of the flag eventually led to a system panic with the panic string "assert failed" on a subsequent cluster reconfiguration. Resolution: GAB is now modified to reflect the correct sequence request status. PHNE_35783: ( SR:8606468172 CR:JAGag23508 ) VERITAS Incident Number: 596273 During a cluster reconfiguration, if the system with the lowest node id is shutdown before the ongoing reconfiguration is completed on all nodes, it is possible that this pending RECONFIG message is not re-broadcast. This can cause all the remaining systems in the cluster to eventually panic. There will be multiple instances of the message "GAB: Port not ready for reconfiguration, will retry" and the eventual panic string will be "GAB: Port halting system due to network failure." Resolution: Make GAB aware of reconfig in near past as a special case for node join. Enhancement: No SR: QXCR1000787691 QXCR1000787694 QXCR1000787703 8606468172 Patch Files: VRTSgab.VRTSGAB-KRN, fr=5.0.0.1, fa=HP-UX_B.11.23_IA, v=HP: VRTSgab.VRTSGAB-KRN, fr=5.0.0.0, fa=HP-UX_B.11.23_IA, v=Symantec: /usr/conf/mod/gab /usr/conf/mod/gab-dbg VRTSgab.GABRUNTIME, fr=5.0.0.1, fa=HP-UX_B.11.23_IA, v=HP: VRTSgab.GABRUNTIME, fr=5.0.0.0, fa=HP-UX_B.11.23_IA, v=Symantec: /sbin/gabdebug VRTSgab.VRTSGAB-KRN, fr=5.0.0.1, fa=HP-UX_B.11.23_PA, v=HP: VRTSgab.VRTSGAB-KRN, fr=5.0.0.0, fa=HP-UX_B.11.23_PA, v=Symantec: /usr/conf/mod/gab /usr/conf/mod/gab-dbg VRTSgab.GABRUNTIME, fr=5.0.0.1, fa=HP-UX_B.11.23_PA, v=HP: VRTSgab.GABRUNTIME, fr=5.0.0.0, fa=HP-UX_B.11.23_PA, v=Symantec: /sbin/gabdebug VRTSgab.VRTSGAB-ENG-MAN, fr=5.0.0.1, fa=HP-UX_B.11.23_IA/PA, v=HP: VRTSgab.VRTSGAB-ENG-MAN, fr=5.0.0.0, fa=HP-UX_B.11.23_IA/PA, v=Symantec: /opt/VRTS/man/man1m/gabconfig.1m cksum(1) Output: VRTSgab.VRTSGAB-KRN, fr=5.0.0.1, fa=HP-UX_B.11.23_IA, v=HP: 2970113080 1112792 /usr/conf/mod/gab 4271496194 2824904 /usr/conf/mod/gab-dbg VRTSgab.GABRUNTIME, fr=5.0.0.1, fa=HP-UX_B.11.23_IA, v=HP: 2839592535 63864 /sbin/gabdebug VRTSgab.VRTSGAB-KRN, fr=5.0.0.1, fa=HP-UX_B.11.23_PA, v=HP: 3399712115 783824 /usr/conf/mod/gab 2568532477 4967184 /usr/conf/mod/gab-dbg VRTSgab.GABRUNTIME, fr=5.0.0.1, fa=HP-UX_B.11.23_PA, v=HP: 3909013442 40960 /sbin/gabdebug VRTSgab.VRTSGAB-ENG-MAN, fr=5.0.0.1, fa=HP-UX_B.11.23_IA/PA, v=HP: 1018755946 5905 /opt/VRTS/man/man1m/gabconfig.1m Patch Conflicts: None Patch Dependencies: None Hardware Dependencies: None Other Dependencies: None Supersedes: PHNE_35783 PHNE_38738 Equivalent Patches: None Patch Package Size: 9660 KBytes Installation Instructions: Please review all instructions and the Hewlett-Packard SupportLine User Guide or your Hewlett-Packard support terms and conditions for precautions, scope of license, restrictions, and, limitation of liability and warranties, before installing this patch. ------------------------------------------------------------ NOTE: This patch is a un-official patch, before installing any offcial patch on top of this patch, please uninstall this patch from all your cluster nodes. ------------------------------------------------------------ To Install the UNOF_40444 patch to your system: 1. Back up your system before installing a patch. 2. Login as root. 3. Install/Upgrade all the cluster nodes to VCS 5.0MP1RP2(MP2). Refer to the VCS Release Notes for HP-UX 11iv2 for 5.0MP1RP2(MP2). 4. Uncompress the patch downloaded from Symantec site to a directory of your choice (ex. /[dir]) 5. Run swinstall command to install the Un-Official patch: swinstall -s /[dir]/patches -x autoreboot=true UNOF_40444 NOTE: Installation of this patch might require a reboot of the system, if gab driver is in loaded state. To unload gab driver, please follow instructions in "VCS 5.0 Installation & Configuration Guide" to stop all modules dependent on gab. 6. Configure the cluster. Refer to the VCS 5.0MP1RP2(MP2) Release Notes for more information. ------------------------------------------------------------ To Remove/Uninstall the UNOF_40444 patch from your system: 1. Back up your system before installing a patch. 2. Login as root. 3. Run the following command to remove the Un-Official Patch from the cluster node: swremove -x autoreboot=true UNOF_40444 NOTE: Uninstall of this patch might require a reboot of the system, if gab driver is in loaded state. To unload gab driver, please follow instructions in "VCS 5.0 Installation & Configuration Guide" to stop all modules dependent on gab. 4. Check with "swlist PHNE_38738" command to confirm VCS 5.0MP1RP2(MP2)/11iv2 patch is installed on the system. ------------------------------------------------------------ For future reference, the contents of the UNOF_40444.text file is available in the product readme: swlist -l product -a readme -d @ /[dir] Special Installation Instructions: None