Date: 2012-07-30 OS: Windows OS Version: 2003, 2008, 2008 R2 Packages: ============================================================================== Architecture/OS Windows Server 2003 Windows Server 2008 / 2008 R2 ============================================================================== x86 CP14_VCSW_51SP2_W2k3_x86.exe CP14_VCSW_51SP2_W2k8_x86.exe ------------------------------------------------------------------------------ x64 CP14_VCSW_51SP2_W2k3_x64.exe CP14_VCSW_51SP2_W2k8_x64.exe ------------------------------------------------------------------------------ ia64 CP14_VCSW_51SP2_W2k3_ia64.exe CP14_VCSW_51SP2_W2k8_ia64.exe ------------------------------------------------------------------------------ Etrack Incidents: 2164975, 2220352, 2237219, 2231216, 2385965, 2407816, 2251689, 2437434, 2401163, 2239785, 2482820, 2392336, 2497664, 2495109, 2522314, 2513842, 2597410, 2639957, 2650336, 2692139, 2695917, 2713126, 2688194, 2722108, 2722228, 2770008, 2817466 Fixes Applied for Products ==========================| Veritas Cluster Server (VCS) 5.1 SP2 for Windows Install instructions ====================| Download the appropriate cumulative patch (CP) executable file to a temporary location on your system. You can install the CP in a verbose mode or in a non-verbose mode. Instructions for both options are provided below. Each CP includes the individual hotfixes that contain enhancements and fixes related to reported issues. See "Errors/Problems Fixed" section for details. Before you begin ----------------: [1] In case of Windows Server 2003, this hotfix requires Microsoft Core XML Services (MSXML) 6.0 pre-installed in your setup. Download and install MSXML 6.0 before installing the hotfix. Refer to the following link for more information: http://www.microsoft.com/downloads/details.aspx?FamilyId=993c0bcf-3bcf-4009-be21-27e85e1857b1&displaylang=en Microsoft posted service pack and/or security updates for Core XML Services 6.0. Please contact Microsoft or refer to Microsoft website to download and install latest updates to Core XML Services 6.0. Refer to the following link for more information: http://www.microsoft.com/downloads/details.aspx?FamilyId=70C92E77-9E5A-41B1-A9D2-64443913C976&displaylang=en [2] Ensure that the logged-on user has the following privileges to install the CP on the systems: - Local administrator privileges - Debug privileges [3] One or more hotfixes that are included with this CP may require a reboot. Before proceeding with the installation ensure that the system can be rebooted. [4] Symantec recommends that you close the Cluster Manager (Java Console) and the Veritas Enterprise Administrator (VEA) Console before installing this CP. [5] Hotfix_5_1_20002_2220352 requires the following two additional steps after installation: - To support non-scoped file shares, the agents read a specific registry key. You must create the registry key after installing this hotfix. See the topic "Configuring the registry parameter to support non-scoped file shares" in the post-install steps section in this readme. - After the hotfix installation is complete, offline the FileShare service group on the node where it was online and then bring it online again. The enhancement will take effect once the service group is online again. [9] Hotfix_5_1_20038_2688194 installation may fail if CP installer fails to stop the MSMQ and its dependent services. In such a case, you should manually stop these services and then retry installing the hotfix. To install in the verbose mode ------------------------------: In the verbose mode, the cumulative patch (CP) installer prompts you for inputs and displays the installation progress status in the command window. Perform the following steps: [1] Double-click the CP executable file to extract the contents to a default location on the system. The installer displays a list of hotfixes that are included in the CP. - On 32-bit systems, the hotfixes executable files are extracted to: "%commonprogramfiles%\Veritas Shared\WxRTPrivates\" - On 64-bit systems, the hotfixes executable files are extracted to: "%commonprogramfiles(x86)%\Veritas Shared\WxRTPrivates\" The installer also lists the hotfixes that require a reboot of the system after the installation. If system reboot is not an option at this time, you can choose not to install these hotfixes. In such a case, exit the installation and then launch the CP installer again from the command line using the /exclude option. See "To install in a non-verbose (silent) mode" section for the syntax. [2] When the installer prompts whether you want to continue with the installation; type Y to begin the hotfix installation. The installer performs the following tasks: - Extracts all the individual hotfix executable files On 32-bit systems the files are extracted at %commonprogramfiles%\Veritas Shared\WxRTPrivates\ On 64-bit systems the files are extracted at %commonprogramfiles(x86)%\Veritas Shared\WxRTPrivates\ - Runs the pre-install tasks - Installs all the hotfixes sequentially - Runs the post-install tasks The installation progress status is displayed in the command window. [3] After all the hotfixes are installed, the installer prompts you to restart the system. Type Y to restart the system immediately, or type N to restart the system later. You must restart the system for the changes to take effect. Note that the installer prompts for a system restart only if hotfixes that require a reboot are included in the CP and are installed. To install in the non-verbose (silent) mode -------------------------------------------: In the non-verbose (silent) mode, the cumulative patch (CP) installer does not prompt you for inputs and directly proceeds with the installation tasks. The installer displays the installation progress status in the command window. Use the VxHFBatchInstaller.exe utility to install a CP from the command line. The syntax options for this utility are as follows: vxhfbatchinstaller.exe /CP: [/Exclude:,...] [/PreInstallScript:] [/silent [/forcerestart]] where, - CPName is the cumulative patch executable file name without the platform, architecture, and .exe extension. For example, if CP executable name is CP14_VCSW_51SP2_W2K8_x64.exe, specify it as CP14_VCSW_51SP2. - HF1.exe, HF2.exe,... represent the executable file names of the hotfixes that you wish to exclude from the installation. Note that the file names are separated by commas, with no space after a comma. The CP installer skips the mentioned hotfixes during the installation. - PreInstallScript.pl is the Perl script that includes the pre-installation steps. These steps forcefully kill the required services and processes in case a graceful stop request does not succeed. Symantec recommends that you use this option and script only in case the CP installer fails repeatedly while performing the pre-installation tasks. - /silent indicates the installation is run in a non-verbose mode; the installer does not prompt for any inputs during the installation. - /forcerestart indicates that the system is automatically restarted, if required, after the installation is complete. Perform the following steps: [1] From the command prompt, navigate to the directory where the CP executable file is located and then run the file to extract the contents to a default location on the system. The installer displays a list of hotfixes that are included in the CP. - On 32-bit systems, the hotfixes executable files are extracted to: "%commonprogramfiles%\Veritas Shared\WxRTPrivates\" - On 64-bit systems, the hotfixes executable files are extracted to: "%commonprogramfiles(x86)%\Veritas Shared\WxRTPrivates\" The installer also lists the hotfixes that require a reboot of the system after the installation. If system reboot is not an option at this time, you can choose not to install these hotfixes. In such a case, launch the CP installer from the command line using the /exclude option. [2] When the installer prompts whether you want to continue with the installation; type N to exit the installer. [3] In the same command window, run the following command to begin the CP installation in the non-verbose mode: vxhfbatchinstaller.exe /CP: /silent For example, to install a VCSW 5.1 SP2 x86 CP for Windows Server 2003, the command is: vxhfbatchinstaller.exe /CP:CP14_VCSW_51SP2 /silent The installer performs the following tasks: - Extracts all the individual hotfix executable files On 32-bit systems the files are extracted at %commonprogramfiles%\Veritas Shared\WxRTPrivates\ On 64-bit systems the files are extracted at %commonprogramfiles(x86)%\Veritas Shared\WxRTPrivates\ - Runs the pre-install tasks - Installs all the hotfixes sequentially - Runs the post-install tasks The installation progress status is displayed in the command window. [4] After all the hotfixes are installed, the installer displays a message for restarting the system. You must restart the system for the changes to take effect. Note that the installer prompts for a system restart only if hotfixes that require a reboot are included in the CP and are installed. The installer automatically restarts the system if you had specified the /forcerestart option in step 3 earlier. VxHFBatchInstaller usage examples: ---------------------------------- [+] Install CP in silent mode, exclude hotfix Hotfix_5_1_20005_2237219_w2k3_x86.exe: vxhfbatchinstaller.exe /CP:CP14_VCSW_51SP2 /Exclude:Hotfix_5_1_20005_2237219_w2k3_x86.exe /silent [+] Install CP in silent mode, restart automatically: vxhfbatchinstaller.exe /CP:CP14_VCSW_51SP2 /silent /forcerestart What's new in this CP =====================| The following hotfixes have been added in this CP: - HotFix_5_1_20043_2817466 For more information about these hotfixes, see the "Errors/Problems Fixed" section in this readme. Errors/Problems Fixed =====================| The fixes and enhancements that are included in this cumulative patch (CP) are as follows: [1] Hotfix name: Hotfix_5_1_20001_2164975 Symptom: This hotfix includes enhancements and fixes to the VCS DiskRes and Mount agents. Description: - VCS DiskRes and Mount agents are enhanced to support storage managed using Microsoft Logical Disk Management (LDM) on Windows Server 2008. You can now configure the VCS Mount and DiskRes agents on Windows Server 2008 systems. - The following issues may occur where a service group containing the VCS Mount agent resource is configured in the cluster: Issue 1 -------- Intermittent Delayed Write Failed errors are observed on nodes where the service group is not online. After switching the service group from an active node to another node, these errors are again seen on the active node where the service group is now offline. The VCS Mount agent log may display the following: VCS ERROR V-16-10051-8029 Mount::offline:Unable to lock the volume. Disk No. = , PartitionNo = . Error : 5 The Windows Event Viewer may contain the following events: Event ID 57: Source: ftdisk Warning: The system failed to flush data to the transaction log. Corruption may occur. Event ID 50: Source: ntfs Warning: {Delayed Write Failed} Windows was unable to save all the data for the file . The data has been lost. This error may be caused by a failure of your computer hardware or network connection. Please try to save this file elsewhere. There may also be an application pop-up message displaying the following message: Event ID 26: Source: Application Pop-up Information: Windows - Delayed Write Failed: Windows was unable to save all the data for the file \\ServerName\VolumeMountPointName\FileName. This data has been lost. This error may be cause by a failure on your computer hardware or network connection. Please try to save this file elsewhere. Issue 2 -------- Mount agent crashes while switching or taking the service group offline. There may also be a pop-up message that displays the following error related to VCSAgDrive.exe: The instruction at "0x78147347" referenced memory at "0x00000000". The memory could not be "written". The Windows Event Viewer may contain the following events: Event ID 1000: Source: Application Error Error: Faulting application VCSAgDriver.exe, version 5.1.10000.320, faulting module msvcr80.dll, version 8.0.50727.762, fault address 0x00017347. Event ID 8029: Source: AgentFramework Error: Mount::offline:Unable to lock the volume. Disk No. = , PartitionNo = . Error : 5 Resolution: - The VCS DiskRes and Mount agents now support configuring LDM managed storage on windows Server 2008. - The Mount agent issues have been fixed in the updated agent. After installing this hotfix, the Mount agent ignores the MountResName attribute. The MountResName attribute was applicable for folder mounts. If you have configured folder mounts in a service group, then after installing this hotfix, the Mount resources may go in to an unknown state. To resolve this, specify the complete mount path value in the MountPath attribute itself. For example, if MountPath value is \Dir and MountResName value is the name of the resource monitoring X:, then change the MountPath attribute value to X:\Dir. The MountPath attribute value must be the complete mount path. You can change the attribute value either before or after installing the hotfix. Binary / Version: DiskRes.dll / 5.1.20001.435 DiskResDiscover.DLL / 5.1.20001.435 DiskRes.inf / NA Mount.dll / 5.1.20001.435 MountDiscover.dll / 5.1.20001.435 havol.exe / 5.1.20001.435 Havol_Lang_en.dll / 5.1.20001.435 -------------------------------------------------------+ [2] Hotfix name: Hotfix_5_1_20002_2220352 Symptom: This hotfix includes the VCS FileShare and Lanman agents that are enhanced to address the limitation of file share scoping on Windows Server 2008 and Windows Server 2008 R2 systems. These agents now support creation of non-scoped file shares configured with VCS. Description: File shares on Windows Server 2003 are accessible using the virtual name (Lanman) or the IP address (administrative IP or virtual IP). On Windows Server 2008 systems, due to file share scoping, the file shares configured with VCS are accessible only using the virtual server name (Lanman). These file shares are not accessible using the IP address. Resolution: The VCS agents (FileShare, Lanman) have been enhanced to support creation of non-scoped file shares on Windows. Along with the virtual name, the file shares are now accessible using the IP address too. To support non-scoped file shares, the agents read a specific registry key. You must create the registry key after installing this hotfix. See the topic "Configuring the registry parameter to support non-scoped file shares" in this readme. Note: - After the hotfix installation is complete, offline the FileShare service group on the node where it was online and then bring it online again. The enhancement will take effect once the service group is online again. - This hotfix includes a design enhancement to the same non-scoped shares support hotfix (Hotfix_5_1_20002_2220352) released in the earlier CP. Binary / Version: Lanman.dll / 5.1.20002.441 Fileshare.dll / 5.1.20002.441 -------------------------------------------------------+ [3] Hotfix name: Hotfix_5_1_20005_2237219 Symptom: This hotfix addresses the issue where the VCS IP resource comes online before the configured IP address becomes available. Description: Before making an IP address usable, Windows server performs a Duplicate Address Detection (DAD) test to check whether the IP address is unique on the network. During this check, the state of the IP address (ipconfig) on the system remains as "Tentative". The VCS IP agent does not wait for the DAD check to complete and reports the status of the IP resource as online, even though the configured IP address is not yet usable. The DAD check typically takes 2-3 seconds. This delay may result in a problem in service groups where an application resource depends directly on the IP resource. The application component fails to start as a result of an unusable IP address. Resolution The VCS IP agent is enhanced to address this issue. The agent does not report the resource status as online until the DAD check is complete and the configured IP address state on the system changes from "Tentative" to "Preferred". Binary / Version: IP.dll / 5.1.20005.443 -------------------------------------------------------+ [4] Hotfix name: Hotfix_5_1_20006_2231216 Symptom: This hotfix addresses an issue where the VCS PrintShare agent erroneously reports the resource as online on the passive nodes, resulting in a concurrency violation. Description: This issue occurs with PrintShare service groups created in a Global Cluster Configuration (GCO) cluster environment. PrintShare resources come online on the nodes at the primary and secondary site at the same time. Taking the resources offline at the secondary site results in a service group fault at the primary site. Bringing the faulted resources online on the primary site also brings the corresponding resources online at the secondary site leading to a concurrency violation. Resolution: This issue has been fixed in the VCS PrintShare agent. Binary / Version: PrintShare.dll / 5.1.20006.445 -------------------------------------------------------+ [5] Hotfix name: Hotfix_5_1_20017_2385965 Symptom: This hotfix addresses an issue wherein the VCS IIS resource fails to probe if IIS site binding is configured for HTTPS only. Description In an IIS service group, the VCS IIS resource fails to probe and may go in to an unknown state if the website binding is configured for https. The IIS agent is unable to detect the configured port if the site binding protocol is set to https. The IIS agent log displays the following message: VCS ERROR V-16-10051-13612 IIS::monitor:Site DEFAULT WEB SITE exists but the site port number doesn’t match with that configured under VCS. Resolution: This issue is fixed in the updated IIS agent. The agent is now able to detect the port if the site binding type is set to https. Binary / Version: IIS.dll / 5.1.20017.482 -------------------------------------------------------+ [6] Hotfix name: Hotfix_5_1_20019_2407816 Symptom: This hotfix addresses the following issues: Issue 1 The VCS IIS resource goes in to an unknown state if the IIS site binding also includes the host name along with the IP and the Port. Issue 2 The VCS IIS resource fails to probe if IIS site binding is configured for HTTPS only. Description: Issue 1 This issue occurs only if IIS 7 WMI Provider is used. The IIS resource goes in to an unknown state if the IIS site binding includes the host name along with the IP and the Port. The IIS agent log contains the following error: VCS ERROR V-16-10051-13620 IIS:IISResource:monitor: exists but the site IP address or port number doesn't match with that configured under VCS Issue 2 In an IIS service group, the VCS IIS resource fails to probe and may go in to an unknown state if the website binding is configured for https. The IIS agent is unable to detect the configured port if the site binding protocol is set to https. The IIS agent log displays the following message: VCS ERROR V-16-10051-13612 IIS::monitor:Site DEFAULT WEB SITE exists but the site port number doesn’t match with that configured under VCS. Resolution: Issue 1 VCS does not need the Host Name to make IIS highly available. Therefore, the IIS agent now ignores the Host Name configured for the site. Issue 2 This issue is fixed in the updated IIS agent. The agent is now able to detect the port if the site binding type is set to https. Binary / Version: IIS.dll / 5.1.20019.488 -------------------------------------------------------+ [7] Hotfix name: Hotfix_5_1_20023_2437434 Symptom: This hotfix addresses an issue wherein BIND DNS servers are unable to be updated in VCS. Description: In a master-slave configuration of BIND DNS server, if you configure the Lanman agent to update the slave BIND DNS server, the Lanman agent fails to do so because the BIND DNS servers do not allow direct updates on the slave BIND DNS servers. In this case, neither an error message is displayed nor the Lanman resource faults even if DNSCriticalForOnline is set to TRUE. Resolution: The Lanman agent's behavior is enhanced to successfully update the slave BIND DNS server. The agent will now query the slave BIND DNS server to discover the corresponding master BIND DNS server and update the master BIND DNS server. Binary / Version: Lanman.dll / 5.1.20023.494 -------------------------------------------------------+ [8] Hotfix name: Hotfix_5_1_20025_2497664 Symptom: This hotfix addresses the issue where Exchange 2010 database resources fail to come online in first attempt because Exchange Active Manager takes time to update. Description: The Microsoft Exchange 2010 database resource failover does not succeed on the first attempt. During the online operation, the agent makes some changes in Active Directory. However Exchange Active Manager takes time to get that information from Active Directory. Due to this, the database mount operation fails. The agent log displays an error message similar to the following: VCS DBG_21 V-16-50-0 Exch2010DB:Common-Users-Database-1-Exch2010DB:online:Error occurred while executing the command. Error message is: Couldn't mount the database that you specified. Specified database: c5e9fdbe-9a2c-4a35-855a-209801f9f412; Error code: An Active Manager operation failed. Error: The database action failed. Error: An error occurred while preparing to mount database 'Common Users Database 1' on server . Please check that the database still exists and that it has a copy on server before retrying the operation. [Database: Common Users Database 1, Server: i.e. local]. LibExchBaseCommand.cpp:CExchBaseCommand::ExecuteEx[105] Resolution: The VCS Exchange 2010 agent is modified to handle the time required for Exchange Active Manager to update. Before mounting a database, the agent executes a cmdlet to get the current owning server for the database. If the owning server of the database is the same as the current node then the agent fires a command to mount the database. If the owning server is not the current node, then the agent waits for 2 seconds and then queries for the owning server again. By default, the agent makes 6 attempts to query the owning server. If Exchange Active Manager requires more time to update, you can modify the number of attempts the agent should make to query the owning server by manually creating the following registry, HKEY_LOCAL_MACHINE\SOFTWARE\Veritas\VCS\EnterpriseAgents\Exch2010DB\ActiveManagerWaitLimit, on the cluster node and setting its value to a reasonable number greater than 6. Perform the following steps to create the registry that is used by the agent to determine the number of attempts it should make to query the owning server: Caution: Incorrectly editing the registry may severely damage your system. Before making changes to the registry, make a backup copy. To create the registry key: 1. Ensure that you have installed this hotfix (Hotfix_5_1_20025_2497664) on all the cluster nodes. 2. To open the Registry Editor, click Start > Run, type regedit, and then click OK. 3. In the registry tree (on the left), navigate to HKEY_LOCAL_MACHINE\SOFTWARE\Veritas\VCS\EnterpriseAgents\. 4. Click Edit > New > Key and create a key by the name Exch2010DB. 6. Select the key that you created in step 4 and add a DWORD type of value. Value name should be ActiveManagerWaitLimit and Value data should be a reasonable number greater than 6. 7. Save and exit the Registry Editor. Note: This registry setting is applicable locally. Create the registry on all Exchange nodes in the service group. The agent uses the node-specific registry value to determine the number of attempts it needs to make to query the owning server. Binary / Version: Exch2010DB.dll / 5.1.20025.496 ExchUtilDL.dll / 5.1.20025.496 -------------------------------------------------------+ [9] Hotfix name: Hotfix_5_1_20027_2522314 Symptom: This hotfix addresses an issue related to the VCS LLT service causing system hangs. Description: LLT spinlock causes a deadlock on multi-CPU servers because a spinlock is held while sending out packets. This deadlock causes the system to hang. Resolution: This issue has been fixed. The change ensures that spinlock is released before transmitting the packet. Binary / Version: llt.sys / 5.1.20027.609 -------------------------------------------------------+ [10] Hotfix name: Hotfix_5_1_20030_2597410 Symptom: This hotfix addresses the following issues: Issue 1 The VCS FileShare agent fails to probe FileShare resources on passive nodes. Issue 2 FileShare resource fails to come online and no specific error is logged. Note: - Fix for issue #1 was earlier released as Hotfix_5_1_20028_2513842. It is now replaced by this hotfix. - For Windows 2008 and Windows 2008 R2, this hotfix has been replaced by Hotfix_5_1_20041_2722108. Description: Issue 1 The VCS FileShare agent is unable to probe FileShare resources on passive nodes if the file share is configured for a root of a volume that is mounted as a folder mount on a folder on a local system drive. The FileShare resource is online on the active node and the share is accessible. However, the status of the resource on the passive nodes displays as unknown instead of offline. The following error is displayed in the FileShare agent log: VCS ERROR V-16-10051-10507 FileShare::monitor:Failed to get properties for folder . The behavior is reversed when the FileShare service group is switched over to a passive node. Issue 2 If Hotfix_5_1_20028_2513842 is installed, and, NetBIOS is disabled, FileShare resource fails to come online. Resolution: Issue 1 The issue is addressed in the updated FileShare agent. Issue 2 The issue has been addressed in this hotfix. Binary / Version: FileShare.dll / 5.1.20030.497 -------------------------------------------------------+ [11] Hotfix name: Hotfix_5_1_20032_2639957 Symptom: This hotfix addresses an issue related to the VCS Mount agent where Copy on Write (COW) snapshot history stored on volumes configured using the Mount agent is lost when the volume resource is taken offline and brought online again. Description: This issue occurs when the Shadow Storage area (differences area) for COW snapshots is configured on volumes that are controlled using the VCS Mount agent. After creating a COW snapshot, if the volume that stores the COW snapshots is taken offline and then brought online again, all the snapshot history is lost. This occurs because the VCS Mount agent, as part of its volume offline function, is sometimes not able to flush all the open handles on the volume. This results in data loss. Resolution: This issue has been fixed in the VCS Mount agent. The Mount agent's offline function is modified to include an additional file system control object (FSCTL_DISMOUNT_VOLUME) that ensures that the agent is able to flush all the open handles to the volume. Binary / Version: Mount.dll / 5.1.20032.497 -------------------------------------------------------+ [12]Hotfix name: Hotfix_5_1_20033_2650336 Symptom: This hotfix addresses an issue related to the VCS PrintSpool agent where information about printers that were newly added in the virtual server context is lost in case of a system crash or unexpected failures that require a reboot. Description: The PrintSpool agent stores printer information to the configuration during its offline function. Therefore if printers are added in the virtual server context but the PrintSpool resource or the PrintShare service group is not gracefully taken offline or failed over, the new printer information does not get stored to the disk. In such a case, if the node where the service group is online hangs, shuts down due to unexpected failures, or reboots abruptly, all the new printer information is lost. The PrintShare service group may also fail to come online again on the node. Resolution: This issue has been fixed in the PrintSpool agent. A configurable parameter now enables the agent to save the printer information periodically. A new attribute, 'SaveFrequency', is added to the PrintSpool agent. SaveFrequency specifies the number of monitor cycles after which the PrintSpool agent explicitly saves the newly added printer information to the cluster configuration. The value 0 indicates that the agent does not explicitly save the information to disk. It will continue to save the printer information during its offline function. The default value is 5. Binary / Version: PrintSpool.dll / 5.1.20033.497 PrintSpool.xml / NA -------------------------------------------------------+ [13] Hotfix name: Hotfix_5_1_20036_2695917 Symptom: This hotfix addresses the following issues identified with the VCS High Availability Engine (HAD) module: Issue 1: The VCS High Availability Engine (HAD) fails to acknowledge resource state changes. As a result, service group resources do not go offline or fail over and remain in the waiting state forever. (2239785, 2392336) Issue 2: In a online global soft group dependency, if more than one parent service group is ONLINE in a cluster, then you cannot switch a child service group from one node to another. (2239785, 2392336) Issue 3: The VCS High Availability Engine (HAD) rejects client connection requests after renewing its authentication credentials as per the CredRenewFrequency cluster attribute. (2401163) Issue 4: The HA commands spike CPU usage to almost 100% for about 10 seconds or more. Also there is a delay in getting the results from the HA commands. Issue 5: The VCS engine log is flooded with information messages in case of network congestion. Note: - Fixes for issue #1, #2, #3, and #4 were released earlier as Hotfix_5_1_20024_2401163. It is now replaced by this hotfix. Description: Issue 1: The VCS engine (HAD) authenticates resource state changes and sends an acknowledgement in the form of an internal message. In some cases, when a service group or a resource is taken offline or switched or failed over to another cluster node, it may happen that HAD fails to send the acknowledgement message for the resource state change. The affected resource continues to wait for the acknowledgement from HAD and does not proceed with the state change. As a result, the offline, switch, or failover operation does not succeed. Issue 2: When you run the 'hagrp -switch' command, the VCS engine checks the state of a parent service group before switching a child service group. If more than one parent service group is ONLINE, the VCS engine rejects the command. The following error message is displayed: hagrp -switch -to VCS WARNING V-16-1-10207 Group dependencies are violated for group on system Issue 3: The CredRenewFrequency is a cluster attribute that determines the number of days after which VCS engine should renew credentials. By default this is set to 0. When the value is set to a non-zero value, the credential is renewed periodically. After this credential is renewed, all HA clients (command line, Java GUI, etc.) fail to connect to VCS engine securely. This situation is corrected only after the VCS engine is restarted. Issue 4: The HA commands spike CPU usage to almost 100% for about 10 seconds or more. Also there is a delay in getting the results from the HA commands. Issue 5: MOMUtil.exe is a VCSAPI client. It uses VCS APIs to get information from VCS and present that to SCOM Server. MOMUtil.exe is invoked after every 5 minutes as part of VCS monitoring. This connects to the VCS Cluster running on the system and tries to fetch the information to display the current status of service groups. In case of a network congestion, the engine is unable to send the data to the client immediately. In this scenario, the following information message is logged after every 5 minutes: Could not send data to peer MOMUtil.exe at this point; received error 10035 (Unknown error), will resend again The engine successfully sends data to the client (MOMUtil.exe) after some time. By this time, the engine log is flooded with the information messages. Resolution: The VCS engine has been modified as follows: Issue 1: The communication logic in the VCS High Availability Engine (HAD) module has been enhanced to address this issue. Issue 2: The VCS engine now proceeds with the switch operation for a child service group even if the parent service groups are ONLINE. Issue 3: The fix ensures that the HA clients can seamlessly connect to VCS engine after the credential renewals. Issue 4: This fix addresses the issue of high CPU usage by VCS High Availability Engine (HAD). Issue 5: The engine has been modified to print the information messages as debug messages. So the messages will appear in the engine log only if DBG_IPM debug level is set. If you need to log the information messages in the engine log, run the following command to set the debug level DBG_IPM: halog -addtags DBG_IPM Binary / Version: had.exe / 5.1.20036.498 hacf.exe / 5.1.20036.498 -------------------------------------------------------+ [14] Hotfix name: Hotfix_5_1_20034_2692139 Symptom: This hotfix addresses an issue related to the VCS NIC agent wherein the NIC resource removes the static IP address assigned to the local network adapter and causes the system to become unreachable on the network. Description: UseConnectionStatus is an optional attribute of the NIC agent that defines whether or not the NIC maintains its connection status. If UseConnectionStatus is disabled (set to False), then a value must be specified for PingHostList, another optional attribute of the NIC agent. However, in case UseConnectionStatus is disabled and PingHostList attribute value is either invalid or left blank, then the NIC resource removes the static IP address assigned to the network adapter. As a result the system becomes unreachable on the network. The VCS high availability engine log contains the following message: NIC is faulted (not initiated by VCS) Resolution: The VCS NIC agent has been updated to address this issue. The agent now does not remove the static IP (admin IP) address assigned to the network adapter, irrespective of the PingHostList attribute value. If the IP address specified in the PingHostList attribute is not reachable, the agent now faults the resource and the following message is displayed in the NIC agent log: VCS ERROR V-16-10051-3510 NIC::monitor:UDP check failed Binary / Version: nic.dll / 5.1.20034.497 -------------------------------------------------------+ [15] Hotfix name: Hotfix_5_1_20037_2713126 Symptom: This hotfix addresses the issue where the cluster fails to come online if the PATH variable is longer than 1976 characters. Description: The cluster is brought online by the VCS High Availability Daemon (HAD). HAD depends on the Veritas VCSComm Startup service, which fails to start when the PATH variable is longer than 1976 characters. As a result, HAD does not start and thus the cluster fails to come online. The following errors are reported in the Windows Event logs: INFORMATION 7036(0x40001b7c) Service Control Manager The Veritas VCSComm Startup service entered the stopped state. INFORMATION 7035(0x40001b7b) Service Control Manager The Veritas VCSComm Startup service was successfully sent a start control. Resolution: This issue has been fixed in this hotfix. NOTE: A reboot may be required if the issue is not resolved after the successful installation of this hotfix. Binary / Version: vcscomm.exe / 5.1.20037.629 -------------------------------------------------------+ [16] Hotfix name: Hotfix_5_1_20038_2688194 Symptom: This hotfix addresses the issue where clustered Microsoft Message Queuing (MSMQ) communication does not work as expected. Description: A clustered instance of MSMQ on a node is not able to acknowledge or send messages to another clustered instance of MSMQ on another node. Resolution: This issue has been fixed in this hotfix. NOTE: After installing or uninstalling this hotfix, perform the following steps for the changes to take effect: 1. Take the MSMQ service groups offline. 2. Stop the default MSMQ service. 3. Bring the MSMQ service groups online. Binary / Version: msmq.dll / 5.1.20038.504 schelp.dll / 5.1.20038.504 -------------------------------------------------------+ [17] Hotfix name: Hotfix_5_1_20041_2722108 Symptom: This hotfix addresses an issue related to the VCS FileShare agent wherein non-scoped file shares are not accessible using virtual server name or IP address if NetBIOS and WINS, or DNS updates are disabled. Description: VCS FileShare and Lanman agents support non-scoped file shares on Windows Server 2008 and 2008 R2 systems if hotfix Hotfix_5_1_20002_2220352 is installed and the DisableServerNameScoping registry key value is set to 1. The VCS FileShare agent depends on NetBIOS and WINS, or DNS to resolve the virtual name. If NetBIOS and WINS are disabled or the DNS is not updated, the agent is unable to resolve the virtual name. This may typically occur when the file share service groups are configured to use localized IP addresses. When the service group is switched or failed over, the virtual name to IP address mapping changes. In such a case if WINS database and the DNS are not updated, the agent is unable to resolve the virtual name. As a result the FileShare resources fault and the shares become inaccessible. The following message is seen in the agent log: VCS INFO V-16-10051-10530 FileShare::online:Failed to access the network path (\\virtualname) Resolution: The FileShare agent is enhanced to address this issue. A new registry key value determines how the FileShare agent behaves if the virtual name is not resolvable. The following registry key is created automatically as part of this hotfix installation: HKLM\SOFTWARE\VERITAS\VCS\BundledAgents\Lanman\__Global__\ DisableStrictVirtualNameCheck If DisableServerNameScoping (file share scoping registry key) and DisableStrictVirtualNameCheck registry key (new registry key introduced with this hotfix) values are set to 1, the agent makes the file shares accessible irrespective of whether or not the virtual name is resolvable. In case the virtual name is not resolvable, the file shares are accessible using the virtual IP. The default value of DisableStrictVirtualNameCheck registry is set to 0. Note that this agent behavior is applicable only for non-scoped file shares. See "Setting the DisableStrictVirtualNameCheck registry key" topic in this readme for details on how to modify this registry key value. Notes: - The registry key DisableStrictVirtualNameCheck will take effect only if DisableServerNameScoping key value is set to 1. - This FileShare agent behavior is applicable only for non-scoped file shares. - This hotfix creates the DisableStrictVirtualNameCheck registry key at the global level. The agent behavior applies to all the file share service groups in the cluster. If there are multiple file share service groups that are to be used in the non-scoped mode, and you want to selectively apply this agent behavior for specific file shares, then you have to create this registry key manually for each virtual server and then set its value. You must create the DisableStrictVirtualNameCheck key parallel to the file share scoping key: HKLM\SOFTWARE\VERITAS\VCS\BundledAgents\Lanman\\DisableServerNameScoping. Here should be the virtual computer name assigned to the file share server. This is the VirtualName attribute of the Lanman resource in the file share service group. - In case these registry parameters are configured at a global level and also configured for individual file share virtual servers, the registry settings for individual virtual servers take precedence. - You must create this key only for Lanman resources that are part of VCS file share service groups. Configuring this key for Lanman resources that are part of other VCS service groups may result in an unexpected behavior. Binary / Version: Fileshare.dll / 5.1.20041.506 -------------------------------------------------------+ [18] Hotfix name: Hotfix_5_1_20039_2722228 Symptom: This hotfix addresses the following issues: Issue 1 This hotfix addresses an authentication issue due to which the VCS Cluster Configuration Wizard (VCW) fails to connect to systems. Issue 2 This hotfix addresses an issue where after configuring a cluster to use single sign-on authentication (secure cluster), the service group configuration wizards and the Cluster Manager (Java Console) are unable to connect to cluster nodes remotely. Note: Fix for issue #1 was earlier released as Hotfix_5_1_20026_2495109. This fix is now a part of this hotfix. Description: Issue 1 While configuring a cluster VCW fails to discover the systems due to an access denied error. The following error is logged: 00005708-00000112: WMI Connection failed. Node='nodename', Error=80070005. 00005708-00000051: CNode::Init failed. Node='nodename', Error=80070005. 00005708-00000072: OpenNamespace for namespace returned 0x80070005 Issue 2 The following issues occur after the VCS Cluster Configuration Wizard (VCW) is used to configure a cluster to use single sign-on authentication (secure cluster): - The VCS service group configuration wizards fail to connect to the remote cluster nodes with the following error: CmdServer service not running on system: <nodename> This error is displayed even though the VCS Command Server service is running on all the nodes. - The Cluster Manager (Java Console) is able to connect to a cluster locally, but fails to connect to the cluster remotely. This happens because VCW fails to explicitly configure the credentials used by HAD and CMDSERVER while configuring the cluster. VCW also fails to clear the older single sign-on authentication credentials if the cluster is being reconfigured. Resolution: Issue 1 VCW uses WMI to connect to the systems and this error occurred due to a DCOM security issue. This issue is addressed in this hotfix. Issue 2 The following enhancements have been made to VCW to address the issues: - VCW by default now deletes older/stale authentication credentials while reconfiguring secure clusters. - VCW now performs all the authentication related steps as part of the reconfigure cluster option. VCW by default reconfigures the security credentials used by the HAD and the CMDSERVER services. - VCW now includes additional logging capabilities for tracking all the AT commands run while configuring single sign-on clusters (secure clusters). VCW logs details about each authentication configuration related command (Setuptrust, DeleteCred, CreatePD, CreatePRPL, Authenticate) it runs on every cluster node. Binary / Version: VCWDlgs.dll / 5.1.20039.507 -------------------------------------------------------+ [19] Hotfix name: Hotfix_5_1_20042_2770008 Symptom: This hotfix addresses an issue related to the VCS Cluster Manager (Java Console) where a service group switch or failover operation to the secondary site fails due to a user privilege error. Description: This issue occurs in secure clusters set up in a disaster recovery (DR) environment. When you switch or failover a global service group to the remote site, the operation fails with the following error: Error when trying to failover GCO Service Group. V-16-1-50824 At least Group Operator privilege required on remote cluster. This error occurs even if the user logged on to the Java Console is a local administrator, which by default has the Cluster Administrator privilege in the local cluster. This issue occurs because the local administrator is not explicitly added to the cluster admin group at the remote site. During a switch or a failover, the Java Console is unable to determine whether the logged-on user at the local cluster has any privileges on the remote cluster and hence fails the operation. If you use the Java Console to grant the local administrator with operator or administrator privileges to the remote cluster, the Java Console only assigns guest privileges to that user. Resolution: This issue is fixed in the Java Console. The Java Console now allows you to assign the local administrator at the local cluster with cluster admin or cluster operator privileges in the remote cluster. This change is applicable only on Windows. Note: This hotfix is applicable to server components only. A separate hotfix is available for client components. Contact Symantec Technical Support for more details. Binary Name / Version: VCSGui.jar / NA -------------------------------------------------------+ [20] Hotfix name: HotFix_5_1_20043_2817466 Symptom: This hotfix addresses an issue where the Microsoft Windows Driver Verifier tool detects a memory violation in the VCS GAB module and causes a system crash. Description: When the GAB membership changes due to LLT network link transitions, the internal GAB data structure is released as it is no longer needed. However, the GAB driver tries to access a pointer that is already freed. This can occur in a different thread while another thread retains the stale pointer and tries to access the memory that was released. The problem is compounded in multi-CPU, multi-threaded systems. The special pool checking option of the Microsoft Windows Driver Verifier tool detects this memory violation issue and causes a system crash. Resolution: The GAB driver has been updated to prevent the release of memory that is still in use. Binary / Version: gab.sys / 5.1.20043.642 -------------------------------------------------------+ Post-install steps ==================| The following section describes the steps that must be performed after installing the hotfixes included in this CP. Note that these steps are applicable only to the hotfixes listed in this section. [1] Hotfix_5_1_20002_2220352 Perform the following steps only if you have installed Hotfix_5_1_20002_2220352 as part of the CP installation. Configuring the registry parameter to support non-scoped file shares --------------------------------------------------------------------+ Perform the following steps to create a registry key that is used by the VCS FileShare and Lanman agents to support non-scoped file shares on Windows Server 2008 and 2008 R2 systems. This key is created in the context of a Lanman resource; if you have multiple VCS file share service groups and wish to use the shares in a non-scoped mode, you must create a corresponding key for each Lanman resource that is configured in the file share service group. Caution: Incorrectly editing the registry may severely damage your system. Before making changes to the registry, make a backup copy. To create the registry key: 1. Ensure that you have installed this hotfix (Hotfix_5_1_20002_2220352) on all the cluster nodes. 2. To open the Registry Editor, click Start > Run, type regedit, and then click OK. 3. In the registry tree (on the left), navigate to HKLM\SOFTWARE\VERITAS\VCS\BundledAgents. If the Lanman key exists, go to step 4 (next step), else go to step 5. 4. Verify if under Lanman key there is a key named __Global__\DisableServerNameScoping. If it exists, change the value of DisableServerNameScoping to 0. Proceed to step 6. 5. Click Edit > New > Key and create a key by the name Lanman. 6. Select the Lanman key and click Edit > New > Key and create a key by the name . Here should be the virtual computer name assigned to the file share server. This is the VirtualName attribute of the Lanman resource in the file share service group. The newly created registry key should look like this: HKLM\SOFTWARE\VERITAS\VCS\BundledAgents\Lanman\. 7. Select the key that you created in step 5 () and add a DWORD type of value. Value name should be DisableServerNameScoping and Value data should be 1. The value 1 indicates that the FileShare and Lanman agents support non-scoped file shares on Windows Server 2008 and 2008 R2 systems. 8. If there are multiple file share service groups to be used in the non-scoped mode, repeat steps 5 and 6 for each Lanman resource that is configured in the file share service group. Note: You must create this key only for Lanman resources that are part of VCS file share service groups. Configuring this key for Lanman resources that are part of other VCS service groups may result in an unexpected behavior. 9. Save and exit the Registry Editor. [2] Hotfix_5_1_20041_2722108 Perform the following steps only if you have installed Hotfix_5_1_20041_2722108 as part of the CP installation. Setting the DisableStrictVirtualNameCheck registry key ------------------------------------------------------+ This hotfix creates the following registry key: HKLM\SOFTWARE\VERITAS\VCS\BundledAgents\Lanman\__Global__\DisableStrictVirtualNameCheck Use this registry key to disable FileShare agent's virtual name check. When DisableStrictVirtualNameCheck is set to 1 (server name check is disabled), the FileShare agent makes the file shares accessible even if the virtual name is not accessible on the network. In case the virtual name is not resolvable, the file shares are accessible using the virtual IP. Caution: Incorrectly editing the registry may severely damage your system. Before making changes to the registry, make a backup copy. To set DisableStrictVirtualNameCheck registry key value: 1. To open the Registry Editor, click Start > Run, type regedit, and then click OK. 2. In the registry tree (on the left), navigate to HKLM\SOFTWARE\VERITAS\VCS\BundledAgents\Lanman\__Global__\. 3. Select the key DisableStrictVirtualNameCheck and change its value data to 1. The value 1 indicates that the FileShare agent makes the non-scoped file shares accessible irrespective of whether or not the virtual name is accessible on the network. In case the virtual name is not resolvable, the file shares are accessible using the virtual IP. 4. Save and exit the Registry Editor. Known issues ============| The following section describes the issues related to the individual hotfixes that are included in this CP. [1] Hotfix_5_1_20001_2164975 Issue 1 If you install this hotfix before configuring the VCS cluster (as in, VCS is installed on the system but the cluster is not yet configured), then the installer will fail while performing the post-installation tasks. The installer fails to update the resource type definition for the DiskRes agent. Workaround: To resolve this issue, perform the following steps after completing the hotfix installation: 1. Configure the VCS cluster on all the systems where you have installed this hotfix. Refer to the Veritas Cluster Server Administrator's Guide for instructions. 2. After configuring the cluster, ensure that the VCS High Availability Engine (HAD) is running on a cluster node. Type the following on the command prompt: hasys -state The output of the command must display the SysState attribute as RUNNING. 3. Navigate to the hotfix directory. - On 32-bit systems, the path is %commonprogramfiles%\VeritasShared\WxRTPrivates\Hotfix_5_1_20001_2164975. - On 64-bit systems, the path is %commonprogramfiles(x86)%\Veritas Shared\WxRTPrivates\Hotfix_5_1_20001_2164975. 4. Run the LDMTypes_Install.bat file. The batch file updates the DiskRes resource type definition. 5. Repeat steps 2 to 4 on all the cluster nodes where you have installed this hotfix. Issue 2 If you have configured folder mounts in a service group, then after installing this hotfix, the Mount resources may go in to an unknown state. Workaround: To resolve this, specify the complete mount path value in the MountPath attribute itself. For example, if MountPath value is \Dir and MountResName value is the name of the resource monitoring X:, then change the MountPath attribute value to X:\Dir. The MountPath attribute value must be the complete mount path. You can change the attribute value either before or after installing the hotfix. Additional notes ================| [+] To confirm the list of cumulative patches installed on a system, run the following command from the directory where the CP files are extracted: vxhfbatchinstaller.exe /list The output of this command displays a list of cumulative patches and the hotfixes that are installed as part of a CP. This command also displays the hotfixes that are included in a CP but are not installed on the system. [+] To confirm the installation of the hotfixes, run the following command: vxhf.exe /list The output of this command lists the hotfixes installed on the system. [+] For details about a particular hotfix, run the following command: vxhf.exe /display: Here, is the name of the hotfix file without the platform and the .exe extension. [+] The CP installer (vxhfbatchinstaller.exe) creates and stores logs at: "%allusersprofile%\Application Data\Veritas\VxHF\VxHFBatchInstaller.txt" [+] The hotfix installer (vxhf.exe) creates and stores logs at: "%allusersprofile%\Application Data\Veritas\VxHF\VxHF.txt" [+] For general information about the hotfix installer (vxhf.exe), please refer to the following technote: http://www.symantec.com/business/support/index?page=content&id=TECH73446 [+] To view a list of hotfixes already installed on a system, please refer to the steps mentioned in the following technote: http://www.symantec.com/business/support/index?page=content&id=TECH73438 [+] For information on uninstalling a hotfix, please refer to the steps mentioned in the following technote: http://www.symantec.com/business/support/index?page=content&id=TECH73443 Disclaimer ==========| This fix is provided without warranty of any kind including the warranties of title or implied warranties of merchantability, fitness for a particular purpose and non-infringement. Symantec disclaims all liability relating to or arising out of this fix. It is recommended that the fix be evaluated in a test environment before implementing it in your production environment. When the fix is incorporated into a Storage Foundation for Windows maintenance release, the resulting Hotfix or Service Pack must be installed as soon as possible. Symantec Technical Services will notify you when the maintenance release (Hotfix or Service Pack) is available if you sign up for notifications from the Symantec support site http://www.symantec.com/business/support and/or from Symantec Operations Readiness Tools (SORT) http://sort.symantec.com.