OS: AIX
OS Version: 5.2, 5.3 & 6.1
Fixes Applied for Products:
	VRTSllt - Veritas Low Latency Transport by Symantec

Additional Instructions:
Please read the instructions below before installing the patch.

 PATCH VRTSllt 5.0 MP3RP2HF2 for LLT 5.0MP3RP2
===============================================================

 Patch Date:  December, 2009

This README provides information on:
 * BEFORE GETTING STARTED
 * CRC AND BYTE COUNT
 * FIXES AND ENHANCEMENTS INCLUDED IN THE PATCH
 * PACKAGES AFFECTED BY THE PATCH
 * INSTALLING THE PATCH IN VCS ENVIRONMENT
 * UNINSTALLING THE PATCH IN VCS ENVIRONMENT
 * INSTALLING THE PATCH IN SFRAC ENVIRONMENT
 * UNINSTALLING THE PATCH IN SFRAC ENVIRONMENT


BEFORE GETTING STARTED:
----------------------
This patch only applies to:
	VRTSllt 5.0.3.200 or higher running on AIX 5.2, 5.3 or 6.1

Ensure that you are running the supported configurations before
installing this patch.


CRC AND BYTE COUNT:
------------------
Ensure that the patch file you have downloaded matches the following
checksum and byte count.

The following command can be used to ascertain this:
# cksum VRTSllt.rte.bff
374053313 3891200 VRTSllt.rte.bff


FIXES AND ENHANCEMENTS INCLUDED IN THE PATCH: 
--------------------------------------------
Etrack Incidents: 1839321, 1860153, 1839299

SDR's of Fixed Symantec Incidents:
--------------------------------
Symantec Incident : 1839321
Symptom:
	While sending a large number of pkts from one node to 
	another node, we see that the receiving side LLT at 
	times trails the sending side LLT by a significant 
	number of pkts. This causes the sender to send pkts 
	in bursts instead of a smooth flow of pkts from 
	the sender to the receiver.
Defect Description:
	The sender does not immediately know about the change 
	in receive side flow control situations. There is some 
	time lag until this information is conveyed to the sender. 
	Thus, some times sender could flood the receiver with 
	pkts when it should wait and at other times there could 
	be a delay in sending the pkts.
Resolution:
	The receiver immediately sends information to the sender 
	about the changes in the flow control situation at receiving side.

Symantec Incident : 1860153
Symptom:
	In the high system load situations LLT fails to send 
	heartbeats to the other nodes, which may cause system panic.
Defect Description:
	LLT sends the heartbeat from the timer context. If the LLT 
	timer handler is not called due to high system load for 
	the LLT peerinact time period, then the peer node declares 
	this node dead and it panics in future.
Resolution:
	With this fix LLT sends heartbeats from the multiple contexts
	(e.g. packet receive), so that even if the LLT timer handler is not
	called on the time the LLT doesn't miss to send heartbeats.

Symantec Incident : 1839299
Symptom:
	The following messages appear in the syslog when a client 
	is generating heavy packet traffic:
	"
	vxvm:vxconfigd: advresp_master: GAB returned EAGAIN,  retrying
	vxvm:vxconfigd: advresp_master: Retry Successful
	vxvm:vxconfigd: advresp_master: GAB returned EAGAIN, retrying
	vxvm:vxconfigd: advresp_master: Retry Successful
	"
Defect description:
	As the client is busy sending data packets to LLT continuously, 
	a race between logic for disabling and enabling generation of 
	more data traffic might occur. Because of which, the above 
	mentioned messages are seen in the syslog. 
Resolution:
	Proper logic has been added to enable and disable generation 
	of more data traffic.


PACKAGES AFFECTED BY THE PATCH:
-------------------------------
This patch updates the following VCS package(s) 
	VRTSllt.rte from 5.0.3.200 or higher to 5.0.3.202


INSTALLING THE PATCH IN VCS ENVIRONMENT:
---------------------------------------
The following steps should be run on all nodes in the VCS cluster:

Stopping the cluster:
--------------------
1. Offline all applications, which are configured on CVM/CFS 
   and are outside VCS control.

    After all applications using CFS and CVM have been taken down,
    run 'slibclean' to unload the libraries from memory.

2. Stop VCS on the current node.
        # /opt/VRTSvcs/bin/hastop -local 
   Verify that ports 'f' (CFS), 'v' and 'w' (CVM), 'h' (VCS) have been closed,
        # /sbin/gabconfig -a
   The display should not have port 'f', 'v', 'w' and 'h' listed

3. If VXFEN is not configured, please go to step 5

4. Unconfigure VxFen:
        #  /sbin/vxfenconfig -U
   Verify that port 'b' has been closed
        # /sbin/gabconfig -a
   The display should not have port 'b' listed  

5. Unconfigure GAB:
        # /sbin/gabconfig -U

6. Unconfigure LLT:
        # /sbin/lltconfig -Uo

7. Unload the LLT driver:
        # /usr/sbin/strload -ud /usr/lib/drivers/pse/llt

8. Verify that the LLT driver has been unloaded
        # /usr/sbin/strload -qd /usr/lib/drivers/pse/llt
        /usr/lib/drivers/pse/llt: no
   If llt is still loaded "yes" will show up in the output above.

NOTE: If you are unable to successfully unload the LLT driver,
the server must be rebooted AFTER the installation of the patch. 
This is so that the new LLT driver gets loaded in the AIX kernel.


Installing the Patch:
--------------------
1. Un-compress the downloaded patch from Symantec.
   Change directory to the unzipped patch location. 
   Install the VRTSllt (5.0.3.202) patch using the following command:
	# installp -a -d [extract dir]/patches/VRTSllt.rte.bff VRTSllt.rte

2. Verify that the new patch has been installed:
	# lslpp -l VRTSllt.rte
   You will find the following output on display with the patch installed properly:
  VRTSllt.rte              5.0.3.202  APPLIED    Veritas Low Latency Transport
                                                 by Symantec
                                                 Veritas-5.0MP3RP2HF2-12/17/09-
                                                 14:29:00 Build Env:AIX
                                                 sheridan 2 5 00023B7A4C00
                                                 IBM, 7029-6E3

Re-starting the cluster:
-----------------------
1. Verify that the new LLT driver has been loaded:
        # strload -qd /usr/lib/drivers/pse/llt
        /usr/lib/drivers/pse/llt: yes

2. If not already loaded, load the newly installed LLT driver:
        # strload -d /usr/lib/drivers/pse/llt

3. Configure LLT:
        # /sbin/lltconfig -c

4. Verify that LLT has been configured properly
        # /sbin/lltconfig
        LLT is running

5. Configure GAB:
        # sh /etc/gabtab

6. Verify that the GAB membership shows up correctly:
        # /sbin/gabconfig -a
   The display should have Port 'a' listed

7. Configure VxFen (if VxFEN was configured previously)
        # /sbin/vxfenconfig -c
   Verify that vxfen has been configured
        # /sbin/gabconfig -a
   The output should list port 'b'

8. Start VCS:
        # /opt/VRTSvcs/bin/hastart
   Verify that VCS is up and running:
        # /sbin/gabconfig -a
   The display should show port 'f', 'v', 'w' and 'h' listed.
   The 'f', 'v' and 'w' port will be listed if CVM and CFS are configured.

9.  Start applications (stopped earlier), which are outside VCS control.

Committing the Patch:
--------------------
1.  To commit the patch:
(Note: The patch cannot be backed out once it is committed.)
        # installp -c VRTSllt.rte
2.  Verify that the fileset is committed:
        lslpp -l VRTSllt.rte
  VRTSllt.rte              5.0.3.202  COMMITTED  Veritas Low Latency Transport
                                                 by Symantec
                                                 Veritas-5.0MP3RP2HF2-12/17/09-
                                                 14:29:00 Build Env:AIX
                                                 sheridan 2 5 00023B7A4C00
                                                 IBM, 7029-6E3

UNINSTALLING THE PATCH IN VCS ENVIRONMENT:
-----------------------------------------
The VRTSllt.rte.bff patch can ONLY be backed out if it has not been committed.

NOTE: Before uninstalling patch, make sure that the APAR changing 
DLPI behaviour is not installed on the system by running 
following commands:
  # instfix  -iv | grep "BRING DLPI DRIVER "TO SPEC""

If above mentioned command returns an APAR then backing out
this point patch will move llt to older version, which will 
cause panic or hang.

Steps to Backout the Patch:
--------------------------
1. Follow the steps provided under "Stopping the cluster" section above, 
   to stop the cluster & unload the drivers.

2. Backout the patches by the following command:
	# installp -r VRTSllt.rte 5.0.3.202

3. Verify that the patch has been removed from the system:
   (Note: The previously installed fileset(s) will be in committed state again.
       It may differ from the mentioned, if a Hotfix was installed on top 
       of VCS 5.0MP3RP2)
        # lslpp -l VRTSllt.rte
  VRTSllt.rte              5.0.3.200  COMMITTED  Veritas Low Latency Transport
                                                 5.0MP3RP2 by Symantec
                                                 07/07/2009-17:19:00 Build
                                                 Env:AIX ogma 3 5 0001D4CAD300
                                                 IBM, 9115-505

4. Restart the node following the steps under 
 "Re-starting the cluster" section above.


INSTALLING THE PATCH IN SFRAC ENVIRONMENT:
-----------------------------------------
The following steps should be run on all nodes in the cluster,
with SFRAC stack installed:

1. Offline all applications, which are configured on CVM/CFS 
   and are outside VCS control.

2. If Oracle database is not configured in VCS, stop it using following command:
        $ srvctl stop instance -d [database name] -i [instance name]

3(a). For Oracle 9iR2, stop 'gsd' using the follwing command as Oracle user
        $ gsdctl stop
   To check the status of gsdctl, run the following command:
        $ gsdctl stat
   The gsdctl command is typically found in $ORACLE_HOME/bin.

3(b). For Oracle 10gR1 and 10gR2, Stop CRS manually, 
      if CRS is not under VCS control.
         # /etc/init.crs stop

4.  After all the oracle instances and other applications using 
    CFS and CVM have been stopped, run 'slibclean' to unload 
    the libraries from memory.

5. Stop VCS on the current node.
        # /opt/VRTSvcs/bin/hastop -local 

6. Verify that ports 'h', 'f', 'v' and 'w' have been closed
        # /sbin/gabconfig -a
   The display should not have ports 'h', 'f', 'v' and 'w' listed

7. Unconfigure VCSMM:
        # /sbin/vcsmmconfig -U
   Verify that port 'o' has been closed
        # /sbin/gabconfig -a
   The display should not have port 'o' listed.
   If it does ensure that Oracle instances are offline.

8. Unconfigure LMX:
        # /sbin/lmxconfig -U

9. Unconfigure VxFen:
        # /sbin/vxfenconfig -U
    Verify that port 'b' has been closed
        # /sbin/gabconfig -a
    The display should not have port 'b' listed

10. Unmount ODM:
        # umount /dev/odm
    Verify that port 'd' has been closed
        # /sbin/gabconfig -a
    The display should not have port 'd' listed

11. At this point all gab ports except port 'a' should have been closed
    Verify this as follows:
        # /sbin/gabconfig -a

12. Follow steps 5 to 8 of "Stopping the cluster" section from 
    "INSTALLING THE PATCHES IN VCS ENVIRONMENT" chapter above.

13. Follow all the instruction in "Installing the patch" section
    from "INSTALLING THE PATCHES IN VCS ENVIRONMENT" chapter above.

14. Follow steps 1 to 7 of "Re-starting the cluster" section from 
    "INSTALLING THE PATCHES IN VCS ENVIRONMENT" chapter above.

15. Configure LMX:
        # /sbin/lmxconfig -c

16. Configure VCSMM:
        # /sbin/vcsmmconfig -c
    Verify that vxfen has been configured
        # /sbin/gabconfig -a
    The output should list port 'o'

17. Mount ODM:
        # mount /dev/odm

18. Start VCS:
        # /opt/VRTSvcs/bin/hastart

19. Check if all ports are now open
        # /sbin/gabconfig -a
    The output should list ports 
    'a', 'b', 'd', 'f', 'h', 'o', 'v', and 'w'.

20(a). For Oracle 10gR1 and 10gR2, start CRS manually,
       if CRS is not under VCS control.
         # /etc/init.crs start

20(b). For Oracle 9iR2, start 'gsd' using the follwing command as Oracle user
        $ gsdctl start
   To check the status of gsdctl, run the following command:
        $ gsdctl stat
   The gsdctl command is typically found in $ORACLE_HOME/bin.

21. If Oracle database is not configured in VCS, 
    start it using following procedure.
        $ srvctl start instance -d [database name] -i [instance name]

22. Online all applications, which are configured on CVM/CFS 
    and are outside VCS control (stopped earlier).

23. To commit the patches follow "Committing the Patch" section from 
    "INSTALLING THE PATCHES IN VCS ENVIRONMENT" chapter above.



UNINSTALLING THE PATCHES IN SFRAC ENVIRONMENT
---------------------------------------------
The VRTSllt.rte patch can ONLY be backed out if it has not been committed.

NOTE: Before uninstalling patch, make sure that the APAR 
changing DLPI behaviour is not installed on the system by
running following commands:
   # instfix  -iv | grep "BRING DLPI DRIVER "TO SPEC""

If above mentioned command returns an APAR then backing out
this point patch will move llt to older version, which will 
cause panic or hang.

Steps to Backout the Patch:
---------------------------
1.  Follow the steps outlined 1 through 12 of chapter 
    "INSTALLING THE PATCHES IN SFRAC ENVIRONMENT"
    to stop and unload the drivers.

2. Backout the patches:
        # installp -r VRTSllt.rte 5.0.3.202

3. Verify that the patch has been backed out:
   (Note: The previously installed fileset(s) will be in committed state again.
       It may differ from the mentioned, if a Hotfix was installed on top 
       of VCS 5.0MP3RP2)

        # lslpp -l VRTSllt.rte
  VRTSllt.rte              5.0.3.200  COMMITTED  Veritas Low Latency Transport
                                                 5.0MP3RP2 by Symantec
                                                 07/07/2009-17:19:00 Build
                                                 Env:AIX ogma 3 5 0001D4CAD300
                                                 IBM, 9115-505

4. Next as before go through the process of loading and configuring
   LLT, GAB and bringing up SFRAC (steps 14 through 23 above of 
   chapter "INSTALLING THE PATCHES IN SFRAC ENVIRONMENT").

 Note: The llt driver will now refer to the old one.