Error Code details
V-16-1-51047
Severity: Warning 
Component: Cluster Server 
Message:
HAD Self Check: Excessive delay in the HAD heartbeat to GAB (10 seconds)
Description:

The Veritas Cluster Server (VCS) high availability daemon (HAD) registers itself with Global Atomic Broadcast (GAB) with a contract that sends a heartbeat to GAB every VCS_GAB_TIMEOUT seconds. For Storage Foundation and High Availability releases earlier than 5.1 SP1, VCS_GAB_TIMEOUT defaults to 15 seconds. For release 5.1 SP1 and later, the default is 30 seconds. 

 

By default, HAD sets an alarm every 10 seconds to check if the heartbeat was sent. If the heartbeat has not been sent when the alarm is invoked, VCS records the message.

 

Veritas solutions
Solution 1 Vote: [Useful] [Not useful]
Last Modified: 2011-11-08 02:46:54
Platform: Generic
Release: Generic
Content:

If you receive this message once, you may ignore it because there may be a temporary, heavy load on the system.


If you receive this message repeatedly, there is a heavy load on the system and HAD is so busy that it cannot send a heartbeat to GAB. If the heartbeat delay is over VCS_GAB_TIMEOUT seconds, GAB kills the HAD process. Check the performance of the system and tune it accordingly.
 

Even if you set VCS_GAB_TIMEOUT to a large value (for example, 100 seconds), you still see these messages every 10 seconds if VCS cannot send a heartbeat after 100 seconds. The reason is because the alarm has been set to repeat every 10 seconds.