glmtrace - Group Lock Manager (GLM) trace utility

glmtrace [ -d ]

glmtrace [ -t locktype ] [ -f fset ] [ -v volid ] [ -i inumber ] [ filename ]

VRTSglm

The glmtrace command prints lines that describe GLM trace events. Currently, the trace events are messages sent or received by a GLM primary, on behalf of the given file. Only messages to or from a primary on the current node are displayed. If the location of the primary is unknown, run glmtrace on each node in the cluster.

Currently, glmtrace prints trace records indefinitely. To stop the printing, send glmtrace a signal, such as by typing Ctrl-C.

Trace records are written to a circular buffer in the kernel before being read and displayed by glmtrace. If this buffer becomes full, new entries are not added until glmtrace reads the oldest entries, which makes more room. The output contains the line "[lost records...]" at the appropriate point.

When glmtrace is invoked, the command clears the current trace buffer, and then waits for new trace records. Therefore, all events reported by glmtrace had occurred after glmtrace was invoked.

Since GLM uses a global trace buffer, you cannot run glmtrace more than once on the same node at the same time, even if the multiple invocations are naming different files.

The glmtrace command is intended primarily for troubleshooting by Veritas personnel. The command’s functionality and output are subject to change.

-t locktype	Limit events to this locktype only. Locktype is rw (rwlock), h (hlock), g (glock), or pg (pglock). If you do not specify -t, glmtrace traces all locks for this file.
-f fset	Limit events to files in this fileset. The argument is the fileset number. The default value of fset is 999.
-v volid	Limit events to files on this volume. The volid argument is the number used by Cluster Volume Manager to identify the volume. If omitted, all files with the given inumber and fset are traced.
-i inumber	Limit events to files with this inumber. This option may be omitted if a filename is supplied.
-d	Disable tracing. This will cause any running glmtrace on the same node to terminate.

Currrently, GLM is unable to determine the fset and volid for the filename on the command line. These must be supplied explicitly if the defaults are unsuitable.

The arguments must include either a filename or an inumber (with -i). Currently, you cannot trace locks that are not associated with an inode, such as a cbuf lock. You also cannot trace locks for multiple files.

There is no option for enabling tracing; this happens when glmtrace is run. Once tracing is enabled, collecting the trace data imposes a small performance overhead. Tracing may be disabled explicitly with the -d option. Once no trace data has been read for 10 minutes, GLM disables tracing automatically.

Each line of output describes a message. The columns are:

time	A timestamp telling when the message was sent or received. The units are seconds; the 3 digits after the decimal point are milliseconds. The timestamp may be usefully compared with other timestamps from the same node only.
msg	The name of the message.
from	The sending node of the message. These are the LLT node numbers, from /etc/llthosts.
to	The receiving node of the message.
type	The locktype. See the -t option.
reqid	The request ID. This is an internal GLM number used for matching requests and responses.
flags	Flags from the message. Values of interest are as follows: 00000002 GLM_GREEDY 00000004 GLM_SHPREF 00000008 GLM_ASYNC 00000020 GLM_TRY 00000200 GLM_SEMI_STRONG 00000400 GLM_INVAL_PBDATA 00001000 GLM_RECOVERY 00002000 GLM_READAHEAD 00020000 GLM_GET_GRANT 08000000 GLM_NORMALIZING 10000000 GLM_STALE 20000000 GLM_GHOST
level	The lock level requested.
range	The range requested. This message is displayed for range locks only.

All numbers display in decimal, except the message flags, which display in hex.

glmstat(1M)