vxclustadm - start, stop, and reconfigure a cluster
vxclustadm [-v] nodestate
vxclustadm -m vcs reinit
vxclustadm setmaster|setprimary nodename
vxclustadm setpreference value
vxclustadm -m vcs -t gab startnode
The vxclustadm utility activates and deactivates cluster functionality on a node in a cluster.
Caution: Use of the clustering functionality of VxVM without a cluster monitor is not supported. Cluster reconfiguration problems may occur if there is no cluster monitor or if GAB is used as the cluster monitor. Ensure that you completely understand the functionality of this command before using it.
Note: Master and Slave words are going to be deprecated in upcoming releases. We will refer to them using Primary and Secondary words respectively.
getpreference Displays the preference values for the node. Cluster Volume Manager (CVM) uses the preference values to determine the most appropriate candidates to take over the Primary role in case of Primary node failover. CVM assigns weight values based on internal criteria, such as disk connectivity. You can also set customized values with the setpreference keyword. The getpreference command displays the customized value as the preference, and displays the preference calculated by CVM as the preference delta. The effective preference value is the total. nidmap Prints a table showing the mapping between node IDs in VxVMs cluster-support subsystem and node IDs in the cluster monitor. nodestate Displays the state of a node in the cluster and the reason for last abort of the node on the standard output. Valid states are:
cluster aborting The node is being aborted from the cluster. cluster member The node is a member of the cluster. All shared volumes in the cluster are accessible. joining The node is in the process of joining a cluster. It has been initialized but is not yet completely in the cluster. The node goes into this state after vxclustadm is executed with the startnode keyword. out of cluster The node is not joined to the cluster. Refer to the Storage Foundation Cluster File System High Availability Administrators Guide for more information about reasons why a node may leave a cluster. For debugging purposes the -v option can be specified to display the node ID, Primary ID, neighbor ID, current state, and reason for a node leaving the cluster (if appropriate). reinit The reinit keyword allows nodes to be added to or removed from a cluster dynamically without stopping the cluster. The command causes vxclustadm to re-read the cluster configuration file, and implement any required changes to the membership of the cluster. The -m vcs option specifies the VCS cluster monitor, which implies the existence of the cluster configuration file, /etc/VRTSvcs/conf/config/main.cf. setmaster|setprimary The setmaster or setprimary keyword migrates the CVM Primary to the specified node. The migration is an online operation. CVM does not internally check for the health of the Primary before triggering a Primary switch. Veritas recommends that you switch the Primary when the cluster is not handling VxVM configuration changes or cluster reconfiguration operations. In most cases, CVM aborts the operation to change the Primary, if CVM detects that any configuration changes are occuring in the VxVM or the cluster. After the Primary change operation starts reconfiguring the cluster, other commands that require configuration changes will fail. You can retry the commands after the Primary change operation is complete. setpreference Sets customized preference values for the node. You can assign an integer value as preference to the nodes. CVM selects the Primary node from the pool of the nodes with the highest preference. The custom weight values do not prevent CVM from assigning weights. CVM increases or decreases the weight values, starting from the custom value, or from 0 if no custom value is set. To make sure that your preference has the effect that you want, consider the effect of the values that CVM generates. For more information about preferences, see the Storage Foundation Cluster File System High Availability Administrators Guide. startnode The startnode keyword initiates cluster functionality on a node using the information supplied in the cluster configuration file. This is the first command that must be issued on a node to bring it into the cluster. The argument to the -m option specifies the cluster monitor, which implies the existence of a cluster configuration file:
vcs The cluster is running in the VCS environment. The cluster configuration file is /etc/VRTSvcs/conf/config/main.cf. Caution: Use VCS commands to edit the main.cf file. Do not edit this file by hand. startnode passes the information in the cluster configuration file to the VxVM kernel. In response to this command, the kernel and the VxVM configuration daemon, vxconfigd, perform the initialization. The argument to the -t option specifies the protocol to be used for messaging:
gab Use GAB as the transport agent for messaging in addition to using GAB as a cluster monitor. If you try to use GAB as a transport agent with a cluster monitor other than GAB (or outside the VCS or HP Serviceguard environment), the kernel changes the transport agent to UDP. When the cluster is running in the VCS environment, the clustering functionality of VxVM should use GAB as the transport agent for messaging. stopnode Stops cluster functionality on a node, and waits for all outstanding I/O to complete and for all applications to close shared volumes or devices.
vxclustadm returns the following exit values:
2 Invalid state. 101 Node is not in cluster. 102 Node is joining the cluster, or is involved in reconfiguration. 103 Node is a cluster member. 104 Node is aborting from cluster.
For a cluster that is operating without a cluster monitor, or that is using GAB as the cluster monitor outside the VCS environment, and which is using UDP as its transport agent for messaging, the cluster configuration file, /etc/vx/cvmtab, contains the following fields:
clustername cluster_name port vxconfigd port_number port vxkmsgd port_number node node_ID name name_on_local_net timeout timeout_value ...
The recommended port numbers for the vxconfigd and vxkmsgd daemons are 4500 and 4501, but any available port numbers greater than 1024 are also acceptable.
name_on_local_net is the nodes IP address or resolvable host name on the clusters private network.
timeout_value is the timeout value in seconds. The clustering functionality of VxVM uses this value during cluster reconfiguration. The appropriate value to use depends on the number of nodes in the cluster and on the size of the shared disk group configuration. In most cases the value of 200 seconds is sufficient but this may need to be increased for larger configurations.
Comment lines in the file start with a #.
If GAB is being used as the transport agent for messaging, fields relating to port numbers and local network names are not required:
clustername cluster_name node node_ID name timeout timeout_value ...
For a cluster running in the VCS environment, VxVM obtains information about the cluster from the VCS cluster configuration file (/etc/VRTSvcs/conf/config/main.cf). Cluster-specific information may be appended to this file by running the vxcvmconfig command. For more information refer to the Storage Foundation Cluster File System High Availability Configuration and Upgrade Guide
A cluster consisting of four nodes, named node0, node1, node2 and node3, operates without a cluster monitor, and has the following cvmtab file when UDP is used as the transport agent for messaging:
# ClusterName clustername CVM1 # Daemon port numbers port vxconfigd 4500 port vxkmsgd 4501 # NodeID Nodename Localname node 0 node0 node0_p node 1 node1 node1_p node 2 node2 node2_p node 3 node3 node3_p # Timeout value timeout 200
If GAB is used as the transport agent for messaging, the cvmtab file only needs to contain the following information:
# ClusterName clustername CVM1 # NodeID Nodename node 0 node0 node 1 node1 node 2 node2 node 3 node3 # Timeout value timeout 200
If node1 is the first node to join the cluster, it becomes the Primary node. The following command confirms that node1 is the Primary node:
vxdctl -c mode
To determine if reconfiguration of node3 is complete, examine the value returned from running the following command on node3:
vxclustadm -v nodestate
To confirm that node3 is a Secondary node, the following command is run on node3:
vxdctl -c mode
node1 remains as the Primary node for its lifetime in the cluster. To remove node1 from the cluster, the following command is run on node1:
vxclustadm does not ensure the consistency of cluster membership information.
vxconfigd(1M), vxdctl(1M), vxintro(1M)
Storage Foundation Cluster File System High Availability Administrators Guide
Storage Foundation Cluster File System High Availability Configuration and Upgrade Guide