Volume reconfiguration is the process of creating, changing, and removing VxVM objects such as disk groups, volumes and plexes. In a cluster, all nodes co-operate to perform such operations. The vxconfigd
daemons (see vxconfigd daemon) play an active role in volume reconfiguration. For reconfiguration to succeed, a vxconfigd
daemon must be running on each of the nodes.
A volume reconfiguration transaction is initiated by running a VxVM utility on the master node. The utility contacts the local vxconfigd
daemon on the master node, which validates the requested change. For example, vxconfigd
rejects an attempt to create a new disk group with the same name as an existing disk group. The vxconfigd
daemon on the master node then sends details of the changes to the vxconfigd
daemons on the slave nodes. The vxconfigd
daemons on the slave nodes then perform their own checking. For example, each slave node checks that it does not have a private disk group with the same name as the one being created; if the operation involves a new disk, each node checks that it can access that disk. When the vxconfigd
daemons on all the nodes agree that the proposed change is reasonable, each notifies its kernel. The kernels then co-operate to either commit or to abandon the transaction. Before the transaction can be committed, all of the kernels ensure that no I/O is underway. The master node is responsible both for initiating the reconfiguration, and for coordinating the commitment of the transaction. The resulting configuration changes appear to occur simultaneously on all nodes.
If a vxconfigd
daemon on any node goes away during reconfiguration, all nodes are notified and the operation fails. If any node leaves the cluster, the operation fails unless the master has already committed it. If the master node leaves the cluster, the new master node, which was previously a slave node, completes or fails the operation depending on whether or not it received notification of successful completion from the previous master node. This notification is performed in such a way that if the new master does not receive it, neither does any other slave.
If a node attempts to join a cluster while a volume reconfiguration is being performed, the result of the reconfiguration depends on how far it has progressed. If the kernel has not yet been invoked, the volume reconfiguration is suspended until the node has joined the cluster. If the kernel has been invoked, the node waits until the reconfiguration is complete before joining the cluster.
When an error occurs, such as when a check on a slave fails or a node leaves the cluster, the error is returned to the utility and a message is sent to the console on the master node to identify on which node the error occurred.
The VxVM configuration daemon, vxconfigd
, maintains the configuration of VxVM objects. It receives cluster-related instructions from the vxclust
utility under Sun Java System Cluster software, or from the kernel when running VCS. A separate copy of vxconfigd
runs on each node, and these copies communicate with each other over a network. When invoked, a VxVM utility communicates with the vxconfigd
daemon running on the same node; it does not attempt to connect with vxconfigd
daemons on other nodes. During cluster startup, the vxclust
utility(
SunCluster) or the kernel (for VCS) prompts vxconfigd
to begin cluster operation and indicates whether it is a master node or a slave node.
When a node is initialized for cluster operation, the vxconfigd
daemon is notified that the node is about to join the cluster and is provided with the following information from the cluster monitor configuration database:
vxconfigd
daemon on each node (if applicable)
On the master node, the vxconfigd
daemon sets up the shared configuration by importing shared disk groups, and informs the vxclust
utility (for SunCluster) or the kernel (for VCS) when it is ready for the slave nodes to join the cluster.
On slave nodes, the vxconfigd
daemon is notified when the slave node can join the cluster. When the slave node joins the cluster, the vxconfigd
daemon and the VxVM kernel communicate with their counterparts on the master node to set up the shared configuration.
When a node leaves the cluster, the kernel notifies the vxconfigd
daemon on all the other nodes. The master node then performs any necessary cleanup. If the master node leaves the cluster, the kernels select a new master node and the vxconfigd
daemons on all nodes are notified of the choice.
The vxconfigd
daemon also participates in volume reconfiguration as described in Volume reconfiguration.
In a cluster, the vxconfigd
daemons on the slave nodes are always connected to the vxconfigd
daemon on the master node. If the vxconfigd
daemon is stopped, volume reconfiguration cannot take place. Other nodes can join the cluster if the vxconfigd
daemon is not running on the slave nodes.
If the vxconfigd
daemon stops, different actions are taken depending on which node this occurred:
vxconfigd
daemon is stopped on the master node, the vxconfigd
daemons on the slave nodes periodically attempt to rejoin to the master node. Such attempts do not succeed until the vxconfigd
daemon is restarted on the master. In this case, the vxconfigd
daemons on the slave nodes have not lost information about the shared configuration, so that any displayed configuration information is correct.
vxconfigd
daemon is stopped on a slave node, the master node takes no action. When the vxconfigd
daemon is restarted on the slave, the slave vxconfigd
daemon attempts to reconnect to the master daemon and to re-acquire the information about the shared configuration. (Neither the kernel view of the shared configuration nor access to shared disks is affected.) Until the vxconfigd
daemon on the slave node has successfully reconnected to the vxconfigd
daemon on the master node, it has very little information about the shared configuration and any attempts to display or modify the shared configuration can fail. For example, shared disk groups listed using the vxdg
list
command are marked as disabled
; when the rejoin completes successfully, they are marked as enabled
.
vxconfigd
daemon is stopped on both the master and slave nodes, the slave nodes do not display accurate configuration information until vxconfigd
is restarted on the master and slave nodes, and the daemons have reconnected.
If the vxclust
utility (for SunCluster) or the CVM agent (for VCS) determines that the vxconfigd
daemon has stopped on a node, vxconfigd
is restarted automatically.
If it is necessary to restart vxconfigd
manually in a VCS controlled cluster to resolve a VxVM issue, use this procedure:
Note
The -r
reset option to vxconfigd
restarts the vxconfigd
daemon and recreates all states from scratch. This option cannot be used to restart vxconfigd
while a node is joined to a cluster because it causes cluster information to be discarded.