vxdg (1M)

NAME

vxdg - manage Veritas Volume Manager (VxVM) disk groups

SYNOPSIS

vxdg [-g diskgroup] [-kp] [-f]
[-o overridessb] adddisk
    [ medianame=]accessname...

vxdg -g diskgroup [-f] addsite sitename

vxdg bootdg

vxdg defaultdg

vxdg [-n newname] [-h newhostid] deport diskgroup...

vxdg [-o coordinator] destroy diskgroup...

vxdg -g diskgroup [-f] detachsite sitename

vxdg flush [diskgroup...]

vxdg [-g diskgroup] [-u unit] [-qa] free [ medianame...]

vxdg [-Cfst] [-n newname] [-o clearreserve]
    [-o useclonedev={on|off}] [-o updateid]
    [-o groupreserve=fencekey [-o verify]]
    [-o noreonline] [-o selectcp=diskid]
    [-o site=sitename] [-o tag=name] import diskgroup

vxdg [-T version] [-s] [-o coordinator={off|on}] [-o groupreserve=fence_key]
init groupname
    [cds={off|on}] [minor=base-minor] [nconfig=config-copies] [nlog=log-copies]
    [medianame=]accessname...

vxdg [-o transretry=retrylimit] join sourcedg targetdg

vxdg [-q] [-s] [-o listreserve] list [diskgroup...]

vxdg [-g diskgroup] [-q] listclone

vxdg [-q] listmeta [diskgroup ...]

vxdg [-o expand] [-o transretry=retrylimit] listmove sourcedg targetdg object...

vxdg [-v] [-g diskgroup] listssbinfo

vxdg [-q] listtag [diskgroup ...]

vxdg -g diskgroup migrateispobjs

vxdg [-o expand] move
    sourcedg targetdg object...

vxdg [-g diskgroup] [-q] nohotuse [ medianame...]

vxdg -g diskgroup [-o clearreserve] [-o overridessb] [-C] reattachsite sitename

vxdg [-o {clean|remove}] recover diskgroup

vxdg [-g diskgroup] [-f] reminor
    [diskgroup] new-base-minor

vxdg [-g diskgroup] [-f] renamesite oldsite newsite

vxdg [-g diskgroup] [-k] repldisk
    unassoc-medianame=spare-medianame...

vxdg [-g diskgroup] [-k] rmdisk medianame...

vxdg -g diskgroup [-f] rmsite sitename

vxdg [-g diskgroup] rmtag [{e|encl|enclr}:enclname] name ...

vxdg -g diskgroup set attribute=value...

vxdg -g diskgroup set siteconsistent={on|off}

vxdg -g diskgroup set tagmeta={on|off} tag=name
    [nconfig=config-copies] [nlog=log-copies]

vxdg [-g diskgroup] [-f] settag [{e|encl|enclr}:enclname] name[=value] ...

vxdg [-g diskgroup] [-q] spare [ medianame...]

vxdg [-Cft] [-o expand] [-o transretry=retrylimit] split
    sourcedg targetdg object...

vxdg [-T version] upgrade diskgroup

DESCRIPTION

The vxdg utility performs administrative operations on disk groups. Operations include the creation of disk groups, the addition of disks to a disk group, disk group split and join, and disk group imports and deports.

Note: This man page uses disk and LUN interchangeably.

The behavior of the vxdg utility depends upon the keyword specified as the first operand.

A diskgroup argument can be either a disk group name or a disk group ID. A groupname argument is a disk group name, not a disk group ID. An accessname argument refers to a system-dependent disk access name (also referred to as a disk device name), as stored in the root configuration by the vxdisk utility. A medianame argument is an administrative name used to define a disk within a disk group.

The following sections describe reserved disk group names and the rules that commands use to determine which disk group to use if this is not specified.

System-Wide Reserved Disk Groups

The following disk group names are reserved, and cannot be used to name any disk groups that you create:
bootdg Specifies the boot disk group. This is an alias for the disk group that contains the volumes that are used to boot the system. VxVM sets bootdg to the appropriate disk group when the root disk is put under VxVM control. Otherwise, bootdg is set to nodg.
Caution: Do not attempt to change the value of bootdg. You may render your system unbootable.
defaultdg Specifies the default disk group to use if the -g option is not specified, or if the VXVM_DEFAULTDG environment variable is undefined. By default, defaultdg is set to nodg (that is, no disk group).
nodg Specifies to an operation that no disk group has been defined.
If you have upgraded your system to VxVM 4.0, you may find it convenient to continue to configure a disk group named rootdg as the default disk group (defaultdg).
Note: defaultdg and bootdg need not refer to the same disk group. Neither the default disk group nor the boot disk group need be named rootdg.

Rules for Determining the Default Disk Group

The recommended best practice is to use the -g option to specify a disk group to VxVM commands. If you do not specify the disk group, VxVM uses the following rules to determine a disk group name, in the order shown:
1. Use the disk group that is specified by the environment variable VXVM_DEFAULTDG. This variable can be set to one of the reserved system-wide disk group names: bootdg, defaultdg, or nodg. See the section System-Wide Reserved Disk Groups above for details.
2. Use the disk group that is assigned to the system-wide default disk group alias, defaultdg.
3. If the operation does not require a disk group name, perform the operation.
If none of these rules succeeds, the operation fails.
Most of the vxdg reporting commands (such as free or list) behave differently when a disk group is not specified. Rather than report on the default disk group, these commands report on all disk groups. The KEYWORDS section provides details.
In a shared disk group, many vxdg operations can be run from either the Cluster Volume Manager (CVM) master node or a CVM slave node. In most cases, Symantec recommends running the command from the master node, if possible.

KEYWORDS

adddisk Adds the specified disk or disks to a disk group. The disk must not belong to any disk group. The accessname specifies the disk to add. If a medianame is not specified, the medianame is set equal to accessname.
When a disk is added to a disk group, the disk group’s configuration record is copied to the disk (space permitting) and the disk is stamped with the system’s host ID, as defined in the file /etc/vx/volboot.
The -f option must be specified if the disk already has a disk group tag name set.
If the -k option is specified, the disk media name must represent a disk media record that was previously dissociated from its disk access record with -k rmdisk; otherwise, a new disk media record is created to represent the disk. With the -k option, plexes requiring recovery are flagged as stale.
Specifying the -p option with the -k option merges contiguous subdisks into one subdisk and aligns them consecutively on their respective disks.
In a Clustered Volume Manager (CVM) environment, before a disk can be added to the cluster-shared disk group, the disk must be physically accessible from all joined nodes in the cluster. When the disk is added, the disk is stamped with the cluster ID (instead of the host ID) and with the shared flag.
Cross-platform Data Sharing (CDS) is a VxVM feature, which allows disk groups to be moved between different OS platforms.
The -o overridessb option allows disks with inconsistent configuration copies due to a serial split brain condition to be added back to a disk group.
addsite Prepares the specified disk group for site-based allocation. This operation must be performed at each site for which site-based allocation from a disk group is required. If siteconsistent is set to on, the disk group is also configured for consistency. The operation checks if the existing site-based volumes in the disk group meet the necessary allocation requirements, and fails if any of the volumes does not have a data plex at the site that is being added.
If the -f (force) option is specified, the operation does not fail, but instead marks all such volumes as being unable to participate in site-based allocation.
The vxdg addsite operation sets the diskdetpolicy to global, and it sets the dgfailpolicy to leave. These settings are the recommended settings for a campus cluster. These settings are persistent, and do not revert to the default settings even if all sites are removed from the disk group with the rmsite option.
bootdg Displays the current boot disk group aliased by the reserved name, bootdg. The boot disk group contains the volumes that are required to boot a system if its root disk is under VxVM control.
defaultdg Displays the current default disk group.
Note: the command vxdctl defaultdg diskgroup changes the default disk group.
deport Disables access to the specified disk group. A disk group cannot be deported if any volumes in the disk group are currently open (for example, they are mounted as file systems, or are in use by a database). When a disk group is deported, the host ID is cleared (unless a new host ID is specified with -h), so the disk group is not reimported automatically when the system is rebooted.
A disk group can be renamed on deport by specifying a new disk group name with -n newname. A lock can be assigned to an alternate host by specifying the host ID of the alternate host with the -h option. This allows the disk group to be auto-imported when the alternate host reboots. The -n and -h options can be combined to export a disk group to be used as a disk group for a new machine and rename that disk group.
In a CVM cluster, when a cluster-shared disk group is deported, the cluster ID and shared flag stored on all disks in the disk group are cleared, so the disk group is not imported automatically when the cluster starts again.
Trying to deport a shared disk group during a cluster reconfiguration fails.
destroy Removes a disk group from the system. Use this option when a disk group and the information on the disks is no longer needed. This frees up space for use by other disk groups. A disk group cannot be destroyed if any volumes in the disk group are open (for example, they are mounted as file systems, or are in use by a database). vxdg destroy only works on imported disk groups.
The destroy operation clears the disk group name, and makes the disks available for use in other disk groups. However, the disk group ID is not cleared, so the disk group can be imported by specifying this value if the disk group has been accidentally destroyed (assuming that the disks have not already been added to another disk group). If both the destroyed disk group and its clone are available, specifying the disk group ID to the import command imports the destroyed (non-clone) disk group. To import the clone, specify the -o useclonedev=on option.
The -o coordinator option is used to specify that the disk group is no longer to be used as the coordinator for the I/O fencing feature of Veritas Cluster Server (VCS).
Caution: Ensure that the coordinator disks are no longer in use before destroying the coordinator disk group.
detachsite Simulates site failure by detaching all devices in the specified disk group at the specified site.
flush Rewrites all on-disk structures (disk headers, configuration copies, and kernel log copies) managed by VxVM for the specified disk groups. Also, if any configuration copies are disabled (for example, due to I/O failures), this command rewrites those configuration copies, and attempts to enable them. The flush command can be run from the master node or a slave node. In most cases, Symantec recommends that you run the command from the master node.
free Lists free space that is available to allocate subdisks. If a disk group is not specified, the free command lists space from all disk groups. If disks are specified by disk media name, the output is restricted to the indicated disks. A region of free space is identified by disk media name, a physical device tag, an offset relative to the beginning of the public region for the media, and a length.
If you specify the -a option, space on spare disks (which is not really allocatable) is listed in addition to regular free space; otherwise, space on spare disks is not listed. The OPTIONS section defines the -q option.
If you specify the -u option with a unit, the command displays the free space length in the specified units. If you specify h or H for the unit, the command displays the length in the highest possible unit. Valid values for units are:
 
h|H The largest possible unit.
b|B Bytes
k|K Kilobytes
m|M Megabytes
g|G Gigabytes
t|T Terabytes
p|P Petabytes
e|E Exabytes
z|Z Zettabytes
import Imports a disk group to make the specified disk group available on the local machine. This makes accessible any data stored on the disk group, including all configuration information such as disk and volume configurations. The diskgroup argument indicates the disk group to import. The diskgroup can be either a disk group name or a disk group unique ID.
Usually, all disks in the disk must be visible ito the importing host. The -f option forces an import if one or more disks are currently unusable or inaccessible. A disk group can be imported successfully without specifying the -f option if all the disks are accessible that were visible when the disk group was last imported successfully. Using the -f option to force the import of an incomplete disk group counts as a successful import, so subsequent imports do not require the -f option.
Note: Be careful when using the -f option because it can import the same disk group twice from disjointed sets of disks. This can make the disk group inconsistent.
When an ISP disk group is specified, the disk group is imported. However, configuration changes are not allowed until the disk group is upgraded to version 150.
When a host imports a disk group, all disks in the disk group are stamped with the host ID of the importing host. Typically, a disk group cannot be imported if any of its disks are stamped with a non-matching host ID. This provides a sanity check in cases where more than one host can access the same disk.
If it is certain that a disk is not in use by another host (for example, because a disk group was not cleanly deported), use the -C option to clear the existing host ID on all disks in the disk group upon import. A host ID can also be cleared using vxdisk clearimport.
Import fails if the specified disk group name is either a reserved name (such as bootdg or nodg) or an existing disk group name. The disk group can be renamed on import using -n newname. If -n is used with the -t option, the stored name of the disk group remains unchanged, but the importing host knows the disk group under the new name; otherwise, the name change is permanent.
Typically, an imported disk group is reimported automatically when the system is rebooted, if at least some of the disks in the disk group remain accessible and usable. This can be disabled using the -t option, which disables the import at the next reboot.
One example of the use of -n and -t is importing a disk group from one host to a second host, to repair the root volume of the second host. The repaired disk group can then be returned to the first host. To do this, identify the disk group ID for the disk group with vxdisk -s list, and import the disk group using -C to clear import locks, -t for a temporary name, and -n to specify an alternate name (to avoid collision with a similarly named disk group on the second host). After repair, deport the disk group using -h to restore the import lock from the first host.
In a CVM cluster, the -s option imports a disk group as cluster-sharable. This is only valid if the cluster is active on the importing host. Ensure that all the disks in a shared disk group are physically accessible by all hosts. A host which cannot access all the disks in a shared disk group cannot join the cluster.
Disks in a shared disk group are stamped with the ID of the cluster and with the shared flag. When a host joins a cluster, it automatically imports disk groups whose disks are stamped with the cluster ID.
Import of a shared disk group during a cluster reconfiguration fails.
By default, VxVM recovers and starts any disabled volumes in the disk group when you import the disk group. Use the vxdefault command to turn off the automatic recovery feature, For example, after importing the disk group, you may want to do some maintenance before starting the volumes.
If the disks in a deported disk group have recently been scanned by vxconfigd, the -o noreonline option may be specified to avoid having to rescan all the disks to make them online again. This can greatly speed up the importing of a large disk group.
Caution: Only use the noreonline option if you know that a re-online has recently been performed on the disks (for example, by running the vxdctl enable or vxdg import commands), and that the configuration of the disks has not changed since that time.
The -o selectcp option specifies a disk ID, diskid, for the disk that has the preferred configuration copy. This option is used to force the import of a disk group after a serial split brain error condition has been detected. Note that this condition can happen for any disk group; not just for private or shared disk groups in a cluster. To choose a preferred configuration copy, use the vxsplitlines script to examine the IDs of the configuration copies on the disks. See the vxsplitlines(1M) manual page and the Veritas Volume Manager Administrator’s Guide for more information.
The -o groupreserve=fence_key option specifies the I/O fencing key for a disk group. If the -o verify option is also specified, the import of a private disk group fails if the SCSI-3 PR keys have not been registered on all the paths to the disks. This ensures that a host is not fenced off while importing the disk group, and that two nodes cannot attempt to fence the disk group at the same time.
When importing a private disk group on which I/O fencing is enabled, the -o groupreserve option must be used to specify the I/O fencing key explicitly. The specified I/O fencing key, fence_key, must be no longer than 7 bytes.
When importing a shared disk group on which I/O fencing is enabled, there is no need to specify the key explicitly provided that the appropriate I/O fencing licenses have been installed, and that cluster-wide fencing has been enabled. The I/O fencing key is automatically created on import.
Use the -o clearreserve option to clear pre-existing reservations on the disks that comprise the disk group.
See the description of vxdg init for more information about I/O fencing.
If the udid_mismatch flag is set on a disk to indicate that the UDID does not correspond to that disk, and a disk with the same UDID already exists in the disk group, the import of the disk fails, to avoid a duplicate disk ID condition.
If udid_mismatch is set, but no disk with the same UDID already exists in the disk group, the disk can be imported but the udid_mismatch flag remains set.
If the -o tag option is specified, only those disks that are tagged with a matching tag name are imported into the disk group. This can result in the partial import of the disk group.
If the -o useclonedev=on option is specified, the import operation only imports devices that have the clone_disk or udid_mismatch flag set into a disk group. This allows the import of cloned disks; for example, disks that have been created as hardware mirrors or snapshots of existing disks in the disk group. By default, the useclonedev option is set to off, which disallows the import of cloned disks. However, if a disk group contains only cloned disks, it is not necessary to specify the -o useclonedev=on option.
Note: The feature for importing clone disks requires that the disk group has been upgraded to version 140 or later.
Non-cloned and cloned disks cannot be imported in the same operation. If the disk group is already imported, use the -n option to specify a new disk group name for the clone disk group during the import operation.
If more than one cloned disk that are to be imported have the same unique disk identifier (UDID), the command fails unless the -o tag option is used to specify which disk is to be imported. (The vxdisk settag command can be used to assign a tag name and tag value to individual disks.) This may result in only a subset of the disks associated with the disk group being imported.
If clone disks have ambiguous UDID values, the -o updateid option can be used to generate new identification values for the disks that are being imported:
o Disk identifier.
o Unique disk identifier (UDID).
o Disk group identifier (DGID).
o Global unique identifier (GUID).
o Base minor number.
 
The -o site option imports only devices from the specified site; devices from other sites are treated as detached. This can be used to simulate site failure during import. This can be combined with the -o tag option to import a subset of suitably tagged disks.
init Defines a new disk group composed of the indicated disks, identified by disk access names. VxVM assigns an internal unique ID to the group, stores a pointer to that group in the root configurates storing a reference to the group on all of the named disks that have a disk header, and stores a disk group record in the disk group’s configuration database. At least one of the disks specified must have space allocated for a configuration copy.
An existing deported disk group may be imported under a different name if it has the same name as that specified for the new disk group.
The init operation fails if another disk group, deported or otherwise, is using any of the specified disks. If vxdg finds an unneeded disk group on the disk, it can be cleaned with the vxdisk -f init command. vxdg init can then be run again.
If a medianame is specified for use with a particular disk, then that medianame names the disk media record used to reference the disk within the disk group (for operations such as rmdisk and subdisk creations). If no medianame is specified, the disk media name defaults to accessname. See vxdisk(1M) for a discussion of definition and initialization of disk access records.
The init operation has the following options:
cds={off|on}
  Specifies whether the disk group being created is compatible with the Cross-platform Data Sharing (CDS) feature. If not specified on the command line, the /etc/default/vxdg file is checked for a value of this attribute. If no value is found, the value cds=on is assumed.
coordinator={off|on}
  Specifies whether a disk group is the coordinator for the I/O fencing feature of Veritas Cluster Server (VCS). The coordinator disk group must contain exactly 3 disks, and disks cannot be added or removed while the coordinator attribute is set to on.
Caution: Ensure that the coordinator disks are no longer in use before setting the value of this attribute to off.
minor=base-minor
  Specifies a base volume device minor number for a disk group. Volume device minor numbers for a disk group start at this value. Minor numbers can range up to 16,777,216, so if it is presumed that no more than 1000 volumes would ever be created in any one disk group, 16,777 different ranges of minor numbers are available for different disk groups.
Note: The minor number range for disk groups that support the Cross-platform Data Sharing (CDS) feature is restricted to 65535 for cross-platform compatibility.
A reasonably sized range should be left at the end for temporary device number remappings (in the event that two device numbers still conflict).
If no minor operand is specified, then VxVM chooses a random number of at least 1000 that is a multiple of 1000, and yields a usable range of 1000 device numbers. This default value is chosen so as to not overlap within a range of 1000 of any currently imported disk groups, and so as to not overlap with any currently allocated volume device numbers.
Since disk groups can be moved between systems, volume device numbers should be allocated in separate ranges for each disk group. That way, you can choose ranges such that all disk groups in a group of machines can be moved around without causing device number collisions. Collisions may occur because VxVM stores device numbers in disk group configurations. This allows device numbers to persist across reboots, which NFS requires. If two systems use the same device numbers for a set of volumes, and if a disk group from one machine is moved to the other, VxVM may be forced to temporarily remap some devices.
Note: The default policy is likely to ensure that a small number of disk groups can be merged successfully between a set of machines. However, in cases where disk groups are merged automatically using fail-over mechanisms, the administrator should select ranges that are known to avoid overlap.
Trying to create a shared disk group during a cluster reconfiguration fails.
Note: Volumes in (CVM) shared disk groups must have the same minor number on all nodes in the cluster. If there is a conflict when a node attempts to join the cluster, the join fails. In that case, the administrator should use the reminor operation on the joined node(s) to resolve the conflict. In a multi-node cluster use a base minor number which does not conflict with those of any of the cluster nodes.
nconfig=config-copies
nlog=log-copies
  Specifies the number of configuration database copies and kernel log copies respectively that are maintained for a disk group.
The config-copies and log-copies values can be a decimal number (including 0 or -1), all or default. A value of all or -1 signifies that all configuration or log copies on all disks in the disk group are to be maintained. A value of default or 0 (synonums) signifies that VxVM is to manage copies that are distributed in a reasonable pattern throughout the disks, controllers and enclosures on the system. Any other number signifies that a particular number of copies should be maintained (or all copies, if that number is larger than the number of available configuration or log copies on all disks).
When a specific number (or default) is requested, configuration copies are distributed across the enclosures on the system. The number of copies in each enclosure is proportional to the number of disks in that enclosure. With the default policy, at least one configuration and log copy is maintained for each enclosure. It is ensured that at least one configuration and log copy is maintained for each host controller connected to an enclosure. If this does not result in allocating at least 4 copies, additional copies are spread uniformly across enclosures.
Refer to vxdisk(1M) for more information on configuration and log copies, and for information on how to create them.
Note: If a policy other than all is used, some disks do not have up-to-date, online configuration and log copies. As a result, it is possible that some number of disk failures can leave a disk group unusable, even if some disks in the disk group remain usable. The default policy allocates a sufficient number of copies, in a sufficient spread of locations, that such a scenario is very unlikely to occur.
The -o groupreserve=fence_key option specifies the I/O fencing key for a disk group. The specified I/O fencing key, fence_key, must be no longer than 7 bytes.
To create the 8-byte SCSI-3 Persistent Reservation key, vxdg adds the character ’A’ to the node ID (0 to number of nodes minus 1) of the host within the cluster to form the first byte, and uses the I/O fencing key for the last 7 bytes (padding with NULL characters as appropriate).
Note: The vxfen driver provides the node ID. This driver must be installed and configured to use I/O fencing.
See the description of vxdg import for more information about I/O fencing.
The -T option specifies the disk group version. If no version is specified, the disk group is initialized with the highest (most current) version supported by the release of VxVM currently running on the system. Specifying a lower version limits the operations and features that the disk group supports. The -T option can be used to make the disk group compatible with previous releases of VxVM, which can be useful if the disk group or copies of the disk group are shared across multiple servers for example, with off-host backup. See the vxdg upgrade operation for more information.
In a CVM cluster, the -s option creates a new cluster-sharable disk group. The user must ensure that disks specified as members of a cluster-sharable disk group are physically accessible from all of the hosts that comprise the cluster.
The disks in a shared disk group are stamped with the cluster ID and with the shared flag. When a host joins a cluster, it automatically imports shared disk groups whose disks are stamped with the cluster ID.
join Moves all objects from the imported source disk group, sourcedg, to the imported target disk group, targetdg. After the move completes, sourcedg is removed.
The source disk group and target disk group to be joined must be either both private or both shared. If one disk group is private and the other is shared, deport the private disk group and reimport it as shared before performing the join.
By default, VxVM recovers and starts any disabled volumes in the target disk group after the join. Use the vxdefault command to turn off the automatic recovery feature.
The join transaction may fail and retry will be done during high I/O load because of the device open/close assertion. The option -o transretry can be used to specify the transaction retry limits. If it is not specified, the retry is unlimited.
list Lists the contents of disk groups. If no diskgroup is specified, all disk groups are listed in a one-line format. If diskgroups are specified, a longer format provides additional status and configuration data.
The OPTIONS section describes the -q option.
In a CVM cluster, if the -s option is specified, all cluster-shared disk groups are listed in a one-line format. If diskgroups are specified, -s has no effect.
The -o listreserve option displays whether the disk group has I/O fencing enabled.
Private disk group that have been fenced off display fencedoff in the flags field. To clear this flag, reboot the host, or deport and then reimport the disk group.
listclone Displays all LUNs with clone_disk or udid_mismatch flags. A LUN with clone_disk and udid_mismatch is a copy of another LUN. For example, an EMC BCV(tm) would appear as udid_mismatch or clone_disk, and the BCV’s corresponding STD would have no flag. VxVM sets udid_mismatch automatically. You may set the clone_disk flag using vxdisk set or vxdisk clear, or VxVM sets clone_disk during vxdg -o useclonedev=on import. Omitting the -g option shows all imported disk groups.
The OPTIONS section describes the -q option.
listmeta Displays the tagmeta values for the configuration database copies and kernel log copies. If no disk group is specified, the information is shown for all imported disk groups.
The OPTIONS section describes the -q option.
listmove Displays all objects, including all objects in hierarchies, that would move from the imported source disk group, sourcedg, to the imported target disk group, targetdg, as implied by the specified object list. The items in the specified object list must be top-level objects, disk media objects or disk access objects.
This command confirms the validity and object content of a proposed move without actually moving any objects.
listssbinfo
  Displays information about conflicting configuration copies. If you import different portions of a disk group on different systems, the on-disk configuration copies on different disks may conflict. In that situation, VxVM may not be able to determine which configuration copy is most current. This condition is termed a serial split brain (SSB) condition. You must specify which disk’s configuration copy to use. in order to import a disk group in this state.
The vxdg listssbinfo command lists which disks have conflicting configuration copies. The vxdg listssbinfo command displays information about two pools of disks. Disks in the first pool have the same configuration copies while disks in the second pool may not have the same configuration copies.
The output from vxdg listssbinfo displays the vxdg commands to run to import the disk group using the available configuration copies. The output suggests a remedial import option for each pool. Choose one of the options to resolve the serial split brain situation. Consider the information that is displayed, such as the number of disks in each pool, when you decide.
The -v verbose option displays additional disk information in each pool which consists of the disk media name, disk access name, the disk private path and the diskid. It also displays the commands to run to see the configuration copy from each disk using the disk private path.
The vxdg listssbinfo command only operates on deported disk groups. Specifying an imported disk group returns an error.
listtag Displays the tag names and tag values that are associated with the specified disk groups, or with all disk groups if no disk group is specified.
The OPTIONS section describes the -q option.
migrateispobjs
  Use this option to migrate ISP disk groups to non-ISP disk groups if upgrading the disk group to version 150 fails to convert the disk groups. Only run this option on a disk group with version 150 that was not converted to non-ISP.
move Moves the specified objects together with their hierarchies from the imported source disk group, sourcedg, to the imported target disk group, targetdg.
The items in the object list must be top-level objects, disk media objects or disk access objects. The list must define a set of self-contained objects, unless the -o expand option is specified. (Self-contained means that the disks used by the selected objects should not contain any objects that are not selected for the move.) If the -o expand option is specified, the object set expands to be self-contained.
The source disk group and target disk group must be either both private or both shared. If one disk group is private and the other is shared, deport the private disk group and reimport it as shared before performing the move.
By default, VxVM recovers and starts any disabled volumes in the target disk group after the move. Use the vxdefault command to turn off the automatic recovery feature.
The move transaction may fail and retry will be done during high I/O load because of the device open/close assertion. The option -o transretry can be used to specify the transaction retry limits. If it is not specified, the retry is unlimited.
nohotuse Lists free space that cannot be used by hot-relocation to replace failed subdisks, in the specified diskgroup, or with all disk groups if the -g option is not specified, If disks are specified by medianame, the output is restricted to the indicated disks. A region of nohotuse space is identified by disk medianame, a physical device tag, an offset relative to the beginning of the public region for the disk, and a length.
The physical device tag is a reference that indicates on which physical device the disk media is defined. It appears as a truncated disk access name.
The OPTIONS section describes the -q option.
reattachsite
  Reattaches the devices in the specified disk group at the specified site. The -o overridessb option may be used to allow disks with inconsistent configuration copies due to a serial split brain condition to be added back to a disk group. Appropriate recovery procedures are initiated to resynchronize the data at the site.
Use the -o clearreserve option to clear pre-existing reservations on the disks from the site that is being reattached.
If you are certain that the disks from the site being reattached are not in use by another host, use the -C option to clear the existing host ID on all disks from the site upon reattach.
recover Attempts to manually recover an incomplete move, split or join operation using either of the disk groups that was involved in the operation.
If the recovery operation cannot complete, the -o clean option clears the MOVE flags from the objects in the disk group.
The -o remove option removes all objects marked with the MOVE flag from the disk group.
reminor Changes the base minor number for a disk group, and renumbers all devices in the disk group to a range starting at that number. If the device for a volume is open, the old device number remains in effect until the system is rebooted or until the disk group is deported and reimported. To cause renumbering to take effect on a volume without rebooting or reimporting, close the volume, then execute vxdg reminor again.
A new device number may also overlap with a temporary renumbering for a volume device. This also requires a reboot or a reimport for the new device numbering to take effect. A temporary renumbering can happen in the following situations: when two volumes in two different disk groups share the same permanently assigned device number, in which case one of the volumes is renumbered temporarily to use an alternate device number; or when the persistent device number for a volume was changed, but the active device number could not be changed to match. The active number may be left unchanged after a persistent device number change either because the volume device was open, or because another volume was using that active number.
vxdg reminor fails if you try to use a range of numbers that is currently in use as a persistent (not a temporary) device number. To force use of the number range, use the -f option. With -f, some device renumberings may not take effect until a reboot or a reimport. If you force volumes in two disk groups to use the same device number, one volume will be temporarily renumbered on the next reboot. The volume to be renumbered is randomly selected, unless the system is booted from a root disk that is under VxVM control. In this case, the device numbers for the boot disk group take precedence.
Only use the -f option to swap the device number ranges used by two or more disk groups. To swap the number ranges for two disk groups, use the -f option when renumbering the first disk group to use the range of the second disk group. Renumbering the second disk group to the first range does not require the -f option.
renamesite Renames the sites that are configured in the specified disk group. The site oldsite needs to be present in the disk group for the rename of site from oldsite to newsite to succeed. The operation makes the existing site-based information of the disk group corresponding to the site oldsite valid for the site newsite, by performing the following tasks:
- Renames the site record in the disk group.
- Modifies auto-tagging information.
- Renames the site tag on the disks.

The rename operation checks if the site newsite conflicts with an existing site, and fails if the site newsite is exists. If the -f (force) option is used, the operation does not fail, instead the existing site is renamed to the site newsite. In this case the auto-tagging information is modified and the site tag on the disks is renamed, but the site record is not renamed. Instead, the site record for oldsite is just deleted. The site information is changed for all hosts, objects and devices at a site. The site information cannot be changed for a disk
repldisk Dissociates the DA record from the DM record named by spare-medianame, and reassociates it with the unassociated DM record named by unassoc-medianame. Both unassoc-medianame and spare-medianame must be members of the disk group named by the diskgroup argument. However, if the -k option is specified, the disk media records for the spare-medianame are retained, although in a removed state.
rmdisk Removes the specified disk or disks from a disk group. It is not possible to remove the last disk containing a valid disk group configuration or log copy from its disk group.
Typically, the rmdisk operation fails if subdisk records point to the named disk media records. However, if the -k option is specified, the disk media records are kept, although in a removed state, and the subdisk records still point to them. The subdisks, and any plexes that refer to them, remain unusable until the disk is re-added using the -k option to the adddisk operation. VxVM disables volumes when all plexes become unusable.
rmsite Removes the site consistency requirement from the specified disk group that was previously configured for site-based allocation. Volume layout and composition is unchanged; all plexes remain and all continue to be updated. If the site is detached or offline, the rmsite operation fails unless the -f (force) option is specified.
rmtag Removes the specified tag names from all of the disks in a disk group. If an enclosure name is associated with a tag, you must specify the enclosure name to the vxdg rmtag command to remove the tag.
set Changes disk group characteristics. Specify changes by entering arguments after the set keyword in the form attribute=value.
The following attributes may be set:
activation=mode
  The activation mode of a disk group determines whether applications are permitted to read and write to volumes in the disk group.
For private disk groups, the valid activation modes are as follows:
off Volumes in the disk group are not available for read or write access.
readonly | ro
  Volumes in the disk group are available for read access only.
readwrite | rw
  Volumes in the disk group are available for read and write access.
For shared disk groups, the activation mode is set per node. The following are the valid activation modes for shared disk groups:
exclusivewrite | ew
  The node has exclusive write access to volumes in the disk group. No other node in the cluster can activate the disk group for write access.
off Volumes in the disk group are not available for read or write access.
readonly | ro
  The node has read access to volumes in the disk group. It has no write access and denies write access to all other nodes in the cluster.
sharedread | sr
  The node has read access to volumes in the disk group, but no write access, However, other nodes can activate the disk group for write access.
sharedwrite | sw
  The node has write access to volumes in the disk group. Other nodes can activate the disk group for shared write access.
Note: to change the activation mode of a shared disk group, first change the activation mode to off before setting it to any other value.
The related attributes-value pairs, enable_activation=true and default_activation_mode=mode, can be used to enable and define a default activation mode for shared disk groups. These attributes can only be set by adding their definitions to the /etc/default/vxdg defaults file.
align={1|8k}
alignment={1|8k}
  Specifies the alignment value for the disk group. Possible values are 1 (one block) or 8k (8KB). For a disk group to use the Cross-platform Data Sharing (CDS) feature, the alignment value must be 8k. The default is 8k.
autotagging={off|on}
  Configures site autotagging for the disk group. When autotagging is set to on, new LUNs that inherit the site tag of their enclosure. when they are added to a disk group. Use the vxdg settag command to assign a site tag to an enclosure. The default is on.
cds={off|on}
  Specifies whether a disk group can use the Cross-platform Data Sharing feature. The default is on.
coordinator={off|on}
  Specifies whether a disk group is the coordinator for the I/O fencing feature of Veritas Cluster Server (VCS). The coordinator disk group must contain exactly 3 disks, and the addition or removal of disks is not permitted while this attribute is set to on. The default is off. If a disk fails, the attribute may temporarily be set to off to allow the number of disks to be restored to three.
Caution: Ensure that the coordinator disks are no longer in use before setting the value of this attribute to off.
dgfailpolicy=policy
  Sets VxVM behavior if the master node cannot update the configuration and log copies in a shared disk group, including upon first write and final close. The dgfailpolicy policy takes precedence over the diskdetpolicy policy. This policy is ignored for private disk groups, and requires disk group version 120 or greater. For site-aware disk groups in a campus cluster,
the dgfailpolicy is set to the leave policy when a disk group is added to a site and made site consistent. You cannot change the dgfailpolicy for site-aware disk groups, unless you specify the -f (force) option for the vxdg set command. The following policies may be set:
dgdisable
  The master node disables the disk group,
This is the default policy.
leave The master node panics. If no node can access the log or configuration copies, all nodes panic in turn as they become the master node.
For site-aware disk groups in a campus cluster, leave is the
  recommended policy.
requestleave
  The master node exits cleanly from the cluster.
diskdetpolicy=policy
  Sets the VxVM behavior when a disk in a shared disk group becomes unusable. This policy is ignored for private disk groups. If the last disk in a shared disk group become unusable, the dgfailpolicy policy takes precedence over the diskdetpolicy policy. For site-aware disk groups in a campus cluster,
the diskdetpolicy is set to the global policy when a disk group is added to a site and made site consistent. You cannot change the diskdetpolicy for site-aware disk groups, unless you specify the -f (force) option for the vxdg set command. The following policies may be set:
global
  If any node in the cluster reports an unusable disk, all nodes detach that disk. An I/O error results if the disk was in the final active plex of a volume.
This is the default policy. It is also the recommended policy for site-aware disk groups in a campus cluster.
local Only those nodes reporting an unusable disk detach that disk. If all nodes report a problem with the disk, all nodes detach that disk. An I/O error results if the disk was in the final active plex of a volume.
maxdev=number
  Specifies the maximum number of VxVM devices that are permitted in a disk group. This value must be a positive integer that is greater than the number of VxVM devices currently in the disk group.
nconfig=config-copies
nlog=log-copies
  Specifies the number of configuration database copies and kernel log copies respectively that are maintained for a disk group.
For more information, see the descriptions under the init operation.
siteconsistent={on|off}
  on enforces site consistency for a disk group.
The vxdg siteconsistent=on operation sets the diskdetpolicy to global, and it sets the dgfailpolicy to leave. These settings are the recommended settings for a campus cluster. These settings are persistent, and do not revert to the default settings even if all sites are removed from the disk group with the rmsite option.
off removes the site consistency enforcement for a disk group.
tagmeta={on|off} tag=name
  Enables (on) or disables (off) the configuration copies and kernel log copies for all disks that are tagged with the specified tag name. This ensures that a suitable number of configuration copies and kernel log copies exist on a set of tagged disks in agreement with any policies that have been set for the nconfig and nlog values. The default is off.
The cds, default_activation_mode and enable_activation attribute values may be defined in the /etc/default/vxdg defaults file. The values defined in this file override the built-in default values, and persist across system reboots. The values defined in the defaults file may themselves be overridden by values specified on the command line.
settag Sets or updates the tag names and optional tag values for all disks in a disk group. The tag name and tag value are strings of up to 128 characters. The string must not include space or tab characters.
To update a previously set tag name, use the -f option.
Note: The tag names site, udid and vdid have specific meanings. The udid and vdid tags are reserved by VxVM and cannot be used.
Use the tag name site to identify to which site in a Remote Mirror configuration the specified disk or enclosure belongs. To specify an enclosure, use encl:encl_name. If the autotagging flag is set to on for the disk groups, disks that are added to the enclosure inherit the site tag of the enclosure.
spare Lists spare space that is available to relocate subdisks during recovery. If no disk group is specified, spare space from all disk groups is listed. If disks are specified by disk media name, the output is limited to the indicated disks. Spare space regions are identified by a disk media name, a physical device tag, an offset relative to the start of the public region for the media, and a length.
The physical device tag indicates on which physical device the disk media is defined on. It appears as a truncated disk access name.
The OPTIONS section defines the -q option.
split Splits a disk group into two disk groups, sourcedg and targetdg. The specified objects move from the imported source disk group, sourcedg, to a newly created target disk group, targetdg. All other objects remain in sourcedg.
Objects specified must be top-level objects or disks. Disk may be specified by disk media name or disk access name. All disks must end up in the source disk group or in the target disk group; a disk cannot span disk groups. Likewise, subdisks, plexes, volumes and volume sets must end up in the same disk group as the disks from which they are built. The list of objects specified must result in all objects sharing a given disk being moved together, otherwise the split operation fails. If the -o expand option is specified, VxVM automatically adds objects to the list so that all objects sharing a given disk move together.
As an example, divide a disk into two subdisks. Create two volumes, one from each subdisk. If you specify one volume but not the other, the split operation fails. If you use the -o expand option, VxVM moves both volumes, both subdisks, and the simple underlying disk to the target disk group.
The split operation fails if it would remove all the disks from the source disk group, or if an imported disk group exists with the same name as the target disk group.
The target disk group is imported as shared if the source disk group is shared; otherwise, it is imported as private. See vxdg import for a description of the -C, -f, and -t options.
See vxdg join for a description of the usage of the -o override and -o verify options.
By default, VxVM recovers and starts any disabled volumes in the target disk group after the split. Use the vxdefault command to turn off the automatic recovery feature.
The split transaction may fail and retry will be done during high I/O load because of the device open/close assertion. The option -o transretry can be used to specify the transaction retry limits. If it is not specified, the retry is unlimited.
upgrade Upgrades the disk group to a newer version. The default is the highest version supported by the running version of VxVM. The -T option upgrades the disk group to a specified version. The following section lists each disk group version, the features it supports, and the VxVM release that introduced it. Note: Some VxVM versions are not supported on all OS platforms.
An ISP disk group must be converted to version 150 to allow any configuration changes to the disk group. If the disk group upgrade succeeds but the ISP disk groups are not converted to non-ISP disk groups, convert the ISP disk groups with the vxdg migrateispobjs command.
10 Supports only the most basic volume management features of mirroring and simple striping. Introduced in VxVM 1.2. Starting with VxVM 3.0, disk groups of version 10 can be imported, but no operations or I/O can be performed on the objects it contains. The only operation supported is to upgrade the disk group to a later release.
All disk group versions greater than 10 are supported by all subsequent releases.
20 RAID-5 Volumes, new-style stripes, recovery checkpointing, disk group configuration/klog copy limiting, and Dirty Region Logging. Introduced in VxVM 2.0.
30 Oracle Resilvering Interface. Introduced in VxVM 2.2.
40 Hot Relocation. Introduced in VxVM 2.3.
60 Online Relayout, safe RAID-5 subdisk moves, Striped Mirrors, and RAID-5 Snapshots. Introduced in VxVM 3.0.
70 Non-Persistent FastResync, Veritas Volume Replicator (VVR) enhancements, and Unrelocate. Introduced in VxVM 3.1.
80 Veritas Volume Replicator (VVR) Enhancements. Introduced in VxVM 3.1.1.
90 CVM support for Oracle Resilvering, disk group move, split and join, Device Discovery Layer (DDL), layered volume support in CVM, ordered allocation, OS independent Naming support, and Persistent FastResync. Introduced in VxVM 3.2.
110 Cross-platform Data Sharing (CDS), Device Discovery Layer (DDL) 2.0, disk group configuration backup and restore, elimination of rootdg as a special disk group, full-sized and space-optimized instant snapshots, Veritas Intelligent Storage Provisioning (ISP), volume sets (for Veritas File System multi-volume file systems.) Introduced in VxVM 4.0.
120 Automatic cluster-wide failback for A/P arrays, DMP co-existence with third-party drivers, persistent DMP policies, shared disk group failure policy, support for EFI disks. Introduced in VxVM 4.1.
130 Veritas Volume Replicator (VVR) Enhancements. Introduced in VxVM 5.0.
140 Data migration, Remote Mirror, coordinator disk groups (used by VCS), linked volumes, snapshot LUN import. Introduced in VxVM 5.0.
150 Enhanced SSD device support, migration of ISP disk groups. Introduced in VxVM 5.1.
160 Automated bunker replay as part of GCO failover, Ability to elect primary during GCO takeover, CVM support for more than 32 nodes and up to 64 nodes, CDS layout support for large luns (> 1 TB) and EFI disks, vxrootadm enhancements. Introduced in VxVM 5.1 Service Pack 1.
170 VVR compression, VVR Secondary logging, CVM availability enhancements, DCO version 30, recovery for synchronization tasks. Introduced in VxVM 6.0.
To determine the version of a disk group, use the vxdg list diskgroup command.

OPTIONS

The following options change the behavior of vxdg:
-g Specifies the disk group name. The space between -g and diskgroup is optional. Only one disk group may be specified. For commands that show -g as optional in the SYNOPSIS section, the operation may still require a disk group. If the disk group is not specified as command input, the operation uses the default disk group. In contrast, if the SYNOPSIS section shows that -g is mandatory, the command input must include the disk group. The default disk group is not sufficient.
-q Omit printing the header of the output fields. The output is more machine readable as a program or script need not parse and discard the header text.

I/O Fencing Options

In clusters using Veritas Cluster Server (VCS) as the cluster monitor, the I/O fencing feature uses the underlying hardware disk reservation mechanism to avoid the split brain problem.

The reservation mechanism is SCSI-3 Persistent Reservation (PR). to enable I/O fencing, set the value of the scsi3_pr attribute to on in the volboot file.

See the description of the vxdg import, init and list commands for information about the clearreserve, coordinator, groupreserve and listreserve options that are used with I/O fencing.

EXAMPLES

To import the disk group, mydg:

vxdg import mydg

To display the free space in the disk group, mydg:


vxdg -g mydg free

To deport the disk group, mydg:


vxdg deport mydg

To import the disk group, mydg, and rename it as newdg:


vxdg -n newdg import mydg

To move volumes, vol1 and vol2, from the disk group, olddg, to newdg:


vxdg move olddg newdg vol1 vol2

To split volumes, vol3 and vol4, from the disk group, olddg, to form a new disk group, mynewdg:


vxdg split olddg mynewdg vol3 vol4

To merge the contents of the disk groups, olddg with the disk group, testdg:


vxdg join olddg testdg

To import a snapshot LUN as a new disk group, snapdg. The original LUN is in disk group mydg:


vxdg -n snapdg -o useclonedev=on -o updateid import mydg

To display information about the configuration copies for a disk group, mydg, with the serial split brain condition:


vxdg -g mydg listssbinfo

This command displays output such as the following:


VxVM vxdg listssbinfo NOTICE V-0-0-0 There are 2 pools
All the disks in the first pool have the same config copies
All the disks in the second pool may not have the same config copies
Number of disks in the first pool: 1
Number of disks in the second pool: 1
To import the diskgroup with config copy from the first pool
issue the command
 /usr/sbin/vxdg (-s) -o selectcp=1215378871.300.vm2850lx13 import mydg
To import the diskgroup with config copy from the second pool
issue the command
 /usr/sbin/vxdg (-s) -o selectcp=1215378869.294.vm2850lx13 import mydg


To display verbose information about the configuration copies for the disks in each pool:


vxdg -v -g mydg listssbinfo


This command displays output such as the following:


VxVM. vxdg listssbinfo NOTICE V-0-0-0 There are 2 pools
All the disks in the first pool have the same config copies
All the disks in the second pool may not have the same config copies
To see the configuration copy from a disk issue the command
/etc/vx/diag.d/vxprivutil dumpconfig <private path>
To import the diskgroup with config copy from a disk
issue the command
/usr/sbin/vxdg (-s) -o selectcp=<diskid> import mydg
                             Pool 0
DEVICE    DISK      DISK ID                     DISK PRIVATE PATH
mydg1     sdp       1215378871.300.vm2850lx13   /dev/vx/rdmp/sdp5


Pool 1 DEVICE DISK DISK ID DISK PRIVATE PATH mydg2 sdo 1215378869.294.vm2850lx13 /dev/vx/rdmp/sdo5

FILES

/etc/vx/volboot File containing VxVM configuration information.
/etc/default/vxdg Default attribute definitions file for the vxdg command.

SEE ALSO

vxcdsconvert(1M), vxconfigd(1M), vxdctl(1M), vxdefault(1M), vxdisk(1M), vxintro(1M), vxplex(1M), vxprint(1M), vxvol(1M)


VxVM 5.1 vxdg (1M)