vxfs_dedup - Share blocks that contain identical data

#include <sys/types.h>

#include <vxfsutil.h>

int vxfs_dedup(int mfd, struct vx_dedup *dedup);

VRTSvxfs

This function’s use is restricted to Veritas-approved internal applications.

The vxfs_dedup() call eliminates redundant data by sharing blocks of identical data, also referred to as deduplication. This function is supported on Version 8 and later disk layouts.

To use this function, specify -l vxfsutil while linking.

The following code is an example call:

vxfs_dedup(mfd, &dedup);

The mfd argument is an open file descriptor of the mount point in which the files to dedup reside. The dedup argument is a pointer to a vx_dedup structure that contains the parameters needed to specify and perform the dedup.

struct vx_dedup {
    uint32_t                  ddup_version;
    uint32_t                  ddup_pad1;
    uint64_t                  ddup_reqlen;
    struct vx_dedup_hint      ddup_hint;
    struct vx_dedup_ireq      ddup_pri;
    struct vx_dedup_ireq     *ddup_sec;
    uint32_t                  ddup_nsec;
    uint32_t                  ddup_pad2[5];
};
struct vx_dedup_ireq {
    uint64_t      dupr_ino;
    uint64_t      dupr_off;
    char         *dupr_fsetname;
    uint64_t      dupr_retlen;
    uint32_t      dupr_syserr;
    uint32_t      dupr_duperr;
};
struct vx_dedup_hint {
    uint32_t      duph_flag;
    uint32_t      duph_match;
    uint32_t      duph_readsz;
    uint32_t      duph_pad[3];
};

The vx_dedup structure has the following fields:

ddup_version	For internal use only.
ddup_pad1	Unused.
ddup_reqlen	The requested amount of data to dedup.
ddup_hint	A struct vx_dedup_hint that provides additional details about the requested dedup.
ddup_pri	A struct vx_dedup_ireq that contains information about the primary dedup file. All files referenced in ddup_sec will be deduped against this primary file.
*ddup_sec	A pointer to an array of struct vx_dedup_ireq entries that contain information about the secondary files that are to be deduped against the primary file.
ddup_nsec	The number of entries in the array pointed to by ddup_sec.
ddup_pad2[5]	Unused.

The vx_dedup_ireq structure has the following fields:

dupr_ino	The inode number of the file to dedup. The primary and secondary inode numbers can refer to different files or the same file.
dupr_off	The offset into the file where the data to dedup begins. The offset must be block aligned.
*dupr_fsetname	A pointer to the fileset name of where the file resides. The primary and secondary files must reside in the same fileset. If a NULL or empty ("") string is passed, the system assumes the files reside in the primary fileset.
dupr_retlen	A return value. See RETURN VALUES.
dupr_syserr	A return value. See RETURN VALUES.
dupr_duperr	A return value. See RETURN VALUES.

The vx_dedup_hint structure has the following fields:

duph_flag	The suggested type of read I/O in which to perform the dedup for both the primary and secondary files. If no flag is specified for either the primary or secondary file, or both, dedup uses the default values of VX_DUPHFLAG_PRIDIO and VX_DUPHFLAG_SECDIO, accordingly. The flags and types of I/O are listed below.
VX_DUPHFLAG_PRIDIO
	Read the primary file as direct I/O. However, if there are pages in the cache, dedup reads from the cache instead. Mutually exclusive with VX_DUPHFLAG_PRICACHE. This is the default flag and action if duph_flag is unspecified for the primary.
VX_DUPHFLAG_PRICACHE
	Read the primary file from the cache. Mutually exclusive with VX_DUPHFLAG_PRIDIO.
VX_DUPHFLAG_SECDIO
	Read the secondary file as direct I/O. However, if there are pages in the cache dedup will read from the cache instead. Mutually exclusive with VX_DUPHFLAG_SECCACHE. This is the default flag and action if duph_flag is unspecified for the secondary.
VX_DUPHFLAG_SECCACHE
	Read the secondary file from the cache. Mutually exclusive with VX_DUPHFLAG_SECDIO.
duph_match	Unused.
duph_readsz	The suggested read size for each I/O when performing the dedup. The read size must be block aligned and less than or equal to VX_MAXBUFSIZE (64K), which is the default value. If a duph_readsz value of 0 is passed, dedup uses the default value. If too small of a value is passed, dedup rounds up to the smallest allowed size.
duph_pad[3]	Unused.

How the vxfs_dedup() ioctl processes the dedup requests, and how well the ioctl performs, is partly based upon a number of global variables. Currently, these variables are only modifiable through a kernel debugger, and should only be changed with Veritas approval.

vx_dedup_maxext	The maximum number of extents processed for each request. The default is 10.
vx_dedup_maxlen	The maximum amount of data processed or read for each request. The default is 1M.
vx_dedup_activeasync
	The number of active asynchronous I/Os allowed when doing direct I/O. The default is 16.

For the secondary files, asynchronous direct I/O is always done, unless there are pages in the cache.

vxfs_dedup() returns zero on success or partial success, and non-zero on failure.

dupr_retlen	A return value of the actual amount of secondary file data that was deduped, which may be less or more than the requested amount. vxfs_dedup() attempts to dedup as much data as it can beyond the initial request. The maximum amount of data that can be deduped in a single vxfs_dedup() call is limited by the size of the extents and by the variable vx_dedup_maxlen, which currently has a value of 1 MB. The dupr_retlen value is set in each dedup request entry of the ddup_sec array of secondary files. It is not set for the ddup_pri primary file.
dupr_syserr	A return value that contains any system-specific error encountered when processing the informaton in the ddup_pri and each ddup_sec array entry.
dupr_duperr	A return value that contains any dedup-specific error encountered when processing the informaton in the ddup_pri and each ddup_sec array entry.

Any error encountered when processing the ddup_pri primary file information is considered a fatal error. If such an error is encountered all ddup_sec requests are ignored and the vxfs_dedup() ioctl returns an error.

Any error encountered when processing the ddup_sec secondary file information only impacts that specific ddup_sec entry and request. It does not impact other ddup_sec requests or the vxfs_dedup() return value.

Any error encountered when processing the ddup_hint structure entries is considered a fatal error. If such an error is encountered, all ddup_sec requests are ignored and the vxfs_dedup() ioctl returns the appropriate dedup error.

The following errors are specific to dedup. Primary pertains to the data in the ddup_pri field. Secondary pertains to the data in an entry in the ddup_sec array.

VX_DEDUP_EPRIINODE
	The primary inode number is invalid or not found.
VX_DEDUP_EPRIOFF
	The primary file offset is not block aligned, exceeds file size, or produces file overlap.
VX_DEDUP_EPRIMMAP
	The primary file is in use by mmap.
VX_DEDUP_ESECINODE
	The secondary inode number is invalid or not found.
VX_DEDUP_ESECOFF
	The secondary file offset is not block aligned, exceeds file size, or produces overlap.
VX_DEDUP_ESECMMAP
	The secondary file is in use by mmap.
VX_DEDUP_EFILESET
	The fileset or names are invalid or non-matching.
VX_DEDUP_ENOMATCH
	The request contains no matching data to dedup.
VX_DEDUP_EVERSION
	There is an internal dedup version mismatch.
VX_DEDUP_EREADSZ
	The duph_readsz value is not block aligned or exceeds vx_dedup_maxlen or system limitations. An ioctl return value.
VX_DEDUP_EFLAG	The duph_flag value specified is invalid. An ioctl return value.
VX_DEDUP_EPRIBUSY
	The primary file is busy.
VX_DEDUP_ESECBUSY
	The secondary file is busy.
The following are system defined error values.
EINVAL	The request length exceeds the file size or is 0.
EINVAL	The ddup_sec variable is NULL or ddup_nsec is 0.
EINVAL	The file is not a regular file or the file systems a clone.
EFAULT	There is an internal copyin/copyout error.
ENOTSUP	The file system disk layout version is not 8 or above.
EROFS	The file system is read-only.
EIO	The file system is disabled.
EBUSY	The file is a swap file.
ENOMEM	Memory could not be allocated to support the API.