Redhat clvm setup

Asymmmetric cluster configurations in which some nodes have access to the storage and others do not are not supported. Figure 2. If you are not able to use either the clvmd daemon or HA-LVM for operational reasons or because you do not have the correct entitlements, you must not use single-instance LVM on the shared disk as this may result in data corruption. If you have any concerns please contact your Red Hat service representative. CLVM requires changes to the lvm.

Level 1 redundancy is the highest possible among all RAID types, with the array being able to operate with only a single disk present. Level 4 uses parity concentrated on a single disk drive to protect data. Parity information is calculated based on the content of the rest of the member disks in the array. This information can then be used to reconstruct data when one disk in the array fails. Because the dedicated parity disk represents an inherent bottleneck on all write transactions to the RAID array, level 4 is seldom used without accompanying technologies such as write-back caching, or in specific circumstances where the system administrator is intentionally designing the software RAID device with this bottleneck in mind such as an array that will have little to no write transactions once the array is populated with data.

RAID level 4 is so rarely used that it is not available as an option in Anaconda. However, it could be created manually by the user if truly needed. The storage capacity of Hardware RAID level 4 is equal to the capacity of the smallest member partition multiplied by the number of partitions minus one. Performance of a RAID level 4 array is always asymmetrical, meaning reads outperform writes.

This is because writes consume extra CPU and main memory bandwidth when generating parity, and then also consume extra bus bandwidth when writing the actual data to disks because you are writing not only the data, but also the parity. Reads need only read the data and not the parity unless the array is in a degraded state.

As a result, reads generate less traffic to the drives and across the buses of the computer for the same amount of data transfer under normal operating conditions. This is the most common type of RAID. By distributing parity across all the member disk drives of an array, RAID level 5 eliminates the write bottleneck inherent in level 4.

The only performance bottleneck is the parity calculation process itself. However, if you have a sufficiently large number of member devices in a software RAID5 array such that the combined aggregate data transfer speed across all devices is high enough, then this bottleneck can start to come into play. As with level 4, level 5 has asymmetrical performance, and reads substantially outperforming writes. The storage capacity of RAID level 5 is calculated the same way as with level 4.

This is a common level of RAID when data redundancy and preservation, and not performance, are the paramount concerns, but where the space inefficiency of level 1 is not acceptable.

Level 6 uses a complex parity scheme to be able to recover from the loss of any two drives in the array. This complex parity scheme creates a significantly higher CPU burden on software RAID devices and also imposes an increased burden during write transactions. As such, level 6 is considerably more asymmetrical in performance than levels 4 and 5. The total capacity of a RAID level 6 array is calculated similarly to RAID level 5 and 4, except that you must subtract 2 devices instead of 1 from the device count for the extra parity storage space.

This RAID level attempts to combine the performance advantages of level 0 with the redundancy of level 1. It also helps to alleviate some of the space wasted in level 1 arrays with more than 2 devices.

With level 10, it is possible for instance to create a 3-drive array configured to store only 2 copies of each piece of data, which then allows the overall array size to be 1. The creation of RAID level 10 is not supported during installation. It is possible to create one manually after installation. In linear RAID, the chunks are allocated sequentially from one member drive, going to the next drive only when the first is completely filled.

Linear RAID also offers no redundancy and decreases reliability. If any one member drive fails, the entire array cannot be used. The capacity is the total of all member disks. To create a RAID logical volume, you specify a raid type as the --type argument of the lvcreate command. The following table describes the possible RAID segment types. For most users, specifying one of the five available primary types raid1 , raid4 , raid5 , raid6 , raid10 should be sufficient.

RAID1 mirroring. This is the default value for the --type argument of the lvcreate command when you specify the -m but you do not specify striping. RAID0 spreads logical volume data across multiple data subvolumes in units of stripe size. This is used to increase performance. Logical volume data will be lost if any of the data subvolumes fail. You can create RAID1 arrays with different numbers of copies according to the value you specify for the -m argument.

You can also specify the stripe size with the -I argument. Note that you specify the number of stripes just as you do for an LVM striped volume; the correct number of parity drives is added automatically. The corrupted data can exist indefinitely on storage devices.

You might not discover this corrupted data until you retrieve and attempt to use this data. However, a RAID configuration does not ensure the integrity of the data itself. Soft corruption, silent corruption, soft errors, and silent errors are terms that describe data that has become corrupted, even if the system design and software continues to function as expected.

DM integrity is used with RAID levels 1, 4, 5, 6, and 10 to help mitigate or prevent data loss due to soft corruption. The RAID layer ensures that a non-corrupted copy of the data can fix the soft corruption errors.

If corruption is detected, the integrity layer returns an error message, and the RAID layer retrieves a non-corrupted copy of the data from another RAID image. The RAID layer automatically rewrites non-corrupted data over the corrupted data to repair the soft corruption. If you experience performance issues, Red Hat recommends that you either use RAID1 with integrity or that you test the performance of a particular RAID configuration to ensure that it meets your requirements.

Logical volume "test-lv" created. Therefore, you must remove the integrity before performing certain operations. The g attribute in the attributes listed under the Attr column indicates that the RAID image is using integrity.

There is an incremental counter that counts the number of mismatches detected on each RAID image. To view the data mismatches detected by integrity on a particular RAID image, run the following command:. You must run the command for each RAID image that you want to view. We can see that integrity has not detected any data mismatches and thus the IntegMismatches counter shows zero 0.

You can also find data integrity information in the kernel message logs, as shown in the following examples. Example of dm-integrity mismatches from the kernel message logs. Example of dm-integrity data corrections from the kernel message logs. This can cause the other LVM operations to slow down. You can control the rate at which a RAID logical volume is initialized by implementing recovery throttling. You specify these options as follows.

The Rate is specified as an amount per second for each device in the array. Setting the recovery rate to 0 means it will be unbounded. You can convert an existing linear logical volume to a RAID device by using the --type argument of the lvconvert command.

Since RAID logical volumes are composed of metadata and data subvolume pairs, when you convert a linear device to a RAID1 array, a new metadata subvolume is created and associated with the original logical volume on one of the same physical volumes that the linear volume is on.

For example, if the original device is as follows:. After conversion to a 2-way RAID1 array the device contains the following data and metadata subvolume pairs:. If the metadata image that pairs with the original logical volume cannot be placed on the same physical volume, the lvconvert will fail. This renames the mirror subvolumes mimage to RAID subvolumes rimage. In addition, the mirror log is removed and metadata subvolumes rmeta are created for the data subvolumes on the same physical volumes as the corresponding data subvolumes.

You can change the number of images in an existing RAID1 array just as you can change the number of images in the earlier implementation of LVM mirroring. When you add images to a RAID1 device with the lvconvert command, you can specify the total number of images for the resulting device, or you can specify how many images to add to the device.

Metadata subvolumes named rmeta always exist on the same physical devices as their data subvolume counterparts rimage. When you add an image to a RAID1 array, you can specify which physical volumes to use for the image. To remove images from a RAID1 array, use the following command. When you remove images from a RAID1 device with the lvconvert command, you can specify the total number of images for the resulting device, or you can specify how many images to remove from the device.

You can also optionally specify the physical volumes from which to remove the device. Additionally, when an image and its associated metadata subvolume volume are removed, any higher-numbered images will be shifted down to fill the slot.

Just as when you are removing a RAID image from an existing RAID1 logical volume, when you remove a RAID data subvolume and its associated metadata subvolume from the middle of the device any higher numbered images will be shifted down to fill the slot. The index numbers on the logical volumes that make up a RAID array will thus be an unbroken sequence of integers.

You can temporarily split off an image of a RAID1 array for read-only use while keeping track of any changes by using the --trackchanges argument in conjunction with the --splitmirrors argument of the lvconvert command. This allows you to merge the image back into the array at a later time while resyncing only those portions of the array that have changed since the image was split.

The format for the lvconvert command to split off a RAID image is as follows. When you split off a RAID image with the --trackchanges argument, you can specify which image to split but you cannot change the name of the volume being split. In addition, the resulting volumes have the following constraints. You can merge an image that was split off with the --trackchanges argument specified by executing a subsequent lvconvert command with the --merge argument.

When you merge the image, only the portions of the array that have changed since the image was split are resynced. The format for the lvconvert command to merge a RAID image is as follows.

The following example creates a RAID1 logical volume and then splits off an image from that volume while tracking changes to the remaining array. The following example splits off an image from a RAID1 volume while tracking changes to the remaining array, then merges the volume back into the array. As long as there are enough devices remaining to support usability, the RAID logical volume will continue to operate. The RAID logical volume is laid out as follows. Note that even though the failed device has been replaced, the display still indicates that LVM could not find the failed device.

This is because, although the failed device has been removed from the RAID logical volume, the failed device has not yet been removed from the volume group. To remove the failed device from the volume group, you can execute vgreduce --removemissing VG. If the allocation fails, you have the option of fixing the drive, then initiating recovery of the failed device with the --refresh option of the lvchange command.

Alternately, you can replace the failed device. Instead, if the device has failed you can replace the device with the --repair argument of the lvconvert command. To replace a RAID device in a logical volume, use the --replace argument of the lvconvert command. The following example creates a RAID1 logical volume and then replaces a device in that volume. The following example creates a RAID1 logical volume and then replaces a device in that volume, specifying which physical volume to use for the replacement.

You can replace more than one RAID device at a time by specifying multiple replace arguments, as in the following example.

LVM mirroring required failed devices to be removed or the mirrored logical volume would hang. RAID arrays can keep on running with failed devices. Therefore, rather than removing a failed device unconditionally and potentially allocating a replacement, LVM allows you to replace a failed device in a RAID volume in a one-step solution by using the --repair argument of the lvconvert command.

If the LVM RAID device failure is a transient failure or you are able to repair the device that failed, you can initiate recovery of the failed device. The volume group includes a physical volume that provides enough free capacity to replace the failed device.

If no physical volume with sufficient free extents is available on the volume group, add a new, sufficiently large physical volume using the vgextend utility. Optional: To manually specify the physical volume that replaces the failed device, add the physical volume at the end of the command:. Until you remove the failed device from the volume group, LVM utilities still indicate that LVM cannot find the failed device.

RAID scrubbing is the process of reading all the data and parity blocks in an array and checking to see whether they are coherent. This might cause the other LVM operations to slow down. You can control the rate of the scrubbing operation by implementing recovery throttling. Add the following options to the lvchange --syncaction commands in the next steps:. Specify the Rate value as an amount per second for each device in the array.

If you provide no suffix, the options assume kiB per second per device. The lvchange --syncaction repair operation does not perform the same function as the lvconvert --repair operation:.

It can be one of the following values:. Bit 9 of this field displays the health of the logical volume, and it supports the following indicators:. Changing the RAID level is usually done to increase or decrease resilience to device failures or to restripe logical volumes.

You use the lvconvert for RAID takeover. Some attributes you can change include RAID layout, stripe size, and number of stripes. The format for using these parameters is as follows. Marks a device in a RAID1 logical volume as write-mostly. All reads to these drives will be avoided unless necessary.

By default, the write-mostly attribute is set to yes for the specified physical volume in the logical volume. It is possible to remove the write-mostly flag by appending :n to the physical volume or to toggle the value by specifying :t. The --writemostly argument can be specified more than one time in a single command, making it possible to toggle the write-mostly attributes for all the physical volumes in a logical volume at once.

Specifies the maximum number of outstanding writes that are allowed to devices in a RAID1 logical volume that are marked as write-mostly. Once this value is exceeded, writes become synchronous, causing all writes to the constituent devices to complete before the array signals the write has completed. Setting the value to zero clears the preference and allows the system to choose the value arbitrarily.

You can override this default value with the -R option of the lvcreate command. After you have created a RAID logical volume, you can change the region size of the volume with the -R option of the lvconvert command.

The RAID volume must be synced in order to change the region size. Use the -s argument of the lvcreate command to create a snapshot volume. A snapshot volume is writable. LVM snapshots are not supported across the nodes in a cluster. You cannot create a snapshot volume in a shared volume group. However, if you need to create a consistent backup of data on a shared logical volume you can activate the volume exclusively and then create the snapshot.

LVM does not allow you to create a snapshot volume that is larger than the size of the origin volume plus needed metadata for the volume. If you specify a snapshot volume that is larger than this, the system will create a snapshot volume that is only as large as will be needed for the size of the origin.

The following procedure creates an origin logical volume named origin and a snapshot volume of the original volume named snap. Create a logical volume named origin from the volume group VG. If the original logical volume contains a file system, you can mount the snapshot logical volume on an arbitrary directory in order to access the contents of the file system to run a backup while the original file system continues to get updated.

Because the snapshot increases in size as the origin volume changes, it is important to monitor the percentage of the snapshot volume regularly with the lvs command to be sure it does not fill. In addition to the snapshot itself being invalidated when full, any mounted file systems on that snapshot device are forcibly unmounted, avoiding the inevitable file system errors upon access to the mount point.

This option allows automatic extension of a snapshot whenever the remaining snapshot space drops below the threshold you set. This feature requires that there be unallocated space in the volume group. Similarly, automatic extension of a snapshot will not increase the size of a snapshot volume beyond the maximum calculated size that is necessary for the snapshot.

Once a snapshot has grown large enough to cover the origin, it is no longer monitored for automatic extension. If both the origin and snapshot volume are not open, the merge will start immediately. Otherwise, the merge will start the first time either the origin or snapshot are activated and both are closed.

Merging a snapshot into an origin that cannot be closed, for example a root file system, is deferred until the next time the origin volume is activated. While the merge is in progress, reads or writes to the origin appear as they were directed to the snapshot being merged.

When the merge finishes, the merged snapshot is removed. You can specify multiple snapshots on the command line, or you can use LVM object tags to specify that multiple snapshots be merged to their respective origins.

If the --background option were used, all snapshot logical volume merges would start in parallel. For further information on the lvconvert --merge command, see the lvconvert 8 man page. This procedure provides an overview of the basic commands you use to create and grow thinly-provisioned logical volumes. For detailed information on LVM thin provisioning as well as information on using the LVM commands and utilities with thinly-provisioned logical volumes, see the lvmthin 7 man page.

You can use the -T or --thin option of the lvcreate command to create either a thin pool or a thin volume. You can also use -T option of the lvcreate command to create both a thin pool and a thin volume in that pool at the same time with a single command.

The following command uses the -T option of the lvcreate command to create a thin pool named mythinpool in the volume group vg and that is M in size. Note that since you are creating a pool of physical space, you must specify the size of the pool.

The -T option of the lvcreate command does not take an argument; it deduces what type of device is to be created from the other options the command specifies. Note that in this case you are specifying virtual size, and that you are specifying a virtual size for the volume that is greater than the pool that contains it.

The following command uses the -T option of the lvcreate command to create a thin pool and a thin volume in that pool by specifying both a size and a virtual size argument for the lvcreate command. This command creates a thin pool named mythinpool in the volume group vg and it also creates a thin volume named thinvolume in that pool. You can also create a thin pool by specifying the --thinpool parameter of the lvcreate command. Unlike the -T option, the --thinpool parameter requires an argument, which is the name of the thin pool logical volume that you are creating.

The following example specifies the --thinpool parameter of the lvcreate command to create a thin pool named mythinpool in the volume group vg and that is M in size:. By default, lvm2 starts with a 64KiB chunk size and increases its value when the resulting size of the thin pool metadata device grows above MiB, this keeps the metadata size compact. However, this may result in some big chunk size values, which are less space efficient for snapshot usage.

In such cases, a smaller chunk size and a bigger metadata size is a better option. Red Hat does not recommend setting a chunk size smaller than the default value. If the chunk size is too small and your volume runs out of space for metadata, the volume is unable to create data.

Monitor your logical volumes to ensure that they are expanded, or create more storage before the metadata volumes become completely full. Ensure that you set up your thin pool with a large enough chunk size so that they do not run out of room for the metadata.

Striping is supported for pool creation. The following command creates a M thin pool named pool in volume group vg with two 64 kB stripes and a chunk size of kB.

You can extend the size of a thin volume with the lvextend command. You cannot, however, reduce the size of a thin pool. The following command resizes an existing thin pool that is M in size by extending it another M. As with other types of logical volumes, you can rename the volume with the lvrename , you can remove the volume with the lvremove , and you can display information about the volume with the lvs and lvdisplay commands.

You can use the --thinpool parameter of the lvconvert command to convert an existing logical volume to a thin pool volume. Converting a logical volume to a thin pool volume or a thin pool metadata volume destroys the content of the logical volume, since in this case the lvconvert does not preserve the content of the devices but instead overwrites the content.

The following example converts the existing logical volume lv1 in volume group vg to a thin pool volume and converts the existing logical volume lv2 in volume group vg to the metadata volume for that thin pool volume. This section provides an overview of the basic commands you use to create and grow thinly-provisioned snapshot volumes. When creating a thin snapshot volume, you do not specify the size of the volume.

If you specify a size parameter, the snapshot that will be created will not be a thin snapshot volume and will not use the thin pool for storing data.

Thin snapshots can be created for thinly-provisioned origin volumes, or for origin volumes that are not thinly-provisioned. You can specify a name for the snapshot volume with the --name option of the lvcreate command. For information on extending the size of a thin volume, see Creating thinly-provisioned logical volumes. A thin snapshot volume has the same characteristics as any other thin volume. You can independently activate the volume, extend the volume, rename the volume, remove the volume, and even snapshot the volume.

By default, a snapshot volume is skipped during normal activation commands. For information on controlling the activation of a logical volume, see Logical volume activation.

You can also create a thinly-provisioned snapshot of a non-thinly-provisioned logical volume. Since the non-thinly-provisioned logical volume is not contained within a thin pool, it is referred to as an external origin. External origin volumes can be used and shared by many thinly-provisioned snapshot volumes, even from different thin pools. The external origin must be inactive and read-only at the time the thinly-provisioned snapshot is created.

To create a thinly-provisioned snapshot of an external origin, you must specify the --thinpool option. The thin snapshot volume is named mythinsnap.

Because the origin volume must be in the same volume group as the snapshot volume, you do not need to specify the volume group when specifying the origin logical volume. You can create a second thinly-provisioned snapshot volume of the first snapshot volume, as in the following command.

For example, if you remove the logical volume stack3 from this sample configuration, the display is as follows. This allows you to display a full thin snapshot dependency chain that includes logical volumes that have been removed from the original dependency chain and have become historical logical volumes.

A historical logical volume retains a simplified representation of the logical volume that has been removed, including the following reporting fields for the volume:.

When a volume is removed, the historical logical volume name acquires a hypen as a prefix. For example, when you remove the logical volume lvol1 , the name of the historical volume is -lvol1.

A historical logical volume cannot be reactivated. To include historical logical volumes in volume display, you specify the -H --history option of an LVM display command. The following series of commands provides examples of how you can display and manage historical logical volumes.

Note that even though the example lvs display command includes the -H option, no thin snapshot volume has yet been removed and there are no historical logical volumes to display. Remove logical volume lvol3 from the snapshot chain, then run the following lvs command again to see how historical logical volumes are displayed, along with their ancestors and descendants.

LVM does not keep historical logical volumes if the volume has no live descendant. This means that if you remove a logical volume at the end of a snapshot chain, the logical volume is not retained as a historical logical volume. Run the following commands to remove the volume lvol1 and lvol2 and to see how the lvs command displays the volumes once they have been removed.

To remove a historical logical volume completely, you can run the lvremove command again, specifying the name of the historical volume that now includes the hyphen, as in the following example. A historical logical volumes is retained as long as there is a chain that includes live volumes in its descendants. This means that removing a historical logical volume also removes all of the logical volumes in the chain if no existing descendant is linked to them, as shown in the following example.

You can add caching to an LVM logical volume to improve performance. The following procedures create a special LV from the fast device, and attach this special LV to the original LV to improve the performance. LVM provides the following kinds of caching. This method speeds up access to frequently used data by caching it on the faster volume. The method caches both read and write operations.

The dm-cache method creates logical volumes of the type cache. This method caches only write operations. The faster volume stores the write operations and then migrates them to the slower disk in the background. The dm-writecache method creates logical volumes of the type writecache. When you enable caching for a logical volume, LVM renames and hides the original volumes, and presents a new logical volume that is composed of the original logical volumes.

The composition of the new logical volume depends on the caching method and whether you are using the cachevol or cachepool option.

The cachevol and cachepool options expose different levels of control over the placement of the caching components:. HA-LVM imposes the restriction that a logical volume can only be activated exclusively; that is, active on only one machine at a time. This means that only local non-clustered implementations of the storage drivers are used.

Avoiding the cluster coordination overhead in this way increases performance. By default, the locking type is set to local. To change this, execute the following command in each node of the cluster to enable clustered locking:. Set up a dlm resource for the cluster. You create the resource as a cloned resource so that it will run on every node in the cluster. Configure clvmd as a cluster resource.

Just as for the dlm resource, you create the resource as a cloned resource so that it will run on every node in the cluster.

Set up clvmd and dlm dependency and start up order. Create the mirror. The first step is creating the physical volumes.