What is qlogic failover configuration

Refer to the technical notes below for supported disk arrays to determine if a given array is supported with multiple paths and with a particular multipath solution. Unless an array is specifically listed as being supported by LifeKeeper with multiple paths and with a particular multipath solution, it must be assumed that it is not. A faster processor may cause the failure to be seen more frequently. A system with more memory may cause the failure to be seen less frequently. In test cases where the problem has been seen, the test was writing an unlimited amount of data to the disk.

Most applications will both read and write data. The servers used in this test were IBM x servers with dual Xeon 2. The change to this parameter results in LifeKeeper waiting approximately 80 seconds before determining that an unresponsive system is dead, rather than the default wait time of approximately 15 seconds.

With some large storage configurations for example, multiple logical volume groups with 10 or more LUNs in each volume group , LifeKeeper may not be able to complete a sendevent within the default timeout of seconds when a failure is detected. This results in the switchover to the backup system failing. All resources are not brought in-service and an error message is logged in the LifeKeeper log.

LifeKeeper will then perform a successful failover to the backup system. Use the qla driver version 6. And user friendly device mapping are not supported. Set the following parameter in "multipath. Server must have interface based on IPMI 2. The test was performed with SPS for Linux v8. Certified by Hewlett-Packard Company. Novell testing was completed by the HP Storage Group. Certified by Hewlett-Packard Company with Fibre Channel in both single path and multipath configurations.

The model used for testing was the MSAsa. HP supports direct connect configurations only at this time. Blade configurations with LifeKeeper are not supported. Model tested was the HP P G2 7. The default kit supports single path storage as well as some multipath storage. RHEL 5. This storage was tested with the following configurations:. StoreVirtual VSA LeftHand OS version StoreVirtual The model numbers of tested storage are XP and XP The connection interface is FC. Use the qla or qla driver, version 6.

Single path i. Multiple path i. Fibre channel switches and hubs are not required with multiple path configurations with RDAC. RDAC is a software package that handles path failovers so that an application is not affected by a path failure.

The steps to install and setup RDAC are slightly different depending on the version being used. RDAC is needed for both single path and multipath. This storage device works in a two node LifeKeeper cluster in both single and multipath configurations.

Use the firmware version 7. Certified by partner testing in a single path configuration. Use the qla driver, version 7. The test was performed with LifeKeeper for Linux v7. The Dell PowerVault storage should not be mixed with any other types of shared storage to be managed by LifeKeeper within the same cluster. Follow the instructions provided with your hardware for configuring the Dell PowerVault storage and the controllers for use in a cluster.

You should then make sure that both controllers can see the same LUNs, and that the Linuxmegaraid driver is properly configured to be loaded. The assigned ID must be unique within the cluster and should be sufficiently constructed to avoid potential future conflicts.

The lkID utility will automatically generate a unique ID for you if desired. Refer to the lkID 8 man page for more details about the use of the utility, the IDs that it generates, where the ID is placed, and any possible restrictions. This is required due to the lack of SCSI reservation support in this configuration. You must also take care to avoid bringing a device hierarchy in-service on both nodes simultaneously via a manual operation when LifeKeeper communications have been disrupted for some reason.

Both are available from the QLogic website at www. This was specifically tested with RHEL4; however, there are no known issues using other LifeKeeper supported Linux distributions or versions.

RDAC is required for both single path and multipath configurations. The testing was completed using iscsi-initiator-utils Use the qla driver, version 6. However, it is also possible to configure a LifeKeeper cluster in which each server is connected directly to a separate controller or port on the Hitachi array, without the use of a switch or hub, as long as each server has only a single path to the storage.

It should be noted that LifeKeeper behaves quite differently from its normal behavior in a split-brain scenario using this configuration.

LifeKeeper normally performs a failover of an active hierarchy in a split-brain scenario causing the original primary node to reboot as a result of the stolen SCSI reservation. When the Hitachi arrays are configured with the servers directly connected to multiple controllers or ports, certain timing peculiarities within the Hitachi arrays prevent LifeKeeper from acquiring a SCSI reservation on the backup node and the failover attempt fails, leaving at least part of the hierarchy running on the original primary node.

For this reason, it is important that all LifeKeeper resources in such a configuration have a direct line of dependencies to one of the disk resources such that no resources can be brought in-service if the disk resources cannot be transferred.

This is particularly true of any IP resources in the hierarchy. For the V array, the following settings are required:. With this configuration, the LifeKeeper behavior in a split-brain scenario is the same as that described above for the Hitachi HDS storage arrays, and the same hierarchy configuration precautions should be observed. In both configurations, the LifeKeeper behavior in a split-brain scenario is the same as that described above for the Hitachi HDS storage arrays, and the same hierarchy configuration precautions should be observed.

The ArrayMasStor L was also tested and certified by our partner organization in a multipath configuration using QLogic and host bus adapters and the QLogic failover driver, version 6. Firmware version WS2. Testing was performed with LifeKeeper for Linux v6. Certified by vendor support statement in a single path configuration and in multipath configurations using the Device Mapper Multipath Recovery Kit. Once the SPS kit is installed, simply creating an application hierarchy that uses one or more of the multipath device nodes will automatically incorporate the new resource types provided by the SPS kit.

This means that devices reserved by one node in the cluster will remain read-accessible to other nodes in the cluster, but those other nodes will be unable to write to the device. Note that this does not mean that you can expect to be able to mount file systems on those other nodes for ongoing read-only access. There are no known dependencies on the version of the SPS package installed.

If new paths are added after the initial reservation, or if failed paths are repaired and SPS automatically reactivates them, those paths will not be registered as a part of the reservation until the next LifeKeeper quickCheck execution for the SPS resource. If SPS allows any writes to that path prior to that point in time, reservation conflicts that occur will be logged to the system message file. The SPS driver will retry these IOs on the registered path resulting in no observable failures to the application.

Once quickCheck registers the path, subsequent writes will be successful. Emulex LP 8. ELsmp kernel. For other supported fibre channel arrays with QLogic adapters, use the qla or qla driver, version 6. For the supported Emulex fibre channel HBAs, you must use the lpfc driver v8. We expect that it should work with other versions, distributions and adapters; however, this has not been tested.

See DataCore for specific support for these and other configurations. One issue was found during failover testing with heavy stress running where multiple server reboots would result in a server only configuring a single path. The test configuration consisted of a 3-node cluster where 2 servers would be killed simultaneously.

A reboot of the server would resolve the problem. This issue was never seen when only a single server was rebooted. This issue has been reported to DataCore. This item is not considered a critical issue since at least one path continues to be available. We expect that LifeKeeper would also work with other versions, distributions and adapters; however, this has not been tested. See Xiotech for specific support for these and other configurations.

The SCSI driver in the 2. The Magnitude 3D is not in that list. This support requires the use of the Secure Path v3. For each server, use Command View v3. Install the HP Secure Path software. This will require a reboot of the system. Verify that Secure Path has properly configured both paths to the storage.

See Secure Path documentation for further details. This support requires the use of the QLogic driver v7. The latest QLogic driver supplied by HP v8. The host connection must be "Linux". To upgrade a cluster from single path to multiple paths, perform the following steps this must be a cluster-wide upgrade :. Upgrade LifeKeeper to the latest version following the normal upgrade procedures. This step can be accomplished as a rolling upgrade such that the entire cluster does not have to be down.

Stop LifeKeeper on all nodes. The cluster will be down until the hardware upgrade is complete and step 5 is finished for all nodes.

Install the HP Secure Path software on each node. Note : This is a change from how the previous version of LifeKeeper supported an upgrade.

LifeKeeper v4. The MSA implements multipathing by having one controller active with the other controller in standby mode. You can add the attribute to the defaults section or the multipaths section. For example:. After it is set up, you can specify the reservation key in the mpathpersist commands. To manually start the service in the running system or to check its status, enter one of the following commands:.

To stop the multipath services in the current session and to disable it, so it will not be started the next time the system is booted, run the following commands:. Whenever you enable or disable the multipath services it is also required to rebuild the initrd , otherwise the system may not boot anymore.

When enabling the multipath services, also run the following command to rebuild the initrd:. Install the Linux HBA driver module. Upon module installation, the driver automatically scans the HBA to discover any SAN devices that have permissions for the host.

It presents them to the host for further configuration. Make sure that the HBA driver you are using does not have native multipathing enabled. After the driver module is loaded, discover the device nodes assigned to specific array LUNs or partitions. If the SAN device will be used as the root device on the server, modify the timeout settings for the device as described in Section Partitioning devices that have multiple paths is not recommended, but it is supported.

You can use the kpartx tool to create partitions on multipath devices without rebooting. You can also partition the device before you attempt to configure multipathing by using the Partitioner function in YaST, or by using a third-party partitioning tool.

Multipath devices are device-mapper devices. Modifying device-mapper devices with command line tools such as parted, kpartx, or fdisk works, but it does not necessarily generate the udev events that are required to update other layers.

After you partition the device-mapper device, you should check the multipath map to make sure the device-mapper devices were mapped.

If they are missing, you can remap the multipath devices or reboot the server to pick up all of the new partitions in the multipath map.

The device-mapper device for a partition on a multipath device is not the same as an independent device. When you create an LVM logical volume using the whole device, you must specify a device that contains no partitions.

If you specify a multipath partition as the target device for the LVM logical volume, LVM recognizes that the underlying physical device is partitioned and the create fails. Default multipath device settings are applied automatically when the multipathd daemon runs unless you create the multipath configuration file and personalize the settings.

This allows you to perform a dry run to verify your changes before they are committed. When you are satisfied with the revised settings, you can update the multipath maps as described in Section Make sure to check if the following sections of the file are matching your configuration:. Refer to your SAN's vendor documentation for the correct settings. Note that different SANs require individual device sections. This section needs to contain all non-multipath devices of your machine. If required, add more sections to the configuration file.

More details are available when running man 5 multipath. When you have successfully verified the configuration, apply it as described in Section See man 5 multipath. These values are used if no values are given in the appropriate device or multipath sections.

Lists the device names to discard as not multipath candidates. Devices can be identified by their device node name devnode , their WWID wwid , or their vendor or product strings device. You can typically ignore non-multipathed devices, such as hpsa , fd , hd , md , dm , sr , scd , st , ram , raw , and loop. For more information and examples, see Section Lists the device names of devices to be treated as multipath candidates even if they are on the blacklist.

You must specify the excepted devices by using the same keyword that you used in the blacklist. For example, if you used the devnode keyword for devices in the blacklist, you use the devnode keyword to exclude some devices in the blacklist exceptions. It is not possible to blacklist devices by using the devnode keyword and to exclude some of them by using the wwid keyword.

Specifies settings for individual multipath devices. Except for settings that do not support individual settings, these values overwrite what is specified in the defaults and devices sections of the configuration file.

Specifies settings for individual storage controllers. These values overwrite values specified in the defaults section of the configuration file. If you use a storage array that is not supported by default, you can create a devices subsection to specify the default settings for it. These values can be overwritten by settings for individual multipath devices if the keyword allows it. Section This command scans the devices, then displays what the setup would look like if you commit the changes.

If the changes are acceptable, continue with the next step. Paths are grouped into priority groups. Only one priority group is in active use at a time. This normally happens automatically on device discovery. For each path, its physical address host:bus:target:lun , device node name, major:minor number, and state is shown.

By using a verbosity level of -v3 in the dry run, you can see all detected paths, multipaths, and device maps. Both WWID and device node blacklisted devices are displayed. Some multiple entries have been omitted to shorten the example. After you make changes, save and close the file, then run the following commands to apply the changes and update the multipath maps:. Run dracut -f to re-create the initrd image on your system, then reboot for the changes to take effect.

If the WWID is the same for two device paths, they are assumed to represent the same device. We recommend not changing the method of WWID generation, unless there is a compelling reason to do so. For more details, see man multipath. The desired default behavior depends on whether the server is a stand-alone server or a node in a high-availability cluster.

It queues messages until a multipath failover occurs and provides a healthy connection. If the field is not otherwise specified in a device section, the default setting is applied for that SAN configuration. The following are the compiled default settings. For information about setting the polling, queuing, and failback policies, see the following parameters in Section You can blacklist devices by WWID wwid keyword , device name devnode keyword , or device type device section.

The preferred method for blacklisting devices is by WWID or by vendor and product. Blacklisting by devnode is not recommended, because device nodes can change and thus are not useful for persistent device identification. They only work if they are matched against common strings. However, the standard configuration of multipath already contains regular expressions for many devices and vendors.

Matching regular expressions with other regular expressions does not work. Make sure that you are only matching against strings shown with multipath -t.

For example, local SATA hard disks and flash disks do not have multiple paths. If you want multipath to ignore single-path devices, put them in the blacklist section. To match an arbitrary string, you must now use ". For example, to blacklist local devices and all arrays from the hpsa driver from being managed by multipath, the blacklist section looks like this:. You can also blacklist only the partitions from a driver instead of the entire array.

For example, you can use the following regular expression to blacklist only partitions from the cciss driver and not the entire array:. You can blacklist by specific device types by adding a device section in the blacklist, and using the vendor and product keywords.

You add exceptions by WWID wwid keyword , device name devnode keyword , or device type device section. You must specify the exceptions in the same way that you blacklisted the corresponding devices. That is, wwid exceptions apply to a wwid blacklist, devnode exceptions apply to a devnode blacklist, and device type exceptions apply to a device type blacklist.

For example, you can enable multipath for a desired device type when you have different device types from the same vendor. Following the reboot, the local devices should no longer be listed in the multipath maps when you issue the multipath -ll command. Simply put, this option prevents multipath and multipathd from setting up multipath maps for devices with only a single path see the man 5 multipath.

In certain configurations, this may save the administrator from the effort of creating blacklist entries, for example for local SATA disks.

It complicates and slows down the system boot, because for every device found, the boot logic needs to wait until all devices have been discovered to see whether a second path exists for the device.

Additionally, problems can arise when some paths are down or otherwise invisible at boot time—a device can be falsely detected as a single-path device and activated, causing later addition of more paths to fail. A multipath device can be identified by its WWID, by a user-friendly name, or by an alias that you assign for it. You can also use a user-friendly name or alias that is mapped to the WWID to identify the device uniquely across reboots.

For an example of multipath. The serial WWID Worldwide Identifier is an identifier for the multipath device that is guaranteed to be globally unique and unchanging. An alias name is a globally unique name that the administrator provides for a multipath device.

This may conflict with an automatically assigned user-friendly name, and give you incorrect device node names. If it is set to no the default , multipath uses the WWID as the name of the device. If an alias name is set up for a multipath device, the alias is used instead of the WWID or the user-friendly name. To avoid this problem, we recommend that you use the default WWID settings for the system root device. You should not use aliases for the system root device.

Because the device name would differ, using an alias causes you to lose the ability to seamlessly switch off multipathing via the kernel command line. For example, this can be done as follows:. Even if the system root device is not on multipath, it is possible for multipath to be included in the initrd. For example, this can happen if the system root device is on LVM.

This disables multipath only in the initrd during system boots. After the system boots, the boot. The alternate path must be available on the system root device where multipath can find it. Uncomment the defaults section and its ending bracket. Optional Specify your own names for devices by using the alias option in the multipath section. The scsi- and wwn- device names represent physical paths to the devices.

Make sure that multipath devices have the same name across all devices by doing the following:. Use UUID and alias names to ensure that multipath device names are consistent across all nodes in the cluster. Alias names must be unique across all nodes. A user-friendly name is unique to a node, but a device might not be assigned the same user-friendly name on every node in the cluster. If you really need to use user-friendly names, you can force the system-defined user-friendly names to be consistent across all nodes in the cluster by doing the following:.

This applies to all affected nodes. In a Linux host, when there are multiple paths to a storage controller, each path appears as a separate block device, and results in multiple block devices for single LUN. This section describes how to specify policies for failover and configure priorities for the paths.

Use the multipath command with the -p option to set the path failover policy:. A priority group is a collection of paths that go to the same physical LUN. The group with the highest calculated value is the primary. When all paths in the primary group are failed, the priority group with the next highest value becomes active. A path priority is an integer value assigned to a path. The higher the value, the higher the priority. An external program is used to assign priorities for each path.

For a given device, the paths with the same priorities belong to the same priority group. The prio line specifies the prioritizer. Specifies the prioritizer program to call to obtain a path priority value. Weights are summed for each path group to determine the next path group to use in case of failure. If no prio keyword is specified, all paths are equal.

These are the arguments for the prioritizer programs that require arguments. Most prio programs do not need arguments. There is no default. The values depend on the prio setting and whether the prioritizer requires any of the following arguments:. Regex is in serial number format. Usually the latency on the remote array will be higher, so you can tune the latency to bring them closer together. Valid values are integers from 2 to Valid values are integer from 2 - The maximum average latency value is s, the minimum is 1us.

You can specify attributes as defaults for all multipath devices. You can also specify attributes that apply only to a given multipath device by creating an entry for that device in the multipaths section of the multipath configuration file. This option can be used in the devices section and the multipaths section. Autogenerate user-friendly names as aliases for the multipath devices instead of the actual ID.

Specifies whether to monitor the failed path recovery, and indicates the timing for group failback after failed paths return to service. When the failed path recovers, the path is added back into the multipath-enabled path list based on this setting. Multipath evaluates the priority groups, and changes the active priority group when the priority of the primary path exceeds the secondary group. Default The failed path is not monitored for recovery. The administrator runs the multipath command to update enabled paths and priority groups.

Only perform automatic failback when the first path of a pathgroup becomes active. This keeps a node from automatically failing back when another node requested the failover.

When the path recovers, wait N seconds before enabling the path. Specify an integer value greater than 0. We recommend failback setting of manual for multipath in cluster environments to prevent multipath failover ping-pong.

Make sure that you verify the failback setting with your storage system vendor. Different storage systems can require different settings. Specifies the number of retries until multipath stops the queuing and fails the path. This causes the resources to fail over when the connection is lost to storage.

Otherwise, the messages queue and the resource failover cannot occur. Make sure that you verify the retry settings with your storage system vendor. This is useful for DASD devices. Logs failure messages in the systemd journal see Chapter 17, journalctl : Query the systemd Journal. Issues an SCSI test unit ready command to the device. This is the preferred setting if the LUN supports it. On failure, the command does not fill up the systemd log journal with messages.

Specifies the path grouping policy for a multipath device hosted by a given controller. Default One path is assigned per priority group so that only one path at a time is used. All valid paths are in one priority group. Traffic is load-balanced across all active paths in the group. One priority group exists for each path priority value. Priorities are assigned by an external program.

One priority group is assigned per target node name. The load-balancing algorithm used to balance traffic across all active paths in a priority group. Specifies path group timeout handling. No value can be specified; an internal default is set. Specifies the time in seconds between the end of one path checking cycle and the beginning of the next path checking cycle.

The default value is 5. A udev attribute that provides a unique path identifier. All paths are active. A single path with the highest priority lowest value setting is active for traffic. Other paths are available for failover, but are not used unless failover occurs. Multiple paths with the same priority fall into the active group. When all paths in that group fail, the device fails over to the next highest priority group.

All paths in the group share the traffic load in a round-robin load balancing fashion. To install the operating system on a multipath device, the multipath software must be running at install time. The multipathd daemon is not automatically active during the system installation. Choose Expert Partitioner on the Suggested Partitioning screen during the installation.

This is the device that should be used for all further processing. Write down the device path and UUID; you will need it later. After all settings are done and the installation is finished, YaST starts to write the boot loader information, and displays a countdown to restart the system.

Stop the counter by clicking the Stop button and press Ctrl — Alt — F5 to access a console. This is necessary because the installation does not distinguish between active and passive paths. Active path: No action is needed. Skip all remaining steps and return to the YaST graphical environment by pressing Ctrl — Alt — F7 and continue with the installation. Passive path: The configuration must be changed and the boot loader must be reinstalled.

If the hd0 entry points to a passive path, change the configuration and reinstall the boot loader:. At the console, run multipath -ll , then check the output to find the active path. Install Linux with only a single path active, preferably one where the by-id symbolic links are listed in the partitioner.

Run dracut -f to update the initrd image. For IBM Z, after running dracut , run zipl. Ideally, you should configure multipathing for devices before you use them as components of a software RAID device. You can use the procedure in this section to get multipathing running for a previously existing software RAID. For example, you might need to configure multipathing for devices in a software RAID under the following circumstances:.

If you create a new software RAID as part of the Partitioning settings during a new install or upgrade. If you did not configure the devices for multipathing before using them in the software RAID as a member device or spare. Except where otherwise directed, use this console to enter the commands in the following steps.

If any software RAID devices are currently mounted or running, enter the following commands for each device to unmount the device and stop it.

Start the multipathd daemon by entering the following command:. Do one of the following:. Devices Are Not Listed: Force the multipath service to recognize them by flushing and rediscovering the devices by entering. Restart the mdmonitor service and the RAID device by entering.