Add custom SATP claimrule for HP 3PAR to VMware ESXi
One of the tasks that I finish before I present the first Virtual Volumes (VV) to hosts is to discuss the need of a custom SATP claimrule with the customer. Requirement for a custom claimrule is usually, that the active and optimized path should be switched after each IO and not after 1000 IOs. Duncan Epping wrote a nice blog post some years ago. I recommend to read it.
Some basics
The Storage Array Type Plug-In (SATP) is responsable for array-specific operations, like health monitoring of physical paths, reporting of path state changes and path failover. Each SATP is linked to a Path Selection Policy (PSP), which controls the selection of active paths for IO. VMware ESXi provides a couple of SATPs:
~ # esxcli storage nmp satp list
Name Default PSP Description
------------------- ------------- ------------------------------------------
VMW_SATP_MSA VMW_PSP_MRU Placeholder (plugin not loaded)
VMW_SATP_ALUA VMW_PSP_MRU Placeholder (plugin not loaded)
VMW_SATP_DEFAULT_AP VMW_PSP_MRU Placeholder (plugin not loaded)
VMW_SATP_SVC VMW_PSP_FIXED Placeholder (plugin not loaded)
VMW_SATP_EQL VMW_PSP_FIXED Placeholder (plugin not loaded)
VMW_SATP_INV VMW_PSP_FIXED Placeholder (plugin not loaded)
VMW_SATP_EVA VMW_PSP_FIXED Placeholder (plugin not loaded)
VMW_SATP_ALUA_CX VMW_PSP_RR Placeholder (plugin not loaded)
VMW_SATP_SYMM VMW_PSP_RR Placeholder (plugin not loaded)
VMW_SATP_CX VMW_PSP_MRU Placeholder (plugin not loaded)
VMW_SATP_LSI VMW_PSP_MRU Placeholder (plugin not loaded)
VMW_SATP_DEFAULT_AA VMW_PSP_FIXED Supports non-specific active/active arrays
VMW_SATP_LOCAL VMW_PSP_FIXED Supports direct attached devices
Claimrules are used to select a valid SATP for a storage system. When a SATP for a given device is searched, the Native Multipathing Provider (NMP) searches the SATP driver part of the rules first. If there is no match, then the SATP vendor/ model part, and finally the SATP transport part is searched. If there is still no match, a valid default SATP rule is selected. And because of this, a custom SATP claimrule is selected instead the default SATP claimrule, because it’s more specific for a given device. The selected SATP determines the PSP.
Adding a custom SATP claimrume
I show you how to add a custom claimrume, in this case for a HP 3PAR storage system. This is an excerpt of some default SATP claimrules.
~ # esxcli storage nmp satp rule list
Name Device Vendor Model Driver Transport Options Rule Group Claim Options Default PSP PSP Options Description
------------------- ------ ------- ---------------- ------ --------- -------------------------- ---------- ----------------------------------- ----------- ----------- --------------------------------------------------------------------------
VMW_SATP_MSA MSA1000 VOLUME system MSA 1000/1500 [Legacy product, Not supported in this release]
VMW_SATP_ALUA LSI INF-01-00 reset_on_attempted_reserve system tpgs_on VMW_PSP_MRU NetApp E-Series arrays with ALUA support
VMW_SATP_ALUA NETAPP reset_on_attempted_reserve system tpgs_on VMW_PSP_RR NetApp arrays with ALUA support
VMW_SATP_ALUA IBM 2810XIV system tpgs_on VMW_PSP_RR IBM 2810XIV arrays with ALUA support
VMW_SATP_ALUA IBM 2107900 reset_on_attempted_reserve system VMW_PSP_RR
VMW_SATP_ALUA system tpgs_on Any array with ALUA support
Usually the highlighted claimrule will cause, that VMW_SATP_ALUA is used for a HP 3PAR storage system, if the ESX hosts use host persona 11 (VMware). For hosts with host persona 6 the VMW_SATP_DEFAULT_AA is used. Please note, that host persona 6 isn’t supported for ESX hosts with 3PAR OS 3.1.3! Since 3PAR OS 3.1.2 you should use host persona 11 instead of 6.
With esxcli you can add a new custom rule:
esxcli storage nmp satp rule add -s "VMW_SATP_ALUA" -P "VMW_PSP_RR" -O iops=1 -c "tpgs_on" -V "3PARdata" -M "VV" -e "HP 3PAR custom SATP Claimrule"
The switch -s adds a rule for VMW_SATP_ALUA, -P sets VMW_PSP_RR as the default PSP for this rule, -O enables that an active path is switches after 1 IO (default is 1000), -c enables ALUA target port group support, -V is a valid string from the vendor part of the SCSI inquiry string, -M is a valid string from the model part of the SCSI inquiry string and -e is for a description.
If you have to add this rule to multiple hosts, you may want to use these lines of PowerCLI code (make sure that you adjust the $esxCluster variable):
$esxCluster = 'LAB'
$esxHosts = Get-Cluster $esxCluster | Get-VMHost | Where { $_.PowerState -eq "PoweredOn" -and $_.ConnectionState -eq "Connected" } | Sort Name
foreach ($esx in $esxHosts) {
$esxcli = Get-EsxCli -VMHost $esx
$result = $esxcli.storage.nmp.satp.rule.add($null,"tpgs_on","HP 3PAR custom SATP Claimrule",$null,$null,$null,"VV",$null,"VMW_PSP_RR","iops=1","VMW_SATP_ALUA",$null,$null,"3PARdata")
Write-Host "Finished $esx"
}
You should add this rule before you present the first VVs to the hosts.
~ # esxcli storage nmp satp rule list -s VMW_SATP_ALUA
Name Device Vendor Model Driver Transport Options Rule Group Claim Options Default PSP PSP Options Description
------------- ------ -------- ------- ------ --------- -------------------------- ---------- ------------- ----------- ----------- ------------------------------------
VMW_SATP_ALUA NETAPP reset_on_attempted_reserve system tpgs_on VMW_PSP_RR NetApp arrays with ALUA support
VMW_SATP_ALUA IBM 2810XIV system tpgs_on VMW_PSP_RR IBM 2810XIV arrays with ALUA support
VMW_SATP_ALUA 3PARdata VV user tpgs_on VMW_PSP_RR iops=1 HP 3PAR custom SATP Claimrule
VMW_SATP_ALUA system tpgs_on Any array with ALUA support
VMW_SATP_ALUA IBM 2107900 reset_on_attempted_reserve system
VMW_SATP_ALUA system tpgs_on Any array with ALUA support
And there it is. Now you can present VVs to your ESX hosts and you can be sure, that this rule is claimed. If you add this rule after you have presented VVs to the ESX hosts, you have to reboot the hosts.