Network Design And IP Plan

Management: 10.0.10.0/24 -> For comunication between Servers.
Heartbeat: 10.0.11.0/24-> For comunication between cluster nodes.
SAN: 10.0.8.0/27 or /24 -> For comunication to the SAN storage.

Server Name	Management IP	SAN1 IP	SAN2 IP	HeartBeat IP
SAN Storage	10.0.10.8	10.0.8.5	10.0.8.6
		10.0.8.7	10.0.8.8
CA-HVC0	10.0.10.10	-	-	-
CA-HVC01	10.0.10.11	10.0.8.11	10.0.8.21	10.0.11.11
CA-HVC02	10.0.10.12	10.0.8.12	10.0.8.22	10.0.11.12
CA-HVC03	10.0.10.13	10.0.8.14	10.0.8.23	10.0.11.13

Requirement

You need at least 3 servers with the same hardware specification with 64 bit processors supports a virtualization Intel Virtualization Technology (Intel VT) or AMD Virtualization (AMD-V) technology.
at least 2x10GB ethernet adapter for management interface (including Heartbeat), and 2x10Gb for SAN.

Confirm compatibility of the network a dapter: Confirm with manufacturers and vendors that the network adapter, including drivers, firmware, and software used for the storage, are compatible with failover clusters in the version of Windows Server that you are running.

Server Basic Setting:

Build a new server using latest Windows 2022 GUI version.
Perform the server basic configuration such us:
- Installing and updating the latest drivers.
- Updating latest BIOS.
- Instaling the RAID monitoring,
- Change the power profile to High Performance either on the server’s BIOS, and on the OS.
- Configure the IDRAC or plug the KVM to configure the server.
- Configure Management and Heartbeat IP.
- Configure Jumbo Frame for SAN interface.
- Rename Server name follows your company standard.
- Join it to the domain.

Rename Interface

On this example, I use 2x PCI network adapters with 2x10GB on PCI Slot 3 and Slot 4 and crossed the cable for redudancy. By this way, in case one of the ethernet is broken in the future, the server still survive even it’s using the 1 left ethernet.

2x10GB: SLOT 3 Port 1 and SLOT 4 Port 1 for management vlanID: 101, and heartbeat vlanID: 102
2x10GB: SLOT 3 Port 2 and SLOT 4 Port 2 for SAN interfaces.

Port Identification and configuration.

Identify the port and cable on the ethernet port adapter and port on switches. Make sure you are plugging and crossing the cable on different port on the server, and on the management and SAN switches. To easiliy identify the adapter on the server, after you are ensuring all ports are installed correctlty on physical perspective, rename the adapter with an easy name on the server as the following: Show all interfaces: Get-NetAdapter -Physical

PS C:\> Get-NetAdapter -Physical

Name                      InterfaceDescription                    ifIndex Status       MacAddress             LinkSpeed
----                      --------------------                    ------- ------       ----------             ---------
SLOT 3 Port 1             Intel Ethernet 10G 2P X520 Adapter      20      Up           33-4D-78-8A-6D-74        10 Gbps
SLOT 3 Port 2             Intel Ethernet 10G 2P X520 Adapter      22      Up           33-4D-78-97-8A-E4        10 Gbps
SLOT 4 Port 1             Intel Ethernet 10G 2P X520 Adapter      30      Up           33-4D-78-F5-96-85        10 Gbps
SLOT 4 Port 2             Intel Ethernet 10G 2P X520 Adapter      32      Up           33-4D-78-68-76-81        10 Gbps

Rename the adapter:

Get-NetAdapter -Physical "SLOT 3 Port 1" | Rename-NetAdapter -NewName "SLOT 3 Port 1-MGMT1"
Get-NetAdapter -Physical "SLOT 4 Port 1" | Rename-NetAdapter -NewName "SLOT 4 Port 1-MGMT2"
Get-NetAdapter -Physical "SLOT 3 Port 2" | Rename-NetAdapter -NewName "SLOT 3 Port 2-SAN1" 
Get-NetAdapter -Physical "SLOT 4 Port 2" | Rename-NetAdapter -NewName "SLOT 4 Port 2-SAN2"

Or based on Mac Address:

Get-NetAdapter -Physical | where {$_.macaddress -eq "33-4D-78-8A-6D-74"} | Rename-NetAdapter -NewName "SLOT 3 Port 1-MGMT1"
Get-NetAdapter -Physical | where {$_.macaddress -eq "33-4D-78-F5-96-85"} | Rename-NetAdapter -NewName "SLOT 4 Port 1-MGMT2"
Get-NetAdapter -Physical | where {$_.macaddress -eq "33-4D-78-97-8A-E4"} | Rename-NetAdapter -NewName "SLOT 3 Port 2-SAN1" 
Get-NetAdapter -Physical | where {$_.macaddress -eq "33-4D-78-68-76-81"} | Rename-NetAdapter -NewName "SLOT 4 Port 2-SAN2"

I don’t remove the Slot and Port number to identify the PCI slot and port used by the adapter.

Create LBFO Teaming

Depend on your setting, you can use LACP (which requires the configuration on networking switches), or use SwitchIndependent (whic doesn’t require any configuration on the networking swtiches)

For LACP

New-NetLbfoTeam -Name "Team0" -TeamMembers "SLOT 3 Port 1-MGMT1","SLOT 4 Port 1-MGMT2" -TeamingMode LACP -LoadBalancingAlgorithm HyperVPort

For Switch Idenpendent

New-NetLbfoTeam -Name "Team0" -TeamMembers "SLOT 3 Port 1-MGMT1","SLOT 4 Port 1-MGMT2" -TeamingMode SwitchIndependent

To check the Team interface and its adapter members, use the following:

Get-NetLbfoTeam
Get-NetLbfoTeam | Get-NetLbfoTeamMember

Create another TeamNIC inteface for Heartbeat

Add-NetLbfoTeamNIC -Team "Team1" -VlanID 102

In case you need to remove the member due to wrong, use the following

Remove-NetLbfoTeamMember -Name "SLOT 3 Port 1-MGMT2" -Team "Team0"

To Add it back, use the following:

Add-NetLbfoTeamMember -Name "SLOT 3 Port 1-MGMT1" -Team "Team0"

Attention! This is strongy recomended to disable the IPv6, and DNS register on the SAN interfaces ensuring the management traffic is not sent through the SAN interfaces.

Disable IPV6

Get-NetAdapter *SAN* | Get-NetAdapterBinding -ComponentID ms_tcpip6
Disable-NetAdapterBinding –InterfaceAlias “Ethernet” –ComponentID ms_tcpip6

Disable DNS Register NIC

Get-NetAdapter *SAN* | Get-DNSClient
Set-DNSClient –RegisterThisConnectionsAddress $False

Enable Jumbo Frame:

Enable the jumbo frame on SAN interfaces only

Don’t enable the jumbo frame on Management, or HeartBeat interfaces as management interfaces requires to comunicate with Active Directory server. Enabling this Jumbo on management interfaces can cause server comunication breaks.

# Set Jumbo Frame.
Get-NetAdapter *SAN* | Set-NetAdapterAdvancedProperty –RegistryValue “9014”
# Show Jumbo Status.
Get-NetAdapter *SAN* | Get-NetAdapterAdvancedProperty -DisplayName "Jumbo*"

Re-IP server

Lets continue to set the IP:

# Management
New-NetIPAddress -InterfaceAlias "Team0" -IPAddress 10.0.10.11 -PrefixLength 24 -DefaultGateway 10.0.10.1
# HeartBeat
New-NetIPAddress -InterfaceAlias "Team1" -IPAddress 10.0.11.11 -PrefixLength 24
# SAN1
New-NetIPAddress -InterfaceAlias "SAN1" -IPAddress 10.0.8.11 -PrefixLength 27
# SAN2
New-NetIPAddress -InterfaceAlias "SAN2" -IPAddress 10.0.8.21 -PrefixLength 27

In case you need to remove the IP due to wrong, or plan to change the IP, you can use the following command to remove the IP:

Remove-NetIPAddress -IPAddress 10.0.10.11 -DefaultGateway 10.0.10.1

Enable RDP

You can use sconfig to enable the RDP, or, use the following command:

Set-ItemProperty -Path 'HKLM:\System\CurrentControlSet\Control\Terminal Server' -name "fDenyTSConnections" -value 0

Ensure that the server’s IP is accesible from the network before continue the steps:

Join Domain

Join the server to the domain. This step requires reboot.

# For Single Server:
Add-computer –domainName "domain.com"  -restart
# For multiple Severs which will be a member of the cluster, create a notepad, and put all the server to it. 
$computers = Get-Content -Path c:\it\computers.txt
Add-Computer -ComputerName $computers -Domain "YourDomainName" -Restart
# In case you need to Unjoin it first, use the following:
Remove-Computer -UnjoinDomaincredential Domain01\Admin01 -PassThru -Verbose -Restart

Install Hyper-V

Because this server will be used to host Virtual Machines, you nee to install the Hyper-V roles. This steps requires reboot.

Install-WindowsFeature -Name Hyper-V -IncludeAllSubFeature -IncludeManagementTools -Restart
Get-WindowsFeature Hyper-V

Attention: On Windows 2022 environment, Do not install the Hyper-V before patching the server to the latest patches as it can break the RDP connection.

Create Teamed Virtual Switch

Create a Trunk virtual switches inteface that can be used by the VMs:

New-VMSwitch  -Name "VSwitch-TRUNK" -AllowNetLbfoTeams $true -AllowManagementOS $true -netadaptername "Team0"

# In case there's a demand of the VMs to be able tp comunicate with the SAN storage, create a new VMSwitch pointed to the SAN interface
New-VMSwitch  -Name "SAN1" -AllowManagementOS $true -netadaptername "VSwitch-SAN1"
New-VMSwitch  -Name "SAN2" -AllowManagementOS $true -netadaptername "VSwitch-SAN2"

Install Failover cluster Roles

Install-WindowsFeature -Name Failover-Clustering -IncludeManagementTools

Install Multipath IO and reboot

Install-WindowsFeature -Name 'Multipath-IO'

Configure the MPIO SupportHW

Depending on the storage / SAN you are using, you will need to register the SAN name to MPIO hardware. If you are using HP Nimble, Dell Equalogic, you don’t to register it manually as after you install the toolkit, typicall it will register it automatically. On this article, I used a PureStorage SAN which needs to add it manually.

# Adding Pure to MPIO Device
New-MSDSMSupportedHw -VendorId PURE -ProductId FlashArray
Get-MSDSMSupportedHw
# Removing default Vendor from MPIO Device
Remove-MSDSMSupportedHw -VendorId 'Vendor*' -ProductId 'Product*'
#Configuring MPIO timer setting
Set-MPIOSetting -NewPathRecoveryInterval 20 -CustomPathRecovery Enabled -NewPDORemovePeriod 30 -NewDiskTimeout 60 -NewPathVerificationState Enabled

Tunning MPIO (For PureStorage Only)

# Default Setting
PS C:\Windows\system32> Get-MPIOSetting
PathVerificationState     : Disabled
PathVerificationPeriod    : 30
PDORemovePeriod           : 20
RetryCount                : 3
RetryInterval             : 1
UseCustomPathRecoveryTime : Disabled
CustomPathRecoveryTime    : 40
DiskTimeoutValue          : 60

# Tunning MPIO
Set-MPIOSetting -NewPathRecoveryInterval 20
Set-MPIOSetting -CustomPathRecovery Enabled
Set-MPIOSetting -NewPDORemovePeriod 30
Set-MPIOSetting -NewDiskTimeout 60
Set-MPIOSetting -NewPathVerificationState Enabled

Set-MPIOSetting -NewPathRecoveryInterval 20 -CustomPathRecovery Enabled -NewPDORemovePeriod 30 -NewDiskTimeout 60 -NewPathVerificationState Enabled

After tunning, the MPIO setting will be show as the following:

PS C:\Windows\system32> Get-MPIOSetting
PathVerificationState     : Enabled
PathVerificationPeriod    : 30
PDORemovePeriod           : 30
RetryCount                : 3
RetryInterval             : 1
UseCustomPathRecoveryTime : Enabled
CustomPathRecoveryTime    : 20
DiskTimeoutValue          : 60

Change the ISCSI service

Get-Service -Name MSiSCSI | Set-Service -Name MSiSCSI -StartupType Automatic | Start-Service

Connect the ISCSI

# Server SAN's interface IP
# CA-HVC1
$san1 = "10.0.8.11"
$san2 = "10.0.8.12"

# Target / SAN's IP
$target1 = "10.0.8.5"
$target2 = "10.0.8.6"
$target3 = "10.0.8.7"
$target4 = "10.0.8.8"

Adding ISCSI target portal

New-IscsiTargetPortal -InitiatorPortalAddress $san1 -TargetPortalAddress $target1 -InitiatorInstanceName "ROOT\ISCSIPRT\0000_0"
New-IscsiTargetPortal -InitiatorPortalAddress $san2 -TargetPortalAddress $target2 -InitiatorInstanceName "ROOT\ISCSIPRT\0000_0"
New-IscsiTargetPortal -InitiatorPortalAddress $san1 -TargetPortalAddress $target3 -InitiatorInstanceName "ROOT\ISCSIPRT\0000_0"
New-IscsiTargetPortal -InitiatorPortalAddress $san2 -TargetPortalAddress $target4 -InitiatorInstanceName "ROOT\ISCSIPRT\0000_0"

Adding ISCSI multipathing

Connect-IscsiTarget -InitiatorPortalAddress $san1 -TargetPortalAddress $target1 -IsMultipathEnabled $true -NodeAddress $targetname.NodeAddress -IsPersistent $true
Connect-IscsiTarget -InitiatorPortalAddress $san1 -TargetPortalAddress $target2 -IsMultipathEnabled $true -NodeAddress $targetname.NodeAddress -IsPersistent $true
Connect-IscsiTarget -InitiatorPortalAddress $san1 -TargetPortalAddress $target3 -IsMultipathEnabled $true -NodeAddress $targetname.NodeAddress -IsPersistent $true
Connect-IscsiTarget -InitiatorPortalAddress $san1 -TargetPortalAddress $target4 -IsMultipathEnabled $true -NodeAddress $targetname.NodeAddress -IsPersistent $true
Connect-IscsiTarget -InitiatorPortalAddress $san2 -TargetPortalAddress $target1 -IsMultipathEnabled $true -NodeAddress $targetname.NodeAddress -IsPersistent $true
Connect-IscsiTarget -InitiatorPortalAddress $san2 -TargetPortalAddress $target2 -IsMultipathEnabled $true -NodeAddress $targetname.NodeAddress -IsPersistent $true
Connect-IscsiTarget -InitiatorPortalAddress $san2 -TargetPortalAddress $target3 -IsMultipathEnabled $true -NodeAddress $targetname.NodeAddress -IsPersistent $true
Connect-IscsiTarget -InitiatorPortalAddress $san2 -TargetPortalAddress $target4 -IsMultipathEnabled $true -NodeAddress $targetname.NodeAddress -IsPersistent $true

Check all configuraton already set on all Servers.

Before you proceed further to test and build the cluster, you will need to setup the same configuration on all cluster nodes.

Perform the cluster testing

This steps is required everytime you want to build a new cluster to verify the hardware compatibility and network connection. To run all cluster validation test on the nodes, run the following:

Test-Cluster -Node CA-HVC01,CA-HVC02,CA-HVC03

To test specific cluster validation, run the following:

Test-Cluster -Node CA-HVC01,CA-HVC02,CA-HVC03 -Include "Storage", "Inventory", "Network", "System Configuration"

Build a new Hyper-V Cluster

If you find any error, warning during the test, you need to fix that prior to proceed rebuilding this cluster. If no issue found, use the following to build the cluster:

New-Cluster -Name CA-HVC0 -Node CA-HVC01,CA-HVC02,CA-HVC03 -StaticAddress 10.0.10.10

Network Design And IP Plan#

Requirement#

Server Basic Setting:#

Rename Interface#

Port Identification and configuration.#

Create LBFO Teaming#

Disable IPV6#

Disable DNS Register NIC#

Enable Jumbo Frame:#

Re-IP server#

Enable RDP#

Join Domain#

Install Hyper-V#

Create Teamed Virtual Switch#

Install Failover cluster Roles#

Install Multipath IO and reboot#

Configure the MPIO SupportHW#

Tunning MPIO (For PureStorage Only)#

Change the ISCSI service#

Connect the ISCSI#

Adding ISCSI target portal#

Adding ISCSI multipathing#

Check all configuraton already set on all Servers.#

Perform the cluster testing#

Build a new Hyper-V Cluster#