Solaris - Configuring link-based IPMP

When the new Solaris servers turned up at my place of work we didn't have anyone trained on Solaris to install the OS and configure the machines - so as part of the deal we paid for a number of contractors to come in and complete this work. One of the tasks I wanted them to do was to configure resilient network connections 2 x 1Gbe LAN and 2 x 10Gbe Storage LAN on most servers.

The lead Engineer then started talking about IPMP.
According to the speil from the manufacturer it does the following:

1. Eliminates a single network adapter as a single point of failure in the the following cases
1. Network adapter failure
2. Network link failure
2. It enables interfaces to fail over within approximately 10 seconds (default configuration) - value can be adjusted in the /etc/default/mpathd file.
3. It can be configured for use with both IPv4 and IPv6.
4. It enables interfaces to be configured as standby interfaces.

There are two different ways of configuring IPMP:

• probe-based - utilises test addresses to monitor the health of the interfaces.
• link-based - the interface kernel driver utilises "Link Up"/"Link Down" status of the interface to monitor interface state of health.

Prep Work

1. View the /etc/hosts file - edit if necessary
2. View the /etc/netmasks file - edit if necessary

NOTE: I've put this step in as you may have two sets of IPMP to setup (which I have) which will require adding additional IP addess and subnet mask.

By following these steps you can setup link-based IPMP with 2 interfaces.

1. Discover network interfaces

Run dladm show-link

(A type of non-vlan or vlan indicates that the hardware is GLDv3 compliant hardware).

For this example I'll be using bge0 and bge2 as the IPMP pairing.

2. Hostname files

Create, or amend, the appropriately named hostname files, add the text listed below (ipmp0 is the IPMP unique group name).

• Check that files exist

   ls -al /etc | grep host

• If they don't create them

   touch /etc/hostname.bge2

• vi /etc/hostname.bge0

   <hostname> netmask + broadcast + group ipmp0 up

• vi /etc/hostname.bge2

   group ipmp0 up

3. Bringing the cards up

• ifconfig bge0 plumb
• ifconfig bge2 plumb
• ifconfig bge0 `cat /etc/hostname.bge0` up
• ifconfig bge2 `cat /etc/hostname.bge2` up

4. Test the configuration

Bring the primary NIC down (bge0) to check the IPMP configuration.

From a console session run and leave it running:
tail -f /var/adm/messages

From a second console run the command below which will disable bge0:
if_mpadm -d bge0

In the console session running the message log output you should have an entry stating a successful fail over has occurred.

Run the following from the 2nd console to enable bge0:
if_mpadm -r bge0

Looking back at the message log console another entry should appear stating that the IPMP setting has failed back to bge0 - CTRL & C to exit

5. Complete

Configuration is now complete