Solaris Link Aggregation

Link aggregation or IEEE 802.1AX-2008, is a computer networking term which describes using multiple network cables/ports in parallel to increase the link speed beyond the limits of any one single cable or port, and to increase the redundancy for higher availability.

Most implementations now conform to what used to be clause 43 of IEEE 802.3-2005 Ethernet standard, usually still referred to by its working group name of “IEEE 802.3ad”. The Link Aggregation definition has since been moved to a standalone IEEE 802.1AX standard. (wikipedia.org)


Here’s the sample of setup aggregation link between :

e1000g0 and e1000g1
e1000g2 and e1000g3

Make sure your eeprom’s local-mac-address? variable is set to true (you don’t need this for x86):

# eeprom local-mac-address?
local-mac-address=false

# eeprom local-mac-address? = true

# eeprom local-mac-address?
local-mac-address?=true

1. Unplumb the interfaces to be aggregated:
# ifconfig e1000g0 down unplumb
# ifconfig e1000g1 down unplumb
# ifconfig e1000g2 down unplumb
# ifconfig e1000g3 down unplumb

2. Create a link-aggregation group with key 1 and 2:

{format= dladm create-aggr -d

interface Represents the device name of the interface to become part of the aggregation.
key Is the number that identifies the aggregation. The lowest key number is 1. Zeroes are not allowed as keys.

{passive mode by default:
# dladm create-aggr -d e1000g0 -d e1000g1 1
# dladm create-aggr -d e1000g2 -d e1000g3 2

{active mode:
# dladm create-aggr -l active -d e1000g0 -d e1000g1 1
# dladm create-aggr -l active -d e1000g2 -d e1000g3 2

*please note that, if you configure LACP passive mode on your switch, you have to configure active mode on your server. if both in passive mode, they do not exchange LACP packets. check your configuration with “dladm show-aggr -L” command.

# dladm show-link aggr1
# dladm show-link aggr2

3. Plumb up the interface aggrkey, add ip address:

# ifconfig aggr1 plumb
# ifconfig aggr2 plumb

# ifconfig aggr1 192.168.1.1 netmask 255.255.255.0 up
# ifconfig aggr2 192.168.1.2 netmask 255.255.255.0 up

4. Show link aggregation status:

# dladm show-aggr >>check the status
# dladm show-aggr -s >>to display statistics
# dladm show-aggr -L >>to display LACP specific information

5. Make the IP configuration of the link aggregation persist across reboots

create “/etc/hostname.aggrkey” file
# vi /etc/hostname.aggr1
192.168.1.1
# vi /etc/hostname.aggr2
192.168.1.2

# dladm modify-aggr -t -l passive 1 {change aggr1 to passive mode; temporary only]
# dladm modify-aggr -t -l active 1 {change aggr1 to active mode; temporary only]

for further reference:

http://docs.sun.com/source/820-3084-10/link_aggregation.html

Read the rest of this entry...

Link aggregarion and Logical Interfaces

The two ends of the network interface configuration spectrum are -

"Link aggregation, or IEEE 802.3ad, is a computer networking term which describes using multiple Ethernet network cables/ports in parallel to increase the link speed beyond the limits of any one single cable or port, and to increase the redundancy for higher availability." (ref: http://en.wikipedia.org/wiki/Link_aggregation)

- and -



Link Aggregation and Logical Interfaces

Logical interfaces allow you to assign multiple IP addresses to a single physical interface, and share the bandwidth of a single interface across multiple IP addresses.



Link Aggregation:
Over the years there have been a couple ways to configure link aggregation on Solaris, the current method is via the 'dladm' command. Here's a script to detect all physical interfaces on your host and create a single aggregated link assigned the name 'aggr1'.

#!/bin/sh

if [ $# -lt 1 ]; then
echo "Usage: ./trunk.sh [ip address]"
exit 1
fi

interfaces=`dladm show-link | grep -v LINK | awk '{print $1}'`
dladm_cmd="dladm create-aggr -l passive"
for i in $interfaces; do
dladm_cmd="${dladm_cmd} -d $i"
done
dladm_cmd="${dladm_cmd} 1"

echo "
To enable trunking, execute the following commands:

${dladm_cmd}
ifconfig aggr1 plumb ${1}/24 up
dladm show-aggr
echo \"${1}\" > /etc/hostname.aggr1
svcadm disable network/physical:nwam
svcadm enable network/physical:default
"
exit 0


Examples of dladm command output used in the script:
# dladm show-link
LINK CLASS MTU STATE OVER
e1000g4 phys 1500 up --
e1000g0 phys 1500 up --
e1000g2 phys 1500 unknown --
e1000g1 phys 1500 up --
e1000g3 phys 1500 unknown --
e1000g5 phys 1500 up --

Create link aggregation named 'aggr1', using interfaces e1000g1, e1000g4 and e1000g5:
# dladm create-aggr -l passive -d e1000g1 -d e1000g4 -d e1000g5 1


Show status of aggr1 interface:
# dladm show-aggr
LINK POLICY ADDRPOLICY LACPACTIVITY LACPTIMER FLAGS
aggr1 L4 auto passive short -----

Note that to use this feature you must coordinate with you network switch administrator. The policy and and aggregated interfaces must configured identically on the other end of the ethernet cables ...


We use the Link Aggregation Control Protocol (LACP) in passive mode to control simultaneous transmission on multiple interfaces. Any single stream is transmitted completely on an individual interface, but multiple simultaneous streams can be active across all interfaces.

Logical Interfaces:
A single physical interface can be assigned multiple IP addresses - you may have already seen this in my earlier post on zones, where I configured a set of zones to share a single physical network interface.


Use /network/physical:default service to manage network interfaces:
# svcadm disable svc:/network/physical:nwam
# svcadm enable svc:/network/physical:default


Example: Configure a second address on interface bge0
# ifconfig bge0
bge0: flags=201000843 mtu 1500 index 2
inet 192.168.1.200 netmask ffffff00 broadcast 192.168.1.255

# ifconfig bge0:1 plumb
# ifconfig bge0:1 172.0.1.111/24 up
# ifconfig bge0:1
bge0:1: flags=201000843 mtu 1500 index 2
inet 172.0.1.111 netmask ffffff00 broadcast 172.0.255.255


That was simple ! Your bge0 interface now participates on two subnets: 192.168.1.0/24 and 172.0.1.0/24.


To make this bge0:1 configuration permanent, create the file /etc/hostname.bge0:1 containing the ip address 172.0.1.111. Your logical interface will be initialized each time you boot your system.

Read the rest of this entry...