Thursday, November 24, 2011

NIC Bonding in Linux


Linux allows binding multiple network interfaces into a single channel/NIC using special kernel module called bonding. The behavior of the bonded interfaces depends upon the mode; generally speaking, modes provide either hot standby or load balancing services. Additionally, link integrity monitoring may be performed.

Step 1: Create a bond0 configuration file

# vi /etc/sysconfig/network-scripts/ifcfg-bond0
Append following lines to it:
DEVICE=bond0
IPADDR=192.168.0.100
NETWORK=192.168.0.0
NETMASK=255.255.255.0
USERCTL=no
BOOTPROTO=none
ONBOOT=yes
Replace above IP address with your actual IP address. Save file and exit to shell prompt.

Step 2: Modify eth0 and eth1 config files:

# vi /etc/sysconfig/network-scripts/ifcfg-eth0
Modify/append directive as follows:
DEVICE=eth0
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none
Save file and exit to shell prompt.

Step 3: Load bond driver/module

Make sure bonding module is loaded when the channel-bonding interface (bond0) is brought up. You need to modify kernel modules configuration file:
# vi /etc/modprobe.conf
Append following two lines:
alias bond0 bonding
options bond0 mode=1 miimon=1000              ###Note: mode 1 is “active-backup”
{Please consider removing the line 'options bonding miimon=100 mode=0' from modprobe.conf and place this line in ifcfg-bond0
BONDING_OPTS="miimon=100 mode=1"
RHEL5 and later support setting the bonding options in the ifcfg-bondX file, not in modprobe.conf.}

Step 4: Probe bonding module and test configuration


# /etc/init.d/network stop
# modprobe -r bonding
# lsmod |grep -i bond
# modprobe bonding

Restart networking service in order to bring up bond0 interface:

# /etc/init.d/network restart

If you get the below output, then everything is fine.

# cat /proc/net/bonding/bond0

Bonding Mode: active-backup
MII Status: up
MII Polling Interval (ms): 0
Up Delay (ms): 0
Down Delay (ms): 0
 
Slave Interface: eth0
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:00:00:00:00:00
 
Slave Interface: eth1
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:00:00:00:00:00

===============================================================

Options for mode types:

You can set up your bond interface according to your needs. In order to do this, you simply change the mode type depicted in the examples below (mode=X). There are seven mode types available. They are as follows:
mode=0
This mode uses the Round-robin policy: Transmit packets in sequential order from the first available slave through the last. This mode provides load balancing and fault tolerance.
mode=1
This mode uses an Active-backup policy: Only one slave in the bond is active. A different slave becomes active if, and only if, the active slave fails. The bond's MAC address is externally visible on only one port (network adapter) to avoid confusing the switch. This mode provides fault tolerance. The primary option affects the behavior of this mode.
mode=2
Transmit based on [(source MAC address XOR'd with destination MAC address) modulo slave count]. This selects the same slave for each destination MAC address. This mode provides load balancing and fault tolerance.
mode=3
Broadcast policy: transmits everything on all slave interfaces. This mode provides fault tolerance.
mode=4
IEEE 802.3ad Dynamic link aggregation. Creates aggregation groups that share the same speed and duplex settings. Utilizes all slaves in the active aggregator according to the 802.3ad specification.
          *Pre-requisites:
1. Ethtool support in the base drivers for retrieving the speed and duplex of each slave.
2. A switch that supports IEEE 802.3ad Dynamic link aggregation. Most switches will require    some type of configuration to enable 802.3ad mode
mode=5
Adaptive transmit load balancing: channel bonding that does not require any special switch support. The outgoing traffic is distributed according to the current load (computed relative to the speed) on each slave. Incoming traffic is received by the current slave. If the receiving slave fails, another slave takes over the MAC address of the failed receiving slave.
*Prerequisite: Ethtool support in the base drivers for retrieving the speed of each slave.
mode=6
Adaptive load balancing: includes balance-transmit load balancing plus receive load balancing for IPV4 traffic, and does not require any special switch support. The receive load balancing is achieved by ARP negotiation. The bonding driver intercepts the ARP Replies sent by the local system on their way out and overwrites the source hardware address with the unique hardware address of one of the slaves in the bond such that different peers use different hardware addresses for the server.