Search This Blog

Friday, January 29, 2016

RHEL 6 - How to configure bonding/teaming

All the system admin would like to avoid server outage by having redundancy for root filesystem using mirroring,  Multiple FC links to SAN with help of multi-pathing and many more.So here the question is how do you provide redundancy in network level ? Having a multiple network card will not give any redundancy.In redhat Linux you need to configure bonding to accomplish the network level redundancy.Once you have configured the bonding/teaming by using two NIC cards, kernel will automatically detect the failure of any NIC and work smartly according to that without any riot.Bonding can be used for load sharing as well between two physical links.

The below diagram will explain how the bonding is happening.



Operating system used: Redhat Linux 6.3
NIC Details:
[root@server2 /]# ifconfig -a|grep eth
eth0      Link encap:Ethernet  HWaddr 00:0C:29:C0:16:56 
eth1      Link encap:Ethernet  HWaddr 00:0C:29:C0:16:6A 
eth2      Link encap:Ethernet  HWaddr 00:0C:29:C0:16:60 
eth3      Link encap:Ethernet  HWaddr 00:0C:29:C0:16:74 

Goal:
Configure bonding between eth2 and eth3 with name of bond0.
Step 1:
1. Network manager should not be running on the system, as NIM doesn’t support bonding. So please stop and disable the networkmanager service.

[root@server2 Desktop]# service NetworkManager status
NetworkManager (pid  2157) is running...
[root@server2 Desktop]# service NetworkManager stop
Stopping NetworkManager daemon:                            [  OK  ]
[root@server2 Desktop]# service NetworkManager status
NetworkManager is stopped
[root@server2 Desktop]#

Step 2:
Add the below line in /etc/modprobe.conf to load the bonding module in to kernel.

alias bond0 bonding

In redhat 6.3, you need to create new file called “bonding.conf” under /etc/modprobe.d/ with below mentioned line.

[root@server2 /]# cat > /etc/modprobe.d/bonding.conf
alias bond0 bonding

Step 3:
Now time to create a bonding interface configuration file in /etc/sysconfig/network-scripts/ directory like the below one.

[root@server2 network-scripts]# pwd
/etc/sysconfig/network-scripts

[root@server2 network-scripts]# cat ifcfg-bond0
#This is congiguration file for bond0.Used NIC's eth2 & eth3
DEVICE=bond0
IPADDR=192.168.10.25
NETMASK=255.255.255.0
USRCTL=no
ONBOOT=yes
BOOTPRO=none
BONDING_OPTS="mode=0 miimon=100"

Step:3
Create a configuration files under “/etc/sysconfig/network-scripts/” for network interfaces if not exists.If exists,have a contents like the below one.

[root@server2 network-scripts]# cat ifcfg-eth2
DEVICE=eth2
HWADDR=00:0C:29:C0:16:60
BOOTPRO=none
ONBOOT=yes
MASTER=bond0
SLAVE=yes
USERCTL=no


[root@server2 network-scripts]# cat ifcfg-eth3
DEVICE=eth3
HWADDR=00:0C:29:C0:16:74
BOOTPRO=none
ONBOOT=yes
MASTER=bond0
SLAVE=yes
USERCTL=no

Note: Do not copy paste the content from above output.MAC and DEVICE name will differ for each system.


Step:4
Now restart the network service to load the configuration.
Note:Do not restart the network service without server maintenance window.

[root@server2 network-scripts]# service network restart
Shutting down interface eth2:  /etc/sysconfig/network-scripts/ifdown-eth: line 121: /sys/class/net/bond0/bonding/slaves: No such file or directory
                                                           [  OK  ]
Shutting down interface eth3:  /etc/sysconfig/network-scripts/ifdown-eth: line 121: /sys/class/net/bond0/bonding/slaves: No such file or directory
                                                           [  OK  ]
Shutting down loopback interface:                          [  OK  ]
Bringing up loopback interface:                            [  OK  ]
Bringing up interface bond0:  Determining if ip address 192.168.10.25 is already in use for device bond0...
                                                           [  OK  ]
[root@server2 network-scripts]#

Step:5
Verify whether “bond0” has come up with IP or not .

[root@server2 /]# ifconfig -a
bond0     Link encap:Ethernet  HWaddr 00:0C:29:C0:16:60 
          inet addr:192.168.10.25  Bcast:192.168.10.255  Mask:255.255.255.0
          inet6 addr: fe80::20c:29ff:fec0:1660/64 Scope:Link
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:142 errors:0 dropped:0 overruns:0 frame:0
          TX packets:18 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:8556 (8.3 KiB)  TX bytes:1236 (1.2 KiB)

eth2      Link encap:Ethernet  HWaddr 00:0C:29:C0:16:60 
          inet6 addr: fe80::20c:29ff:fec0:1660/64 Scope:Link
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:1395 errors:0 dropped:0 overruns:0 frame:0
          TX packets:19 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:86730 (84.6 KiB)  TX bytes:1898 (1.8 KiB)
          Interrupt:16 Base address:0x2080

eth3      Link encap:Ethernet  HWaddr 00:0C:29:C0:16:60 
          inet6 addr: fe80::20c:29ff:fec0:1660/64 Scope:Link
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:1390 errors:0 dropped:0 overruns:0 frame:0
          TX packets:15 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:86384 (84.3 KiB)  TX bytes:1578 (1.5 KiB)
          Interrupt:18 Base address:0x2480

[root@server2 /]#

In the above output,you can see NIC eth2 and eth3 have flag “SLAVE” and interface “bond0” has flag MASTER.Another thing you note both, the NIC interface will show same MAC address.

Step:6
Performing the live test to ensure bonding is providing the fault tolerance.
First i am removing the LAN cable from eth3 and let see what happens.

[root@server2 /]# ifconfig -a
bond0     Link encap:Ethernet  HWaddr 00:0C:29:C0:16:60 
          inet addr:192.168.10.25  Bcast:192.168.10.255  Mask:255.255.255.0
          inet6 addr: fe80::20c:29ff:fec0:1660/64 Scope:Link
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:5012 errors:0 dropped:0 overruns:0 frame:0
          TX packets:39 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:312100 (304.7 KiB)  TX bytes:2682 (2.6 KiB)


eth2      Link encap:Ethernet  HWaddr 00:0C:29:C0:16:60 
          inet6 addr: fe80::20c:29ff:fec0:1660/64 Scope:Link
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:3830 errors:0 dropped:0 overruns:0 frame:0
          TX packets:29 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:238520 (232.9 KiB)  TX bytes:2534 (2.4 KiB)
          Interrupt:16 Base address:0x2080

eth3      Link encap:Ethernet  HWaddr 00:0C:29:C0:16:60 
          BROADCAST SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:3825 errors:0 dropped:0 overruns:0 frame:0
          TX packets:26 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:238138 (232.5 KiB)  TX bytes:2388 (2.3 KiB)
          Interrupt:18 Base address:0x2480

[root@server2 /]#

Still the bond0 interface is UP and RUNNING fine.At the same time,”RUNNING” flag has disappear from eth3.

Now i have connected LAN cable back to eth3 and pulling out from eth2.

[root@server2 /]# ifconfig -a
bond0     Link encap:Ethernet  HWaddr 00:0C:29:C0:16:60 
          inet addr:192.168.10.25  Bcast:192.168.10.255  Mask:255.255.255.0
          inet6 addr: fe80::20c:29ff:fec0:1660/64 Scope:Link
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:5115 errors:0 dropped:0 overruns:0 frame:0
          TX packets:39 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:318376 (310.9 KiB)  TX bytes:2682 (2.6 KiB)

eth2      Link encap:Ethernet  HWaddr 00:0C:29:C0:16:60 
          BROADCAST SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:3926 errors:0 dropped:0 overruns:0 frame:0
          TX packets:29 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:244376 (238.6 KiB)  TX bytes:2534 (2.4 KiB)
          Interrupt:16 Base address:0x2080

eth3      Link encap:Ethernet  HWaddr 00:0C:29:C0:16:60 
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:3832 errors:0 dropped:0 overruns:0 frame:0
          TX packets:26 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:238558 (232.9 KiB)  TX bytes:2388 (2.3 KiB)
          Interrupt:18 Base address:0x2480

Still “bond0” interface running with UP & RUNNING flag.So you have successfully configured bonding on Redhat Linux 6.

To see complete bonding info,use below command.

[root@server2 /]# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: load balancing (round-robin)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0

Slave Interface: eth2
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 1
Permanent HW addr: 00:0c:29:c0:16:60
Slave queue ID: 0

Slave Interface: eth3
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 1
Permanent HW addr: 00:0c:29:c0:16:74
Slave queue ID: 0

To verify the current bonding mode,use below command. 

[root@server2 /]# cat /sys/class/net/bond0/bonding/mode
balance-rr 0

You can modify the bonding mode by editing “mode” in the ifcfg-bond0  configuration file.

[root@server2 /]# cat /etc/sysconfig/network-scripts/ifcfg-bond0 |grep -i mode
BONDING_OPTS="mode=0 miimon=100"

Policy Name
Code
Description
balance-rr
0
Round-Robin policy for fault tolerance
active-backup
1
Active-Backup policy for fault tolerance
balance-xor
2
Exclusive-OR policy for fault tolerance
broadcast
3
All tansmissions are sent on all slave interfaces
802.3ad
4
Dynamic Link Aggregation Policy
balance-tlb
5
Transmit Load Balancing Policy for fault tolerance
balance-alb
6
Active load Balancing Policy for fault tolerance

To list the currently configured bonds,

[root@server2 /]# cat /sys/class/net/bonding_masters
bond0


Thank you for reading.
For Reading other article, visit to “https://sites.google.com/site/unixwikis/

No comments:

Post a Comment