Author: Ben

Carry out multi-channel network via Linux Bonding and Cisco Etherchannel

Preface

Linux kernel supporting functions are too numerous to enumerate. You can experience the limitless power of Linux from the following instance. Recently a student who serves MIS management asked me a question, “In the same LAN how much money do I need to pay and what networking devices do I need to prepare in developing one host which has more bandwidth than surrounding PCs?”. The answer is pretty easy! Several NICs that cost less than NT$1000 can make you easily increase bandwidth of Linux host.

Actually such issue of how to increase bandwidth via the combination of several NICs based on Linux system is often raised for lab practice in class previously. In this document we will demonstrate how to increase networking transit rate with the combination of NICs and corresponding bandwidth. Also we can make such device support networking redundancy.

Equipment

1. Linux machine, two Ethernet NICs above
2. Cisco 3550 Switch

Linux kernel & firmware

1. Linux kernel version 2.4.32, bonding command: ifenslave
2. Cisco 3550 Switch firmware: c3500xl-c3h2s-mz.120-5.WC15.bin

Essential knowledge

1. Linux 802.1Q configuration (see Linux Guide Issue 11)
2. Etherchannel and 802.1Q configuration on Cisco 3550 Switch

Definition

Bonding refers to combining networking bandwidth via software manipulation (Linux bonding technique). If every NIC transit rate is 100Mb, for instance, we will obtain one NIC with 400Mb/s transit rate by combining four NICs.

How to make Linux support bonding

There are some required conditions for Linux bonding function support and user allowance of bonding command implementation:

1. Linux kernel should support bonding
   In the kernel option you can choose “Bonding driver support” in the menu of “Network Device Support”. Here we want to remind you of choosing kernel version 2.4.32 because the bonding function of earlier Linux kernel version does not support 802.3ad protocol that we will use lately. Additionally, please notice that 802.3ad protocol is named as PAgP in Linux bonding configuration while LACP in Cisco Switch configuration.
2. Ifenslave command is necessary
   Ifenslave is the only command of bonding configuration and highly depended on kernel version. If you use kernel version 2.4.32, you should be able to find ifenslave source code: ifenslave.c under /Documentation/networking. You can also find ‘bonding.txt’ in the same directory, which simply describes what functions that current Linux version supports.

Thus, we can compile ifenslave command that is appropriate for kernel version 2.4.32.

So far we ensure that system kernel version 2.4.32 supports bonding and corresponding ifenslave command.

Introduction to Cisco port aggregation control protocol

LACP (Link Aggregation Control Protocol) is one way of the Cisco Switch port aggregation and also a part of IEEE 802.3ad protocol. This protocol regulates that different physical port on Switch can be connected logically and further regarded as one physical line. In addition to LACP, Cisco device also has other similar functions, e.g. Etherchannel and PAgP (Port Aggregation Protocol).

Objective

In this experiment we need to make a connection between Linux Server and Cisco Catalyst 3550 Switch via 802.3ad protocol in achieving port aggregation.

This experiment is focusing on OSI Layer 2 connection and the network topology is shown above. The procedure is as follows:

1. Linux Server configuration
2. Cisco Switch configuration
3. Networking test
4. Network redundancy test

1.Linux Server configuration
First, we already have a Linux machine with kernel version 2.4.32 and compiled ifenslave command which saved in /root. Then we also have a four-port NIC where each port has its independent MAC address so they can be regarded as four subdivided NICs which are named as eth2, eth3, eth4 and eth5 respectively.

After ensuring the NIC usability, we need to develop a virtual aggregation interface (bond0) so as to aggregate these four interfaces. First, we have to insert the bonding kernel module and then inform the kernel about bonding mode. The current kernel version 2.4.32 supports seven bonding modes and 802.3ad is the fourth one. Thus, we need to add either ‘mode=802.3ad’ or ‘mode=4’ module argument when inserting bonding module. Additionally, we can add a argument ‘miimon=(number)’ in monitoring each port connection. At the following figure we can realize the whole process of bonding module placement.

By running ‘ifconfig –a’ we can see the NIC aggregation interface named as bond0 in which we have to start it and set its IP address.

At the following we need to run ‘./ifenslave –v bond0 eth2 eth3 eth4 eth5’ in aggregating each physical NIC into bond0.

So far we have completed port aggregation. However, we should remind you of confirming every configuration you’ve done. Everyone should know /proc in Linux OS is a virtual file system which includes all the contents of procedure and the supporting functions and devices of kernel. Thus, the kernel supporting list demonstrates related interface codes in /proc/net/bonding when bonding function is incorporated. We can observe the file content /proc/net/bonding/bond0 in realizing the bond0 supporting protocol and function of port aggregation and how many physical NICs are connected.

Run ifconfig in ensuring all interfaces are included after observing file content.

So far we have developed Linux Server but Cisco 3550 Switch not yet. Therefore, when we run ‘ping 192.168.10.2’, packet cannot be transferred to Router because 802.3ad protocol is not incorporated in 3550 Switch so that the communication protocol on either side is different in resulting data transferring failure.

2.Cisco Switch configuration
At this stage we use FastEthernet port 13-16 corresponding to eth2, eth3, eth4 and eth5 on Linux Server. On the IOS interface of Cisco Switch we can simultaneously run the same command toward selected ports by using ‘range’. As the following figure shows, we can utilize range command to enter port configuration mode and the procedure is as follows:

1. Select VLAN ID
2. Port mode configuration
3. Incorporate 802.3ad protocol support, i.e. LACP

Then run ‘show running’ command to observe the configuration status of FastEthernet port 13-16. At last Cisco Switch configuration completes.

3.Networking test
At the previous stages we have accomplished 802.3ad protocol configuration on Linux Server and Cisco Switch. In the following figure we prove that the configuration is workable via ping between Linux Server and Router on the other side.

4.Network redundancy test
Another advantage of port aggregation is providing network redundancy. In this experiment, for instance, we use four ports connecting to Switch. We should obtain bandwidth with 400Mb/s transit rate in theory and such networking mechanism also provides redundancy possibility. Among these four paths, if one or two are disconnected, 802.3ad protocol will detect such networking status and make the other two workable paths replace the work.

In making readers can observe such status, first we run ping from Linux Server to Router then disconnect eth2 and eth3 when the 4th icmp answers. From the output of running ‘mii-tool’, we can see eth2 and eth3 are disconnected so that the packets via eth2 and eth3 cannot reach Router. Nevertheless, after one minute or more, networking has recovered after the 77th icmp packet transmitted.

Conclusion

No doubt Linux has the most powerful function because it provides one networking & server integration environment for enterprise via coherent protocol in achieving perfect combination with commercial network device via the most economic, stable and convenient way.

Due to the page limitation, we cannot demonstrate all the details but much more foci on fundamental development. The instance in this document is only a small part of Linux application. Linux always has powerful support in any OSI layers and too numerous too enumerate. The last but not the least, the most important thing is whether readers can discover and realize its extensive application.