Bonding/teaming internet connections

So, perhaps you are too far from your local exchange to get a decent DSL speed, or you have an urgent large download and want to use a secondary internet connection to boost your speed. This article will allow you to combine 2 connections together. No ISP support is required. Connections could be all DSL, 2.4Ghz wireless, or HSDPA or any combination of, indeed any type of internet connection (latency permitting).

There are a number of expensive products on the market which do this. I want to bring this into the realms of the sys admin/computer hobbyist, after all this is the information age.

In this example I am using 1 DSL connection, with 1 HSDPA connection, but assuming you know enough to be able to implement this you should be able to change interface names or implement for more than 2 connections if you want.

You will need:-

A computer of some sort with Ubuntu loaded on it (only for the purposes of this article, could easily be any other distro if you are prepared to adapt it).
A dedicated server of some sort on the internet with Ubuntu (again another distro if you want to adapt, I can recommend a Kimsufi 2G for €14.99 per month ex VAT, can’t get better value than that)
2 internet connections, obviously.

We will be using a VPN and multi-link PPP to bundle the 2 connections together. The traffic will go through the dedicated server onto the net as follows:-

LAN <-> Masquerading/SNAT <-> VTUN <-> multi-link PPP <-> VTUN <-> SNAT <-> Internet.

First of all we have to setup policy routing so that the 2 VPN tunnels go through the correct interfaces, otherwise this is all a waste of time. We are going to setup 2 routing tables 1 for each connection.

/etc/iproute2/rt_tables

#
# reserved values
#
255     local
254     main
253     default
0       unspec
#
# local
#
#1      inr.ruhep

# Routing table for DSL connection
101     dsl-connection
# Routing table for HSDPA connection
102     hsdpa-connection

We then have to remove any default route/gateway from the network setup and add the policy routing rule which will tell the computer to route traffic through the correct interface.

/etc/network/interfaces

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
        address 192.168.1.254
        netmask 255.255.255.0

# Rule to route everything originating from the DSL interface through the DSL gateway
post-up ip rule add from 192.168.1.254 lookup dsl-connection
# Gateway for DSL connection
post-up ip route add default via 192.168.1.252 table dsl-connection

The following file is the setup for the HSDPA connection (in my case to Three), adjust accordingly for your own provider. Refer to http://johnlewis.ie/mobile-broadband-from-the-command-line-in-ubuntu/ for the chat script and pap-secrets.

/etc/ppp/peers/provider

user "user"

connect "/usr/sbin/chat -v -f /etc/chatscripts/pap -T *99#"

# Serial device to which the modem is connected.
/dev/ttyUSB0

# Do not use this connection as the default route.
nodefaultroute

# Makes pppd "dial again" when the connection is lost.
persist

# no compression - ppp is used only until the modem
novj
novjccomp
nopcomp
nodeflate
noccp

The following file sets up the correct routing for the HSDPA connection once it is up.

/etc/ppp/ip-up.d/bond-route

#!/bin/sh

# Rule to route everything originating from the HSDPA interface through the HSDPA gateway
# Notice we use a network here as opposed to a singular IP address or interface name
# This is because the IP address is dynamic but always in the 10.0.0.0 range
# And also because routing rules by interface name don't appear to work at the moment
ip rule add from 10.0.0.0/8 lookup hsdpa-connection
# Gateway for HSDPA connection
# Similarly using the fake gateway assigned by pppd in the absence of provider supplied
# Succesfully routes out of the correct interface, again as interface name doesn't work
ip route add default via 10.64.64.64 table hsdpa-connection

Install Vtun and edit config as necessary.

/etc/vtund.conf

options {
  port 5000;            # Listen on this port.

  # Path to various programs
  ppp           /usr/sbin/pppd;
  ifconfig      /sbin/ifconfig;
  route         /sbin/route;
  firewall      /sbin/tables;
  ip            /sbin/ip;
}

# Default session options
default {
#  type tun;            #tun, ether, tty(default), pipe
#  proto tcp;           #udp, tcp
  compress no;          # no, yes, zlib, lzo
  encrypt no;          #yes, no
  stat yes;             #yes, no: check /var/log/vtund/SessionName_X
  speed 0;              # By default maximum speed, NO shaping
}

# PPP tunnel example.
DSL-PPP-Tunnel {
  passwd  yourpassword;     # Password
  type  tty;             # PPP tunnel.
  proto tcp;            # UDP/TCP protocol
#  compress  lzo:9;      # LZO compression level 9
  encrypt  no;         # Encryption
#  keepalive yes;       # Keep connection alive
#  persist yes;

    srcaddr {
        iface eth0;     # Use first up-/ downstream-device
    };

  up {
        # Connection is Up
        ppp "file /etc/ppp/options.vtund";
  };

  down {
        # Connection is down
  };
}

HSDPA-PPP-Tunnel {
  passwd  yourpassword;     # Password
  type  tty;             # PPP tunnel.
  proto tcp;            # UDP/TCP protocol
#  compress  lzo:9;      # LZO compression level 9
  encrypt  no;         # Encryption
#  keepalive yes;       # Keep connection alive
#  persist yes;

    srcaddr {
        iface ppp0;     # Use first up-/ downstream-device
    };

  up {
        # Connection is Up
        ppp "file /etc/ppp/options.vtund";
  };

  down {
        # Connection is down
  };
}

Here we set the options for PPP.

/etc/ppp/options.vtund

noauth
lock
debug
dump
#logfd 1
logfile /var/log/vtund.log
passive
updetach
#To Enable PPP compression Comment the following line. --START--
noccp
novj
novjccomp
nopcomp
noaccomp
#To Enable PPP compression Comment the following line. --END--

# Set the tunnel endpoint as the default gateway for the entire machine
defaultroute
# Enable multlink so this connection can be used in a bundle
mp

/etc/default/vtun

# Defaults for vtun initscript
# sourced by /etc/init.d/vtun
# Created by the maintainer scripts

#
# This is a POSIX shell fragment
#
# Should the standalone server be started?
# RUN_SERVER=no
# SERVER_ARGS="-P 5000"

# Client sessions to start. Up to ten instances can be configured.
#
# Session name
# CLIENT0_NAME=viper
#
# Destination host
# CLIENT0_HOST=vtun-server.somewhere.com.au
#
# Optional parameters
# CLIENT0_ARGS=
#
# CLIENT1_NAME=
# CLIENT1_HOST=
# CLIENT1_ARGS=

CLIENT0_NAME=DSL-PPP-Tunnel
# Real IP address of dedicated server running vtun, adjust accordingly
CLIENT0_HOST=188.165.0.43

CLIENT1_NAME=HSDPA-PPP-Tunnel
# Real IP address of dedicated server running vtun, adjust accordingly
CLIENT1_HOST=188.165.0.43

/etc/sysctl.conf

net.ipv4.ip_forward=1

/etc/rules-save

# Generated by iptables-save v1.4.10 on Thu Aug 11 20:02:56 2011
*filter
:INPUT ACCEPT [308:33472]
:FORWARD ACCEPT [150:16924]
:OUTPUT ACCEPT [259:26848]
COMMIT
# Completed on Thu Aug 11 20:02:56 2011
# Generated by iptables-save v1.4.10 on Thu Aug 11 20:02:56 2011
*nat
:PREROUTING ACCEPT [87:6487]
:INPUT ACCEPT [2:412]
:OUTPUT ACCEPT [37:2646]
:POSTROUTING ACCEPT [37:2646]
# Rule to Masquerade/NAT the local network change accordingly
-A POSTROUTING -s 192.168.1.0/24 -o ppp1 -j MASQUERADE
COMMIT
# Completed on Thu Aug 11 20:02:56 2011

/etc/rc.local

#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.

# Start HSDPA connection
pon

# Load ftp connection tracking module
modprobe nf_conntrack_ftp

# Restore  previous iptables config
iptables-restore /etc/rules-save

# Restart the local DNS caching server if you have one
service dnsmasq restart

exit 0

At the server end we have to setup Vtun, PPP and source NAT.

/etc/vtund.conf

options {
  port 5000;            # Listen on this port.

  # Path to various programs
  ppp           /usr/sbin/pppd;
  ifconfig      /sbin/ifconfig;
  route         /sbin/route;
  firewall      /sbin/tables;
  ip            /sbin/ip;
}

# Default session options
default {
#  type tun;            #tun, ether, tty(default), pipe
#  proto tcp;           #udp, tcp
  compress no;          # no, yes, zlib, lzo
  encrypt yes;          #yes, no
  stat yes;             #yes, no: check /var/log/vtund/SessionName_X
  speed 0;              # By default maximum speed, NO shaping
}

DSL-PPP-Tunnel {
  passwd  yourpassword;     # Password to authenticate the vtund client
  type  tty;            # PPP tunnel.
  proto tcp;            # Use UDP or TCP protocol
#  compress  lzo:9;      # LZO compression level 9
  encrypt  no;         # Encryption
  keepalive yes;        # Keep connection alive

  up {
        # Connection is Up
        ppp "file /etc/ppp/options.vtund";

  };

  down {
        # Connection is down
  };
}

HSDPA-PPP-Tunnel {
  passwd  yourpassword;     # Password to authenticate the vtund client
  type  tty;            # PPP tunnel.
  proto tcp;            # Use UDP or TCP protocol
#  compress  lzo:9;      # LZO compression level 9
  encrypt  no;         # Encryption
  keepalive yes;        # Keep connection alive

  up {
        # Connection is Up
        ppp "file /etc/ppp/options.vtund";

  };

  down {
        # Connection is down
  };
}

/etc/ppp/options.vtund

noauth
lock
#debug
#dump
#logfd 2
#To Enable PPP compression Comment the following line. --START--
noccp
novj
novjccomp
nopcomp
noaccomp
#To Enable PPP compression Comment the following line. --END--

# Enable multilink bundling
mp

#Set local and remote IP addresses for the PPP connection
192.168.10.1:192.168.10.2

/etc/default/vtun

# Defaults for vtun initscript
# sourced by /etc/init.d/vtun
# Created by the maintainer scripts

#
# This is a POSIX shell fragment
#
# Should the standalone server be started?
RUN_SERVER=yes
# SERVER_ARGS="-P 5000"

# Client sessions to start. Up to ten instances can be configured.
#
# Session name
# CLIENT0_NAME=viper
#
# Destination host
# CLIENT0_HOST=vtun-server.somewhere.com.au
#
# Optional parameters
# CLIENT0_ARGS=
#
# CLIENT1_NAME=
# CLIENT1_HOST=
# CLIENT1_ARGS=

/etc/rc.local

#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.

iptables -t nat -A POSTROUTING -s 192.168.10.0/24 -o eth0 -j SNAT --to 188.165.0.43

exit 0

http://www.opennet.ru/soft/vpn_table/vtund-ppp1.html
http://www.dslreports.com/forum/r20456553-MLPPP-Guide-on-Linux

Umer

“2. A dedicated server of some sort on the internet with Ubuntu” (Can this be the same computer as in 1, so we can do all this in one computer with ubuntu.)

As an ordinary user, is it possible to setup my computer(with ubuntu in it) to utilize my DSL & 3G internet connections without setting up a server. If you can guide me about using my two different internet connections (as one being used in firefox and other in chrome or opera).
John Peter Lewis

Short answer, no. You have to have a server, of some sort, on the internet, to be the other end of the tunnel/vpn. It also doesn’t work particularly well with links that drop lots of packets like 3G, aka HSPA, connections do. However, I am working on that.
Sean

Hi John,

Firstly, this is a great article and i’d like to thankyou dearly for it. I have a question, im not sure if this is possible much but, would it be possible to Link 2 Internet Connections coming into the eth0 and eth1 interfaces (two gateways obviously) . If so how would everything above have to be modified in order to get that done?

Sean
John Peter Lewis

Hi Sean,

Have you got a dedicated server on the net? If you haven’t you can’t do this.

John.
http://twitter.com/FarrisGoldstein FarrisGoldstein

I’m interested in using this sort of setup for an RV. I have all the major requirements, but I’m wondering if you know of any resources to help me make it more robust: easily support connecting/disconnecting more 3g/4g devices via tethering as their available (for collaboration among ~10 band and crew members), more fine-tuned load-balancing to make use of those flaky 3g connections you warn against.
John Peter Lewis

I’m working on that side of things. The main problem with this setup is tunnelling TCP with TCP connections. It’s never going to work well, causing “ack wars” and tunnel disconnects. l2tp overcomes these problems. In terms of fine grained control of load balancing, I’m also working on that. In a nutshell you have to tell the kernel PPP code what speed each connection so that it divides the traffic appropriately instead of just dividing it by 2/3/etc. I’m am only at dirty hack stage, and still haven’t got it working properly. Once I have I imagine there would have to be some sort of automated test every 5 minutes, when idle, to determine the relative link speeds and adjust accordingly.