Circumventing Path MTU Discovery issues with MSS Clamping (for ADSL, cable, PPPoE & PPtP users)

Righ, so you know I'm having hassles with clienbts behind my gateway. I haven't quite solved them yet, but for those of you using a gateway as a firewall to do network address translation or anything else that results in your packet size increasing, the follwoing is relevant.

From Linux.com

Path MTU Discovery doesn't work as well as it should anymore. If you know for a fact that a hop somewhere in your network has a limited (<1500) MTU, you cannot rely on PMTU Discovery finding this out.

Besides MTU, there is yet another way to set the maximum packet size, the so called Maximum Segment Size. This is a field in the TCP Options part of a SYN packet.

Recent Linux kernels, and a few PPPoE drivers (notably, the excellent Roaring Penguin one), feature the possibility to 'clamp the MSS'.

The good thing about this is that by setting the MSS value, you are telling the remote side unequivocally 'do not ever try to send me packets bigger than this value'. No ICMP traffic is needed to get this to work.

The bad thing is that it's an obvious hack - it breaks 'end to end' by modifying packets. Having said that, we use this trick in many places and it works like a charm.

In order for this to work you need at least iptables-1.2.1a and Linux 2.4.3 or higher. The basic command line is:

# iptables -A FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu

Does Windows XP Pro behind a Linux gateway running PPPoE cause disconnections?

It has taken me days to isolate this problem.

My Debian gateway (running kernel 2.6.12.2) repeatedly disconnects its PPPoE (over iBurst wireless internet) when I plug my new HP notebook running Windows XP into the network behind it. I am running rp-pppoe.so

The network is running on a 192.168.0.0 range and the gateway does NAT. IPtables clamps the MTU and MRU which is in addition set to 1432 (I've tried every number).

I discovered that when another machine running linux connects behind the gateway, there is no problem. The minute I plug the WinXP machine in, the PPPoE disconnects and tried to continually reconnect wihtout success.

My Syslog reflects:

Jul 10 12:57:04 server1 pppd[3097]: Couldn't increase MTU to 1500
Jul 10 12:57:04 server1 pppd[3097]: Couldn't increase MRU to 1500
Jul 10 12:57:04 server1 pppd[3097]: PAP authentication succeeded

*****
Jul 10 12:57:04 server1 pppd[3097]: peer from calling number 00:C0:EE:C3:XX:XX authorized
*****
Jul 10 12:57:04 server1 pppd[3097]: Cannot determine ethernet address for proxy ARP
Jul 10 12:57:04 server1 pppd[3097]: local IP address x.x.x.x
Jul 10 12:57:04 server1 pppd[3097]: remote IP address 196.30.31.100
Jul 10 12:57:04 server1 pppd[3097]: primary DNS address 196.30.31.193
Jul 10 12:57:04 server1 pppd[3097]: secondary DNS address 196.46.70.1
Jul 10 12:57:04 server1 named[1881]: loading configuration from '/etc/bind/named.conf'
Jul 10 12:57:04 server1 named[1881]: listening on IPv4 interface ppp0, 196.2.102.212#53
Jul 10 12:57:07 server1 pppd[3097]: Connect time 0.1 minutes.
Jul 10 12:57:07 server1 pppd[3097]: Sent 389 bytes, received 128 bytes.

Interestingly, the issue continues even when I drop all IP traffic from the internal network - indicating that some signal the WinXP machine is sending at an OSI layer below TCP is causing my woes. See the bold line above for evidence.

Could this be because my WinXP MTU has not been manually altered down? If so, hope do you implement a network behind a gateway without manually altering everone's MTU down?

My Debian gateway has two interfaces. One (eth0) is configured with no IP and that is connected to my iBurst terminal (wireless broadband). The other (eth1) is configured for a private 192.168.0.0 network. There is also a wireless card (wlan0) which I have disabled until I sort this out. Currently, the internal interface (eth0) is connected directly to either one of my laptops. When it is a Linux laptop, no problem. When it is the WinXP machine, the connection works fine (I ssh in, run VNC to the Linux box, etc) the PPPoE connections drop.

I thought it might be PPPoE deamons on the XP machine and carefully ensured these were off, so I don't think it was that.

AAAAAhhhh!

Compiling your own Linux kernel

This is very abbreviated and there are many more thorough guides than this, but this works for me.

This guide is for compiling a kernel for Debian linux!

SERIOUS DISCLAIMER: this works for me. I cannot be held responsible if it does not work for you.

Get the version of the kernel you desire from http://kernel.org

Unpack the archive to /usr/src/[your new kernel version]

Remove the old symbolic link to your old kernel source if it exists:

Make sure it is a link you're removing and not your old kernel source! (Unless you want to do this)

rm /usr/src/linux

Link the new kernel source to the new source directory:

ln -s /usr/src/ /usr/src/linux

Change to the new /usr/src/linux directory

Now you'll need to repeat the following for each time you recompile.

First clean up your environment:

make mrproper

make clean

Copy your previous kernel's config to your new kernel source:

cp /boot/.config /usr/src/linux

Now configure your new kernel:

Either make menuconfig (in a console) or make xconfig from a console to launch a GUI session (you'll need QT installed).

Now run make-kpkg kernel_image to create the Debian package containing your new kernel.

Change to the parent directory cd /usr/src

Run DPKG to install the new kernel dpkg -i kernel-image-[your kernelversion]

Now a very cool trick. If you've just compiled your new kernel on a server somewhere in a backroom through a remote session, your nightmare would be a kernel panic locking up the machine and necessitating a hard reboot and manual selection of another kernel on the server console.

This assumes your Debian package has installed your new kernel selection to the GRUB boot loader

You're going to need to edit Grub:

nano /boot/grub/menu.lst


If you look there are a series of repeated lines. Each one of these is a different kernel that can be booted.

***PLEASE NOTE*** You need to modify the root (hdx,x) to match your machines partition configuration. The drive will be different depending on the individual server drive and partition configuration.

Check your Grub file to ensure that your new kernel selection is the first entry on the menu. (Entry O)

Make sure and change the default= one number higher then before since you added one at the very top. If it is 0 and you leave it at 0 and you have trouble with your server you will not be able to boot it.

After that save out and run grub

grub

Once it is done probing the drives enter:

savedefault --default=0 --once
quit

If all goes well (check by typing uname -a and looking at the running kernel number), then go back into the GRUB menu.lst file and set default= to zero.

If things do go wrong, you may still need to perform a hard reset on the server, but the next boot will default to your previous kernel.

Good luck - but everything above is at your own risk!!