Path MTU Discovery

March 20, 2016

The maximum transmission unit (MTU) is the largest packet that can be transmitted on a link. It naturally follows that the MTU of a given path is the smallest MTU that would be experienced along any given hop on a packet’s journey to its destination. While many of us have become accustomed to the default Ethernet LAN MTU of 1500 bytes, different transmission technologies may have a more constrained MTU. With IPv4, a host didn’t necessarily have to know the MTU of a given path. IPv4 allowed routers along a path to fragment packets on behalf of a host, unless the “don’t fragment” bit happened to be set in the IPv4 header. Details about IPv4 fragmentation can be found in RFC 791.

IPv6 changes this functionality. Section 4.5 of RFC 2460 specifies that routers must no longer fragment packets on behalf of hosts, as this places an unnecessary burden on network infrastructure. It is now incumbent on each host to determine the MTU for a path and only transmit packets that meet this size constraint. This is accomplished through Path MTU Discovery (PMTUD), as detailed in RFC 1981. Essentially, PMTUD is accomplished through the use of the ICMP Packet Too Big message type 2, which is specified in Section 3.2 of RFC 4443. Naturally, the necessary use of ICMP for a critical function of packet transmission spells disaster for engineers who, wielding Maslow’s trusty hammer, believe that the best security strategy is to filter all ICMP messages, everywhere, all the time. Security posture for IPv6 will need to consider the necessity of ICMP for proper IPv6 operation, strategies that most competent security professionals have probably been doing for quite some time.

The overall process for PMTUD can be found in RFC 1981 and is simple. If a router along the transmission path receives a packet that is too large then it will send an ICMPv6 Packet Too Big message to the originating host and drop the packet. The host, upon receiving the ICMPv6 Packet Too Big message, will review the acceptable MTU provided in the message and adjust the packet size for the flow accordingly. The Packet Too Big message also contains as much of the original message as will fit, which helps the host to identify the packet that was dropped.

Topology

Our topology for this activity is fairly simple, but still one of the more complex topologies that we’ve dealt with in this series. We’ve strung together a series of routers, each with a progressively smaller MTU as we move from the left to right side of the topology.

At each end are CentOS hosts residing on networks with the default Ethernet MTU of 1500 bytes. One acts as an IPv6 web server, running Apache without any special configuration and having ip6tables disabled. The other host acts as web client. Our goal is to see how the IP stack on the hosts determine the MTU of the entire path, which is 1300 bytes for our topology. Remember: the path MTU is the smallest MTU along a given path, as it acts as the constraining MTU for the end-to-end transmission of an IP packet.

It’s also worth noting that the minimum MTU for IPv6 is 1280 bytes, as specified by Section 5 of RFC 2460.

The configs for the routers are below. There is nothing special about the configurations for the CentOS Server or Client. The server has a basic Apache installation, and both are using Stateless Address Autoconfiguration for address configuration.

Router configs:

• R1
• R2
• R3
• R4

The packets

To capture the data needed for this experiment, I curled the website hosted on the CentOS server and captured the packets. We’re going to break up the packet capture into manageable sections, as it would be a bit overwhelming to look at the entire thing in one image. First, we can see the TCP three-way handshake between the CentOS Client and CentOS Web Server. This is largely uneventful, as the handshake consists mainly of header information at this point, and no serious application data is being exchanged. Therefore, this process works without issue.

Note: I really won’t be concerned with looking at the TCP headers and explaining their values, as the operation of TCP is beyond the scope of this discussion. However, RFC 793 is a worthwhile read for any IT engineer.

Next, we can see an attempt by the server to transfer three packets to the client with a size of 1514 bytes. This corresponds to the default Ethernet MTU, accounting for Ethernet header and CRC. Let’s take a closer look at the first packet.

The first thing that is worth examining about this packet is the payload length of 1460 bytes specified in the IPv6 header. This matches up with the default Ethernet MTU of 1500 bytes (40 byte IPv6 header + 1460 bytes of payload = 1500 bytes). Next, we’ll make note that the TCP sequence number of this segment is 282137334. Knowing the TCP sequence number will be helpful later.

Continuing on with our packet trace, we can see an ICMPv6 Packet Too Big message, followed by several additional packets of size 1464. For the sake of brevity, we won’t be examining every single TCP segment. Rather, we will just follow the journey of the segment with sequence number 282137334 until its finally acknowledged by the client.

Taking a look at the ICMPv6 Packet Too Big message, we can see that it was sourced by router R1 at the IP layer. We can see that it is of type 2, which is the ICMPv6 type for Packet Too Big, as specified by Section 3.2 of RFC 4443. Next, we can see the actual MTU of the next hop network: 1450 bytes.

After the ICMPv6 header, we see something interesting: a collection of IPv6, TCP, and HTTP2 headers. The TCP header lists sequence number 282137334, which indicates that this is our dropped packet. When a router drops a packet and sends a Packet Too Big message to a host, it also includes as much of the dropped packet as allowed by the MTU, a behavior that is specified in Section 3.2 of RFC 4443. This is useful, as the end-host doesn’t have to just blindly guess about which packet was dropped, nor does it have to wait for timeouts of higher-layer protocols, such as TCP. Rather, it can immediately identify the dropped data (in this case, the same packet we looked with TCP sequence 282137334) and take action.

The MTU provided by R1 by the Packet Too Big message is 1450. However, we still have two more networks that the packet must traverse, each one with an increasingly smaller MTU. By setting up a capture filter that specifies TCP sequence number 282137334, we are able to see a Packet Too Big Message from each router along the path to the packet’s destination. Finally, we are able to see a frame with size 1314, which corresponds to a packet size of 1300 bytes once we account for the Ethernet header and CRC. Our host has now discovered the MTU for the entire path to the destination.

We can now see that packets successfully reach their destination once the path MTU from the CentOS Web Server to the CentOS Client is known.

Conclusion

The path MTU discovery process provides an important function for IPv6 hosts, especially due to the way that IPv6 handles packets that exceed the MTU. While IPv4 hosts can often rely on routers to handle packet fragmentation, IPv6 hosts must perform this function for themselves. This alleviates strain on network routing infrastructure, but it also necessitates the permitting of ICMPv6 traffic throughout the network. IPv6 networks are less sympathetic to blocking all ICMP traffic on the network, and this is an important consideration to be aware of when developing the security posture of an IPv6 network.