Skip to content

Some sites won’t load over VPN, RDP freezes: MTU and MSS issue

  • Home
  • Blog
  • Some sites won’t load over VPN, RDP freezes: MTU and MSS issue
Some sites won’t load over VPN, RDP freezes: MTU and MSS issue
Short answer

If the VPN tunnel is up, ping works and some applications open, yet HTTPS pages hang halfway, large file transfers fail or RDP freezes, the problem is most likely MTU-related. Tunnel headers make packets larger, full-size packets no longer fit, and because a firewall in between blocks the “fragmentation needed” ICMP messages, the sender never finds out (a PMTUD black hole). The fix is to set the tunnel MTU correctly and apply TCP MSS clamping at the tunnel endpoints.

Symptoms: “Packet needs to be fragmented but DF set.”

  • The VPN connection shows as “connected” and small ping packets go back and forth without trouble, but some websites or internal applications sit on a blank page.
  • HTTPS connections stall after the TCP handshake, during the TLS stage; the browser stays “connecting” for a long time and then times out.
  • A Remote Desktop (RDP) session opens but freezes when screen updates arrive; small files copy over file shares while large files stop halfway.
  • On Windows, trying “ping -f -l 1472” returns: Packet needs to be fragmented but DF set.
  • The same application works without problems outside the VPN, directly over the internet.

Root cause: tunnel headers, the DF bit and blocked ICMP

The typical MTU on Ethernet is 1500 bytes. In IPv4, the MSS, the largest amount of data TCP can carry, is found by subtracting the 20-byte IP and 20-byte TCP headers from the MTU, which gives 1460 bytes. A VPN tunnel adds its own headers in front of every packet: according to Cisco’s documentation, GRE adds 24 bytes, and in IPsec tunnel mode with ESP encryption and authentication the overhead can reach 58 bytes. So a 1500-byte packet on the inside exceeds 1500 bytes on the outside.

Path MTU Discovery (RFC 1191) was designed to solve this: the sender transmits packets with the DF (Do not Fragment) bit set; a router that hits a link with a smaller MTU drops the packet and reports the next-hop MTU to the sender with a “fragmentation needed and DF set” message (ICMP type 3, code 4). The sender then reduces the packet size. However, this mechanism depends entirely on that ICMP message getting back. RFC 2923 notes that firewalls are often misconfigured to suppress all ICMP messages. The result is a “black hole”: the TCP handshake succeeds with small packets, the transfer stops with the first large packet and the connection hangs until it times out.

That is why some sites load and others do not. Pages that return small responses work, while responses that arrive in full-size packets, such as a server’s certificate chain or large page content, vanish in the tunnel. If you use FortiGate, there is a special case: according to Fortinet, on FortiOS versions 7.6.1 through 8.0.0 the device no longer automatically clamps the TCP MSS for traffic traversing an IPsec tunnel and relies on PMTUD instead; the behaviour was reverted in 8.0.1.

Step-by-step fix

  1. First measure the path MTU. From a client at one end of the VPN, send a ping with the DF bit set to a server on the other side. On Windows -l, and on Linux -s, is only the size of the data field; 20 bytes of IP and 8 bytes of ICMP header are added on top. 1472 bytes of data correspond to a 1500-byte packet.
  2. Reduce the size until you find the largest value that succeeds. Adding 28 to that value gives the real path MTU inside the tunnel. For example, if 1372 succeeds and 1373 fails, the path MTU is 1400 bytes.
  3. Calculate the required MSS: for IPv4, subtract 40 from the path MTU. For a 1400-byte path MTU, the MSS is 1360.
  4. Set the tunnel interface MTU to the measured value. Cisco recommends an IP MTU of 1400 bytes on the tunnel because it covers most GRE and IPsec combinations.
  5. Apply TCP MSS clamping at both ends of the tunnel. This setting lowers the MSS value in TCP SYN packets passing through the tunnel, so both ends send packets that fit the tunnel from the very start.
  6. Do not block ICMP type 3 code 4 messages on the firewalls in between; allow PMTUD to work.
  7. Repeat the ping test and retry the problematic applications.

Measure in both directions; if the paths are asymmetric, the MTU can differ between the two directions. For measurements on Windows and Linux clients (replace the target address with one from your network):

REM Windows: DF bit set, 1472 bytes of data = 1500-byte packet
ping -f -l 1472 10.20.0.10
ping -f -l 1372 10.20.0.10

# Linux: -M do sets the DF bit, -s is the data size
ping -M do -s 1472 -c 4 10.20.0.10
ping -M do -s 1372 -c 4 10.20.0.10

Example configuration for a tunnel interface on a Cisco IOS router:

interface Tunnel0
 ip mtu 1400
 ip tcp adjust-mss 1360
end

On FortiGate, the MSS value can be set manually in the relevant IPsec firewall policy; on a Linux-based gateway, the iptables TCPMSS target can be used:

config firewall policy
    edit <policy-id>
        set tcp-mss-sender 1360
        set tcp-mss-receiver 1360
    next
end

# Linux gateway
iptables -t mangle -A FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu

Tunnel MTU and MSS changes affect every session passing through the tunnel and may require open connections to be re-established. Try these settings outside business hours, on a single tunnel first, and back up the current configuration.

Permanent prevention

MSS clamping only works for TCP. RFC 4459 stresses that the technique does not help UDP or other protocols without an MSS field, and that it assumes all traffic passes through endpoints that perform clamping. So do not rely on MSS alone: set the tunnel MTU correctly, keep ICMP “fragmentation needed” messages explicitly allowed in firewall rules, and document that this exception must be kept when rules are tightened. Repeat the measurement whenever the ISP line changes (for example, when moving to a connection such as PPPoE that adds extra headers) or a second tunnel layer is added. After firewall software updates, check the release notes for changes in MSS behaviour; the FortiOS 7.6.1 example shows that an update can silently change this behaviour.

When should you call an expert?

If measurements are inconsistent end to end, if there are tunnels between several branches and firewalls from different vendors, or if the problem appears only in a particular application, a two-way analysis with packet captures is required. As part of its network solutions, Workas in Denizli provides support with MTU/MSS tuning for site-to-site and remote access VPNs, reviewing firewall rules and resolving tunnel performance problems; for emergencies we respond with priority during business hours.

Frequently Asked Questions

Why can there be an MTU problem if ping works?

The default ping packet is very small and fits easily into the tunnel. The problem only shows up with full-size packets, which is why you need to test with a large ping that has the DF bit set.

Is setting the MSS too low harmful?

It does not break connections, but it reduces efficiency because each packet carries less data. Deriving the value by subtracting 40 from the measured path MTU is the most balanced approach.

Isn’t opening ICMP completely a security risk?

You do not need to open all ICMP. For PMTUD it is enough to allow “destination unreachable, fragmentation needed” (type 3, code 4) messages.

Does MSS clamping also fix UDP-based applications?

No. MSS only exists in TCP. UDP traffic needs a correctly set tunnel MTU and working PMTUD.

Sources

  1. Cisco: Resolve IPv4 Fragmentation, MTU, MSS, and PMTUD Issues with GRE and IPsec
  2. RFC 1191: Path MTU Discovery
  3. RFC 2923: TCP Problems with Path MTU Discovery
  4. RFC 4459: MTU and Fragmentation Issues with In-the-Network Tunneling
  5. Microsoft Learn: ping command reference

Let's Get Started

Let's Work Together!

Let's plan your company's IT needs together — we'll quickly find the right solution for you.

x

Since 2002, Workas Software Technology has provided businesses in Denizli with ERP, cybersecurity, network, server and backup, custom software and AI solutions.

Address
Kınıklı, Hüseyin Yılmaz Cd. No:67,
B Blok Z06, 20160 Pamukkale/Denizli, Türkiye
Get in Touch
0549 669 99 06
0535 089 29 29
Working Hours
Mon - Sat: 8:00 - 18:00