Re: Fun and surprises with IPv6 fragmentation

Mikkel Fahnøe Jørgensen <> Sun, 04 March 2018 14:55 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 966A51270AC for <>; Sun, 4 Mar 2018 06:55:57 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.698
X-Spam-Status: No, score=-2.698 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, UNPARSEABLE_RELAY=0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id 0m5ovSTqtpyx for <>; Sun, 4 Mar 2018 06:55:56 -0800 (PST)
Received: from ( [IPv6:2607:f8b0:4001:c06::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 33BD81243F6 for <>; Sun, 4 Mar 2018 06:55:56 -0800 (PST)
Received: by with SMTP id v6so15270071iog.7 for <>; Sun, 04 Mar 2018 06:55:56 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=from:in-reply-to:references:mime-version:date:message-id:subject:to :cc; bh=fyB7H0KwPF77SDWbEV8TM0BoV5iSVYlwAnFlNCRocWw=; b=MYaJqUY6hZBocCzaD70HTFn48gXN/QqMTFwKoxKQ2oq4Bsl61KiryR+N3EHCQeCzJM pxa8G4q+D8DL/UDtX+G6DQtqkdcvi6yQqDQ6l+iDXyGsX0zMQzy0F6Ek1D36SZqBDKKJ 9tUL5hl/Gpfpgx+hN24FUY4NSV4x6YetXekUMx2VXeFipHUSHpyLLimOYHsV8eYUK8kw pyPgK668nyym192jYDLS+A9JTZ7HVxnHT+BCG5H14K+qr7xeuVZsFerPlHCys50zOdBT 6CsaCp8CGu7ZktGjCfb026YYywcOO1aubb8oBul/KC/kQZurmXiFL5crSKm0gq/pR3em Q56Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=x-gm-message-state:from:in-reply-to:references:mime-version:date :message-id:subject:to:cc; bh=fyB7H0KwPF77SDWbEV8TM0BoV5iSVYlwAnFlNCRocWw=; b=f3JT3X+AuXvUUXWsx/QihfLQFRknId5CnWYWyDz9KalVas2f7lcDqvN7rTZ3hMD23W aFV4NWXoSCbKPLJD6PdKTUJaIIXgFO+3Sf2CHVDuIPrpYknCBaIHDBQuw/gsN5Ccxwhl EmWQEeDFu4AFFEB7dcZaQvkqCteD/LyafzXDQwQdJZV4JStpA+gdfebFYeSLq/pvAqRT R80yeGJN6MBfiwmSNlmfuH9gjmMAZ5QKlDymUpOO4niN5iB2z6Qzguo+2W5Y3lmEcaQm ypdycDx3fSdA7qI3mz40YNHz9zbdv8DsZoPP9J8XehpUjZhBpdv/2fNpC7l5x3lKopQh rb4Q==
X-Gm-Message-State: AElRT7E3C5REPg5XF6EYWp//Bvhj3G4NU7QDBYFq7EBuwqGonBOk7uV/ ZtSJZ/dKsmHLXL5XuVnzz53OWrw+6baApWF/k3+dcA==
X-Google-Smtp-Source: AG47ELuORLIYmJwrhdsUNiyGctk7ioJu74vWvrXpeuqdfmeqeuQGO7nX3nEjyiqyZL0jG8txX/gVyZ01KgR7eUgMpmo=
X-Received: by with SMTP id a76mr14231751ioa.36.1520175355605; Sun, 04 Mar 2018 06:55:55 -0800 (PST)
Received: from 1058052472880 named unknown by with HTTPREST; Sun, 4 Mar 2018 09:55:54 -0500
From: Mikkel Fahnøe Jørgensen <>
In-Reply-To: <>
References: <> <>
X-Mailer: Airmail (420)
MIME-Version: 1.0
Date: Sun, 04 Mar 2018 09:55:54 -0500
Message-ID: <>
Subject: Re: Fun and surprises with IPv6 fragmentation
To: Erik Kline <>, Christian Huitema <>
Cc: "" <>
Content-Type: multipart/alternative; boundary="001a113fd0621139ef05669768f6"
Archived-At: <>
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Main mailing list of the IETF QUIC working group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Sun, 04 Mar 2018 14:55:58 -0000

It would be interesting to see what a raw socket interface would do, sender

I’m also wondering if the hypervisor might be interfering, though it should
act as middleware.

On 4 March 2018 at 13.00.29, Erik Kline (

> I tried to work around the issue by setting the "don't fragment" bit on
> socket, but somehow that doesn't work.

I don't see how an app can do any sort of PathMTU on its own without

Without this, the sender OS can (by default) generate fragments if it
receives an ICMPv6 PTB. I.e. IPV6_DONTFRAG seems to me like a
requirement for MTU detection.

Put another way, I think you probably /always/ want to perform the
step you describe, whenever probing for MTU.

You said (elsewhere) you had a pcap showing fragments arriving at the
client, but have you checked that the sender isn't receiving PTBs and
reacting accordingly?

If PTBs are being received and the fact fragments are still being
generated with IPV6_DONTFRAG, that could point to a bug in the sender

If something in the Amazon infrastructure or along the path is doing
IPv6 fragmentation of its own accord it's violating the text at the
top of:

even that text isn't particularly strongly worded. (I looked for a
MUST or MUST NOT elsewhere, but haven't found such text yet.) Devices
doing this should probably be identified and expurgated from the