RE: Fun and surprises with IPv6 fragmentation

"Lubashev, Igor" <ilubashe@akamai.com> Sun, 04 March 2018 17:30 UTC

Return-Path: <ilubashe@akamai.com>
X-Original-To: quic@ietfa.amsl.com
Delivered-To: quic@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id C630B126C0F for <quic@ietfa.amsl.com>; Sun, 4 Mar 2018 09:30:47 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ietf.org; s=ietf1; t=1520184647; bh=PzsC9kzLFO4C6ll0fyJ86mmIqs35lKey+XIEL6SVQaU=; h=From:CC:Subject:Date:References:In-Reply-To:To:To:To; b=MI/AJSfrZUBYn7drAkv+IBO9mccWHPNr9yZywCi9TXGPRGypbOP4K9KPdHACm9oVL Vq55x9JJeEsaoR8vlYUjsXpAZX9jmeP7Hd1ugjM+D7iHgpzls6f2l5Lh1TJon8Pv1K S8yCiPwrqfNwJXpLRqGyx7vmTV0BZDgWFyDfPBDE=
X-Mailbox-Line: From ilubashe@akamai.com Sun Mar 4 09:30:47 2018
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 76591124B18 for <quic@ietf.org>; Sun, 4 Mar 2018 09:30:47 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ietf.org; s=ietf1; t=1520184647; bh=PzsC9kzLFO4C6ll0fyJ86mmIqs35lKey+XIEL6SVQaU=; h=From:CC:Subject:Date:References:In-Reply-To:To:To:To; b=MI/AJSfrZUBYn7drAkv+IBO9mccWHPNr9yZywCi9TXGPRGypbOP4K9KPdHACm9oVL Vq55x9JJeEsaoR8vlYUjsXpAZX9jmeP7Hd1ugjM+D7iHgpzls6f2l5Lh1TJon8Pv1K S8yCiPwrqfNwJXpLRqGyx7vmTV0BZDgWFyDfPBDE=
X-Original-To: dmarc-reverse@ietfa.amsl.com
Delivered-To: dmarc-reverse@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 37018126C0F for <dmarc-reverse@ietfa.amsl.com>; Sun, 4 Mar 2018 09:30:47 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.711
X-Spam-Level:
X-Spam-Status: No, score=-0.711 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, HTTPS_HTTP_MISMATCH=1.989, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=akamai.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id J-mjpg2OtEhh for <dmarc-reverse@ietfa.amsl.com>; Sun, 4 Mar 2018 09:30:45 -0800 (PST)
Received: from mx0b-00190b01.pphosted.com (mx0b-00190b01.pphosted.com [IPv6:2620:100:9005:57f::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 24C77124B18 for <ek=40google.com@dmarc.ietf.org>; Sun, 4 Mar 2018 09:30:45 -0800 (PST)
Received: from pps.filterd (m0122331.ppops.net [127.0.0.1]) by mx0b-00190b01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w24HR9Ks025051; Sun, 4 Mar 2018 17:30:44 GMT
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=akamai.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : mime-version; s=jan2016.eng; bh=Ru+D69Q6XbTLiNDWvxEUfbOC/H1qQXRkJaMmfBFsS54=; b=RhFFMqQrNIdwOgN1Y/lo5t8imfQo5+lsJfo4kkdPEtIqyiP313tUEd3Dh/CVnmXaWP7x A/g5deLrjLnOQvDWN+KsR8VDiKqe/MNv0vZlm78DBWMOkeKlhKB66lUj74WXqt9eN3ML 6EFbRypwZ3heQ+A+dz9D50hTLQZu47rD8FGNfZKB9z9z0/b9mE+oCW2ytfNrJqj5gfba 3Xhf6VumZA/3rtXquspqHWKCAp2nfp51bxMnRVoQgz+oc3SkxgI39XanLt2w4APdvyXE RuDtagYgLJJ1yZyZk57wxciGaFNeM7Vy+7g1Puc7mDshSjshnLztMW60bbsMTaK7JBhE Ag==
Received: from prod-mail-ppoint2 (prod-mail-ppoint2.akamai.com [184.51.33.19]) by mx0b-00190b01.pphosted.com with ESMTP id 2gfr2ck0a4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 04 Mar 2018 17:30:44 +0000
Received: from pps.filterd (prod-mail-ppoint2.akamai.com [127.0.0.1]) by prod-mail-ppoint2.akamai.com (8.16.0.21/8.16.0.21) with SMTP id w24HUgPB007845; Sun, 4 Mar 2018 12:30:43 -0500
Received: from email.msg.corp.akamai.com ([172.27.123.31]) by prod-mail-ppoint2.akamai.com with ESMTP id 2gfqwxua0c-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sun, 04 Mar 2018 12:30:43 -0500
Received: from USMA1EX-DAG1MB5.msg.corp.akamai.com (172.27.123.105) by usma1ex-dag1mb2.msg.corp.akamai.com (172.27.123.102) with Microsoft SMTP Server (TLS) id 15.0.1263.5; Sun, 4 Mar 2018 12:30:42 -0500
Received: from USMA1EX-DAG1MB5.msg.corp.akamai.com ([172.27.123.105]) by usma1ex-dag1mb5.msg.corp.akamai.com ([172.27.123.105]) with mapi id 15.00.1263.000; Sun, 4 Mar 2018 12:30:42 -0500
From: "Lubashev, Igor" <ilubashe@akamai.com>
CC: "quic@ietf.org" <quic@ietf.org>
Subject: RE: Fun and surprises with IPv6 fragmentation
Thread-Topic: Fun and surprises with IPv6 fragmentation
Thread-Index: AQHTsqzSw7eHcFi/aUaNSiLj0J473aPATxmAgAAxKAD//9dvxQ==
Date: Sun, 4 Mar 2018 17:30:41 +0000
Message-ID: <1f0f0060365f4d60a73cf68b7344a271@usma1ex-dag1mb5.msg.corp.akamai.com>
References: <681fcc96-4cf9-100d-9ad6-b3c7be9189a5@huitema.net> <CAAedzxq5G_gAaBzdizv5x-yobomW5+8sSZ_1rn4ApGYpQZmpvg@mail.gmail.com>, <CAN1APdcF0DeKQSDd_=CPbSJRuvSWQRF_K+EUpzCrp7OMytzvNw@mail.gmail.com>
In-Reply-To: <CAN1APdcF0DeKQSDd_=CPbSJRuvSWQRF_K+EUpzCrp7OMytzvNw@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-transport-fromentityheader: Hosted
Content-Type: multipart/alternative; boundary="_000_1f0f0060365f4d60a73cf68b7344a271usma1exdag1mb5msgcorpak_"
MIME-Version: 1.0
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2018-03-04_09:, , signatures=0
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1711220000 definitions=main-1803040227
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2018-03-04_09:, , signatures=0
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1711220000 definitions=main-1803040226
To: "ek=40google.com@dmarc.ietf.org" <ek@google.com>
To: "huitema@huitema.net" <huitema@huitema.net>
To: "mikkelfj@gmail.com" <mikkelfj@gmail.com>
Archived-At: <https://mailarchive.ietf.org/arch/msg/quic/G64Cr5jDhGOBOQqFq2FYx68HheA>
X-BeenThere: quic@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Main mailing list of the IETF QUIC working group <quic.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/quic>, <mailto:quic-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/quic/>
List-Post: <mailto:quic@ietf.org>
List-Help: <mailto:quic-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/quic>, <mailto:quic-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 04 Mar 2018 17:30:48 -0000

Depends on what you mean by raw socket. A raw IP socket will have sender kernel send IP fragments per its interface MTU or destination MTU cache, if available. A raw Packet socket will error out, if you try to send larger than interface MTU.

-----Original Message-----
From: Mikkel Fahnøe Jørgensen [mikkelfj@gmail.com]
Received: Sunday, 04 Mar 2018, 9:56AM
To: Erik Kline [ek=40google.com@dmarc.ietf.org]; Christian Huitema [huitema@huitema.net]
CC: quic@ietf.org [quic@ietf.org]
Subject: Re: Fun and surprises with IPv6 fragmentation

It would be interesting to see what a raw socket interface would do, sender side.

I'm also wondering if the hypervisor might be interfering, though it should act as middleware.



On 4 March 2018 at 13.00.29, Erik Kline (ek=40google.com@dmarc.ietf.org<mailto:ek=40google.com@dmarc.ietf.org>) wrote:

> I tried to work around the issue by setting the "don't fragment" bit on the
> socket, but somehow that doesn't work.

I don't see how an app can do any sort of PathMTU on its own without following:

https://tools.ietf.org/html/rfc3542#section-11.2<https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_rfc3542-23section-2D11.2&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=Djn3bQ5uNJDPM_2skfL3rW1tzcIxyjUZdn_m55KPmlo&m=0_EwNIr1oLcj1zLJnxR98mB2q-UQbuPwlhQ6wAvsvqk&s=gLmtYGMeHbWnHwRmuTzdlBs7j3QbbtXjcHhmU9evKeI&e=>

Without this, the sender OS can (by default) generate fragments if it
receives an ICMPv6 PTB. I.e. IPV6_DONTFRAG seems to me like a
requirement for MTU detection.

Put another way, I think you probably /always/ want to perform the
step you describe, whenever probing for MTU.


You said (elsewhere) you had a pcap showing fragments arriving at the
client, but have you checked that the sender isn't receiving PTBs and
reacting accordingly?

If PTBs are being received and the fact fragments are still being
generated with IPV6_DONTFRAG, that could point to a bug in the sender
OS.

If something in the Amazon infrastructure or along the path is doing
IPv6 fragmentation of its own accord it's violating the text at the
top of:

https://tools.ietf.org/html/rfc8200#section-4.5<https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_rfc8200-23section-2D4.5&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=Djn3bQ5uNJDPM_2skfL3rW1tzcIxyjUZdn_m55KPmlo&m=0_EwNIr1oLcj1zLJnxR98mB2q-UQbuPwlhQ6wAvsvqk&s=TEunT0TN8uqVxX5LC9IAIH7dkUkHZZ3EJ2i6NY-0NRY&e=>

even that text isn't particularly strongly worded. (I looked for a
MUST or MUST NOT elsewhere, but haven't found such text yet.) Devices
doing this should probably be identified and expurgated from the
network.