RE: Non-Last Small IPv6 Fragments

"Lubashev, Igor" <ilubashe@akamai.com> Sat, 12 January 2019 16:22 UTC

Return-Path: <ilubashe@akamai.com>
X-Original-To: ipv6@ietfa.amsl.com
Delivered-To: ipv6@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 70F82124BE5 for <ipv6@ietfa.amsl.com>; Sat, 12 Jan 2019 08:22:11 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.253
X-Spam-Level:
X-Spam-Status: No, score=-5.253 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-4.553, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, KHOP_DYNAMIC=2, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=akamai.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pn1D7tzFzpOy for <ipv6@ietfa.amsl.com>; Sat, 12 Jan 2019 08:22:09 -0800 (PST)
Received: from mx0b-00190b01.pphosted.com (mx0b-00190b01.pphosted.com [IPv6:2620:100:9005:57f::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id EAC9C12426E for <ipv6@ietf.org>; Sat, 12 Jan 2019 08:22:08 -0800 (PST)
Received: from pps.filterd (m0122330.ppops.net [127.0.0.1]) by mx0b-00190b01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x0CGKF6Y016835; Sat, 12 Jan 2019 16:22:06 GMT
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=akamai.com; h=from : to : subject : date : message-id : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=jan2016.eng; bh=BKwFQdEFBDugr+pJPv+ddEm0y+w9SKKcm+j5E8s6uZ0=; b=Nd/AFgl+Y7klnxRs12nmoAazEZHxBVMB1XMJrOFSsqTaxPWqcY+Nas1COccTpofB2tsH jIGL29ZCc9lHM6701X4O4LuC+o8K9SEJNQbSUmj9k0eIv6XXcQz+ny23QM/m7CkyMCqj x9hKqGmkbiU1xIcIPEiMHnkE+D4cftaJUysWymsQohYvBAIT0DXdMJOOEjBJUSWoZ9B3 5V/IojnUSb2KFrHHjMPQEAYGCxtYAsS0vjNHLQ7JDq07d+XKwpqJCf9zsiazcmv0Wazm SdPTZn9ID11dp3C5Tt2GHGTXBSyMiv7OQlLOjqgNCAYS5tzIUWkHTnD+H8QsKZ59Dvk0 iQ==
Received: from prod-mail-ppoint4 (a96-6-114-87.deploy.static.akamaitechnologies.com [96.6.114.87] (may be forged)) by mx0b-00190b01.pphosted.com with ESMTP id 2pyb00s8yf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 12 Jan 2019 16:22:05 +0000
Received: from pps.filterd (prod-mail-ppoint4.akamai.com [127.0.0.1]) by prod-mail-ppoint4.akamai.com (8.16.0.27/8.16.0.27) with SMTP id x0CGH0pw016638; Sat, 12 Jan 2019 11:22:05 -0500
Received: from email.msg.corp.akamai.com ([172.27.27.21]) by prod-mail-ppoint4.akamai.com with ESMTP id 2pycf0h685-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sat, 12 Jan 2019 11:22:05 -0500
Received: from USTX2EX-DAG1MB5.msg.corp.akamai.com (172.27.27.105) by ustx2ex-dag1mb2.msg.corp.akamai.com (172.27.27.102) with Microsoft SMTP Server (TLS) id 15.0.1365.1; Sat, 12 Jan 2019 10:21:52 -0600
Received: from USTX2EX-DAG1MB5.msg.corp.akamai.com ([172.27.27.105]) by ustx2ex-dag1mb5.msg.corp.akamai.com ([172.27.27.105]) with mapi id 15.00.1365.000; Sat, 12 Jan 2019 10:21:52 -0600
From: "Lubashev, Igor" <ilubashe@akamai.com>
To: Ole Troan <otroan@employees.org>, 6man WG <ipv6@ietf.org>
Subject: RE: Non-Last Small IPv6 Fragments
Thread-Topic: Non-Last Small IPv6 Fragments
Thread-Index: AQHUqPn6UJUsAabdDUW2I5kvXsPj7aWpK8eAgAAdU4CAAAKvgIAAAucAgAACcACAAAjMgIAASb0AgAAjOwCAAK+8AIAAA6OAgAALsQCAAEZLgIAAE/EAgAAXVQCAAAPVgIAAEn4AgAAEHQCAAE3PgIAABeQAgAAOCICAAFcDgP///Yzg
Date: Sat, 12 Jan 2019 16:21:51 +0000
Message-ID: <722a0cba0f934e7f89c6f433c02a5726@ustx2ex-dag1mb5.msg.corp.akamai.com>
References: <CAOSSMjV0Vazum5OKztWhAhJrjLjXc5w5YGxdzHgbzi7YVSk7rg@mail.gmail.com> <BYAPR05MB4245B9305E6EC57EDD45509FAE840@BYAPR05MB4245.namprd05.prod.outlook.com> <7453645f-ff91-e866-b087-e7d4f1450ab6@gmail.com> <0e792b48-4360-6977-9ae8-9cdfdc78c7b8@gmail.com> <16A642DC-D3A4-452C-B7D1-20CA0EEEDDA2@lists.zabbadoz.net> <CAOSSMjWS9po2XuBHJ5hbN9hfNDKZ1diecH08Kt697-15jRtAvg@mail.gmail.com> <0e0c3141-889e-ff60-2787-2889b3a9af6b@si6networks.com> <748DA428-5AB2-4487-A4FB-F2DABF5BF8BE@thehobsons.co.uk> <8b43af81-1c49-5cea-6472-97703674e661@si6networks.com> <CAN-Dau1HwG5RndacpSA+si+zKuTdpSvA=QA1A11A==rMNe=4+w@mail.gmail.com> <CALx6S35KNhV2gFp9OdU+M1zy5WUuEAEvXkDXNDWWxi7uQ4e_cw@mail.gmail.com> <CAN-Dau0rTdiiF2SjByxcMG6nhPCEjUH2pYBCOeK_FSGJ_ucDQw@mail.gmail.com> <CALx6S34AyV9OpvnjQhQc56n5vfeVgU5Zd3kheP0g+XvsMbBV9g@mail.gmail.com> <1b2e318e-1a9f-bb5d-75a5-04444c42ef20@si6networks.com> <CALx6S37TJr++fC=pVoeS=mrO1fHc4gL_Wtu-XkVTswzs2XxXCA@mail.gmail.com> <9352bb4d-f4d9-ebc6-34f9-dcb2f3ec24f1@si6networks.com> <312A771E-9E5B-4E33-926F-EDF46C4CB925@employees.org>
In-Reply-To: <312A771E-9E5B-4E33-926F-EDF46C4CB925@employees.org>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-transport-fromentityheader: Hosted
x-originating-ip: [172.19.33.142]
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-01-12_09:, , signatures=0
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1901120141
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-01-12_09:, , signatures=0
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1901120139
Archived-At: <https://mailarchive.ietf.org/arch/msg/ipv6/RYZx2kGHuJb9xZtm-tYJFcu4wKI>
X-BeenThere: ipv6@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "IPv6 Maintenance Working Group \(6man\)" <ipv6.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ipv6>, <mailto:ipv6-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ipv6/>
List-Post: <mailto:ipv6@ietf.org>
List-Help: <mailto:ipv6-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ipv6>, <mailto:ipv6-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 12 Jan 2019 16:22:11 -0000

> - how long are fragment chains (both in packets and in time span)
> - ratio of out of order fragments
> - which applications use fragments
> - ratio of attack traffic using fragments
> - average/min/maximum sizes, same for numbers of fragments in chain
> - ratio of complete fragment chain following the same path in the network
> - ratio of fragments to total traffic

Ole, I do not think a study of ratios and averages will provide much value.  A general implementation like Linux kernel must (will want to) support all use cases. No regressions!  I've seen several applications, all homegrown so you are unlikely to encounter them for any study, where the use of fragments is an effect of the application's long life and evolution -- it started with reasonably-sized UDP packets but demands on the amount of data in a single datagram grew over the years.


As for the Linux implementation, I think the limit there is not a complete solution but an assertion that non-final fragments shorter than X cannot possibly be legit.  This assertion increases the costs on the attacker.

Ultimately, Fragment Smack and Segment Smack are NOT attacks on memory of the machine but on the bookkeeping Algorithm. For obvious reasons, Linux tries hard to avoid memcopy to consolidate the fragments as they arrive, so it keeps each fragment in its own buffer, which has a significant memory overhead.  Once the entire IP packet is received and processed (discarded, forwarded, or copied to userspace), all these buffers are freed.  However, Linux does careful memory accounting, and if the memory for these fragment buffers grows past a certain threshold, Linux attempts to perform compaction (the dreaded memcopy), which requires a linear scan through all the accumulated fragment buffers.  In a well-behaved case, that compaction will free up a lot of extra memory, but in the attack case, that compaction will free only a little memory (just enough for one more fragment).  All subsequent attack packets will cause the linear compaction scan.

The algorithmic improvement to the above could include a faster compaction (extra bookkeeping to know in advance which fragments can be coalesced and not spend any time examining the ones that cannot be coalesced) and/or a simple heuristic after a single compaction that determines whether "enough" progress has been made and/or a "quality" heuristic that is maintained as fragments are being received.

- Igor

> -----Original Message-----
> From: Ole Troan <otroan@employees.org>
> Sent: Saturday, January 12, 2019 4:49 AM
> To: 6man WG <ipv6@ietf.org>
> Subject: Re: Non-Last Small IPv6 Fragments
> 
> Fernando said:
> > I agree with the above (please see the ref I posted to RFC6274, which
> > basically argues about this). HOowever, it is still not clear to me
> > what's the problem you are addressing by enforcing a minimum fragment
> > size. If you allow MIN_MTU/2, then you require first frags to be 640
> > bytes.. and teh attacker is required to send 640 as opposed to 60
> > bytes or  so. Doesn't seem to be much of a difference to me.
> > And as noted by Bob, I still don't see what's the big deal with the
> > small fragments.
> >
> > There are some DoS attacks where you send a last frag and possibly a
> > first frag, and the implementation allocates a buffer that would fit
> > the rest of the missing fragments -- obviously exacerbating the
> > problem a bit. BUt still i this cases, the size of the first frag is not a big deal.
> 
> In e.g. VPP limiting fragment size would not help at all.
> The scarce resource is number of available buffers. One fragment occupies at
> least one buffer regardless of how much smaller than 2K (default buffer size)
> it is.
> 
> Mitigations against attack is highly likely to be implementation specific.
> As an example see:
> https://github.com/FDio/vpp/blob/master/src/vnet/ip/ip6_reassembly.c
> 
> Default reassembly timeout is 100ms (constrasting that with the RFC 60s)
> Maximum consequtive reassemblies in progress is limited to 1024.
> And in other virtual reassembly algorithms we have also limited maximum
> allowed number of fragments in chain to 5.
> 
> Now, if we in the IETF think we can provide some guidance here, I will second
> Erik’s suggestion of getting more research.
> 
> The packet traces I have looked at in the past, the majority of fragmeted
> packets looked like attacks. UDP port 80, DNS queries to
> “thisdomainnamewillgiveaverylongresponsepurelyforthepurposeofattack.or
> g”,
> didn’t reassemble, 64K size… But it would be interesting to understand
> fragmentation behaviour in various parts of the network.
> 
> - how long are fragment chains (both in packets and in time span)
> - ratio of out of order fragments
> - which applications use fragments
> - ratio of attack traffic using fragments
> - average/min/maximum sizes, same for numbers of fragments in chain
> - ratio of complete fragment chain following the same path in the network
> - ratio of fragments to total traffic
> 
> Cheers,
> Ole
> 
> 
> 
> 
> --------------------------------------------------------------------
> IETF IPv6 working group mailing list
> ipv6@ietf.org
> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> --------------------------------------------------------------------