RE: Is the invariants draft really standards track?

"Lubashev, Igor" <ilubashe@akamai.com> Fri, 19 June 2020 20:42 UTC

Return-Path: <ilubashe@akamai.com>
X-Original-To: quic@ietfa.amsl.com
Delivered-To: quic@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7B67F3A0E71 for <quic@ietfa.amsl.com>; Fri, 19 Jun 2020 13:42:28 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.999
X-Spam-Level:
X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, HTTPS_HTTP_MISMATCH=0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=akamai.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZnGusEDUTeBs for <quic@ietfa.amsl.com>; Fri, 19 Jun 2020 13:42:26 -0700 (PDT)
Received: from mx0a-00190b01.pphosted.com (mx0a-00190b01.pphosted.com [IPv6:2620:100:9001:583::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 394203A0E6D for <quic@ietf.org>; Fri, 19 Jun 2020 13:42:26 -0700 (PDT)
Received: from pps.filterd (m0122332.ppops.net [127.0.0.1]) by mx0a-00190b01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 05JKcLmv002348; Fri, 19 Jun 2020 21:42:16 +0100
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=akamai.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : mime-version; s=jan2016.eng; bh=OjpyhO0WCNaWTdPH4Sv/gn1wlqmljjyv4gcTIN+39dA=; b=II3Q2yPr8SEqsSpAzwqCopGYtHjo+ozdS1jS6tT9BBTfx+Idko1lgFsXAoEcuON1E6sf fqkLHV5etjVgFkBDG52XBfea+6NNzcCLU++LJprFMMk+IQ2cTRULpH3TMdVcEhrj4ZZe n38X5RKcPqwJ7oyoFoqaJsXj8PBWgo+ctyoQwYCFM48py0fb6Zj3Q6/c/cXThxTlfG5v YPKuu37jmcC7lJrSj+jX9Ou0dRddsyy1UYQ9CtbhMCnp2nrXPoz/RMwqS72CPHo/UgF/ EY2PYMr7QTFH5aNcKkFsT1An9zUh+M52tMmg9IK2N8vnc4E1Tj3CXOpIqfZbd0GZhTUO EA==
Received: from prod-mail-ppoint1 (prod-mail-ppoint1.akamai.com [184.51.33.18] (may be forged)) by mx0a-00190b01.pphosted.com with ESMTP id 31qqyc5gy9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 19 Jun 2020 21:42:15 +0100
Received: from pps.filterd (prod-mail-ppoint1.akamai.com [127.0.0.1]) by prod-mail-ppoint1.akamai.com (8.16.0.42/8.16.0.42) with SMTP id 05JKZc33022889; Fri, 19 Jun 2020 16:42:14 -0400
Received: from email.msg.corp.akamai.com ([172.27.123.31]) by prod-mail-ppoint1.akamai.com with ESMTP id 31qjmbwt33-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Fri, 19 Jun 2020 16:42:14 -0400
Received: from USMA1EX-DAG1MB5.msg.corp.akamai.com (172.27.123.105) by usma1ex-dag1mb4.msg.corp.akamai.com (172.27.123.104) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Fri, 19 Jun 2020 16:42:13 -0400
Received: from USMA1EX-DAG1MB5.msg.corp.akamai.com ([172.27.123.105]) by usma1ex-dag1mb5.msg.corp.akamai.com ([172.27.123.105]) with mapi id 15.00.1497.006; Fri, 19 Jun 2020 16:42:13 -0400
From: "Lubashev, Igor" <ilubashe@akamai.com>
To: Martin Duke <martin.h.duke@gmail.com>
CC: Christian Huitema <huitema@huitema.net>, Kyle Rose <krose@krose.org>, "Ian Swett" <ianswett=40google.com@dmarc.ietf.org>, Lars Eggert <lars@eggert.org>, IETF QUIC WG <quic@ietf.org>, Jared Mauch <jared@puck.nether.net>
Subject: RE: Is the invariants draft really standards track?
Thread-Topic: Is the invariants draft really standards track?
Thread-Index: AQHWM4bg7m/IhU9tKEaMc9qs7omEcqi7wjeAgACBhgCAAA8vAIAAAZ8A///Fg1CAId32AIACNYSAgACBoAD//9wAkA==
Date: Fri, 19 Jun 2020 20:42:13 +0000
Message-ID: <9c2e300c30f74d1794d11cf4334ea07b@usma1ex-dag1mb5.msg.corp.akamai.com>
References: <CAM4esxQBqfrz24riPQA_VGKcGp_TzW0pqb97KfFMtNdW9pUfDg@mail.gmail.com> <833A693C-62E6-4889-9954-FCE65A839A7C@eggert.org> <CAKcm_gPMO2DtqvKucqVw0zDjSniSOmFD4p1Tp4YLjr9WSWdEUw@mail.gmail.com> <CAJU8_nUN42gGmQof24XD9-EjXedyzcarDyRP8MGe1qW-BZ=+Aw@mail.gmail.com> <9cd91c24-c730-22a4-7aa0-baf61613b3ce@huitema.net> <f4922cdb59014202900de44cc5fea0ff@usma1ex-dag1mb5.msg.corp.akamai.com> <CAM4esxQvwkTvpUcu6-+W5zWo22m-R1jvN7DcCpXfuw8Hb55qsw@mail.gmail.com> <95dd02c92b32472d9cab0dd47b98c637@usma1ex-dag1mb5.msg.corp.akamai.com> <CAM4esxQxxXn27rZEY75-AsHD5VF0fqiV1VDyeSrzQ=-sM7JNCg@mail.gmail.com>
In-Reply-To: <CAM4esxQxxXn27rZEY75-AsHD5VF0fqiV1VDyeSrzQ=-sM7JNCg@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-transport-fromentityheader: Hosted
x-originating-ip: [172.19.118.2]
Content-Type: multipart/alternative; boundary="_000_9c2e300c30f74d1794d11cf4334ea07busma1exdag1mb5msgcorpak_"
MIME-Version: 1.0
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.216, 18.0.687 definitions=2020-06-19_22:2020-06-19, 2020-06-19 signatures=0
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 malwarescore=0 bulkscore=0 adultscore=0 mlxlogscore=999 mlxscore=0 phishscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2004280000 definitions=main-2006190146
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.216, 18.0.687 definitions=2020-06-19_22:2020-06-19, 2020-06-19 signatures=0
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 malwarescore=0 bulkscore=0 lowpriorityscore=0 impostorscore=0 priorityscore=1501 phishscore=0 suspectscore=0 mlxlogscore=999 mlxscore=0 clxscore=1015 adultscore=0 cotscore=-2147483648 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2004280000 definitions=main-2006190146
Archived-At: <https://mailarchive.ietf.org/arch/msg/quic/mUHxp00uj15kBoImrVVUqLb2Rkg>
X-BeenThere: quic@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Main mailing list of the IETF QUIC working group <quic.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/quic>, <mailto:quic-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/quic/>
List-Post: <mailto:quic@ietf.org>
List-Help: <mailto:quic-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/quic>, <mailto:quic-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 19 Jun 2020 20:42:29 -0000

> There is no need for servers to decrypt CIDs in QUIC-LB. Presumably the server has a lookup table for its CIDs.

Sending a stateless reset in response to a junk packet would cost more CPU than verifying CID integrity.  But, yes, a server may choose to not send stateless resets to anyone when under attack.


From: Martin Duke <martin.h.duke@gmail.com>
Sent: Friday, June 19, 2020 2:44 PM

>Unfortunately, Retry system protects only server's memory state and some CPU cycles spent on crypto.  (Servers still need to decrypt CID to decide it is invalid, and if the attacker is clever enough to establish one valid connection and use that CID in a flood, the server will also be decrypting packets.)

There is no need for servers to decrypt CIDs in QUIC-LB. Presumably the server has a lookup table for its CIDs.

It is true that Retry Services (and indeed, the Retry concept as a whole) does nothing to protect network capacity.

On Fri, Jun 19, 2020 at 8:08 AM Lubashev, Igor <ilubashe@akamai.com<mailto:ilubashe@akamai.com>> wrote:
It looks like https://tools.ietf.org/html/draft-ietf-quic-load-balancers-02#section-5<https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_draft-2Dietf-2Dquic-2Dload-2Dbalancers-2D02-23section-2D5&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=Djn3bQ5uNJDPM_2skfL3rW1tzcIxyjUZdn_m55KPmlo&m=eTT-BoZ1fMitywKRVSxpU3js0lhO0qkspYTKljvj-ys&s=1x7AN2a51X_zV5xSyK0uCL8vZ5cugD3n4lbFWZDqVQg&e=> is an excellent discussion of Retry mechanics.  It definitely deserves a reference from this manageability draft.

The Retry mechanisms described in LB draft are all cooperating boxes, and servers must be aware of them.  Unfortunately, Retry system protects only server's memory state and some CPU cycles spent on crypto.  (Servers still need to decrypt CID to decide it is invalid, and if the attacker is clever enough to establish one valid connection and use that CID in a flood, the server will also be decrypting packets.)  Retry does nothing to protect network resources.

The PR I opened (https://github.com/quicwg/ops-drafts/pull/94<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_quicwg_ops-2Ddrafts_pull_94&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=Djn3bQ5uNJDPM_2skfL3rW1tzcIxyjUZdn_m55KPmlo&m=eTT-BoZ1fMitywKRVSxpU3js0lhO0qkspYTKljvj-ys&s=VpvhQ9n79rANIk3O5Sm7PZyToFQEennoPt9iqFpWbq8&e=>) is about uncooperating devices that try to mitigate volumetric network attacks.


From: Martin Duke <martin.h.duke@gmail.com<mailto:martin.h.duke@gmail.com>>
Sent: Wednesday, June 17, 2020 8:16 PM
Hi Igor, you might want to check out the "Retry Services" bit of the QUIC-LB draft. This has something to do with the DDoS use case you discuss.

On Wed, May 27, 2020 at 9:07 AM Lubashev, Igor <ilubashe@akamai.com<mailto:ilubashe@akamai.com>> wrote:
I’m working on a manageability draft PR for this (how to rate limit UDP to reduce disruption to QUIC if you have to rate limit UDP).  ETA end of the week (if I do not get pulled into something again).

The relevant observation is that DDoS with UDP that is indistinguishable from QUIC will happen.  UDP is already the most prevalent DDoS vector, since it is easy for a compromised non-admin app to send a flood of huge UDP packets (with TCP you get throttled by the congestion controller).  So there WILL be DDoS protection devices out there to try to mitigate the problem, possibly by observing both directions of the flow and deciding whether a packet belongs to a valid flow or not.

Since such middle boxes will be created, the more explicit and normative Invariants are about what one can expect, the less such middle boxes may decide for themselves.  For example (I did not think long about it), if some elements of path validation could land into Invariants (roughly, “no more than X packets/bytes can be sent on a new path w/o a return packet”), a DDoS middle box may use this fact and active connection migration might still have a chance during an attack (NAT rebinding could be linked by DDoS boxes to an old connection via unchanged CID).


  *   Igor


From: Christian Huitema <huitema@huitema.net<mailto:huitema@huitema.net>>
Sent: Wednesday, May 27, 2020 11:34 AM
On 5/27/2020 8:28 AM, Kyle Rose wrote:
On Wed, May 27, 2020 at 10:34 AM Ian Swett <ianswett=40google.com@dmarc.ietf.org<mailto:40google.com@dmarc.ietf.org>> wrote:
I was agreeing with MT, but I'm happy to see some more MUSTs added if people feel that'd be helpful.

Coincidentally, we were just talking about this internally at Akamai yesterday. IMO, an invariants document isn't really helpful if it isn't normative, and for it to be normative it (or a related practices doc for operators) really needs to spell out clear boundaries for operators with MUSTs..

The example that came up yesterday was around operators filtering QUIC in the event of a DDoS: one recommendation based on some conversations going back at least to Prague 2019 was to hash packets on 4-tuple and filter those below a hash value chosen for a desired ingress limit instead of doing what most operators do with UDP today, which is to cap UDP throughput and just drop packets randomly or tail drop.

Interesting. Did they consider using the CID, or a fraction of it? This looks entirely like the scenario for which we developed stateless retry.

This recommendation certainly imposes some constraints on future protocol development that motivate new invariants: for instance, it would preclude sharding a connection across multiple source ports (not that there is necessarily a reason to do this; it's just an example). But more importantly, it goes beyond invariants: it's one among many practices compatible with the current set of invariants, some reasonable and some terrible.

This would break the "preferred address" redirection. Preferred address migration may or may not be spelled out in the invariants.

Operators are going to do things to QUIC traffic, so it would be good to offer them recommendations that are compatible with broad deployability.



Yes, we do need the invariants for that.

-- Christian Huitema