Re: Packet number encryption

Jana Iyengar <jri@google.com> Mon, 05 February 2018 23:32 UTC

Return-Path: <jri@google.com>
X-Original-To: quic@ietfa.amsl.com
Delivered-To: quic@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8E92A124E15 for <quic@ietfa.amsl.com>; Mon, 5 Feb 2018 15:32:32 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.71
X-Spam-Level:
X-Spam-Status: No, score=-2.71 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PZIdjk9Fi37r for <quic@ietfa.amsl.com>; Mon, 5 Feb 2018 15:32:30 -0800 (PST)
Received: from mail-yw0-x22e.google.com (mail-yw0-x22e.google.com [IPv6:2607:f8b0:4002:c05::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E2DE8120721 for <quic@ietf.org>; Mon, 5 Feb 2018 15:32:29 -0800 (PST)
Received: by mail-yw0-x22e.google.com with SMTP id j128so45210ywg.7 for <quic@ietf.org>; Mon, 05 Feb 2018 15:32:29 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=mRBYje+CSvdpQ/qWA0RgG7HurCno/YkalYKq7fiIpRc=; b=QG3PMJrFb+XBXHQybDfqcRkIDUzbpdTbXRO6/YXZaEI3PGE3DI5rJxO7+1N9VgsvQp 9LPjWCPmOswO2HNfV3Lau/foc+UDtwghdNtep2b1n/JivviVFCMIexQf1z+LcztFX9i4 pR6seAd0C1VQ2kClTGObYoEzzqecDVs4eWQPTXC/e8cf5OrH2Eqm6VZa6NZCMC6N9G7z PbC16VkoSGjn4qye5R9LSE05fziLAmGoN2jj3rgGmoVmNX1NlFQHNH1HP7j76Whzxl4O vSqke06sU2uBryA8myrKHMYsdYhdLX9W//2CmxvKnf7V0HmSIUOmpqddlvKl8q02vz8F 0gCA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=mRBYje+CSvdpQ/qWA0RgG7HurCno/YkalYKq7fiIpRc=; b=epFo1m6cQ/6DsOruTbInbS4FpRVkmd4vVsIrVE1dv6gojy2cxtePapAhV9X/EBn5Yc mku3u2hbxEJ/pFyttVpfutoRriJcF8ecdWxT3vYSP01IfmMXy8kxDkJWOLzvpCQe72pa JlIlfBIIwyCFBGnLTVYQavI8ba6Ivgv+bRjNv4lF0/rJx0K3qR7UO9X78u5a3feQPL/4 ko8FQOQ8Bk/LVZ1wE3caST/0tqBEEErYZ/nhh3PUbEzPKPmSO/KRPmJzjYeZoZgtEMYd fk3A0s4saWrSpqFpiaN0CYh/+uiMbvWqAL0LL1LregS4h7ohMTua7u3fAHMU0jRar3k9 tPWw==
X-Gm-Message-State: APf1xPDhczxQVIsg16g0p4CHIcWV04EQbmPsXAvydQ6AAZcq0x8DXsL4 ckRSvu2z/QosNK0cBtqP+wj7jA3Ks8sMwMY6mNyxAw==
X-Google-Smtp-Source: AH8x2258OfNhSyxgpqD26ETQpJHkKAnN51eO5OXWgfjgWgVdJHA4UE4yjtOgZuNIfxETuOXzKUKvcKxYP86Tm81uzhI=
X-Received: by 10.129.131.139 with SMTP id t133mr302795ywf.59.1517873548089; Mon, 05 Feb 2018 15:32:28 -0800 (PST)
MIME-Version: 1.0
Received: by 10.37.135.77 with HTTP; Mon, 5 Feb 2018 15:32:27 -0800 (PST)
In-Reply-To: <CY4PR21MB01334E30C7AF6AE75F58EEFDB6FE0@CY4PR21MB0133.namprd21.prod.outlook.com>
References: <CABkgnnVyo3MmWtVULiV=FJTnR528qfY8-OmKGWAs0bCvri-a_g@mail.gmail.com> <1F7FB3B8-A94C-4354-9944-FB09FB8DB68B@trammell.ch> <CABcZeBMbwdwyC9TxxHBLYaZKfNB-FG2wCGjqUZ_mNR-A1R47FA@mail.gmail.com> <9096e5ec-581e-875a-b1dd-bff0b05206fd@huitema.net> <CABkgnnWRQSAufwPss+qf=xAzCwRYeNNH8XLPm3yFaHxOb+ba4g@mail.gmail.com> <BF80500A-6277-45DC-8525-9C3FE138B76D@tik.ee.ethz.ch> <5A7191E0.6010003@erg.abdn.ac.uk> <5214AD93-8376-4B25-922F-AF5551CC2E95@netapp.com> <F990E064-E6F8-41A3-B791-F776C9955E15@nokia.com> <CAGD1bZab0GaZFsHwC+nw3AxxC4VusxMJ6oDanzk3dSDdWKAXdw@mail.gmail.com> <2C515BE8694C6F4B9B6A578BCAC32E2F83BA1443@MBX021-W3-CA-2.exch021.domain.local> <BY2PR15MB07757473DB9788558B902EB5CDF80@BY2PR15MB0775.namprd15.prod.outlook.com> <6E58094ECC8D8344914996DAD28F1CCD861B7F@DGGEMM506-MBX.china.huawei.com> <e529144067624fcba636fc8c24ee3ff4@usma1ex-dag1mb5.msg.corp.akamai.com> <BY2PR15MB07754D83A1721F2BD742359BCDFE0@BY2PR15MB0775.namprd15.prod.outlook.com> <2CD9DC43-D69B-43F0-8474-DFE798850A52@akamai.com> <CAGD1bZaUuNxqpDkn62B0wWcFD8=mCUWrAwWGG-rAOxH7Mf1=cQ@mail.gmail.com> <CY4PR21MB01334E30C7AF6AE75F58EEFDB6FE0@CY4PR21MB0133.namprd21.prod.outlook.com>
From: Jana Iyengar <jri@google.com>
Date: Mon, 05 Feb 2018 15:32:27 -0800
Message-ID: <CAGD1bZaxrqzdkk0wxRaULwOTgg6wnrSrXNBK31s4uxdozaACBA@mail.gmail.com>
Subject: Re: Packet number encryption
To: Praveen Balasubramanian <pravb@microsoft.com>
Cc: "Salz, Rich" <rsalz@akamai.com>, "Lubashev, Igor" <ilubashe@akamai.com>, Roberto Peon <fenix@fb.com>, QUIC WG <quic@ietf.org>
Content-Type: multipart/alternative; boundary="001a114ecd54a6a64405647f7904"
Archived-At: <https://mailarchive.ietf.org/arch/msg/quic/tpsS4E-ucgoXB6_0BvOV8gVz74Y>
X-BeenThere: quic@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Main mailing list of the IETF QUIC working group <quic.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/quic>, <mailto:quic-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/quic/>
List-Post: <mailto:quic@ietf.org>
List-Help: <mailto:quic-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/quic>, <mailto:quic-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 05 Feb 2018 23:32:32 -0000

Praveen,

That was precisely my point about the ecosystem evolving. Now that we have
the ability to classify flows, various things have been built around being
able to tell TCP flows. This obviously has benefits as you point out, but
the downside is that that I can't get good network utilization with a
single flow. I understand the scaling point, but speaking of common cases,
any server in the wild is usually serving a large number of connections at
moderate speeds, not a single one at 25-35 Gbps... which makes that sort of
scaling less exciting than in a microbenchmark.

FWIW, the other downside of ECMP is that multipath transport doesn't work
-- you need to hash on something else besides the 4-tuple. (We had to work
around this in Google's deployment of connection migration.)

I'll disagree with your point about reordering being the uncommon case.
While that's true today, this is again an expectation that the network
works hard to maintain, though the ecosystem and what we expect as "usual"
would be quite different had we not built TCP's requirements into the
network. There's nothing wrong with having n-modal latencies... we can
engineer around that. Any sensible load distribution scheme within the
network will give you increased variance but it should limit the worst-case
latency. You'd perhaps agree that that's a net win.

This was an example of how exposing packet number or not can have long term
ecosystem effects. My point was about the fact that exposing can cause
ossification.

- jana

On Mon, Feb 5, 2018 at 9:24 AM, Praveen Balasubramanian <pravb@microsoft.com
> wrote:

> >> This was definitely true for implementations of TCP, but that is TCP's
> problem, not the network's
>
> Flow classification happens through the network and also on the end host
> with RSS where a flow is mapped to a core. This allows for building high
> performance receive processing that could be lock free for the most part.
> The only downside of ECMP on the network and RSS on the CPU is that a
> single flow will not take multiple paths or get processed on more than one
> core. Windows on server machines today can saturate 25-35 Gbps for a single
> TCP connection before being CPU limited. This is a  reasonable trade off
> because I don’t know of cases where a single flow is driving more than that
> amount of traffic. Assumptions about how flows get classified help make for
> more efficient processing. They also lead to consistent latency. You do not
> want the traffic to take a bi-modal or N-modal paths (on the network or the
> host) to being processed because the fluctuations in latency will hurt the
> workload. I am not arguing for TCP not being resilient to reordering but
> IMO that should be the uncommon case not the common case. With QUIC if
> streams were exposed on the network you could take advantage of stream
> level ECMP and RSS to scale better than TCP but we have chosen to keep the
> 4-tuple (or the 5-tuple) as the flow classifier on the network which is ok
> by me since it leads to TCP parity.
>
>
>
> Thanks
>
>
>
> *From:* QUIC [mailto:quic-bounces@ietf.org] *On Behalf Of *Jana Iyengar
> *Sent:* Sunday, February 4, 2018 8:35 PM
> *To:* Salz, Rich <rsalz@akamai.com>
> *Cc:* Lubashev, Igor <ilubashe@akamai.com>; Roberto Peon <fenix@fb.com>;
> QUIC WG <quic@ietf.org>
> *Subject:* Re: Packet number encryption
>
>
>
> Privacy-protection can't be a user choice, for the reasons others have
> noted on this thread.
>
>
> That said, my primary argument is for encryption to avoid ossification.
> Not that it matters now, but I'll note that much of GQUIC's original
> motivation for encrypting headers was to avoid ossification.
>
>
>
> I'll reiterate that fields we expose will get ossified and there are
> long-term ecosystem effects to this. Let me illustrate this with precisely
> the packet number field. Middleboxes commonly assume that a TCP flow can
> only handle packets in-order. This assumption comes from the fact that TCP
> implementations get poor performance when packets are reordered. This was
> definitely true for implementations of TCP, but that is TCP's problem, not
> the network's. However, almost all load-balancers I know of now will pin
> all packets within a TCP flow to one path, leading to sub-optimal
> performance in the network, and destroying incentives for the endpoints to
> deploy reordering-resilient TCP implementations (even though there are
> plenty of ways of doing this.)
>
>
>
> Exposing QUIC's packet number field (as any field) is likely to have
> similar consequences and a similar ecosystem arc.
>
>
>
>
>
> On Sun, Feb 4, 2018 at 7:51 PM, Salz, Rich <rsalz@akamai.com> wrote:
>
> > Optional security tends to devolve to non-secure.
>
>
>
> That’s a great aphorism.  And sadly all too true.
>
>
>