Re: Exercising Version Negotiation

Roberto Peon <fenix@fb.com> Mon, 26 March 2018 22:05 UTC

Return-Path: <prvs=66230deeec=fenix@fb.com>
X-Original-To: quic@ietfa.amsl.com
Delivered-To: quic@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DC63F124B18 for <quic@ietfa.amsl.com>; Mon, 26 Mar 2018 15:05:15 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.701
X-Spam-Level:
X-Spam-Status: No, score=-2.701 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=fb.com header.b=YM3CCH5C; dkim=pass (1024-bit key) header.d=fb.onmicrosoft.com header.b=Mg3cFORX
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PYjEHFQHZ-rL for <quic@ietfa.amsl.com>; Mon, 26 Mar 2018 15:05:14 -0700 (PDT)
Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 39FBC1276AF for <quic@ietf.org>; Mon, 26 Mar 2018 15:05:14 -0700 (PDT)
Received: from pps.filterd (m0044012.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w2QM4bST018716; Mon, 26 Mar 2018 15:05:12 -0700
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : mime-version; s=facebook; bh=BeYVcPkjEkGSO9dzvSBvinC3ytdZj4Eirf0ahig8yIk=; b=YM3CCH5CWI7S5j9bw8qBtbFVFpKC6aIHzHGOKjn6RHHp+WdWGiHsARonTXebWB+xOyLX DxbSaIRMbXe8OutHwoNwc0MYX6DVpWY3eJRZZ0/ut1YrVTlZ14kzKD6tF5UGQ2L3eCc9 WKrNojY4vXaLbFmer0imMiJRqfpu4e2MW/c=
Received: from maileast.thefacebook.com ([199.201.65.23]) by mx0a-00082601.pphosted.com with ESMTP id 2gy9af009f-19 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NOT); Mon, 26 Mar 2018 15:05:12 -0700
Received: from NAM02-BL2-obe.outbound.protection.outlook.com (192.168.183.28) by o365-in.thefacebook.com (192.168.177.32) with Microsoft SMTP Server (TLS) id 14.3.361.1; Mon, 26 Mar 2018 18:04:33 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.onmicrosoft.com; s=selector1-fb-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=BeYVcPkjEkGSO9dzvSBvinC3ytdZj4Eirf0ahig8yIk=; b=Mg3cFORXnAgj6YlCUVdzosC02Au7DHHEjitKNuogTS2yCT8pCp43JnpnaIg0tLUeo41j4LobAGyjWLIxJ2zxKSSgA+hktcHKFK6DySaqassLN6VMAClRR/y0r05xvSqIOFYAIJnELKmob/ui+94mk3L4BWE/cALPsqubRPNYXuw=
Received: from BY2PR15MB0775.namprd15.prod.outlook.com (10.164.171.11) by BY2PR15MB0326.namprd15.prod.outlook.com (10.163.65.140) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.609.10; Mon, 26 Mar 2018 22:04:31 +0000
Received: from BY2PR15MB0775.namprd15.prod.outlook.com ([fe80::78a6:3a7a:1195:dd02]) by BY2PR15MB0775.namprd15.prod.outlook.com ([fe80::78a6:3a7a:1195:dd02%13]) with mapi id 15.20.0609.012; Mon, 26 Mar 2018 22:04:31 +0000
From: Roberto Peon <fenix@fb.com>
To: Ryan Hamilton <rch=40google.com@dmarc.ietf.org>
CC: Eric Rescorla <ekr@rtfm.com>, IETF QUIC WG <quic@ietf.org>, Mike Bishop <mbishop@evequefou.be>
Subject: Re: Exercising Version Negotiation
Thread-Topic: Exercising Version Negotiation
Thread-Index: AQHTwdYDUZhb3cJH5UigdAPrv4PipqPcS1kAgAAEWICAABZ4AIAAVeCAgADHmYCAAMElAP//liUAgACNhYCAAAC5BYAEdfoA//+WXoCAAIIhAP//rCwA
Date: Mon, 26 Mar 2018 22:04:31 +0000
Message-ID: <3CD5DE02-61DE-4201-873E-D682AD7345F2@fb.com>
References: <CABcZeBMv5BqZOtgVA2wfqaaGCd94gcNPB9bTXkrvNXXRveU8wA@mail.gmail.com> <CAJ_4DfQ6zqVeUUF7XcoT110kVcP1BJFEtqVR-+FN5XD2UuRMMA@mail.gmail.com> <CABcZeBMVNy151rFntLutSPtctPsd2Ei3Qy-ChuEXVMVpz4pgdQ@mail.gmail.com> <SN1PR08MB1854DD71FDCB9FBA48DD26E9DAA90@SN1PR08MB1854.namprd08.prod.outlook.com> <CAJ_4DfQxcxNGNP9CW3poPqnFgUsrNy269dO2Lf0dvWBFKKsSHg@mail.gmail.com> <CABcZeBNfw6MjjgeqQct11g6Kf=4xy+zVZiBRMwwzhtKB2rWHZg@mail.gmail.com> <CAJ_4DfRCNRHajNfFdedg6O54mjydWt+ooRESJZQ5L0sZmcwmXg@mail.gmail.com> <9A92D490-1F4F-457D-9209-C19E154E4881@fb.com> <CAJ_4DfSNA0wMnvzxSy1bHU5VTdKujP6oZAtwn8DUNwnX97yLDQ@mail.gmail.com> <BY2PR15MB0775C6A8859735FF359A2974CDA80@BY2PR15MB0775.namprd15.prod.outlook.com> <CAJ_4DfStVJ7kY8V-LLFHL8cJejP8BtWW2+Gjbgc-WRhRfMP6yg@mail.gmail.com> <5302B914-3586-4C94-83B3-09309F630C98@fb.com> <CAJ_4DfRm6wTuJLuX2CT6Fdu-LGakaxLKeqhyp5oj5iuouDLm+A@mail.gmail.com>
In-Reply-To: <CAJ_4DfRm6wTuJLuX2CT6Fdu-LGakaxLKeqhyp5oj5iuouDLm+A@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
user-agent: Microsoft-MacOutlook/10.9.0.180116
x-originating-ip: [2620:10d:c090:200::4:191d]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; BY2PR15MB0326; 7:JHsIjv36rEfVn7AeCSy5rjF5mrdGONkXM/1dWDIRYOMn7Y7lzOSKaNMHIscM6BvOwUMjsddtnEZcnKk08l0vPUs5Vw+ueclxfPRy4I4FfSgyrUCvaJV8PQGcEY08grOZMO25vr9fsZgfjiKEHlg3snmn7Gr+oeqYDRapo2avwl+Xa60VezQzWTA7Ox2quqzHtXi0Rbi/e8gF8LHDSWutNLm1a+qy9+aVpNxnpbeOZqoczVd5YGVL2LnX3UCiLVmv; 20:KJD1v8nvZB3YzZmArEcgpDPrwHE1zgFu90z4LDFDIAUxP6mTOMGhGG3ZuOxSlPLRfh3egvQn9zWNtYLt2JWGQ4NxbzD/eNLVjvtdKR0RZ/leaNxOFIdsWjavaZighTY//sWtvn+MZkj8mbxei7dfY/4Eft0rVW3zvVskz3Q7y8I=
x-ms-exchange-antispam-srfa-diagnostics: SOS;
x-ms-office365-filtering-correlation-id: 2db1004b-ade7-47b9-49c8-08d593658d7d
x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4604075)(3008032)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:BY2PR15MB0326;
x-ms-traffictypediagnostic: BY2PR15MB0326:
x-microsoft-antispam-prvs: <BY2PR15MB032638FCCE72F1A254072E5CCDAD0@BY2PR15MB0326.namprd15.prod.outlook.com>
x-exchange-antispam-report-test: UriScan:(28532068793085)(158342451672863)(67672495146484)(211936372134217)(153496737603132)(21748063052155);
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(10201501046)(3002001)(93006095)(93001095)(3231221)(11241501184)(944501327)(52105095)(6041310)(20161123564045)(20161123558120)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(6072148)(201708071742011); SRVR:BY2PR15MB0326; BCL:0; PCL:0; RULEID:; SRVR:BY2PR15MB0326;
x-forefront-prvs: 06237E4555
x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(979002)(39380400002)(346002)(376002)(366004)(39860400002)(396003)(189003)(199004)(51444003)(58126008)(76176011)(46003)(68736007)(236005)(53936002)(6306002)(790700001)(99286004)(6512007)(7736002)(54896002)(53946003)(2900100001)(478600001)(6116002)(105586002)(11346002)(33656002)(2906002)(476003)(561944003)(446003)(229853002)(14454004)(3660700001)(6436002)(3280700002)(2616005)(6246003)(6486002)(82746002)(7116003)(36756003)(53546011)(86362001)(5660300001)(93886005)(5250100002)(83716003)(4326008)(186003)(25786009)(97736004)(6506007)(54906003)(8936002)(81166006)(106356001)(316002)(81156014)(102836004)(8676002)(3480700004)(486005)(486005)(42262002)(969003)(989001)(999001)(1009001)(1019001); DIR:OUT; SFP:1102; SCL:1; SRVR:BY2PR15MB0326; H:BY2PR15MB0775.namprd15.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en;
received-spf: None (protection.outlook.com: fb.com does not designate permitted sender hosts)
x-microsoft-antispam-message-info: /gGQ+pUvmbagGFHNvtT+1RplPVE8DnmFIi9GfbU2Vkz0DkTNZ/j8n2/dkbXYqXBiX1IQB2JfETCw1bSoVhH1x0bJCH9xfMrZ2tGYGJ6LxPl9ZWuKUU+D9g56nllD9eJHGNR9sXtZXv+4HGjNIs6/2b2EAdu4LdQvU6l0rbzPMpCUv2uW2Oh5DSG7lJrcoJ1NMpw+z/pLwmS9P+UyPqC8xkCMVyrLX+LBIAZc64Eo7Ccs71ATViILjM7mEkJ2PY1zD+UuickmkfqC9WwIsmFxDdQ26U6R7TbcFxpnQs/F5FeCk2tlIh2s8urkrzu7sbbGFmk9HPk4PDFaFCdPyLatZg==
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: multipart/alternative; boundary="_000_3CD5DE0261DE4201873ED682AD7345F2fbcom_"
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: 2db1004b-ade7-47b9-49c8-08d593658d7d
X-MS-Exchange-CrossTenant-originalarrivaltime: 26 Mar 2018 22:04:31.6477 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY2PR15MB0326
X-OriginatorOrg: fb.com
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2018-03-26_10:, , signatures=0
X-Proofpoint-Spam-Reason: safe
X-FB-Internal: Safe
Archived-At: <https://mailarchive.ietf.org/arch/msg/quic/-BRaiFwVixWnxTMasftC9VIYWuQ>
X-BeenThere: quic@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Main mailing list of the IETF QUIC working group <quic.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/quic>, <mailto:quic-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/quic/>
List-Post: <mailto:quic@ietf.org>
List-Help: <mailto:quic-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/quic>, <mailto:quic-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 26 Mar 2018 22:05:16 -0000

There are two sets of entities that we’re greasing.

  1.  Middleboxes
  2.  Implementation endpoints.

#1 is greased whenever multiple versions are spoken by any set of clients/servers
#2 is greased whenever the client/server use multiple versions simultaneously. There is a subset of the problems space which is negotiation. Within negotiation there are two subsets:

  1.  Negotiation where the client picks something it likes
  2.  Negotiation where the client must throw away old state and use something new.

Users incur a hit only when 2b happens. <bad joke> 2b or not 2b, that is the question. </bad joke>
How often we pick 2b is the question. If we’re slowly rotating between versions, then how often depends on which path the client uses to know it should do QUIC.
If it is being told to use a version via alt-svc, then we’re unlikely to see the QUIC version negotiation happen on the wire.
If it is not told via alt-svc, then, once a week or so (if following my example), the client will learn that its version is no longer the version the server wishes to speak. It will incur that penalty once per week per connection per ip.

-=R

From: QUIC <quic-bounces@ietf.org> on behalf of Ryan Hamilton <rch=40google.com@dmarc.ietf.org>
Date: Monday, March 26, 2018 at 1:05 PM
To: Roberto Peon <fenix@fb.com>
Cc: Eric Rescorla <ekr@rtfm.com>, IETF QUIC WG <quic@ietf.org>, Mike Bishop <mbishop@evequefou.be>
Subject: Re: Exercising Version Negotiation

So if a client sees Alt-Svc v+, and the server supports v+, then there will be no version negotiation. Same for:
  Alt-Svc v- Server supports v-
  Alt-Svc v+, v- Server supports v+, v-
I think we'll only get version negotiation when the client attempts to speak a version of QUIC that the server does not support. But the client will only attempt to speak a version of QUIC that the server advertises. So to get negotiation we need one of:
  Alt-Svc v+,v- Server supports v- (client prefers v+)
​  ​
Alt-Svc v-
​, v+​
Server supports v
​+​ (client prefers v-)
​If this is the setup, I think that every connection the client makes to the server will incur a version negotiation 1-RTT hit. I don't think it will be only a single hit per day. Can you explain how this setup will only impact the client once per day (for connections to this server?)

Cheers,

Ryan

On Mon, Mar 26, 2018 at 12:18 PM, Roberto Peon <fenix@fb.com<mailto:fenix@fb.com>> wrote:
I was thinking that you’d advertise different versions based on the client IP and day, including the alt-svc stuff, thus (as you point out) the QUIC version negotiation logic would match up with the alt-svc logic.
This depends to some extent on how we actually advertise QUIC in alt-svc (e..g. do we advertise all versions, or just say ‘quic’, and let the client sort it out?).

I’d imagine that 33% of the time, a particular IP will see ALT-SVC: v+, and QUIC will negotiate v+. 33% of the time, that IP will see ALT-SVC: v+, v- and QUIC will negotiate either, etc.
I suspect that there are better proposals which could exercise more of the state space (v-, v+ advertised instead of v+, v-, etc.), but the overall mechanism would be the same—a particular client sees the world as a slowly changing rainbow of version combinations.

-=R

From: Ryan Hamilton <rch@google.com<mailto:rch@google.com>>
Date: Monday, March 26, 2018 at 11:38 AM
To: Roberto Peon <fenix@fb.com<mailto:fenix@fb.com>>
Cc: Eric Rescorla <ekr@rtfm.com<mailto:ekr@rtfm.com>>, IETF QUIC WG <quic@ietf.org<mailto:quic@ietf.org>>, Mike Bishop <mbishop@evequefou.be<mailto:mbishop@evequefou.be>>

Subject: Re: Exercising Version Negotiation

Mixing day as well as client IP into the experiment group function makes sense, but it's not clear to me how this only ends up impacted 1 connection per day, because of Alt-Svc. Is it your vision that the server will advertise v- and v+ for all users, or will the QUIC version negotiation logic match up with the Alt-Svc advertisement logic? I ask because if the server keeps advertising v- and v+, and the client prefers, say v+, I would imagine that the client would attempt to use v+ on each new connection attempt.

On Fri, Mar 23, 2018 at 3:35 PM, Roberto Peon <fenix@fb.com<mailto:fenix@fb.com>> wrote:

This is roughly what I'm thinking:

auto versions = std::vector<std::string>();

int day = date_as_unix_epoch_in_ms() / ms_in_day;

int day_mod = (day + hash(client_ip)) % 3;

switch (day_mod) {

  case 0:

    versions.push_back("v+");

    break;

  case 1:

    versions.push_back("v-");

    break;

  case 2:

    versions.push_back("v+");

    versions.push_back("v-");

    break;
}

With such a scheme, the client would incur an extra RTT on a new connection once a day.

-=R

________________________________
From: QUIC <quic-bounces@ietf.org<mailto:quic-bounces@ietf.org>> on behalf of Ryan Hamilton <rch=40google.com@dmarc.ietf.org<mailto:40google.com@dmarc.ietf.org>>
Sent: Friday, March 23, 2018 3:26:59 PM
To: Roberto Peon
Cc: Eric Rescorla; IETF QUIC WG; Mike Bishop
Subject: Re: Exercising Version Negotiation

Maybe. :) I'm just trying to get my arms around a concrete proposal. For example, if the server always advertises both versions, but only supports 1 version based on hash(client IP), and those client typically prefer the other version, then they'll incur an extra RTT on every connection, which seems undesirable. But I'm not sure if that's the proposal.

On Fri, Mar 23, 2018 at 2:00 PM, Roberto Peon <fenix@fb.com<mailto:fenix@fb.com>> wrote:

Wouldn’t it be best to do all 3 combinations? ☺

-=R



From: QUIC <quic-bounces@ietf.org<mailto:quic-bounces@ietf.org>> on behalf of Ryan Hamilton <rch=40google.com@dmarc.ietf.org<mailto:40google.com@dmarc.ietf.org>>
Date: Friday, March 23, 2018 at 1:20 PM
To: Eric Rescorla <ekr@rtfm.com<mailto:ekr@rtfm.com>>
Cc: IETF QUIC WG <quic@ietf.org<mailto:quic@ietf.org>>, Mike Bishop <mbishop@evequefou.be<mailto:mbishop@evequefou.be>>
Subject: Re: Exercising Version Negotiation



On Fri, Mar 23, 2018 at 1:48 AM, Eric Rescorla <ekr@rtfm.com<mailto:ekr@rtfm.com>> wrote:

On Thu, Mar 22, 2018 at 8:53 PM, Ryan Hamilton <rch@google.com<mailto:rch@google.com>> wrote:

Agreed. As specified, I don't see how the server would be induced to send a version negotiation packet in this case. Ekr, perhaps you can provide some sample flows. If the server supports versions A and B, and the client supports versions A and B, then it doesn't matter which version the client prefers. In either case, the server supports the version the client is attempting to speak and will not send a version negotiation packet.



I'm not suggesting that you flip the preferences, but rather that the each side configure itself with one version for a given connection. This is obviously trivial for a client. For the server, you can just hash the client's IP address to get a stable choice.



​Ah, I misunderstood what you meant by, "We would then suggest to people that they somewhat randomize their preferences (e.g., 99% of the time prefer v1+i, 1% of the time prefer v1-i)."



How would you envision this working with Alt-Svc advertisements? Would the server advertise support for both versions but only support a single one, or would it advertise only the single version?