Subject: Re: Is "Version Greasing" a new benfit or a new obstacle?
From: "Brian Trammell (IETF)" <[email-masked]>
To: G Fairhurst <[email-masked]>
Cc: Roberto Peon <[email-masked]>, Praveen Balasubramanian <[email-masked]>, Ian Swett <[email-masked]>, Christian Huitema <[email-masked]>, Mirja Kuehlewind <[email-masked]>,Mikkel Fahnøe Jørgensen <[email-masked]>, "[email-masked]" <[email-masked]>, "Border, John" <[email-masked]>
Date: Mon, 15 Apr 2019 11:19:48 +0200
List-Id: Main mailing list of the IETF QUIC working group <quic.ietf.org>

hi Gorry, all,

> On 13 Apr 2019, at 13:03, G Fairhurst <[email-masked]> wrote:
> 
> Bringing this thread back - to me it seems there has been lots of discussion about how "version greasing" can be added in various ways, and very little on what makes this necessary and worth the impact on the network.
> 
> I still don't do not see that the side effects are worse when the protocol requires the version to be unobfusticated (as in most other IETF transport protocols).

+1. I continue to be confused about the amount of effort we're apparently willing to put into version negotiation protection of a protocol that is *explicitly* designed to be blockable outright (by virtue of the TCP fallback requirement). I haven't seen anything in this thread, or any of these discussions preceding it on this subject, that indicates that the actual risk of harm with respect to version blocking is discernibly worse (or even discernibly different) than the risk of harm with respect to blocking of the entire protocol regardless of version. 

I can see two reasons a version negotiation process could get blocked in the network:

(1) As an explicit choice by a traffic inspection function that it must have a specific version (or versions) of a protocol because it inspects information only available in that version of the wire image.

Note that QUIC is different from all other situations from which we could derive experience, since there simply isn't that much wire image for an inspection function to grab on to, and I can't imagine a future in which a future version would omit information it'd be worth blocking the version over (as opposed to all of QUIC, forcing TCP). Indeed, note that we made the latency spin bit probabilistic, as opposed to negotiated, in part to remove a potential incentive to ossify version negotiation.

It's possible that a device might want to block non-V1 versions because the handshake protection keys are hardcoded into the device. I'm not sure I see how dynamic aliasing could be reliably made to address this problem, though publishing v1 with N aliases (i.e. static aliasing) with different keys would at least ensure that devices with only one key slot don't work.

(2) As an overcorrection by a traffic inspection function that mistakes thinks that a particular sequence of version negotiation packets is a protocol identifier.

As I see it, the current version negotiation proposals only make this worse, not better, by creating a semantically indistinguishable set of aliases for the same operation. This problem could be nicely sidestepped by punting version negotiation into the future.

Cheers,

Brian

> Gorry
> 
> 
> On 12/04/2019, 17:41, Roberto Peon wrote:
>> 
>> I’m not sure that I agree there—there is no need to have it standardized so long as the LB is involved in picking it. Each LB system can choose whatever it wants. Thus, every LB **deployment** could use a different scheme with different grease. Every LB deployment could change the scheme it uses pseudo-randomly or according to the lunar calendar, etc.
>> 
>> Because there is a dynamic (and controlling) component (namely the part of the LB system which vends and then later routes CIDs), there is no need for anything else.
>> 
>> -=R
>> 
>> *From: *Praveen Balasubramanian <[email-masked]>
>> *Date: *Friday, April 12, 2019 at 9:28 AM
>> *To: *Roberto Peon <[email-masked]>, Christian Huitema <[email-masked]>, Ian Swett <[email-masked]>
>> *Cc: *"Gorry (erg)" <[email-masked]>, Mirja Kuehlewind <[email-masked]>, Mikkel Fahnøe Jørgensen <[email-masked]>, "[email-masked]" <[email-masked]>, "Border, John" <[email-masked]>
>> *Subject: *RE: Is "Version Greasing" a new benfit or a new obstacle?
>> 
>> Re. Encoding in CID. Then it needs to be standardized and if it is standardized what’s the point in greasing? Let’s not ignore the smaller services who use the cloud i.e. people deploying say NGINX web servers in AWS, Azure or GCP. Not every QUIC deployment is a large first party service which controls all aspects how LB and DDoS are done. In public clouds LB and DDoS protection are network functions (services) and are middleboxes under a completely different administrative domain.
>> 
>> Christian, this is about source address validation by the middlebox. This is how it works for TCP (think SYN cookies) and we need to have parity for QUIC. When under attack, short header packets will be dropped for addresses that have not been validated, but this is the same for non-SYN packets for TCP. I don’t understand how invariants can be used here. If we don’t have parity for DoS protection in network then QUIC traffic will be rate limited which is going to be really bad for performance. FWIW there is already DoS traffic that looks like QUIC.
>> 
>> *From:* Roberto Peon <[email-masked]>
>> *Sent:* Friday, April 12, 2019 9:16 AM
>> *To:* Christian Huitema <[email-masked]>; Ian Swett <[email-masked]>; Praveen Balasubramanian <[email-masked]>
>> *Cc:* Gorry (erg) <[email-masked]>; Mirja Kuehlewind <[email-masked]>; Mikkel Fahnøe Jørgensen <[email-masked]>; [email-masked]; Border, John <[email-masked]>
>> *Subject:* Re: Is "Version Greasing" a new benfit or a new obstacle?
>> 
>> For any large deployment, I suspect they’ll need LB.
>> 
>> For most kinds of LB-ing, I’d expect the LB system to either directly or indirectly be involved in choosing/vending connection IDs.
>> 
>> If it is involved in vending/choosing connection IDs, it can encode version information into the CID.
>> 
>> -=R
>> 
>> *From: *Christian Huitema <[email-masked] <mailto:[email-masked]><mailto:[email-masked] <mailto:[email-masked]>>>
>> *Date: *Friday, April 12, 2019 at 9:13 AM
>> *To: *Ian Swett <ianswett=[email-masked] <mailto:ianswett=[email-masked]><mailto:ianswett=[email-masked] <mailto:ianswett=[email-masked]>>>, Praveen Balasubramanian <pravb=[email-masked] <mailto:pravb=[email-masked]><mailto:pravb=[email-masked] <mailto:pravb=[email-masked]>>>
>> *Cc: *"Gorry (erg)" <[email-masked] <mailto:[email-masked]><mailto:[email-masked] <mailto:[email-masked]>>>, Roberto Peon <[email-masked] <mailto:[email-masked]><mailto:[email-masked] <mailto:[email-masked]>>>, Mirja Kuehlewind <[email-masked] <mailto:[email-masked]><mailto:[email-masked] <mailto:[email-masked]>>>, Mikkel Fahnøe Jørgensen <[email-masked] <mailto:[email-masked]><mailto:[email-masked] <mailto:[email-masked]>>>, "[email-masked] <mailto:[email-masked]> <mailto:[email-masked] <mailto:[email-masked]>>" <[email-masked] <mailto:[email-masked]><mailto:[email-masked] <mailto:[email-masked]>>>, "Border, John" <[email-masked] <mailto:[email-masked]><mailto:[email-masked] <mailto:[email-masked]>>>
>> *Subject: *Re: Is "Version Greasing" a new benfit or a new obstacle?
>> 
>> On 4/12/2019 8:57 AM, Ian Swett wrote:
>> 
>>    *From:* Praveen Balasubramanian
>>    *...*
>> 
>>    Version greasing will create problems for DDoS protection. The
>>    DDoS protection middlebox will have to treat all traffic with
>>    unknown version numbers as plain old QUIC and rate limit it. Any
>>    solution that requires per connection state to track version
>>    numbers doesn’t scale. Especially in the cloud scenario customers
>>    bring their own web servers so this makes any opt-out solution not
>>    good enough. So I do think this is an obstacle for many deployments.
>> 
>> Don't we have an example of middle-box ossification just there?
>> 
>> Praveen's example points to the perils of the "shortest path" approach to middle-box design. Taking the shortest path means finding something that works now, and once you have found it deploy it without worry about consequences -- which very often include ossification. If the DDOS protection box looks for "QUIC versions that it knows" and just rate limits the other types of packets, then the box will by design prevent deployment of versions that it does not know.
>> 
>> I assume that big organizations like Azure will update their protections once a new version is out, but the experience shows that many other organizations don't. And even when they do, the focus on published versions will block experimenting with new versions, not to mention interfering with other features like connection migration. I take Praveen's example as an argument why we should in fact standardize version greasing, precisely to avoid this "shortest path middle-box design".
>> 
>> In the case of DDOS protection, I will point out that DDOS protection also needs to work with short header packets, which do not carry any version number. In order to avoid DDOS by short header packets, the boxes have to be able to distinguish between packets that belong to authorized connections and packets that don't. And if you do that, then you don't really need to look at the version numbers, you can simply rely on the invariants.
>> 
>> -- Christian Huitema