Re: Back to work

[Eric Kinnear <[e-mail]>]

Hi Martin, 

> On Oct 28, 2020, at 10:17 PM, Martin Thomson <[e-mail]> wrote:
> 
> Hi Eric,
> 
> I don't know what you mean by: "How worried are we about the 3x limit?".

The one place where it seems that the proposed change (which is otherwise very nice, general, and fairly clean) could introduce some issues is around NAT rebinding — consider what could be a fairly common case: 

Client is fetching a decently large web resource from a server over QUIC. NAT rebinds and so the server sees un-padded QUIC packets arriving on a different port. We’ve said that the server does not need to reset the CC state if it has strong reason to believe that it’s likely the same path, and in this case the fact that the address did not change and it was solely the port is sufficient for that server implementation to match that condition.

With the 3x amplification limit, the server can only send 3x the bytes that are coming in — normally, this is fine because an endpoint is also getting padded PATH_CHALLENGE packets from the remote endpoint. However, in this case the client isn’t aware that it has migrated (this is also the case with a number of other MOTS attacks), and so is just sending a steady stream of small packets containing ACKs.

> I got the impression that Christian's request was that we avoid padding where it is unnecessary.

This case described above is what I thought Christian was describing in the final paragraph of his message, but it’s quite likely I’m wrong there. :) 

When we enter into this situation, we’re now getting fairly minimally-sized packets for at least the first RTT until the client responds with a padded PATH_RESPONSE to our (maybe not padded due to the 3x limit) PATH_CHALLENGE.

After that first RTT, the fact that we’re getting back a padded PATH_RESPONSE doesn’t matter for path validation because the path is already considered valid and we’re free to continue sending full-sized flights as allowed by our not-reset congestion controller.

And of course, I completely agree with you that we can use the actual data we’re trying to deliver for that MTU validation rather than wasting space with PADDING frames.

I’m not bothered by a requirement to do another PATH_CHALLENGE to validate MTU (wiring that up anywhere would work for PMTUD, path validation seems fine) afterward, the potential problem is simply the larger-than-before reduction in delivery rate.

The reason I ask “how worried” we are about that is because it’s a bit of an edge case, things were never going to be perfectly smooth during a NAT rebinding, etc. This happens to be one specific case that’s almost the most common of the unintentional migration cases, but even here we don’t usually expect a NAT rebinding in the middle of an active flow that’s trying to send significant amounts of data in each flight.

So, it seems to me that a completely reasonable consensus here would be “yeah, that’s a thing, but we don’t think that will matter in practice”. :) 

> 
> The two things we need to balance here are:
> * Amplification (the reason we have a 3x limit)
> * Odd failures when things only work sometimes (the reason we require padding of probes)
> 
> My belief is that the proposed PR #4264 achieves this balance in a minimal fashion.
> 
> Yes, there are two things you need to verify for a new path, but for the most part, probes can be padded and so will verify both.  Both that the address is live and that the path supports a reasonable MTU.  These are separated into two separate checks if checking the latter would violate the amplification limit.
> 
> Padding packets sent on either path is sub-optimal if you already know that the MTU is OK, so I'd be happy to create a carve-out for that.  It seems like a fine optimization, but the space can be used for things other than PADDING frames, so I don't see it as critical.  My migration implementation (almost finished, thanks) will include other frames in these packets where possible.

This totally makes sense, I think the amplification issue introduced by the padding requirement could be sorted out if we only applied that padding requirement to the one of two path validation situations that need it (i.e. the one where the client is sending PATH_CHALLENGEs, if we’re requiring the PATH_RESPONSE to come back on the same path). So just noting that this is one area where there may be further fallout and since we were discussing reducing change and risk of fallout it seemed like it would be good to consider options that reduce the scope of the previous change, rather than building on it.

If we look at that and say, “yeah, but this approach is unlikely to cause real-world fallout and the 3x limit is more general and clean”, I’m totally in agreement with that and have no issue with #4264.

Thanks,
Eric


> 
> Cheers,
> Martin
> 
> On Thu, Oct 29, 2020, at 13:13, Eric Kinnear wrote:
>> I was originally shying a bit away from that option, Christian, but the 
>> more I think about it the less painful it seems: back out the padding 
>> requirement, add in a note that “after a new path has been validated, 
>> endpoints can/should/must do PMTUD to validate the MTU on that path”. 
>> 
>> The natural responses that an endpoint goes through during path 
>> validation (or abrupt migration) leave it with either a working or 
>> failing path when trying to migrate; this was always the case and is no 
>> more or less the case due to MTU concerns. 
>> 
>> Alternately, you also note that we might want to avoid forcing the 
>> server specifically to send large packets — in my opinion, that would 
>> be another equally valid way to tackle this: require the padding only 
>> for PATH_CHALLENGES sent from the client and any PATH_RESPONSE to a 
>> padded challenge.
>> That’s really nice in a number of ways, especially for the MOTS cases 
>> where I can make the server think that every new packet is on a new 
>> path, etc.
>> 
>> So… lots of words to say: I very much agree with what you’re saying. :) 
>> 
>> It seems like these would be good options to use if we think that 
>> there’s something we *don’t* like about the 3x limit (perhaps how it 
>> interacts with NAT rebinding). Otherwise, the 3x limit is nice and 
>> general.
>> 
>> How worried are we about the 3x limit?
>> 
>> Thanks,
>> Eric
>> 
>> 
>>> On Oct 28, 2020, at 5:41 PM, Christian Huitema <[e-mail]> wrote:
>>> 
>>> 
>>> On 10/28/2020 5:17 PM, Jana Iyengar wrote:
>>>> I like the universality of the 3x limit. The reasoning applies broadly
>>>> and there's no reason to separately reason about how a server responds
>>>> to new addresses, be it at the start of a connection or
>>>> mid-connection. Overall, I have a few minor suggestions to make, but
>>>> I'm happy with the way the PR is headed.
>>> 
>>> 
>>> I think that we can devise proper solutions for "organized migrations",
>>> such as requiring both receipt and acknowledgement of a long enough
>>> packet before validating a path. I am not going to belabor that. But NAT
>>> traversal is a can of worms, and some of the norms are going to bite us.
>>> 
>>> I order to support NAT traversal, we request servers to take action if
>>> they see a packet arriving from a new address of the client. This opens
>>> a huge hole through which enterprising MOTS or "specially crafted
>>> clients" can harass servers. We have an OK defense: send a challenge to
>>> both old and new IP to force "proof of address ownership". That defense
>>> is OK because it uses minimal length packets, and thus does not cause
>>> too much amplification. I think we should leave it at that, and not
>>> force the server to send large packets. There is a need to then validate
>>> that the path can carry traffic, which will happen naturally when the
>>> server tries to send a large packet. But if the server only sends these
>>> large packets after address ownership has been verified, things ought to
>>> be quite good.
>>> 
>>> -- Christian Huitema
>>> 
>>> 
>> 
>