Re: Packet number encryption

[Jana Iyengar <[e-mail]>]

I don't think we're converging here, so I'd like to try and encourage us to
move towards it. I have my opinion, but I care more about forward progress
at this point.

There are three designs so far:
1. Packet numbers as is, with random gaps around migration.
2. Packet numbers encrypted, as per Martin's PR.
3. Packet numbers encrypted, plus a small modulo sequence number somewhere.

The concrete ones here are (1) and (2).  I'm listing (3) but am not
inclined to consider it seriously, since it's not concrete enough, (and I
don't want more bits to be spent in the QUIC packet header if that's where
this info goes.)

There are opinions about ossification, privacy, and manageability. I'm not
hearing bloody murder, so for what it's worth, I'd like to stipulate first
that we can all live with the final design, whichever one it is.  If anyone
disagrees, please say so -- I think it'd be useful to hear if there are
folks who think a particular design is fundamentally problematic.

My opinion is still for (2) over (1), but I can live with either.

On Mon, Feb 5, 2018 at 3:32 PM, Jana Iyengar <[e-mail]> wrote:

> Praveen,
>
> That was precisely my point about the ecosystem evolving. Now that we have
> the ability to classify flows, various things have been built around being
> able to tell TCP flows. This obviously has benefits as you point out, but
> the downside is that that I can't get good network utilization with a
> single flow. I understand the scaling point, but speaking of common
> cases, any server in the wild is usually serving a large number of
> connections at moderate speeds, not a single one at 25-35 Gbps... which
> makes that sort of scaling less exciting than in a microbenchmark.
>
> FWIW, the other downside of ECMP is that multipath transport doesn't work
> -- you need to hash on something else besides the 4-tuple. (We had to work
> around this in Google's deployment of connection migration.)
>
> I'll disagree with your point about reordering being the uncommon case.
> While that's true today, this is again an expectation that the network
> works hard to maintain, though the ecosystem and what we expect as "usual"
> would be quite different had we not built TCP's requirements into the
> network. There's nothing wrong with having n-modal latencies... we can
> engineer around that. Any sensible load distribution scheme within the
> network will give you increased variance but it should limit the worst-case
> latency. You'd perhaps agree that that's a net win.
>
> This was an example of how exposing packet number or not can have long
> term ecosystem effects. My point was about the fact that exposing can cause
> ossification.
>
> - jana
>
> On Mon, Feb 5, 2018 at 9:24 AM, Praveen Balasubramanian <
> [e-mail]> wrote:
>
>> >> This was definitely true for implementations of TCP, but that is TCP's
>> problem, not the network's
>>
>> Flow classification happens through the network and also on the end host
>> with RSS where a flow is mapped to a core. This allows for building high
>> performance receive processing that could be lock free for the most part.
>> The only downside of ECMP on the network and RSS on the CPU is that a
>> single flow will not take multiple paths or get processed on more than one
>> core. Windows on server machines today can saturate 25-35 Gbps for a single
>> TCP connection before being CPU limited. This is a  reasonable trade off
>> because I don’t know of cases where a single flow is driving more than that
>> amount of traffic. Assumptions about how flows get classified help make for
>> more efficient processing. They also lead to consistent latency. You do not
>> want the traffic to take a bi-modal or N-modal paths (on the network or the
>> host) to being processed because the fluctuations in latency will hurt the
>> workload. I am not arguing for TCP not being resilient to reordering but
>> IMO that should be the uncommon case not the common case. With QUIC if
>> streams were exposed on the network you could take advantage of stream
>> level ECMP and RSS to scale better than TCP but we have chosen to keep the
>> 4-tuple (or the 5-tuple) as the flow classifier on the network which is ok
>> by me since it leads to TCP parity.
>>
>>
>>
>> Thanks
>>
>>
>>
>> *From:* QUIC [mailto:[e-mail]] *On Behalf Of *Jana Iyengar
>> *Sent:* Sunday, February 4, 2018 8:35 PM
>> *To:* Salz, Rich <[e-mail]>
>> *Cc:* Lubashev, Igor <[e-mail]>; Roberto Peon <[e-mail]>;
>> QUIC WG <[e-mail]>
>> *Subject:* Re: Packet number encryption
>>
>>
>>
>> Privacy-protection can't be a user choice, for the reasons others have
>> noted on this thread.
>>
>>
>> That said, my primary argument is for encryption to avoid ossification.
>> Not that it matters now, but I'll note that much of GQUIC's original
>> motivation for encrypting headers was to avoid ossification.
>>
>>
>>
>> I'll reiterate that fields we expose will get ossified and there are
>> long-term ecosystem effects to this. Let me illustrate this with precisely
>> the packet number field. Middleboxes commonly assume that a TCP flow can
>> only handle packets in-order. This assumption comes from the fact that TCP
>> implementations get poor performance when packets are reordered. This was
>> definitely true for implementations of TCP, but that is TCP's problem, not
>> the network's. However, almost all load-balancers I know of now will pin
>> all packets within a TCP flow to one path, leading to sub-optimal
>> performance in the network, and destroying incentives for the endpoints to
>> deploy reordering-resilient TCP implementations (even though there are
>> plenty of ways of doing this.)
>>
>>
>>
>> Exposing QUIC's packet number field (as any field) is likely to have
>> similar consequences and a similar ecosystem arc.
>>
>>
>>
>>
>>
>> On Sun, Feb 4, 2018 at 7:51 PM, Salz, Rich <[e-mail]> wrote:
>>
>> > Optional security tends to devolve to non-secure.
>>
>>
>>
>> That’s a great aphorism.  And sadly all too true.
>>
>>
>>
>
>