Re: [EXTERNAL] Re: WGLC review of draft-ietf-quic-recovery-29

[Gorry Fairhurst <[e-mail]>]

Thanks Praveen,

I'd like to talk more, see below:

On 09/07/2020 17:11, Praveen Balasubramanian wrote:
>
> Within data center mainly. To hit 40 Gbps line rate you effectively 
> have to batch as much as you can to minimize the CPU processing costs. 
> The burst size today is capped to 64k in most cases for TCP. Now that 
> we have segmentation offload for UDP the same should apply. Even for 
> Internet facing traffic, the bursts primarily impact the first hop 
> within the datacenter because subsequently you get statistical 
> multiplexing from ToR switch handling traffic forwarding for many 
> flows and ports.
>
I get this for this use-case and comments on high-speed NICs. I agree 
offload is important.

> Ok understood about there being no conflict. But I still contend that 
> the MUST regarding burst size is way too restrictive. We should allow 
> for experimentation here.
>
I am not sure. The IW spec for TCP was based on evidence back around 
2010, and the IETF agreed in RFC6928 that a burst size of 10 was 
acceptable as an EXP spec for TCP.

I think we can safely refer to that - even perhaps take the 
leap-of-faith that EXP for TCP is equivalent to PS for QUIC, given 
subsequent experience.  I think the IW  **is** the burst size that the 
IETF should specify (because it's dictated by buffering that is expected 
on the path).

I see opportunities here for experimentation and measurement 
opportunities, and new specs can be proposed. Or the Specs can be recast 
with more detail. I also know that practical experience is that many 
people have configured (probably tested, I hope) much larger IW's - and 
I'd hope they also make a reasonable effort to monitor for possible 
interactions with other Internet applications and services (as per 
section 12 of RFC6928).

> Or present data that this is universally bad on all networks.
>
I think you touch on something that the IETF hasn't been great at 
talking about:

I could myself indeed likely produce data this shows this is "bad" in 
some networks that I value, and also show side-effects of collateral 
damage to other flows and adverse effects of line-rate bursts into 
equipment with small buffers. I don't know how useful that would be at all.

The IETF has typically defined a spec for endpoint. The way a modern 
endpoint is implemented can be quite different in a DC from a single 
device connected to the Internet. In a DC, I understand it is common  to 
distribute transport functions to other parts of the network. e.g.: DDOS 
protection could be done on a server; or by a box in front of it - 
similarly what matters to the network CC is the  bursts of traffic that 
leaves a DC within a flow to a local receiver, not what leaves a 
particular server within the DC network - I think in future we may do 
well to consider these differences?

> Current production use of TCP shows there is no major problems with 
> bursting higher. In fact for low latency same rack scenarios this will 
> boost receive performance as well due to better opportunity for LRO etc.
>
It's also possible that QUIC LRO (or whatever) includes some notion of 
per-flow burst-mitigation. Is that impossible?

However, I personally would not hold-up QUIC, to arrive at a new consensus.

Gorry

> *From:* Gorry Fairhurst <[e-mail]>
> *Sent:* Thursday, July 9, 2020 1:39 AM
> *To:* Praveen Balasubramanian <[e-mail]>; IETF QUIC WG 
> <[e-mail]>
> *Subject:* [EXTERNAL] Re: WGLC review of draft-ietf-quic-recovery-29
>
> Hi Praveen,
>
> On 08/07/2020 22:29, Praveen Balasubramanian wrote:
>
>     Section 7.9
>
>     "Implementations MUST either use pacing or another method to limit
>     such bursts to the initial congestion window; see Section 7.2."
>
>     This seems to preclude use of segmentation offload of sizes
>     greater than IW. In datacenters we routinely send bursts that are
>     higher without causing loss.
>
> Is that within data centres? or from data centres to Internet 
> destinations? ... if the latter, how can the sender be sure this is 
> not impacting other traffic?
>
>     The MUST here seems unnecessary. It also conflicts with the
>     RECOMMENDED in an earlier sentence.
>
> I actually don't see this as conflicting. The recommendation in this 
> section is to pace. The requirement is that sender limits burst, using 
> some method, to the size of IW.
>
> Gorry
>