Re: [EXTERNAL] Re: WGLC review of draft-ietf-quic-recovery-29

[Gorry Fairhurst <[e-mail]>]

I'd like to argue a little more for something like the current text we 
arrived at:

"Implementations MUST either
    use pacing or another method to limit such bursts to the initial
    congestion window;"

I've been watching, and have comments see below:

On 20/07/2020 23:11, Spencer Dawkins at IETF wrote:
> Hi, Ian,
>
> On Mon, Jul 20, 2020, 16:33 Ian Swett <[e-mail] 
> <mailto:[e-mail]>> wrote:
>
>     Most of the items which I felt were somewhat deployment-specific
>     constants(ie: Initial Window and Initial RTT) are 'RECOMMENDED' in
>     the current recovery draft, so I thought it'd be preferable to
>     keep that convention.
>
>
> Thanks for helping me understand - this is a stylistic change.
>
> I've had a couple of conversations outside of QUIC recently with 
> people who were interpreting the two terms somewhat differently, so 
> thought it a good time to ask.
>
> Best,
>
> Spencer
>
>
>     On Mon, Jul 20, 2020 at 5:30 PM Spencer Dawkins at IETF
>     <[e-mail]
>     <mailto:[e-mail]>> wrote:
>
>         Hi, Ian,
>
>         On Mon, Jul 20, 2020 at 11:49 AM Ian Swett
>         <ianswett=[e-mail]
>         <mailto:[e-mail]>> wrote:
>
>             I think it's ok to include burst limits in the set of
>             things we expect may be different in DCs, but if we do
>             that, I'd prefer to use the RECOMMENDED rather than SHOULD,
>
>
>         Aren't RECOMMENDED and SHOULD equivalent in
>         https://tools.ietf.org/html/rfc2119?
>
>         Or am I misunderstanding your point?
>
>
>             3 <https://tools.ietf.org/html/rfc2119#section-3>. SHOULD
>
>         This word, or the adjective "RECOMMENDED", mean that there may
>         exist valid reasons in particular circumstances to ignore a
>         particular item, but the full implications must be understood
>         and carefully weighed before choosing a different course.
>
>         Best,
>
>         Spencer
>
>             since there are a number of SHOULDs which I believe apply
>             both to the public internet and to DCs.  Then we could add
>             a note at the top about how there are some values which
>             are RECOMMENDED, including X, Y and Z, and those
>             recommendations are expected to be good choices for most,
>             but not all environments.
>
>             In terms of text, I'd suggest taking Neal's suggestion and
>             dropping an explicit mention of the public internet:
>
>             Implementations MUST either use pacing or another method
>             to limit such bursts.
>
>             It is RECOMMENDED that implementations limit bursts to the
>             initial congestion window; seeSection 7.2  <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftools.ietf.org%2Fhtml%2Fdraft-ietf-quic-recovery-29%23section-7.2&data=02%7C01%7Cpravb%40microsoft.com%7C9df3e5f4f0ca4e586f0908d824e0a3ad%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637299894526657906&sdata=ze%2F7SpKUbrdgwI6m%2BS6uEGSBgc2NapVtPn9vkOYFjVs%3D&reserved=0>.
>
>
>             On Sun, Jul 19, 2020 at 4:08 PM Praveen Balasubramanian
>             <pravb=[e-mail]
>             <mailto:[e-mail]>> wrote:
>
>                 >> There's a way of satisfying both desires: have the
>                 NIC handle the pacing.
>
>                 Yes letting the NIC handle pacing will make improve
>                 CPU efficiency and improve accuracy due to fine
>                 grained hardware timers. But that’s in the future.
>                 Today’s NICs don’t pace large send offloads for TCP or
>                 UDP.
>
>                 >> Perhaps we can have a principle here:
>                 recommendations that are specific for Internet use are
>                 just that, and we use SHOULDs for those. IW10 makes
>                 sense based on this, and I would then also be fine
>                 with changing the MUST to a SHOULD. Perhaps we can
>                 state this principle upfront.
>
>                 I like the idea of stating that principle up front.
>                 SHOULD would be sufficient resolution for the burst
>                 size issue.
>
>                 *From:* QUIC <[e-mail]
>                 <mailto:[e-mail]>> *On Behalf Of * Jana
>                 Iyengar
>                 *Sent:* Friday, July 17, 2020 7:18 PM
>                 *To:* QUIC WG <[e-mail] <mailto:[e-mail]>>
>                 *Subject:* [EXTERNAL] Re: WGLC review of
>                 draft-ietf-quic-recovery-29
>
>                 There's a protocol question and there's a question of
>                 endpoint behavior. In terms of the protocol itself,
>                 yes, there's no real need to distinguish between
>                 Internet and DC environments; we've tried to ensure
>                 that the protocol can be used broadly. My point was
>                 that the constants in the spec were based on what we
>                 believe to be true for the public Internet, and not
>                 for DC environments.
>
>                 That said, perhaps I was a bit too hasty. IW10 and
>                 InitialRTT values are the others I was thinking about,
>                 but those are recommendations in the spec, not
>                 requirements. And as Ian notes, there's no minimum
>                 timeout anymore.
>
>                 Perhaps we can have a principle here: recommendations
>                 that are specific for Internet use are just that, and
>                 we use SHOULDs for those.
>
I don't agree, see below.
>
>                 IW10 makes sense based on this, and I would then also
>                 be fine with changing the MUST to a SHOULD. Perhaps we
>                 can state this principle upfront.
>
>                 Ian, I share your hesitation that we don't want to
>                 make a distinction between private and public
>                 networks, but we already allow for implementations to
>                 do that with a different IW and Initial RTT. Is it
>                 different when talking about burst limits?
>
>                 On Fri, Jul 17, 2020 at 3:07 PM Jeremy Harris
>                 <[e-mail] <mailto:[e-mail]>> wrote:
>
>                     On 08/07/2020 22:29, Praveen Balasubramanian wrote:
>                     > Section 7.9
>                     > "Implementations MUST either use pacing or
>                     another method to limit such bursts to the initial
>                     congestion window; see Section 7.2."
>                     > This seems to preclude use of segmentation
>                     offload of sizes greater than IW.. In datacenters
>                     we routinely send bursts that are higher without
>                     causing loss. The MUST here seems unnecessary. It
>                     also conflicts with the RECOMMENDED in an earlier
>                     sentence.
>
>                     There's a way of satisfying both desires: have the
>                     NIC handle the
>                     pacing.
>                     -- 
>                     Cheers,
>                       Jeremy
>

I am unsure that trying to differentiate DC from ISP-Edge from Enterprise Server
or anything else will help in the spec. Sure, guidance may differ - but I'd like
to think (???) the IETF Spec describes the "Internet case, which
is expected to be good choices for most, but not all environments... and if a system distributes
functions within your own network segment then that's your decision - and yes,
and doing this in NICs would be great ... although NICs also can generate bursts
when transports pace at a higher level...

The pacing/burst-limiting could happen in various ways by various actors, but
that's not the way transport Specs are written, I thought that's why we ended-up with
"other methods".

For me at least, it was a conscious decision to not to explicitly make this "10 packets",
because an endpoint sender that made an informed decision to use a larger IW
has at the same time decided to configure in a way that expects the network path can safely
support this burst size. I expect this matters most when the
connection sending rate is low (see below). But we missed something.

I think the suggested addition from Ian, could really help a lot:
"unless the pacing system can't provide enough sending opportunities to send at the rate specified by the congestion controller."

- This would make the language less clear, but I think in a very good way, because I agree that as the connection **sending** rate increases (and hence also the CC understanding of
the path characteristics), the effect of bursts might be expected to be less significant. At higher rates queues in devices clear more quickly, and buffering 1mS (or whatever) of data, buffers more.

Can we discuss that proposal more?

Gorry