Re: [tcpm] AD Review of draft-ietf-tcpm-hystartplusplus-09

[Praveen Balasubramanian <[e-mail]>]

That text looks good to me. I am for removing the reference to RFC3465.

Re. L>=2 I am fine with it. The only reason I used L>=1 is because RFC 5681
unfortunately has this text:

We note that [RFC3465 <https://www.rfc-editor.org/rfc/rfc3465>] allows
for cwnd increases of more than SMSS
   bytes for incoming acknowledgments during slow start on an
   experimental basis; however, such behavior is not allowed as part of
   the standard.


We can publish a new revision with these changes.




On Thu, Sep 8, 2022 at 7:52 PM Neal Cardwell <[e-mail]> wrote:

>
>
> On Thu, Sep 8, 2022 at 10:03 PM Praveen Balasubramanian <
> [e-mail]> wrote:
>
>> Thanks for the review and the comments. I agree with Neal that burst
>> control, pacing, and L limits should be separated from Hystart++, and
>> should have their own RFC. One path forward would be to still refer to
>> RFC3465 for context on ABC and origin of the L parameter but provide an
>> updated recommendation for L based on deployment experience. Would the
>> following updates be acceptable?
>>
>> In Section 4.2
>>
>> The following pseudocode integrates Appropriate Byte Counting as
>>    described in [RFC3465 <https://datatracker.ietf.org/doc/html/rfc3465>]. See section 4.3 for an updated recommendation of variable L based on deployment experience.
>>
>>
> I think Martin is suggesting that it would be nice to not have the
> Hystart++ RFC depend on ABC in any direct way, and I like that direction.
>
> What do folks think about something like the following text, which Martin
> and I have been collaborating on:
> """
> 4.2. Algorithm Details
>
> The following pseudocode uses a limit, L, to control the aggressiveness of
> the cwnd increase during both standard slow start and CSS. While an
> arriving ACK may newly acknowledge an arbitrary number of bytes, the
> Hystart++ algorithm limits the number of those bytes applied to increase
> the cwnd to L*MSS bytes.
> """
>
>
>> In Section 4.3
>>
>> A paced TCP implementation SHOULD use L=Infinity. Burst concerns are mitigated by pacing and this setting allows for optimal cwnd growth on modern networks.
>>
>> A non-paced TCP implementation SHOULD use L >=1 and L <=8. L values larger than 8 can cause increase in burstiness and loss rates and result in poor performance.
>>
>>
> This sounds good to me, although I would recommend "L >=2 and L <=8".
>
> RFC3465 already says: "This document specifies that TCP implementations
> MAY use L=2*SMSS bytes", so based on that and experience with Linux's
> L=infinity I think it's definitely fine to have Hystart++ say stacks MAY
> use L=2. I would argue that stacks SHOULD use at least 2, given delayed
> ACKs.
>
> In Section 4.3, how about something like:
>
>
> """
>
> A paced TCP implementation SHOULD use L=Infinity. Burst concerns are mitigated by pacing and this setting allows for optimal cwnd growth on modern networks.
>
>
> A non-paced TCP implementation SHOULD use L >=2 and L <=8. L values
> smaller than 2 can suffer performance problems from the negative impact of
> the standard delayed acknowledgment algorithm, which often sends one
> acknowledgment per 2*SMSS bytes. L values larger than 8 can cause
> increases in burstiness and loss rates and result in poor performance.
> """
>
>
>> This draft is already updating slow start behavior from RFC 5681, so I
>> think its within scope to update the recommendation on use of L based on
>> deployment experience on the Internet and modern networks.
>>
>> Suggestions are welcome.
>>
>
> I agree it makes sense to consider an update to the recommendation on L
> values, but I like Martin's idea to try to avoid dependencies on ABC, so
> that ABC can be retired at some point.
>
> cheers,
> neal
>
>
>
>>
>> On Thu, Sep 8, 2022 at 2:24 PM Martin Duke <[e-mail]>
>> wrote:
>>
>>> I am happy with that formulation, but want community consensus for the
>>> numbers (1 and infinity)
>>>
>>> On Thu, Sep 8, 2022 at 2:12 PM <[e-mail]> wrote:
>>>
>>>> > On 8. Sep 2022, at 18:53, Martin Duke <[e-mail]>
>>>> wrote:
>>>> >
>>>> > Hi Neal, thanks for the data.
>>>> >
>>>> > Given its wide deployment, I'm happy for the spec to allow L_max =
>>>> infinity, assuming the community is fine with it.
>>>> So would you be happy with:
>>>>
>>>> The positive integer L SHOULD be 1 and MAY be larger than 1.
>>>>
>>>> Best regards
>>>> Michael
>>>> >
>>>> > I don't care that much, in theory, whether L limits are defined in
>>>> this document or in another one. However, in practice adding about 2
>>>> paragraphs to this document would allow us to obsolete 3465. If we can
>>>> rapidly reach consensus on this, it seems like the expeditious thing to do.
>>>> >
>>>> > The alternative is to simply eliminate L from Hystart++, and we
>>>> "really know" what people will do in production.
>>>> >
>>>> > On Thu, Sep 8, 2022 at 9:42 AM Neal Cardwell <[e-mail]>
>>>> wrote:
>>>> >
>>>> >
>>>> > On Wed, Sep 7, 2022 at 5:46 PM <[e-mail]> wrote:
>>>> > > On 7. Sep 2022, at 22:44, Martin Duke <[e-mail]>
>>>> wrote:
>>>> > >
>>>> > >
>>>> > >
>>>> > > On Wed, Sep 7, 2022 at 1:15 PM <[e-mail]> wrote:
>>>> > > > On 7. Sep 2022, at 21:40, Martin Duke <[e-mail]>
>>>> wrote:
>>>> > > >
>>>> > > > I would be happier with some sort of limit on L. Maybe: the
>>>> positive integer L SHOULD be 2 and MUST be no more than 8(?) But whatever
>>>> numbers the community is comfortable with are fine with me.
>>>> > > Hi Martin,
>>>> > >
>>>> > > the crucial point here is that we know that MS used L = 8. But we
>>>> don't have any information,
>>>> > > whether other values of L are worse or better, what the tradeoffs
>>>> are and in particular
>>>> > > what traffic patterns impact the choice of L. Wouldn't we need such
>>>> an analysis / results
>>>> > > from an experiment for selecting an upper limit L_MAX? I would
>>>> assume that it might
>>>> > > have an impact whether pacing is used or not.
>>>> > >
>>>> > > Well the community has at least some data that L = 8 is safe on the
>>>> internet. Maybe some other practitioners have data for 16 or 32 or
>>>> whatever. It would be reasonable to set some sort of limit based on the
>>>> limits of our empirical limit.
>>>> > Sure. My point was just that the results provided my MS show that
>>>> L_MAX <= 8 seems to
>>>> > be safe in the scenario they looked at. But there is no statement
>>>> that L_MAX > 8 is
>>>> > bad...
>>>> > >
>>>> > > It would be fine to say "one SHOULD NOT exceed L = foo if you're
>>>> not pacing"
>>>> > >
>>>> > >
>>>> > > >
>>>> > > > I'd also be happier with a cut-and-paste of those considerations
>>>> from 3465, which should not be a lot of work.
>>>> > > >
>>>> > > > The lower-effort approach is just to strike L from the document
>>>> and let people do what they've doing, which is use Ls well in excess of 2.
>>>> But I'd prefer we'd actually write it down because
>>>> > > > (1) our standards are better if they reflect widespread behavior
>>>> in the internet; and
>>>> > > I agree with the above.
>>>> > > > (2) it means we can finally make 3465 historic, as we have
>>>> standards-track documents that cover virtually all of its material.
>>>> > > >
>>>> > > > If the community really cannot converge on sensible limits for L,
>>>> then I'd rather publish with no L than wait for a deadlock to resolve.
>>>> > > In my view, the selection of L is not part of the core of the
>>>> document.
>>>> > >
>>>> > > This document standardizes a new slow start algorithm, which
>>>> includes L as a parameter, for which there is no standard.
>>>> > My point was that there is no L_MAX given here.
>>>> > >
>>>> > > >
>>>> > > > But as a way forward, I'd suggest:
>>>> > > > 1) the authors propose text that sets bounds on L (a SHOULD and a
>>>> MUST) and cut-and-pastes the considerations.
>>>> > > Aren't the consideration in RFC 3465 coming to the conclusion: L
>>>> SHOULD be 1, MAY be 2, MUST NOT be larger than 2?
>>>> > > How to use the same considerations to come to a different
>>>> conclusion: L SHOULD be 2, MUST NOT be larger than L_MAX
>>>> > > for L_MAX > 2?
>>>> > >
>>>> > > Re: SHOULD 1 or 2, I don't have a strong opinion but I feel like
>>>> the 3465 experiment shows that 2 is safe, and somewhat beneficial given the
>>>> prevalence of ack thinning? If there is no consensus for this, I'm happy to
>>>> go with 1.
>>>> > >
>>>> > > L_MAX: My preference is to set a value that is widely deployed with
>>>> good data that it is safe to do so
>>>> > Let us see if people provide data which values are good and which
>>>> values are bad...
>>>> >
>>>> > FWIW, Linux TCP has been using L_MAX=infinity since 2013:
>>>> >
>>>> >
>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9f9843a751d0a2
>>>> >
>>>> > IMHO trying to limit bursts with an L parameter is misguided:
>>>> >
>>>> > + AN L limit is only a partial solution; TCP RFCs still allow massive
>>>> line-rate bursts of cwnd when restarting from idle, which is super-common
>>>> in some very common Internet workloads: web traffic, streaming video, RPC.
>>>> >
>>>> > + To avoid such bursts, whether restarting from idle or not, TCP
>>>> stacks should enable pacing.
>>>> >
>>>> > + Once pacing is enabled, having an L limit purely shoots your flow
>>>> in the foot, causing it to fail to double cwnd each round trip in slow
>>>> start in the very common case of the many ubiquitous aggregation mechanisms
>>>> that cause ACKs to arrive for dozens of packets at a time (e.g.. Linux
>>>> GRO/LRO commonly aggregate 45 packets into a single unit that is processed
>>>> and ACKed as a unit).
>>>> >
>>>> > IMHO ideally burst control, pacing, and L limits should be separated
>>>> from Hystart++, and should have their own RFC.
>>>> >
>>>> > And whatever is done, at a minimum Hystart++ should not depend on
>>>> RFC3465 (ABC) (though it could mention it in passing to provide context).
>>>> >
>>>> > my two cents,
>>>> > neal
>>>> >
>>>> > _______________________________________________
>>>> > tcpm mailing list
>>>> > [e-mail]
>>>> > https://www.ietf.org/mailman/listinfo/tcpm
>>>>
>>>>