Re: [tsvwg] start of WGLC on L4S drafts

[Bob Briscoe <[e-mail]>]

Jake,

I thought we were only waiting for David B's promised reviews, but I've 
just noticed I never sent my reply to your last email in this thread. 
Sorry about that. See [BB] inline...

On 08/11/2021 14:52, Holland, Jake wrote:
> Hi Bob,
>
> Thanks for your response. For the most part I'm satisfied, but with
> regard to the responses to points #2 and #3, I'll top-post my remaining
> problem and ask that you suggest something appropriate to address it.
>
> The current text, as far as I can tell, does not *require* any attempt
> to detect problems due to "legacy queue" issues (only "SHOULD" language
> here, I think?), and then makes the safety response of failing over to
> classic CC conditional on "detecting problems".  Certainly if there is
> a "MUST" or "SHALL" requirement, it's hard for me to nail down exactly
> where.

[BB] Doesn't the first sentence of the Classic ECN AQM requirement 
(bullet #3 under §4.3) say what you're wanting?:
See 
https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id-22#section-4.3

Possibly you're saying this doesn't make detection mandatory, only 
monitoring. If that's your concern, we can easily fix that.  For 
instance, at the beginning of the second para of this requirement bullet 
we could add the following highlighted sentence.

I don't believe any implementers will object to this, because I imagine 
most thought detection was already mandatory. But if any do object, pls 
do so now!

    3.  In uncontrolled environments, monitoring MUST be implemented to
        support detection of problems with an ECN-capable AQM at the path
        bottleneck that appears not to support L4S and might be in a
        shared queue.  Such monitoring SHOULD be applied to live traffic
        that is using Scalable congestion control.  Alternatively,
        monitoring need not be applied to live traffic, if monitoring has
        been arranged to cover the paths that live traffic takes through
        uncontrolled environments.

        *A function to detect the above problems with an ECN-capable AQM ****MUST also be implemented. *The detection function SHOULD be capable of making the congestion
        control adapt its ECN-marking response to coexist safely with
        Classic congestion controls such as standard Reno [RFC5681  <https://datatracker.ietf.org/doc/html/rfc5681>], as
        required by [RFC5033  <https://datatracker.ietf.org/doc/html/rfc5033>].  Alternatively, if adaptation is not
        implemented and problems with such an AQM are detected, the
        scalable congestion control MUST be replaced by a Classic
        congestion control.


(For context, recall that we separated monitoring and detection to allow 
for the possibility of live monitoring but offline detection, perhaps 
involving analysis of the monitoring logs supported by further offline 
testing).


>
> As has been discussed, without some quite complex detection efforts,
> detection of problems is a major challenge with this technology, and
> thus the current text overall seems too weak to me on this point.
>
> IMO, the docs need to make it clear that sending with this technology
> comes with a mandatory duty to look for and respond to likely problems
> caused to other senders due to the non-backward-compatible signaling
> response.

[BB] Offline detection of a Classic ECN AQM (shared queue or FQ) 
involves a straightforward test (see table 2 at the top of 
https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id-22#page-50 
).



Bob

>
> Best,
> Jake
>
>
> From: Bob Briscoe <[e-mail]>
> Date: Sun,2021-11-07 at 8:40 AM
> To: "Holland, Jake" <[e-mail]>, Wesley Eddy <[e-mail]>, "[e-mail]" <[e-mail]>
> Subject: Re: [tsvwg] start of WGLC on L4S drafts
>
> Jake,
>
> Thank you for your new reviews. Pls see [BB] inline...
> On 21/08/2021 21:11, Holland, Jake wrote:
> Overall:
> The documents are in much better shape than the last time I reviewed
> them, thanks for all the improvements.
>
> I'm reviewing l4s-id and l4s-arch but didn't have time to get to
> dualq-aqm yet.  But I wanted to make sure these got posted, and
> might not have more time before the deadline.
>
> ----
>
> draft-ietf-tsvwg-ecn-l4s-id-19:
> There's a few major issues.  These really should be fixed before
> sending it to the IESG, IMO, but with these fixed I'd be happy to
> see it shipped.
>
> Major:
>
> 1. This should be proposed standard, not experimental.
>
> [snip]
>
> [BB] See separate thread.
>
>
>
>
> 2. This text from Section 4.3 should be strengthened to recommend
> detection of paths that might be classic-marking L4S traffic (shared
> queue or not):
>
>     o  In uncontrolled environments, monitoring MUST be implemented to
>        support detection of problems with an ECN-capable AQM at the path
>        bottleneck that appears not to support L4S and might be in a
>        shared queue.
>
> Even in fq, the standing queue that will be built by a scalable cc with
> classic marking is contrary to "1-2ms latency at 99th percentile" goal
> of L4S, even leaving aside the harm to colliding flows (though of course
> the harm to colliding flows, including from hash collisions in fq, is
> also another reason to make this change).
>
> [BB] If we make this requirement so that monitoring MUST report all Classic ECN AQMs, operators could just have a sea of data that doesn't say whether there are any of the single-queue AQMs out there - much more likley to be a potential problem than FQ AQMs. The key word in this requirement is 'problems'.
>
> Also note that the word 'might' in 'might be in a shared queue' allows for the case of a flow-queue that is exhibiting problems at the time it is monitored (e.g. during real time monitoring when within a VPN). BTW, we created a matrix of cases, and checked all the possible cases when we wordsmithed this requirement.
>
> Also, detecting a flow-queue that is ECN marking but appears to exceed expected L4S delay is already in the flow's self-interest. So I wouldn't worry about trying to legislate over-precisely about this case. For instance, of the two OS developers that I have had detailed conversations with about this, both are working on the basis that it's worth taking sthg like the max of the responses to increasing delay and to ECN marks, which will automatically lead to a Classic-like response in an FQ-CoDel queue, as well as in other cases like suddenly reduced capacity.
>
>
>
> 3. This text from Section 4.3 should be strengthened to require that
> sending L4S traffic in uncontrolled environments does not happen when
> classic marking of L4S traffic is detected for a shared queue, and at
> least recommend that it not happen even for fq:
>
>     o  In uncontrolled environments, monitoring MUST be implemented to
>        support detection of problems with an ECN-capable AQM at the path
>        bottleneck that appears not to support L4S and might be in a
>        shared queue.  Such monitoring SHOULD be applied to live traffic
>        that is using Scalable congestion control.  Alternatively,
>        monitoring need not be applied to live traffic, if monitoring has
>        been arranged to cover the paths that live traffic takes through
>        uncontrolled environments.
>
> The current text requires monitoring, but only gives a single SHOULD for
> live traffic, plus non-normatively permits one alternative.  This allows
> operators to monitor but not cut off (so this requirement as currently
> written would be satisfied by write-only logging for instance, with the
> SHOULD easily dismissable with an "implementation complexity" hand-wave
> while still following the spec).
>
> Suggested alternative, feel free to edit:
>
>     o  In uncontrolled environments, L4S traffic MUST NOT be sent without
>        monitoring to detect marking of L4S traffic by non-L4S bottlenecks.
>        This monitoring can for example be performed on live traffic, or
>        can rely on monitoring that covers the paths live traffic takes
>        through uncontrolled environments.  Where non-L4S bottlenecks are
>        observed marking L4S traffic, L4S sending MUST be disabled if the
>        bottleneck is a shared queue, and SHOULD be disabled if it is FQ.
>
> [BB] Aside: Your middle sentence has (intentionally?) lost the 'SHOULD' preference for monitoring live, rather than out of band.
>
> I think the next para in the draft (not given in your quote) with its mandatory final sentence is stronger than your alternative.
>        The detection function SHOULD be capable of making the congestion
>        control adapt its ECN-marking response to coexist safely with
>        Classic congestion controls such as standard Reno [https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/rfc5681__;!!GjvTz_vk!Hnqqz6_j8SIdAfFyCkZ-d1cMB2kGbO96HKTk-QSU1VUYsO2YkIK23b-uFrhRnEc$], as
>        required by [https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/rfc5033__;!!GjvTz_vk!Hnqqz6_j8SIdAfFyCkZ-d1cMB2kGbO96HKTk-QSU1VUYsO2YkIK23b-uqCDg5WI$].  Alternatively, if adaptation is not
>        implemented and problems with such an AQM are detected, the
>        scalable congestion control MUST be replaced by a Classic
>        congestion control.
> The differences are:
> * Replacement is a 'MUST' not a 'SHOULD', but conditional on 'problems'
> * It doesn't distinguish between shared queue and FQ.
>
> We didn't think it was reasonable to give the sender a requirement that depends on what type of queue it encounters, because the sender doesn't know the type for sure. That's why we made this conditional on 'problems' instead. The last sentence of this passage from the referenced Appendix A.1.5 explains why:
>     CDN servers placed within an access ISP's
>     network can be considered as a single controlled environment, but any
>     onward networks served by the access network, including all the
>     attached customer networks, would be unlikely to fall under the same
>     degree of coordinated control.  Monitoring is expressed as a 'MUST'
>     for these uncontrolled segments of paths (e.g.  beyond the access ISP
>     in a home network), because there is a possibility that there might
>     be a shared queue Classic ECN AQM in that segment.  Nonetheless, the
>     intent is to only require occasional monitoring of these uncontrolled
>     regions, and not to burden CDN operators if monitoring never uncovers
>     any potential problems, given it is anyway in the CDN's own interests
>     not to degrade the service of its own customers.
>
> In summary I think your suggestion of conditioning on the type of queue is likely to be infeasible and therefore prone to being ignored.
> This is not easy but, on balance, I don't think I've seen anything here that is tighter than the existing wording.
>
>
>
>
>
> 4. Although l4s-arch claims that l4s-id satisfies the RFC 4774
> requirements, it's hard to tell whether it does so.  Specifically:
>
> 4.a. From section 7 of RFC 4774:
>     Specifications of alternate ECN semantics must clearly state how they
>     address the issues raised in this document, particularly the issues
>     discussed in Section 2.
>
> I don't see how issues 2 and 3 from section 2 are covered in l4s-id.
> >From section 4 of RFC 4774:
>     (2) How does the possible presence of old routers affect the
>         performance of the alternate-ECN connections?
>
>     (3) How does the possible presence of old routers affect the
>         coexistence of the alternate-ECN traffic with competing traffic
>         on the path?
>
>     When alternate semantics are defined for the ECN field, it is
>     necessary to ensure that there are no problems caused by old routers
>     along the path that don't understand the alternate ECN semantics.
>
> 4.b. Section 4 goes on to describe 3 options for how alternate ECN
> semantics should be treated.  I don't see a claim in the L4S docs
> specifying which of the 3 options the L4S spec for alternate ECN semantics
> matches, but it implicitly appears to be trying to say it's option 3
> (unsafe) and asserting that the detection and adaptive response satisfies
> the requirement for isolation on option 3, I think?
>
> Maybe there are sections in l4s-id that intend to cover these points,
> and there just needs to be text listing what they are, but I don't think
> the link is obvious enough to satisfy the "clearly state" requirement
> from RFC 4774.  So it would be very helpful to add a list of references
> to sections that are intended to address these RFC 4774 requirements.
>
> [BB] After guidance from the chairs, the co-authors have been working offlist on a fairly lengthy new subsection in ecn-l4s-id about where the RFC4774 requirements are and are not satisfied, and how that is to be dealt with. Rather than pasting it all here (it's long) you should see it later today once the secretariat allow the draft through (or otherwise when the servers re-open in the morning). And I'm sure the chairs will give everyone time to read it over, and review it.
>
>
>
>
>
>
> Nits:
> - the list in section 7.1 has a weird formatting problem for the sub-
>    list:
>
>     o  Did use of L4S over the Internet result in improvements to the
>        following metrics:
>
>     o
>
>        *  queue delay (mean and 99th percentile) under various loads;
>
> [BB] Thx - (had already noticed this one myself as well).
>
> Continued...
>
>
>
>
> ----
>
> draft-ietf-tsvwg-l4s-arch-10:
> Summary: Almost ready
>
> +1 to Gorry's comments here, especially regarding the use of "all traffic":
> https://urldefense.com/v3/__https:/mailarchive.ietf.org/arch/msg/tsvwg/vMMsQpXs65lk1E7NpV5RlmpyqdI/__;!!GjvTz_vk!Hnqqz6_j8SIdAfFyCkZ-d1cMB2kGbO96HKTk-QSU1VUYsO2YkIK23b-ul4tJbII$
>
> [BB] Accepted - see response to Gorry/Alex.
>
>
>
>
> Minor:
> 1. l4s-arch section 1:
>     It has been demonstrated that, once access network bit rates reach
>     levels now common in the developed world, increasing capacity offers
>     diminishing returns if latency (delay) is not addressed.
> - This needs a reference.
>
> [BB] I've added:
>    [Dukkipati15]
>                Dukkipati, N. and N. McKeown, "Why Flow-Completion Time is
>                the Right Metric for Congestion Control", ACM CCR
>                36(1):59--62, January 2006,
>                https://urldefense.com/v3/__https:/dl.acm.org/doi/10.1145/1111322.1111336__;!!GjvTz_vk!Hnqqz6_j8SIdAfFyCkZ-d1cMB2kGbO96HKTk-QSU1VUYsO2YkIK23b-un5qa5gw$.
>    [Rajiullah15]
>                Rajiullah, M., "Towards a Low Latency Internet:
>                Understanding and Solutions", Masters Thesis; Karlstad
>                Uni, Dept of Maths & CS 2015:41, 2015, https://urldefense.com/v3/__https:/www.diva-portal.org/smash/get/diva2:846109/FULLTEXT01.pdf__;!!GjvTz_vk!Hnqqz6_j8SIdAfFyCkZ-d1cMB2kGbO96HKTk-QSU1VUYsO2YkIK23b-ud1apytw$.
>
>
>
> Editorial:
> 1. l4s-arch 4.2 section a:
> - This is a confusing wall of text.  I think it would be better to
>    give a much briefer summary here with a reference.  Exposition
>    like "the obvious part" and "the less obvious part" are a minus
>    here--I don't think the obviousness claims made here generalize
>    well.
>
> [BB] When it only referenced the DualQ draft and only gave a short summary, we were asked to summarize more fully how the DualQ works here (sigh!).
>
> So we've kept the explanation here, but de-confused and de-walled it a bit:
> * Removed the 'obvious/non-obvious' wording.
> * Within the largest block: Introduced sub-bullets for the two parts of the semi-permeable membrane, each starting "Latency isolation:" and "Bandwidth pooling:". Then continued the part about the scheduler with 2 further bullets, again starting each with "for latency isolation" and "for bandwidth pooling".
> * Cut out some redundancy.
> * I considered shifting out the last para of the DualQ bullet, which is about the DualQ document, but I decided there was no better place for it.
>
> It's not brilliant, but I think it's better.
>
>
>
>
> 2. All the uses of underlining for emphasis are a minus.  The
>    places where it seems necessary or useful are a good hint that
>    the text on its own is not adequately capturing the intended
>    meaning.
>    Leaning on this kind of toned emphasis makes assumptions about
>    connotations that don't hold very well even for native English
>    speakers and break down entirely for non-native speakers, so they
>    are generally out of place in a technical document that will need
>    to be correctly interpreted by an international audience with many
>    non-native speakers, IMO.
>
> [BB] I agree, use of emphasis is excessive. I've taken out /some/ but not /all/ ;)
>
> The ones left are:
> * "on /average/" (because the comparison with the other P99 is 3 lines of dense numbers away)
> * "without the /need/ for per-flow operations" (to help the reader not to just run over the word without thinking)
> * "without /requiring/ inspection" (---ditto---)
>
> Hopefully, this is now down to a non-irritating level for those who don't like it, and now conforms to the guidance from the only source I could quickly find on international technical writing style:
> "Do not use italics for
> • mere emphasis. (Italics are acceptable if emphasis might otherwise be lost; in general, however, use syntax to provide emphasis.)"
> American Psychological Association (APA, 2009, pp. 104-106)
> via https://urldefense.com/v3/__https:/writing.stackexchange.com/questions/10750/when-should-i-use-italics-in-scientific-writing__;!!GjvTz_vk!Hnqqz6_j8SIdAfFyCkZ-d1cMB2kGbO96HKTk-QSU1VUYsO2YkIK23b-uHTBTecE$
>
> (BTW, underscores in ASCII can mean either italics or underlining. In the HTML rendering, these words come out in italics - which was the intention)
>
>
>
>
> ----
>
> I'm going to be too pressed for time to do a more detailed review,
> but I wanted to get the above comments in.
>
> As a final aside, I'd like to see this happen.  The only goal I'm
> pursuing at this point wrt this work is avoiding preventable harm
> from pushing this out in a way that's likely to cause confusion
> when and if problems are encountered in production.  (In particular,
> I have given up on efforts to improve the signaling design, since
> the authors have rejected all such suggestions and this work needs
> to get moved out of the wg one way or another.)
>
> [BB] On signalling design, I'd like to point out that it's not just the authors - the chairs have judged that the WG wants to go this way.
>
> Whatever, thank you - for your reviews, and for your continuing perseverance and diligence.
>
> Cheers
>
> Bob
>
>
>
>
> Best,
> Jake
>
>
> On 07-29, 9:18 AM, "Wesley Eddy" mailto:[e-mail] wrote:
>
> This message is starting a combined working group last call on 3 of the
> L4S drafts:
>
> - Architecture: https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/draft-ietf-tsvwg-l4s-arch/__;!!GjvTz_vk!Hnqqz6_j8SIdAfFyCkZ-d1cMB2kGbO96HKTk-QSU1VUYsO2YkIK23b-u3NnimRI$
>
> - DualQ:
> https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/draft-ietf-tsvwg-aqm-dualq-coupled/__;!!GjvTz_vk!Hnqqz6_j8SIdAfFyCkZ-d1cMB2kGbO96HKTk-QSU1VUYsO2YkIK23b-uscUhkHw$
>
> - ECN ID: https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/draft-ietf-tsvwg-ecn-l4s-id/__;!!GjvTz_vk!Hnqqz6_j8SIdAfFyCkZ-d1cMB2kGbO96HKTk-QSU1VUYsO2YkIK23b-usmeCGjo$
>
> The WGLC will last through 4 weeks from today, and then we'll see what
> to do next.  Please submit any comments you have on these to the TSVWG
> list in that timeframe.
>
> The chairs are considering a possible virtual interim following the
> close in order to work through feedback received.
>
> The work on the L4S operational guidance draft is continuing in
> parallel, but that draft is not being last called yet.
>
>
>
>
>

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/