Re: [tsvwg] start of WGLC on L4S drafts

["Holland, Jake" <[e-mail]>]

Hi Bob,

Thanks for your response. For the most part I'm satisfied, but with
regard to the responses to points #2 and #3, I'll top-post my remaining
problem and ask that you suggest something appropriate to address it.

The current text, as far as I can tell, does not *require* any attempt
to detect problems due to "legacy queue" issues (only "SHOULD" language
here, I think?), and then makes the safety response of failing over to
classic CC conditional on "detecting problems".  Certainly if there is
a "MUST" or "SHALL" requirement, it's hard for me to nail down exactly
where.

As has been discussed, without some quite complex detection efforts,
detection of problems is a major challenge with this technology, and
thus the current text overall seems too weak to me on this point.

IMO, the docs need to make it clear that sending with this technology
comes with a mandatory duty to look for and respond to likely problems
caused to other senders due to the non-backward-compatible signaling
response.

Best,
Jake


From: Bob Briscoe <[e-mail]>
Date: Sun,2021-11-07 at 8:40 AM
To: "Holland, Jake" <[e-mail]>, Wesley Eddy <[e-mail]>, "[e-mail]" <[e-mail]>
Subject: Re: [tsvwg] start of WGLC on L4S drafts

Jake,

Thank you for your new reviews. Pls see [BB] inline...
On 21/08/2021 21:11, Holland, Jake wrote:
Overall:
The documents are in much better shape than the last time I reviewed
them, thanks for all the improvements.

I'm reviewing l4s-id and l4s-arch but didn't have time to get to
dualq-aqm yet.  But I wanted to make sure these got posted, and
might not have more time before the deadline.

----

draft-ietf-tsvwg-ecn-l4s-id-19:
There's a few major issues.  These really should be fixed before
sending it to the IESG, IMO, but with these fixed I'd be happy to
see it shipped.

Major:

1. This should be proposed standard, not experimental.

[snip]

[BB] See separate thread.




2. This text from Section 4.3 should be strengthened to recommend
detection of paths that might be classic-marking L4S traffic (shared
queue or not):

   o  In uncontrolled environments, monitoring MUST be implemented to
      support detection of problems with an ECN-capable AQM at the path
      bottleneck that appears not to support L4S and might be in a
      shared queue.

Even in fq, the standing queue that will be built by a scalable cc with
classic marking is contrary to "1-2ms latency at 99th percentile" goal
of L4S, even leaving aside the harm to colliding flows (though of course
the harm to colliding flows, including from hash collisions in fq, is
also another reason to make this change).

[BB] If we make this requirement so that monitoring MUST report all Classic ECN AQMs, operators could just have a sea of data that doesn't say whether there are any of the single-queue AQMs out there - much more likley to be a potential problem than FQ AQMs. The key word in this requirement is 'problems'. 

Also note that the word 'might' in 'might be in a shared queue' allows for the case of a flow-queue that is exhibiting problems at the time it is monitored (e.g. during real time monitoring when within a VPN). BTW, we created a matrix of cases, and checked all the possible cases when we wordsmithed this requirement.

Also, detecting a flow-queue that is ECN marking but appears to exceed expected L4S delay is already in the flow's self-interest. So I wouldn't worry about trying to legislate over-precisely about this case. For instance, of the two OS developers that I have had detailed conversations with about this, both are working on the basis that it's worth taking sthg like the max of the responses to increasing delay and to ECN marks, which will automatically lead to a Classic-like response in an FQ-CoDel queue, as well as in other cases like suddenly reduced capacity.



3. This text from Section 4.3 should be strengthened to require that
sending L4S traffic in uncontrolled environments does not happen when
classic marking of L4S traffic is detected for a shared queue, and at
least recommend that it not happen even for fq:

   o  In uncontrolled environments, monitoring MUST be implemented to
      support detection of problems with an ECN-capable AQM at the path
      bottleneck that appears not to support L4S and might be in a
      shared queue.  Such monitoring SHOULD be applied to live traffic
      that is using Scalable congestion control.  Alternatively,
      monitoring need not be applied to live traffic, if monitoring has
      been arranged to cover the paths that live traffic takes through
      uncontrolled environments. 

The current text requires monitoring, but only gives a single SHOULD for
live traffic, plus non-normatively permits one alternative.  This allows
operators to monitor but not cut off (so this requirement as currently
written would be satisfied by write-only logging for instance, with the
SHOULD easily dismissable with an "implementation complexity" hand-wave
while still following the spec).

Suggested alternative, feel free to edit:

   o  In uncontrolled environments, L4S traffic MUST NOT be sent without
      monitoring to detect marking of L4S traffic by non-L4S bottlenecks.
      This monitoring can for example be performed on live traffic, or
      can rely on monitoring that covers the paths live traffic takes
      through uncontrolled environments.  Where non-L4S bottlenecks are
      observed marking L4S traffic, L4S sending MUST be disabled if the
      bottleneck is a shared queue, and SHOULD be disabled if it is FQ.

[BB] Aside: Your middle sentence has (intentionally?) lost the 'SHOULD' preference for monitoring live, rather than out of band.

I think the next para in the draft (not given in your quote) with its mandatory final sentence is stronger than your alternative.
      The detection function SHOULD be capable of making the congestion
      control adapt its ECN-marking response to coexist safely with
      Classic congestion controls such as standard Reno [https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/rfc5681__;!!GjvTz_vk!Hnqqz6_j8SIdAfFyCkZ-d1cMB2kGbO96HKTk-QSU1VUYsO2YkIK23b-uFrhRnEc$], as
      required by [https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/rfc5033__;!!GjvTz_vk!Hnqqz6_j8SIdAfFyCkZ-d1cMB2kGbO96HKTk-QSU1VUYsO2YkIK23b-uqCDg5WI$].  Alternatively, if adaptation is not
      implemented and problems with such an AQM are detected, the
      scalable congestion control MUST be replaced by a Classic
      congestion control.
The differences are:
* Replacement is a 'MUST' not a 'SHOULD', but conditional on 'problems'
* It doesn't distinguish between shared queue and FQ.

We didn't think it was reasonable to give the sender a requirement that depends on what type of queue it encounters, because the sender doesn't know the type for sure. That's why we made this conditional on 'problems' instead. The last sentence of this passage from the referenced Appendix A.1.5 explains why:
   CDN servers placed within an access ISP's
   network can be considered as a single controlled environment, but any
   onward networks served by the access network, including all the
   attached customer networks, would be unlikely to fall under the same
   degree of coordinated control.  Monitoring is expressed as a 'MUST'
   for these uncontrolled segments of paths (e.g.  beyond the access ISP
   in a home network), because there is a possibility that there might
   be a shared queue Classic ECN AQM in that segment.  Nonetheless, the
   intent is to only require occasional monitoring of these uncontrolled
   regions, and not to burden CDN operators if monitoring never uncovers
   any potential problems, given it is anyway in the CDN's own interests
   not to degrade the service of its own customers.

In summary I think your suggestion of conditioning on the type of queue is likely to be infeasible and therefore prone to being ignored.
This is not easy but, on balance, I don't think I've seen anything here that is tighter than the existing wording.





4. Although l4s-arch claims that l4s-id satisfies the RFC 4774
requirements, it's hard to tell whether it does so.  Specifically:

4.a. From section 7 of RFC 4774:
   Specifications of alternate ECN semantics must clearly state how they
   address the issues raised in this document, particularly the issues
   discussed in Section 2.

I don't see how issues 2 and 3 from section 2 are covered in l4s-id.
>From section 4 of RFC 4774:
   (2) How does the possible presence of old routers affect the
       performance of the alternate-ECN connections?

   (3) How does the possible presence of old routers affect the
       coexistence of the alternate-ECN traffic with competing traffic
       on the path?

   When alternate semantics are defined for the ECN field, it is
   necessary to ensure that there are no problems caused by old routers
   along the path that don't understand the alternate ECN semantics.

4.b. Section 4 goes on to describe 3 options for how alternate ECN
semantics should be treated.  I don't see a claim in the L4S docs
specifying which of the 3 options the L4S spec for alternate ECN semantics
matches, but it implicitly appears to be trying to say it's option 3
(unsafe) and asserting that the detection and adaptive response satisfies
the requirement for isolation on option 3, I think?

Maybe there are sections in l4s-id that intend to cover these points,
and there just needs to be text listing what they are, but I don't think
the link is obvious enough to satisfy the "clearly state" requirement
from RFC 4774.  So it would be very helpful to add a list of references
to sections that are intended to address these RFC 4774 requirements.

[BB] After guidance from the chairs, the co-authors have been working offlist on a fairly lengthy new subsection in ecn-l4s-id about where the RFC4774 requirements are and are not satisfied, and how that is to be dealt with. Rather than pasting it all here (it's long) you should see it later today once the secretariat allow the draft through (or otherwise when the servers re-open in the morning). And I'm sure the chairs will give everyone time to read it over, and review it.






Nits:
- the list in section 7.1 has a weird formatting problem for the sub-
  list:

   o  Did use of L4S over the Internet result in improvements to the
      following metrics:

   o

      *  queue delay (mean and 99th percentile) under various loads;

[BB] Thx - (had already noticed this one myself as well).

Continued...




----

draft-ietf-tsvwg-l4s-arch-10:
Summary: Almost ready

+1 to Gorry's comments here, especially regarding the use of "all traffic":
https://urldefense.com/v3/__https:/mailarchive.ietf.org/arch/msg/tsvwg/vMMsQpXs65lk1E7NpV5RlmpyqdI/__;!!GjvTz_vk!Hnqqz6_j8SIdAfFyCkZ-d1cMB2kGbO96HKTk-QSU1VUYsO2YkIK23b-ul4tJbII$

[BB] Accepted - see response to Gorry/Alex.




Minor:
1. l4s-arch section 1:
   It has been demonstrated that, once access network bit rates reach
   levels now common in the developed world, increasing capacity offers
   diminishing returns if latency (delay) is not addressed.
- This needs a reference.

[BB] I've added:
  [Dukkipati15]
              Dukkipati, N. and N. McKeown, "Why Flow-Completion Time is
              the Right Metric for Congestion Control", ACM CCR 
              36(1):59--62, January 2006,
              https://urldefense.com/v3/__https:/dl.acm.org/doi/10.1145/1111322.1111336__;!!GjvTz_vk!Hnqqz6_j8SIdAfFyCkZ-d1cMB2kGbO96HKTk-QSU1VUYsO2YkIK23b-un5qa5gw$.
  [Rajiullah15]
              Rajiullah, M., "Towards a Low Latency Internet:
              Understanding and Solutions", Masters Thesis; Karlstad
              Uni, Dept of Maths & CS 2015:41, 2015, https://urldefense.com/v3/__https:/www.diva-portal.org/smash/get/diva2:846109/FULLTEXT01.pdf__;!!GjvTz_vk!Hnqqz6_j8SIdAfFyCkZ-d1cMB2kGbO96HKTk-QSU1VUYsO2YkIK23b-ud1apytw$.



Editorial:
1. l4s-arch 4.2 section a:
- This is a confusing wall of text.  I think it would be better to
  give a much briefer summary here with a reference.  Exposition
  like "the obvious part" and "the less obvious part" are a minus
  here--I don't think the obviousness claims made here generalize
  well.

[BB] When it only referenced the DualQ draft and only gave a short summary, we were asked to summarize more fully how the DualQ works here (sigh!).

So we've kept the explanation here, but de-confused and de-walled it a bit:
* Removed the 'obvious/non-obvious' wording.
* Within the largest block: Introduced sub-bullets for the two parts of the semi-permeable membrane, each starting "Latency isolation:" and "Bandwidth pooling:". Then continued the part about the scheduler with 2 further bullets, again starting each with "for latency isolation" and "for bandwidth pooling".
* Cut out some redundancy.
* I considered shifting out the last para of the DualQ bullet, which is about the DualQ document, but I decided there was no better place for it.

It's not brilliant, but I think it's better.




2. All the uses of underlining for emphasis are a minus.  The
  places where it seems necessary or useful are a good hint that
  the text on its own is not adequately capturing the intended
  meaning.
  Leaning on this kind of toned emphasis makes assumptions about
  connotations that don't hold very well even for native English
  speakers and break down entirely for non-native speakers, so they
  are generally out of place in a technical document that will need
  to be correctly interpreted by an international audience with many
  non-native speakers, IMO.

[BB] I agree, use of emphasis is excessive. I've taken out /some/ but not /all/ ;)

The ones left are:
* "on /average/" (because the comparison with the other P99 is 3 lines of dense numbers away)
* "without the /need/ for per-flow operations" (to help the reader not to just run over the word without thinking)
* "without /requiring/ inspection" (---ditto---)

Hopefully, this is now down to a non-irritating level for those who don't like it, and now conforms to the guidance from the only source I could quickly find on international technical writing style:
"Do not use italics for
• mere emphasis. (Italics are acceptable if emphasis might otherwise be lost; in general, however, use syntax to provide emphasis.)" 
American Psychological Association (APA, 2009, pp. 104-106)
via https://urldefense.com/v3/__https:/writing.stackexchange.com/questions/10750/when-should-i-use-italics-in-scientific-writing__;!!GjvTz_vk!Hnqqz6_j8SIdAfFyCkZ-d1cMB2kGbO96HKTk-QSU1VUYsO2YkIK23b-uHTBTecE$

(BTW, underscores in ASCII can mean either italics or underlining. In the HTML rendering, these words come out in italics - which was the intention)




----

I'm going to be too pressed for time to do a more detailed review,
but I wanted to get the above comments in.

As a final aside, I'd like to see this happen.  The only goal I'm
pursuing at this point wrt this work is avoiding preventable harm
from pushing this out in a way that's likely to cause confusion
when and if problems are encountered in production.  (In particular,
I have given up on efforts to improve the signaling design, since
the authors have rejected all such suggestions and this work needs
to get moved out of the wg one way or another.)

[BB] On signalling design, I'd like to point out that it's not just the authors - the chairs have judged that the WG wants to go this way. 

Whatever, thank you - for your reviews, and for your continuing perseverance and diligence.

Cheers

Bob




Best,
Jake


On 07-29, 9:18 AM, "Wesley Eddy" mailto:[e-mail] wrote:

This message is starting a combined working group last call on 3 of the 
L4S drafts:

- Architecture: https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/draft-ietf-tsvwg-l4s-arch/__;!!GjvTz_vk!Hnqqz6_j8SIdAfFyCkZ-d1cMB2kGbO96HKTk-QSU1VUYsO2YkIK23b-u3NnimRI$ 

- DualQ: 
https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/draft-ietf-tsvwg-aqm-dualq-coupled/__;!!GjvTz_vk!Hnqqz6_j8SIdAfFyCkZ-d1cMB2kGbO96HKTk-QSU1VUYsO2YkIK23b-uscUhkHw$ 

- ECN ID: https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/draft-ietf-tsvwg-ecn-l4s-id/__;!!GjvTz_vk!Hnqqz6_j8SIdAfFyCkZ-d1cMB2kGbO96HKTk-QSU1VUYsO2YkIK23b-usmeCGjo$ 

The WGLC will last through 4 weeks from today, and then we'll see what 
to do next.  Please submit any comments you have on these to the TSVWG 
list in that timeframe.

The chairs are considering a possible virtual interim following the 
close in order to work through feedback received.

The work on the L4S operational guidance draft is continuing in 
parallel, but that draft is not being last called yet.





-- 
________________________________________________________________
Bob Briscoe                               https://urldefense.com/v3/__http:/bobbriscoe.net/__;!!GjvTz_vk!Hnqqz6_j8SIdAfFyCkZ-d1cMB2kGbO96HKTk-QSU1VUYsO2YkIK23b-ucdq-oCs$