Re: [tsvwg] L4S drafts: Next Steps

[Gorry Fairhurst <[e-mail]>]

I also agree on that...

On 12/03/2021 02:43, Black, David wrote:
> [...skipping to the end ...]
>
>>> To purse such a course of action, "MUST implement monitoring to detect non_L4S ECN AQM..." cannot survive as
>>> a normative requirement (MUST) for use of the ECT(1) codepoint in the L4S experiment, although it would be fine
>>> to encourage that monitoring to be implemented.  If this sort of approach of converting problematic requirements
>>> to implementation guidance and/or goals to be evaluated by the experiment is followed for all of the "Needs experimental data"
>>> items, then the resulting WG consensus call on feasibility of the remaining requirements (effectively the rest of the contents
>>> of Koen's slides, including removal of the objectionable documentation requirements) should be quick and easy.
>>>
>>> The l4s-ops draft would need to align with any resulting changes.
>> [BB] This is fine, and good, and consistent with what we've been led to understand. Thank you.
> If this is what the L4S team intends to do, then we're aligned, and I see no need to respond to most of the discussion skipped above.
>
> I'll attempt clarify one more thing:
>
>> Say, for instance that the second implementer likely to do the first full set of testing is Google with BBRv2.
>> They have done person-years of work but, with this sudden call to bring you a different rock, all their work
>> suddenly won't count, 'cos their implementation is over TCP, which is the same transport protocol as TCP Prague.
> For the purposes under discussion, TCP Prague and TCP BBRv2 look like different transport protocols to me.
>
> Thanks, --David

+1.  I was looking for different ways of instantiating the method - not 
necessarilly in different protocols, and this was helpful to me.

Gorry

>
> -----Original Message-----
> From: Bob Briscoe <[e-mail]>
> Sent: Thursday, March 11, 2021 8:48 PM
> To: Black, David
> Cc: [e-mail]; Jonathan Morton
> Subject: Re: [tsvwg] L4S drafts: Next Steps
>
>
> [EXTERNAL EMAIL]
>
> David,
>
> On 12/03/2021 00:20, Black, David wrote:
>>> What I believe David Black is referring to - and I'm sure he'll
>>> correct me if necessary - is the (hopefully uncontroversial) notion that a specification is useless if it cannot be implemented, and indeed not very useful if distinct implementations are not automatically interoperable with each other.
>> That's the core idea.  In more detail, the transport requirements for use of the L4S service (section 4 of draft-ietf-tsvwg-ecn-l4s-id-14, particularly section 4.3 on congestion response) define a functional interface between the network and transport protocols, an interface that needs to interoperate with a variety of transport protocols in order to realize the L4S goal of being a widely usable service.  Looking back over the past few years, in 20/20 hindsight, some of these requirements were initially written too strictly to the point of being infeasible.
> I am not questioning that there is work to be done to implement those requirements (by us - the authors). I am questioning the notion that at least one developer (other than the authors) has to do all the same work as well, irrespective of how many other developers do many parts of the work.
>
> The functional interfaces are a tiny part of the job, i.e. the ECT(1) codepoint and e2e transport feedback.
>
> However, 90% of the requirements are about performance impact on other hosts (mostly latency but also throughput). I.e. all the safety requirements in 4.3. These are /not/ functional interfaces.
>
> Functional interface requirements are relatively easy to set up tests for. In contrast performance safety testing requires huge resources to test satisfactorily. And the results are easy to rubbish as meaningless if not done over the real Internet (Jonathan can give you the playbook on that). But no-one can do this over the real Internet without the codepoint. So this type of testing requires even more resources to set up an artificial but realistic environment.
>
> I agree this stuff has to be done. It just makes a hugely hard task even harder if you arbitrarily set the bar so that at least one of the developers has to do it /all/ *as well as the authors*. When it would be no less meaningful if testing of each requirement was covered by at least one of the developers (as well as the authors).
>
>> Turning to the current situation, the claim (in email below) that "No-one expects that [meeting these requirements] to be a problem for L4S implementers" does not align with the two "Needs experimental data" slides in Koen's meeting presentation on these requirements (see slides 6 and 7 of: https://datatracker.ietf.org/meeting/110/materials/slides-110-tsvwg-sessb-73-prague-requirements-00).
> [BB] You're now quoting me out of context (presumably due to Jonthan's snippers). My words were:
>>> Where does the concern about transport protocol independence come
>>> from? No-one expects that to be a problem for L4S implementers.
> I was arguing against your specific focus on *transport protocol independence*. That suddenly adds another dimension to the test matrix.
>
> Say, for instance that the second implementer likely to do the first full set of testing is Google with BBRv2. They have done person-years of work but, with this sudden call to bring you a different rock, all their work suddenly won't count, 'cos their implementation is over TCP, which is the same transport protocol as TCP Prague. Yes, Google could probably do the work over QUIC instead". But only if you had asked them 2 years ago. Suddenly inventing this new requirement throws a spanner into everyone's plans. The projects are set up. The people involved have expertise in the particular protocols, then suddenly there's a click of the fingers for a different rock.
>
> Which transport protocol is used is not even important to the requirements in the draft. it's just an arbitrarily different rock.
>
>
>> I have no problem with evaluation of some of these requirements being goals of the L4S experiment - if that is done, then mandating that the requirements involved be met in order to participate in the L4S experiment appears to set the bar for experimental use of ECT(1) too high.  To make this concrete, one of the "Needs experimental data" items (first item on slide 7) is:
>>
>> 		o MUST implement monitoring to detect non_L4S ECN AQM...
>> 		   - Is detection itself required?
>> 		   - Robust detection scheme needs real deployment experience.
>> 		   - Combination with delay based control could minimize potential issues
>> 		   - Develop during experiment as needed.
>>
>> To purse such a course of action, "MUST implement monitoring to detect non_L4S ECN AQM..." cannot survive as a normative requirement (MUST) for use of the ECT(1) codepoint in the L4S experiment, although it would be fine to encourage that monitoring to be implemented.  If this sort of approach of converting problematic requirements to implementation guidance and/or goals to be evaluated by the experiment is followed for all of the "Needs experimental data" items, then the resulting WG consensus call on feasibility of the remaining requirements (effectively the rest of the contents of Koen's slides, including removal of the objectionable documentation requirements) should be quick and easy.
>>
>> The l4s-ops draft would need to align with any resulting changes.
> [BB] This is fine, and good, and consistent with what we've been led to
> understand. Thank you.
> It's the sudden demands for different rocks I am picking up on: sudden
> demands about how many transport protocols, sudden constraints on which
> implementers have to do how much, etc.
>
>
>
> Bob
>
>> Thanks, --David
>>
>> -----Original Message-----
>> From: Jonathan Morton <[e-mail]>
>> Sent: Thursday, March 11, 2021 5:56 PM
>> To: Bob Briscoe
>> Cc: Black, David; [e-mail]
>> Subject: Re: [tsvwg] L4S drafts: Next Steps
>>
>>
>> [EXTERNAL EMAIL]
>>
>
>
>>> On 12 Mar, 2021, at 12:07 am, Bob Briscoe <[e-mail]> wrote:
>>>
>>> The concern is requiring unnecessary work to prove things that are "bring me a different rock" exercises.
>> That does appear to be *your* concern at the moment.  I am merely trying to interpret the concerns of other people for your benefit.  Neither you nor I are the sole arbiters of what valid concerns may or may not exist.  I would however encourage you to consider what I have recently said about "externalised risk"; I can find a link to the YouTube recording of my talk the other day, if that would be helpful.
>>
>> What I believe David Black is referring to - and I'm sure he'll correct me if necessary - is the (hopefully uncontroversial) notion that a specification is useless if it cannot be implemented, and indeed not very useful if distinct implementations are not automatically interoperable with each other.  This is part of the IETF credo of "rough consensus and running code".
>>
>> As he explained, it is not necessary for two independent implementations to actually exist for an Experimental or Proposed Standard RFC, so long as there's a reasonable expectation that a second complete implementation is achievable by an ordinarily skilled practitioner of the art (to borrow some patent law jargon).  The existence of *one* complete implementation would, however, appear to be a prerequisite for such a determination.  If such an implementation is open-source and well documented, it's reasonable to expect a second implementer to refer to it in minor cases of doubt - but the major facets of requirement must also be well described in the specification.
>>
>> I have entirely separate concerns about the specification itself, which I will keep to other threads.
>>
>>    - Jonathan Morton