Re: [tsvwg] L4S drafts: Next Steps

[Bob Briscoe <[e-mail]>]

David,

On 12/03/2021 00:20, Black, David wrote:
>> What I believe David Black is referring to - and I'm sure he'll correct me if necessary - is the (hopefully uncontroversial) notion that a specification
>> is useless if it cannot be implemented, and indeed not very useful if distinct implementations are not automatically interoperable with each other.
> That's the core idea.  In more detail, the transport requirements for use of the L4S service (section 4 of draft-ietf-tsvwg-ecn-l4s-id-14, particularly section 4.3 on congestion response) define a functional interface between the network and transport protocols, an interface that needs to interoperate with a variety of transport protocols in order to realize the L4S goal of being a widely usable service.  Looking back over the past few years, in 20/20 hindsight, some of these requirements were initially written too strictly to the point of being infeasible.

I am not questioning that there is work to be done to implement those 
requirements (by us - the authors). I am questioning the notion that at 
least one developer (other than the authors) has to do all the same work 
as well, irrespective of how many other developers do many parts of the 
work.

The functional interfaces are a tiny part of the job, i.e. the ECT(1) 
codepoint and e2e transport feedback.

However, 90% of the requirements are about performance impact on other 
hosts (mostly latency but also throughput). I.e. all the safety 
requirements in 4.3. These are /not/ functional interfaces.

Functional interface requirements are relatively easy to set up tests 
for. In contrast performance safety testing requires huge resources to 
test satisfactorily. And the results are easy to rubbish as meaningless 
if not done over the real Internet (Jonathan can give you the playbook 
on that). But no-one can do this over the real Internet without the 
codepoint. So this type of testing requires even more resources to set 
up an artificial but realistic environment.

I agree this stuff has to be done. It just makes a hugely hard task even 
harder if you arbitrarily set the bar so that at least one of the 
developers has to do it /all/ *as well as the authors*. When it would be 
no less meaningful if testing of each requirement was covered by at 
least one of the developers (as well as the authors).

>
> Turning to the current situation, the claim (in email below) that "No-one expects that [meeting these requirements] to be a problem for L4S implementers" does not align with the two "Needs experimental data" slides in Koen's meeting presentation on these requirements (see slides 6 and 7 of: https://datatracker.ietf.org/meeting/110/materials/slides-110-tsvwg-sessb-73-prague-requirements-00).

[BB] You're now quoting me out of context (presumably due to Jonthan's 
snippers). My words were:
>
>> Where does the concern about transport protocol independence come 
>> from? No-one expects that to be a problem for L4S implementers.

I was arguing against your specific focus on *transport protocol 
independence*. That suddenly adds another dimension to the test matrix.

Say, for instance that the second implementer likely to do the first 
full set of testing is Google with BBRv2. They have done person-years of 
work but, with this sudden call to bring you a different rock, all their 
work suddenly won't count, 'cos their implementation is over TCP, which 
is the same transport protocol as TCP Prague. Yes, Google could probably 
do the work over QUIC instead". But only if you had asked them 2 years 
ago. Suddenly inventing this new requirement throws a spanner into 
everyone's plans. The projects are set up. The people involved have 
expertise in the particular protocols, then suddenly there's a click of 
the fingers for a different rock.

Which transport protocol is used is not even important to the 
requirements in the draft. it's just an arbitrarily different rock.


> I have no problem with evaluation of some of these requirements being goals of the L4S experiment - if that is done, then mandating that the requirements involved be met in order to participate in the L4S experiment appears to set the bar for experimental use of ECT(1) too high.  To make this concrete, one of the "Needs experimental data" items (first item on slide 7) is:
>
> 		o MUST implement monitoring to detect non_L4S ECN AQM...
> 		   - Is detection itself required?
> 		   - Robust detection scheme needs real deployment experience.
> 		   - Combination with delay based control could minimize potential issues
> 		   - Develop during experiment as needed.
>
> To purse such a course of action, "MUST implement monitoring to detect non_L4S ECN AQM..." cannot survive as a normative requirement (MUST) for use of the ECT(1) codepoint in the L4S experiment, although it would be fine to encourage that monitoring to be implemented.  If this sort of approach of converting problematic requirements to implementation guidance and/or goals to be evaluated by the experiment is followed for all of the "Needs experimental data" items, then the resulting WG consensus call on feasibility of the remaining requirements (effectively the rest of the contents of Koen's slides, including removal of the objectionable documentation requirements) should be quick and easy.
>
> The l4s-ops draft would need to align with any resulting changes.

[BB] This is fine, and good, and consistent with what we've been led to 
understand. Thank you.
It's the sudden demands for different rocks I am picking up on: sudden 
demands about how many transport protocols, sudden constraints on which 
implementers have to do how much, etc.



Bob

>
> Thanks, --David
>
> -----Original Message-----
> From: Jonathan Morton <[e-mail]>
> Sent: Thursday, March 11, 2021 5:56 PM
> To: Bob Briscoe
> Cc: Black, David; [e-mail]
> Subject: Re: [tsvwg] L4S drafts: Next Steps
>
>
> [EXTERNAL EMAIL]
>



>> On 12 Mar, 2021, at 12:07 am, Bob Briscoe <[e-mail]> wrote:
>>
>> The concern is requiring unnecessary work to prove things that are "bring me a different rock" exercises.
> That does appear to be *your* concern at the moment.  I am merely trying to interpret the concerns of other people for your benefit.  Neither you nor I are the sole arbiters of what valid concerns may or may not exist.  I would however encourage you to consider what I have recently said about "externalised risk"; I can find a link to the YouTube recording of my talk the other day, if that would be helpful.
>
> What I believe David Black is referring to - and I'm sure he'll correct me if necessary - is the (hopefully uncontroversial) notion that a specification is useless if it cannot be implemented, and indeed not very useful if distinct implementations are not automatically interoperable with each other.  This is part of the IETF credo of "rough consensus and running code".
>
> As he explained, it is not necessary for two independent implementations to actually exist for an Experimental or Proposed Standard RFC, so long as there's a reasonable expectation that a second complete implementation is achievable by an ordinarily skilled practitioner of the art (to borrow some patent law jargon).  The existence of *one* complete implementation would, however, appear to be a prerequisite for such a determination.  If such an implementation is open-source and well documented, it's reasonable to expect a second implementer to refer to it in minor cases of doubt - but the major facets of requirement must also be well described in the specification.
>
> I have entirely separate concerns about the specification itself, which I will keep to other threads.
>
>   - Jonathan Morton

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/