Re: [tcpm] Thoughts on EXP vs. PS in TCPM

Hi Michael,

while I agree with the two broad acceptance criterial you laid out for
EXP and PS, unless my recollection is wrong, at least for IW10 and TLS
at the time I wouldn't have seen them to pass the PS criteria.

However, as both of these mechansims were quite well understood, with a
good amount of discussion of unindended consequences (IW10), and a
limited impact if they would (unexpectedly) fail to meet their goal
(TLS), I believe the criterial for PS should actually slightly wider...

A change that is in a reasonably well defined space, with no
(forseeable) dramatic consequences to the operation of the tcp protocol
itself, could be allowed for PS even before multiple independent
implementations are available...

Alternatively (which is, what came to pass if I remeber correctly), the
status may need to be desided only very late in the process - if
additional implementations were in the works once the technical details
of a draft settled down.

Stiking a new balance in tcpm for the intended status seems to be in order.

Regards,
   Richard

Am 19.11.2019 um 13:08 schrieb Scharf, Michael:
> [chair hat off]
>
> In TCPM we repeatedly run into the question whether to publish specs as PS or as EXP. We also have several worked examples in which TCPM first published a EXP RFC and later a PS.
>
> This somehow implies that TCPM often follows a process with three maturity levels (EXP, PS, and Internet Standard), while the IETF actually has two maturity levels (see RFC 6410). Formal definitions are:
> * EXP in https://tools.ietf.org/html/rfc2026#section-4.2.1
> * PS in https://tools.ietf.org/html/rfc7127#section-3.1
> * Internet Standard in https://tools.ietf.org/html/rfc6410
>
> As questions on EXP and PS emerges repeatedly, I think it would be useful to discuss two questions:
>
> 1/ What is the minimum requirement for publishing an EXP RFC in TCPM?
>
> 2/ What is the minimum requirement for publishing an PS RFC in TCPM?
>
> I have been thinking myself about these questions quite a bit, most notably since I have recently also worked in working groups outside TSV (such as RTG and OPS). As far as I can tell, other IETF working groups don't make use of EXP as frequently as TCPM. In many working groups PS is the norm.
>
> I start to get worried by that. At the end of the day, the IETF is one standardization organization and TCPM documents matter also to other areas of the IETF, as well as the networking industry as a whole. So, TCPM does not operate in isolation.
>
> Of course, I am partly accountable for that, given that I have run several consensus calls on the target status of RFCs in the last years. I think the community feedback was pretty clear in all these cases, typically in favor of EXP instead of PS. Yet, after more insight in the rest of the IETF, I wonder if maybe TCPM sometimes decides to early to go for EXP if PS is actually possible?
>
> For instance, as individual contributor, I believe that IW10 and TLP could fulfill the PS requirements now. And, in retrospective, I think they were not far away from PS at the time of publication. So, in some sense, TCPM has been very careful and conservative when publishing these RFCs. This now implies that we would have to reissue a PS to actually clean up the RFC services.
>
> Given that other working groups apparently operate differently regarding PS as target, do we need to rethink our minimum bars?
>
> I believe that it would be useful to discuss this question, and I'd be in particular interested in the opinions of implementers.
>
> As a starting point, I offer my own understanding of how to answer the two questions - with chair hat off. So, feel free to disagree... Of course, a case-by-case analysis is always needed, so this is not really a checklist, but I guess we should have some general understanding of what "maturity" really implies.
>
> Michael's thought on question 1/:
>
> In order to start a WGLC in TCPM for EXP status, there SHOULD be one implementation in the main source code tree of an important TCP/IP stack; a default to "off" is perfectly acceptable. Alternatively, there SHOULD be at least one implementation outside the main tree and a certain level of operational experience e.g. in very controlled environments. Comprehensive testing by simulation can also be understood as operational experience if the simulation results are verified independently. (As example, the converter specification meets this requirement in my point of view.)
>
> Note that this requirement is significantly higher than what is written in RFC 2026 - in principle, thousands of research papers on TCP could meet the EXP requirement of RFC 2026. But, at least to me, there is little value in documenting an experiment without proof that it is actually implementable in a TCP stack.
>
> Michael's thought on question 2/:
>
> The requirement follows from RFC 7127, which strongly suggests implementation and operational experience, but it is not strictly required. So, there is some room for interpretation. The bar for PS is _not_ widespread deployment in the Internet. For instance, to me a WGLC for an PS could be started if there are two independent implementations, at least one being deployed in an important TCP stack (i.e., one does not have to be production quality), and some operational experience over a subset of the Internet. For instance, such operational experience can also be tests, it does not require enabling by default. (As example, RACK easily meets these requirements as far as I understand.)
>
> Note that with two independent implementations one gets already relatively close to the requirements of an Internet Standard, but in that case also widespread deployment and some other conditions are required (see RFC 6410). So, the bar for PS must be lower than that.
>
> Please feel free to comment and/or disagree...
>
> Michael
>
> _______________________________________________
> tcpm mailing list
> tcpm@ietf.org
> https://www.ietf.org/mailman/listinfo/tcpm
>