Re: [tcpm] Thoughts on EXP vs. PS in TCPM

["Scharf, Michael" <[e-mail]>]

> Hi Michael,
> 
> while I agree with the two broad acceptance criterial you laid out for
> EXP and PS, unless my recollection is wrong, at least for IW10 and TLS
> at the time I wouldn't have seen them to pass the PS criteria.

Yes, the IETF consensus was that IW10 and TFO (which I wanted to write) didn't pass the PS criteria. I don't want to revisit this. My point is what to do in future in an IETF that publishes many PS documents...
 
> However, as both of these mechansims were quite well understood, with a
> good amount of discussion of unindended consequences (IW10), and a
> limited impact if they would (unexpectedly) fail to meet their goal
> (TLS), I believe the criterial for PS should actually slightly wider...
> 
> A change that is in a reasonably well defined space, with no
> (forseeable) dramatic consequences to the operation of the tcp protocol
> itself, could be allowed for PS even before multiple independent
> implementations are available...

I agree that for relatively straightforward TCP changes we could possibly publish a PS even with a single implementation. It may be a case-by-case decision, and possibly we would have to precisely define for which well-defined space such a PS is applicable. But, the IETF mandates two independent implementations for Internet Standard, not for PS. So, in principle nothing prevents us from at least trying PS status if there is a way to reason why the requirements in RFC 7127 are fulfilled.
 
> Alternatively (which is, what came to pass if I remeber correctly), the
> status may need to be desided only very late in the process - if
> additional implementations were in the works once the technical details
> of a draft settled down.

Indeed, we discussed this variant in a number of cases.

In TCPM, our default was often EXP as target, in particular if there were unknowns at WG adoption time. But IMHO we should ask ourselves if we should more frequently _try_ to produce a PS. Of course, if that is not possible or not possible in a timely fashion, downgrade to EXP later in the process is always possible. But at the end of the day IETF is a standardization organization - so we are assumed to write standards, no?

Whether more PS specs are doable in TCPM is not a trivial question. Yet, I believe we should think about that.

Michael


> Stiking a new balance in tcpm for the intended status seems to be in order.
> 
> Regards,
>    Richard
> 
> 
> Am 19.11.2019 um 13:08 schrieb Scharf, Michael:
> > [chair hat off]
> >
> > In TCPM we repeatedly run into the question whether to publish specs as PS
> or as EXP. We also have several worked examples in which TCPM first
> published a EXP RFC and later a PS.
> >
> > This somehow implies that TCPM often follows a process with three maturity
> levels (EXP, PS, and Internet Standard), while the IETF actually has two maturity
> levels (see RFC 6410). Formal definitions are:
> > * EXP in https://tools.ietf.org/html/rfc2026#section-4.2.1
> > * PS in https://tools.ietf.org/html/rfc7127#section-3.1
> > * Internet Standard in https://tools.ietf.org/html/rfc6410
> >
> > As questions on EXP and PS emerges repeatedly, I think it would be useful to
> discuss two questions:
> >
> > 1/ What is the minimum requirement for publishing an EXP RFC in TCPM?
> >
> > 2/ What is the minimum requirement for publishing an PS RFC in TCPM?
> >
> > I have been thinking myself about these questions quite a bit, most notably
> since I have recently also worked in working groups outside TSV (such as RTG
> and OPS). As far as I can tell, other IETF working groups don't make use of EXP
> as frequently as TCPM. In many working groups PS is the norm.
> >
> > I start to get worried by that. At the end of the day, the IETF is one
> standardization organization and TCPM documents matter also to other areas
> of the IETF, as well as the networking industry as a whole. So, TCPM does not
> operate in isolation.
> >
> > Of course, I am partly accountable for that, given that I have run several
> consensus calls on the target status of RFCs in the last years. I think the
> community feedback was pretty clear in all these cases, typically in favor of
> EXP instead of PS. Yet, after more insight in the rest of the IETF, I wonder if
> maybe TCPM sometimes decides to early to go for EXP if PS is actually
> possible?
> >
> > For instance, as individual contributor, I believe that IW10 and TLP could
> fulfill the PS requirements now. And, in retrospective, I think they were not far
> away from PS at the time of publication. So, in some sense, TCPM has been
> very careful and conservative when publishing these RFCs. This now implies
> that we would have to reissue a PS to actually clean up the RFC services.
> >
> > Given that other working groups apparently operate differently regarding PS
> as target, do we need to rethink our minimum bars?
> >
> > I believe that it would be useful to discuss this question, and I'd be in
> particular interested in the opinions of implementers.
> >
> > As a starting point, I offer my own understanding of how to answer the two
> questions - with chair hat off. So, feel free to disagree... Of course, a case-by-
> case analysis is always needed, so this is not really a checklist, but I guess we
> should have some general understanding of what "maturity" really implies.
> >
> > Michael's thought on question 1/:
> >
> > In order to start a WGLC in TCPM for EXP status, there SHOULD be one
> implementation in the main source code tree of an important TCP/IP stack; a
> default to "off" is perfectly acceptable. Alternatively, there SHOULD be at least
> one implementation outside the main tree and a certain level of operational
> experience e.g. in very controlled environments. Comprehensive testing by
> simulation can also be understood as operational experience if the simulation
> results are verified independently. (As example, the converter specification
> meets this requirement in my point of view.)
> >
> > Note that this requirement is significantly higher than what is written in RFC
> 2026 - in principle, thousands of research papers on TCP could meet the EXP
> requirement of RFC 2026. But, at least to me, there is little value in
> documenting an experiment without proof that it is actually implementable in a
> TCP stack.
> >
> > Michael's thought on question 2/:
> >
> > The requirement follows from RFC 7127, which strongly suggests
> implementation and operational experience, but it is not strictly required. So,
> there is some room for interpretation. The bar for PS is _not_ widespread
> deployment in the Internet. For instance, to me a WGLC for an PS could be
> started if there are two independent implementations, at least one being
> deployed in an important TCP stack (i.e., one does not have to be production
> quality), and some operational experience over a subset of the Internet. For
> instance, such operational experience can also be tests, it does not require
> enabling by default. (As example, RACK easily meets these requirements as far
> as I understand.)
> >
> > Note that with two independent implementations one gets already relatively
> close to the requirements of an Internet Standard, but in that case also
> widespread deployment and some other conditions are required (see RFC
> 6410). So, the bar for PS must be lower than that.
> >
> > Please feel free to comment and/or disagree...
> >
> > Michael
> >
> > _______________________________________________
> > tcpm mailing list
> > [e-mail]
> > https://www.ietf.org/mailman/listinfo/tcpm
> >