Re: [rfc-i] Table of conformance requirements.

[Larry Masinter <[e-mail]>]

At the time, and also now, being clear enough about conformance requirements would be a requirement for the STD series, not the RFC series.

 

Given that you can’t just concatenate XML of separate RFCs to make a STD which has more than one component, there is a need for some other kind of way of specifying Internet Standards.

I imagine  Internet Standards which are a document series which might start out as pointers to the relevant RFCs (and errata) but which could be updated as the standard evolved.

 

 

 

--

 <https://larrymasinter.net/> https://LarryMasinter.net  <https://going-remote.info/> https://going-remote.info

 

From: rfc-interest <[e-mail]> On Behalf Of Christian Huitema
Sent: Thursday, June 18, 2020 9:42 AM
To: Robert Sparks <[e-mail]>; Larry Masinter <[e-mail]>; [e-mail]
Subject: Re: [rfc-i] Table of conformance requirements.

 

I think Robert's example shows the limit of the exercise. Take this requirement from the list:

  MUST-3: There are certain operations that an application **MUST** be able to perform when interacting with QUIC streams.

How are you going to test that snippet of text? In the original document, what follows is the list of these operations that the application is supposed to perform. But if you just take out of context the snippet of text surrounding the MUST statement, you get something meaningless.

 

-- Christian Huitema

 

On 6/18/2020 9:31 AM, Robert Sparks wrote:


On 6/18/20 11:29 AM, Larry Masinter wrote: 



A previous proposal 
https://tools.ietf.org/html/draft-ietf-newtrk-interop-reports-00 
and experiment 
https://tools.ietf.org/html/draft-saintandre-xmpp-interop-report-00 

Yeah, well. Fwiw: https://datatracker.ietf.org/doc/rfc5657/ 





-- 
https://LarryMasinter.net https://going-remote.info 




-----Original Message----- 
From: rfc-interest  <mailto:[e-mail]> <[e-mail]> On Behalf Of 
Robert Sparks 
Sent: Thursday, June 18, 2020 6:38 AM 
To: [e-mail] <mailto:[e-mail]>  
Subject: Re: [rfc-i] Table of conformance requirements. 

I have a script that builds such lists, and have used it for 20-ish years at this 
point in various contexts. 

Some example output: 

https://www.nostrum.com/~rjsparks/draft-ietf-quick-transport-29- 
enumreqs.txt 

https://www.nostrum.com/~rjsparks/rfc3261-enumreqs.txt 

For the most part, its best use has been to help identify when there are 
meaningless, duplicate, or contradictory uses of the BCP14 keywords. 

In interop testing (primarily the SIPit), having these requirements 
enumerated this way didn't help drive test behavior or the creation of test 
suites. It turned out that far more context was needed as a predicate to "I'm 
going to check this thing" than the isolated sentence containing the 
requirement. And agreeing on that context often turned out to be a 
subjective exercise (when that happened, I'd help put pressure on changing 
the spec to reduce the probability of different interpretations). 

So, it's been somewhat helpful in my experience, but not helpful enough to 
try to build extra support for it into, say, the v3 grammar. 

RjS 

On 6/18/20 2:08 AM, Carsten Bormann wrote: 



There is a whole industry around requirements tracking. 
I’m sure the Ribose people can tell us more about that. 




Each MUST in a specification should have at least one corresponding unit 

test to check compliance. 



Yeah sure. 

RFC 4120: 
     *  Principals MUST keep their secret keys secret. 

I’d love to see that unit test :-) 

The other problem is that we simply don’t state all requirements in BCP14 

language. 



Very often, most requirements are stated in describing an architecture or a 

protocol; they are phrased as statements of fact.  Extracting and labeling 
these requirements for requirements tracking is an art form. 



And this needs to be expressible somehow in XML. 

I played around some with using kramdown’s auto-indexing feature for 

BCP14 keywords.  With today’s xml2rfc, that gives you an index of BCP14 
usage.  Not exactly what you want, but just a few lines of kramdown 
markdown. 



I never used the results of these experiments in a document, mainly 

because the lack of BCP14 language on so many of the actual requirements 
made the result too sketchy. 



Grüße, Carsten 

_______________________________________________ 
rfc-interest mailing list 
[e-mail] <mailto:[e-mail]>  
https://www.rfc-editor.org/mailman/listinfo/rfc-interest 

_______________________________________________ 
rfc-interest mailing list 
[e-mail] <mailto:[e-mail]>  
https://www.rfc-editor.org/mailman/listinfo/rfc-interest 

_______________________________________________ 
rfc-interest mailing list 
[e-mail] <mailto:[e-mail]>  
https://www.rfc-editor.org/mailman/listinfo/rfc-interest 

_______________________________________________
rfc-interest mailing list
[e-mail]
https://www.rfc-editor.org/mailman/listinfo/rfc-interest