Re: [rfc-i] Table of conformance requirements.

[Robert Sparks <[e-mail]>]

On 6/18/20 11:29 AM, Larry Masinter wrote:
> A previous proposal
> https://tools.ietf.org/html/draft-ietf-newtrk-interop-reports-00
> and experiment
> https://tools.ietf.org/html/draft-saintandre-xmpp-interop-report-00
Yeah, well. Fwiw: https://datatracker.ietf.org/doc/rfc5657/
>
>
> --
> https://LarryMasinter.net https://going-remote.info
>
>> -----Original Message-----
>> From: rfc-interest <[e-mail]> On Behalf Of
>> Robert Sparks
>> Sent: Thursday, June 18, 2020 6:38 AM
>> To: [e-mail]
>> Subject: Re: [rfc-i] Table of conformance requirements.
>>
>> I have a script that builds such lists, and have used it for 20-ish years at this
>> point in various contexts.
>>
>> Some example output:
>>
>> https://www.nostrum.com/~rjsparks/draft-ietf-quick-transport-29-
>> enumreqs.txt
>>
>> https://www.nostrum.com/~rjsparks/rfc3261-enumreqs.txt
>>
>> For the most part, its best use has been to help identify when there are
>> meaningless, duplicate, or contradictory uses of the BCP14 keywords.
>>
>> In interop testing (primarily the SIPit), having these requirements
>> enumerated this way didn't help drive test behavior or the creation of test
>> suites. It turned out that far more context was needed as a predicate to "I'm
>> going to check this thing" than the isolated sentence containing the
>> requirement. And agreeing on that context often turned out to be a
>> subjective exercise (when that happened, I'd help put pressure on changing
>> the spec to reduce the probability of different interpretations).
>>
>> So, it's been somewhat helpful in my experience, but not helpful enough to
>> try to build extra support for it into, say, the v3 grammar.
>>
>> RjS
>>
>> On 6/18/20 2:08 AM, Carsten Bormann wrote:
>>> There is a whole industry around requirements tracking.
>>> I’m sure the Ribose people can tell us more about that.
>>>
>>>> Each MUST in a specification should have at least one corresponding unit
>> test to check compliance.
>>> Yeah sure.
>>>
>>> RFC 4120:
>>>      *  Principals MUST keep their secret keys secret.
>>>
>>> I’d love to see that unit test :-)
>>>
>>> The other problem is that we simply don’t state all requirements in BCP14
>> language.
>>> Very often, most requirements are stated in describing an architecture or a
>> protocol; they are phrased as statements of fact.  Extracting and labeling
>> these requirements for requirements tracking is an art form.
>>>> And this needs to be expressible somehow in XML.
>>> I played around some with using kramdown’s auto-indexing feature for
>> BCP14 keywords.  With today’s xml2rfc, that gives you an index of BCP14
>> usage.  Not exactly what you want, but just a few lines of kramdown
>> markdown.
>>> I never used the results of these experiments in a document, mainly
>> because the lack of BCP14 language on so many of the actual requirements
>> made the result too sketchy.
>>> Grüße, Carsten
>>>
>>> _______________________________________________
>>> rfc-interest mailing list
>>> [e-mail]
>>> https://www.rfc-editor.org/mailman/listinfo/rfc-interest
>> _______________________________________________
>> rfc-interest mailing list
>> [e-mail]
>> https://www.rfc-editor.org/mailman/listinfo/rfc-interest
_______________________________________________
rfc-interest mailing list
[e-mail]
https://www.rfc-editor.org/mailman/listinfo/rfc-interest