Re: [DNSOP] draft-ietf-dnsop-kskroll-sentinel-12

[Job Snijders <[e-mail]>]

On Tue, May 08, 2018 at 11:05:50AM +1000, Mark Andrews wrote:
> >> We have also taken the implementation comments posted to the WG
> >> mailing list and collected them in a new section titled
> >> "Implementation Experience” in the light of Suzanne’s request
> >> 
> >> So we would like to pass this draft back to the WG Chairs at this
> >> point for your review for potential submission as an RFC.
> > 
> > 1/ While this is a step in the right direction, I'm not yet entirely
> > impressed by the 'Implementation Experience' section. Is it somehow
> > hard to write an implementation report that describes which
> > implementations comply with the BCP 14 / RFC 8174 terms used in
> > draft-ietf-dnsop-kskroll-sentinel? Are the authors experiencing some
> > blocker in this regard?
> 
> Is it a implementation report or a compliance report?

Well, the current section on the implementations isn't much of either,
it is very sparse. This working group should try to raise the bar for
RFC publication, not explore what the bare minimum is. 

I've used this example before: what I was hoping for is reports that
allude to the BCP14 terms and implementation's compliance similar to
what is done here: https://trac.ietf.org/trac/idr/wiki/draft-ietf-idr-large-community%20implementations
Overviews like these are easy to consume for humans, and are derived
from extensive tests like the one you showed below.

I'd like to note that describing people's intention to implement in a
section called 'Implementation experience' of course doesn't add much
value. :-)

> To actually do a full compliance report for this draft it would
> require including a complete DNSSEC compliance report and in turn a
> complete STD 13 compliance report to meet this “MUST”.
>
>    DNSSEC-Validating resolvers that implement this mechanism MUST
>    perform validation of responses in accordance with the DNSSEC
>    response validation specification [RFC4035].
> 
> A exhaustive compliance report would run to a couple of thousand tests
> without doing a DNSSEC compliance report due to the combinatorial
> nature of this draft.
> 
> You would need validate as secure zone, a validate as bogus zone, a
> validate as insecure w/ signatures, a validate as insecure w/o
> signatures.  You then for all of these need zones with and without A
> and AAAA records at the test names where without includes both NODATA
> and NXDOMAIN.  You then need to multiply that by servers configured to
> validate and those configured not to validate.  Then you need to
> multiply this by having the functionality enabled or disabled.  You
> then need to do a test for each of the conditions in the list of
> conditions that must be met for the modified responses to be returned.
> You also need to do that where the label looked up after a CNAME and a
> DNAME.
> 
> We skipped some of these tests when we built our system tests by I
> believe we hit enough of them to be happy that the code is operating
> correctly.

Yes, I think you are touching upon the core of the issue. Discussing
_how_ to test a specification is exactly the type of discussion to be
held in this working group.

Its commonly accepted that even seemingly simple specifications may be
stringing together a number of complex features. Extensive testing and
demonstrations of such testing are a necessity to be able to claim
'running code and rough consensus'.

> S:rootkeysentinel:Mon May  7 17:55:35 UTC 2018
> T:rootkeysentinel:1:A
> A:rootkeysentinel:System test rootkeysentinel
> [snip]

Thanks for sharing this!

So, the current status is that there is no longer zero, but now a single
implementation of draft-ietf-dnsop-kskroll-sentinel-12?
I see progress, but not last call material.

Kind regards,

Job