Re: [dmarc-ietf] indeterminisim of ARC-Seal b= value

Yes, +1000 to many test suites.

But no test suite can function without a deterministic order of fields in
the header unless it also includes an ARC implementation for
signing/validating, which is where this conversation started and why we
care so much about discussing this ordering.

On Thu, Mar 30, 2017 at 6:11 PM, HANSEN, TONY L <tony@att.com> wrote:

> One of the reasons DKIM was successful in becoming interoperable was not
> that we had a test suite to force conformance, but instead that we had
> >multiple< test suites that were able to successfully validate DKIM
> signatures from multiple implementations. My test suite caught corner cases
> that Ned’s and Yahoo’s and Google’s did not, and vice versa for each one of
> them. (My company actually had two totally independent implementations of
> DKIM so that we could do internal testing against a different platform. It
> was just prudent to do it that way.)
>
>
>
> When there was an issue in someone’s test suite, we were then able to
> discuss whether it was an issue in the test suite that needed to be fixed,
> or it was an issue in the implementation. A single test suite is never
> sufficient – that’s one reason we prefer multiple interoperating
> implementations in order for something to be considered an internet
> standard.
>
>
>
> By the way, none of us that implemented DKIM test suites had an issue with
> the header order being determined by the sender. It makes the validation
> infinitesimally trickier than having a predetermined header order.
>
>
>
> -1000 to the suggestion of having a single conformance test suite that
> everyone lives and dies by. +1000 to multiple test suites that multiple
> people write and can be used to test against.
>
>
>
> I’m all for you, Seth and Peter, to write a test suite. But nothing you’ve
> said so far has convinced me that a predetermined header order or a single
> conformance test suite is worth pursuing. One of many, sure.
>
>
>
>                 Tony Hansen
>
>
>
> *From: *dmarc <dmarc-bounces@ietf.org> on behalf of Seth Blank <
> seth@valimail.com>
> *Date: *Thursday, March 30, 2017 at 8:10 PM
> *To: *"dcrocker@bbiw.net" <dcrocker@bbiw.net>
> *Cc: *"Murray S. Kucherawy" <superuser@gmail.com>, "dmarc@ietf.org" <
> dmarc@ietf.org>, Scott Kitterman <sklist@kitterman.com>, Peter Goldstein <
> peter@valimail.com>, "ned+dmarc@mrochek.com" <ned+dmarc@mrochek.com>
> *Subject: *Re: [dmarc-ietf] indeterminisim of ARC-Seal b= value
>
>
>
> Dave, If we were only talking about ARC Signing messages, I'd generally
> agree with the comments on this list.
>
>
>
> However, ARC is fundamentally different. It is about a chain of messages
> that validate each other. And ARC's complexity is not in the signing (where
> most of this conversation has been focused), but in this chain validation
> and the cascading issues that can be created by any member of the chain
> doing something slightly wrong. And since *everyone* in the chain except
> the initial signer is also a receiver, the final receiver simply can't make
> up for mistakes caused by a less savvy intermediary - the chain will
> already be broken and unrecoverable.
>
>
>
> ARC also has a lot of subtlety that is missable at interops because of
> this. For instance, if I'm modifying a message, I need to validate the
> incoming chain, then make my modifications, and then sign the outgoing
> message. An intermediary doing this wrong (say, validating and then signing
> after message modification) isn't a bug a final receiver can fix. And
> without a seriously complex interop with a ton of intermediaries and
> sending relationships mapped out for testing, it's not likely an error in
> any specific implementation here will be discovered until it's too late.
>
>
>
> To my understanding, working on the test suite shook these issues out
> because they were what caused problems running the test suite at the
> interop in the first place. So this is actually a good and healthy part of
> the process, and is the natural consequence of the interops. What worked
> here for DKIM is insufficient with the nuances of ARC.
>
>
>
> What we've been discussing doesn't just guarantee a test suite can be made
> that works for everyone (which is huge in itself), but constrains the
> problem space for everyone involved in an ARC chain. This is clean and
> desirable.
>
>
>
> Since the spec is not finalized, this is the perfect time to set a
> deterministic order for headers that no one has even seen before.
>
>
>
> Seth
>
>
>
>
>
> On Thu, Mar 30, 2017 at 4:15 PM, Dave Crocker <dhc@dcrocker.net> wrote:
>
> On 3/30/2017 12:13 PM, ned+dmarc@mrochek.com wrote:
>
> And given the relationship with DKIM people are going to expect ARC
> to work in the same general way, maing such a change a least astonishment
> principle violation.
>
>
>
> +10.
>
> This thread has been active for 6 days.  That is 5.5 days longer than it
> should have lasted, and I'm being generous.
>
> Internet technologies succeed through interoperability testing, not
> conformance tests, or the like.  Conformance testing, and the like, are
> useful for early-stage bug-finding, but not for 'certification'.  This
> point has been a distinguishing feature of Internet technical work since
> before the Internet.
>
> ARC is a small re-casting of DKIM.  DKIM has worked for quite a few
> years.  Some details of ARC need to be different from DKIM, but the basic
> paradigm -- including the model of achieving field operation through
> interoperability testing -- should be the same.
>
> My reading of this too-long message thread is that there is essentially no
> support for making ARC's header-related issues any different from DKIM's,
> so I encourage the chair to shut this thread down.
>
> d/
>
> --
> Dave Crocker
> Brandenburg InternetWorking
> bbiw.net
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__bbiw.net&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=Kz8VdgPVctDNSNPJ6PsBaw&m=M5zzV_Ak3GCR8lZhAHLFfMA0SvGqSztWEPI0H6a5Y2E&s=ZX1C85lFDoWiX0J86FUeLGQx37E3YQcIT3LIGyCaeWU&e=>
>
>
>
> _______________________________________________
> dmarc mailing list
> dmarc@ietf.org
> https://www.ietf.org/mailman/listinfo/dmarc
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_dmarc&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=Kz8VdgPVctDNSNPJ6PsBaw&m=M5zzV_Ak3GCR8lZhAHLFfMA0SvGqSztWEPI0H6a5Y2E&s=mJ57Dtkqee8RcYkQnZ9yx6EdyO8ycHhuJTmem3tZxy8&e=>
>
>
>
>
>
> --
>
> [image: mage removed by sender. logo for sig file.png]
>
> *Bringing Trust to Email*
>
> Seth Blank | Head of Product for Open Source and Protocols
>
> seth@valimail.com
> +1-415-894-2724 <415-894-2724>
>
> _______________________________________________
> dmarc mailing list
> dmarc@ietf.org
> https://www.ietf.org/mailman/listinfo/dmarc
>
>

-- 

[image: logo for sig file.png]

Bringing Trust to Email

Seth Blank | Head of Product for Open Source and Protocols
seth@valimail.com
+1-415-894-2724 <415-894-2724>