Re: [dmarc-ietf] indeterminisim of ARC-Seal b= value

[Peter Goldstein <[e-mail]>]

Scott,

I will confess to not following all the details on a regular basis, but
> doesn't the paragraph you referenced give a required sequence of multiple
> header fields?  Maybe that's not a problem any more (it's been some years
> since I looked), but that definitely used to be something it was risky to
> rely
> on.


No.  Precisely the opposite.  The referenced paragraph in the ARC spec is a
set of instructions of how to create an ARC Seal for a message.  The point
of referencing the instructions is to show that the algorithm is already
resilient to header re-ordering, and nothing being proposed here would
change that.

The test case comment was based on my understanding that you wanted
> deterministic ordering because if the ordering wasn't deterministic it
> would
> be too hard to test.  I think that can be managed.


I think there's a misunderstanding here.  You seem to be under the
impression that supporting the current non-deterministic behavior can be
managed with 'more tests' that can be auto-generated and run against all
implementations.  That's not the issue.

The non-deterministic ordering requires either:

1. Implementations that use different tag/value ordering/formatting cannot
share a test suite.  Signing tests that are valid for one implementation
will not be valid for an implementation using a different tag/value
ordering or formatting.

or

2. The test suite needs to include a 'reference' ARC verifier

#1 explicitly fails your requirement that "test cases are validated against
multiple implementations", unless the implementations use the same
tag/value canonicalization.  This thread is simply proposing that we pick
one such canonicalization, and define it in the RFC, so that all ARC
implementations share the same canonicalization.

I've discussed why I think #2 is a bad idea - it's a lot of work (and it's
unclear who will do it), much more fragile and complex, and yields no
benefit that has been actually identified.

Best,

Peter




On Tue, Mar 28, 2017 at 5:16 AM, Scott Kitterman <[e-mail]>
wrote:

> I will confess to not following all the details on a regular basis, but
> doesn't the paragraph you referenced give a required sequence of multiple
> header fields?  Maybe that's not a problem any more (it's been some years
> since I looked), but that definitely used to be something it was risky to
> rely
> on.
>
> The test case comment was based on my understanding that you wanted
> deterministic ordering because if the ordering wasn't deterministic it
> would
> be too hard to test.  I think that can be managed.
>
> Scott K
>
> On Monday, March 27, 2017 10:30:37 PM Peter Goldstein wrote:
> > Scott,
> >
> > Header reordering is irrelevant to this discussion - the ARC RFC
> specifies
> > in some detail how to construct the ARC Seal from the AMS and AAR
> headers -
> > https://tools.ietf.org/html/draft-ietf-dmarc-arc-protocol-02#section-5.2
> .
> > And the AMS headers have the same behavior as DKIM - header ordering and
> > canonicalization is also well-defined, so header re-ordering doesn't
> impact
> > the signature.  The discussion at hand is literally about the ordering
> and
> > formatting of tag/value pairs within the individual ARC headers.
> >
> > So the only brittleness question is whether existing mail infrastructure
> > will potentially alter the internal structure of the AMS header or ARC
> Seal
> > header.  That is...unlikely.  We can conclude it's unlikely, because a
> > similar alteration of a DKIM signature header would break the DKIM
> > signature of forwarded email.  So if DKIM generally works (and it does),
> we
> > should be able to conclude that specifying ordering/formatting of the
> > tag/value pairs in an AMS/ARC header does not represent an increase in
> > brittleness.
> >
> > I'm not really sure I understand the point regarding the # of test cases.
> > We're not worried about keeping the # of test cases small - we're
> > interested in ensuring that the same test cases will run successfully
> > against any valid ARC implementation.  Can you please clarify?
> >
> > And yes, something will probably be developed in Ruby at some point - but
> > it's just as likely to be a de novo effort, as it is a modification of an
> > existing DKIM gem.  There is only 1 pure Ruby DKIM signing gem I'm aware
> of
> > - https://github.com/jhawthorn/dkim - and it hasn't had any updates in
> 2+
> > years and lacks verification logic.  So it's a less than ideal basis for
> > developing an ARC gem.
> >
> > Best,
> >
> > Peter
> >
> > On Mon, Mar 27, 2017 at 9:49 PM, Scott Kitterman <[e-mail]>
> >
> > wrote:
> > > What's more difficult to is identify all the ways that things get
> > > reordered,
> > > mangled, etc as they transit the various elements of the mail
> > > architecture.
> > > If you over specify the allowed order, aren't you risking increasing
> the
> > > brittleness of the solution.
> > >
> > > If test cases are automatically generated, then they are cheap and we
> > > shouldn't worry about unduly constraining things to keep the number
> > > small.  As
> > > long as, at some point, the test cases are validated against multiple
> > > implementations, I think it's fine.  If you've got a handful of
> > > implementations, then the risk of them all having the same bug is
> pretty
> > > low.
> > >
> > > In addition to Perl, I'd expect something in Ruby to appear in due
> course.
> > >
> > > Scott K
> > >
> > > On Monday, March 27, 2017 07:54:43 PM Peter Goldstein wrote:
> > > > John,
> > > >
> > > > I'm familiar with the definition of the FWS in the ABNF, as well as
> the
> > > > generic definition ABNF for message headers.  I'm also aware of the
> > > > challenges with trying to normalize such headers, and how that can
> > > > impact
> > > > email authentication - breaking forwarded DKIM signatures and such.
> But
> > > > none of that is actually relevant here - we are not interested in
> > >
> > > arbitrary
> > >
> > > > messages.
> > > >
> > > > We are interested in creating a set of valid input messages and
> ensuring
> > > > that these messages are signed in a consistent, reliable way by any
> > > > valid
> > > > ARC implementation.  This is an extremely narrow case - so yes, I do
> > >
> > > think
> > >
> > > > the above is basically all it would take to make such signature
> headers
> > > > identical.  Perhaps I'll be disappointed, but based on the sample
> > >
> > > messages
> > >
> > > > I have on hand from existing implementations, that seems unlikely.
> > > >
> > > > As for the hypothetical developers who will be adapting DKIM
> libraries
> > > > to
> > > > do ARC signing, we've been talking about them since M3AAWG in October
> > > > 2015.  So far they haven't materialized.  Honestly, it's not even
> clear
> > >
> > > to
> > >
> > > > me that there are that many DKIM libraries out there to adapt.
> > > >
> > > > Instead we've seen ~5 implementations (Google, AOL, Dkimpy, OpenARC,
> > > > MailerQ), with potential support for implementations in 1-2
> additional
> > > > languages (e.g. Perl) probably driven by one or more of the
> implementers
> > >
> > > of
> > >
> > > > the existing implementations.  It should be relatively easy to
> > > > coordinate
> > > > such a change across the small number of existing implementations.
> > > >
> > > > Best,
> > > >
> > > > Peter
> > > >
> > > > On Mon, Mar 27, 2017 at 7:21 PM, John R Levine <[e-mail]>
> wrote:
> > > > > I think tightening up some currently allowed ambiguity in the ARC
> > > > >
> > > > >> specification is a much simpler and much better solution.  I'm not
> > >
> > > sure
> > >
> > > > >> why
> > > > >> there's such concern about canonicalizing the format and ordering
> of
> > >
> > > some
> > >
> > > > >> tag/value pairs.
> > > > >
> > > > > If you think that's all it would take to make signature headers
> > >
> > > perfectly
> > >
> > > > > identical, you will be deeeply disappointed.  (Take a look at FWS
> in
> > >
> > > the
> > >
> > > > > ABNF and all of the other generic ABNF for message headers.)
> > > > >
> > > > > I think what we've been saying is that the SMTP mail ecosystem has
> > >
> > > never
> > >
> > > > > tried to make stuff bit-for-bit reproducible, and even if you could
> > >
> > > hammer
> > >
> > > > > on the spec to make super strict rules for one particular header,
> it's
> > > > > unlikely that the people who are adapting their DKIM code would pay
> > > > > attention.
> > > > >
> > > > > R's,
> > > > > John
> > >
> > > _______________________________________________
> > > dmarc mailing list
> > > [e-mail]
> > > https://www.ietf.org/mailman/listinfo/dmarc
>
> _______________________________________________
> dmarc mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/dmarc
>



-- 


[image: logo for sig file.png]

Bringing Trust to Email

Peter Goldstein | CTO & Co-Founder

[e-mail]
+1.415.793.5783