Re: [dmarc-ietf] Which DKIM(s) should be reported? (Ticket #38)

[Douglas Foster <[e-mail]>]

Is this already a settled issue?  The specification already calls for a
complete A-R data set, so all signatures are supposed to be included if
they are evaluated.  Are the largest reporting sources already providing a
complete list of DKIM signatures?

However, there are significant technical problems with aggregating a list
with a variable number of members, because the list must be converted into
a list with a fixed number of elements before aggregation can be
performed.

- One technique is to convert the list into a variable-length text string,
so that the entire list is handled as one element.   Including all
signatures in an A-R record, and then grouping on the A-R text, would be an
example of this approach.  The technique will work up to the maximum
allowed text string supported by the data management system.   The maximum
number of list elements will depend on the mechanism used to build the text
string, the information being reported, and the maximum text size.   The
maximum number of supported list elements becomes unpredictable, but in
many data management systems will be larger than the expected number of
signatures in a message, unless a message is specifically constructed to
trigger a denial-of-service attack.

- Another approach, based on E.F.Codd's data normalization rules for
relational databases, is to have a table of messages which is keyed on a
message ID, and a table of signatures, which is keyed on message ID and
sequence number.   Then an outer join can be used to append the list
element with sequence number # to the message record.   A separate outer
join is required for each sequence number being appended, so the
implementation must choose a maximum number of list elements to append.
 One recent poster said that he was using this approach.    Outer joins are
generally inefficient, and this approach might work for up to 4 list
elements, but it will not work acceptable for a list with 100 elements.

For report sources with a fixed limit, it seems appropriate to have a
metadata element where the report provider states the maximum number of
signatures that might be reported by his system.   An indicator would be
needed to indicate "many, with no pre-determined limit"

Doug

On Tue, Jan 26, 2021 at 7:50 AM Alessandro Vesely <[e-mail]> wrote:

> On Tue 26/Jan/2021 13:02:46 +0100 Douglas Foster wrote:
> > DKIM Scopes
> > I have not heard a compelling argument to require information about
> > authentication tests that are unrelated to alignment testing.    For
> DKIM
> > specifically, I think one scope should be sufficient, on this hierarchy:
> >
> > - The best-aligned scope that verified, or
> > - the best-aligned scope that failed verification, or
> > - a no-signature result otherwise.
> >
> > Anything more complex imposes a gratuitous data collection burden on the
> > reporting domain and reduces aggregation significantly.   On the
> technical
> > side, it has already been noted that variable-length lists are
> particularly
> > problematic for calculating aggregates.
>
>
> Let me attach an HTML rendering of a report I received today, so we can
> talk
> about something real.
>
> Lines with IP 4.31.198.44 bear a ietf.org identifier.  I see no reason to
> remove it.  It is useful for understanding the mailflow, which is what
> DMARC
> reporting is designed to do.
>
>
> > Aggregation Controls
> >
> > We have discussed whether the target domain should be included in the
> > report.  I understand that doing so is not reasonable for the large
> hosting
> > services.   On the other hand, including the target domain would be a
> > trivial matter for smaller operations, and I think it would be valuable
> for
> > some research.    Similarly, DKIM scopes are known to be useful for most
> > investigations, but John has already observed that proliferation of DKIM
> > scopes can be used to force disaggregation down to the individual
> recipient
> > level.
>
>
> Even if this is a small example, learning the disaggregated, or even
> individual
> recipients does not help my understanding.  Authentication is obviously
> conditioned by how the Mediator treats my messages.
>
> I expect that Fastmail Pty Ltd carries out SPF and DKIM validation using
> the
> same algorithm, irrespective of the recipient.  That is what I, as a
> sender, am
> interested in.  Splitting the report in 66 lines wouldn't tell me anything
> more, it would just consume more eyeballs.  And is useless for people who
> sum
> up all reports and just look at the totals.  In any case, I cannot verify
> if
> the messages I didn't send directly are real.
>
> If a multi-domain host allows personalized validation algorithms for some
> domains, I'd expect they send separated aggregate reports, if any.
>
>
> Best
> Ale
> --
>
>