Re: [dmarc-ietf] Aggregate Reporting - "Not Evaluated" result

[Douglas Foster <[e-mail]>]

Scope considerations
----------------------------
For those who can speak on behalf of domain owners, it would be helpful to
clarify scope:

- If a message has multiple signatures for the same domain, do you want all
or one included in the DMARC aggregate report?   I can imagine that having
both PASS and FAIL for the same domain could cause confusion, but perhaps
that confusion has already been prevented in all running code.

- If a message has a valid ARC chain, and the evaluator performs ARC
checking on the message, what signatures do you want included in the
aggregate report?   Note that ARC sets will increase the likelihood of
duplicate results.  If ARC signatures are to be included, we need to
provide details on how this is done.

- As requested previously, what is the preferred behavior if the evaluator
does not evaluate all signatures but is aware that additional signatures
were present in a particular message?  My specific suggestion is included
at the bottom of this post.


Message Limits
---------------------
https://www.rfc-editor.org/rfc/rfc6376.html#section-4.2 says:
"To limit potential denial-of-service attacks, Verifiers MAY limit the
total number of signatures they will attempt to verify."

I would choose stronger language:  "Verifiers SHOULD limit the total number
of signatures that they will attempt to verify, based on the design and
testing limits of the software being used."

For interoperability, aggregate reports need to stay within the safe design
limits of both the generating and the receiving applications, even the the
design limits of downstream systems are unknown.  Our very limited data
suggests that legitimate messages will have 10 or fewer DKIM signatures,
and possibly 10 or fewer ARC signatures.  Our specification should limit
the maximum number of included signatures to ensure that neither the report
generator nor the report recipient can be sabotaged by an oddball message
with an unexpectedly large number of signatures.   This also reduces the
overhead imposed on the report generating system.


Signatures not Evaluated
----------------------------------
Based on the above, a message may have signatures which lack reported
results for any of these reasons:
- The verifier evaluated signatures until its hard limit on the number of
signatures was reached, then stopped.
- The verifier evaluated more signatures than the reporting specification
allows, so it could not report all of them.
- The verifier evaluated only those signatures needed to obtain a PASS
result, then stopped evaluating.

I suggest that all three of these situations can be handled with an
additional counter.
The data on each row would include:
(a) the number of messages that are covered by the aggregation key, and
(b) the number of messages within that total which had additional
signatures which were not included in the aggregation key.

The aggregation key should only include evaluated signatures and their
results, so the "Not Evaluated" result is discarded because the domains of
any such signatures are not itemized.

 Doug Foster






On Tue, Oct 4, 2022 at 5:23 AM Laura Atkins <[e-mail]> wrote:

>
>
> On 3 Oct 2022, at 23:31, Douglas Foster <
> [e-mail]> wrote:
>
> The primary key for aggregation is the SMTP domain (up to 255 characters),
> plus each DKIM domain (up to 255 characters) and its DKIM scope (up to 64
> characters).    For 100 DKIM domains to be included, the primary key
> becomes up to 201 fields with up to 25,919 characters.  This is
> unmanageable with readily available tools like a SQL database.
>
> I suggest that we start simplifying the scope, by limiting the items of
> interest to:
> (a) SPF whether pass or fail,
> (b) aligned DKIM domains, whether pass or fail, and
> (c) non-aligned DKIM domains that pass.
>
> I don't see any value in knowing about failed DKIM signatures for
> non-aligned domains.   Perhaps someone with a sophisticated report parsing
> capability will be kind enough to correct me and explain.
>
>
> Knowing what domain is signing is important if you think things should be
> signed but they’re not aligning. Knowing what they’re being signed with
> helps treat the problem.
>
> A very simple case is: if you have 2 domains you’re using at Google
> Workspace sometimes Google will pick the wrong one to sign. So your mail
> fails DKIM alignment. Without knowing the domain, it’s impossible to
> troubleshoot what’s going on here.
>
> Then I suggest that the DKIM upper limit by 10, not 100.   At some point,
> an excess of domains becomes a DDoS attack on the evaluator, and to a much
> lesser extent, the report recipient.
>
>
> This is the type of suggestion that has broken SPF. I do not support it.
>
> laura
>
>
> Doug Foster
>
>
> On Mon, Oct 3, 2022 at 2:17 PM Alessandro Vesely <[e-mail]> wrote:
>
>> On Mon 03/Oct/2022 18:01:06 +0200 Murray S. Kucherawy wrote:
>> > On Mon, Oct 3, 2022 at 10:26 AM Brotman, Alex <[e-mail]>
>> wrote:
>> >
>> >> So we would likely need a section in the core document with a SHOULD
>> for
>> >> evaluation (if it’s not already there), and then a section in the
>> aggregate
>> >> reporting for a MUST for reporting on evaluated information (if they
>> choose
>> >> to send reports at all), correct?
>> >
>> > [...]
>> >
>> > From the actual protocol standpoint, the filtering part of DMARC
>> operates
>> > just fine if you make the shortcut Doug is proposing, so the first
>> SHOULD
>> > is probably apt but the MUST is moot because it doesn't change
>> > interoperability.
>>
>>
>> Let me add that reporting /all/ identifiers can be unfeasible.  (I, for
>> example, collect identifiers to be reported using SQL left join of
>> various
>> copies of the table, which delivers results as just that many columns,
>> although
>> more might have been evaluated at reception time.)
>>
>> Reporting just the "most important" identifiers could be an option.
>>
>>
>> Best
>> Ale
>> --
>>
>>
>>
>>
>>
>> _______________________________________________
>> dmarc mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/dmarc
>>
> _______________________________________________
> dmarc mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/dmarc
>
>
> --
> The Delivery Experts
>
> Laura Atkins
> Word to the Wise
> [e-mail]
>
> Email Delivery Blog: http://wordtothewise.com/blog
>
>
>
>
>
>
> _______________________________________________
> dmarc mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/dmarc
>