Re: [dmarc-ietf] I-D Action: draft-ietf-dmarc-dmarcbis-03.txt

[Alessandro Vesely <[e-mail]>]

On Thu 19/Aug/2021 14:52:41 +0200 Todd Herr wrote:
> On Thu, Aug 19, 2021 at 7:19 AM Alessandro Vesely <[e-mail]> wrote:
>> On Wed 18/Aug/2021 22:17:57 +0200 Todd Herr wrote:
>>>
>>> The main update in this draft is removal of the "pct" tag, with an
>>> explanation as to why, and an introduction of the "t" tag in an effort
>>> to maintain the functionality provided today by "pct=0" and "pct=100".
>>
>> As held earlier, I disagree with such gratuitous breaking of the
>> existing installed base and published records.
> 
> I disagree with your characterization of removal of the "pct" tag as
> "gratuitous breaking"; the spec has long contained the following text:
> 
> Only tags defined in this document or in later extensions, and thus
> added to that registry, are to be processed; unknown tags MUST be
> ignored. >
> and so should a DMARC protocol without the "pct" tag be formally adopted,
> there should be no breaking of any existing DMARC implementations.


Excuse me, but I don't get it.  An existing DMARC implementation which 
treats pct=, should a DMARC protocol without it be formally adopted, 
MUST be rewritten, re-tested, re-installed.  IOW, the current 
installation is broken.

As for existing records, they have to work with the updated 
implementations as well as with the ones now formally broken.


>> It goes without saying that domains who are publishing pct=0 will
>> slowly adapt by adding t=y and never removing pct.  Those who publish
>> pct=50 and are satisfied with it will have to give up, despite their
>> own operational experience.
>>
>> In any case, I object to the use of the Probability Mass Function as
>> applied to Binomial Distributions argument.  It presumes that the
>> percentage in question refers to the number of messages sent during a
>> given day, which was never specified.  The spec said "Percentage of
>> messages from the Domain Owner's *mail stream*".  The random function
>> applied to such stream is equivalent to computing a Monte Carlo
>> integration on a finite set.  Since *all samples* are eventually
>> considered, the result tends to the exact value.
> 
> I will continue to contend that the first sentence of the existing
> definition of the "pct" tag was incorrectly worded. It reads:
> 
>     Percentage of messages from the Domain Owner's mail stream to
>     which the DMARC policy is to be applied.
> 
> and as I've written in other threads, such phrasing makes no sense to me,
> because a DMARC policy cannot be applied to a message which passes DMARC
> verification checks.
> 
> I believe my argument to be supported by the existing definitions of
> "quarantine" and "reject", which read in part:
> 
>     quarantine:  The Domain Owner wishes to have email that fails the
>     DMARC mechanism check be treated by Mail Receivers as suspicious.
> 
>     reject:  The Domain Owner wishes for Mail Receivers to reject
>     email that fails the DMARC mechanism check.
> 
> There is nothing in the text there that talks of applying a policy of
> "quarantine" or "reject" to messages that pass the DMARC mechanism check,
> so it follows for me that the "pct" tag was never intended to apply to the
> entire mail stream, only to those messages that failed the check.


Agreed.  The language of RFC 7489 seems to talk about applying the 
DMARC check and then applying the policy if the check fails.  This in 
no way limits the mail stream to a single day.


> Given my
> assertion here, I believe that the Probability Mass Function as applied to
> Binomial Distributions is the correct argument to make against the pct tag,
> and I was rather pleased with myself in how I described it in an earlier
> revision of the spec:
> 
>           if (random mod 100) < pct then
> 
>              selected = true
> 
>           else
> 
>              selected = false
> 
> 
>     The pseudocode shown above is an example of that approximation,
>     relying on a random number generator to effectively produce a whole
>     number between 0 and 99, inclusive.  If that number is less than the
>     value of the "pct" tag, then a message producing a DMARC "fail"
>     result will be subject to the DMARC policy in question; if not, it
>     will be subject to the lesser policy.


Correct.


>     Over time and given enough
>     iterations of the pseudocode, this should produce a roughly uniform
>     distribution of all values across the range, which we will refer to
>     going forward as "the pool".


This definition of "the pool" is not in the spec.  It is enough to 
have a decent random function.  Anyway...


>     However, mathematics teaches us that
>     the pool cannot be guaranteed to produce the desired result.
>     The sampling done to honor the "pct" tag is known in mathematics as a
>     Binomial Distribution, where a number of independent samples of the
>     pool are taken, with each one having the same probability of
>     producing a number that is less than the value of the "pct" tag.


Here you deviate from the spec.  A binomial distribution is 
characterized by a fixed number of samples n and a probability pct. 
The spec only talks about the probability pct.  Your argument seems to 
assume that a mail stream is made of 5 messages or less, which is 
completely nonsensical w.r.t. to common understanding of DMARC usage.

Indeed, most servers end up relaying more than five messages to any 
receiver within a given mail stream.

Now, I confess that I don't have a formal definition of mail stream at 
hand.  I tend to consider a stream, for example, the series of 
messages signed with a given DKIM selector, or emitted by a given 
relay.  Roughly, a mail stream is the entity that a reputation tracker 
attaches a value to, isn't it?


>     A Binomial Distribution is expressed by the following function, known
>     as a probability mass function (PMF):
> 
>                   n!         x          n-x
>       f(x) = ----------- *  p  * (1 - p)
>              (n-x)! * x!
> 
>     In English, the PMF is a way to calculate the probability that x
>     items from a sample of n items will have the desired result when p is
>     the probability that any one item will have the desired result.


The spec does not say that pct=20 requires a receiver to not applying 
the policy in one message out of the first five ones.  My 
understanding is that it says that /on average/ a receiver will not 
apply the policy in the 20% of the failed messages of the whole mail 
stream.


>     For example, for a DMARC policy record with pct=20, we let p = 0.2,
>     and to calculate the probability that 1 out of every 5 messages will
>     be assigned the requested policy, we have:
> 
>             5!           1            5-1
>        ----------- *  0.2  * (1 - 0.2)     =
>        (5-1)! * 1!
> 
> 
>           120       1     4
>          ----- * 0.2 * 0.8  = 5 * 0.2 * 0.4096 = 0.4096
>           24
> 
>          0.4096 * 100 = 40.96%
> 
>     The above demonstrates that for every five messages producing a DMARC
>     "fail" result, there is a slightly less than 41% chance that just one
>     of the five will have the requested policy applied to it.  The table
>     below shows the percent probability for all possible results:
> 
>          -----------------------------------
>          | X  | Percent chance that X of 5 |
>          |    |  will have policy applied  | >          -----------------------------------
>          | 0  |        32.768%             |
>          -----------------------------------
>          | 1  |        40.96%              |
>          -----------------------------------
>          | 2  |        20.48%              |
>          -----------------------------------
>          | 3  |         5.12%              |
>          -----------------------------------
>          | 4  |         0.64%              |
>          -----------------------------------
>          | 5  |         < 0.1%             |
>          -----------------------------------


Whatever the definition of stream, the binomial distribution is not 
the right tool to compute what's happening with realistic numbers. 
Now it's about dinner time and I'm not willing to program a binary 
distribution for a decently high number of trials.  What would you 
reckon it is the probability that, out of 1000 failed messages, the 
percent chance probability of applying the policy to X message is such 
that 150 < X < 250?  That would be an inefficient way to compute the 
precision of Monte Carlo approximation.


> Your mileage may vary, of course, but the larger point here is that while
> you personally have objected to removal of the "pct" tag, there has
> seemingly been rough consensus supporting the idea of removing all values
> except 0 and 100, and quite a lot of agreement that having a tag named
> "pct" that only had valid values of 0 and 100 didn't make sense, so this
> rev is a first attempt to find a path to documenting something that does
> make sense, given that there is support for keeping the functionality that
> pct=0 provides.


I agree pct with only valid values of 0 and 100 makes no sense.


Best
Ale
--