Re: [spfbis] Review of draft-ietf-spfbis-experiment-05

[Scott Kitterman <[e-mail]>]

On Monday, April 23, 2012 11:13:35 AM Dotzero wrote:
> On Mon, Apr 23, 2012 at 10:34 AM, Murray S. Kucherawy <[e-mail]> 
wrote:
> >> -----Original Message-----
> >> From: Dotzero [mailto:[e-mail]]
> >> Sent: Monday, April 23, 2012 5:58 AM
> >> To: Murray S. Kucherawy
> >> Cc: [e-mail]
> >> Subject: Re: [spfbis] Review of draft-ietf-spfbis-experiment-05
> >> 
> >> It is absolutely incorrect to say that their accuracies are about the
> >> same. I can consistently game PRA to get a neutral outcome regardless
> >> of what the originating domain publishes in it's record. If there is a
> >> Sender field then that is the PRA per the spec. I cannot do the the
> >> same for SPF.
> > 
> > Based on the data collected about actual live mail streams, SPF and Sender
> > ID reach the same pass/fail conclusion about messages at least 80%, and
> > as much as 95%, of the time.  Do you have different data?
> > 
> > We're specifically not doing a comparison of the weaknesses of the two.
> >  We're only citing empirical data.
> You stated that their accuracies are comparable. Given the known
> weakness of PRA (based on emperical data/testing), that is an
> incorrect statement. The whole point of SPF (and presumably SIDF) is
> to mitigate abuse. You have not provided complete details about the
> dataset you are referring to so it is hard to draw detailed conlusions
> regarding key points:
> 
> 1) What percentage of the dataset does not have a Sender field?
> 2) For data points where there is a Sender field, what percentage of
> those data points have a Sender field that is aligned with the Mail
> From (and conceivably From)?
> 3) What percentage of the messages were "abusive"? Would the mail
> stream selected be a likely target for the particular type of abuse I
> am pointing out? If not, why would you expect to see this particular
> type of abuse?
> 4) To what extent is the dataset representative of the mail streams
> that various types and sizes of receivers might receive? That is, is
> the dataset truly representative or are there potential issues with
> self selection, etc?

Perhaps it would be enough to say that there isn't any evidence of accuracy 
improvement associated with the additional complexity of Sender ID?

Scott K