Re: [Endymail] spam versus cleartext

On Sun, Sep 7, 2014 at 8:27 AM, Dave Crocker <dcrocker@gmail.com> wrote:
> On 9/7/2014 8:10 AM, Kathleen Moriarty wrote:
>> How does handing not only spam, but other attack like phishing and spear phishing evolve when e2e messaging is the norm?
>
>
> Spam and other abuse continue to occupy 90-98% of the email traffic
> across the net.  Life is tolerable only because the receiving operators
> have gotten quite good at keeping these barbarians outside of the gate.
>  Note that a change of only a few percent in filtering efficacy will
> likely double the amount of spam/abuse the receivers sees.  And double
> is a best case scenario.
>
> Modern filtering engines use an amazing array of information to assess
> incoming mail.  IP Address, message meta data, content, traffic
> analysis, etc.  Some of the filtering does not require looking at any
> content (envelope, header, body).  Some does.
>
> To the extent that particular content is hidden from the filtering
> engine, that portion of the engine is useless.  (This observation is in
> the realm of "duh", but it's needed for the sequence here.)
>
> If that efficacy is to be retained/recovered, we need to find a way to
> give the filtering engine access to that data, but without compromising
> the confidentiality model.
>
> As this has been discussed in other conversations, the only way I see
> that happening is to move the relevant portions of the engine into the
> recipient's MUA, and then have that sub-engine consult with the main
> engine.  ("Consult" is a code word for needing an open protocol between
> the MUA and the filtering engine.)

To connect to server side filtering, the filtering engine on the
server just needs to put probabilities it thinks that the message is
spam in the headers, as well as have a standardized means for the
client to report spam or ham. This doesn't seem that complicated: just
a double and some sort of forwarding info to get the backchannel.
(This assumes naive Bayes as a filter design)

>
> This will let more bad mail get to the inbox, but would still limit how
> much actually burdens the recipient.

True: how much does DKIM+sender based blacklists do vs. filtering
based on content? For mobile someone raised the issue of excessive
notifications and battery life, so we do need to worry a little about
server-side. But I think it's clear we can engineer a solution to spam
that doesn't look much different than today.

Sincerely,
Watson Ladd
>
> d/
>
> --
> Dave Crocker
> Brandenburg InternetWorking
> bbiw.net
>
> _______________________________________________
> Endymail mailing list
> Endymail@ietf.org
> https://www.ietf.org/mailman/listinfo/endymail

-- 
"Those who would give up Essential Liberty to purchase a little
Temporary Safety deserve neither  Liberty nor Safety."
-- Benjamin Franklin