Re: [Asrg] Adding a spam button to MUAs

Douglas Otis <dotis@mail-abuse.org> Thu, 17 December 2009 19:59 UTC

Return-Path: <dotis@mail-abuse.org>
X-Original-To: asrg@core3.amsl.com
Delivered-To: asrg@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 79AF23A69C1 for <asrg@core3.amsl.com>; Thu, 17 Dec 2009 11:59:33 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.085
X-Spam-Level:
X-Spam-Status: No, score=-6.085 tagged_above=-999 required=5 tests=[AWL=0.358, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, SUBJECT_FUZZY_TION=0.156]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BJ4635jEZfAi for <asrg@core3.amsl.com>; Thu, 17 Dec 2009 11:59:32 -0800 (PST)
Received: from harry.mail-abuse.org (harry.mail-abuse.org [168.61.5.27]) by core3.amsl.com (Postfix) with ESMTP id 78B7428C166 for <asrg@irtf.org>; Thu, 17 Dec 2009 11:59:32 -0800 (PST)
Received: from sjc-office-nat-214.mail-abuse.org (gateway1.sjc.mail-abuse.org [168.61.5.81]) by harry.mail-abuse.org (Postfix) with ESMTP id 16676A94442; Thu, 17 Dec 2009 19:59:14 +0000 (UTC)
Message-ID: <4B2A8D91.80606@mail-abuse.org>
Date: Thu, 17 Dec 2009 11:59:13 -0800
From: Douglas Otis <dotis@mail-abuse.org>
User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.1.5) Gecko/20091204 Thunderbird/3.0
MIME-Version: 1.0
To: asrg@irtf.org, ARF mailing list <abuse-feedback-report@mipassoc.org>
References: <alpine.BSF.2.00.0912082138050.20682@simone.lan> <20091216014800.GA29103@gsp.org> <DBF77720-200E-4846-949F-924388F9CC15@blighty.com> <20091216120742.GA28622@gsp.org> <20091216185904.3B9032421D@panix5.panix.com> <4B296458.5070603@mail-abuse.org> <16C1C8A4-D223-435B-93BC-A9D44F5965A1@guppylake.com> <B14EC7430355853625D0D4EA@lewes.staff.uscs.susx.ac.uk> <BBF2AC03-3C88-4557-9346-343347C196A9@guppylake.com> <20091217182526.192832421D@panix5.panix.com> <01C75B49-E78B-4293-B5EC-B2690B7ECFD9@guppylake.com>
In-Reply-To: <01C75B49-E78B-4293-B5EC-B2690B7ECFD9@guppylake.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Subject: Re: [Asrg] Adding a spam button to MUAs
X-BeenThere: asrg@irtf.org
X-Mailman-Version: 2.1.9
Precedence: list
Reply-To: Anti-Spam Research Group - IRTF <asrg@irtf.org>
List-Id: Anti-Spam Research Group - IRTF <asrg.irtf.org>
List-Unsubscribe: <http://www.irtf.org/mailman/listinfo/asrg>, <mailto:asrg-request@irtf.org?subject=unsubscribe>
List-Archive: <http://www.irtf.org/mail-archive/web/asrg>
List-Post: <mailto:asrg@irtf.org>
List-Help: <mailto:asrg-request@irtf.org?subject=help>
List-Subscribe: <http://www.irtf.org/mailman/listinfo/asrg>, <mailto:asrg-request@irtf.org?subject=subscribe>
X-List-Received-Date: Thu, 17 Dec 2009 19:59:33 -0000

On 12/17/09 10:34 AM, Nathaniel Borenstein wrote:
> There's a difference between having an "A or B" bit, and having
> separate "A" and "B" bits that may be muddled.  If you don't know the
> right way to interpret the two bits, then the single bit is
> potentially much less misleading.
>
> But as I said, this is something that could really be resolved by
> empirical study, and I'd prefer not to trust my intuition or anyone
> else's if we can figure out the facts about how people would actually
> use two buttons.  If the error rate is low enough, then two buttons
> will make sense, but I've rarely managed to underestimate the
> competence of the average user.   Two buttons is way more than twice
> as confusing as one, in this case.  -- Nathaniel
>
> On Dec 17, 2009, at 1:25 PM, Seth wrote:
>
>> Nathaniel Borenstein<nsb@guppylake.com>  wrote:
>>> On Dec 17, 2009, at 11:27 AM, Ian Eiloart wrote:
>>
>>>> Twitter seems to think that users are smart enough to
>>>> distinguish between "unwanted" and "spam". They give you a
>>>> button for each. It's an important distinction that most people
>>>> can make.
>>>
>>> Twitter isn't always right, and my intuition differs from yours
>>> on this one.  Fortunately it's something that could be resolved
>>> empirically.  I'd like to see such a study, because it wouldn't
>>> take very many users who *can't* properly make that distinction
>>> to render the two-button solution counterproductive.  I'd rather
>>> have one bit of meaningful data than two bits of muddled data.
>>> -- Nathaniel
>>
>> One button is the "OR" of the two buttons, so there's no less
>> information available.  Given enough data, it should be easy to
>> get pretty accurate statistics on how reliable _each_ user is, and
>> the unreliable ones can be mapped into the one-button treatment.

Agreed. After all, can users recognize an auto-response in Chinese, or
know the difference between real or spoofed DSNs?  Of course not.

All that can be determined from user feedback metrics is that 'X' number
of messages from some source were "unwanted".  Filters can be trained at
removing "unwanted" for specific users based upon their feedback,
however this information's usefulness is limited without knowing overall
source volumes, and whether messages might have been either foreign
language auto-responses, or valid DSNs.

Having "unwanted" considered equivalent of "spam-trap" feedback will
create a number of complaints, especially when dealing with a diverse
range of users.  Spam-trap email-addresses do not solicit email and are
good at excluding auto-responses and valid DSN.  This allows spam-trap
feedback be applied over a larger range of users with less regard to
overall source volume.  Unfortunately, since no provider is able to
ensure all customers are not 0wned, even "spam-trap" feedback can
benefit when overall volume is shared.

Our process consolidates metrics for all activity seen from each
specific source and evaluates related metrics in a non-linear fashion.
Offering this information within ARF feedback would not entail a sizable
increase in report size to include a field of "source activity rate" as
a method to share the number of message events (good or bad) seen over
some number of seconds.

Volume metrics added to an ARF report might look something like:

IP-Activity: [source-IP], [#messages], [#seconds];
DKIM-Activity: [d=], [#messages], [#seconds];

Such as
IP-Activity: 192.0.2.128,34566,86400;
DKIM-Activity: example.com,4013,86400;

When metrics are passed through a feedback conduit, each source should
be qualified as representing spam-trap feedback, or user feedback, where
feedback metrics from multiple sources would be consolidated into at
least two separate categories.

-Doug