Re: [Asrg] Adding a spam button to MUAs

Ian Eiloart <iane@sussex.ac.uk> Tue, 22 December 2009 11:30 UTC

Return-Path: <iane@sussex.ac.uk>
X-Original-To: asrg@core3.amsl.com
Delivered-To: asrg@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id CDE453A68D2 for <asrg@core3.amsl.com>; Tue, 22 Dec 2009 03:30:53 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.574
X-Spam-Level:
X-Spam-Status: No, score=-0.574 tagged_above=-999 required=5 tests=[AWL=-0.731, BAYES_50=0.001, SUBJECT_FUZZY_TION=0.156]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xLx-Fys5yWHw for <asrg@core3.amsl.com>; Tue, 22 Dec 2009 03:30:52 -0800 (PST)
Received: from lynndie.uscs.susx.ac.uk (lynndie.uscs.susx.ac.uk [139.184.14.87]) by core3.amsl.com (Postfix) with ESMTP id BAF7E3A6AE5 for <asrg@irtf.org>; Tue, 22 Dec 2009 03:30:52 -0800 (PST)
Received: from lewes.staff.uscs.susx.ac.uk ([139.184.134.43]:58343) by lynndie.uscs.susx.ac.uk with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.64) (envelope-from <iane@sussex.ac.uk>) id KV1XEI-0006IV-EE for asrg@irtf.org; Tue, 22 Dec 2009 11:32:42 +0000
Date: Tue, 22 Dec 2009 11:30:35 +0000
From: Ian Eiloart <iane@sussex.ac.uk>
Sender: iane@sussex.ac.uk
To: Anti-Spam Research Group - IRTF <asrg@irtf.org>
Message-ID: <E12D471E9BA7C62AC1CE7453@lewes.staff.uscs.susx.ac.uk>
In-Reply-To: <4B2FBD46.1010809@leisi.net>
References: <alpine.BSF.2.00.0912082138050.20682@simone.lan> <20091216014800.GA29103@gsp.org> <DBF77720-200E-4846-949F-924388F9CC15@blighty.com> <20091216120742.GA28622@gsp.org> <20091216185904.3B9032421D@panix5.panix.com> <4B296458.5070603@mail-abuse.org> <16C1C8A4-D223-435B-93BC-A9D44F5965A1@guppylake.com> <B14EC7430355853625D0D4EA@lewes.staff.uscs.susx.ac.uk> <BBF2AC03-3C88-4557-9346-343347C196A9@guppylake.com> <240DB04672256506ED548857@lewes.staff.uscs.susx.ac.uk> <4B2A7E8D.8060104@nd.edu> <AF09C4BE1E9DB501F0D2CDF3@paine.local> <940DF2A7-C912-49C1-B967-E82CA69649D3@guppylake.com> <4B2FBD46.1010809@leisi.net>
Originator-Info: login-token=Mulberry:01VqXgm78GTgCXEjT9wsvJGlMOvB0B9bnLyxA=; token_authority=support@its.sussex.ac.uk
X-Mailer: Mulberry/4.0.8 (Mac OS X)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
X-Sussex: true
X-Sussex-transport: remote_smtp
Subject: Re: [Asrg] Adding a spam button to MUAs
X-BeenThere: asrg@irtf.org
X-Mailman-Version: 2.1.9
Precedence: list
Reply-To: Anti-Spam Research Group - IRTF <asrg@irtf.org>
List-Id: Anti-Spam Research Group - IRTF <asrg.irtf.org>
List-Unsubscribe: <http://www.irtf.org/mailman/listinfo/asrg>, <mailto:asrg-request@irtf.org?subject=unsubscribe>
List-Archive: <http://www.irtf.org/mail-archive/web/asrg>
List-Post: <mailto:asrg@irtf.org>
List-Help: <mailto:asrg-request@irtf.org?subject=help>
List-Subscribe: <http://www.irtf.org/mailman/listinfo/asrg>, <mailto:asrg-request@irtf.org?subject=subscribe>
X-List-Received-Date: Tue, 22 Dec 2009 11:30:53 -0000

--On 21 December 2009 19:24:06 +0100 Matthias Leisi <matthias@leisi.net> 
wrote:

>
> Am 21.12.09 18:46, schrieb Nathaniel Borenstein:
>
>> distinction, it's whether the users can.  If the users can't use the
>> client's two buttons with sufficiently low error rates, then the
>> resulting data can't possibly be useful to the admins.  In other
>
> When I was responsible for spamfilter operation at a former job, error
> rates of "human spamfilters" were considerably higher than FP rates of
> any possible solution. This is not scientific evidence, but it
> illustrates it nicely:

I nice story. It says that we must not block mail on a single report, 
especially a report from a different user. That's fair enough, but it's not 
what anyone is suggesting.

The current question is how can we make it easier for users to report spam 
to administrators. The question of what administrators do with the reports 
is secondary - though it might inform the kind of information that they 
want to gather.



> It was very important for the CEO of the company not to lose mail. It
> was thus decided that his assistant would go through his inbox and spam
> folder and use "Mark as Spam" and "Mark as Not Spam" buttons to clean up
> things.
>
> This assistant was very diligent, and highly capable at her job.
> Nevertheless, she had a surprisingly high error rate -- for every
> hundred mails which she marked as (not) spam, she mis-categorized maybe
> five to ten mails.
>
> I did not watch the other users as closely, but they had similar error
> rates. The feedback loop from the users could never have been used for
> automated actions due to the low quality (maybe it would have been
> possible with a larger user base and appropriate statistical analysis).
> The buttons were still an important source for fine tuning of the filter
> and to find emerging trends in spammer behaviour. Nothing more, but
> nothing less.
>
> -- Matthias
> _______________________________________________
> Asrg mailing list
> Asrg@irtf.org
> http://www.irtf.org/mailman/listinfo/asrg



-- 
Ian Eiloart
IT Services, University of Sussex
01273-873148 x3148
For new support requests, see http://www.sussex.ac.uk/its/help/