Re: [Asrg] Adding a spam button to MUAs

Jose-Marcio Martins da Cruz <Jose-Marcio.Martins@mines-paristech.fr> Sun, 20 December 2009 12:54 UTC

Return-Path: <Jose-Marcio.Martins@mines-paristech.fr>
X-Original-To: asrg@core3.amsl.com
Delivered-To: asrg@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 5D51D3A6912 for <asrg@core3.amsl.com>; Sun, 20 Dec 2009 04:54:45 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: 0.507
X-Spam-Level:
X-Spam-Status: No, score=0.507 tagged_above=-999 required=5 tests=[BAYES_50=0.001, HELO_EQ_FR=0.35, SUBJECT_FUZZY_TION=0.156]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wDWwD2afau69 for <asrg@core3.amsl.com>; Sun, 20 Dec 2009 04:54:44 -0800 (PST)
Received: from boipeva.ensmp.fr (cobra.ensmp.fr [194.214.158.101]) by core3.amsl.com (Postfix) with ESMTP id 083963A683F for <asrg@irtf.org>; Sun, 20 Dec 2009 04:54:43 -0800 (PST)
Received: from localhost.localdomain (joe.j-chkmail.org [88.168.143.55]) (authenticated bits=0) by boipeva.ensmp.fr (8.14.3/8.14.3/JMMC-11/Feb/2009) with ESMTP id nBKCsMVe002336 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for <asrg@irtf.org>; Sun, 20 Dec 2009 13:54:23 +0100 (MET)
Message-ID: <4B2E1E76.9000400@mines-paristech.fr>
Date: Sun, 20 Dec 2009 13:54:14 +0100
From: Jose-Marcio Martins da Cruz <Jose-Marcio.Martins@mines-paristech.fr>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.23) Gecko/20090908 Fedora/1.1.18-1.fc11 SeaMonkey/1.1.18
MIME-Version: 1.0
To: Anti-Spam Research Group - IRTF <asrg@irtf.org>
References: <alpine.BSF.2.00.0912082138050.20682@simone.lan> <20091216014800.GA29103@gsp.org> <DBF77720-200E-4846-949F-924388F9CC15@blighty.com> <20091216120742.GA28622@gsp.org> <20091216185904.3B9032421D@panix5.panix.com> <4B296458.5070603@mail-abuse.org> <16C1C8A4-D223-435B-93BC-A9D44F5965A1@guppylake.com> <B14EC7430355853625D0D4EA@lewes.staff.uscs.susx.ac.uk> <BBF2AC03-3C88-4557-9346-343347C196A9@guppylake.com> <240DB04672256506ED548857@lewes.staff.uscs.susx.ac.uk> <4B2A7E8D.8060104@nd.edu> <20091217200605.8E99E2421D@panix5.panix.com> <4B2B0E4B.3050509@dcrocker.net>
In-Reply-To: <4B2B0E4B.3050509@dcrocker.net>
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 8bit
X-Miltered: at boipeva with ID 4B2E1E7E.000 by Joe's j-chkmail (http : // j-chkmail dot ensmp dot fr)!
X-j-chkmail-Enveloppe: 4B2E1E7E.000/88.168.143.55/joe.j-chkmail.org/localhost.localdomain/<Jose-Marcio.Martins@mines-paristech.fr>
Subject: Re: [Asrg] Adding a spam button to MUAs
X-BeenThere: asrg@irtf.org
X-Mailman-Version: 2.1.9
Precedence: list
Reply-To: Jose-Marcio.Martins@mines-paristech.fr, Anti-Spam Research Group - IRTF <asrg@irtf.org>
List-Id: Anti-Spam Research Group - IRTF <asrg.irtf.org>
List-Unsubscribe: <http://www.irtf.org/mailman/listinfo/asrg>, <mailto:asrg-request@irtf.org?subject=unsubscribe>
List-Archive: <http://www.irtf.org/mail-archive/web/asrg>
List-Post: <mailto:asrg@irtf.org>
List-Help: <mailto:asrg-request@irtf.org?subject=help>
List-Subscribe: <http://www.irtf.org/mailman/listinfo/asrg>, <mailto:asrg-request@irtf.org?subject=subscribe>
X-List-Received-Date: Sun, 20 Dec 2009 12:54:45 -0000

Dave CROCKER wrote:
> 
> 
> On 12/17/2009 12:06 PM, Seth wrote:
>> We don't know that it _will_ be the worst case.  To the extent there
>> are _some_ users who can tell the difference (and the admin can tell
>> the difference between them and other users), having two buttons gives
>> better information.
> 
> 
> unless there are _some_ users who are more confused by the choice.  UI 
> complexity is not reduced by having more buttons; reducing complexity 
> leads to better UI, particularly for the mass market.
> 
> as nathaniel notes, these types of design choices for UI design require 
> testings.

I strongly agree.

Quoting a research paper from Gordon Cormack [1] :

"When explicitly asked to classify messages, human subjects have been reported to exhibit error 
rates of 3%-7% [30, 16]. Tacitly derived labels, such as those obtained from a “report spam” button, 
where it is assumed that unreported messages are ham, may have even higher rates."

The references 30 and 16 in this paper comes from experiments with real data and real subjects 
(SpamOrHam - Graham-Cumming and Hotmail - Yih/Kolcz)

These error rates are most of the time bigger than what can be achieved by spam filters. So it's 
probably a bad idea to consider that user feedback is reliable. User interface shall be as simple as 
possible.

[1] Cormack, G. V. and Kolcz, A. 2009. Spam filter evaluation with imprecise ground truth. In 
Proceedings of the 32nd international ACM SIGIR Conference on Research and Development in 
information Retrieval (Boston, MA, USA, July 19 - 23, 2009). SIGIR '09. ACM, New York, NY, 604-611. 
http://plg.uwaterloo.ca/~gvcormac/cormacksigir09-spam.pdf

JM

> 
> for folks who do serious UX work, they do not guarantee that their 
> suggestions are right, merely that they are worth testing.
> 
> d/


-- 
  ---------------------------------------------------------------
  Jose Marcio MARTINS DA CRUZ           http://j-chkmail.ensmp.fr
  Ecole des Mines de Paris
  60, bd Saint Michel                      75272 - PARIS CEDEX 06
  mailto:Jose-Marcio.Martins@mines-paristech.fr