Re: [Asrg] Soundness of silence

[Bill Cole <[e-mail]>]

Alessandro Vesely wrote, On 6/16/09 7:28 AM:
> Bill Cole wrote:
>> Different people (and mail systems) have different spam problems.
>
> I tend to understand that as different classes of spam. For an example,
> consider a creditor of mines who solicits payment by sending me
> reminders. Assume I'm not going to pay and I just discard them. If, by
> chance, they end up in the spam folder, would I be willing to train my
> Bayesian filter to avoid that? Probably no. And, are those reminders
> spam? In some acceptation of the term, yes. Thus, a fax or a registered
> letter is better than email...

It goes beyond that sort of edge case of defining spam as "mail I don't 
like". There are envelope characteristics that exist in distinct types of 
mail that are mostly seen by different sets of receiving systems, such as 
messages with more than 10 recipients. For microdomains and mass-market mail 
providers, such mail is almost always archetypal spam: sent without any 
prior relationship to addresses harvested from the net or bought from a 
harvester. For many businesses, such mail is almost entirely legitimate mail 
from existing business partners: service providers, suppliers, etc.  On 
different mail systems, the same low-cost rule may correlate well to the 
spam/non-spam classification, *but in opposite directions.*



>> Many people have come up with "good enough" solutions for their own
>> spam problems, but they are no all the same solutions. The idea that
>> there is or could be one solution that works for everyone has largely
>> fallen into disrepute because all of the attempts at it have fallen
>> far short of the goal. Unfortunately, many of the de facto best
>> current practices are completely unsuited for technical
>> standardization. I don't think anyone wants to see any sort of RFC
>> that recommends using any specific DNSBL, but for many people running
>> mail systems of a wide variety the use of the Spamhaus Zen DNSBL is
>> their most effective single anti-spam tactic. Recommending the
>> shunning of specific whole countries certainly does not belong in
>> anything that anyone might see as a "standard" but as a matter of
>> practicality, many mail systems do so to great benefit and at no
>> tangible cost.
>
> I don't see why such techniques are not amenable to standardization.
> Actually, there is a couple of DNSBL drafts that are slowly moving forward.

Which are good efforts, but they don't actually tell readers which DNSBL's 
are highly effective and which are dangerous to their mail. Or which might 
be both. For the overwhelming majority of mail systems, the most effective, 
cost-effective, and safe tool to shun spam is the Spamhaus Zen list, but it 
would be a very bad idea for any RFC to say that. Similarly, there are very 
safe, cheap, and effective ways to stop spam before DATA based on rDNS and 
HELO names that could never pass muster for an RFC.

> It should be possible for my SMTP server to accept mail only from, say,
> an office opposite with whom I do most business, and shunning all the
> rest except, say, Gmail, thereby relying on their filtering. There's
> nothing wrong with that, except for technical problems that make it
> difficult to set it up properly.

No RFC will (or should) ever recommend such an approach.

That is not because such an approach will never be the best one for any 
system, but because it is not a widely deployable solution and it relies 
upon a characteristic of the mail world that may well be transient.

>> Because spam is fundamentally a social problem rather than a technical
>> problem, the technical approaches to fixing it are all imperfect, many
>> subsets are subject to "arms race" problems, and the only
>> generalizable solution is that everyone running a mail system should
>> apply a mix of tactics suited to their spam and their non-spam (based
>> on the locally relevant definition of "spam") and pay attention to how
>> those tactics work *for them* rather than seek to locally deploy some
>> global solution.
>
> Yes, that's the conclusion I also reached. Spam is a universal plague
> and we must live with it. It is indecent to egoistically take oneself
> away from it. Therefore, solutions to get rid of spam are not wanted,
> not even discussed. BTW, discussion implies that someone will try to
> also get rid of direct marketing, in the bargain. So, let's keep all of
> it, even the inadmissible zombie-generated spam.

I disagree. :)

I think you are misunderstanding my point. The existing tools are good 
enough that most mail system operators can put together some set of them to 
assure that a large majority of their users see spam rarely and have very 
little legitimate mail blocked, while the non-zero level of errors in both 
directions have made users more acclimated to and forgiving of such 
imperfections. This has raised the bar significantly for new technical 
approaches, which will not even get attention unless they are very good, 
very low-cost, and very easy to deploy.


[...]
>> Your proposal is complex enough that making a careful analysis takes
>> real effort. A casual scan of the document doesn't yield obvious fatal
>> flaws, nor does it provide any instant 'AHA!' response of how the VHLO
>> mechanism would provide a clear fix for a major problem. That results
>> in it seeming like a low-yield chore to go through 23 pages of details
>> to figure out whether this idea is sound and useful. Maybe improving
>> sections 1.1-1.3 to more directly and concisely define the problem
>> VHLO is meant to address would encourage more attention.
>
> That's what I've been trying to do in the last two rounds. Any explicit
> hint?

Replace the tutorial on mail filtering fundamentals with a concise problem 
definition and concise explanation of how VHLO provides a solution.

[...]
>> More telling: I'm not convinced that any new technical approach to
>> spam control has any chance of widespread adoption or even careful
>> attention. The jungle of existing tactics combined with a drop in user
>> expectations has resulted in a circumstance where the demand for
>> better mail service is not enough to get significant new technical
>> approaches deployed.
>
> Great! I cannot tell it better than that. It obviously implies that
> email is going to die out.

Not at all. I just don't expect that it will every be like 1993 again. I 
think we've reached something like a dynamic equilibrium over the past few 
years, and it will take a really big push to change that. There are many 
mail systems out there shunning 97%+ of all messages while delivering less 
than a spam per week per user and stopping less than one legitimate message 
per year per user. 5 years ago, that sort of accuracy took an anti-spam 
craftsman tending a garden of homegrown tools (and customizations of open 
tools) with users screaming bloody murder over every error. Today you can 
buy it in a box or as a service, and the users are largely resigned to the 
fact that sometimes mail goes missing and sometimes they get solicited for 
dubious drugs and money-making schemes. Perversely, users have also become 
shockingly dependent on Internet email, and expect it to do things that they 
never would have asked back before mail administrators evolved into a breed 
of artful destroyers of most mail.

 > Newcomers don't perceive it as something new
> and exciting, but rather as an obsolete communication system used
> predominantly by elder people, generally left in a state of regrettable
> neglect.

That perception is IMHO largely shaped by the fact that the newest of 
newcomers are people who do not actually operate as autonomous adults.