82% of email is spam

http://cr.yp.to/im2000.html

I've posted this on Slashdot a few times, and whiners always come back with reasons why it won't work. I have, with very little difficulty, been able to think of a reasonable answer to every objection.

So before you gripe here about why it won't work, please spend a few minutes yourself trying to think of a way to get around your supposed problem.

The biggest problem by far is getting people to switch mail systems. But if we do nothing, spam will keep getting worse. IM2000, along with a good blacklist system (which would work MUCH better than blacklists with SMTP), should go a long way to stopping spam. No centralized mail servers, certificates, or government intervention required.

Last I checked a few months ago, there were some good proposals swimming around on the Net. Need to look again for progress. Unfortunately, I don't know of an implementation yet, but it shouldn't be *that* hard. Once we get that, I bet people will start switching. Then as they drop SMTP accounts, others will need to switch to IM2000 to communicate with them. :)

Will people disobey the law? Sure. But after a few people lose all their money, or go to jail, this exploitation will quickly reduce to levels that can be handled by current technology.

We need both laws and technology. One without the other is like fighting with one hand tied behind our back.

Europe is already leading this way; they've already passed these kinds of laws. But without cooperation from the US, it won't be enough. Once there are only a few rogue states, their cooperation can be more easily acquired... by refusing to accept any of their email until they update their laws. This is just like the computer break-in laws a few decades ago; at first, many people didn't even understand that break-ins were like trespass. Then, over time, laws were passed to forbid people from harming others using computers.

Wood also attributes the rise in spam to the huge number of open proxies on the Net.

Virus writers began teaming up with spammers last year, and so far it's been a dangerous combination. Virus writers send out malicious code that infects computers and opens a back door in the machine.

Wood estimates that 70 percent of spam is sent through open proxies.

Well, then, this isn't a problem, because as everyone knows, Microsoft is now taking security seriously! So I'm sure all those zombies will be dead real soon now.

This isn't a complete solution by any means, but it is likely to be a neccessary step in making a complete solution possible which doesn't break too many things worth keeping.

This has cut the spam I receive down to a small trickle from the couple hundred a day, to say nothing for the other users of the system I admin. (The only spam I get now is stuff sent via massive BCCs using a rather traditional e-mail client)

The nice thing about this is that it happens before mail is delivered, so you save on the bandwidth, storage, and cpu costs of post-delivery filtering.

http://projects.puremagic.com/greylisting/

There are multiple implementations of this technique now.

--- quote:

Your post advocates a

( ) technical
(X) legislative
( ) market-based
( ) vigilante

approach to fighting spam. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)

( ) Spammers can easily use it to harvest email addresses
( ) Mailing lists and other legitimate email uses would be affected
( ) No one will be able to find the guy or collect the money
( ) It is defenseless against brute force attacks
( ) It will stop spam for two weeks and then we'll be stuck with it
( ) Users of email will not put up with it
( ) Microsoft will not put up with it
( ) The police will not put up with it
( ) Requires too much cooperation from spammers
(X) Requires immediate total cooperation from everybody at once
( ) Many email users cannot afford to lose business or alienate potential employers
( ) Spammers don't care about invalid addresses in their lists
( ) Anyone could anonymously destroy anyone else's career or business

Specifically, your plan fails to account for

( ) Laws expressly prohibiting it
(X) Lack of centrally controlling authority for email
(X) Open relays in foreign countries
( ) Ease of searching tiny alphanumeric address space of all email addresses
(X) Asshats
(X) Jurisdictional problems
( ) Unpopularity of weird new taxes
( ) Public reluctance to accept weird new forms of money
( ) Huge existing software investment in SMTP
( ) Susceptibility of protocols other than SMTP to attack
( ) Willingness of users to install OS patches received by email
( ) Armies of worm riddled broadband-connected Windows boxes
( ) Eternal arms race involved in all filtering approaches
( ) Extreme profitability of spam
( ) Joe jobs and/or identity theft
( ) Technically illiterate politicians
( ) Extreme stupidity on the part of people who do business with spammers
(X) Dishonesty on the part of spammers themselves
( ) Bandwidth costs that are unaffected by client filtering
( ) Outlook

and the following philosophical objections may also apply:

(X) Ideas similar to yours are easy to come up with, yet none have ever
been shown practical
( ) Any scheme based on opt-out is unacceptable
( ) SMTP headers should not be the subject of legislation
( ) Blacklists suck
( ) Whitelists suck
( ) We should be able to talk about Viagra without being censored
( ) Countermeasures should not involve wire fraud or credit card fraud
( ) Countermeasures should not involve sabotage of public networks
( ) Countermeasures must work if phased in gradually
( ) Sending email should be free
( ) Why should we have to trust you and your servers?
( ) Incompatiblity with open source or open source licenses
( ) Feel-good measures do nothing to solve the problem
( ) Temporary/one-time email addresses are cumbersome
( ) I don't want the government reading my email
( ) Killing them that way is not slow and painful enough

Furthermore, this is what I think about you:

(X) Sorry dude, but I don't think it would work.
( ) This is a stupid idea, and you're a stupid person for suggesting it.
( ) Nice try, assh0le! I'm going to find out where you live and burn your
house down!

At work, we use a 3-tiered strategy that's pretty effective. Firstly, we use RBL blacklists, and reject connections from malconfigured mail servers (eg with DNS problems); secondly, we use SpamAssassin on everything that reaches our queue; thirdly, we've put most of our users onto Thunderbird, and the built-in adaptive filtering is pretty good at mopping up anything that gets through SpamAssassin, once it's been trained.

For me personally, the biggest problem is accessing my (POP-based) mail via the webmail gateway when I'm away from home: with no spam filtering at that stage, I have to wade through 3 screenfuls of crap to find the 1 legitimate mail waiting for me....

Besides, spam e-mails are generally small. I have 13 in my filtered area now, the largest is 12 KB. People regularly send me 1MB files in the mail.

Mailing lists also could work this way, carrying both the mailing list signature and the signature of the poster.

This would work for 90% of users or more. Those needing to receive truly anonymous mail, could set special mailboxes for that.

Thinking of that, is anyone aware of any anti-spam filter that works on signatures? (Just curious: my personal small spam problems are easily solved by pressing 'd' in mutt some 8-10 times per day ).

Unfortunately, trust metrics are difficult enough to understand (and I've done such a poor job evangelizing people so they're motivated to understand), that they haven't made much impact on the world.

I resemble a few of those check-boxes on the form posted above, but I do suspect that one of two things is true: the spam problem won't be solved in our lifetime because of a combination of technical difficulty and lack of willpower; or that the solution will use some form of peer certification to distinguish spammers from legitimate email correspondents. These days I'm more inclined to believe the former, but that's probably just my legendary bitterness shining through.

But on a more serious side, I'm coming to the conclusion that the only long-term solution to SPAM must involve white-listing. i.e. I *only* accept mail from addresses (SPF-verifiable addresses) that I trust. This would require MUA support so that anyone I send mail to automatically gets white-listed, and things like that.

It would also mean that when people (e.g. companies) ask for your Email address, their would need to give your their email address in return (We will only send you mail from info@clever-company.com.) which you would have to white-list.

If people who you don't know want to send you mail, that has to be possible, but it also should be expensive (roughly the cost of a postage stamp or a 'phone call). This might involve finding a common correspondant to introduce you, or it might involve some "proof-of-cpu-power-spent" similar to HashCash, or it could even involve an exchange of money (e.g. I will read your mail if you can prove that you have bought a $1 e-postage stamp from World Vision). It could use any other challenge-response that a recipient is happy to impose on potential senders.

The big problem today with challenge-response is that you risk sending challenges to innocent third-parties whose address has been used inappropriately. This is where I think SPF really gives value. With SPF, I can tell if I can trust a return-address, and so I can know if it is safe to send a challenge.

If the MTA add an appropriate header with SPF status, this can all be done in the MUA.

This doesn't address the bandwidth/server-load problem. It isn't clear to me that there can be a better total solution to that than pushing the whitelist+challenge-response into the MTA (there are lots of partial solutions which can heurisically drop a lot of bad mail, but they will eventually cause the spammers to get smarter).

1) I get spam in my mailbox.
2) I forward the spam to a "spam alert" service that verifies it as spam and then extracts the meat from the spam (the human intervention part), then feeds the meat to a pattern builder.
3) The pattern is forwarded to all ISPs to be used in their spam filters. The filter would reject any email that contained text that fit the pattern within some reasonably high confidence level. The rejected email would be returned to the sender, hopefully by the sender's own ISP. Rejecting spam at the network edge would be a huge win.
4) In the case of a false positive, the sender would have to sufficiently restate the email's text. This could be a pain, but I doubt it would happen all that often, percentage-wise.
5) The spammer is also forced to sufficiently restate his spam.

So what do we gain? The spammer has to work a lot harder (harder than adding some bogus word strings, anyway). With adequate speed of response from spam recipients, many of the spammer's targets would never see his spam. And the pattern building is largely centralized and so can be more easily improved as we become more clever. Additionally, catching spam at the network edge allows easier identification of the spam source.

The monetary cost is primarily in the "spam alert" service. I suspect that cost will be very low compared to the size of the spam problem. The social cost (or effort) will include getting most of the ISPs to use the pattern filtering, but I suspect seeing success at a few ISPs will provide a lot of momentum. I for one am certainly willing to badger my ISP to use it.

There will be no silver bullet. As we all know, there are always antisocials with too much time and misdirected energy. The best we can do is to keep far enough ahead of them to keep their damage to a tolerable level. I think that will be good enough, if we can do it.

This all is about humans, not about the computers. But computers can help achieve the goal. Say, any mail server that detects a spam messsage attaches a promotion text just above the text of the message, explaining the problem briefly and saying that the only way to stop spam is to stop accepting *any* offerings in that spam.

That is, attaching anti-spam to any spam. The more spam one gets, the more probably he will read the banner at last and understand the problem.

Well, in case most people are failing to read that banners and understand the problem, all the humanity will keep getting spam -- in that case, it actually deserves it. That sounds bitter, of course, but that's the economy -- while there is an interest, there is an offering.