LWN: Comments on "82% of email is spam"

82% of email is spam

coolian — Mon, 17 May 2004 12:01:26 +0000

Why do you think Microsoft will EVER change, unless given a monetary reason to do so? They dominate for now, and they rarely change until they are forced to. Bush couldn't force them, but maybe in a few years, when Linux is making MS toss its salad...

Why does everyone bring this up every time?

coolian — Mon, 17 May 2004 11:58:27 +0000

1. Regulating all mail servers with government intevention

>> Oh yeah, I don't think so.

2. Using only centralized mail servers

>> Meaning what?

3. Using certificates for mail servers, with each server needing to pay out the arse for a certificate

>> Why? Do you understand how certs work?

4. IM2000, which has a couple minor disadvantage for a few uses, and which can be worked around

>> Minor disadvantages? The whole system not working is more than minor. There are MAJOR unresolved issues with it, that you have zero answers for. They can be worked around? By who? You?

5. 82% of our bandwidth being wasted on spam.

>> This is the least evil of the ones you list. Sorry, but most would agree. I don't want people like you, who have zero answers, running my mail. Thanks, but no.

Problem with your 'Solution'

coolian — Mon, 17 May 2004 11:52:14 +0000

" Dan Bernstein made the proposal, not me."

But you suggested it and said it would "of course" be the answer. If you don't have any actual knowledge of it, then how could you know it's the answer? Jesus, you must be 12.

Problem with your 'Solution'

coolian — Mon, 17 May 2004 11:50:11 +0000

The original "IM2000" guy is obviously a hard-core democrat. I'm no Republican, but this "legislate my butt out of existence" and "trust the ISP not to go down" thing is WAY too much trust for me.

Problem with your 'Solution'

coolian — Mon, 17 May 2004 11:45:33 +0000

It's idiotic. You said the answer was "of course" this guy...It wouldn't work. Why? Because it wouldn't.

I think the only thing that will work without legislation is filtering/whitelist/challenge-response.

spam filtering by ISPs

giraffedata — Sat, 15 May 2004 17:19:56 +0000

We may be using different definitions of ISPs; I believe this thread is about providers of email mailboxes. AOL, Yahoo, Hotmail, Earthlink, ...

I know these guys filter spam based on content. I see it in my own Yahoo mail account, and Earthlink advertises it. So why is spam still effective? Are they not filtering enough? Are users opting out?

82% of email is spam

bronson — Sat, 15 May 2004 17:00:49 +0000

Big ISPs never look at the content of what they are transmitting, lest they lose their common carrier status. Also, the tinfoil hat crowd claims that the big ISPs make good money on spam so it's not in their best interest to stop it.

82% of email is spam

Wol — Thu, 13 May 2004 12:47:58 +0000

My box got compromised (W2K server).

Indeed, it was so bad, that it took me about four attempts before I could download the necessary security patches to fix the problem. The spam software - which I didn't know what it was although I knew something was wrong - was pinching nearly all available cpu and bandwidth so the system had nothing left to download the patches with :-(

Cheers,
Wol

82% of email is spam

ikm — Mon, 10 May 2004 15:17:45 +0000

I think the roots of the problem are very simple. Spam exists just because it is welcomed by most people. That's right -- they read it, and in case they are not interested about the offering they read about, they pronanly just think -- "what the hell, spam sucks!"... but if they are interested about the offering, they go for it! They go for it! That way spam proves as an effective marketing solution. And the only real way to stop it is to make it ineffective.

This all is about humans, not about the computers. But computers can help achieve the goal. Say, any mail server that detects a spam messsage attaches a promotion text just above the text of the message, explaining the problem briefly and saying that the only way to stop spam is to stop accepting *any* offerings in that spam.

That is, attaching anti-spam to any spam. The more spam one gets, the more probably he will read the banner at last and understand the problem.

Well, in case most people are failing to read that banners and understand the problem, all the humanity will keep getting spam -- in that case, it actually deserves it. That sounds bitter, of course, but that's the economy -- while there is an interest, there is an offering.

82% of email is spam

giraffedata — Sat, 08 May 2004 17:13:10 +0000

It seems to me that that doesn't have to be done with a complex inter-ISP system. It can be done within each individual ISP (and I thought they were already doing it). I believe the vast majority of the world's email accounts are within giant ISPs. If we can filter spam from giant ISP users, the spammers will give up.

So why isn't it working? Is it technologically infeasible? Are big ISP customers not availing themselves of the filtering?

Thrusted mail ?

giraffedata — Sat, 08 May 2004 16:56:21 +0000

Your system doesn't even require signatures. It's exactly what I do today, based on the from: header. The from: header is successfully forged (i.e. has one of the 2,000 addresses in my white list) in less than 1% of my spam.

But I still have a big problem. I route the stranger mail to a special folder, which I check daily, but there are 250 spams a day. I've had to start automatically deleting some of it (e.g. any all-html email), but that's risky.

I think people who believe the only interesting email they will get is from acquaintances lack imagination. You can't just delete mail from strangers; you have to spend some time looking at it to see if it's spam.

I occasionally have my outgoing emails bounced by overzealous spam filters, and in every case the recipient would have been glad to get my email.

82% of email is spam

rgoates — Fri, 07 May 2004 21:21:42 +0000

My 2 bits. Most of the spam I get is very repetitious in form, disregarding the list of bogus words used to bypass filters. I suspect a little human intervention added to filters can get rid of most spam, and doing it at the ISP level can reduce the impact on the net. Here's how it might work:

1) I get spam in my mailbox.
2) I forward the spam to a "spam alert" service that verifies it as spam and then extracts the meat from the spam (the human intervention part), then feeds the meat to a pattern builder.
3) The pattern is forwarded to all ISPs to be used in their spam filters. The filter would reject any email that contained text that fit the pattern within some reasonably high confidence level. The rejected email would be returned to the sender, hopefully by the sender's own ISP. Rejecting spam at the network edge would be a huge win.
4) In the case of a false positive, the sender would have to sufficiently restate the email's text. This could be a pain, but I doubt it would happen all that often, percentage-wise.
5) The spammer is also forced to sufficiently restate his spam.

So what do we gain? The spammer has to work a lot harder (harder than adding some bogus word strings, anyway). With adequate speed of response from spam recipients, many of the spammer's targets would never see his spam. And the pattern building is largely centralized and so can be more easily improved as we become more clever. Additionally, catching spam at the network edge allows easier identification of the spam source.

The monetary cost is primarily in the "spam alert" service. I suspect that cost will be very low compared to the size of the spam problem. The social cost (or effort) will include getting most of the ISPs to use the pattern filtering, but I suspect seeing success at a few ISPs will provide a lot of momentum. I for one am certainly willing to badger my ISP to use it.

There will be no silver bullet. As we all know, there are always antisocials with too much time and misdirected energy. The best we can do is to keep far enough ahead of them to keep their damage to a tolerable level. I think that will be good enough, if we can do it.

Part of the solution: laws that work

brouhaha — Fri, 07 May 2004 21:09:28 +0000

you'd probably need some kind of penalty for having your products advertised via spam.

Some people have proposed the death penalty for spammers. Most such proposals are probably somewhat tongue-in-cheek. I used to think that the death penalty for spam was too severe, but after a small amount of back-of-the-envelope analysis, I'm not so sure. A single spam campaign easily wastes 10 seconds each of the lives of 100,000,000 people. In total, that's more than a third of a human lifetime. So one spam campaign is equivalent to murdering a 50 year old person. The death penalty (or life imprisonment with no possibility of parole) thus seems entirely reasonable.

Solution

brouhaha — Fri, 07 May 2004 21:02:14 +0000

OK, so I've spent a few minutes thinking about problems with IM2000 in terms of spam avoidance, and trying to think up a solution.

The problem I see is that this doesn't address spam at all. It just changes the delivery mechanism. Today, the spammers bombard your MTA (e.g., Sendmail, Postfix, or Exchange) with spam. Your MTA uses a lot of resources (CPU, memory, disk) dealing with this. Depending on the MTA and configuration, it may or may not be able to do some filtering to avoid actually storing some of the spam in your mail queue. Then you check your inbox using an MUA (mail client such as Evolution, Mozilla Mail, Eudora, or Outlook). The MUA may also do some filtering, but at the very least has to inspect the headers of each message the MTA has received for you.

With IM2000, the spammer's machine doesn't directly send the spam to you. Instead, it tells your MTA (or whatever the IM2000 equivalent of an MTA is) that there is mail waiting for you on the spammer's machine. So now instead of getting 1200 spam emails a day, you get 1200 email waiting notifications from random machines all over the internet, many of which are probably zombies. Ultimately your software doesn't have any better way to tell which are spam than in today's architecture; it will simply have to poll each of the 1200 sender mail servers to get the mail headers and try to filter them.

This has *perhaps* reduced the internet backbone bandwidth consumed somewhat, if the filtering can be done with inspection of headers only and not the full message body, but it has not solved the spam problem. In fact, given that the IM2000 model provides for the sender's mail server to repeat the "mail available" notifications to the receiver, it may not even affect the bandwidth consumption. And it certainly does not improve the bandwidth consumption on the user's local link.

As far as I can see, IM2000 is a solution to a non-problem. It has attempted to solve the problem of reducing the amount of disk space the recipient needs to store the email, but disk space is the least significant problem associated with spam.

IM2000 appears to solve the "mailing list problem", except that there isn't really a "mailing list problem" either.

Thrusted mail ?

dd9jn — Fri, 07 May 2004 12:37:30 +0000

It is not only businesses receiving a lot of mail from yet unknown people but also people from the Free Software community, especially authors and maintainers. There is as well the problem of resending and forwarding messages.

Mailing lists are another problem. Of course the mailing list software could sign all message to be send out but that won't help. For a closed mailing list this will currently help but I have already encountered faked From addresses (which are the current way of authenticating subscribers) which led spam slip through. Open mailing lists (everyone is allowed to post) are already nice spam exploders and it won't help to have a signature applied by the ML software. Over short or long we have to change the authentication of mailing lists anyway to a stronger one (i.e. only accept signed posts), but this will require manual approval of subscription requests to sort out spammers. For some mailing lists this will not be possible at all - think of a help list for the signing or MUA software ;-).

Thrusted mail ?

bockman — Fri, 07 May 2004 11:22:56 +0000

Uhm, maybe my usage of e-mail is anomalous, but 99% of the mail I receive, at work and at home, belongs to one of these categories:
- people I know (and then I can get their publick key)
- mailing list or newsletted which I subscribed to (and also here, a public key could be distributed by the mailing list )
- Spam/viruses (ok, viruses could also come from known people, but spam usually doesn't)

Once per month, or even less, I may receive a mail from an unknown person, that wants to get in touch with me.

Therefore, for me would be very easy to define a list of 'good signatures' that could be used to filter my mail (possibly at server level). If I don't want to loose the mail from unauthenticated sources, I could isolate it in a separate mail folder, to check when/if I want. This mail folder could collect spam, but at least it is nicely isolated (and if I get too much annoyed, I could simply bounce unauthenticated mail).

Now, I understand that people, and especially business, can use e-mail to get in touch daily with unknown people. But I believe that a lot of people have an e-mail behaviour very similar to mine.
Consider also that you don't need the same 'level of thrust' used for secure transactions (you are just talking by e-mail, not doing finantial transactions). Therefore you could also exchange public key by mail, when you start corresponding with someone else.

Problem with your 'Solution'

fergal — Fri, 07 May 2004 08:19:57 +0000

Are you suggesting we would have blacklists for ISPs that don't keep their spam filters up to date?

How exactly would you keep that blacklist up to date? Wouldn't you have to aim spam probes at various ISPs and see if they block them. Good luck with that project.

Thrusted mail ?

dd9jn — Fri, 07 May 2004 08:14:26 +0000

Sorry, that does not work.

The problem is the definition of what makes up a "good signature". Yeah, we know about the Web of Trust and hierachical trust models but this will only work between people who know each other. This would lead to a system of closed groups like BBSes decades ago.

If you don't care about the validity of the signature, but merely check for the mathematical correctness, spammers will simply create new keys for each spam and sign it. No, there is no performance problem given the legions of zombies waiting for their evil masters.

82% of email is spam

neilbrown — Fri, 07 May 2004 03:13:44 +0000

Maybe the best way to beat this statistic is to simply generate more genuine mail. If everybody subscribed to a few high-volume mailing lists..... :-) (I must admit that linux-kernel is one reason that my spam ratio is quite low. "host -t MX cse.unsw.edu.au" might give you a hint at another).

But on a more serious side, I'm coming to the conclusion that the only long-term solution to SPAM must involve white-listing. i.e. I *only* accept mail from addresses (SPF-verifiable addresses) that I trust. This would require MUA support so that anyone I send mail to automatically gets white-listed, and things like that.

It would also mean that when people (e.g. companies) ask for your Email address, their would need to give your their email address in return (We will only send you mail from info@clever-company.com.) which you would have to white-list.

If people who you don't know want to send you mail, that has to be possible, but it also should be expensive (roughly the cost of a postage stamp or a 'phone call). This might involve finding a common correspondant to introduce you, or it might involve some "proof-of-cpu-power-spent" similar to HashCash, or it could even involve an exchange of money (e.g. I will read your mail if you can prove that you have bought a $1 e-postage stamp from World Vision). It could use any other challenge-response that a recipient is happy to impose on potential senders.

The big problem today with challenge-response is that you risk sending challenges to innocent third-parties whose address has been used inappropriately. This is where I think SPF really gives value. With SPF, I can tell if I can trust a return-address, and so I can know if it is safe to send a challenge.

If the MTA add an appropriate header with SPF status, this can all be done in the MUA.

This doesn't address the bandwidth/server-load problem. It isn't clear to me that there can be a better total solution to that than pushing the whitelist+challenge-response into the MTA (there are lots of partial solutions which can heurisically drop a lot of bad mail, but they will eventually cause the spammers to get smarter).

Solution

yodermk — Thu, 06 May 2004 23:46:18 +0000

> Those "brief notification" messages must either contain a little information, like a subject line, or people will have to click on them not knowing what they're going to get.

I'd suggest they contain a subject line and sender name. Any obvious spam would then not need to be transmitted over the Net at all, at least to users who care, which would take care of a lot right there!

> Either way, these will become the new method of delivering spam.

But remember that it would be much easier for a responsible ISP to stop this before the worst part of the problem than it is under SMTP. If an ISP detects spam, it deletes the message at the source before most people have downloaded it. If a customer's computer is spamming with its own IM2000 server, it could simply block the receiving port to that IP. Under SMTP, if it was caught at any point after the spam was sent, it's too late.

> If there's an open relay out there for "brief notification" messages, it would have to be closed so spammers don't send fake ones.

It would be impossible to send fake notification messages, because the end-user's box needs the IP of the server from which to fetch the mail!

Trust metrics are the FUSSP

raph — Thu, 06 May 2004 23:42:49 +0000

I've given a lot of though to e-mail spam as one of the possible applications of trust metrics. I've done a piss-poor job of self promotion, but if you look at the eigenvector-based diary ratings on Advogato, they're fabulously accurate, and the site itself runs a near-zero spam/abuse rate, in spite of the very open access policies, and the near total neglect of administrative duties on my part.

Unfortunately, trust metrics are difficult enough to understand (and I've done such a poor job evangelizing people so they're motivated to understand), that they haven't made much impact on the world.

I resemble a few of those check-boxes on the form posted above, but I do suspect that one of two things is true: the spam problem won't be solved in our lifetime because of a combination of technical difficulty and lack of willpower; or that the solution will use some form of peer certification to distinguish spammers from legitimate email correspondents. These days I'm more inclined to believe the former, but that's probably just my legendary bitterness shining through.

Why does everyone bring this up every time?

yodermk — Thu, 06 May 2004 23:41:04 +0000

There's a difference between "isn't going to work" and a few minor disadvantages. The way I see it, we have a choice:

1. Regulating all mail servers with government intevention
2. Using only centralized mail servers
3. Using certificates for mail servers, with each server needing to pay out the arse for a certificate
4. IM2000, which has a couple minor disadvantage for a few uses, and which can be worked around
5. 82% of our bandwidth being wasted on spam.

Which is least evil?

Problem with your 'Solution'

yodermk — Thu, 06 May 2004 23:37:43 +0000

Dan Bernstein made the proposal, not me. Granted, his page isn't that detailed. If you Google for "IM2000" you can find a few other more specific proposals.

> How is this different than a spam where the body is essentially just an image tag (or a redirect) to an advertisement on a remote server?

That would still end up as an email in someone's box. An IM2000 spam would have a subject, but may be unavailable upon mail check. Even though that type of spam is "short", it's still far better that it never reaches the recipient, to make spamming less profitable.

> Combine that with an ISP that just doesn't care about bandwidth usage

I do think it would be detectable, and any ISP that simply didn't care would be blacklisted.

> but B has a .forward file passing it along to bar.com. I've had forwarding chains of about four hops, and I suspect there are many people with longer chains. Does each hop have to cache the message?

Couple possibilities ... 1) each hop caches the message, 2) each hop notifies the originating ISP of the new recipient. That may have privacy concerns though.

> What about messages of the type "Our server will be down for maintenance on Thursday"?

That would be something to think about. This really does depend on mail servers being reliable. But, for the most part, they are.

Problem with your 'Solution'

yodermk — Thu, 06 May 2004 23:28:17 +0000

> Why do you think ISPs will keep their spam spotting software right up to date when even today we still encounter the trivially fixable problem of open relays?

Because they'd enter blacklists if they don't.

Problem with your 'Solution'

yodermk — Thu, 06 May 2004 23:26:31 +0000

1. These "unique" subjects are only to get around mail filters with SMTP. I don't think those type of filters will be as necessary with IM2000.

2. In the long run it's no real disadvantage over SMTP even if you *do* have to store a separate copy for each recipient. And, if joe@isp.com sends a million distinct messages, some flag would still be raised, and it would be easy for an admin to nuke all his messages.

Heck, you could set a space limit for outgoing mail ... maybe 100MB or so (up to the ISP of course). That should eliminate a million distinct messages.

You need to allow for people running there own servers

yodermk — Thu, 06 May 2004 23:21:26 +0000

Get a virtual server to run your mail server. More expensive than running it over a cable modem? Yeah, but you can afford it. And as another reply said, running SMTP over a cable modem isn't reliable anyway, since they tend to get blacklisted.

Problem with your 'Solution'

yodermk — Thu, 06 May 2004 23:19:08 +0000

Ok, by that definition of zombie, IM2000 would mostly stop it, if not totally. In most cases, it would have to provide the IP address of the compromised box in order to "work", and the box would have to run a mail server, and not have a firewall. That kind of thing would be easy to detect, but of course I suppose a few idiots will let it go on unnoticed. And a compromised box IP would likely be blacklisted quickly.

With the distributed blacklist (assuming it is administered in a trustworthy manner) would almost certainly make the cost to spam way too high.

As for mailing lists, the protocol changes somewhat drastically. See Bernstein's site, he talks about it.

> what happens if the senders server is down or unreachable when I want to read the message?

That is the second biggest problem with the system, after transition difficulties. Basically, someone (your ISP or you) needs to run a reliable mail server. That's a minor disadvantage perhaps, but keep in mind how it compares to 1200 spams a day.

> this idea would work in a world where everyone has pleanty of bandwidth and storage and everything is always up, but in the real world it's little better then a dream to toy with. some good may come of it, but this is nowhere near being a deployable system.

I really don't see how you can come to that conclusion. Of course it's deployable. It doesn't even use more bandwidth and storage in the long run. It will use much less bandwidth because of less spam. And storage would be less because it would only store one copy of an outgoing message to many users. Receiver-side storage would only be on his local computer, not on the server. (Ok, an IMAP-like mode could change that.)

Solution

melauer — Thu, 06 May 2004 19:19:02 +0000

>The solution, of course, is Dan Bernstein's IM2000. Read it. Be happy.
>
>http://cr.yp.to/im2000.html

I'm not convinced by this "solution". Those "brief notification" messages must either contain a little information, like a subject line, or people will have to click on them not knowing what they're going to get. Either way, these will become the new method of delivering spam. Then we'd be back where we started from. If a spammer runs their own mail server, we'd need to blacklist it. If there's an open relay out there for "brief notification" messages, it would have to be closed so spammers don't send fake ones. And so on.

The proposed method would cut down on the total bandwidth used by e-mail, though. That's kind of nice.

Incidentally, this system has basically already been implemented. Any number of private web forums use this. When one forum member sends a private message to another, the recipient gets a "brief notification" in the form on an e-mail. Then they login using a link provided in the e-mail and view the contents of the private message, which is of course stored on the forum's server.

Problem with your 'Solution'

shapr — Thu, 06 May 2004 19:08:20 +0000

These are actually part of the advantages of this system.

If a zombie sends notifications, it must be at the same hostname or IP for them to be picked up. that also means that blacklists become much more effective when a server is 'accountable' for its actions.

As for mailing lists, I think the list host would pick up and then host the mail, allowing for pushed spam. But then this isn't a silver bullet, just an improvement.

Right, once a mail is in the system it's trusted and pushed. The cost of storing and delivering mails is on the system, not the sender. If you move those costs to the sender, spam becomes less economically viable.

The essence is that you and I pay for spam in a push system like we have now, and we only pay for notifications in im2000.

One major advantage here is that those most likely to respond to spam, namely Internet newbies who check their mail once a week, are much less likely to get spam, since geeks like us will have gotten a spammed notify, and had time to do something about it (update blacklist, remove virus, etc).
That further cuts down on the economic advantages of spam.

Anyway, that's just my take on how to make spam less profitable for the spammers.

Thrusted mail ?

bockman — Thu, 06 May 2004 18:50:35 +0000

IF (but what a big IF it is) everyone started to use signature in their mail (I don't, actually), then one could think of mail clients that only accept mail with given signatures, or better yet we could instruct our ISP mail server to reject any e-mail addressed to us but not coming with a 'good' signature.

Mailing lists also could work this way, carrying both the mailing list signature and the signature of the poster.

This would work for 90% of users or more. Those needing to receive truly anonymous mail, could set special mailboxes for that.

Thinking of that, is anyone aware of any anti-spam filter that works on signatures? (Just curious: my personal small spam problems are easily solved by pressing 'd' in mutt some 8-10 times per day ).

Solution: Sender Policy Framework

Ross — Thu, 06 May 2004 16:28:57 +0000

Maybe. It depends on what type of authentication the server requires.
Assuming the worst case, no authentication, it would still be an
inprovement because the messages would really be from the domain they
appear to be from. This makes it easier to contact admins, implement
filters, etc.

Why does everyone bring this up every time?

Ross — Thu, 06 May 2004 16:24:50 +0000

I haven't read an article about spam in the last 6 months where there
wasn't a post extolling the virtues of DB's proposal. And every time
people point out major reasons why it isn't going to work. Those issues
go unresolved and yet when I read the next article...

Bandwidth

smoogen — Thu, 06 May 2004 16:10:17 +0000

If you think only about yourself then it is small. Here at a place with 12000 email accounts it actually goes over the amount that video etc are using according to our plots and graphs. I think that may be due to the fact that we can cache/proxy the video/web much better than 12000 slightly different emails to 12000 accounts

Problem with your 'Solution'

mmarsh — Thu, 06 May 2004 15:55:37 +0000

>First, I said to spend a few minutes thinking of a solution for any
>flaws before mentioning them, not spend a few minutes thinking of flaws...

That's an unfair request. If it's all been thought of and debated before, provide a URL or something. You're making a proposal, so it's up to you to defend it. A capsule description provides insufficient detail to support a position or adequately describe a system.

>Say a user of an ISP sends a million spams. All million (short)
>notification messages go out to the clients. BUT, someone at the ISP
>would almost certainly get some clue rather quickly that a spam was sent.
>*ZAP* and the message is gone from the ISP's server, and whoever hadn't
>checked their mail yet will be fortunate enough not to see it or waste
>bandwidth downloading it!

How is this different than a spam where the body is essentially just an image tag (or a redirect) to an advertisement on a remote server? Since notification latency will vary from recipient to recipient and not everyone checks email immediately on arrival, there might never be a noticeable spike. Combine that with an ISP that just doesn't care about bandwidth usage, and you're really no better off than when you started.

>Also, all mail will need to go through some permanantly connected mail
>server. Dialup certainly won't work. A cable modem/DSL mail server might
>work, but would possibly be less reliable. But virtual servers are cheap
>enough that anyone who really needs their own mail server should be able
>to afford one.

So email availability is dependant on the reliability of the sender? If the sender's ISP has to keep track of whether all of the recipients have retrieved the message, how does that affect forwarding? Will mail that gets forwarded be retrievable at all? Here I don't mean A sends a message to B who then forwards it to C. That's a no-brainer: B has to store it. I mean A sends a message to B at foo.com, but B has a .forward file passing it along to bar.com. I've had forwarding chains of about four hops, and I suspect there are many people with longer chains. Does each hop have to cache the message?

What about messages of the type "Our server will be down for maintenance on Thursday"? If a recipient doesn't check mail on Wednesday, he'll see that there's a message waiting from him, possibly flagged as important, but he won't be able to retrieve it until it's no longer relevant.

82% of email is spam

mmarsh — Thu, 06 May 2004 15:31:35 +0000

Microsoft's product quality, from a security standpoint, would likely improve more quickly if there were a demand for it from customers. I don't mean "supply and demand"-type demand, I mean "fix your damn software!"-type demand. The best way to do this is for ISPs to go back to TOSing users whose machines are compromised and used as zombies. It all goes back to good digital hygiene, and most users today have never had an Internet Health class. One of the reasons that FLOSS has a better security record is that the Unix community went through its baptism by fire a few decades ago, and now the bulk of the Unix-ish systems users _know_ that they want a secure system and _expect_ it from their providers. FLOSS is even more susceptible to this, since there's a lot less lock-in with it, making it easy for users to switch to alternatives if they're not satisfied.

Bandwidth

vondo — Thu, 06 May 2004 14:21:09 +0000

I've heard this bandwidth argument from the Cantor and Seigel (sp?) days, and never been convinced. Even if 82% of e-mail is spam, (and for me it's about 1/2 that), all the e-mail I get in a typical day probably doesn't add up to the amount of bandwidth I consume when I visit CNN's homepage. If I watch streaming video or audio, or download 5 ISOs for the latest version of Mandrake, e-mail is just a drop in the bucket.

Besides, spam e-mails are generally small. I have 13 in my filtered area now, the largest is 12 KB. People regularly send me 1MB files in the mail.

82% of email is spam

alspnost — Thu, 06 May 2004 13:59:41 +0000

Well, the 82% figure sounds about right for me. For a while, it seemed to be getting better, but it's now dramatically worse again. It's depressing, because SpamAssassin used to work brilliantly, but it just doesn't seem to be coping any more. Its success rate seems to have dropped from about 95% to around 50%.

At work, we use a 3-tiered strategy that's pretty effective. Firstly, we use RBL blacklists, and reject connections from malconfigured mail servers (eg with DNS problems); secondly, we use SpamAssassin on everything that reaches our queue; thirdly, we've put most of our users onto Thunderbird, and the built-in adaptive filtering is pretty good at mopping up anything that gets through SpamAssassin, once it's been trained.

For me personally, the biggest problem is accessing my (POP-based) mail via the webmail gateway when I'm away from home: with no spam filtering at that stage, I have to wade through 3 screenfuls of crap to find the 1 legitimate mail waiting for me....

82% of email is spam

rwmj — Thu, 06 May 2004 13:32:56 +0000

My standard response to spam (originally from here: http://yro.slashdot.org/comments.pl?sid=104138&cid=8868069 ) ...

--- quote:

Your post advocates a

( ) technical
(X) legislative
( ) market-based
( ) vigilante

approach to fighting spam. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)

( ) Spammers can easily use it to harvest email addresses
( ) Mailing lists and other legitimate email uses would be affected
( ) No one will be able to find the guy or collect the money
( ) It is defenseless against brute force attacks
( ) It will stop spam for two weeks and then we'll be stuck with it
( ) Users of email will not put up with it
( ) Microsoft will not put up with it
( ) The police will not put up with it
( ) Requires too much cooperation from spammers
(X) Requires immediate total cooperation from everybody at once
( ) Many email users cannot afford to lose business or alienate potential employers
( ) Spammers don't care about invalid addresses in their lists
( ) Anyone could anonymously destroy anyone else's career or business

Specifically, your plan fails to account for

( ) Laws expressly prohibiting it
(X) Lack of centrally controlling authority for email
(X) Open relays in foreign countries
( ) Ease of searching tiny alphanumeric address space of all email addresses
(X) Asshats
(X) Jurisdictional problems
( ) Unpopularity of weird new taxes
( ) Public reluctance to accept weird new forms of money
( ) Huge existing software investment in SMTP
( ) Susceptibility of protocols other than SMTP to attack
( ) Willingness of users to install OS patches received by email
( ) Armies of worm riddled broadband-connected Windows boxes
( ) Eternal arms race involved in all filtering approaches
( ) Extreme profitability of spam
( ) Joe jobs and/or identity theft
( ) Technically illiterate politicians
( ) Extreme stupidity on the part of people who do business with spammers
(X) Dishonesty on the part of spammers themselves
( ) Bandwidth costs that are unaffected by client filtering
( ) Outlook

and the following philosophical objections may also apply:

(X) Ideas similar to yours are easy to come up with, yet none have ever
been shown practical
( ) Any scheme based on opt-out is unacceptable
( ) SMTP headers should not be the subject of legislation
( ) Blacklists suck
( ) Whitelists suck
( ) We should be able to talk about Viagra without being censored
( ) Countermeasures should not involve wire fraud or credit card fraud
( ) Countermeasures should not involve sabotage of public networks
( ) Countermeasures must work if phased in gradually
( ) Sending email should be free
( ) Why should we have to trust you and your servers?
( ) Incompatiblity with open source or open source licenses
( ) Feel-good measures do nothing to solve the problem
( ) Temporary/one-time email addresses are cumbersome
( ) I don't want the government reading my email
( ) Killing them that way is not slow and painful enough

Furthermore, this is what I think about you:

(X) Sorry dude, but I don't think it would work.
( ) This is a stupid idea, and you're a stupid person for suggesting it.
( ) Nice try, assh0le! I'm going to find out where you live and burn your
house down!

Problem with your 'Solution'

fergal — Thu, 06 May 2004 13:20:44 +0000

Say a user of an ISP sends a million spams. All million (short) notification messages go out to the clients. BUT, someone at the ISP would almost certainly get some clue rather quickly that a spam was sent. *ZAP* and the message is gone from the ISP's server, and whoever hadn't checked their mail yet will be fortunate enough not to see it or waste bandwidth downloading it!

Your faith in the attentiveness of ISPs is a little misplaced. Why do you think ISPs will keep their spam spotting software right up to date when even today we still encounter the trivially fixable problem of open relays?

82% of email is spam

pizza — Thu, 06 May 2004 12:29:24 +0000

Perhaps the most promising solution I've seen is greylisting. The idea is that all "unknown" senders get sent a temporary "retry later" error, and legit MTAs will do just that, but spammers/worms/virii/etc won't bother, and will require a much more sophisticated "zombie" to send mail.

This has cut the spam I receive down to a small trickle from the couple hundred a day, to say nothing for the other users of the system I admin. (The only spam I get now is stuff sent via massive BCCs using a rather traditional e-mail client)

The nice thing about this is that it happens before mail is delivered, so you save on the bandwidth, storage, and cpu costs of post-delivery filtering.

http://projects.puremagic.com/greylisting/

There are multiple implementations of this technique now.