| |||||||||||||
|
| |||||||||||||
SolutionSolutionPosted May 6, 2004 2:57 UTC (Thu) by yodermk (subscriber, #3803)Parent article: 82% of email is spam The solution, of course, is Dan Bernstein's IM2000. Read it. Be happy. http://cr.yp.to/im2000.html I've posted this on Slashdot a few times, and whiners always come back with reasons why it won't work. I have, with very little difficulty, been able to think of a reasonable answer to every objection. So before you gripe here about why it won't work, please spend a few minutes yourself trying to think of a way to get around your supposed problem. The biggest problem by far is getting people to switch mail systems. But if we do nothing, spam will keep getting worse. IM2000, along with a good blacklist system (which would work MUCH better than blacklists with SMTP), should go a long way to stopping spam. No centralized mail servers, certificates, or government intervention required. Last I checked a few months ago, there were some good proposals swimming around on the Net. Need to look again for progress. Unfortunately, I don't know of an implementation yet, but it shouldn't be *that* hard. Once we get that, I bet people will start switching. Then as they drop SMTP accounts, others will need to switch to IM2000 to communicate with them. :)
(Log in to post comments)
Problem with your 'Solution' Posted May 6, 2004 3:42 UTC (Thu) by lakeland (subscriber, #1157) [Link] Ok, I've spent your few minutes thinking of flaws...A large percentage of spam is sent by zombies. Your solution would enable these zombies to continue spamming as before. I also wonder: if the spammer can afford the bandwidth of delivering each message to the ISP of the user, why it would be so much more expensive delivering them directly to the users. After all, the spammer could trivially manage to only store one copy of the message on their ISP. Though I admit I haven't thought this one through so carefully.
Problem with your 'Solution' Posted May 6, 2004 5:05 UTC (Thu) by yodermk (subscriber, #3803) [Link] First, I said to spend a few minutes thinking of a solution for any flaws before mentioning them, not spend a few minutes thinking of flaws...> A large percentage of spam is sent by zombies. Your solution would enable First, what do you mean by zombies? I know (roughly) the tactics of spammers, but I'm not familiar with which one(s) the term "zombie" applies to. But no, I don't believe you're correct that it would enable them to keep spamming as before. Here's how I understand it: Say a user of an ISP sends a million spams. All million (short) notification messages go out to the clients. BUT, someone at the ISP would almost certainly get some clue rather quickly that a spam was sent. *ZAP* and the message is gone from the ISP's server, and whoever hadn't checked their mail yet will be fortunate enough not to see it or waste bandwidth downloading it! Also, all mail will need to go through some permanantly connected mail server. Dialup certainly won't work. A cable modem/DSL mail server might work, but would possibly be less reliable. But virtual servers are cheap enough that anyone who really needs their own mail server should be able to afford one. Blacklisting as I understand it in the IM2000 model: If joe@isp.com sends a spam, his particular address is added to the blacklist, which would probably be a "push" type deal. Eventually it would trickle down to all the clients. (Ok, a "pull" system might be better, where it queries for every email, that's up for debate.) If, say, three or more users of a certain ISP or domain name or maybe subnet send spam, that entire network goes onto the blacklist. It probably won't stop 100% of spam, but I don't think our dear grumpy editor would be receiving 1200 spams a day with this protocol. I can also see how it could cause some minor inconvenience for people in some circumstances, say, those in the middle of nowhere with spotty satellite connections. But, I'm 100% sure that it can be made to work somehow, even if it looses a feature or two of this setup. Above all, please ask yourself this: Is any conceivable disadvantage to this system anywhere near as big a problem as 1200 spams a day??? > After all, the spammer could Yeah, think you need to read and think about this method more. :-) Actually one of the great things about this system is that it allows only one master copy to be stored on the ISP. For legitimate bulk mail, such as mailing lists, that's a rather significant benefit.
Problem with your 'Solution' Posted May 6, 2004 5:29 UTC (Thu) by dlang (✭ supporter ✭, #313) [Link] a zombie is a users machine that has been taken over by the bad guys.get a few thousand machines controlled by worms to each send a few dozen messages and you have a system that won't trigger any alarms at any one ISP. also what happens to mailing lists? I have a low bandwidth DSL line to my mail server (144k IDSL, the only thing I can get to my location) if I sendd a message to the linux-kernel mailing list do I now have several tens of thousands of people trying to download the message over my slow link? what happens if the senders server is down or unreachable when I want to read the message? this idea would work in a world where everyone has pleanty of bandwidth and storage and everything is always up, but in the real world it's little better then a dream to toy with. some good may come of it, but this is nowhere near being a deployable system.
Problem with your 'Solution' Posted May 6, 2004 8:13 UTC (Thu) by rjw (guest, #10415) [Link] Do you have an ISP?You get them to store the mail. Normal caching strategies also work. The mail you send would be stored in the same place the mail you receive is stored today.
You need to allow for people running there own servers Posted May 6, 2004 10:30 UTC (Thu) by alex (subscriber, #1355) [Link] You don't always want an ISP to process your mail. I run a mail server of my cable modem because I host several domains and mailing lists. Its cheaper to use my system at home where I have control rather than pay an ISP for stuff I can do myself.
You need to allow for people running there own servers Posted May 6, 2004 11:51 UTC (Thu) by copsewood (subscriber, #199) [Link] Unfortunately for this kind of mail system, one of the most effective means of blocking spam is to see if it comes directly from the kind of d IP address generally allocated to broadband users, by doing a PTR DNS lookup on the IP and looking for distinctive patters ( e.g. 4 numbers between . Addresses in this category are already getting into a number of DNS blacklists, and if more people get sufficiently fed up with spam to adopt this solution, legitimate mail systems hanging on this kind of connection without using a smart host are likely to become collateral damage. You would avoid this by configuring your ISPs outgoing SMTP server as your outgoing smart host, which would avoid this problem, and make better use of your limited bandwidth (only 1 copy of each list message uses your uplink).
You need to allow for people running there own servers Posted May 6, 2004 23:21 UTC (Thu) by yodermk (subscriber, #3803) [Link] Get a virtual server to run your mail server. More expensive than running it over a cable modem? Yeah, but you can afford it. And as another reply said, running SMTP over a cable modem isn't reliable anyway, since they tend to get blacklisted.
Problem with your 'Solution' Posted May 6, 2004 19:08 UTC (Thu) by shapr (subscriber, #9077) [Link] These are actually part of the advantages of this system.If a zombie sends notifications, it must be at the same hostname or IP for them to be picked up. that also means that blacklists become much more effective when a server is 'accountable' for its actions. As for mailing lists, I think the list host would pick up and then host the mail, allowing for pushed spam. But then this isn't a silver bullet, just an improvement. Right, once a mail is in the system it's trusted and pushed. The cost of storing and delivering mails is on the system, not the sender. If you move those costs to the sender, spam becomes less economically viable. The essence is that you and I pay for spam in a push system like we have now, and we only pay for notifications in im2000. One major advantage here is that those most likely to respond to spam, namely Internet newbies who check their mail once a week, are much less likely to get spam, since geeks like us will have gotten a spammed notify, and had time to do something about it (update blacklist, remove virus, etc). Anyway, that's just my take on how to make spam less profitable for the spammers.
Problem with your 'Solution' Posted May 6, 2004 23:19 UTC (Thu) by yodermk (subscriber, #3803) [Link] Ok, by that definition of zombie, IM2000 would mostly stop it, if not totally. In most cases, it would have to provide the IP address of the compromised box in order to "work", and the box would have to run a mail server, and not have a firewall. That kind of thing would be easy to detect, but of course I suppose a few idiots will let it go on unnoticed. And a compromised box IP would likely be blacklisted quickly.With the distributed blacklist (assuming it is administered in a trustworthy manner) would almost certainly make the cost to spam way too high. As for mailing lists, the protocol changes somewhat drastically. See Bernstein's site, he talks about it. > what happens if the senders server is down or unreachable when I want to read the message? That is the second biggest problem with the system, after transition difficulties. Basically, someone (your ISP or you) needs to run a reliable mail server. That's a minor disadvantage perhaps, but keep in mind how it compares to 1200 spams a day. > this idea would work in a world where everyone has pleanty of bandwidth and storage and everything is always up, but in the real world it's little better then a dream to toy with. some good may come of it, but this is nowhere near being a deployable system. I really don't see how you can come to that conclusion. Of course it's deployable. It doesn't even use more bandwidth and storage in the long run. It will use much less bandwidth because of less spam. And storage would be less because it would only store one copy of an outgoing message to many users. Receiver-side storage would only be on his local computer, not on the server. (Ok, an IMAP-like mode could change that.)
Problem with your 'Solution' Posted May 6, 2004 12:25 UTC (Thu) by pizza (subscriber, #46) [Link] >Say a user of an ISP sends a million spams. All million (short) notification messages go out to the clients. BUT, someone at the ISP would almost certainly get some clue rather quickly that a spam was sent. *ZAP* and the message is gone from the ISP's server, and whoever hadn't checked their mail yet will be fortunate enough not to see it or waste bandwidth downloading it!Except you are forgetting one very important point. The majority of these spams now contain unique subjects and/or bodies, be it via random dictionary words or whatever. So much to only one copy, eh?
Problem with your 'Solution' Posted May 6, 2004 23:26 UTC (Thu) by yodermk (subscriber, #3803) [Link] 1. These "unique" subjects are only to get around mail filters with SMTP. I don't think those type of filters will be as necessary with IM2000.2. In the long run it's no real disadvantage over SMTP even if you *do* have to store a separate copy for each recipient. And, if joe@isp.com sends a million distinct messages, some flag would still be raised, and it would be easy for an admin to nuke all his messages. Heck, you could set a space limit for outgoing mail ... maybe 100MB or so (up to the ISP of course). That should eliminate a million distinct messages.
Problem with your 'Solution' Posted May 6, 2004 13:20 UTC (Thu) by fergal (subscriber, #602) [Link] Say a user of an ISP sends a million spams. All million (short) notification messages go out to the clients. BUT, someone at the ISP would almost certainly get some clue rather quickly that a spam was sent. *ZAP* and the message is gone from the ISP's server, and whoever hadn't checked their mail yet will be fortunate enough not to see it or waste bandwidth downloading it!Your faith in the attentiveness of ISPs is a little misplaced. Why do you think ISPs will keep their spam spotting software right up to date when even today we still encounter the trivially fixable problem of open relays?
Problem with your 'Solution' Posted May 6, 2004 23:28 UTC (Thu) by yodermk (subscriber, #3803) [Link] > Why do you think ISPs will keep their spam spotting software right up to date when even today we still encounter the trivially fixable problem of open relays?Because they'd enter blacklists if they don't.
Problem with your 'Solution' Posted May 7, 2004 8:19 UTC (Fri) by fergal (subscriber, #602) [Link] Are you suggesting we would have blacklists for ISPs that don't keep their spam filters up to date?How exactly would you keep that blacklist up to date? Wouldn't you have to aim spam probes at various ISPs and see if they block them. Good luck with that project.
Problem with your 'Solution' Posted May 17, 2004 11:50 UTC (Mon) by coolian (guest, #14818) [Link] The original "IM2000" guy is obviously a hard-core democrat. I'm no Republican, but this "legislate my butt out of existence" and "trust the ISP not to go down" thing is WAY too much trust for me.
Problem with your 'Solution' Posted May 6, 2004 15:55 UTC (Thu) by mmarsh (subscriber, #17029) [Link] >First, I said to spend a few minutes thinking of a solution for any>flaws before mentioning them, not spend a few minutes thinking of flaws... That's an unfair request. If it's all been thought of and debated before, provide a URL or something. You're making a proposal, so it's up to you to defend it. A capsule description provides insufficient detail to support a position or adequately describe a system. >Say a user of an ISP sends a million spams. All million (short) How is this different than a spam where the body is essentially just an image tag (or a redirect) to an advertisement on a remote server? Since notification latency will vary from recipient to recipient and not everyone checks email immediately on arrival, there might never be a noticeable spike. Combine that with an ISP that just doesn't care about bandwidth usage, and you're really no better off than when you started. >Also, all mail will need to go through some permanantly connected mail So email availability is dependant on the reliability of the sender? If the sender's ISP has to keep track of whether all of the recipients have retrieved the message, how does that affect forwarding? Will mail that gets forwarded be retrievable at all? Here I don't mean A sends a message to B who then forwards it to C. That's a no-brainer: B has to store it. I mean A sends a message to B at foo.com, but B has a .forward file passing it along to bar.com. I've had forwarding chains of about four hops, and I suspect there are many people with longer chains. Does each hop have to cache the message? What about messages of the type "Our server will be down for maintenance on Thursday"? If a recipient doesn't check mail on Wednesday, he'll see that there's a message waiting from him, possibly flagged as important, but he won't be able to retrieve it until it's no longer relevant.
Problem with your 'Solution' Posted May 6, 2004 23:37 UTC (Thu) by yodermk (subscriber, #3803) [Link] Dan Bernstein made the proposal, not me. Granted, his page isn't that detailed. If you Google for "IM2000" you can find a few other more specific proposals.> How is this different than a spam where the body is essentially just an image tag (or a redirect) to an advertisement on a remote server? That would still end up as an email in someone's box. An IM2000 spam would have a subject, but may be unavailable upon mail check. Even though that type of spam is "short", it's still far better that it never reaches the recipient, to make spamming less profitable. > Combine that with an ISP that just doesn't care about bandwidth usage I do think it would be detectable, and any ISP that simply didn't care would be blacklisted. > but B has a .forward file passing it along to bar.com. I've had forwarding chains of about four hops, and I suspect there are many people with longer chains. Does each hop have to cache the message? Couple possibilities ... 1) each hop caches the message, 2) each hop notifies the originating ISP of the new recipient. That may have privacy concerns though. > What about messages of the type "Our server will be down for maintenance on Thursday"? That would be something to think about. This really does depend on mail servers being reliable. But, for the most part, they are.
Problem with your 'Solution' Posted May 17, 2004 11:52 UTC (Mon) by coolian (guest, #14818) [Link] " Dan Bernstein made the proposal, not me."But you suggested it and said it would "of course" be the answer. If you don't have any actual knowledge of it, then how could you know it's the answer? Jesus, you must be 12.
Problem with your 'Solution' Posted May 17, 2004 11:45 UTC (Mon) by coolian (guest, #14818) [Link] It's idiotic. You said the answer was "of course" this guy...It wouldn't work. Why? Because it wouldn't.I think the only thing that will work without legislation is filtering/whitelist/challenge-response.
Why does everyone bring this up every time? Posted May 6, 2004 16:24 UTC (Thu) by Ross (subscriber, #4065) [Link] I haven't read an article about spam in the last 6 months where therewasn't a post extolling the virtues of DB's proposal. And every time people point out major reasons why it isn't going to work. Those issues go unresolved and yet when I read the next article...
Why does everyone bring this up every time? Posted May 6, 2004 23:41 UTC (Thu) by yodermk (subscriber, #3803) [Link] There's a difference between "isn't going to work" and a few minor disadvantages. The way I see it, we have a choice:1. Regulating all mail servers with government intevention Which is least evil?
Why does everyone bring this up every time? Posted May 17, 2004 11:58 UTC (Mon) by coolian (guest, #14818) [Link] 1. Regulating all mail servers with government intevention>> Oh yeah, I don't think so. 2. Using only centralized mail servers >> Meaning what? 3. Using certificates for mail servers, with each server needing to pay out the arse for a certificate >> Why? Do you understand how certs work? 4. IM2000, which has a couple minor disadvantage for a few uses, and which can be worked around >> Minor disadvantages? The whole system not working is more than minor. There are MAJOR unresolved issues with it, that you have zero answers for. They can be worked around? By who? You?
>> This is the least evil of the ones you list. Sorry, but most would agree. I don't want people like you, who have zero answers, running my mail. Thanks, but no.
Solution Posted May 6, 2004 19:19 UTC (Thu) by melauer (guest, #2438) [Link] >The solution, of course, is Dan Bernstein's IM2000. Read it. Be happy.> >http://cr.yp.to/im2000.html I'm not convinced by this "solution". Those "brief notification" messages must either contain a little information, like a subject line, or people will have to click on them not knowing what they're going to get. Either way, these will become the new method of delivering spam. Then we'd be back where we started from. If a spammer runs their own mail server, we'd need to blacklist it. If there's an open relay out there for "brief notification" messages, it would have to be closed so spammers don't send fake ones. And so on. The proposed method would cut down on the total bandwidth used by e-mail, though. That's kind of nice. Incidentally, this system has basically already been implemented. Any number of private web forums use this. When one forum member sends a private message to another, the recipient gets a "brief notification" in the form on an e-mail. Then they login using a link provided in the e-mail and view the contents of the private message, which is of course stored on the forum's server.
Solution Posted May 6, 2004 23:46 UTC (Thu) by yodermk (subscriber, #3803) [Link] > Those "brief notification" messages must either contain a little information, like a subject line, or people will have to click on them not knowing what they're going to get.I'd suggest they contain a subject line and sender name. Any obvious spam would then not need to be transmitted over the Net at all, at least to users who care, which would take care of a lot right there! > Either way, these will become the new method of delivering spam. But remember that it would be much easier for a responsible ISP to stop this before the worst part of the problem than it is under SMTP. If an ISP detects spam, it deletes the message at the source before most people have downloaded it. If a customer's computer is spamming with its own IM2000 server, it could simply block the receiving port to that IP. Under SMTP, if it was caught at any point after the spam was sent, it's too late. > If there's an open relay out there for "brief notification" messages, it would have to be closed so spammers don't send fake ones. It would be impossible to send fake notification messages, because the end-user's box needs the IP of the server from which to fetch the mail!
Solution Posted May 7, 2004 21:02 UTC (Fri) by brouhaha (subscriber, #1698) [Link] OK, so I've spent a few minutes thinking about problems with IM2000 in terms of spam avoidance, and trying to think up a solution.The problem I see is that this doesn't address spam at all. It just changes the delivery mechanism. Today, the spammers bombard your MTA (e.g., Sendmail, Postfix, or Exchange) with spam. Your MTA uses a lot of resources (CPU, memory, disk) dealing with this. Depending on the MTA and configuration, it may or may not be able to do some filtering to avoid actually storing some of the spam in your mail queue. Then you check your inbox using an MUA (mail client such as Evolution, Mozilla Mail, Eudora, or Outlook). The MUA may also do some filtering, but at the very least has to inspect the headers of each message the MTA has received for you. With IM2000, the spammer's machine doesn't directly send the spam to you. Instead, it tells your MTA (or whatever the IM2000 equivalent of an MTA is) that there is mail waiting for you on the spammer's machine. So now instead of getting 1200 spam emails a day, you get 1200 email waiting notifications from random machines all over the internet, many of which are probably zombies. Ultimately your software doesn't have any better way to tell which are spam than in today's architecture; it will simply have to poll each of the 1200 sender mail servers to get the mail headers and try to filter them. This has *perhaps* reduced the internet backbone bandwidth consumed somewhat, if the filtering can be done with inspection of headers only and not the full message body, but it has not solved the spam problem. In fact, given that the IM2000 model provides for the sender's mail server to repeat the "mail available" notifications to the receiver, it may not even affect the bandwidth consumption. And it certainly does not improve the bandwidth consumption on the user's local link. As far as I can see, IM2000 is a solution to a non-problem. It has attempted to solve the problem of reducing the amount of disk space the recipient needs to store the email, but disk space is the least significant problem associated with spam. IM2000 appears to solve the "mailing list problem", except that there isn't really a "mailing list problem" either.
|
|
||||||||||||