Weekly Edition
Return to the Security page
| |||||||||||||
Spam-proofing the mail system
One of the major problems with Simple Mail Transport Protocol (SMTP) is that
it allows email senders to forge information about who they are. The lack
of sender authentication allows unscrupulous users to send email that
appears to come from a domain other than where it truly originates. Spammers
use this 'feature' to disguise their email and to cause any bounces or
responses to be handled by someone else.
There are several proposals for combating this problem that are currently being worked on; we will describe some of them below. Before we do, however, a bit of a review on how SMTP currently works is in order. When a host wants to send mail, it looks at the DNS Mail Exchanger (MX) record for the destination domain and makes a connection to the host that is indicated. The sending host identifies itself, the email address of the sender of the message, and the address of the recipient of the message to the destination host via SMTP messages. This is known as the envelope of the message and, if it is accepted by the destination host, the sender proceeds to send the body of the message. The message body contains RFC822 headers (From:, To:, Subject:, etc.) that are used by Mail User Agents (MUAs) to identify the message to users. SMTP servers traditionally do not do any kind of checking on the envelope data they receive, believing that other hosts will not deceive them. Any part of the envelope and RFC822 headers can be forged (except, of course, the recipient in the envelope). Obviously, SMTP has its roots in a much friendlier Internet where trusting other hosts was the norm. Recently, Yahoo announced an initiative that is meant to combat spam called Domain Keys. Technical details are somewhat sketchy, but the basic idea is that the DNS records for a domain would include a public key. Email that originates from that domain would use the corresponding private key to encrypt some data (it is not clear exactly what, but a cryptographic hash of the message contents would seem an obvious choice) that would be placed in an email header. Mail Transfer Agents (MTAs) that received the message could decrypt it using the public key in the DNS record and if the decrypted value was correct, the MTA would know that the message originated from the domain that was claimed. Sender Permitted From (SPF) is a proposal to add information to the DNS records for a domain specifying what machines legitimately send email for that domain. This information is the reverse of the MX record, rather than specifying hosts that receive email for the domain, they specify hosts that send it. This would allow MTAs to check the IP address of the sender and the host name provided in the SMTP envelope along with the SPF information in DNS to determine whether that IP address is a legitimate sender for that domain. (LWN covered SPF in more detail last October). The Trusted Email Open Standard (TEOS) is a wide-ranging proposal that has three implementation steps and would eventually allow for third-party certification of email messages as coming from a trusted source. This scheme would operate in some ways like the SSL Certificate Signing Authorities; an MTA could verify that a message came from a source trusted by the third party. The first step that TEOS proposes is similar to the Domain Keys proposal; it would provide a way to authenticate email senders. The second stage adds the ability for senders to make assertions about the contents of the email, saying, for example, that it contains advertising or an opt-in mailing, or that the sender and recipient have a business or family relationship. Users would be able to filter the mail based on the assertions (or lack thereof). If the sender incorrectly categorizes a message, the authentication will not allow the blame to be shifted elsewhere, providing a large incentive to be truthful when making the assertions. The Tripoli proposal envisions an entirely new email infrastructure, at first running in parallel with the current SMTP-based system, but eventually supplanting it. The underlying principle is that the receiver of email should have greater control than any of the other parties involved, including the sender, ISPs that transmit the email, or governments. The system proposed would eventually have end-to-end encryption for all email traffic. Associated with each email would be a cryptographic token that is certified by a third-party to a particular level of authentication; email recipients could then choose the level of authentication that they wish to require and can reject any messages that fall below this standard. These proposals are a testament to just how problematic and widespread the spam problem has become. The scope of some of these proposals, particularly TEOS and Tripoli, show how far some people are willing to go to try and combat it. Adding third-parties to email sending could have a number of security and privacy concerns and would almost certainly add a cost to sending email. If that cost breaks the current economic model of spamming, however, it may be effective, but it would also impact lots of other bulk email uses today (legitimate mailing list traffic, opt-in newsletters and the like). On the other hand, Domain Keys and SPF could be circumvented by spammers willing to create throwaway domains that conform to the requirements. Once the domains are identified as spam domains, they can be added to blacklists, of course, but there have been any number of problems with that particular solution as well. Authenticating senders might help track down spammers, but until the risk of detection and the cost of conviction are greatly increased, it is likely to only slow things down and perhaps not by much. It should be interesting to watch the battle over our email inboxes play out over the next few years. It may well be that one or more of these proposals is adopted (or some combination of them) by a significant portion of email users and providers. Unfortunately, in the meantime, less technical email users are suffering at the hands of the spammers to the point where email is no longer a useful communications medium for many. (Log in to post comments)
Spam-proofing the mail system Posted Dec 18, 2003 4:48 UTC (Thu) by freemars (subscriber, #4235) [Link] The systems described concentrate on identifying the source of the email. I think a more direct approach would be an extension to SMTP requiring a micropayent to the recipient. At some point as two computers are negotiating to set up a SMTP session the recipient computer can request a micropayment. If the originating computer doesn't comply it gets a brush-off message and the connection is closed. If the originating computer sends a micropayment token the receiving computer immediately takes it to the bank. If the bank OKs the payment the email body is accepted. Otherwise the connection closes with a brush-off message. In the traditional ISP model this will take place at the ISP level. The customer will eventually log in and retrieve mail. It is vitally important the micropayment get passed on to the final recipient; that's the only person who can tell if an email is legetimate or if it's spam. When you log into your account and get your mail, it will come with dime-sized credits attached. As you read your mail, you get to decide: if a message is spam, you keep the dime. If it is legetimate, you return a micropayment to the sender, and probably add the sender to a whitelist. (Don't want to refund the payment? You run the risk of getting labeled a deadbeat, and never getting email from anyone.) Whitelists and blacklists for each email address would be kept at the ISP, and are essential to running this system efficiently. If someone is on your whitelist, the request for a payment portion of the SMTP session never takes place, saving the bank overhead for two transactions (the payment and the refund). Whitelists allow good things like mailing lists to continue operating without a major financial hit. If a sender is on your blacklist your ISP will happily accept the micropayment and message, but silently disgard the latter.
This plan needs several things to work:
Anonymous e-mail? Posted Dec 18, 2003 5:38 UTC (Thu) by lovelace (guest, #278) [Link] Doesn't your system preclude the possibility of sending anonymous e-mail (since every e-mail account must be hooked up to a bank account)? And what about people who don't have a bank account? While I hate spam as much as the next person, it seems to me your system gives away too much.
Anonymous e-mail? Posted Dec 18, 2003 14:53 UTC (Thu) by tonnesen (guest, #3589) [Link] Any of these proposals would limit the possibility of sending anonymous e-mails. What would we lose if, say, the domain keys proposal was limited? A whistle blower couldn't send an e-mail to a newspaper without worrying that he could be identified later? I'm sure there are many other instances where anonymous e-mail is desirable. How is anonymous e-mailing done now? You can't just forge the sender. The originating IP address will still show in the headers and be traceable.Should anonymous e-mailing be a pay-as-you-go system? You can send mail from your account at wekeepyouhidden.com which you pay $20/month for. What about the "free" e-mail systems like hotmail, yahoomail, etc. They'll presumably get domain keys, and continue allowing spamers to send spam through them. What percentage of current spam is sent through these systems now? They wouldn't solve the anonymity problem because, as far as I know, they all attach the originating IP address to the messages they send. Should anonymous e-mailing be a pay-as-you-go system? You can send mail from your account at wekeepyouhidden.com which you pay $20/month for. My gut tells me this is the only way to get true anonymity now (short of cracking computers and sending from there, but I can't exactly endorse breaking the law as a solution). My feeling is that blacklists like SpamCop et al would be significantly more effective under the domain keys proposal. Blacklist systems could easily link all messages from one domain together and aggregate the complaints they receive about that domain.
Anonymous e-mail? Posted Dec 18, 2003 18:06 UTC (Thu) by iabervon (subscriber, #722) [Link] Anonymous email would still be possible under the SPF system, at least,by simply permitting mail from hosts that don't claim to be anyone in particular. Such email wouldn't be verified at all, since it didn't claim to be anything in particular. Most people would just throw it away, but anonymous tip lines wouldn't (since they wouldn't get anything else, obviously), and spammers wouldn't tend to spam such addresses anyway (do spammers really want to reach investigative reporters at their work?). Furthermore, if people are actually using an anonymizing network, the email will come out with a perfectly valid and legitimate address which simply can't be tracked back to the sender, and is authenticated as being an anonymous message from this remailer. These systems are generally pseudonymous, in any case, so that the original sender can send additional messages authentically, and these systems would merely authenticate the pseudonym. Again, this mail would be filtered by most people, but would get through to people who intend to pick up anonymous tips.
Yes, anonymous e-mail Posted Dec 18, 2003 16:03 UTC (Thu) by freemars (subscriber, #4235) [Link] Doesn't your system preclude the possibility of sending anonymous e-mail (since every e-mail account must be hooked up to a bank account)? And what about people who don't have a bank account? I believe in the value of anonymous email, and believe this plan supports it better than the ones suggsted above. While it's true the ISP must be hooked up with a bank an individual sender needn't be. Imagine a system where you go to a convenience store and buy a CD with 200 email stamps worth a dime each. The CD might cost $25 -- $20 for the value of the stamps, $2.50 markup for the store, $2.00 for the bank, and $0.50 for the CD, packaging & distribution. You could pay cash for the CD and take it to your local Starbucks/Internet Cafe to send emails. Since you're trying to be annonymous, you forge your return address, so you never get your rebate. It either gets lost in cyberspace (and never cashed in; the bank thanks you) or goes to your favorite charity. A poor person might have to scrape to come up with the $25 purchase price, but doesn't need to jump through all the hoops of getting a bank account. If most of your mail goes to people who know you -- and have you on their whitelists -- your initial supply of stamps will last a long time.
Spam-proofing the mail system Posted Dec 18, 2003 5:51 UTC (Thu) by sfeam (subscriber, #2841) [Link] Your proposal sounds counterproductive. As you describe it, the spammer only gets charged if I read the spam. The spammer might well be OK with that, but my goal is to never see the spam in the first place. The more successful people become at filtering, the less penalty is applied to the spammer. Surely that is not what you intended.
Spam-proofing the mail system Posted Dec 18, 2003 10:16 UTC (Thu) by ekj (subscriber, #1524) [Link] This is true, on the surface.But the thing is, the requirement alone to attact a dime to every outgoing spam is likely more than enough to make spamming a non-issue. If you have to attach say $0.10 to every outgoing message, then even if 90% of the spam is never read so you eventually get the money back (I assume there'd be a timeout of some sort), you'd still be paying $0.01 a message. Sending a million spam-emails at this rate would cost you $10K. Personally, I think this is low. Because the economics of reading spam would completely change. I would, for example, happily read and delete spam if I knew it would gain me, and cost the spammer, $0.10 for each one. Why, with my current load, and assuming I could process one spam a second, I would make around $5 a day, and would be spending less than 10 minutes to do so. In practice offcourse, the chanse of any spammer paying me, even a single cent, for me to read their message is miniscule.
Spam-proofing the mail system Posted Dec 19, 2003 1:58 UTC (Fri) by giraffedata (subscriber, #1954) [Link] Hear hear. Between my automatic deleters and my morning manual deletion ritual, I determine to be spam and delete about 200 messages a day. The morning ritual takes about two minutes. Programming the automatic deleters, an hour a month. Well worth $20. Of course, most of those spams would not be in my mailbox at all if they had to have $.10 attached. Based on the fact that I never buy anything and never fall for scams, I doubt I'd get more than a few spams a week.
Spam-proofing the mail system Posted Dec 18, 2003 14:52 UTC (Thu) by freemars (subscriber, #4235) [Link] The more successful people become at filtering, the less penalty is applied to the spammer. Surely that is not what you intended. Indeed that's not what I intended. Filtering spam would have the same effect as what I described as blacklisting; accept the payment for the spam but throw away the message unread. It costs them a dime but they have no idea if the message was ever seen by human eyeballs.
Spam-proofing the mail system Posted Dec 19, 2003 0:15 UTC (Fri) by Ross (guest, #4065) [Link] Who would fund mailing lists? I'm sure you couldn't depend on every usersubscribed to a list to return the funds (no to mention the list admin is out those funds until they are returned? This would be a huge problem for large mailing lists. Who would manage the micropayments? Would we all get PayPal (shudder)
Spam-proofing the mail system Posted Dec 19, 2003 2:00 UTC (Fri) by giraffedata (subscriber, #1954) [Link] With a decent micropayment system (as postulated), it would be easy to charge minuscule subscription fees for mailing lists, which would cover the occasional non-refund and the transaction fees.
Spam-proofing the mail system Posted Dec 19, 2003 4:50 UTC (Fri) by brouhaha (subscriber, #1698) [Link] Who would fund mailing lists?In a micropayment email system, the recipient sets the amount of the micropayment, which can vary depending on the sender. Most people would whitelist their friends and charge nothing. Presumably they would do this for mailing lists as well, and if the mailing list attempted to deliver to a recipient demanding a payment, would simply drop that recipient from the list. That leaves open the possibility of a spammer masquerading as a list server, but other anti-spam techniques such as SPF easily thwart such misrepesentation. It also would be possible to set the amount fairly high, but refund it in the case of unsolicited but non-spam email. For instance, I'd probably set my software to charge at least $0.25, and perhaps as much as $1, for unsolicited email from non-whitelisted senders. But for non-spam email, for instance people inquiring about free software I've written, I would refund it. Probably a fee as low as a penny would discourage a lot of spam, but having the fee higher would serve to also discourage email from students that want me to do their EE or CS homework for them. (I get such email routinely as the result of having a web page about PIC microcontrollers, despite the web page clearly stating that I don't have time to offer free help.)
Spam-proofing the mail system Posted Dec 19, 2003 13:46 UTC (Fri) by freemars (subscriber, #4235) [Link] Who would fund mailing lists? I'm sure you couldn't depend on every user subscribed to a list to return the funds (no to mention the list admin is out those funds until they are returned? This would be a huge problem for large mailing lists. Mailing lists would depend on whitelists. (This means that any new software supporting micropayments must support whitelists also. Because the whitelist would be stored at the ISP both the userland mail-reading software and the ISP SMTP software would have to support whitelists.) Mailing lists could operate much like current "double opt-in" lists do today... when you subscribe the list sends a 'welcome' email which asks you to whitelist the mailing list address. When you do (sending a rebate to the list) the software notices and adds you to the list. To remove yourself from a list, simply remove it from your whitelist. When the list software gets a request for a micropayment (not whitelisted) it declines to pay and removes the recipient from the list of subscribers. Who would manage the micropayments? Would we all get PayPal (shudder) accounts? What prevents micropayments from being stolen by ISPs or someone hijacking accounts?
You've found the potential showstoppers. Ideally, there would be a few banks in the business so there would be some pressure on them to hold rates down. It will be up to individual ISPs to decide if they will accept payments from Fred's Bank (where Fred's Bank might be located in some unsavory nation). I would expect ISPs would quickly settle on accepting payments only from member banks of some International Collaboration of Cooperative Micropayment Banks, all of which agree to the following standards: I'm hoping a few companies -- probably current-day banks -- would step up and offer such a micropayment system. If all email were to convert to a micropayment system there would be enough use to justify setting up the kind of heavily automated system that would be needed to be affordable. Spammers will no doubt try to hijack computers which have access to email 'stamps'. Cautious people will not store large numbers of 'stamps' on their computers. Prediction: someone will decide it's a really cool idea to keep the user's credit card information in the computer so the software can automatically buy extra email 'stamps' from the International Bank of Microsoft as needed -- some people will get burned. "Bad" ISPs stealing people's micropayments ... you ask tough questions ... perhaps the legislation protecting ISPs from anti-trust lawsuits should only apply to those which pass on 100% of the micropayments to their customers. "Bad" ISPs could also mess with customer whitelists, but I don't see what they would gain from doing that.
Spam-proofing the mail system Posted Dec 28, 2003 20:07 UTC (Sun) by rogblake (guest, #18258) [Link] You seem to be forgetting that not everyone has a bank account, or a credit card. It looks like your solution to the spam problem would have the side effect of not allowing such people to use email at all!
What about cracked machines? Posted Dec 18, 2003 11:41 UTC (Thu) by NAR (subscriber, #1313) [Link] There were some worms lately than turned the cracked computers into spam-sending machines. Would any of these proposals give a solution for this case? Or would it only make things worse (imagine that a home PC is cracked, used to send millions of spams and the owner has to pay for these spams)?
Yes, mostly Posted Dec 19, 2003 0:30 UTC (Fri) by Ross (guest, #4065) [Link] The cracked machines would in all likelyhood not be allowed to send emailfor the domain they are in (probably an ISP's domain). Normal use would require sending the messages through the ISP's outgoing mail server where the user could be authenticated. But spammers are determined people. They could develop better hijacking
What about cracked machines? Posted Dec 19, 2003 16:24 UTC (Fri) by giraffedata (subscriber, #1954) [Link] That's easy. Knowing how complex computers are, no one would tolerate a micropayment system where the computer can automatically spend away his entire bank account. The micropayment system would, therefore, have built in limits. I might tell it, for example, not to let me spend more than $5 a day on sending email unless I log in and type a password or something to raise the limit.Back in the days when computer users were charged by the resource, the computers had such budgets for the same reason: I could say, "Don't let me use more than $5 worth of disk space." This would eliminate the incentive for crackers to hijack computers, and also create incentive for people to harden and repair their computers.
DJB's Internet Mail 2000 Posted Dec 18, 2003 22:55 UTC (Thu) by jhohm (guest, #7225) [Link] I'd like to remind everyone of D. J. Bernstein's Internet Mail 2000 concept, which puts the burden for message storage on the sender instead of the recipient. This would handily turn around the economics of mass mailing, without micropayments or public-key cryptography.
DJB's Internet Mail 2000 Posted Dec 19, 2003 0:25 UTC (Fri) by Ross (guest, #4065) [Link] There would be a few problems with that. For one, what keeps people fromchanging messages after you read them, or even deleting them before you read them? What happens when their site is down? What happens when that ISP goes out of business? That's a different type of communication that what we know of as email. Making local copies on your ISP's mail server would eleminate many of those problems but that kind of defeats the purpose. And worst of all spammers would still be a nuisance... they would not take
DJB's Internet Mail 2000 Posted Dec 25, 2003 21:45 UTC (Thu) by jbayko (guest, #3493) [Link] There would be a few problems with that. For one, what keeps people from changing messages after you read themYou could save them locally, as now (when messages are stored on an ISP email server, vs. stored on the sending server). or even deleting them before you read them? I think it's fair for the sender to change their mind. Your email client would simply compare the tag with what's on the server, notice the message is gone, and not even show it to you (unless you have it configured to show you deleted messages) What happens when their site is down? I'm sure you could configure your software to retry over a period of time, or a number of attempts - the same as when you send email, and the receiving server is down. What happens when that ISP goes out of business? What happens when your ISP goes out of business before you download your email? As with that case, the sender has the option of re-sending (and they're more likely to know things went wrong on their end than on your end). That's a different type of communication that what we know of as email. Making local copies on your ISP's mail server would eleminate many of those problems but that kind of defeats the purpose. Not really - the purpose is to make the sender incur a reasonable, but unavoidable and deterring cost. [...] But the bigger problem with spam is the distraction and annoyance to the users when they checking their email. Now if they want to do filtering they are required to connect to the spammer's server to view the content. Not necessarily:
This lets the spammer know what addresses are alive and being used. And vice-versa. Any automated filtering will be slowed to a crawl due to all the outgoing connections. This could be a problem. But email is typically text, and takes less bandwidth than the images that are part of web pages, and web traffic isn't too bad. I can even imagine situations where spammers could use forged headers to cause mail servers to perform denial of service attacks against third parties. This could be a problem, if headers were not encrypted using some sort of strong public key. And finally there would be an annoying amount of latency between selecting a messages and reading the contents. Not any more than messages stored on an ISP - they (valid messages) can be downloaded while you're looking at the subject lines.
DJB's Internet Mail 2000 Posted Dec 19, 2003 16:40 UTC (Fri) by giraffedata (subscriber, #1954) [Link] In this proposal, the text of an email is stored at the sending end, and the recipients are all sent a brief message telling them to go to the sending system and request the text.The only problem this seems to solve is one where a recipient has a small email storage quota and spammers keep filling it up. This does not turn the economics around -- the spammer would presumably store one copy of the spam, at insignificant cost, but the recipients would still bear the expense of reviewing all the spam. And it happens to be what we have today much of the time. A spammer sends a trivially small email telling you to go to his web site and read an ad.
DJB's Internet Mail 2000 Posted Dec 25, 2003 21:56 UTC (Thu) by jbayko (guest, #3493) [Link] This does not turn the economics around -- the spammer would presumably store one copy of the spam, at insignificant cost, but the recipients would still bear the expense of reviewing all the spam. It's not a matter of storage space, and only partially of bandwidth, but mainly time. A spammer would need to keep the spam available for however long the last recipient they want to read it will take - could be a month. Not only that, but will need to keep off any sort of easily- checked blacklist. Good ISPs may take a day or two to cut off service (thus saving millions of recipients from even seeing the spam header). What would make it harder, there would need to be some reasonable authentication or encryption used, to prevent unauthorized users from eading each other's email. The spammer's system would have to keep track of all users who the email was sent to, to handle all this authentication or encryption/decryption (there are many possible variations for security).
|
|||||||||||||