Blocking forgeries and spam with SPF [LWN.net]

Anybody who has spent any amount of time dealing with spam (i.e. just about anybody with an email address) knows that a great deal of it comes with forged return addresses. Email worms attacking certain proprietary systems also have a habit of generating mail with fake return addresses. If there were a way to filter out mail with bogus sender addresses, a great deal of spam and other unpleasant mail could be automatically removed from our mailboxes.

A technique called "Sender Permitted From" (SPF) is being readied to attempt to make this sort of filtering possible. Those looking for details can find them in the draft RFC, but the core concept is simple: the DNS database for each domain should be augmented with information on which systems are authorized to originate email for that domain. This information is added as a DNS "text" record, so no changes to the DNS protocol are required.

So, for example, the DNS zone file for a domain which never, ever sends mail could be made SPF-compliant by adding one line:

    example.com  IN  TXT  "v=spf1 default=deny"

The "v=spf1" portion indicates that this is an SPF version 1 entry, and the rest says to deny all mail from that domain.

In most interesting cases, however, people will want to be able to send mail from a domain. So the SPF entry must be modified to tell mail recipients which systems can send mail for the domain. The simplest way of doing that, perhaps, is to simply state that the domain's MX servers can originate mail:

    example.com  IN  TXT "v=spf1 mx default=deny"

There are, of course, many ways of specifying, in great detail, exactly which systems can legitimately send mail for the domain of interest; see the RFC for details.

None of this will work until receiving systems perform SPF tests, of course. One of the nice features of SPF is that the check can be done before the body of a message is received. If the message will be filtered, this filtering can be done at the SMTP level and a meaningful message returned to the sender - if, indeed, there is a real sender. Patches exist for a number of MTAs now; expect more as the SPF specification solidifies. There are also plans to add SPF support in other places; apparently SpamAssassin 2.70 will support it, for example.

SPF certainly will not solve the spam problem; spammers will just use domains that lack SPF information, open relays, or throwaway domains of their own. But it does place one more obstacle in their way, and will doubtless reduce the flow somewhat. The real value of SPF may be in its ability to make the forgery of email more difficult. In a fully SPF-compliant world, Linux users would no longer be flooded with "virus notifications" every time a new worm starts digging through peoples' address books. A dedicated attacker would probably still be able to forge email from a specific victim, but the days of easy, casual forgery would, one hopes, be over. And that is worth something.

to post comments

Blocking forgeries and spam with SPF

Posted Oct 23, 2003 3:05 UTC (Thu) by roelofs (guest, #2599) [Link] (2 responses)

Does any of this have anything to do with the "From: " line, or is it all limited to the "From " line (which reflects the "actual sender" as seen by the MTA, I think, regardless of what I do in the user agent)?

In other words, will pobox.com, ieee.org, acm.org, etc., all have to include an SPF wildcard record--rendering the whole system fairly pointless--or can I continue to happily "forge" my "From: " address without any changes on their side, safe in the knowledge that all this SPF stuff will only affect the "From " address, which always points to my current ISP no matter what I do?

Sorry for the imprecise From:/From description, but I'm not exactly a mail-system guru... I just want my "public" address to be the one over which I have long-term control, not one that disappears as soon as I move to another state.

Greg Roelofs

Blocking forgeries and spam with SPF

Posted Oct 23, 2003 8:33 UTC (Thu) by sjdg (guest, #5384) [Link] (1 responses)

According to the RFC, it works on the envelope header ("From "), not the embedded "From:" header. Your benign faking should be ok.

Simon.

Blocking forgeries and spam with SPF

Posted Oct 23, 2003 15:43 UTC (Thu) by zone (guest, #3633) [Link]

Umm, no. Practically half the site is devoted to answering the "From:" vs. "From " question, and addressing the problems SPF will create with regards to that. In summary, if your users send mail with your domain, but don't always use your mail server, you need to specify "softdeny", at least until you can transition them to using your outgoing mail server exclusively. After all of your users are using your mail server exclusively, then you can set the default SPF to "deny".

This is really a very large roadblock to SPF adoption for certain domains. Convincing thousands of your users to change their habits and not to use their mail server at their ISP, at work, at the cybercafe, at a friend's house, etc., but instead to always remember to specify the domain's real mail server, is not an easy task. It will be made substantially easier if MUA developers get on the SPF bandwagon and steer the user into sending through the proper mail server, but it will also take a concerted effort by ISPs and employers to educate their users.

In an ideal SPF world where the Global Sunrise Date of April 12, 2004 were to actually occur (i.e. all legitimate domains have SPF records and all are set to default=deny), there would be no more "vanity domains" in the traditional sense. You'll either have to send through the domain's mail server or use the new Sender Rewriting Scheme or possibly some other contrived way. The SPF authors are apparently relying on sites that forward mail for their users to take the initiative to implement SPF-compliant solutions and notify their users.

From what I can tell, SPF adoption can and probably will happen, but it's going to take a lot more than simple DNS records and spam filter adjustments. It's going to take cooperation from ISPs, corporate network administrators, mail agent developers, and unfortunately, end users. Until a significant portion of emails are sent from domains with default=deny, spam filters will be forced to take a non-aggressive SPF stance, and rely on traditional techniques when analyzing emails from domains with default=softdeny or with no SPF information at all.

Blocking forgeries and spam with SPF

Posted Oct 23, 2003 9:53 UTC (Thu) by rwmj (subscriber, #5474) [Link]

Cool - I just updated my domain with this extra information:

$ dig annexia.org txt

[...]

;; ANSWER SECTION:
annexia.org.            604800  IN      TXT     "v=spf1 mx default=deny"
[...]

Everyone else should do the same. Hopefully the final RFC won't be substantially different from the draft. Rich.

Blocking forgeries and spam with SPF

Posted Oct 23, 2003 14:09 UTC (Thu) by Luyseyal (guest, #15693) [Link]

It should definitely help stem the tide of virus-based spam.
-l

Blocking forgeries and spam with SPF

Posted Oct 23, 2003 16:03 UTC (Thu) by copsewood (subscriber, #199) [Link] (1 responses)

My first thoughts as an author of a minor anti-spam program and as a victim of spam pretending to come from my own domain , were that an approach like this could not work because of a catch 22: people won't implement blocks on this basis if there are many false positives, and if there are few blockers, DNS admins have less incentive to update their domains.

Further thoughts suggest that blocks probably can be usefully implemented on the easiest targets. These are likely initially to be faked mail coming from servers not belonging to the most heavily used mail domains on outgoing envelope headers, e.g. Hotmail and AOL. This would make it more difficult for people legitimately using Hotmail and AOL addresses from sending their mail from other clients on other networks but very few legitimate web-mail users would know how to do this, and those who wanted to do this would very soon acquire more reputable addresses. Spammers would then choose other domains to fake, but this fact provides a good incentive for any domains which don't want to be faked to update their DNS records and provide access to authenticated outgoing SMTP relays for their legitimate users.

Whether this succeeds in having a wider network effect might depend upon how many people are willing to block all mail coming from all domains which don't have the required DNS records, as well as all mail coming from unauthorised relays from domains with the required DNS records. The problem here is that the first people to block in this manner risk having a very high false positive rate to start with.

This proposal also only provides a start to prevent some of the worst abuse. When spammers have to use their own domains to send their spam, we will still need to be able to coordinate reporting and blocking of these spamming domains and relays, but hopefully this will make this job a bit easier.

Blocking forgeries and spam with SPF

Posted Oct 23, 2003 17:46 UTC (Thu) by iabervon (subscriber, #722) [Link]

I don't think there should be a significant number of false positives (at least, that don't get resolved), because domains won't add spf until they actually mean to, which will probably be prompted by problems caused by forged addresses.

The tricky part, I believe, is actually handling forwarding addresses. If you run a domain which has any addresses which are supposed to receive mail and forward it to some other address (where it is actually read), it will be tricky to support sending mail legitimately from that address, and it will be tricky to use spf for any other addresses in the domain.

Of course, one possibility would be to set up the machine to relay for a forwarding address if its domain has spf and the message is coming from an authorized host, but that depends on the user's reading domain having spf set up (and requires the user to configure their mailer to relay through the right place).

I don't think it would be a problem for vanity domains (in the sense of "x@y.name" is actually really "xy@isp.com", where isp.com handles everything, and there is not a separate SMTP server for y.name, just an MX record which points to isp.com).

It would be a problem for webmail people who want to send through a different interface, but that's either something the webmail sites don't like anyway, or something they could support (by allowing relays from authenticated users using the SMTP-POP/IMAP hack).

Blocking forgeries and spam with SPF

Posted Oct 23, 2003 17:45 UTC (Thu) by edstoner (guest, #4496) [Link] (2 responses)

Won't all spammers just then use a utility (i.e. dig) to get the SPF info for the domain they want to specify in the FROM: and use that info to get a valid address?

Blocking forgeries and spam with SPF

Posted Oct 24, 2003 3:15 UTC (Fri) by skybrian (guest, #365) [Link] (1 responses)

But to use that information, they have to forge the IP address in the TCP/IP connection to sendmail. Or break into a trusted machine. Both are harder than forging email addresses now, so it raises the bar a little.

Blocking forgeries and spam with SPF

Posted Oct 24, 2003 10:40 UTC (Fri) by liw (subscriber, #6379) [Link]

I have not yet studied SPF and can't comment on it in detail. I will have to read about the proposed solutions for e-mail forwarding systems (I use one such).

A minor point, however: I don't think SPF will stop spam, it will just take things to the next level. In the progress, the amount of spam might reduce, but possibly not.

Spammers can use virus and worm techniques to break into random workstations, which have the right to send mail using some valid, trusted mail server. They can then send spam despite SPF. Suppose the workstation is foo.example.com and the mail server is mail.example.com. The spammers can forge sender addresses to be @example.com, and SPF checks will not help.

I seem to recall a news article somewhere about a group of criminals selling exactly that kind of service to spammers, though I don't seem to have bookmarked it. They claim to control a few hundred thousand workstations, and given the ease in which viruses and worms spread, getting new ones probably won't be a problem.

Thus, widespread adoption of SPF might result in much higher levels of computer break-ins. This will, hopefully, in the end, result in better overall security. *That* might drastically remove the amount of spam, but it might also just cause spammers to invent yet more workarounds.

SPF will hurt Pobox.com?

Posted Oct 26, 2003 12:04 UTC (Sun) by PaulDickson (subscriber, #478) [Link] (1 responses)

I currently set my "From:" address to my pobox.com mailbox, but have my ISP
deliver the messages (via a local mail server on my LAN, with a faux domain).

SPF seems to require pobox.com users to send their out-going E-mail through
pobox.com. Currently, I use my ISP because it's convenient, not because
port 25 is blocked. But I can see SPF hurting companies like pobox.com.

There are ways around this, but it will require experience that most E-mail
users lack.

SPF will hurt Pobox.com?

Posted Oct 29, 2003 18:41 UTC (Wed) by jmason (guest, #13586) [Link]

Note: Meng Wong, author of the SPF proposed std, runs pobox.com ;)

Basically, the idea is that forwarding services like pobox.com will either:

1. allow users to specify the IP addresses allowed to forward -- so you would specify "pobox.com, for user foo@pobox, include the SPF records from bigisp.com". or

2. use SRS, an encapsulation scheme for the (hidden) "envelope-from" field, which allows a message to be re-sent while encapsulating the sender's original envelope-from address.

Blocking forgeries and spam with SPF

Posted Oct 31, 2003 2:14 UTC (Fri) by cypherpunks (guest, #1288) [Link]

This is an incredibly bad idea. You're suggesting every domain name on the Internet add a TXT record so spammers have to go through a tiny extra bit of work? Give me a break.

See Bogus ``solutions'' to spam for more on this common spam fallacy.