Spammers Most Likely Users Of E-Mail Authentication (TechWeb) [LWN.net]

Spammers Most Likely Users Of E-Mail Authentication (TechWeb)

Posted Jul 13, 2005 16:33 UTC (Wed) by elanthis (guest, #6227) [Link] (9 responses)

This isn't news, really. E-mail authentication is not an anti-spam measure. The very name implies that; it's "authentication," not "authorization."

Combine an authorization service - like a real-time blacklist - with authorization, and then you have an anti-spam solution. The authorization allows you to know with certainty where the mail came from, and then you can compare that with the blacklist. Authorization forces spammers to use their own domains, and then you can just blacklist the whole domain. No more need for blacklisting individual hosts and trying to figure out which host a spam came from (think relays) and so on.

Spammers Most Likely Users Of E-Mail Authentication (TechWeb)

Posted Jul 13, 2005 16:48 UTC (Wed) by bfields (subscriber, #19510) [Link] (8 responses)

Combine an authorization service - like a real-time blacklist - with authorization, and then you have an anti-spam solution.

Also, it solves the problem of bounces going to the wrong address, doesn't it? If people stopped sending bounces when they couldn't authenticate the from address, that'd cut down significantly on my "spam", much of which is just automatic responses to mail that someone else sent with my from address.

Spammers Most Likely Users Of E-Mail Authentication (TechWeb)

Posted Jul 13, 2005 17:00 UTC (Wed) by dwmw2 (subscriber, #2063) [Link] (2 responses)

If people stopped sending bounces when they couldn't authenticate the from address, that'd cut down significantly on my "spam", much of which is just automatic responses to mail that someone else sent with my from address.

Alternatively, you could just stop accepting those bounces to mail you didn't send. See my configuration, SES, or BATV for more information.

The simple principle is that you stop sending MAIL FROM:<your@address.here> and always send from a timestamped, hash-signed address. Then you stop accepting bounces to the 'raw' address.

Far more effective than SPF, and doesn't involve throwing away valid mail, like SPF does. See http://david.woodhou.se/why-not-spf.html for further discussion of that.

Spammers Most Likely Users Of E-Mail Authentication (TechWeb)

Posted Jul 15, 2005 15:23 UTC (Fri) by giraffedata (guest, #1954) [Link] (1 responses)

That method has a serious drawback, applied unilaterally -- it defeats white lists. Every mail is from an address the recipient has never seen or sent to before. It also screws up various other email sorting on the receiver's end.

I use a simplified version of that -- I always include my name in the From: header (in the conventional manner). Bounce messages usually contain a copy of the headers. If I see a From: header with my email address but not my name, I know I did not send it. Spammers rarely have both my name and my address.

This is a filter just for my personal mailbox, of course.

Spammers Most Likely Users Of E-Mail Authentication (TechWeb)

Posted Jul 18, 2005 7:58 UTC (Mon) by farnz (subscriber, #17727) [Link]

Most whitelisting is on the Internet message "From:" header, not on the SMTP "MAIL FROM:" header. Thus, you tend not to break whitelists, but you do ensure that anything that harvests your "From:" header for spamming doesn't trigger bounces back to you, as you know that that address never accepts bounces.

Spammers Most Likely Users Of E-Mail Authentication (TechWeb)

Posted Jul 13, 2005 17:10 UTC (Wed) by TwoTimeGrime (guest, #11688) [Link] (4 responses)

> If people stopped sending bounces when they couldn't authenticate the from
> address, that'd cut down significantly on my "spam"

I just stopped accepting all mail from parties where I couldn't validate the from address. That elimitated about 70% of my spam. There's still a lot of spam sent with fake addresses.

Spammers Most Likely Users Of E-Mail Authentication (TechWeb)

Posted Jul 15, 2005 15:16 UTC (Fri) by giraffedata (guest, #1954) [Link] (3 responses)

>I just stopped accepting all mail from parties where I couldn't validate >the from address. That elimitated about 70% of my spam. And plenty of your ham, too. Not many people hate spam enough to do that. In fact, you're most of the way toward the ultimate spam solution -- don't accept any mail at all.

Spammers Most Likely Users Of E-Mail Authentication (TechWeb)

Posted Jul 16, 2005 7:34 UTC (Sat) by dwmw2 (subscriber, #2063) [Link] (2 responses)

And plenty of your ham, too. Not many people hate spam enough to do that. In fact, you're most of the way toward the ultimate spam solution -- don't accept any mail at all.

Not really. You don't lose much ham by rejecting mail which has an invalid reverse-path, and in the cases that you do it tends to improve the actual communication.

Consider an analogy... if you are sending me a letter by snail mail and you accidentally put an invalid return address on it, how am I ever going to reply or actually manage to have a conversation with you? It's much better for that letter never to leave your local post office where your postie knows you and will hand it back to you.

That's basically what happens when my mail servers reject a mail from an invalid recipient. Most of the time it's actually spam, and the spambot just moves on to its next victim. But on the rare occasion that it's actually a genuine mail from someone who has misconfigured their mailer, the fact that I reject their mail means that their own mail server can tell them about it. If I'd accepted the mail, that chance for them to fix the problem would have been lost.

Spammers Most Likely Users Of E-Mail Authentication (TechWeb)

Posted Jul 16, 2005 17:44 UTC (Sat) by giraffedata (guest, #1954) [Link] (1 responses)

I guess you're talking about a different kind of validating the from address.

The context of this article is SPF, which is about determining that a From: address in an email, while it may be a valid email address, is not the address of the person who sent the email.

But while SPF provides a way to prove the From: address is genuine, most ham doesn't participate in SPF, so one wouldn't want to reject mail just because SPF failed to validate the from address.

Spammers Most Likely Users Of E-Mail Authentication (TechWeb)

Posted Jul 18, 2005 7:54 UTC (Mon) by farnz (subscriber, #17727) [Link]

SPF validates the "MAIL FROM:" part of the SMTP protocol. Unlike Sender ID, it has nothing to do with the "From:" header of the mail message.

A compliant SMTP server is supposed to return bounces (not rejects) to the address given in "MAIL FROM:"; spammers don't want bounces, so they tend to use false addresses in "MAIL FROM:", while legitimate users do want bounces, so they use genuine addresses in "MAIL FROM:"

This is to be expected

Posted Jul 13, 2005 16:59 UTC (Wed) by Ross (guest, #4065) [Link] (1 responses)

People are trying to paint this as something bad but it's great that spammers
are being forced to be honest about being the origin of their spam. It will
make filter lists more effective and prevent innocent third parties from
being unfairly blamed.

This is to be expected

Posted Jul 19, 2005 9:00 UTC (Tue) by job (guest, #670) [Link]

No, it does not. The normal user will only care about what's in the From-field and you can still type anything there.

I know you are probably thinking of automatic blacklist filters on the SPF information now, but what would be more effective is to filter out specific IP networks instead. Domains can be easily changed and if this gets widespread we will only see more top domains like .nu which allows you to change your domain name as much as you want for a yearly fee.

IP based filtering is much more effective, and it may surprise you to find out that this has been used for nearly(?) ten years to a great success, technically speaking. We got rid of most open relays that way. The big problem turned out to be political: Who will administer the list and how to you remove false listings? This will be a problem with any blacklist.

So SPF accomplishes nothing except getting rid of some joe jobs, at the price of breaking the email model. The collateral damange includes email forwarding via SMTP and the DNS TXT record, as the RFC stands right now. But this may change in the future. There are much cleaner ways to get rid of false bounces by tagging the legitimate ones if this is very important to you.

No one with an understanding of the email RFCs would want to use SPF. But don't take my word for it. Listen instead to Brad Knowles or Eric Allman of sendmail fame, they really know their stuff.

Spammers Most Likely Users Of E-Mail Authentication (TechWeb)

Posted Jul 13, 2005 20:25 UTC (Wed) by einstein (subscriber, #2052) [Link]

The "it pundits" who came up with this story are missing one obvious fact: Spammers are the most likely users of email, PERIOD. Therefore it follows that they are the "most likely users" of any mandatory standards which must be followed in order to send email.

Despite the FUD against SPF, it does make the spammers job more difficult, and it makes spam detection more effective.

Misleading stats

Posted Jul 13, 2005 20:53 UTC (Wed) by pjs (guest, #10927) [Link] (1 responses)

Reading that 9% of domains used SPF and 84% were spam sure makes it sound like hardly anybody is using SPF, and of those it's mostly spam. One might even wrongly conclude that a SPF record is an indicator of spam.

The important (and potentially misleading) word is "domains". That's a very different word than "messages". In fact, the paragraph that gives us these figures begins by telling us the data is compiled from 17.7 million messages. But all the data is regarding domains, and they don't tell us how many. And we have no way of relating this info about domains back to what really matters: messages!

Consider that 16% of the 9% of domains that used SPF (or 0.0144% of all domains). AOL.COM is probably one of those 0.0144%. HOTMAIL.COM and MSN.COM are also on that list. It's hard to imagine 17.7 million messages wouldn't contain at least a few from these three domains.

In fact, it's hard to imagine that these three domains don't add up to anything less than a substantial portion of all received messages. What percentage, who knows? They don't give us that info in the article. It's entirely possible that 20%, 30%, maybe even 50% of all messages were from the 9% of domains using SPF (or the 0.014% using it for non-spam), since only a small handful of domains concentrate a vast number of users.

It's also hard to imagine that spammers reuse the same domain name for very long. Most likely, they're cycling through lots and lots of them. If this is the case, it's pretty easy to see how a spammer using SPF and buring through lots of lots of domains could easily accumulate a very high percentage of all per-domain SPF usage. AOL & Microsoft do exactly the opposite, using a few domains that they never change... just a tiny drop in the ocean when SPF is measured on a per-domain scale.

What matters are messages. What percentage of messages that pass SPF checks are legitimate vs spam?

Who knows? MX Logic probably does. Gregg Keizer probably does not, since his article is just a rehash their press release. Gregg probably didn't even notice that paragraphs begins with talk of 17.7 million messages to set a tone of relevance to spam levels but then gives per-domain stats that reflect the tendancy of legitimate sends to continue using their long established domain name and spammers to burn through massive numbers of domains.

It's hard to fault Gregg (who probably cares much more about word counts and deadlines than these "details"), but MX Logic really should know better.

Misleading stats

Posted Jul 15, 2005 15:49 UTC (Fri) by giraffedata (guest, #1954) [Link]

Reading that 9% of domains used SPF

I think you misparsed the sentence. It does give per centages of messages. It says, "... a sampling of 17.7 million messages ... 9 per cent [of those messages] from domains with published SPF records..."

And this means that AOL, Yahoo, and Hotmail are sending very little email. Assuming they publish SPF records, at most 9% of email comes from those 3 domains combined.

The article doesn't mention how the study knows from what domain the mail comes, so I think we should assume it means the domain in the From: header. There's a big difference between saying email came from a domain that publishes SPF records and email came with a From: header identifying a domain that publishes SPF records. Much of my spam comes with a @yahoo.com From: header but does not come from Yahoo. Spammers do this for two reasons: 1) a per centage of it gets through anyway; and 2) they haven't kept up with the times.

SPF should be killed

Posted Jul 14, 2005 11:46 UTC (Thu) by ahoh (guest, #17291) [Link] (5 responses)

I hate SPF.
It complicates my mail setup and makes tracking down problems related to
mail routing more difficult.

I am used to deliver mail directly (via postfix) to the MX of the target
address. If there is anything going wrong I notice that in the logs and so
I am able to solve the problem with the peer affected.

With SPF I am supposed to route the mail through a gateway (different ones
for each FROM I have). Not only I see no use in my mailprovider reading my
mails, I am also sucked if he manages to get blacklisted (again) and if
the mail vanishes in a black hole I have no idea who is the next in the
chain to track down the problem.

All in all I think SPF makes mail communication (for the experienced ones)
more fragile and errorprone without rewarding any relief of spam.

I like solutions like http://greylisting.org/ though. It doesn't require
adoption of new standards and you can hook in a lot of clever tricks for
burning the mail directed to you (intentionally or unintentionally ;-) )
without annoying unrelated systems.

ahoh

SPF should be killed

Posted Jul 14, 2005 20:13 UTC (Thu) by copsewood (subscriber, #199) [Link] (2 responses)

If you send your mail direct to the MX of the recipient, why not just create SPF records for your outgoing domains pointing to the machine you send this mail from ?

If you and many others send legitimate mail for these domains from different machines with different IP addresses, and these domains have all authorised different outgoing relays and your IP address varies, then I accept that this is more of a problem. Most travelling salesmen sending email from hotel rooms only need to send for 1 domain, in which case authenticating to the relay authorised by this domain is going to be the same regardless of where you send from.

SPF should be killed

Posted Jul 16, 2005 10:55 UTC (Sat) by DennisJ (subscriber, #14700) [Link] (1 responses)

I have users who for verious reasons prefer to use addresses from their home isp or some webmail service instead of the address we offer them, but send through the server I administer when they are here.

Unless I'm completely missing something, there's no way for me to set up SPF records for those domains, I just see a lot of work for me if SPF really catches on.

SPF should be killed

Posted Jul 16, 2005 15:47 UTC (Sat) by copsewood (subscriber, #199) [Link]

The more recent CSV proposal seems likely to be more secure and simpler than SPF, but with SPF its the domain owners who decide which server can legitimately send mail on that domains behalf, and it is up to them not you either to provide relaying facilities, or to authorise your relay for their domain. If you must block outgoing port 25 to all addresses but your own mail relay server, make sure that you leave 587 open, so they can use the authenticated MSA relay setup by their domain owner for this purpose. Webmail domain owners may or may not provide 587 MSA facilities - but that is between them and their users. If they have you delegated as the technical contact for the domain, all you have to do is add a DNS SPF record for the domain delegated to point to your own relay.

Personally, having read about this in the last few days, I think the CSV approach will be simpler for admins than SPF and possibly avoid potential denial of service security problems which the SPF macro expansion capabilities allow for. With CSV the DNS setup looks simpler, and if you relay customer mail outgoing in respect of their own domains you just HELO as your own domain, listen to the abuse address at the domain you HELO as and take responsibility for what you relay.

SPF should be learned before destruction is advocated

Posted Jul 14, 2005 20:32 UTC (Thu) by Ross (guest, #4065) [Link] (1 responses)

You seem to hate it because you don't understand it. You are in control of
where mail can be sent from -- that's the whole point of the SPF record. If
you don't publish one then you can send from anywhere just like before. How
does that complicate things?

SPF should be learned before destruction is advocated

Posted Jul 20, 2005 7:09 UTC (Wed) by ahoh (guest, #17291) [Link]

"You are in control of where mail can be sent from -- that's the whole
point of the SPF record. If you don't publish one then you can send from
anywhere just like before. How does that complicate things?"

The statement is not quite right. I am only in control of where mail can
be sent from if I am administrating the mail domain of the email in
question.
That is ok if I dish out company mail where I have control over the DNS
records of the FROM address.

The problem starts as soon as you deal with FROM addresses of various mail
providers (Think about a firewall gating "private" mail of various people
each having its own webmail account).

In such an environment you want to dump the outgoing mail as close as
possible to the recipient, but the replies should be delivered to whatever
is specified in the mail header (after all we are talking about SMTP and
thats one of its features). My mailer is only authoritative for receiving
the company mail after all and I assume we agree that changing THAT would
be a really bad idea in most environments ;-)

As far as I found out SPF forces me to route the mail via the mail gate of
the webmail provider if HE has a SPF record (note as administrator of a
different domain or as a road warrior I have no influence on that).

Spammers Most Likely Users Of E-Mail Authentication (TechWeb)

Posted Jul 16, 2005 4:16 UTC (Sat) by 0015517 (guest, #2065) [Link]

A small amount of the SPAM that's getting thru to me passes SPF. So far all of it has been sent thru big Mail Service Providers like Hotmail. A Spammer get's a hotmail account, sends spam and hotmail's outgoing mail server matches it's SPF record.

If Spammers are creating SPF records for their zombies, it's not getting to me thru MX Logic's servers, unless they decide to run an open relay. If Spammers are forced to use SPF or other sender authentication, they will be quickly added to lists of Spamming domains.

SPF is working!

Steve