Happy New Year from SpamAssassin

[Posted January 1, 2010 by corbet]

Those of you using SpamAssassin to filter your mail may want to watch things a bit more closely than usual; it seems that current versions still include the rule known as FH_DATE_PAST_20XX, which adds 2-3 points to any message with a 2010 date in the headers. Surprisingly enough, such dates have suddenly become common, with the result that SpamAssassin may be generating more false positives than usual. The fix is to add:

    score FH_DATE_PAST_20XX 0.0

to the local.cf file.

Happy New Year from SpamAssassin

Posted Jan 1, 2010 17:04 UTC (Fri) by edwill (guest, #62771) [Link] (3 responses)

fuhchee - This issue was first identified over a year ago, and a fix constructed about half a year ago. See "SpamAssassin 2010 Bug" for information on how it went down.

Happy New Year from SpamAssassin

Posted Jan 1, 2010 17:55 UTC (Fri) by joey (guest, #328) [Link] (2 responses)

But no release, and sa-update even seems to be pulling down the old broken
rule rather than a fixed one.

Happy New Year from SpamAssassin

Posted Jan 1, 2010 22:57 UTC (Fri) by csamuel (✭ supporter ✭, #2624) [Link] (1 responses)

Seems to be available via sa-update now:

header FH_DATE_PAST_20XX Date =~ /20[2-9][0-9]/ [if-unset: 2006]

Happy New Year from SpamAssassin

Posted Jan 5, 2010 17:17 UTC (Tue) by ajross (guest, #4563) [Link]

Right. So the fix just introduces a 2020 bug. Sigh...

Happy New Year from SpamAssassin

Posted Jan 1, 2010 17:17 UTC (Fri) by seveas (guest, #57389) [Link]

The bugreport as filed in 2010: https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6269

Happy New Year from SpamAssassin

Posted Jan 1, 2010 17:46 UTC (Fri) by mbanck (subscriber, #9035) [Link]

This has hit two major german ISPs already. I wonder whether this Y2K10 bug will make more damage on GNU/Linux systems (or to users who get served by them) than the Y2K issues.

Happy New Year from SpamAssassin

Posted Jan 1, 2010 18:49 UTC (Fri) by PO8 (guest, #41661) [Link] (11 responses)

I would argue that the right fix for this problem is to enhance the pattern language so that it can match relative dates. Then you could have a rule that flags messages that are more than a year in the past or a year in the future, without worrying that the rule will go stale. It looks like this would be doable with the eval: construct in spamassassin, but I haven't the energy to try it right now, so I did some dumb kludge that will fail in 2050. :-)

Happy New Year from SpamAssassin

Posted Jan 1, 2010 18:57 UTC (Fri) by dark (guest, #8483) [Link] (3 responses)

The problem with such time-dependent rules is that it is hard to measure their effectiveness. Their evaluation of existing archives (of preclassified mail) is going to change with time, usually for the worse.

And it does need to be measured. There's no obvious reason why spam would be more likely to have bad date headers than ham. I've been using Debian long enough to remember when the alpha port had system clock issues; I have a lot of ham from 2020 in my mail folders :)

Happy New Year from SpamAssassin

Posted Jan 1, 2010 19:31 UTC (Fri) by rbuchmann (guest, #52862) [Link] (1 responses)

Some spammers try to have their spam always on top in the mailbox by using dates in the future.

Happy New Year from SpamAssassin

Posted Jan 2, 2010 4:59 UTC (Sat) by jengelh (subscriber, #33263) [Link]

That is why I use Sort By Arrival in the re-alpine MUA. Well, I use it not because of the spammers, but because of course people from all timezones can send me mail, and I rather prefer that new items get tacked onto the inbox list at the end rather than somewhere in the middle (a little prone to oversee it when you have lots).

Happy New Year from SpamAssassin

Posted Jan 1, 2010 19:47 UTC (Fri) by iabervon (subscriber, #722) [Link]

That depends on what relative dates are relative to. Generally the top "Received" date is trustworthy, since it's provided by the user's own mail server, and it's what ought to be close to the "Date" date. It doesn't matter when the mail is scanned, of course.

Happy New Year from SpamAssassin

Posted Jan 1, 2010 23:10 UTC (Fri) by dom (guest, #37940) [Link] (6 responses)

I think everyone agrees :) The fix pushed out today was a minimal fix for the immediate problem; they have another decade to push out a proper one :)

https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6271

Happy New Year from SpamAssassin

Posted Jan 3, 2010 1:47 UTC (Sun) by Cyberax (✭ supporter ✭, #52523) [Link] (5 responses)

Ha! That's what they though about Y2K in 80-s.

Happy New Year from SpamAssassin

Posted Jan 3, 2010 2:05 UTC (Sun) by jordanb (guest, #45668) [Link] (4 responses)

Considering that we ended up having very few problems with Y2K, it seems like the 1980s assessment was correct. The Y2K problem was successfully dealt with in the 1990s and caused no major disruptions.

Although it did show just how crappy many perl programs are.

BTW, to all Perl programmers out there: Happy new year, 19110!

Happy New Year from SpamAssassin

Posted Jan 3, 2010 4:14 UTC (Sun) by rahvin (guest, #16953) [Link] (1 responses)

It could be argued that the only reason Y2K didn't cause a computer metldown is so many people ran around screaming that it was going to cause a meltdown along with all the people who tried to capitalize on those people by selling guns, food, generators and other "end of civilization" products. It's interesting to consider that had there not been so much publicity and the end of the world crowd not gotten involved that the y2k bug would have resulted in serious problems although certainly not the end of the world.

Happy New Year from SpamAssassin

Posted Jan 3, 2010 5:33 UTC (Sun) by JoeBuck (guest, #2330) [Link]

Exactly. Y2K wasn't a problem because billions were spent fixing all of the software systems before it happened. Without those fixes (and many of the bugs that were fixed would have been catastrophic), the software simply wouldn't have worked, and we would have had a huge mess.

Also, if it weren't for all of this investment, the dot-com bubble would have exploded a couple of years earlier. Thanks to all that dot-com spending, we didn't see a tech collapse until spring of 2000.

Happy New Year from SpamAssassin

Posted Jan 3, 2010 9:22 UTC (Sun) by cventers (guest, #31465) [Link] (1 responses)

> BTW, to all Perl programmers out there: Happy new year, 19110!

Ah, but you see, any Perl program that erroneously string-concatenates the
year would have already been rendered broken at the turn of the
millennium (19100).

Or perhaps my sarcasm detector is broken :p

Happy New Year from SpamAssassin

Posted Jan 3, 2010 10:34 UTC (Sun) by drag (guest, #31333) [Link]

I think that the point was that some software sucks so badly that it's been
broken for 10 years and has not been fixed yet!

Happy New Year from SpamAssassin

Posted Jan 2, 2010 4:24 UTC (Sat) by frazier (guest, #3060) [Link] (1 responses)

Although I'm not a Sysadmin, this is why part of why I subscribe to LWN. Thanks for making these sorts of things more known.

Happy New Year from SpamAssassin

Posted Jan 3, 2010 3:07 UTC (Sun) by ejr (subscriber, #51652) [Link]

Indeed, thank you! This explains recent false positives at my ISP, and now I can point them at the reason directly.