A look at Rspamd

SpamAssassin has been the dominant spam-filtering program in the free-software community for at least a decade. But the relative newcomer Rspamd is making significant gains, and may be a project worth serious consideration by those running email servers. It is designed to be easier to extend with new functionality, and the architecture supports running multiple, parallel processes even on clusters.

Vsevolod Stakhov started work on Rspamd in 2010 with the goal of building a more modular alternative to SpamAssassin—both in the architecture of program and in how its spam-filtering tests are designed. The latest major release was version 0.9.0 in May, which added support for native SpamAssassin rules, Domain-based Message Authentication, Reporting and Conformance (DMARC), and many other improvements. A series of minor updates have followed, the latest being 0.9.8 on June 25. The program is packaged for Debian, Ubuntu, Fedora, openSUSE, and is available in the FreeBSD ports collection.

Regarding its modular architecture, Rspamd is designed to run in multiple processes, with one controller process coordinating several workers that may be running locally or on other machines. The workers can each handle a separate task, such as performing statistical analysis of message content or conducting traditional rule-based filtering. The various processes communicate over HTTPS. The system is event-driven, with the intent to not block anywhere in the code, so that it can process many incoming messages simultaneously.

As for the actual spam-filtering tests, Rspamd's core modules are written in C, but the project also supports a flexible system for writing new filters as Lua plugins. The documentation lists eight Lua modules that ship with the default release. They include support for real-time blacklists (RBLs), recognizing common mailing-list signatures, detecting phishing URLs, and more.

The C modules, though, constitute what the project considers the core filtering functionality. They include support for Sender Policy Framework (SPF) validation checking, SURBL blacklists, DomainKeys Identified Mail (DKIM), and more. Like SpamAssassin, Rspamd uses the output of its various filters to assign a score to each message, indicating the likelihood that it is spam. The central filtering tool is a regular-expression engine that scans message body and header content, much like the one found in SpamAssassin.

In fact, there is a separate plugin available that lets one use unaltered SpamAssassin rules with Rspamd. But one difference that the Rpsamd project takes particular pride in is that it can support more sophisticated rules. Its expression-matching engine uses a more complex classifier than the traditional, single-word Bayesian algorithm used in SpamAssassin. Rspamd's algorithm, known as OSB-Bayes (for "Orthogonal Sparse Bigrams") was originally described in a 2006 paper [PDF].

In essence, OSB-Bayes adjusts the "spam probability" factor calculated for a message by de-emphasizing those trigger features (such as suspicious words) that seem to be evenly distributed in a message and emphasizing those trigger features that appear in clusters. The purported advantage is that this approach detects trigger features that are found in statistically unlikely groups, minimizing the "noise" caused by features that one would expect to find, randomly, spread out over the whole message. Rspamd's implementation uses five-word chunks, skipping small words. The process of tuning the implementation has been an ongoing one; the 0.9.0 release notes mention that the latest revisions significantly reduced the false-positive rate.

One of the other core modules implements the fuzzy hash-checking filter. This filter uses the Shingles algorithm (which chops up the text into chunks and searches for matches) to detect phrases that are similar to, but not exact matches for, suspected spam content. Rspamd provides separate fuzzy storage workers to save the hashes calculated on incoming message contents to a local database. That allows the fuzzy-checking module to learn based on a mail server's particular message traffic. Storing the hashes rather than plain text, of course, has privacy benefits.

All the filtering features in the world mean nothing if the system cannot be used in the real world, however. On the deployment front, Rspamd offers an SMTP proxy mode that can be used as an intermediary between any Mail Transfer Agent (MTA) and Mail User Agent (MUA)—although most users will probably prefer to take advantage of the direct MTA integration features. Currently, Rspamd supports integration with Exim, Postfix, Sendmail, and Haraka.

Postfix and Sendmail integration relies on Stakhov's rmilter mail filter. Haraka includes an Rspamd plugin, while Exim integration requires patching the Exim source in several places.

One other distinction between Rspamd and SpamAssassin is that Rspamd includes a web-based administration interface. It allows the user to monitor the status of worker processes, see statistics about the incoming mail traffic, and adjust several of the configuration knobs for calculating filter scores and triggering the resultant actions. There are several competing web interfaces for SpamAssassin, although they are developed by third parties.

Rspamd has certainly not supplanted SpamAssassin for many system administrators, although the project's documentation and release notes frequently mention that it is in use in real-world, high-volume deployments. In 2013, Stakhov told the SpamAssassin mailing list that he had originally written Rspamd for a single client, and has been in the process of growing the project since.

Whatever its origins, one does increasingly see it mentioned in discussions and blog posts about spam filtering—particularly for running a personal mail server, which bodes well for the long-term future. Smaller deployments are often less risky, but they are frequently how a new project gains a foothold with administrators. The project is also a 2015 Google Summer of Code mentoring organization. Between those factors and the rapid pace of development over the past year or two, Rspamd appears to be a healthy project on firm footing—one that mail server administrators might be wise to keep an eye on.