Kernel-based malware scanning

On November 28, a message from an engineer at Sophos turned up on the kernel mailing list. It seems that Sophos has a security module (called "Talpa") which performs virus scanning; they would like for the LSM interface to remain so that this module can continue to be loaded. Of course, only free software modules are considered relevant for this discussion, but there is no problem with that: Talpa is available on SourceForge and has been since, well, November 23. Talpa was described this way:

So Talpa creates a hook whereby a (presumably proprietary) user-space process can scan file contents for bad stuff and, when it is found, block access to that stuff.

There was quite a bit of discussion about the approaches taken in this code, many of which are seen as being "significantly less nice." Suffice to say that any kernel running this module is not really Linux any more. But it also turns out that there is a lot of skepticism about the goal of this work, regardless of how it's implemented.

One might well be able to create a shiny, proprietary file scanner which can, with 100% accuracy, identify any file containing malware. But that still does not really solve the problem because it is very hard to guarantee that the file's contents as seen by the scanner match those which are later processed by an application. One can imagine a course of events something like this:

• A user asks the system administrator to look at a PDF file using some sort of plausible reason probably involving the highly humorous content which can be found inside.

• The administrator (as root, naturally) runs a PDF viewer application which opens the file.

• The user-space malware-scanning application springs into action; after extensive analysis it pronounces the file clean and allows the access to continue.

• Before the PDF viewer gets going again, the evil user overwrites the file with content crafted to exploit one of the several-hundred recent PDF library vulnerabilities.

• The now-compromised viewer immediately repurposes the system toward the task of posting Ron Paul-related comments on weblogs worldwide.

The obvious thing might be to prevent the evil user from writing malware to the file by scanning data as it is written as well. Simple forms of this technique could be circumvented by jumping around the file and writing the bad stuff in small, seemingly innocuous pieces. A much easier and more effective approach, though, is for the attacker to simply mmap() the file, allowing it to be rewritten without the need for any system calls at all. One could try to enforce exclusive access to the file, but that would require an effective revoke() system call, which has proved to be tremendously hard to implement. At this time, there really is no effective defense against that sort of attack.

Defenders of active scanning respond that this sort of local-attacker scenario is not really part of their threat model. If one is concerned about the content of files which arrive on the system via a web browser, a web server, an email client, or something along those lines, then attacks which involve race conditions exploited by local bad guys are not really part of the picture. For this case, active file scanning might be sufficient to detect most potential attacks. In the end, no security mechanism is going to be perfect. A mechanism which makes attacks harder is the best which can be done, and it should, at least, succeed in raising the bar. The response to that assertion is that, for such a simple model, the requisite scanning could easily be done entirely in user space.

In the short term, the Talpa module is not going to get anywhere near the mainline kernel. There is clearly a demand for that sort of feature, though; some segments of the user community feel that it will improve their security and they are willing to pay for it. So the real question that comes out of this discussion is how to provide this kind of feature in a way which plays well with the rest of the kernel. The word is that some of the relevant people are getting together to talk about new approaches; expect this topic to return sometime soon.

Note: you can avoid this step in the future by logging into your LWN account.