Kernel-based malware scanning

Readers of LWN's Kernel Page have long been aware of the ongoing debate over the value of the Linux Security Module (LSM) framework and the security policies which have been implemented using it. One result from that discussion has been the removal of the modular interface in the upcoming 2.6.24 kernel. If that change stands (Linus has said that he could yet be talked out of it), security modules will have to be built into the kernel; there will be no way to load them at runtime. Arguments for this change include ending abuses of the LSM interface, the need to have a security policy in place when the kernel is first booted, and the fact that there is no safe way to unload most known security modules.

On November 28, a message from an engineer at Sophos turned up on the kernel mailing list. It seems that Sophos has a security module (called "Talpa") which performs virus scanning; they would like for the LSM interface to remain so that this module can continue to be loaded. Of course, only free software modules are considered relevant for this discussion, but there is no problem with that: Talpa is available on SourceForge and has been since, well, November 23. Talpa was described this way:

So Talpa creates a hook whereby a (presumably proprietary) user-space process can scan file contents for bad stuff and, when it is found, block access to that stuff.

There was quite a bit of discussion about the approaches taken in this code, many of which are seen as being "significantly less nice." Suffice to say that any kernel running this module is not really Linux any more. But it also turns out that there is a lot of skepticism about the goal of this work, regardless of how it's implemented.

One might well be able to create a shiny, proprietary file scanner which can, with 100% accuracy, identify any file containing malware. But that still does not really solve the problem because it is very hard to guarantee that the file's contents as seen by the scanner match those which are later processed by an application. One can imagine a course of events something like this:

• A user asks the system administrator to look at a PDF file using some sort of plausible reason probably involving the highly humorous content which can be found inside.

• The administrator (as root, naturally) runs a PDF viewer application which opens the file.

• The user-space malware-scanning application springs into action; after extensive analysis it pronounces the file clean and allows the access to continue.

• Before the PDF viewer gets going again, the evil user overwrites the file with content crafted to exploit one of the several-hundred recent PDF library vulnerabilities.

• The now-compromised viewer immediately repurposes the system toward the task of posting Ron Paul-related comments on weblogs worldwide.

The obvious thing might be to prevent the evil user from writing malware to the file by scanning data as it is written as well. Simple forms of this technique could be circumvented by jumping around the file and writing the bad stuff in small, seemingly innocuous pieces. A much easier and more effective approach, though, is for the attacker to simply mmap() the file, allowing it to be rewritten without the need for any system calls at all. One could try to enforce exclusive access to the file, but that would require an effective revoke() system call, which has proved to be tremendously hard to implement. At this time, there really is no effective defense against that sort of attack.

Defenders of active scanning respond that this sort of local-attacker scenario is not really part of their threat model. If one is concerned about the content of files which arrive on the system via a web browser, a web server, an email client, or something along those lines, then attacks which involve race conditions exploited by local bad guys are not really part of the picture. For this case, active file scanning might be sufficient to detect most potential attacks. In the end, no security mechanism is going to be perfect. A mechanism which makes attacks harder is the best which can be done, and it should, at least, succeed in raising the bar. The response to that assertion is that, for such a simple model, the requisite scanning could easily be done entirely in user space.

In the short term, the Talpa module is not going to get anywhere near the mainline kernel. There is clearly a demand for that sort of feature, though; some segments of the user community feel that it will improve their security and they are willing to pay for it. So the real question that comes out of this discussion is how to provide this kind of feature in a way which plays well with the rest of the kernel. The word is that some of the relevant people are getting together to talk about new approaches; expect this topic to return sometime soon.

---

(Log in to post comments)

As a Ron Paul supporter, I laughed like crazy upon reading the remark :p

http://en.wikipedia.org/wiki/Ru_Paul, in double-secret-political drag.

What I'd really like is a fs with checksums. Once you know the checksum of the 'clean' file,
just check that of the one you're opening.

That would also help my backup, subversion, scons... which use checksums to detect file
modification.

Another way is to provide a user-space open() which loads the data, checks it, then makes it
available to the original caller I suppose.

How does that work with mmap? Any time a process writes to memory backed by the file, the
checksum has to be recomputed?

It doesn't work if checksummed on a file-by-file basis: if the file is 
large enough you can DoS-attack the system just by changing a single byte 
every so often (oops, the whole file has to be reread...)

Checksummed blocks *do* work, and are useful to detect a variety of 
disk-incurred problems like writes that accidentally landed in the wrong 
place and so on. But this doesn't help to detect malware because malware 
is using exactly the same syscalls as non-malware. If used for that 
purpose it becomes like the UK ID card, where what they really want is an 
`I am not a terrorist' card...

What Talpa is doing, instead, is using what amounts to a huge mugshot 
database of known bad guys. Unfortunately for them there's no right time 
to check the data written against the mugshot, and whatever method they 
use the next generation of malware will specifically evade...

If they want a filestore with an integrated scanner running in userspace, why don't they
implement it with FUSE (and an ordinary filestore as the data store)?

Or better, just use LD_PRELOAD to run the scanner when wanted, all in user space.

How's that work out for statically linked apps?

What statically linked apps?

And more precisely, which ones do add new files to your system that you want to check?
Remember that you trust the current apps, just not new files that are added by them.

It won't work for those of course, but the current approach doesn't always work either. As a
last resort you can always use inotify or whatever to scan the files generated by static apps,
and that doesn't have to be horribly slow either if you have a clue where the files are added.
But static apps are rare, so I don't see the problem.

How about the statically linked emergency boot shell?  Now every shell script is a "statically
linked app."  Also, someone could purposefully statically link an otherwise innocuous bit of
code and use it as a conduit.  That is, the "installation" procedure for some bit of malware
might include an additional level of indirection.

LD_PRELOAD could work for many things, but it strikes me as leaving too many holes, more than
the "scan on open" approach does.  (Now, if "scan on open" also made a temporary read-only
copy for all readers/executers, a'la RCU, you might have something!)

Why would a malicious app bother opening other malicious apps if it can do whatever it wants
all ready? You're missing the point. The only purpose of the file scanning talked about here
is to detect malicious software when it's installed/downloaded/saved on disk. When you have
malicious software doing whatever it wants you've already lost. "Scan on open" isn't good
enough to prevent malicious apps from writing other malicious files anyway. For more details
read the lkml thread.

Shell scripts aren't statically linked apps at all, it's just the shell running, in general a
dynamically linked bash, so LD_PRELOAD will work for them fine.

We're talking about damn virus scanners here, not a security framework (The former is mostly
about detection, the latter mostly about damage mitigation). If you want your own brew of
security then write an LSM module, or SELinux ruleset, but if you want to do something as
simple as file scanning then just do it with a preloaded lib.

The only real solution to the local user problem, I would think, is walls of separation. At
the simplest level, never open a file a user asks you to open; always copy it and run chmod
600 over it first, then check it, then open it. At a larger level, design things with clear
gateways; documents have to be loaded onto the server with FTP/CVS/SVN/HTTP, wherein they get
checked before distributed. Don't trust anyone with log-on access that you wouldn't trust with
all the data on the computer.

This doesn't work in some environments, like a shared university system, but I would consider
those secured as much by the university's power over those who access it then any software
protection. I certainly wouldn't trust such a system for anything even slightly sensitive.

The root PDF example is facetious, a competent security module could 
decline access to a file that could be re-written by a non-privileged 
user.  Also it could deny read access, to files currently held open for 
Write by other users.  Similarly, it could decline write access to files, 
that have open file descriptors held by other users, similar to the 
default file locking used by OS like VMS.

SunOS 4 had a union type file system, that was COW, it was used as basis 
for a source code management system.  That might also be an interesting 
approach, at price of losing POSIX filesystem semantics.

Actually a COW filesystem, overlay for chroot-ed daemons, would allow 
hard-linking of most of the files, so it wouldn't just be useful when some 
kind of file scanning was intended.