A recently discovered Linux rootkit has a number of interesting attributes
that make it worth a look. While it demonstrates the power that a rootkit
has (to perform its "job" as well as hide itself from detection) this
particular rootkit also has some fairly serious bugs—some that are
almost comical. What isn't known, at least yet, is how the system where it
was discovered became infected; there is no exploit used by the rootkit to
The rootkit was reported to the
full-disclosure mailing list on November 13 by "stack trace". Customers
had noticed that they were being redirected to malicious sites by means of
an <iframe> in the HTTP responses from Stack Trace's site. Stack Trace
eventually found that the Nginx web server on the system was not
<iframe> and tracked it to a loadable kernel module, which
was attached to the posting. Since
then, both CrowdStrike
and Kaspersky Lab's Threatpost
have analyzed the module's behavior.
The first step for a rootkit is to get itself running in the kernel. That
can be accomplished by means of a loadable kernel module. In this
case, the module is
at the most recent 64-bit
kernel version used by Debian 6.0 ("Squeeze"):
The presence of that file would indicate infection, though a look at the
process list is required to determine if the rootkit is actually loaded.
Once loaded, the module has a number of different tasks to perform that
are described below. The CrowdStrike post has even more detail for those
The rootkit targets HTTP traffic, so that it can inject an
<iframe> containing an attack: either a
that in a relatively undetectable way, it must impose itself into the
kernel's TCP send path. It does so by hooking tcp_sendmsg().
Of course, that function and other symbols that the rootkit wants to access
are not exported symbols that would be directly accessible to a kernel
module. So the rootkit uses /proc/kallsyms to get the addresses
it needs. Amusingly, there is code to fall back to looking for the proper
System.map to parse for the addresses, but it is never used due to
a bug. Even though the kernel version is hardcoded in several places in
the rootkit, the
System.map helper function actually uses uname -r to
get the version. The inability to fall back to checking
System.map, along with this version-getting
oddity make it seem like multiple people—with little or no
inter-communication—worked on the code. Other odd bugs in the
rootkit only add to that feeling.
For example, when hooking various functions, the rootkit carefully saves
away the five bytes it needs to overwrite with a jmp instruction, but then
proceeds to write 19 bytes at the start of the function. That obliterates
14 bytes of code, which eliminates any possibility of unhooking the
function. Beyond that, it can't call the unhooked version of the function
either, so the rootkit contains private copies of all the functions it hooks.
Beyond hooking tcp_sendmsg(), the rootkit also attempts to hide
its presence. There is code to hide the files that it installs, as well as
its threads. The file hiding works well enough by hooking
vfs_readdir() and using a list of directories and files that
should not be returned. Fortunately (or unfortunately, depending on one's
perspective), the thread hiding doesn't work at all. It uses the same
file-hiding code, but doesn't look in /proc nor convert the names
into PIDs, so ps and other tools show the threads. In the original
report, Stack Trace noted two threads named get_http_inj_fr and
write_startup_c; those names are fairly descriptive given the
behavior being seen. The presence of one or both of those names in the
process list would mean that the system has the rootkit loaded.
The rootkit does successfully remove itself from the list of loaded
modules. It directly iterates down the kernel's module list and deletes
the entry for itself. That way lsmod will not list the module,
but it also means that it cannot be unloaded, obviating the "careful"
preparations in the hooked functions for that eventuality.
As with other malware (botnets in particular), the rootkit has a "command
and control" client. That client contacts a particular server (at a
hosting service in Germany) for information about what to inject in the web
pages. There is some simple, weak encryption used on the link for both
authentication and obfuscation of the message.
Beyond just missing a way to propagate to other systems, the rootkit is
also rather likely to fail to persist after a reboot. It has code to
continuously monitor and alter /etc/rc.local to add an
insmod for the rootkit module. It also hooks vfs_read()
to look for the exact insmod line and adjusts the buffer to hide
that line from anyone looking at the file. But it just appends the command
to rc.local, which means that on a default installation of
Debian Squeeze it
ends up just after an exit 0 line.
Like much of the rest of the rootkit, the HTTP injection handling shows an
odd mix of reasonably sensible choices along with some bugs. It looks at
the first buffer to be sent to the remote side, verifies that its source
port is 80 and that it is not being sent to the loopback address. It also
compares the destination IP address with a list of 1708 search engine IP
addresses, and does no further processing if it is on the list.
One of the bugs that allowed Stack Trace to diagnose the problem is the
handling of status codes. Instead of looking for the 200 HTTP success
code, the rootkit looks for three strings on a blacklist that correspond to HTTP
list is not exhaustive, so Stack Trace was able to see the injection in a
400 HTTP error response. Beyond that, the rootkit cleverly handles chunked
Transfer-Encodings and gzip Content-Encodings, though the latter does an
in-kernel decompress-inject-compress cycle that could lead to
noticeable server performance problems.
None of the abilities of the rootkit are particularly novel, though it is
interesting to see them laid bare like this. As should be obvious, a rootkit
can do an awful lot in a Linux system, and has plenty of ways to hide its
tracks. While this rootkit only hid some of its tracks, some of that may
have happened after the initial development. The CrowdStrike conclusion is
instructive here: "Rather, it seems that this is contract work of an
intermediate programmer with no extensive kernel experience, later
customized beyond repair by the buyer."
The question of how the rootkit was installed to begin with is still open.
Given the overall code quality, CrowdStrike is skeptical that some
"custom privilege escalation exploit" was used. That implies
that some known but unpatched vulnerability (perhaps in a web application)
or some kind of credential leak (e.g. the root password or an SSH
key) was the culprit. Until and unless some mass exploit is used to
propagate an upgraded version of the rootkit, it is really only of academic
interest—except, of course, to anyone whose system is already infected.
to post comments)