User: Password:
|
|
Subscribe / Log in / New account

Known-exploit detection for the kernel

Did you know...?

LWN.net is a subscriber-supported publication; we rely on subscribers to keep the entire operation going. Please help out by buying a subscription and keeping LWN on the net.

By Jake Edge
December 18, 2013

An attacker might try a number of different kernel exploits before actually getting one that works with a specific running kernel. If the kernel were instrumented to detect the failed attempts, it could alert system administrators about an in-progress attack in addition to returning an error code to the attacker. That's the idea behind a patch set proposed by Vegard Nossum: complain loudly when someone attempts to exploit a closed security hole.

Any given kernel will have both patched and unpatched vulnerabilities; hopefully the latter are far fewer than the former. When targeting a system, attackers can either figure out which kernel version is running and what vulnerabilities it is likely to have, or they can just try to exploit a bunch of recent vulnerabilities. Many attacks seem to be of this untargeted, mass-attack style, so recognizing and flagging failed attempts may help administrators put a stop to the attacks.

Nossum suggested adding an exploit() annotation to the fixes for specific kinds of now-closed security holes. For example, in sock_alloc_send_pskb() from net/core/sock.c:

    if (npages > MAX_SKB_FRAGS) {
        exploit("CVE-2012-2136");
        goto failure;
    }

In suitably configured kernels, that would put out a rate-limited message to the system logs noting a "possible exploit attempt". But, he said, the annotations should not be for all bugs, nor should they have an unlimited lifespan:

I propose limiting the annotation of known exploits to the most serious type of exploit, namely where the attacker otherwise silently gains root/elevated capabilities. For sure, there is little point in calling exploit() where an older kernel would just panic or OOM.

I also propose to keep each exploit() annotation around for only ~5 years after the bug was discovered/fixed. This will allow us to catch most of the intrusion attempts while still not littering the kernel code forever.

The reaction has largely been positive, though there were some concerns and quibbles. Ted Ts'o wondered if malware writers would just start checking kernel versions more carefully to avoid setting off the alarms. But Kees Cook is not convinced that testing for kernel versions will be all that effective:

The reality of the situation is that the kernels running on an end-user's system is rarely a stock upstream kernel. As a result, they usually have organization-specific versioning, which makes version-only autodetection useless to an attacker. While it is possible to keep track of all distro versions in a massive table, even the public exploits rarely do this, instead focusing on maybe one or two distros. But when attacking systems with kernels built custom by various organizations that don't publish their kernel trees, it becomes impossible to rely on just the version. Given all the forks, and stable vs mainline, and backported patches vs not, the version tends to only give a gross ball-park idea. Probing is still useful to an attacker, and this proposes reporting those probes.

Ts'o disagreed, as he believes the landscape mostly consists of distribution kernels: "testing for various distribution kernel versions, as well as specific ChromeOS and Android kernel versions, wouldn't be that difficult for an attacker, and would probably allow them to avoid detection for 99% of the Linux systems found in the wild". Cook noted that careful attackers aren't really the focus of this work. He believes that there are a lot more custom kernels installed than Ts'o does, but recognizes there is no real way to know. There are "sloppy attackers", though, who will probe all kinds of kernels without doing any checks first—Nossum's patches would potentially catch some of those.

Furthermore, Ts'o is skeptical that the enterprise distributions will even build with the CONFIG_EXPLOIT_DETECTION flag turned on. The support burden for explaining false positives (and actual attacks for that matter) might be rather high. While not speaking in any official capacity, Jiri Kosina said that he suspected SUSE would indeed turn off exploit detection to try to "maintain sanity of our support engineers".

Ryan Mallon is worried that attackers will just clean out the logs as soon as they hit upon a successful attack. But Ingo Molnar pointed out that many sites do not rely on log files on the local disk, but instead use the network or append-only media to protect their logs.

The logistics of adding the annotations as well as maintaining them going forward was another area of concern. Cook volunteered to help add annotations, but wanted to make sure that he wasn't the only one doing so. Dave Jones wondered about tests to ensure the triggers are still firing correctly when code around them changes. James Morris is also concerned about the long-term maintenance of the triggers. He is not at all sure that the feature belongs in the mainline "without at least first being proven in the field".

Adding tests could also help ensure that a vulnerability doesn't get reintroduced, which is something that has happened several times in the past, as Molnar pointed out. In addition, he said, annotating earlier bugs will help alert kernel developers to "'hotspot' areas in the kernel that tend to attract more bugs than others". The annotations will also help point out dangerous patterns in the code.

Nossum has been maintaining the patch set for around six months at this point. It consists of two base patches, one that adds the exploit() call and another to hook it into the audit subsystem, and then seven patches to add annotations for CVEs from the past two years. The latter patches are largely "one-liners in the error path of a specific input validation check". He doesn't believe there is much of a maintenance burden for the triggers and plans to maintain a public git repository with the patches going forward.

Linus Torvalds was generally in favor of the idea: "I think that it's a good idea to at least have the option to complain about certain errors, and leave markers in the logs about things that look suspicious." But he doesn't want to see annotations added for random CVEs, just those that are actually being used by rootkits or other malware. Cook and Nossum both seem to be on the same page with Torvalds; that only "serious privilege escalation issues" (in Cook's words) get annotated.

While it may not catch that many attackers, catching even one is clearly better than none. Given that the patch is lightweight, and has a low maintenance burden, it wouldn't be a surprise to see it get added to the mainline before too long. As Molnar suggested, for more security-sensitive installations, exploit() could be turned into a more active deterrent that freezes all tasks being run by the suspected UID. It could certainly be a useful tool in the ever-escalating battle between administrators and attackers.


(Log in to post comments)

Known-exploit detection for the kernel

Posted Dec 18, 2013 23:56 UTC (Wed) by spender (subscriber, #23067) [Link]

There's a 100% chance that if this were merged, some distro will compile in support for it despite it being useless. Yet another in a line of features designed to provide the illusion of security.

It is already trivially circumvented via a single function usable by any other exploit, present in enlightenment:
http://grsecurity.net/~spender/exploits/enlightenment.tgz

But it's cute that discussion continues while those involved are completely oblivious to its futility.

-Brad

Known-exploit detection for the kernel

Posted Dec 19, 2013 0:49 UTC (Thu) by ncm (subscriber, #165) [Link]

Boldly demonstrating my ignorance... is this comment meant to suggest that there is no need for an attacker to risk possibly-fixed security holes when a variety of holes known not to be fixed is available in a convenient tarball from Brad?

Known-exploit detection for the kernel

Posted Dec 19, 2013 0:58 UTC (Thu) by spender (subscriber, #23067) [Link]

Not at all correct -- that would be completely irrelevant to the topic at hand since it has nothing to do with 0-days (nevermind that some involved in the discussion threw the term around incorrectly).

Reading the code would be more instructive: cve_chicken_out() in exploit.c

-Brad

Known-exploit detection for the kernel

Posted Dec 19, 2013 1:27 UTC (Thu) by ncm (subscriber, #165) [Link]

Demonstrating again my ignorance, this seems easily defeated by "chmod 0 /boot/vmlinuz*", perhaps along with similar work in /lib/modules.

Known-exploit detection for the kernel

Posted Dec 19, 2013 1:34 UTC (Thu) by spender (subscriber, #23067) [Link]

For distro kernels that are easily available on the Internet? (this is why it allows fallback to a vmlinuz in the current directory)

-Brad

Known-exploit detection for the kernel

Posted Dec 19, 2013 8:10 UTC (Thu) by ncm (subscriber, #165) [Link]

So the real workaround you propose is to determine the distro and kernel version and then check the intended exploits against it before attacking the real kernel. Maybe you use a table of all the likely kernel versions and their well-documented (in-)vulnerabilities.

Suppose, then, that in addition to making /boot and /lib/modules inaccessible, uname() is made to lie or be otherwise uninformative. Are you left with any alternative than to try out your catalog of exploits? If you try the newest one first, the kernel under attack might not have that fault inserted yet. If it fails, you have not exposed yourself, but you still have to try the older ones, and risk exposure. Or maybe any EINVALID return is logged, with the PC where it was noted, so the first try exposes you anyway.

Maybe you restrict yourself to faults that have always been there, and only recently patched, if at all. Then, either you get in silently, or you trip the alarm. This still seems better than being able to try out your whole catalog without worry of being noticed. No doubt I'm still missing something crucial.

Known-exploit detection for the kernel

Posted Dec 19, 2013 12:35 UTC (Thu) by spender (subscriber, #23067) [Link]

I gave you the exact file and function to look at, and it's clear you haven't even done that, so this will be my last post in this thread. You don't want to learn, you want validation that you could somehow wrap this turd of a feature with the appropriate metal lining to make it useful. It's just not going to happen.

What you propose (which is also what Ingo proposed) will not prevent invisible detection of the vulnerable kernel. I have *many* other methods I can use even in the event everything you mention is implemented.

This is my promise: if any form of this security theater makes its way upstream and into any distro, I will write any code necessary to expose it for what it is.

-Brad

Known-exploit detection for the kernel

Posted Dec 19, 2013 16:06 UTC (Thu) by dgm (subscriber, #49227) [Link]

> So the real workaround you propose is to determine the distro and kernel version and then check the intended exploits against it

Spender's code shows that it's actually easy to look for signs of fixes in the kernel image before trying any exploit, thus defeating the detection scheme.

This could be prevented by making the kernel image unreadable except for root, and in case of using SELinux, only for the bootloader tools.

Known-exploit detection for the kernel

Posted Dec 20, 2013 4:03 UTC (Fri) by shmget (subscriber, #58347) [Link]

or the good old gentoo way of not leaving /boot mounted. ?
to mount /boot you need to be root.. and if you are root already what's the point....

Known-exploit detection for the kernel

Posted Dec 24, 2013 6:18 UTC (Tue) by drag (subscriber, #31333) [Link]

I am sure that there are other ways to determine what version of he Linux kernel that is running.

'uname -a' comes to mind.

This 'threat detection' is just one of those things that, if you are lucky, can be used to detect some old hack a script kiddie may try on your computer as they just randomly try different things. But any threat beyond the most trivial it would quickly become completely worthless.

Known-exploit detection for the kernel

Posted Dec 24, 2013 9:22 UTC (Tue) by dgm (subscriber, #49227) [Link]

The question is what percentage of the threats a Real World box is exposed to is "trivial". This is not a tool to make the OS invulnerable, but to improve the bottom-line.

Known-exploit detection for the kernel

Posted Dec 19, 2013 7:17 UTC (Thu) by josh (subscriber, #17465) [Link]

You might want to add support for non-gzipped kernels, but yeah, it does look about that simple.

The obvious workaround would be to not put the CVE number in the kernel, but to instead just print "exploit attempt detected at $file:$line" and let it be decoded later. Still easily worked around if you're targeting a particular distro kernel, but if you're targeting a *particular* distro kernel anyway then you could just know which exploits to run against it.

Known-exploit detection for the kernel

Posted Dec 19, 2013 12:19 UTC (Thu) by spender (subscriber, #23067) [Link]

Some trivial static analysis (the likes of which I already do in enlightenment for other purposes) will defeat that "workaround" easily. Obviously I'm not going to waste time writing code for every possible change that could be made to the "feature" but the fact remains that there's no asymmetry of information here. Whatever eventually gets merged will be known to all and defeated. I guarantee it.

It's clear to me many people want more than anything to believe that this feature will work, regardless of any facts.

-Brad

Known-exploit detection for the kernel

Posted Dec 19, 2013 14:32 UTC (Thu) by josh (subscriber, #17465) [Link]

> Some trivial static analysis (the likes of which I already do in enlightenment for other purposes) will defeat that "workaround" easily.

I'd be curious what kind of static analysis you have in mind.

> It's clear to me many people want more than anything to believe that this feature will work

I don't think anyone believes this mechanism will stop or detect sophisticated attacks. The question remains: will it detect some subset of unsophisticated attacks, enough to make it worth including, or not?

Known-exploit detection for the kernel

Posted Dec 24, 2013 6:30 UTC (Tue) by drag (subscriber, #31333) [Link]

> The question remains: will it detect some subset of unsophisticated attacks, enough to make it worth including, or not?

The social aspect of it is the problem.

Sure the kernel implementing features like this may succeed at detecting ham-fisted attacks, but the problem you run into is that people will naturally assume that this security feature has merit and will unfortunately actually try to depend on it.

People will write blog posts on how critically important it is to make sure you use a kernel version with this detection enabled in it and so on and so forth. It'll end up in Google's cache when people search 'make sure Linux is secure' or such things and then it'll just get increasingly stupid and self-defeating from then on.

You can see the same thing happen all over the place with various 'linux security features'.

A perfect example of this is things like 'chkrootkit'. People may end up with a hacked website and then install chkrootkit and run it thinking that somehow it will actually be effective at detecting root kits, which it is not. Or schemes that involve checking file checksums using 'rpm' utility. Or things like 'fail2ban' being used to 'secure' ssh, or port knocking, or any other number of silly and hokum things that people try to do to 'improve' their security.

Like anti-virus in Windows there is a actual limited useful application for some of this stuff, but most people are not able to know what that is.

Software security is already hard enough without creating features that add to the confusion with no real benefit.

Known-exploit detection for the kernel

Posted Dec 19, 2013 14:39 UTC (Thu) by vegard (subscriber, #52330) [Link]

Hi Brad,

I'm really grateful that we have people like yourself who are both competent and view security with a critical eye. Your feedback is appreciated. Keep up the good work!

Vegard

Known-exploit detection for the kernel

Posted Dec 20, 2013 0:57 UTC (Fri) by tialaramex (subscriber, #21167) [Link]

"despite it being useless"

Like dusting for prints is useless, because criminals all wear gloves, right? Like it's pointless to own a burglar alarm because any "real" burglar would know how to disable it, right? You're actually making the same mistake the people you don't like usually make, assuming perfection where it's unwarranted.

Ordinarily it's the good guys who have to be perfect. One missed special case and you're exploitable. But in this sort of situation the roles are reversed, one slip by the bad guys and the alarms go off. _Everything_ they try has to be properly insulated so that it can't trigger this sort of alarm, because if just one thing isn't, and the relevant alarm is installed, game over.

Known-exploit detection for the kernel

Posted Dec 20, 2013 1:54 UTC (Fri) by PaXTeam (guest, #24616) [Link]

i'm not entirely sure whether you grasped the proposed feature and spender's criticism (the perfection strawman and silly real life non-examples indicate otherwise) but here it is in a nutshell:

detecting and reacting to incidents is a useful thing (grsec has been probably doing it for longer and more efficiently than most) but this feature will not achieve anything, not just because there're so many ways to know when a kernel is equipped with it or because the potential for false positives, but also because in real life nobody reacts to logs on systems where script kiddies are considered an actual threat (these are the lowest value systems, cf. the kernel.org compromise whose dmesg with the sign of a backdoor laid in public for weeks before anyone figured out that something wasn't right with it). and where they're not a threat, the remaining attackers are sophisticated enough to use 0-day to begin with (many of which the mechanisms in grsec will handle unlike the proposed feature).

Known-exploit detection for the kernel

Posted Dec 20, 2013 10:52 UTC (Fri) by tialaramex (subscriber, #21167) [Link]

The issue of ignoring alarms is a far more widespread one than can be covered in this thread, or indeed LWN at all.

In the maritime industry almost every other accident report will involve alarms that were disabled, non-functional, or ignored. In other transport industries it's less bad, but e.g. a London Underground driver overrode or disabled all the safety alarms in his train and was only alerted to the fact that he'd driven out of a station with the doors still open by the screaming of passengers. They couldn't alert him using the provided passenger alarm because he'd switched that off too.

Teaching people to investigate alarms rather than ignore them, and to test and maintain alarm systems so that they have confidence that the alarms reported are real is a bunch of work, but it's not impossible.

Known-exploit detection for the kernel

Posted Dec 24, 2013 6:34 UTC (Tue) by drag (subscriber, #31333) [Link]

The most critical step in that is to have alarms that are actually meaningful and functional.

In most of those cases you mentioned you'd find that those alarms were disabled because they had so many false positives. A alarm that goes off once because a train left it's doors open is useless when it also goes off a thousand times when the doors are actually closed.

Known-exploit detection for the kernel

Posted Dec 19, 2013 1:07 UTC (Thu) by gerdesj (subscriber, #5446) [Link]

This is very well intentioned (and I'm thankful that someone is thinking laterally) but I think it is not the right way.

The kernel should restrict itself with doing kernel related stuff - doing complicated things to and with my hardware. It should not try and be an IDS.

For IDS I would like my perfect kernel to provide access, in a reasonably safe way, to a wealth of information about what is happening in there.

As far as I can tell (as a non KDev) it does that rather well already and getting better as are the u/s tools.

If I want to somehow attach a form of Ksnort/KclamAV or whatever to the kernel then I will (I wont - it's daft) but I'm not convinced that you will do it better (but your's will be far cooler and bloody quick).

KDevs - please continue doing what you do remarkably well - we sysadmins will err... try and keep up.

I keep re-reading the article and also note the list of names of people I respect in there but I can't help but think that this is not a good approach to securing the Linux kernel - what am I missing?

Cheers
Jon

ooo look ... me .. yes: me

Posted Dec 19, 2013 1:27 UTC (Thu) by gerdesj (subscriber, #5446) [Link]

Got it at last.

Kernel devs: Do your best at making the bestest, fastest, safest and most featureful kernel you can.

Me (sysadmin): Keep an eye out for vulnerabilities. Notify the kernel devs accordingly.

That'll do pig.

Cheers
Jon

Known-exploit detection for the kernel

Posted Dec 19, 2013 1:28 UTC (Thu) by dlang (subscriber, #313) [Link]

This isn't a way to secure the kernel as much as a way to have it act as a honypot if a problem that's been fixed gets triggered.

There is currently no way to detect that this sort of thing is taking place, and short of logging every syscall, it's just not possible without explicit support like this.

As long as the maintinance of this is not a burden, I don't see a problem with this (done sanely, rate limited with decent log messages)

you already have the kernel logging a lot of things, this is just a little more to go into the logs that you can either ignore or take advantage of.

Known-exploit detection for the kernel

Posted Dec 19, 2013 2:23 UTC (Thu) by PaulWay (subscriber, #45600) [Link]

As I see it, this makes sense for sysadmins who:

* use kernels with the CONFIG_EXPLOIT_DETECTION turned on (either rolling their own or using a distro's enabled kernel)
* have systems that are likely to be probed by script-kiddies
* want to know if they're being probed
* watch their logs

Now, that isn't everyone - but it's a lot of people, including me. Even if I discover the probing after the fact, there's a good chance that the script-kiddie won't clean the logs, so I can then shut the system down and restore from a good backup.

So I for one think this a good idea.

Have fun,

Paul

Known-exploit detection for the kernel

Posted Dec 19, 2013 6:14 UTC (Thu) by wahern (subscriber, #37304) [Link]

Script kiddies, by definition, don't need to worry about cleaning the logs. That's what their scripts are for. If the scripts don't do it now, they will. Script kiddies do the minimum possible. If you raise the bar, there's no reason to believe that it slows them down one iota.

Adding any security which can be easily circumvented is no security at all. All it takes is one person to write the circumvention code and to share it.

There are people doing the thankless job of actually preemptively scanning the code for vulnerabilities and fixing them. Those guys are priceless. Why they keep doing it when all the fame and adulation goes to this kind of stuff, port knocking, and other crazy schemes.... well, I wish their commitment could spread the same way interest in these schemes do.

These schemes only work as long as they're not widely adopted. Once they're widely adopted, they get added into the kiddie scripts. Then you're left with a bunch of useless code which only adds to your attack surface.

Known-exploit detection for the kernel

Posted Dec 19, 2013 7:11 UTC (Thu) by noxxi (subscriber, #4994) [Link]

From my understanding the code is not to fight new exploits, but to detect if a known and fixed exploit was tried. And because the exploit was fixed the script kiddie will have no way to clean the logs (unless it has another and better exploit).
Thus the log entry can be used as a canary by the admin to detect if an account might be compromised and lock it down before worse exploits will be tried.

Known-exploit detection for the kernel

Posted Dec 19, 2013 7:50 UTC (Thu) by dlang (subscriber, #313) [Link]

the better shops also ship their logs off of the local systems so that attempts to scrub the logs will fail.

Known-exploit detection for the kernel

Posted Dec 19, 2013 9:36 UTC (Thu) by zlynx (subscriber, #2285) [Link]

A really well informed attacker can try to jam the log server with nonsense UDP or TCP resets. He'd need access to the log server network of course.

If he can DOS the log server, it won't record anything except a pile of junk. Once he gets root he can kill -9 the log service, clean the logs and restart it.

Just another thing to watch out for.

Known-exploit detection for the kernel

Posted Dec 19, 2013 16:30 UTC (Thu) by Funcan (subscriber, #44209) [Link]

A sufficiently advanced attacked can also break in and steal the log server. I doubt most people are facing that level of APT most of the time though...

Known-exploit detection for the kernel

Posted Dec 19, 2013 9:34 UTC (Thu) by Trou.fr (subscriber, #26289) [Link]

This is so stupid. I rather wish the kernel developers spend the time lost on actually improving the kernel security.
I find it baffling that even security-minded people like Kees Cook support the idea.

Known-exploit detection for the kernel

Posted Dec 19, 2013 12:44 UTC (Thu) by iq-0 (subscriber, #36655) [Link]

You seem to suggest that not doing this would change the focus of said kernel developers to do more kernel security improvements. That is rather wishful thinking, they would do something different but not necessarily that.

The idea itself is also not bad. It's the equivalent of doing authentication attempt logging and detecting possible attacks. Does it increase security? No. Does it tell you there might be something going on? Yes. Can attackers circumvent those checks? Yes. Does that make them less valuable? No, the attackers have to take more precautions, take more actions which might be detected or risk tripping an alarm.

An analogy would be to install a video cameras in your house. Any burglar could easily either circumvent them, disable them or ignore them and afterwards erase the tapes. But it makes live more difficult for them (even if slightly), might deter them to look for easier targets or increase the chance of detection because of all the steps involved. And there is a good chance some are not that diligent and get caught in the act.

You can never assume to be secure, you can only try and hope that it's as uneconomically possible for others to abuse the weaknesses.
But often it's just as important to just know somebody might have (tried) to do that, so you can take appropriate measures to minimize possible (indirect) damage.

Known-exploit detection for the kernel

Posted Dec 19, 2013 23:16 UTC (Thu) by Trou.fr (subscriber, #26289) [Link]

I perfectly understand the reasoning. But I think this is quite representative of the current mindset of developpers regarding security : no consideration for effective measure and it sometimes seems no consideration for actual security.

The single "recent" feature which led to actual security improvements in the kernel I can think of is seccomp-bpf, which is a brilliant and very efficient idea.

As you said, developpers would probably turn their attention to something else entirely. If only some grsec features could be included in the kernel.

Known-exploit detection for the kernel

Posted Dec 25, 2013 23:02 UTC (Wed) by nix (subscriber, #2304) [Link]

If only some grsec features could be included in the kernel.
That seems unlikely to happen while the grsec people remain incapable of working with other people or taking criticism of any kind without blowing up like two-year-olds. (Apologies to my two-year-old niece, who blows up very rarely and is usually quite charming and sweet.)

Known-exploit detection for the kernel

Posted Dec 25, 2013 23:56 UTC (Wed) by PaXTeam (guest, #24616) [Link]

did you move to the southern hemisphere recently? i can't think of many other places where it's hayfever season. keep up the nonsense and spender and i wish you a merry christmas (try without the drugs though).

Known-exploit detection for the kernel

Posted Dec 26, 2013 22:22 UTC (Thu) by nix (subscriber, #2304) [Link]

... aand my point is proved. I'd rather bite my own nose off than work with people this nasty.

Known-exploit detection for the kernel

Posted Dec 20, 2013 15:10 UTC (Fri) by roblucid (subscriber, #48964) [Link]

I don't understand your objections, it's a fallacy to suppose that the guy who maintains this patch set, would necessary be capable of solving unknown kernel issues.

Reading logs manually is way too slow, so good sysadmins generate alarms filtering for where action may be required.

It might not be only Internet facing boxes, where for example remote or local root exploit attempts might be of interest. Engineers & other employees to get curious and try things, sometimes things they really are not supposed to. Just because the most clueful and careful attacker won't trip something doesn't mean a measure is useless.

Known-exploit detection for the kernel

Posted Dec 19, 2013 16:25 UTC (Thu) by tshow (subscriber, #6411) [Link]

It seems to me that rather than just sending something to syslog, this would be more useful if it could feed into userspace hooks and things like firewall rules. If I could rig exploit() to:

if(user is nonlocal)
{
log them out & lock the account
add their IP to the ban list
send the admin a "user tried exploit" message
}

I could see that being useful. Obviously there's potential pitfalls (people boobytrapping each other, or managing to get daemons kickbanned if the system isn't smart enough), but a tripwire hooked to a nuke can be a useful line in a layered defense.

Known-exploit detection for the kernel

Posted Dec 19, 2013 18:09 UTC (Thu) by RobSeace (subscriber, #4435) [Link]

Sounds like something better suited to a userspace daemon parsing the kernel log messages and acting on them... Sort of like "Denyhosts" does now with failed SSH logins...

Known-exploit detection for the kernel

Posted Dec 19, 2013 20:35 UTC (Thu) by tshow (subscriber, #6411) [Link]

That's what I'm implying; it would be nice if this could export via some sort of file descriptor through which you could feed a userspace daemon, rather than shuffling stuff off to the logger. Logging is great, but if there's an exploit attempt I want to deal with it now, not after something gets done parsing the logs.

Known-exploit detection for the kernel

Posted Dec 19, 2013 20:45 UTC (Thu) by dlang (subscriber, #313) [Link]

you can feed it to your userspace daemon through syslog pretty darn fast, how fast do you think it needs to be acted on? and are you talking about an automated action or a manual action? if it's a manual action, the time taken to get a human's attention is FAR longer than the delay that syslog will introduce.

In addition, I'm not sure this is something you want to jump on every time (that the reason for rate limiting the alerts), you may just get flooded with alerts.

Known-exploit detection for the kernel

Posted Dec 20, 2013 16:21 UTC (Fri) by tshow (subscriber, #6411) [Link]

I'd want it for automated response, with a subsequent notification to the admin that the automated response has been made.

If someone is script kiddying their way through a series of exploits to see what sticks, I want the userspace daemon to be able to log them out and ban their IP (and possibly lock their account) *now*, this microsecond, before they get a chance to try the next one. Yes, that does imply potential scheduling hackery to get the data out the exploit alert pipe to the userspace daemon before control is allowed to return to the offending process.

In theory you'd only get flooded with alerts if the system was flooded with exploit attempts. You'd probably want to know if that was happening, but rate limiting could be done in the userspace daemon.

I think you'd want to be able to differentiate between local and remote exploits; if someone logged in is trying an exploit it's a very different kettle of fish from someone trying a remote exploit against the external interface.

Feeding the userspace daemon through syslog is kind of silly; a dedicated pipe the daemon could connect to would be better. Avoiding syslog as a transport means:

- being able to specify a consistent communication protocol
- possibly being able to avoid parsing entirely, but at the least the protocol can be designed to make the parsing trivial
- not needing to worry about some other syslog output source being tricked into spoofing valid-looking but manufactured exploit logs
- the daemon being able to sleep on data availability in the pipe rather than having to constantly chew on syslog and spit out the (1 - epsilon) it doesn't need, so the daemon will mostly just be asleep until its needed

Known-exploit detection for the kernel

Posted Dec 20, 2013 18:47 UTC (Fri) by dlang (subscriber, #313) [Link]

> I'd want it for automated response, with a subsequent notification to the admin that the automated response has been made.

There are a whole lot more tools available to do the alerting based on syslog message than there are going to be if you invent a new protocol.

> If someone is script kiddying their way through a series of exploits to see what sticks, I want the userspace daemon to be able to log them out and ban their IP (and possibly lock their account) *now*, this microsecond, before they get a chance to try the next one. Yes, that does imply potential scheduling hackery to get the data out the exploit alert pipe to the userspace daemon before control is allowed to return to the offending process.

This requires much more significant changes, and really is going to require that you put your response code into the kernel inside the detection routine.

> In theory you'd only get flooded with alerts if the system was flooded with exploit attempts. You'd probably want to know if that was happening, but rate limiting could be done in the userspace daemon.

given that the effort involved in generating the log compared to the action that triggers the log, not having this be rate limited would result in a very good DOS vector.

> Feeding the userspace daemon through syslog is kind of silly; a dedicated pipe the daemon could connect to would be better. Avoiding syslog as a transport means:

> - being able to specify a consistent communication protocol

why can't you use a consistent protocol in your syslog message?

> - possibly being able to avoid parsing entirely, but at the least the protocol can be designed to make the parsing trivial

you can make the parsing trivial in syslog as well

> - not needing to worry about some other syslog output source being tricked into spoofing valid-looking but manufactured exploit logs

you can do this inside syslog with filters today (only send to this output if the source is the kernel and the message contains blah)

> - the daemon being able to sleep on data availability in the pipe rather than having to constantly chew on syslog and spit out the (1 - epsilon) it doesn't need, so the daemon will mostly just be asleep until its needed

again, with filtering in syslog, your userspace daemon can do this today.

And as I noted up at the top, there are a lot more alerting engines around that understand syslog than ones that will understand your new protocol.

Any significant company is already going to have an alerting tool in place, it's far better to integrate with that than to require a new tool be written.

Known-exploit detection for the kernel

Posted Dec 29, 2013 8:06 UTC (Sun) by lamawithonel (subscriber, #86149) [Link]

this patch set uses the audit framework without any rate limiting, and in the somewhat more structured audit format. that sounds like what you want.
+ audit_log_format(ab, "exploit id=%s pid=%u uid=%u auid=%u ses=%u comm=",
+                  id, pid, uid,
+                  from_kuid(&init_user_ns, audit_get_loginuid(task)),
+                  audit_get_sessionid(task));

Known-exploit detection for the kernel

Posted Dec 25, 2013 23:03 UTC (Wed) by nix (subscriber, #2304) [Link]

The problem with that sort of thing is that as soon as it becomes popular enough that attackers can speculate that you might be running it (which is not very popular at all), it suddenly becomes a wonderful DoS tool.

Known-exploit detection for the kernel

Posted Jan 6, 2014 3:03 UTC (Mon) by speedster1 (subscriber, #8143) [Link]

> The problem with that sort of thing is that as soon as it becomes popular enough that attackers can speculate that you might be running it (which is not very popular at all), it suddenly becomes a wonderful DoS tool.

There shouldn't be much DoS potential for script kiddies to abuse if there were a reliable mechanism for automatic account-locking like tshow wanted.

On the other hand, count me among those who predict this feature will quickly become worked-around by all the popular exploit kits -- at least on any systems lacking admins who are big enough on security to be running custom kernels with generic uname info. Those admins who do tweak their uname and hide /boot /lib/modules are probably not the ones who the kernel devs need to worry about protecting from script kiddies (their custom kernels probably include grsecurity...)

Known-exploit detection for the kernel

Posted Jan 6, 2014 13:22 UTC (Mon) by nix (subscriber, #2304) [Link]

Locking an account *is* a DoS for the account's legitimate owner.

Known-exploit detection for the kernel

Posted Jan 7, 2014 2:59 UTC (Tue) by speedster1 (subscriber, #8143) [Link]

That makes no sense to me, perhaps you can explain more... are these DoS victims naive users who are running random things that script kiddies left for them? Or a daemon which has been tricked into running evil code? Either way, if a script kiddie gets to run stuff from an account, that account has been compromised and deserves to be locked out until it can be cleaned up. Maybe instead you are thinking of a way to lock out a non-compromised account with this feature?

Known-exploit detection for the kernel

Posted Jan 8, 2014 16:38 UTC (Wed) by nix (subscriber, #2304) [Link]

Imagine that someone has found a way to exploit, say, procmail, or some other daemon run on the user's behalf that accepts code from the network and is ultimately invoked from the network.

The right thing to do in that situation is probably to halt mail delivery and just queue everything -- but your proposal would lock the entire account. An attacker that can determine what accounts exist (perhaps via said exploit) could then DoS-attack the entire system trivially.

(But, of course, if they can execute arbitrary code as one user they can probably do that anyway, in about a million ways, and probably get root too. So perhaps my concerns are unjustified. It might well elevate a failed breakin, via an exploit that doesn't actually work, to a partial DoS, but I'm finding it hard to be too worried about that.)


Copyright © 2013, Eklektix, Inc.
This article may be redistributed under the terms of the Creative Commons CC BY-SA 4.0 license
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds