fixed in v2.6.31.1

Posted Sep 25, 2009 8:45 UTC (Fri) by hppnq (guest, #14462)
In reply to: fixed in v2.6.31.1 by PaXTeam
Parent article: Kernel release status

what we find and fix ourselves is released in our patches

You do not submit a patch for the Linux kernel. Instead, an exploit is released, with a lot of fanfare indeed. Selectively quoting Linus about the greatness of shame is not going to convince people that it is Linux security you are after. In fact, I am not even so convinced anymore that you are just plugging your own software at the expense of someone else's.

fixed in v2.6.31.1

Posted Sep 25, 2009 12:17 UTC (Fri) by spender (guest, #23067) [Link] (7 responses)

Why would we submit a patch for the Linux kernel when a patch already existed for the bug and was queued for the 2.6.31.1 kernel? We can't control the fact that the stable team sat on the release for 2 weeks while public exploits circulated (I wasn't the only one to write one). And what fanfare are you talking about? Ingo complained that there wasn't enough fanfare apparently, that I should have emailed full-disclosure/dailydave like I usually do and gotten the information out to everyone, and then surely again I would be insulted by people like you (and Ingo) for being an attention seeker by doing so.

-Brad

fixed in v2.6.31.1

Posted Sep 25, 2009 18:09 UTC (Fri) by dlang (guest, #313) [Link] (6 responses)

if the fix was already in the release process, why release an exploit at all?

also, it's very common for the -stable release to wait until after the initial merge window before releasing 2.6.x.1 the same developers who do the -stable work also work on the main kernel and that is a very busy two weeks.

fixed in v2.6.31.1

Posted Sep 25, 2009 18:20 UTC (Fri) by foom (subscriber, #14868) [Link] (5 responses)

> if the fix was already in the release process, why release an exploit at all?

Apparently it takes the existence of a public exploit for a kernel bug's severity to be properly
described as an arbitrary code execution exploit, instead of just a denial-of-service crash.

fixed in v2.6.31.1, also caught by StackProtector

Posted Sep 26, 2009 9:55 UTC (Sat) by mingo (guest, #31122) [Link] (4 responses)

It simply was not known as an exploitable bug. When we know something is exploitable a -stable kernel is released immediately, within hours.

(Let me know about a labeling method that actually works in practice - i'm not aware of any. Post facto labeling does not actually prevent bugs from getting mislabeled - it mostly only increases the security theatre - i.e. rewards the wrong kind of behaviour and the people who leech off of that.)

Also, note that if you had StackProtector enabled (CONFIG_CC_STACKPROTECTOR_ALL=y), the bug is not exploitable into a local root hole but gets caught and the break-in attempt gets exposed:

[   66.928106] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: ffffffff810c49d2
[   66.928109] 
[   66.939059] Pid: 2628, comm: exploit Not tainted 2.6.31 #17228
[   66.944947] Call Trace:
[   66.947423]  [ffffffff81622ba6] panic+0x84/0x12f
[   66.952238]  [ffffffff810c49d2] ? sys_perf_counter_open+0x660/0x672
[   66.958729]  [ffffffff810500cd] __stack_chk_fail+0x27/0x57
[   66.964419]  [ffffffff810c49d2] sys_perf_counter_open+0x660/0x672

It's a good thing we revived this feature in recent kernels, it already caught real kernel bugs and also catches exploit attempts.

fixed in v2.6.31.1, also caught by StackProtector

Posted Sep 26, 2009 12:47 UTC (Sat) by spender (guest, #23067) [Link] (1 responses)

So I have the answer to my question above then: "Is it the case that yourself, Paul Mackerras, Xiao Guangrong, Greg Kroah-Hartman, Peter Zijlstra, Eugene Teo and the other members of oss-security *ALL* can not plainly see that trivial, classic stack overflows, like the one signed-off on the fix for, are exploitable for arbitrary code execution? Did you all really believe that it was purely a "kernel crash" as noted in the commit message you all reviewed and signed off on?"

Let the record show that the Linux kernel development community and oss-security can't spot the exploitability of a trivial stack overflow.

It would be funny if it weren't so pathetic.

-Brad

fixed in v2.6.31.1, also caught by StackProtector

Posted Sep 26, 2009 13:16 UTC (Sat) by mingo (guest, #31122) [Link]

Yes, it indeed happens.

I take responsibility for this having slipped through (i should have caught it), and let me defend those other hard working people, who take an active part in the upstream kernel development process. Yes, nobody is infallible, and no, i dont think your repeated attempts to ridicule them is fair or justified.

If you think you could do better then i'd invite you to take part in the process instead of taking pot shots at it externally. Otherwise you'll never know whether you could do better than those whom you attack so viciously.

The fundamental question there (if you care): are you able to participate in a constructive community? Judging by your messages your life seems to be defined by hatred.

fixed in v2.6.31.1, also caught by StackProtector

Posted Sep 26, 2009 13:20 UTC (Sat) by spender (guest, #23067) [Link] (1 responses)

I seem to recall someone talking about another exploit being unexploitable from stackprotector. Right, it was you:
http://www.pubbs.net/kernel/200904/62088/
"the vmsplice exploit would only have been caught by the -ALL variant."

How quickly you forgot this:
http://copilotconsulting.com/mail-archives/linux-kernel.2...

and the fact that you actually committed his fix (and so were involved in the discussion above):
http://copilotconsulting.com/mail-archives/linux-kernel.2...

Did you have any proof for that one? That SSP stops exploitation of a vuln that doesn't even involve overwriting a return address?

-Brad

fixed in v2.6.31.1, also caught by StackProtector

Posted Sep 26, 2009 13:45 UTC (Sat) by mingo (guest, #31122) [Link]

Did you have any proof for that one? That SSP stops exploitation of a vuln that doesn't even involve overwriting a return address?

Indeed, i was wrong about that in the changelog, mea culpa. StackProtector has its place, but it would not have stopped the vmsplice exploit.

Thanks,

Ingo

fixed in v2.6.31.1

Posted Sep 25, 2009 19:25 UTC (Fri) by PaXTeam (guest, #24616) [Link] (8 responses)

> You do not submit a patch for the Linux kernel. Instead, an exploit is released, with a lot of fanfare indeed.

wow, lots of accusations there. let's see.

first of all, i didn't release let alone write this exploit, spender did, so you'll have to take that up with him, not me.

second, since you're seemingly unfamiliar with the Linux kernel development process, let me explain the situation briefly. your first task is to read Documentation/SubmittingPatches which is a file in the kernel source tree. in there you will find something called the Developer's Certificate of Origin (in section 12 as of .31). careful reading will reveal the following bits:

| then you just add a line saying
|
| Signed-off-by: Random J Developer <random@developer.example.org>
|
| using your real name (sorry, no pseudonyms or anonymous contributions.)

notice that anonymous contributions are explicitly not allowed (those words were actually added because of a patch of mine a few years ago). so there is the answer to your accusation.

> Selectively quoting Linus about the greatness of shame is not going to convince people that it is Linux security you are after.

why was that selective again? but more to the point, it seemingly works and makes people take a second look and tell something closer to the truth in turn. that can only be good for security, or do you believe that covering up security bugs is better?

> In fact, I am not even so convinced anymore that you are just plugging your own software at the expense of someone else's.

did you even read what i responded to? it was a suggestion about releasing our own security fixes as an alternative to the so-called 'stable' series (apparently that implies neither timely nor honest). obviously when we've already done that work for many years now, i'll not withhold the fact. as for the expense part, it sounds like as if you didn't like the taste of your own medicine.

fixed in v2.6.31.1

Posted Sep 26, 2009 10:45 UTC (Sat) by mingo (guest, #31122) [Link] (2 responses)

notice that anonymous contributions are explicitly not allowed (those words were actually added because of a patch of mine a few years ago). so there is the answer to your accusation.

I don't question your honesty, but FYI, your level of paranoia is reaching clinical levels.

If you don't trust others to know your real name, why do you expect millions of others to trust your anonymous word that the code you have contributed is indeed yours, with no legal strings attached? (such as an employer you work for currently and who thus does not know (and cannot know, due to the pseudonym) that you contribute in corporate time, etc.)

As far as it being added to the SOB requirements based on your case - that might simply be because in 99.99% of the cases people are actually proud of being mentioned in the Linux kernel source (it's even a good point for your career so generally considered stupid if you forfeit it), so this situation was simply not contemplated before?

Nevertheless i do always look at your fix patches and accept them (for subsystems i maintain) if they are good. I add my own signoff so i take responsibility for your patches, and credit you for the fix. I do think you are genuinely interested in (and worried about) Linux security, you are competent and capable, and I think you are doing good fixes.

As an example look at this upstream fix that you wrote:

e003208: x86: fix stackprotector canary updates during context switches

So there's absolutely no impediment to you contributing fixes to the Linux kernel, if you chose to do so. All it takes is to find someone with a real name who trusts you enough to take fixes from you.

(bigger patches are still a problem of course.)

the so-called 'stable' series (apparently that implies neither timely nor honest).

Do you have proof of (or even an indication of) that -stable is dishonest? I don't have such an indication, and i take honesty and security seriously.

Have you considered alternative possibilities, such as honest mistakes or lack of information? Or the possibility of maintainers being genuinely worried about and interested in a far wider spectrum of system stability than the important but (technically) narrow sub-set of security related bugs?

fixed in v2.6.31.1

Posted Sep 27, 2009 10:34 UTC (Sun) by nix (subscriber, #2304) [Link] (1 responses)

We've had this flamewar before. From those past flamewars, 'dishonest' in
PaXspeak appears to mean 'does not research every single patch to
determine if it fixes an exploitable hole, then pause the release while a
CVE is assigned and rewrite the changelog to include the CVE number and
exploit'. If a single hole lacks a CVE we get flames on LWN (why pick on
LWN, which is just reporting them? Search me. Maybe he wants as many
people as possible to dislike him. LWN used to have a pleasant collegial
comment atmosphere before these PaXTeam and Brad Spengler came along. Now
it often feels like alt.flame: I certainly want a killfile again.)

So we have PaXTeam on one side damning the stable team for not doing all
*that* and Brad on the other damning them for not releasing quickly. You
can't win.

fixed in v2.6.31.1

Posted Sep 27, 2009 10:35 UTC (Sun) by nix (subscriber, #2304) [Link]

Allow me to thank Greg et al for continuing with stable maintenance
despite getting 'thanks' and accusations of incompetence, lying, and
malice with virtually every release. I know I'd have given up long ago and
told these horrible people to do the releases themselves if they cared so
much about them. Greg is more mature than that thankfully. :)

fixed in v2.6.31.1

Posted Sep 27, 2009 7:21 UTC (Sun) by hppnq (guest, #14462) [Link] (4 responses)

did you even read what i responded to? it was a suggestion about releasing our own security fixes as an alternative to the so-called 'stable' series

busterb suggested:

Brad, why don't you and PaXTeam just release an intermediate .x stable kernel patch series to fix the problems you find?

You could set up a git repository with the fixes you find from which the -stable maintainers could pull, instead of wasting your time and energy on arguing with people over things that have nothing to do with security.

As for real security: I ran the perf_counter exploit through Brad's framework on my pristine, default Ubuntu distribution and it fails right after it killed pulseaudio, so it was only a bit annoying.

fixed in v2.6.31.1

Posted Sep 27, 2009 12:32 UTC (Sun) by spender (guest, #23067) [Link] (3 responses)

You ran the wrong exploit then. I made two versions of the exploit, one that requires a NULL mapping, and one that doesn't. The one that doesn't won't do anything at all with pulseaudio.
./run_nonnull_exploits.sh and chose "Ingo m0wnar"

Run the wrong exploit, obviously get the wrong results ;)

-Brad

fixed in v2.6.31.1

Posted Sep 27, 2009 14:06 UTC (Sun) by hppnq (guest, #14462) [Link] (2 responses)

I was just pulling your leg, my system -- pristine, default Ubuntu -- is not running 2.6.31 in the first place. ;-) You may think this is silly, but I think that this observation makes a lot of sense when it comes to "real" security. The first question, when doing a vulnerability assessment, is not "Is it remotely exploitable?", it is "Are we running that stuff?". So in this case, anyone who is running 2.6.31 with perf counters on a system that handles untrusted user data is likely to be vulnerable. Not too many people, I should think, fall in that category without knowing it.

That said, these bugs need to be fixed. But your sense of urgency and your tone are easily ignored -- sometimes much too easily -- and therefore, the question whether you plan to ever cooperate better with the kernel developers remains valid and to the point. If you would strip all communication of its unneeded emotion, and for instance, simply set up a repository that contains the fixes you find during your research, this could prove to be very fruitful to all parties.

fixed in v2.6.31.1

Posted Sep 27, 2009 14:19 UTC (Sun) by spender (guest, #23067) [Link] (1 responses)

As I mentioned elsewhere, anyone using a distro config will have perf counters automatically enabled without knowing it (since it gets turned on if PROFILING=y, which the vendors all enable).

As for the repository:
http://grsecurity.net/test/grsecurity-2.1.14-2.6.31.1-200...

our code has always been open for anyone to pull fixes from.

PS: what system doesn't handle untrusted user data?

-Brad

fixed in v2.6.31.1

Posted Sep 27, 2009 20:32 UTC (Sun) by hppnq (guest, #14462) [Link]

As I mentioned elsewhere, anyone using a distro config will have perf counters automatically enabled without knowing it (since it gets turned on if PROFILING=y, which the vendors all enable).

You mean, in the case where users download 2.6.31 and blindly reuse a non-matching distribution-supplied config? That is not a safe practice. Otherwise, I have to assume that either the distribution or the user knows what is turned on, with what impact.

our code has always been open for anyone to pull fixes from.

Incorporating fixes in your own patches is not the same as fixing bugs in the vanilla kernel. (The link doesn't work, so I am assuming it points to a full-blown grsecurity patch, which is not part of the vanilla kernel.) That said, you seem to write excellent code, and I wish more of your efforts would end up in the kernel in a way that satisfies you, the kernel developers, and Linux users alike. A git repository seems like a good idea. It's not so much the code contribution that's important (crucial fixes will always be picked up), but the cooperation it would help shape.

PS: what system doesn't handle untrusted user data?

Well, in the office we of course work with data classifications (and there is no "completely trusted" one), but here I would say a home or test system that is not connected to a public network fits the bill. When referring to systems that do handle untrusted data, I was thinking of Internet facing systems that, for instance, allow FTP or shell access, and I couldn't think of a reason why an admin would turn on profiling on them, or why the admin would not have a security policy based on the assumption that these systems will be hacked to pieces sooner or later.