"Stable" kernel 2.6.25.6 [LWN.net]

"Stable" kernel 2.6.25.6

Posted Jun 10, 2008 10:24 UTC (Tue) by PaXTeam (subscriber, #24616) [Link]

you might be right if it weren't for some (sad) real life *facts* of life. what facts? Andrew
and a bunch of other top people you find on lkml (Linus included) are directly and consciously
complicit in this downplaying of bugs. where else do you go to complain? at least LWN has the
advantage of being a lot less noisy than lkml and being read by regular users as well.

"Stable" kernel 2.6.25.6

Posted Jun 10, 2008 16:12 UTC (Tue) by iabervon (subscriber, #722) [Link]

All that actually matters is convincing Greg KH, who is solely responsible for writing the
line in the announcement that says how important the updates are. If you send him exploits for
a random selection of the bugs fixed here, he's likely to change how he reports such things in
the future.

"Stable" kernel 2.6.25.6

Posted Jun 10, 2008 16:54 UTC (Tue) by PaXTeam (subscriber, #24616) [Link]

do you know how the -stable tree works? the rule is that if a bug is present in the Linus
tree, it must be fixed there *before* it goes into -stable. so if there's a problem with
commit messages in -stable it's because there is a problem with them in the Linus tree already
(incidentally, Brad's links pointed to the Linus tree as well, not that of -stable). so one
more time: the problem is the policy held at the very top of linux development, it is not
about accidentally mislabeled bugs but a seemingly conscious strategy. let me give you another
example:
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-...
and you tell me if it was just too hard to call it what it is (buffer overflow as it was also
clearly spelled out in the linked bugzilla entry) or someone tried really hard to evade
grepping the changelogs.

and frankly, i hope you didn't seriously suggest that it has to take an exploit to convince
someone that downplaying of commit messages is not acceptable.

"Stable" kernel 2.6.25.6

Posted Jun 10, 2008 18:40 UTC (Tue) by nix (subscriber, #2304) [Link]

This looks like conspiracy-mongering to me.

Your complaint appears to be that Linus isn't spending his time looking at 
every single commit he merges, determining if it could be a security hole, 
and rewriting/rejecting it until the changelog mentions that.

Am I the only person who thinks this would be an enormous waste of his 
time, and that if the changelogs don't mention such things, perhaps you 
could pin the blame on the people who *wrote* the changelogs, and who 
could have many reasons for not mentioning that (god knows not all crap 
commit logs are crap because of some enormous conspiracy).

"Stable" kernel 2.6.25.6

Posted Jun 10, 2008 20:41 UTC (Tue) by PaXTeam (subscriber, #24616) [Link]

> Your complaint appears to be that Linus isn't spending his time looking
> at every single commit he merges, determining if it could be a security
> hole, and rewriting/rejecting it until the changelog mentions that.

nice strawman there, but i didn't say anything like that. i was talking about commits that
went in with the full knowledge and agreement of said kernel developers.

"Stable" kernel 2.6.25.6

Posted Jun 10, 2008 21:36 UTC (Tue) by nix (subscriber, #2304) [Link]

The vast majority don't, including at least two of the three you 
specifically called out (Linus doesn't look at every commit to cpufreq, 
and the other one is networking: expecting David Miller to rewrite commit 
logs for everything in the net subsystem is similarly madness: 
particularly if it's a security hole, delaying its propagation to fix 
*commit logs* is surely not sensible).

"Stable" kernel 2.6.25.6

Posted Jun 10, 2008 22:15 UTC (Tue) by PaXTeam (subscriber, #24616) [Link]

> The vast majority don't,

another strawman, i never said that the vast majority did.

> including at least two of the three you specifically called out

i didn't call out three, only one. and that one is hard to believe to have escaped attention
given it was already public in the bugzilla with its full impact properly described. *someone*
did everything to not mention 'buffer overflow' there and engaged in creative rewording, you
can't dance around that fact. whether it was discussed on the kernel security list is anyone's
guess (why aren't they public after a while anyway?).

> (Linus doesn't look at every commit to cpufreq, and the other one is
> networking: expecting David Miller to rewrite commit logs for everything
> in the net subsystem is similarly madness: particularly if it's a
> security hole, delaying its propagation to fix *commit logs* is surely
> not sensible).

you keep talking about this need/expectation to rewrite commit logs, i don't know where you
got that from, certainly not from me. i always talked about commits that went in (attention,
i'm about to repeat myself) with the full knowledge and agreement of said kernel developers.
that means that they were well aware of the security impact of the bug and did still choose to
omit that fact. i will throw in another one for you and you tell me the history behind it:
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-... .

"Stable" kernel 2.6.25.6

Posted Jun 10, 2008 23:04 UTC (Tue) by nix (subscriber, #2304) [Link]

You're saying that specific commits which are 'obviously' (to you but 
perhaps not to their authors) security holes are not being noted as such. 
Then you say that this is a 'policy held at the very top of Linux 
development'. This is ludicrous fearmongering at best, given that the 
commits in question were not written by the people at the very top of 
Linux development, and that it is silly to expect them to rewrite the 
commit messages or reject the changes because their commit messages are 
wrong, and even sillier to expect them to review every change to see if it 
is a security hole *specifically so they can complain about or fix the 
commit messages*.

(I can't figure out what else you might expect them to do: given that 
you've complained about the commit logs and the 'very top of Linux 
development', it is reasonable to believe that you expect those at the 
very top of Linux development to reject commits fixing security holes or 
fix their commit logs themselves. If that's *not* what you mean, please be 
clearer, because I don't have a clue what else you could be talking 
about.)

"Stable" kernel 2.6.25.6

Posted Jun 10, 2008 23:26 UTC (Tue) by PaXTeam (subscriber, #24616) [Link]

you say a lot of things i never said, so forgive me if i save time by not quoting every
sentence of yours again ;). i thought i was quite clear already but let me spell it out again:

1. there's no problem with commits not telling the whole truth if noone was the wiser at the
time of the commit.

2. there *is* however a problem with commits where the impact of the bug fixed by said commit
is known *yet* it's not mentioned at all or cleverly worded so that noone will notice it by
just glancing at the commit looking for keywords.

the second commit i cited falls into category 2 (and we have yet to see the story of the first
one, not that there was any shortage of them i could have cited) and what i expect from the
kernel devs is to stop contributing to that category.

"Stable" kernel 2.6.25.6

Posted Jun 11, 2008 0:11 UTC (Wed) by nix (subscriber, #2304) [Link]

Badly worded, sure: a bugzilla link is not a good changelog entry. But we 
have plenty of bad changelog entries from people who haven't yet got up to 
speed or are in a hurry (perhaps because, well, they just found a nasty 
crash bug and want to fix it sharpish). Bad changelog entries do not 
conspiracies or intentional concealment of security holes make. You're 
assuming malice without cause.

"Stable" kernel 2.6.25.6

Posted Jun 11, 2008 0:30 UTC (Wed) by PaXTeam (subscriber, #24616) [Link]

how do you know i don't have cause? when you see how
https://bugzilla.redhat.com/show_bug.cgi?id=194215#c2 turns into this
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-... commit, there is cause.

"Stable" kernel 2.6.25.6

Posted Jun 11, 2008 2:51 UTC (Wed) by tialaramex (subscriber, #21167) [Link]

As spooky as it undoubtedly seems to you, the commit messages aren't addressed to PaXTeam (are
you actually an individual or a corporate entity?). Unless stated elsewhere they aren't
intended as a functional description of the changes (read the source for that), a summary of
the security consequences, an entry into a poetry contest, a running commentary on the
developer's lunch plans or a source of entropy for a random number generator. On occasion they
may serve as one or more of those things, but generally, they're a memo to the developers
working on that code. So if they fix a line which incorrectly sets foozle to NULL, don't be
surprised if the commit message is "don't touch foozle in baz" rather than "Potential security
flaw fixed, NULL dereference PaXTeam take note".

If you would like to provide blow-by-blow security commentary for every merge, I suspect at
least some people on LKML would welcome it, perhaps to a different list.

The general situation is that most bugs involving privileged code will have security
consequences. Since any large system, including all the popular general purpose operating
systems, will have such bugs, this is not very helpful information.

"Stable" kernel 2.6.25.6

Posted Jun 11, 2008 8:49 UTC (Wed) by PaXTeam (subscriber, #24616) [Link]

you *still* don't understand what i was saying. for the Nth time: i am *not* talking about
commits lacking security impact information when it is *not* known at the time of the commit
(although one would think that in this age at least a few exploitable bug classes should be
known to kernel developers and be recognized and thus documented as such in the commit).

what i *am* talking about is commits where the security impact is already *known* yet the
commit very deliberately does *not* mention it or tries to downplay it as much as possible.
you want proof? the last one i mentioned, look at
https://bugzilla.redhat.com/show_bug.cgi?id=194215#c0 in the Red Hat bugzilla. the actual
author of that one was not Marcel Holtmann but Paul Mackerras himself. the problem? Marcel did
not quote the full email, he omitted one of the last paragraphs, as you will see below, for a
good reason (good for him and other kernel devs trying to evade full disclosure, something
they supposedly support, at least when it's for PR purposes, that is):

I'd appreciate advice about the best way forward here, and to what
degree we should be candid about the reason for the patch in the
commit message when it gets committed to Linus' git repository.

and candid it became indeed. if you want more juicy stuff, go ask your supposedly honest
kernel devs for the kernel security list archives.

PS: i hope you're not a kernel contributor, you certainly haven't got a clue about what
belongs into a commit message. you also don't have a clue about what bugs have a security
impact, it's most certainly not *most bugs*. but i understand, what i've been saying sounds
unbelievable to you, so it's easier to attack the messenger, instead of actually checking out
the situation for yourself.

"Candid" doesn't mean hidden

Posted Jun 11, 2008 9:48 UTC (Wed) by tialaramex (subscriber, #21167) [Link]

What you're talking about is commits which lack security impact information.

I explained that commit messages aren't specifically called out as a medium for security
impact information. So it doesn't matter if they don't have security impact information, any
more than it matters if they don't rhyme.

And you've responded to inform me that I "still don't understand" by which, so far as I've
been able to discover you mean simply that I didn't agree with you about how terrible this is.

Candid means frank and open. You may be confused by seeing it used in the context "candid
camera" or "candid photographs" which refer to the fact that the subject isn't aware that
they're being watched and so they act candidly. The developers involved decided /not/ to be
candid about the bug in your example. You're cross about that, we get it. But you still
haven't given me a reason to care.

I've encountered this claim that only a handful of bugs have security impact before. It's
disappointing, particularly when it comes from someone who claims to actually care about
security. I just picked at random a harmless sounding item from the latest stable tree commit
logs

"eCryptfs: remove unnecessary page decrypt call"

... sounds like it's probably a performance fix? Let's take a look. Oh, the bug overwrites
mmap() changes, undoing them. The kernel promises not to do that to you, breaking the promise
converts to a security impact because userspace programs that rely on this promise are now
broken and their security assumptions are violated.

"Candid" doesn't mean hidden

Posted Jun 11, 2008 13:04 UTC (Wed) by PaXTeam (subscriber, #24616) [Link]

> "Candid" doesn't mean hidden

i never said it did. what Paul was asking the kernel devs was about how honest he should be in
the commit. the mind boggles at that very moment as this was on a list whose declared
disclosure policy is full-disclosure (see Documentation/SecurityBugs if in doubt). then what
do we get in the commit? complete omission of the more serious security impact (arbitrary
kernel memory read). you call that whatever you want, i call it dishonesty. it's not terrible
per se, it's just it's in direct contradiction to what was communicated to the public before
and what this supposedly superiour development model is so proud of among others. as they say,
you can't have it both ways.

also, this one at least got a CVE that, as far as i know, you can't say about that ptrace bug
i posted somewhere above (not even the corresponding -stable commit mentions anything at least
yet the -stable guys were fully aware of its impact - this by the way directly contradicts
Chris Wright's claim in http://marc.info/?l=linux-kernel&m=121312896523183&... ).

> I explained that commit messages aren't specifically called out as a medium for security
impact information.

says who? 'tialaramex' or is it the agreed-upon (read: documented for the public) commit
policy of kernel developers?

as for what percentage of kernel bugs have a security impact (or did you mean something other
than 'kernel code' by 'privileged code'?), i stand by my claim that it's not *most* of them (i
never said 'only a handful' either, nice strawman though). this is based on my personal
experience.

PS: congratulations on finding yet another silently fixed potential security bug.

"Candid" doesn't mean hidden

Posted Jun 11, 2008 17:33 UTC (Wed) by tialaramex (subscriber, #21167) [Link]

If you use words in the opposite sense to their correct meaning without any trace of irony
then people certainly are entitled to assume that's because you don't actually know what they
mean. I see that a dose of your own medicine doesn't suit you.

You've several times provided links which actually don't say what you claim they do. This time
you suggested that the Linux security list has a policy of full disclosure, when in fact the
document you mentioned uses weasel words to declare that disclosure should happen "as soon as
possible" which means nothing at all.

No congratulations are necessary. I did not find a bug, I just read a randomly chosen commit
log. Anyone can do it. Maybe more people should.

"Candid" doesn't mean hidden

Posted Jun 11, 2008 21:11 UTC (Wed) by PaXTeam (subscriber, #24616) [Link]

i see you're getting sidetracked into various things which is a good sign that i raised valid
points you don't contest.

> You've several times provided links which actually don't say what you
> claim they do.

this is a blanket statement without any proof/explanation. elaborate please.

> This time you suggested that the Linux security list has
> a policy of full disclosure, when in fact the document you mentioned
> uses weasel words to declare that disclosure should happen "as soon as
> possible" which means nothing at all.

said document talks about 'full disclosure', not mere 'disclosure' (nice attempt but you got
caught). second, if you cared to read the comment at the bottom of this thread, you'd realize
that said document doesn't at all raise the issue of the amount of disclosure, only its
potential timing. so we're in agreement that the disclosure policy is 'full disclosure' (and
which is what wasn't observed in several cases i cited). i've said nothing else and you
haven't contradicted it either. and frankly, if you're not an active party in all this
non-or-half-assed-disclosure process, you have zero idea about what you're talking.

"Stable" kernel 2.6.25.6

Posted Jun 12, 2008 2:47 UTC (Thu) by tialaramex (subscriber, #21167) [Link]

Sorry everyone, I should not have fed the troll.

"Stable" kernel 2.6.25.6

Posted Jun 12, 2008 3:14 UTC (Thu) by spender (subscriber, #23067) [Link]

Clever: when your lies are uncovered (see above post re: "full-disclosure"), you sneak out
while claiming the moral high ground.

If you're looking for a troll, you need only look in the mirror.

-Brad

"Stable" kernel 2.6.25.6

Posted Jun 12, 2008 19:03 UTC (Thu) by nix (subscriber, #2304) [Link]

This thread really is a confluence of the pointlessly combative, isn't it? 
There's not an instant's consideration given to the possibility that any 
disliked event might *not* be due to malice, `lies', `suppression'... 
sheesh.

"Stable" kernel 2.6.25.6

Posted Jun 12, 2008 20:49 UTC (Thu) by spender (subscriber, #23067) [Link]

One of us is privy to certain insider information, and one of us is not.  I'm quite sure you
don't know what goes on in the private kernel security mailing lists (I'll include vendorsec
in on this as well).  After a certain period of time, there's no reason to keep the
discussions of these lists private unless the members are embarrassed of the information that
would be found.

Go ask Linus if he didn't intentionally decide to cover up that ptrace bug mentioned in this
thread.  I have proof that he did, and that as a result of the coverup no CVE was assigned and
the fix was not backported to distro kernels.

Go ask the members of the vendorsec list if they haven't been covering up a DoS on the Itanium
architecture reported by Intel over a year ago, for which two fixes were provided.  The flaw
is in hardware so all kernels are vulnerable, and the worst part in this situation is that the
vendorsec members collectively agreed not to provide either of the two software fixes for the
bug.

You claim there's been no proof, but yet I've already pointed out in my previous exploit a
clear coverup of what was at least known to the committer to be a local DoS (I demonstrated
that it was trivial arbitrary code execution), but the commit message said only "fix
sys_tee()" and no CVE was created.

So many of those you blindly trust are involved in coverups like these.  They erroneously
think they can get away with it because they don't think anyone's watching the watchers.  So
people like Chris Wright will claim in public that there are no coverups, but he knows as we
know that he's taken part in it out of the public eye.

So I seriously encourage you or anyone else to go ask the people I've mentioned about the
specific events I've mentioned.  Send this to LKML if you wish.  I'll give them the
opportunity to give their side of the events in question.  It should be clear to from the
details I've already released here that this is not a frivolous accusation, and that I'm aware
of things they hoped the public would never be aware of.  If they choose not to, or continue
to lie about what happens behind the scenes, I'll release what information I have.  I think
they'll be surprised by what is known among certain individuals about what they're doing.

I'm probably making enemies from all sides by even mentioning the details above, but in the
end a more honest security leadership will only help users of Linux.  I'd also like to
reiterate the importance of making the discussions of the private kernel security lists public
after an agreed upon acceptable amount of time, so that this culture of coverups cannot be
allowed to grow any more.

-Brad

"Stable" kernel 2.6.25.6

Posted Jun 12, 2008 22:02 UTC (Thu) by nix (subscriber, #2304) [Link]

OK, you're resorting to argument from authority. There's absolutely no 
point talking to you at all, is there?

(And no, I'm not going to go around randomly accusing kernel hackers of 
coverups and conspiracies. You're the one arguing that point and referring 
to secret evidence to 'support' your claim, not me.)


(I'd agree that the various private security lists should have delayed 
public archives. Transparency is good, and the only reason to keep those 
lists private is to prevent the bad guys from responding before the 
distros do, so after they've responded there's no point keeping the 
traffic secret anymore.)