Stable kernel 2.6.25.7 released [LWN.net]

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 1:46 UTC (Tue) by dlang (subscriber, #313) [Link]

like it or not, despite the fact that SELinux is in the kernel, most kernel folks (like most
other admins) don't use it. As such the fact that root can do something nasty is not
considered a critical/exploitable fix

as for the one where you complain about no CVE number, is there a number allocated for this,
or are you complaining becouse the person who found and fixed the problem didn't jump through
whatever hoops are needed to create one first?

for the one that's in Linus' tree but didn't get into -stable, did anyone send it to -stable
and they rejected it, or is it one of the many patches that got upstream that the author
didn't send to -stable?

as for the one in vendorsec, it is disappointing that vendors are taking this long to deal
with it.

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 2:32 UTC (Tue) by spender (subscriber, #23067) [Link]

I'm not sure when the last time you used Linux was, but SELinux is enabled by default on
Fedora and RHEL (and others I'm sure, but I haven't checked).  Your usage argument doesn't
apply anyways, since it doesn't come into play when labeling bugs as security-related for
obscure hardware which may only affect a tiny number of users (much much smaller than the
number of users of SELinux).  Either these bugs need to be taken seriously (and they have in
the past, so I'm not sure why this one was declared a non-issue) or people need to be told
that SELinux is completely worthless against preventing complete compromise as any root-based
vulnerability that can totally subvert its security isn't considered a security bug and thus
won't be backported to any distribution-maintained kernel.

Generally when a kernel bug is identified as security-relevant, someone creates a CVE for it.
The problem in this case was the committer (Al Viro) knew of the security relevance of the bug
but covered it up in his changelog, which resulted in it getting covered up at the -stable
level.  Now tell me how any distribution is supposed to know to backport this security fix if
for this 2.6.25.7 release, as with the 2.6.25.6 release with silently fixed vulnerabilities,
no security implications whatsoever are mentioned?

I don't know about the one that didn't make it into -stable, I was just pointing out that it
belongs to the class of trivially exploitable bugs.  I'm sure when it makes it into -stable
(if it does), its exploitability will again be covered up.

-Brad

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 3:02 UTC (Tue) by bojan (subscriber, #14302) [Link]

> Now tell me how any distribution is supposed to know to backport this security fix if for
this 2.6.25.7 release, as with the 2.6.25.6 release with silently fixed vulnerabilities, no
security implications whatsoever are mentioned?

Excellent point. I actually asked Fedora kernel maintainers to label the latest F9 kernel as a
security fix (I went with the official .5 explanation, although LWN discussion about .6 made
me do it):

https://admin.fedoraproject.org/updates/F9/FEDORA-2008-5308

Although I'm neither security expert nor kernel developer, that double-free in the log made me
think that there could be security issues being fixed. I'm not really sure what's going on
with all this, but I reckon it's always better to err on the side of caution and open CVEs
etc.

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 6:15 UTC (Tue) by dlang (subscriber, #313) [Link]

I have been using linux as my desktop full time for the last 11 years. I am aware of the fact
that RedHat chooses to enable SELinux by default, but I am also aware of the fact that many
(if not most) sysadmins (as opposed to casual users) end up disabling SELinux as a fairly
routine thing as they start to configure the boxes to be something other then the defaults
that RedHat ships. and from various discussions on the kernel mailing list it's clear that the
most of the kernel developers do as well (those that have RedHat based systems).

the useage argument is that since the kernel developers don't use it, they don't have the
mindset to take it into account when evaluating the severity of bugs, and therefor something
that requires root access to do is not considered that bad as root can already get around the
limitations (even on a RedHat system root is the account that can disable SELinux, so
pretending that SELinux can protect you from rootis wishful thinking)

distribution-maintained kernels will backport or not backport patches on teir own criteria,
not matter what the kernel devs decide to put in a -stable kernel (and note that this patch
did make it into stable, it's not that they didn't fix the problem, it's that you are
complaining that the commit message didn't identify it as a security problem)

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 7:28 UTC (Tue) by nix (subscriber, #2304) [Link]

Oh, in our previous go-round he was quite clear that the problem wasn't 
the commit message... after spending half a dozen messages whining about 
nothing else.

I'm still unclear what the problem actually *is*. There's meant to be some 
sort of huge conspiracy, perhaps focused around not doing whatever Brad 
says the instant he says it (it would help if he didn't say it in such a 
profoundly unpleasant fashion.)

(And I thought I was bad at diplomacy. At least I have interaction modes 
besides `J'accuse' and don't try that one first. If you want to get people 
to do something, telling them they're all in a conspiracy and 
intentionally attacking the general public(?) is not the way to go.)

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 9:32 UTC (Tue) by PaXTeam (guest, #24616) [Link]

> Oh, in our previous go-round he was quite clear that the problem wasn't 
> the commit message... after spending half a dozen messages whining about 
> nothing else.

would you stop distorting the facts? first, it was me who posted quite a few messages, not
Brad, second, the problem *is* with the commit messages in that they intentionally omit
important and known-to-the-developers security information. you were given several examples,
not one of which you contested. it's funny how you keep running around with a huge post sign
shouting conspiracy whereas noone of us said anything like that. what we did talk about was
good old human dishonesty hidden in an unaccountable mailing list. as you quite rightly said
in your last post on that other thread, you should not be talking to us as we can't change the
situation. you should be asking questions to your kernel developers and make them accountable.
unfortunately it seems it's easier to pull strawmen than actually trying to get to the bottom
of this problem.

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 13:47 UTC (Tue) by nix (subscriber, #2304) [Link]

Ah, right, you were talking in such similar ways that I confused you :/ However, you're
assuming that a large number of people are each acting dishonestly for no obvious gain in
different places, with the strong implication of coordination (in that they have all chosen to
be 'dishonest' in exactly the same way: by not mentioning things you consider a security hole
in their commit logs). That's called a conspiracy theory. If you remove the assumption of
dishonesty, it's simply human error, or perhaps that they have lower thresholds for defining
something a security hole than you do. (It would be a miracle if this were not true, given
that most of these people are not security paranoids by nature, and thus are almost bound to
have lower thresholds for such things than you are!)

I'm not contesting the facts: I'm contesting your unnecessarily unpleasant interpretation of
them.

In fact your interpretation makes no sense at all: why would people spend time coordinating to
hide security holes when knowingly doing that could have no consequence other than to harm the
reputation of the system they're working on? Doing that would be ridiculous. Ergo, they aren't
doing that: there is no magically coordinated decision to fix security holes while hiding them
by the single means of describing them differently in commit logs, no conspiracy, no bad
intent. Simply differently-set thresholds.

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 14:47 UTC (Tue) by PaXTeam (guest, #24616) [Link]

i'm not assuming or interpreting anything, i speak of facts i have. for one, it's not a large
number of people, it's the membership of secret lists which by its nature is quite limited.
second, it doesn't take a lot of coordination when you're within your normal social circle
anyway (the same kernel devs who interact on lkml, at conferences, real life, etc), it's
normal human behaviour to stick to the flock lest you get ejected. so no, this is not a
conspiracy (you've been caught again ;).

speaking of facts, what do you have to say about the Paul Mackerras quote and its visible
result in the commit message? what is your interpretation? why was he even raising the point
of how candid he should be in the commit?

> In fact your interpretation makes no sense at all: why would people
> spend time coordinating to hide security holes when knowingly doing that
> could have no consequence other than to harm the reputation of the
> system they're working on? Doing that would be ridiculous.

why would there be consequences when all of this can take place outside of the public eye, in
a completely unaccountable manner?

> Ergo, they aren't doing that:

and you determined that by...? oh right, you didn't. why don't you go ask for the mailing list
archives? that would surely answer all questions, do you agree? and if there's nothing bad to
hide in there, there shouldn't be any objections to getting them.

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 15:40 UTC (Tue) by nix (subscriber, #2304) [Link]

Er, if the holes get exploited, of *course* there's a consequence. And if there's no
possibility that the holes would be exploited, then modifying the commit messages to conceal
the security-related nature of the commits makes no sense.

You're blatantly contradicting yourself now. I've had enough of this thread.

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 16:09 UTC (Tue) by PaXTeam (guest, #24616) [Link]

> Er, if the holes get exploited, of *course* there's a consequence.
> And if there's no possibility that the holes would be exploited, then
> modifying the commit messages to conceal the security-related nature of
> the commits makes no sense.

i have absolutely no idea what you are talking about now. probably another of your strawmen,
but just in case: you were trying to speculate why it makes no sense to downplay/hide security
information in commit messages (mind you, a few posts below you argued the *opposite*, so much
for contradiction ;) since that would only endanger the reputation of their work. except you
forgot the little fact that such coverup took place in a secret list, hence there was no
danger of exposure, on the other hand there was a perceived advantage of not getting bad PR
about the many silently fixed security bugs. and now, out of the blue, you come with this
commit message modification and how it makes no sense for bugs that don't have a security
impact. guess what, if a bug cannot possibly be exploited then it doesn't have a
security-related nature. that's a tautology and i'm not sure what you tried to say with that.
incidentally, we weren't talking about bugs without security impact either. another strawman
down ;).

> You're blatantly contradicting yourself now.

hmm, where? you're making no sense to me now, sorry. but feel free to elaborate. of course if
you just wanted a cheap cop-out, let this rest.

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 18:07 UTC (Tue) by nix (subscriber, #2304) [Link]

As I said, I'm dropping this because we're talking past each other. ('Cheap cop-out'.
Charming.)

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 3:39 UTC (Tue) by gregkh (subscriber, #8) [Link]

> In this issue of "we have no clue what we're doing," we have:

<snip>

Hm, if you have questions/comments/suggestions about our kernel releases, then please, send us
email and cc: the mailing lists.

I will be glad to discuss this there, in public with all of the developers, and put to rest
any issues raised about these releases, how we are doing them, and any potential problems that
you feel are still outstanding and not yet addressed.

But not on a web site comment system, that's just not where the kernel developers do their
work, sorry.

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 4:26 UTC (Tue) by avik (subscriber, #704) [Link]

He will immediately get 13 (unwarranted) "Do not feed the troll" replies.

Negative comments about the Linux process on lkml receive very negative feedback, especially
if they come from someone who is not a regular contributor.

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 6:08 UTC (Tue) by wtarreau (guest, #51152) [Link]

> He will immediately get 13 (unwarranted) "Do not feed the troll" replies.
> Negative comments about the Linux process on lkml receive very negative
> feedback, especially if they come from someone who is not a regular
> contributor.

Then he can post privately to the -stable team. But I think that unfortunately, Brad is not
trying to improve Linux but to demonstrate it's a pile of crap, maybe in order to promote
security add-ons such as grsecurity. But it's doing no service to him either because acting
this way will scare away people who look for a reliable system.

IMHO, he would be of great help if he would accept to be on the -stable reviewers list. He
will be able to remind all of us about security impacts when not perceived, suggest missing
patches just in time for inclusion, he will get general gratitude for this useful work and
will feel proud of really improving the kernel security.

Attacking the work of the stable team as he does now is unfair. The stable series is the thing
which made 2.6 usable, and I know it's a tough work for Greg and Chris. Suggesting and helping
would be more welcome than whining.

And it's not like Brad does not have the skills to do so, which is really a shame.

The core issue

Posted Jun 17, 2008 6:29 UTC (Tue) by bojan (subscriber, #14302) [Link]

For me, the main issue here is this: are these things real security problems or not?

- if yes, CVEs should be opened and logs clearly marked
- if no, then someone should should let us know that they are not

I'm really not in the position to evaluate validity of Brad's claims, but from what I saw in
other discussions, other more qualified people usually cannot refute them.

The core issue

Posted Jun 17, 2008 7:21 UTC (Tue) by wtarreau (guest, #51152) [Link]

> For me, the main issue here is this: are these things real security problems or not?

It's not as simple. Security means everything and nothing at the same time. There are people
who consider security only about risk of intrusion. Other people consider the risk of remote
or local DoS. Others the risk of data leak. If we want to be picky, everything which can
permit a non-privileged user to cause a malfunction resulting in a degradation of performance,
integrity, availability, confidentiality or traceability is a security issue.

As you can see, the smallest driver bug can become a security issue. If someone plugs an
usb-serial cable into a machine and makes it crash, it is a security issue if it is normally
not allowed to enter this machine.

Brad tends to be particularly picky about security issues (and it is good to have someone who
really sees a lot of risks as he does). Other people like Al Viro consider that anything which
requires root to impact the security is not a problem, because once your box is rooted, you
have lost. These differences explain why a lot of bugs are not marked as security issues and
are complained about afterwards.

Another example I reported a long time ago : make a chroot, mount /proc into it, install a
shell there, chroot overthere, then "cd /proc/1/cwd". Bingo, you're out of the chroot. Some
people say that since this "cd" already requires to be root, it's not an issue. Others
(including myself) consider it is an issue. It's just a matter of point of view unfortunately.

Overall, I think the security issues are correctly taken by the middle chain (2.4 and
2.6-stable), but the fact that information sometimes gets lost at the starting point makes it
difficult to dig for the whole thing. It depends on who reported the issue and how it was
merged into mainline in fact. Sometimes, big security fix will get immediately merged, even
before a CVE gets assigned. But those ones are not the most difficult to track.

The most difficult ones are the ones which require special knowledge in a specific area and
which are not covered as security problems by the person who initially fixes them (by lack of
knowledge too). This is often the case with drivers.

The core issue

Posted Jun 17, 2008 10:47 UTC (Tue) by PaXTeam (guest, #24616) [Link]

> It's not as simple. Security means everything and nothing at the same
> time.

Willy, please do read the previous discussion before commenting. the problem isn't that people
are unable to determine whether a given bug has a security impact or not. that is a separate
issue and is not the point raised now. the problem we exposed is that of covering up security
impact information when it *is* already known to the kernel developers. you're privy to that
information yourself and in silent agreement with that policy, don't pretend you don't know
(care to explain to the rest of the world that itanium hardware bug still unfixed after 2
years?).

> There are people who consider security only about risk of intrusion.
> Other people consider the risk of remote or local DoS. Others the risk
> of data leak. If we want to be picky, everything which can permit a
> non-privileged user to cause a malfunction resulting in a degradation of
> performance, integrity, availability, confidentiality or traceability is
> a security issue.

and where's the problem? you simply include the relevant info in the commit and let the users
decide which ones they're interested in. the current problem is that by omitting or
obfuscating any such info makes it *impossible* to quickly select the interesting commits.
you're not improving linux security by making it hard for users to make decisions.

> Overall, I think the security issues are correctly taken by the middle
> chain (2.4 and 2.6-stable),

once again, to stress the point: no, they are *not* correctly handled. ask Chris Wright about
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-... .

> but the fact that information sometimes gets lost at the starting point
> makes it difficult to dig for the whole thing.

it doesn't get 'lost', it gets intentionally omitted. and it's not sometimes but pretty much
every time now. case in point,
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-... .
knowing Ilja's past work, i can't imagine he didn't report this as a bug with security impact
(not that it's hard to figure out even if he didn't) yet there's no mention of it in the
commit, not to mention a backport to -stable.

The core issue

Posted Jun 17, 2008 19:08 UTC (Tue) by wtarreau (guest, #51152) [Link]


> Willy, please do read the previous discussion before commenting.

I did, and I do not agree with what was said, it is exagerated.

> the
> problem isn't that people are unable to determine whether a given bug
> has a security impact or not. that is a separate issue and is not the
> point raised now. the problem we exposed is that of covering up security
> impact information when it *is* already known to the kernel developers.

Well, I know that Linus does not like to give too much information in
the commits because he believes it will help the bad guys, reason why
he does not add the methods to reproduce the problems nor the CVE when
he does the commits himself. I don't share his opinion here, but that's
personal taste, and after all, it's *his* project.

You have to keep in mind that most often, a commit is a summary of 10-20
mail exchanges with reproducers, tests and results. You cannot put all
that in a commit. OK Linus tends to go to extremes (and believe me, it's
hard for -stable and I to follow those bugs). But I don't think there are
many more people oversimplifying the commits in such circumstances.

Both -stable and I keep the original message when backporting, and look
for any existing CVE information then add it. I have already missed CVE
numbers before committing. It happens.

What I'm really sicked about is the accusations of "general agreement to
deliberately hide security informations from commits". That's false IMHO,
really. And in fact, examples given by Brad were on public lists, so that
does not make any sense.

What we agree about is not to divulgate information when joining the list.
That's the minimalist requirement you can expect from members who you will
send sensitive information. There are times you would like to shake a bit
the tree to get something to speed up, and you cannot. Those moments are
a bit frustrating, but you agreed about the initial rules. But I have
never seen anything there such as "hey shhht! don't tell anybody, let's
fix it silently". It's more like "pfff!! needs to be root anyway, not
relevant".

> you're privy to that information yourself and in silent agreement with
> that policy, don't pretend you don't know (care to explain to the rest
> of the world that itanium hardware bug still unfixed after 2 years?).

As ruled above, I cannot explain this. However, I can paraphrase what
Brad already reported there : http://lwn.net/Articles/285928/
Basically, the fix was deemed wrong and needed some rework. Then no
time to work on it. I recently asked for news about the situation,
because I don't like a fix to remain pending, and response was
that the fix was wrong and not sufficient anyway because of the
reasons Brad already explained. If a fix cannot be developped,
game over! And please don't ask someone to publicly forward the
patch, it will only harm in helping exploitation without bringing
any value towards a possible fix.

Hint: I found the wrong fix is already present in at least one publically
downloadable distro kernel update.

> and where's the problem? you simply include the relevant info in the
> commit and let the users decide which ones they're interested in.

Believe it or not, it is already the hardest part of my work, determining
the relevant information. It is no different than any bug tracking for any
software, people ask for tests, some seem valid and point to one direction,
some seem invalid, some tracks are finally proven wrong. It's very hard to
build a relevant commit message. Sometimes when I read Linus' commits, I
think "oh he's abusing..." but when I try to do the work myself, I
sometimes wonder why I take 30 minutes to try to collect relevant
information to build a useful commit message when it could simply be
a one-liner.

> the
> current problem is that by omitting or obfuscating any such info makes
> it *impossible* to quickly select the interesting commits. you're not
> improving linux security by making it hard for users to make decisions.

I know that, believe me! And this is not limited to security impacts. I
regularly read random 2.6 commits to find if there are fixes relevant to
2.4 or stable. Sometimes I don't know. Sometimes I think I should merge
them, then someone explains me that I'm wrong because the bug cannot be
triggered in my situation.

And please, do not say "obfuscate" when speaking about commit logs,
it's more like "summarize", even if extreme summarization may lead to
obfuscation.

> > Overall, I think the security issues are correctly taken by the middle
> > chain (2.4 and 2.6-stable),
> once again, to stress the point: no, they are *not* correctly handled.
> ask Chris Wright about
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-... .

Could you elaborate please ? This fix is in Linus tree, and I don't see
Chris in the sign chain. Did he report it first ? Also, I'm sorry, but
I fail to see the security implications of this patch, eventhough seeing
"ptrace_attach" catches my attention.

> it doesn't get 'lost', it gets intentionally omitted. and it's not
> sometimes but pretty much every time now. case in point,
> http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-...
> knowing Ilja's past work, i can't imagine he didn't report this as a bug
> with security impact (not that it's hard to figure out even if he didn't)
> yet there's no mention of it in the commit, not to mention a backport to
> -stable.

From what I have seen, this was finally judged as a non-issue since other
checks are in place, which probably explains why you don't have it in
stable. It is very often that fixes for supposedly real vulnerabilities
finally end in mostly code cleanups.

Don't get me wrong, I'm all about transparency, and we're not good enough
at this IMHO. But it is not *that* bad.

What amazes me, to be honnest, is the ability you have to spot patches in
mainline that take a lot of time to other people. I'd love to know how you
proceed, it would save me a lot of time. Or maybe you spend whole days at
this, in which case all users would benefit from your findings. Why don't
you post on LKML, to -stable or even on security lists links to the patches
you consider suspicious security-wise ? Even assuming everyone's honnest
in the chain, and the other people considered the bug inoffensive when you
think the contrary, you could educate people about your methods of
discovering this.

For instance, people are slowly getting used to CC stable when releasing
a fix for an annoying bug. It still takes a lot of efforts to get those
bugs automatically submitted, but after about 2 years, people start to
educate themselves. If you regularly prove them wrong when they claim
that a bug has no security impact, they will more often ask for a second
opinion, or try to a bit more descriptive in their commits.

But you cannot expect too much from the people who already do this on their
minuscule spare time, really.

The core issue

Posted Jun 17, 2008 22:08 UTC (Tue) by PaXTeam (guest, #24616) [Link]

> Well, I know that Linus does not like to give too much information in
> the commits because he believes it will help the bad guys, reason why
> he does not add the methods to reproduce the problems nor the CVE when
> he does the commits himself.

that's obviously utter crap. the bad guys are smarter than him or any kernel contributor.
they'll spot the bugs *before* they do. they'll also spot the bugs in the commits regardless
of what the commit says. does anyone remember the collective bruhaha in the security world
when CVE-2006-2451 became public? IIRC, that was a full year after the original commit and
also a full year after at least two working private exploits for it. on a sidenote, IIRC, that
bug still isn't fixed properly, as soon as a sysadmin enables the suid coredump feature, it's
exploitable again.

but actually i don't care about his opinion. what i do care about is that his published
commitment to full disclosure matches what he actually does and that's where the problem is.
you see, withholding security impact information doesn't full disclosure make. you guys can't
have it both ways because the people relying on you expect you to hold up to that commitment
you made in Documentation/SecurityBugs and if you violate that promise, people will make wrong
decisions and innocent people may suffer as a result. so if you want to stick to withholding
security impact info from commits, say so and the world will adjust. but let me guess, that'd
also be bad PR and you don't like that part.

> You have to keep in mind that most often, a commit is a summary of 10-20
> mail exchanges with reproducers, tests and results. You cannot put all
> that in a commit.

that's a strawman Willy, noone asked for it. what we did ask for was inclusion of known
security impact details. it doesn't overflow anyone's mailbox or git repo if you include a
*single sentence* saying that this is a potentially exploitable buffer overflow or NULL deref
or whatever. people can grep for the keywords and make further judgements based on whatever
extra research they deem necessary (including asking the committer for more details). once
again, this info doesn't help the bad guys because they'll spot this regardless of how obscure
the commit message is.

> What I'm really sicked about is the accusations of "general agreement to
> deliberately hide security informations from commits". That's false IMHO,
> really. And in fact, examples given by Brad were on public lists, so that
> does not make any sense.

when you publicly commit to full disclosure, yet, by your own admission now, you don't
actually practice it, then what do you call that? i call that general (silent or otherwise)
agreement of all participants. after all, participation in handling the kernel security bugs
is voluntary, noone forces you or anyone else to do it against his better judgement. therefore
if you play ball there, you de facto accept the not-exactly-full-disclosure policy (you later
admit that regarding vendor-sec, except everyone knows that vendor-sec does not practice
full-disclosure so people don't build that expectation into their decision making processes).
do you understand the problem better now?

> What we agree about is not to divulgate information when joining the list.

that's not the whole truth. what you also agree to is full disclosure after the embargo. go
read Documentation/SecurityBugs if you don't believe me. do you admit that you don't actually
practice it? if you do, we can take the discussion further and figure out whether you want to
actually update Documentation/SecurityBugs or really adopt full-disclosure. if you don't,
we'll keep on exposing the fallacy because we have no other way to affect you and at the same
time help the world by letting them know that their security risk analysis processes are based
on false promises.

> But I have never seen anything there such as "hey shhht! don't tell
> anybody, let's fix it silently".

i quoted Paul Mackerras a few times now. what do you have to say about that quote? why would
someone practicing full disclosure even *ask* that question and then actually withhold the
more important security impact?

> [stuff about the itanium hw bug]
> If a fix cannot be developped, game over!

not true. did the FDIV or F00F bugs mean game over? i don't think so. last time i checked, the
world hadn't stopped back then. what did happen was a huge PR disaster for Intel (in part
because of the same coverup attempt they're doing again). and history is repeating itself,
except this time you guys are actively party to the coverup. you should all be ashamed and the
conscientous of you should have left the list the moment Intel first had asked you to keep
silent because they didn't feel like fixing the chips. something tells me that there's another
PR disaster in the making.

> Believe it or not, it is already the hardest part of my work, determining
> the relevant information.

this is another strawman Willy, as this should not be your work at all and noone asked for it.
what we do ask for is that if you guys learn about security impact information during the
course of bug examination, then you *do* include those details in the commit. noone is
expecting anyone to spend time on thorough bug impact analysis, most of you are not even
qualified for that, it's really hard and time consuming work (just look at this very thread
and see how i missed stuff ;). but, say, if you see a kernel stack overflow then you say so
and don't try to be creative by calling it 'copying overly long strings' and similar crap. of
course there's always the alternative of publicly admitting that you do not in fact want to
practice full disclosure. it won't look cool in PR events but you can at least sleep with
clear conscience.

> And please, do not say "obfuscate" when speaking about commit logs,
> it's more like "summarize", even if extreme summarization may lead to
> obfuscation.

sorry, but i have no better words when i see something like
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-... .
do you really call that 'summary'? i hope not.

> Could you elaborate please ? This fix is in Linus tree, and I don't see
> Chris in the sign chain. Did he report it first ? Also, I'm sorry, but
> I fail to see the security implications of this patch, eventhough seeing
> "ptrace_attach" catches my attention.

don't you think there's a reason you fail to see the security impact and that i cited this
particular commit? go ask Chris about the history of it, you'll be surprised. in short, this
*is* a local DoS, the kernel devs were provided proof-of-concept code *yet* Linus deliberately
did not include any of that information. then Chris deliberately omitted it from the -stable
commit as well. wild accusations? or the unbelievable truth? it's yours to find out and you
even know how.

> From what I have seen, this was finally judged as a non-issue since other
> checks are in place, which probably explains why you don't have it in
> stable. It is very often that fixes for supposedly real vulnerabilities
> finally end in mostly code cleanups.

it would have been nice to mention this in the commit as well.

> What amazes me, to be honnest, is the ability you have to spot patches in
> mainline that take a lot of time to other people. I'd love to know how
> you proceed, it would save me a lot of time.

uhm, there's nothing magical in this. i sometimes check the kernel shortlog looking for
potential conflicts with PaX and obviously my eyes catch sometimes other 'interesting' looking
commits which i then go and look at in detail.

> Or maybe you spend whole days at this, in which case all users would
> benefit from your findings.

not at all, as i said somewhere below, i do it on my free time, always have in fact. and these
days, i'm trying to actually spend my time on other things. but as long as i can't give PaX
away to a new crew, i'll have to have a look every now and then.

> Why don't you post on LKML, to -stable or even on security lists links
> to the patches you consider suspicious security-wise ?

what would that change? more flames, more excuses for why it's not a good idea to support the
bad guys, etc. and in the end, as i said above, that's not what i care about but rather the
consistency between the promise made to the public and the actual actions. i can't fix any of
that because it's not of my making.

> Even assuming everyone's honnest in the chain, and the other people
> considered the bug inoffensive when you think the contrary, you could
> educate people about your methods of discovering this.

that's a whole separate issue of figuring out that a bug has a security impact after the
commit. i'm afraid such research requires a full staff, and i'm not in the position to fund
anything like that. heck, i'm happy when i make ends meet here ;).

Kernel bug 9416

Posted Jun 18, 2008 2:28 UTC (Wed) by vonbrand (subscriber, #4458) [Link]

What is wrong here? The commit message cites a bug report saying that the kernel might have a buffer overflow. And that is somehow "hiding" potential security implications, when it took me a few seconds to see that it could be a very serious issue?

Come on, now... a real conspiracy theory needs to hint at more secrecy than that!

Kernel bug 9416

Posted Jun 18, 2008 3:28 UTC (Wed) by spender (subscriber, #23067) [Link]

Let's get straight what you were actually commenting on here, since you were turning it into
something it wasn't.

Willy said:
"And please, do not say "obfuscate" when speaking about commit logs,
it's more like "summarize", even if extreme summarization may lead to
obfuscation."

So the PaX team gave an example, where the commit subject was "isdn: avoid copying overly-long
strings".  I agree with them that this isn't a summary but rather a strained attempt at
obfuscating the issue.  Nothing would have been wrong with just copying the description from
the bugzilla entry, it wouldn't have required the invention of clever new phrases, and
wouldn't have required that people click on a bugzilla entry to figure out what was actually
fixed.

Please read and comprehend before posting.

-Brad

Kernel bug 9416

Posted Jun 18, 2008 4:20 UTC (Wed) by vonbrand (subscriber, #4458) [Link]

"Avoid copying overlong strings" does make my (mostly untrained!) alarm bell go nuts. If that is "obfuscation"...

In all this (by now extremely tiresome) discussion I have seen not a shred of evidence of wrongdoing. Perhaps carelessness, perhaps people not seeing potential security problems. Bugs get fixed, most developers care that it is a bug and don't care much if it might be a security problem. Others try to filter "important" (by whatever measures) fixes to apply to the "-stable" (by their measure) tree. If you disagree, you are wellcome to set up the "-secure" tree and do your own filtering and applying. In doing so, you won't be able to rely blindly on the commit messages (the bug fixer might be completely incompetent at seeing security implications) or the discussions that went before (they could all very well be completely off track), so this is hard, thankless work. If you moreover succeed in recruiting a bunch of hackers to help out, more power to you. That would be a real help, flinging all sort of conspiracy theories and ill will accusations around is counterproductive. If the people here (myself included) had spent their time chasing bugs instead of flaming around, we would all be better off.

Kernel bug 9416

Posted Jun 18, 2008 12:17 UTC (Wed) by PaXTeam (guest, #24616) [Link]

> In all this (by now extremely tiresome) discussion I have seen not a
> shred of evidence of wrongdoing.

would that be because you haven't actually seen/read everything? if you have, please tell me
the history of this commit/bug:
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-... .
if you don't see it immediately from the linked commit it's because it was intentionally
omitted. but you can always ask the committer. will you?

> Perhaps carelessness, perhaps people not seeing potential security
> problems. Bugs get fixed, most developers care that it is a bug and
> don't care much if it might be a security problem.

that shows how much of the discussion you saw. pretty much nothing. the issue is *not* with
people not realizing the security impact of bugs (noone expects people to disclose what they
don't know), but rather with intentional withholding/downplaying the same when it *is* known
to them. i gave you a lead above, try to find out what happened there and be shocked.

Kernel bug 9416

Posted Jun 18, 2008 12:10 UTC (Wed) by PaXTeam (guest, #24616) [Link]

> What is wrong here? The commit message cites a bug report saying that
> the kernel might have a buffer overflow.

you just said it yourself ;). 'buffer overflow' is *the* very common and usual term to
describe this kind of bug, not 'copying overly-long strings'. mind you, it's even shorter to
type and would immediately match everyone's mental filters for security related commits (it
happened to match my 'look for funny looking commits' filter only, and i've grown one only
because i realized there was a need regarding kernel commits. pretty sad.). so, why was it
omitted/rephrased in an unusual way (a simple google fight shows that 'buffer overflow' has
over a million hits, whereas the other phrase gives a few thousand), especially since the
original bugreport said so already?

The core issue

Posted Jun 18, 2008 21:53 UTC (Wed) by wtarreau (guest, #51152) [Link]

Hi,

I'm replying quickly because I'm fed up with wasting my time writing in
HTML textareas, and I still have bugs to chase down in other areas.

> it doesn't overflow anyone's mailbox or git repo if you include a
> *single sentence* saying that this is a potentially exploitable buffer
> overflow or NULL deref or whatever. people can grep for the keywords and
> make further judgements based on whatever extra research they deem
> necessary (including asking the committer for more details)

Well, if what you want to see is mostly keywords, it changes things a
*lot*. As I said, building a proper commit is a tough work. Simply adding
keywords or phrases such as "maybe exploitable, costs nothing to merge
anyway" is not hard. We've been trying to work like this at least on 2.4,
noticeably with Dann Frazier from Debian who reported quite a bunch of
fixes. But in general, adding simple keywords such as "security" or "CVE"
on subject line helps a lot.

Now, about the lists' "full disclosure" after the embargo, I disagree
with your interpretation. For instance, vendor-sec charter explicitly
states :
   "...security problems first reported on vendor-sec should not be
    discussed publicly...".

Saying that an embargo is over and a bug is public does not mean you can
talk about everything with everybody. Most common examples are exploits.
So there's no "full disclosure" commitment, in fact it's pretty the
opposite. More like a confidentiality commitment.

> you should have left the list the moment Intel first had asked you to
> keep silent

You don't get it. Nobody asked anyone to keep silent. Nobody simply set
an end to the embargo. You may find this cowardly because noone is
responsible that way. But on another point of view, the guy may finally
come up with a usable fix and the risk of massive exploitation will have
remained lower than with a gratuitous disclosure without a workaround.

Also, it was not "intel". Just one individual, not authoritative of
Intel. That makes a difference too because threatening him will not
threaten intel, but he would simply stop working on the subject. But I
don't know why I'm explaining this, you seem well informed already.

Last, about "obfuscated commits", the suspicious examples you collected
concern Chris, Linus and Paul. Why not talk to them directly, showing
evidence of those bugs being actively used for years before their fix
and trying to convince them to be more verbose, instead of whining on
a forum they might never read about general kernel devs practises ?

Build up a collection of evidence, show them privately or publicly as
you like, and explain what issues you find there and why you think
it's bad to proceed that way. It will be quite more productive IMHO.

Once again, I'm all for giving enough information to users to justify
upgrades, while not giving enough to script kiddies. I'm for
"responsible disclosure" instead of "full disclosure". The real bad guys
are not dangerous, they have their targets and already know about the bug.
Dangerous ones are the script kiddies destructing systems to impress their
friends. That has happened a lot with the vmsplice bug. I've even read
public records of IRC channels where stupid kiddies compared their
performance on their respective friends' colocated systems. In that sense,
giving a link to a CVE and a minor adequate summary is the right thing to
do in my opinion. It can be left to the reader to push analysis further if
he wants to.

Also, keep in mind that most bug reports come from vendors' customers,
and at least for that reason you cannot divulgate too much of their
context, otherwise your security channel gets a bad reputation and you
quickly don't get bug reports anymore.

Last, I would say that most people interested in security updates are
already on security lists : distro vendors and stable maintainers. Rest
of the world either rely on their vendor or on mainline kernel. If you
feel like you'd need to get more information on the security issues,
you may ask to join those lists, and even participate in elaborating
more suitable commit messages. If you can respect a certain level of
confidentiality, you may finally get the information which seems to
be missing you too much, and finally notice that there is no such
common agreement about deliberate obfuscation.

Willy

The core issue

Posted Jun 19, 2008 0:47 UTC (Thu) by PaXTeam (guest, #24616) [Link]

> Well, if what you want to see is mostly keywords, it changes things a
> *lot*. As I said, building a proper commit is a tough work.

it wasn't me who decided that the kernel security bug disclosure policy should be that of full
disclosure. you're free to change it if you can't live up to it for whatever reason.
alternatively, you can make the archives public after the embargo so that people can get the
whole picture. the current situation is not good as it hurts users.

> But in general, adding simple keywords such as "security" or "CVE"
> on subject line helps a lot.

indeed and this is what's missing among others in 2.6 commits (which is what we've been
complaining about, not 2.4).

> Now, about the lists' "full disclosure" after the embargo, I disagree
> with your interpretation.

'full disclosure' is not subject to interpretation. if you withhold information, then it's not
'full', period. check the full-disclosure mailing list if you don't believe me (it's full of
exploit code, there's no embargo observed, etc). you're free to call yours something else and
then everyone will know what to expect.

> Saying that an embargo is over and a bug is public does not mean you can
> talk about everything with everybody. Most common examples are exploits.
> So there's no "full disclosure" commitment, in fact it's pretty the
> opposite. More like a confidentiality commitment.

i know all that Willy, that's why i said that vendor-sec was known *not* to practice full
disclosure. the kernel security list however, as of now, has a declared full disclosure
policy.

> You don't get it. Nobody asked anyone to keep silent.

not true. you stated yourself the vendor-sec list rules that you don't talk about issues in
public. this one was no different, the list rules still apply. if the list had been public, do
you think Intel would have told you about this bug?

> Nobody simply set an end to the embargo.

not true, there was an embargo set but it came and went, nothing happened except the patch got
into at least one distro afterwards, that effectively made it public and thus ended the
embargo.

> You may find this cowardly because noone is responsible that way. But on
> another point of view, the guy may finally come up with a usable fix

you know now that it's not possible because the bug can only be fixed in hw properly.

> and the risk of massive exploitation will have remained lower than with a
> gratuitous disclosure without a workaround.

so you're saying that the world would have been better off if the FDIV and F00F bugs had never
become public? are you seriously justifying this new Intel coverup?

> Also, it was not "intel". Just one individual, not authoritative of
> Intel. That makes a difference too because threatening him will not
> threaten intel, but he would simply stop working on the subject.

someone posting from @intel.com cannot state on his own behalf that Intel would not provide
the code sequences that trigger the bug. and holding Intel accountable is not threatening
them, except their bottom line if angry customers hold them, well, accountable for their bad
decisions.

> But I don't know why I'm explaining this, you seem well informed already.

the world isn't Willy, that's the problem. and when Intel does finally fix the bug in a future
chip, what will happen to the people running with the old ones? will you/they keep them
ignorant about their buggy chips forever?

> Last, about "obfuscated commits", the suspicious examples you collected
> concern Chris, Linus and Paul. Why not talk to them directly,

and tell them what they already know? i don't see the point...

> showing evidence of those bugs being actively used for years before
> their fix and trying to convince them to be more verbose, instead of
> whining on a forum they might never read about general kernel devs
> practises ?

we're not whining, we're telling the world that things are not exactly as one would naively
believe. although i don't expect kernel devs to read everything on lwn, i know that many of
them read this one and they got the message. question now is what they'll do with it, in
particular, will they live up to full disclosure, or will they keep downplaying security
issues.

> Build up a collection of evidence, show them privately or publicly as
> you like, and explain what issues you find there and why you think
> it's bad to proceed that way. It will be quite more productive IMHO.

the evidence is in secret mailings lists. will you make the archives public? this is the point
we made: noone is accountable because everything is done in secret. how can *we* change that?

> I'm for "responsible disclosure" instead of "full disclosure".

i posted somewhere below what Linus had to say about 'responsible disclosure' (look for
'mental masturbation'). you're in wild disagreement, to say the least.

> Also, keep in mind that most bug reports come from vendors' customers,
> and at least for that reason you cannot divulgate too much of their
> context, otherwise your security channel gets a bad reputation and you
> quickly don't get bug reports anymore.

noone asked for that (although the kernel devs' commitment to full disclosure would imply
disclosing even such information, personally i don't care or think it's interesting or
necessary).

> Last, I would say that most people interested in security updates are
> already on security lists : distro vendors and stable maintainers.

not true at all. one must meet some definition of 'elite' to get on these lists. you know full
well how individual researchers are *not* welcome on vendor-sec for example.

> If you feel like you'd need to get more information on the security
> issues, you may ask to join those lists, and even participate in
> elaborating more suitable commit messages.

i may ask and i will get rejected. thanks, but no, thanks.

> If you can respect a certain level of confidentiality, you may finally
> get the information which seems to be missing you too much, and finally
> notice that there is no such common agreement about deliberate
> obfuscation.

i don't need to be on any list to notice when something's covered up. FWIW, i'm not the first
one to have made such an observation: http://marc.info/?l=bugtraq&m=115255401805462 - you know
who he is, right?

The core issue

Posted Jun 17, 2008 11:50 UTC (Tue) by bojan (subscriber, #14302) [Link]

> If we want to be picky, everything which can permit a non-privileged user to cause a
malfunction resulting in a degradation of performance, integrity, availability,
confidentiality or traceability is a security issue.

Kernel is done for a wide audience and, as you mentioned, different people would like to know
about different types and severity of security problems. So, the best policy is just to
describe them all and let people (including distributors) decide if they want to bump up or
not.

In other words, if things are logged in such a way that even the pickiest of them are OK with
it, then the ones that are less picky will be OK too. Provided the information about the
security impact is known, of course.

It's done exactly in this way... in a sense

Posted Jun 17, 2008 17:08 UTC (Tue) by khim (subscriber, #9252) [Link]

So, the best policy is just to describe them all and let people (including distributors) decide if they want to bump up or not.

Better for whom exactly? There are exist one potential group which will be VERY negatively affected: top-level kernel developers (and to lesser extent all other kernel developers). For them detailed list of security implications in descriptions is unneeded noise, not something important. They look and review THOUSANDS of such descriptions each and every week so OF COURSE they want to reduce this noise. If issue looks severe enough - they keep it in description.

In other words, if things are logged in such a way that even the pickiest of them are OK with it, then the ones that are less picky will be OK too.

And the pickiest users are of course kernel developers - they want to avoid clutter in various typed of reports and so demand short and concise descriptions. So they remove unimportant (for them) parts. Including detailed descriptions of possible security implications. In all cases except the most severe ones. So in fact they did what you asked them to do - but somehow you are not satisfied.

Now if you think they go overboard in some cases - you can join the system and help to prevent this. If you want totally different style of commit messages (extensive with lots and lots of details) in general - you are free to fork the project and put any kind of descriptions you want.

It's done exactly in this way... in a sense

Posted Jun 17, 2008 21:54 UTC (Tue) by bojan (subscriber, #14302) [Link]

> Better for whom exactly?

For people trying to keep their systems secure?

Security issues are not noise, especially when people producing patches for them are already
aware of their nature.

> So they remove unimportant (for them) parts.

Yeah, that's kinda the problem. Even the most trivial of security issues may be important to
someone. Removing information about it puts users that may be affected at a disadvantage.

> If you want totally different style of commit messages (extensive with lots and lots of
details) in general - you are free to fork the project and put any kind of descriptions you
want.

So, mentioning in the release notes of .7 that some security issues have been addressed, which
would take less than 100 words, requires a kernel fork? I don't understand where all this
hostility is coming from.

If issues are known, release notes should specify that. It's that simple.

The core issue

Posted Jun 18, 2008 1:07 UTC (Wed) by vonbrand (subscriber, #4458) [Link]

Or create a device for /, mount it somewhere, and cd there. Or a hundred other creative ways to escape, or to thoroughly thrash the system... As root in (traditional) Unix there is nothing that stops you. And that with SELinux you can make UID 0 powerless doesn't mean that that is common today, even where SELinux is enabled by default.

Sure, if there is a bug that could make the system crash or misbehave, it should be fixed. But if only root can exploit it, it is definitively much lower priority than the same that any user can trigger locally or (even worse) remotely.

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 10:07 UTC (Tue) by PaXTeam (guest, #24616) [Link]

> Then he can post privately to the -stable team.

in case you didn't read http://lwn.net/Articles/285438/ , the problem is not with -stable per
se, it's just one of the causalties of the policies played on the kernel security mailing list
(although it's probably not a coincidence  as there's overlap between the members). it should
also be clear by now that we are NOT going to play ball with that list. i even told you so in
the past already (remember the random driver stack overflow bug?). the reason is very simple
and sad at the same time: this list has become the primary place to discuss then hide security
information about bugs. in case it's not clear, the problem is not with the 'discuss' part but
the 'hide' one.

> But I think that unfortunately, Brad is not trying to improve Linux but
> to demonstrate it's a pile of crap, maybe in order to promote security
> add-ons such as grsecurity.

how is exposing the dishonesty of certain kernel developers not improving linux? do you want
people to live without knowing what bugs have security related consequences? if said
developers are unwilling to practice full disclosure (despite public reassurances, mind you)
then someone else has to. and how on earth is this supposed to promote grsecurity (or any
other access control system)? if anything, they all suffer collateral damage from this policy
of not-exactly-full-disclosure.

> But it's doing no service to him either because acting this way will
> scare away people who look for a reliable system.

i did that more than 3 years ago already:
http://forums.grsecurity.net/viewtopic.php?p=3805#p3805 and i still stand by what i said back
then. in fact, the situation has become a whole lot worse since.

> IMHO, he would be of great help if he would accept to be on the -stable reviewers list.

it's too late by then, the problems start with the kernel security list, -stable feeds on
that. and no amount of private lists will help the accountability problem you're having now.
the way of solving that is to make the archives public after a certain period (there's nothing
to hide after the bug is fixed and public, right?) then we can talk about having improved
linux security.

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 19:24 UTC (Tue) by wtarreau (guest, #51152) [Link]

stable@ is not a list, just an address to post things you think should
be fixed in stable releases. There is no secrecy there, you can simply
CC LKML if you want (and preferably the patch reporter first in order
to get information about relevance). I sometimes proceed like this.

I have no problem with your policy about not posting to private lists.
Davem does the same, and I respect this. Then you'd better check the
oss-security list : http://oss-security.openwall.org/wiki/

It is public, talks about security issues in opensource software
(including linux), and many of the closed lists members are there.

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 20:35 UTC (Tue) by PaXTeam (guest, #24616) [Link]

> stable@ is not a list, just an address to post things you think should
> be fixed in stable releases.

yes, i figured it out in the meantime. problem with it still is that anything that makes it
there is already *too late* because it must have been entered the Linus tree already
(mandatory condition for a submission to be accepted). and if the commit is misleading there,
it's game over for everyone else. in any case, you can certainly CC me on anything you need my
input on *but* consider that my work has nothing to do with neither the kernel nor security
and, as of this year, i'm spending less and less of my free time on this as well, with the
eventual goal of completely stopping it. in other words, don't expect me to spend a lot of
time on this, my quota for linux/security is already pretty much exhausted.

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 21:21 UTC (Tue) by spender (subscriber, #23067) [Link]

Though it won't solve many of the problems mentioned, particularly:
1) Bugs that originate in private through the numerous private mailing lists
2) Obvious and explicit security information mentioned in attached bugzilla entries getting
omitted from changelogs
3) root-based bugs not only being downplayed, but specifically labeled as not security
relevant (defeats the purpose of SELinux, SMACK, capabilities)

It may help in raising more awareness regarding the invalid userland dereference class of
bugs, which seem to most often get ignored.  (Microsoft hopefully is paying similar attention:
http://www.immunitysec.com/downloads/DriverImpersonationA...)

With that said, you're of course welcome to CC me on relevant mails as well.

-Brad

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 12:31 UTC (Tue) by hmh (subscriber, #3838) [Link]

I suggested this very thing in the comments for 2.6.25.6, and guess what :-) I was told to
"reread the thread because I had missed the point".  Heh.

The point, apparently, is that those who could help lessen the damage of security fixes going
unnoticed are NOT interested on doing it (because of various reasons) for -stable.  I won't
judge their reasons for this, but that's what it boils down to.   And that's really
unfortunate, because while it would not be the full round-trip to the moon, it would still be
a damn nice first step.

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 6:58 UTC (Tue) by Los__D (subscriber, #15263) [Link]

And how is a comment starting with In this issue of "we have no clue what we're doing," NOT auto-labeling itself as a troll comment?

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 10:14 UTC (Tue) by PaXTeam (guest, #24616) [Link]

you might want to read and understand this before labeling someone as a troll. in particular, the problems we're exposing are neither irrelevant nor off-topic nor are they meant to incite emotional responses but rather a rational discussion of the problem. but the first step towards that is to actually admit that there's a problem which is where the kernel devs are stuck at the moment.

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 10:26 UTC (Tue) by Los__D (subscriber, #15263) [Link]

Starting discussions with people by labeling them as clueless as the first sentence, is doing
a pretty terrible job about not inciting emotional responses.

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 10:52 UTC (Tue) by PaXTeam (guest, #24616) [Link]

it wasn't a start but the continuation of http://lwn.net/Articles/285438/ . second, it's not a
discussion apparently, but just talking to ourselves and maybe telling the world to look at
certain problems, the discussion itself is apparently to be had on lkml with the kernel devs,
not us. finally, maybe you wanted to say 'flame' instead of 'trolling'? there's a big
difference and lkml is full of the former yet life continues, problems get solved. why can't
this one be?

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 11:27 UTC (Tue) by Los__D (subscriber, #15263) [Link]

<p><i>it wasn't a start but the continuation of http://lwn.net/Articles/285438/ </i><br/>
Another post started in exactly the same mocking way, and not intended FOR the developers, but
for spewing gall at them, on a semi-random news site.

Please note that I don't disagree at with the technical parts, just the serving.

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 13:53 UTC (Tue) by nix (subscriber, #2304) [Link]

Likewise. Obviously it would be nice if more things that were security holes got so labelled,
but short of educating everyone who commits anything to think in a suitably paranoid manner (a
nice long-term goal but ridiculous in the short term), I can't see any way in which this
problem could be instantly fixed, except if the people who *could* determine that particular
fixes fix security problems were to help with that.

But at least one of them (sorry Brad, mixed you up, it may be that only PaXTeam is alleging
some sort of widespread dishonesty conspiracy theory and you're merely alleging insufficient
paranoia, which I'd agree with) would apparently rather moan than help fix the problem.

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 14:25 UTC (Tue) by PaXTeam (guest, #24616) [Link]

1. none of us talked about 'conspiracy', you did. you keep coming up with this strawman and
i'll keep exposing it. but keep trying ;).

2. as you were told about a dozen times already, the problem isn't with not recognizing a bug
for its security impact (or rather, that's a separate problem), but the intentional omission
of such information when it is already known. you have yet to explain the ptrace self-attach
commit, why don't you say something about that?

3. we can't help fix the problem because the problem isn't with us but rather those kernel
developers who decided (but failed to inform the public about) that full-disclosure is a PR
act only, in reality they don't practice it. the consequence of this double-play is that
people who trust them on their word will make false security evaluations when looking at the
commits (and no, they have nothing else to judge because these bugs are discussed on private
lists).

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 15:07 UTC (Tue) by nix (subscriber, #2304) [Link]

Did you not read what I wrote at all? You're insisting that your assertions of coordinated
identical intentional malpractice on the part of many dozens of unrelated people does not
constitute a conspiracy theory... but coordinated identical intentional malpractice is the
very *definition* of conspiracy.

i.e. you're now arguing with the dictionary, not with me.

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 15:24 UTC (Tue) by PaXTeam (guest, #24616) [Link]

i read what you wrote but i think we're having a definition crisis of some sort ;). how do you
define 'coordination'? somewhere above i told you it doesn't take much in a close-knit circle
of people who belong to the same social group anyway. if you mean written edicts issued in
secret or something like that, then it's definitely not that. if you mean the 'see what Linus
does, do as Linus does' kind of 'coordination', then you may call it that but it doesn't make
it so. ever heard of unspoken/unwritten rules that people understand and abide by? doesn't
make them all conspirative, does it? nevertheless, it can still be bad practice and in this
case, it is.

but in the end, you know what, whatever word makes you happy. can you now make the next step
and actually do something about checking the facts out for yourself?

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 15:42 UTC (Tue) by nix (subscriber, #2304) [Link]

I can't do that because the facts are largely on private lists I don't have access to. (That's
why I trusted that the facts were as you stated.)

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 15:47 UTC (Tue) by PaXTeam (guest, #24616) [Link]

> I can't do that because the facts are largely on private lists I don't
> have access to.

and what prevents you from asking for them? are you not curious? don't you think that it may
be a good idea to make them public? or just afraid of finding your beliefs shattered a bit?

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 18:09 UTC (Tue) by nix (subscriber, #2304) [Link]

Mostly I'm not interested enough to bother people over it (and I have other things to do). If
the holes get fixed, it's good enough for me personally...

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 20:53 UTC (Tue) by ncm (subscriber, #165) [Link]

Sorry, nix, the "definition of conspiracy" involves lawbreaking.  I see no evidence of crimes
here, and no accusations of crimes, hence no conspiracy and no accusation of conspiracy.  

What we do appear to have is a belief in security-by-obscurity, abetted by preference for
convenience and tidiness, and by publicity-shyness, finally coupled with disrespect for
SELinux.  None are crimes, but the combined effect on security is no less fortunate.

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 22:36 UTC (Tue) by nix (subscriber, #2304) [Link]

There's a dictionary definition and a legal definition, and they're 
different. The dictionary definition doesn't mention lawbreaking (at least 
not in my dictionary).

(And no, nobody's alleged crimes, although some of the allegations might 
be read to allege criminal *intent*.)

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 10:18 UTC (Tue) by mb (subscriber, #50428) [Link]

> In this issue of "we have no clue what we're doing," we have:

So can you please join the -stable review list to improve this situation? Or are you just out
for blood and don't really want to _improve_ the situation?
I'd be happy, if you'd review my patches and point out (security) errors in the patch or
patch-description, that I didn't think of.

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 11:12 UTC (Tue) by PaXTeam (guest, #24616) [Link]

> So can you please join the -stable review list

can you point us to the archives of this mailing list?

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 11:24 UTC (Tue) by mb (subscriber, #50428) [Link]

> can you point us to the archives of this mailing list?

Well, it's not a mailing list AFAIK. It's a list of CCs that the -stable patches are sent to
for review. Just ask greg, if you want to be added.

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 18:16 UTC (Tue) by wtarreau (guest, #51152) [Link]

> > So can you please join the -stable review list
> can you point us to the archives of this mailing list?

It's not a mailing list, it's just a list of people who automatically
receive the patches planned for next stable release when they are sent
to LKML. I asked myself to be subscribed because it was hard to spot
those in the huge LKML noise. It helps me track fixes which I may have
to backport to 2.4, and maybe one day I'll be able to contribute back
by spotting bugs before they get merged.

All you get is 100% public, it's just a convenience to avoid polling
LKML. If you or Brad have things to add to the patch descriptions,
it might be a good idea.

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 11:22 UTC (Tue) by NAR (subscriber, #1313) [Link]

According to Napoleon, never ascribe to malice, that which can be explained by incompetence.
How do you know, it's malice and not incompetence? Some of the comments imply malice.

Of course, incomptence is not that reassuring either, but for example Al Viro's comit seems to
be two months older than the DoS bug report so I don't see how could he knowingly omit the
security information from the bug report...

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 12:00 UTC (Tue) by spender (subscriber, #23067) [Link]

First, a correction to my original post.  The fix for the TPM bug on vendorsec came from
Michael Halcrow, not Serge Hallyn.

I should have linked to the proper URL, which is:
http://git.kernel.org/?p=linux/kernel/git/stable/linux-2....

Here we see Greg committed it into -stable, and Al Viro and Chris Wright both signed off on
it.  Unless Greg or any of the other people signing off on the commit didn't read any of the
link which was pasted (which I find very hard to believe) the information about it being a
local DoS was intentionally suppressed.

If you don't bother to spend the time to go and read the links to everything the PaX team and
I have posted, then it would be easy to assume that this is just some huge series of
incompetence (which extends back far beyond these last two releases where I started to
actually point things out).  If you go and actually read the references though, I think you'd
have to be pretty naive and distort the evidence in every case to come to a conclusion of mere
incompetence.

Like I mentioned before though, the public doesn't have all of the information at hand since
so much goes on behind the scenes in private mailing lists with no accountability.  I alluded
to a few of the instances earlier if you actually read all my previous posts, but I'm
compiling the full details of this information and will be submitting it to LKML and other
mailing lists, since it's too good to waste here.  It should clear up any lingering doubts.

-Brad

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 12:31 UTC (Tue) by NAR (subscriber, #1313) [Link]

Well, I haven't noticed the DoS in the bug report for the first look, I had to do a search in
the browser.

I think there are very very few security consious developers like yourself. Most of them won't
notice that a "double free" is a security bug. Most of them start to read the bugzilla entry
from the "Description" part and would miss the summary.

Given the speed of kernel development, the amount of changes and the lack of common bug
database (and it's integration with the version control system), I still think it's
incompetence, not malice. I haven't examined all of the links in details, but haven't read a
"it's a security fault, let's hide it" comment which would prove malice.

Not that's it all that reassuring.

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 13:11 UTC (Tue) by PaXTeam (guest, #24616) [Link]

> I haven't examined all of the links in details, but haven't read a
> "it's a security fault, let's hide it" comment which would prove malice.

well, you really should check them out, but here's a choice quote from Paul Mackerras posted
to the security list a while ago:

  I'd appreciate advice about the best way forward here, and to what
  degree we should be candid about the reason for the patch in the
  commit message when it gets committed to Linus' git repository.

how 'candid' it became you can check in that thread and the posted links (hint: no mention of
the more serious security impact).

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 15:10 UTC (Tue) by nix (subscriber, #2304) [Link]

I imagine this is because the bad guys must be assumed to read the commit logs too, before
releases are made, and it's intentionally hard to edit commit logs immediately before
releasing, especially if it's a big tree (i.e., a new major release, not -stable): it would
likely mean a massive rebase.

"Stable" kernel 2.6.25.7 released

Posted Jun 17, 2008 15:43 UTC (Tue) by PaXTeam (guest, #24616) [Link]

> I imagine this is because the bad guys must be assumed to read the
> commit logs too, before releases are made,

this mentality is called security by obscurity. congratulations for having made the basic
mistake in computer security ;-).

more seriously, you're saying that the bad guys in desperation read commit logs and use the
little time between that and the actual release/widespread adoption to exploit the holes (that
takes some skill in case of kernel bugs), because the same (skilled) bad guys are unable to
find exploitable bugs themselves? do you have the *slightest* evidence to support this view
(yeah i was paraphrasing, hope you got the point and won't get sidetracked in my choice of
words)?

second, what about the good guys who need to know the same information (think IDS/IPS vendors
for one, then the many trees tracking that of Linus and trying to keep their side secure)? oh
yeah, the two sides of the coin. the problem is that you, the ultimate end user and most
affected person didn't get to flip it. maybe you personally don't want to, but there're many
others who do.

third, about this particular commit: why does it make sense to mention the DoS at all then?
don't you think it draws attention from the bad guys who can sniff an exploitable bug where
others smell a DoS only? wouldn't it have been better (from your point of view, that is) to
simply not mention anything at all? i think your imagination needs a little more consistency.

last but not least, now that you're done using your imagination, will you actually try to do
something to figure out what's really going on? you know, talking here won't get you far, your
answers are in the mailing list archives.

Congratulations are in order!

Posted Jun 17, 2008 17:37 UTC (Tue) by khim (subscriber, #9252) [Link]

this mentality is called security by obscurity. congratulations for having made the basic mistake in computer security ;-).

This one comment explains it all: you DON'T care about security. What you DO care about is your reputation. You will happily increase risk for users as long as everything is done by your rules.

Why? Oh, it's simple: anyone who claims that security by obscurity never works is a troll. Actual rule is "You can fool all the people some of the time, and some of the people all the time, but you cannot fool all the people all the time". Case to the point: PSP had easily exploitable backdoor in firmware from the day one - and still has it today. Yet crackers needed two years to find and exploit it. Why? Because of the obscurity. And you can be pretty sure A LOT OF guys wanted to this: hardware solitions were offered, rare disks (needed to use previously known cracks) were sold for $100 or more, etc. BIG BUSINESS. BIG MONEY. Yet no crack for TWO YEARS. THAT is the power of "security by obscurity". It DOES work. Forever? Of course not! But for a long time.

If you don't accept thios simple fact that there are nothing to discuss further - you have different set of axioms then kernel developers do.

about this particular commit: why does it make sense to mention the DoS at all then? don't you think it draws attention from the bad guys who can sniff an exploitable bug where others smell a DoS only? wouldn't it have been better (from your point of view, that is) to simply not mention anything at all?

Bingo! This particular commit is bad NOT because it does not mention all security implications, but because it talks about DoS at all. Poor phrasing, but not the end of the world.

Congratulations are in order!

Posted Jun 17, 2008 18:58 UTC (Tue) by zakalwe2 (subscriber, #50472) [Link]

What you've just said is that experts who spent their time reverse engineering a binary
firmware failed to find an easily exploitable method, but instead found a method that enabled
them to make money.  That is human nature.  What guarantees do I have that some linux
contributers don't deliberately introduce or obscure exploitable bugs so that they can profit
from them in some way?  The vast majority of other people who understand and find them are
likely to keep them to themselves, or sell them on.

If there is no accountability or transparency in how these serious flaws are dealt with,  no
incentives to disclose or not put them there in the first place, bad things will almost
certainly happen.  It's inevitable.  These corruptions happen in every other field, it's naive
to think they won't happen here.

Everything that is even remotely a "security" threat should be labeled as such.  If this makes
linux look bad when some smart researcher compares all the major OS' security by counting
publicly known flaws, so be it.  That is the incentive to not put them there in the first
place.  Hiding them is, at best, leading to a culture of complacency, and at worst an
indication of malice.

I salute spender and the PaX team for bringing these issues to the fore.  They have made a
considerable impact in the field of security,  when they speak it is worth listening.

Not so fast

Posted Jun 17, 2008 22:04 UTC (Tue) by man_ls (subscriber, #15091) [Link]

But wait, you have to realize the consequences of the way advocated by spender and PaXTeam: let us follow the path suggested by nix below. First find a bug, and search for a related vulnerability; then publish it along with an exploit so people take you seriously, and obtain a CVE identifier. That is the moment to send a patch to a public list pointing to the CVE so people can make informed decisions.

Et voilà! You have an unpatched vulnerability (with a published exploit in the wild) on millions of machines worldwide, and everyone who doesn't update their kernels daily is vulnerable. Is this responsible disclosure?

You may think that Linus et al are burying their heads in the sand, but try to picture yourself in their shoes; there is little else they can do but try to raise the bar, however little they can, for exploitation. Occasionally a really serious issue arises with obviously dangerous implications. Such rare events can be disclosed and publicized so hopefully everyone will patch their kernel before an exploit is published, but it cannot be done for every bug with potential security implications (i.e. a hundred times a month). Kernel users would stop paying attention quickly if every esoteric bug was magnified by the maintainers.

Not so fast

Posted Jun 17, 2008 22:38 UTC (Tue) by PaXTeam (guest, #24616) [Link]

besides a few strawmen, you missed the entire point of what we were saying. it's not about
what disclosure policy is best for the kernel. i don't care, really. what i do care about is
that what the kernel devs commit to in Documentation/SecurityBugs (as of now, full
disclosure), they stick to it. and i explained many times now the reason for that: people
assessing security risks need to know whether the kernel commits are a reliable source of
information of that or not. because if they aren't then these people will know they'll have to
assign resources to that. as simple as that, it's called risk management.

Not so fast

Posted Jun 17, 2008 23:28 UTC (Tue) by man_ls (subscriber, #15091) [Link]

besides a few strawmen [...]

"Straw man" commonly refers to the misrepresentation of someone's position. I don't see where I misrepresented your position, but I'm sorry that you feel I did.

The policy for security bugs you cite is specifically for people who contact the security team. If a specific bug doesn't reach them I don't see why that policy should apply at all. In some of the bugs you have brought up the security implications seem minor at best, so maybe that is why they were not sent to the security team; or maybe it was due to incompetence. Anyway for these bugs kernel devs are free to apply whatever disclosure policy they see fit (i.e. not bound by any document), and we are free to discuss if they do it sensibly, whether you care about it or not.

people assessing security risks need to know whether the kernel commits are a reliable source of information of that or not.

People assessing security risks should make their own assessments. They should not trust commit messages, and I don't see how you can suggest that they might even think about it.

Not so fast

Posted Jun 18, 2008 0:41 UTC (Wed) by PaXTeam (guest, #24616) [Link]

> "Straw man" commonly refers to the misrepresentation of someone's
> position. I don't see where I misrepresented your position, but I'm
> sorry that you feel I did.

where did we suggest that 'our way' is to unleash full blown exploits on the unsuspecting
public in order to stress the bug's importance?

> The policy for security bugs you cite is specifically for people who
> contact the security team.

stop right there. we weren't talking about anything else but security issues discussed and
subsequently covered up in secret lists. what happens in public forums is not under
discussion.

> If a specific bug doesn't reach them I don't see why that policy should
> apply at all.

in that case what's the question at all? what they don't know they can't cover up.

> In some of the bugs you have brought up the security implications seem
> minor at best, so maybe that is why they were not sent to the security
> team; 

which ones are you talking about? FYI, every one of the commits we brought up had been
discussed on either the kernel security list or vendor-sec. besides, what do you call 'minor'?
does your definition of 'minor' also match that of the rest of the world? did you make sure?
then what entitles you to make that judgement call instead of the world? see, you already
showed the exact bad mindset that plauges the kernel devs engaged in covering up security
bugs. *you* don't get to decide what is important information for *other* people. the *other*
people do. understand?

> Anyway for these bugs kernel devs are free to apply whatever disclosure
> policy they see fit

what bugs again? if bugs are not on a secret list then they're in a public one, what's there
to 'disclose' that isn't already public?

> They should not trust commit messages,

says who? 'man_ls' or is it the agreed-upon kernel policy? do you even realize what you just
said?

Not so fast

Posted Jun 18, 2008 6:57 UTC (Wed) by man_ls (subscriber, #15091) [Link]

where did we suggest that 'our way' is to unleash full blown exploits on the unsuspecting public in order to stress the bug's importance?

I think my scenario (find vuln, exploit it, publish and get CVE id, then send commit with CVE in message) is almost as dangerous without a published exploit. I can see why giving this kind of information away in the kernel commits (even a CVE id) scares people. I don't think it was a misrepresentation, but whatever.

FYI, every one of the commits we brought up had been discussed on either the kernel security list or vendor-sec.

If it is vendor-sec then Documentation/SecurityBugs does not apply. When it is the kernel security list it applies, but this policy document doesn't say what you seem to think it says. It clearly states that kernel security officers will disclose what they are told as they see fit, and anyone reporting the bug cannot rely on secrecy upon their part.

From what we have seen bugs are disclosed, just not the way you like. You are getting all worked up because commit messages do not reflect the kind of information you would like to be there, and that is fine, but it does not follow from the document that it should be there. "Full disclosure" is not "crying wolf".

does your definition of 'minor' also match that of the rest of the world?

In this thread we have seen a bug which cannot be exploited ("still, this pattern is dangerous, someone had better audit the code for it."), another one only exploitable by root and a local DoS (i.e. a crash). I sincerely hope the rest of the world also thinks these things are minor.

says who? 'man_ls' or is it the agreed-upon kernel policy?

I say that security professionals who rely upon kernel commits to perform security assessments are crazy. If you think it is sound professional conduct, let me know where you get your security. If for "agreed-upon kernel policy" you are referring again to Documentation/SecurityBugs, then you are still deluding yourself. Where does it say that "kernel commits will contain all available information for security professionals"? Bugs are disclosed and that is what they agreed to.