| |||||||||||||
Kernel.org's road to recoveryKernel.org's road to recoveryPosted Oct 11, 2011 8:46 UTC (Tue) by PaXTeam (subscriber, #24616)In reply to: Kernel.org's road to recovery by vonbrand Parent article: Kernel.org's road to recovery
let's see these 'answers'.
> There are very, very few of those; way too few to be of any relevance
first, noone has ever shown actual numbers, i.e., all this 'very few of those' is purely made up, with *nothing* whatsoever to back it up (i bet you don't answer back with actual numbers much like you ignored the rest of my questions so far, you know how well that advances your arguments ;).
second, what does it matter whether there's a lot or only a few of such fixes where the security impact is known? are they doing the same judgement call when they're fixing other kinds of bugs? like, are we to assume now that file system corruption bugs in ext* are also suppressed because "there are very very few of those"? they can't have it both ways.
third, it's none of their business what any given user is trying to do with that information. the rest of the world publishes security errata all the time with varying level of details, but the very fact of having a security bug is not usually suppressed (save for a few companies and apparently linux left in the dark ages).
> We worry there are people out there who will think that only the commits
first, why is it a loss when someone backports a security fix he learns about? does it reduce his security or something?
second, and i asked this already in this very thread, who are these 'people out there' who think this way? show me a single and relevant one (i.e., not your grandma but someone's responsible for the kernel of some sizable chunk of the linux user world). i bet you can't show anyone, and you just made this excuse up to appear 'caring' yet you're achieving the exact opposite.
third, and i asked this already, but let's see if you'll avoid it: what do you think about LWN's having an entire page dedicated to security resources (errata, etc)? are they too "encouraging said selectiveness" causing a net loss? i'm sure the LWN folks are all ears to hear you out on this one.
> Furthermore, there are miscreants grepping changelogs for stuff
first, show me evidence of this but then i'm pretty sure you also made this up. little hint for the future: evidence based arguments fare much better in a discussion than your imagination.
second, and i explained this too some time ago, you're assuming the existence of a person with impossible qualities. namely, you assume that a person can write an exploit based on a patch when he can read its commit message but he's unable to write said exploit based on the same patch when he cannot read the commit message. you should probably talk to some exploit writers one of these days and ask them whether they write exploits based on a few words in a commit *message* or the actual *code* being fixed. you'll be surprised, i guarantee you that ;).
third, you're assuming that exploit writers write exploits based on the commit that fixes the bug vs. the one that introduces it. what evidence is this assumption based on?
> Yes, security through obscurity, which is useful as long as it isn't for
why is security by obscurity useful when it's not for the long term? and what other security measures do the kernel devs have in place?
> What could be won with the flagging is minuscule, what would be lost is,
neither of this has been established yet, but keep trying ;).
> If you want to research the security impact of bugs, knock yourself out.
so on one hand the loss (caused by disclosing security impact of fixes) is much more than the gain but on the other hand they're encouraging others to do the very same and hence cause a net loss (harm). now that's some logic there. unfortunately, they can't have it both ways.
(Log in to post comments)
Kernel.org's road to recovery Posted Oct 11, 2011 9:07 UTC (Tue) by mpr22 (subscriber, #60784) [Link] I have actually encountered people who should know better engaging in behaviours sufficiently similar to "security fixes only!", though not on Linux. In this case it was approximately "fixes for our known problems only, cherry-picked from the more recent patches so that we can play semantic games with the qualification authority to avoid requal", and they subsequently ran into a problem that had been fixed in the latest patch, which they had been sent. They were somewhat upset when they were told that they wouldn't get support unless they applied the patches properly. So yes, these people exist, and what matters is not the detail metric "how large a portion of the general-public user base do they feed kernels to?", but the overall metric "how important is it that they not screw up?".
Kernel.org's road to recovery Posted Oct 11, 2011 10:34 UTC (Tue) by PaXTeam (subscriber, #24616) [Link]
right and we're talking about linux here, so give it another try. like, show me poeple who work for companies/distros of any significance (in terms of user base) *and* live under this mistaken belief. otherwise you've got no argument. also, this software you mentioned, do its authors publish security errata? do they mark security fixes explictly? did you tell them that they must at once stop doing that because it'll cause unspeakable damage?
Kernel.org's road to recovery Posted Oct 11, 2011 18:58 UTC (Tue) by dlang (✭ supporter ✭, #313) [Link]
I can't talk about the 'large part of the customer base' part of this question, but I work in a large (8000+ person) company that runs thousands of servers and I see this mindset of "if it's not tagged as a security issue, we don't really need to apply it" continuously.
Far too many people have the opinion that change, _any_ change should be avoided and so they avoid doing any changes that aren't either tagged as security fixes or causing an outage.
Kernel.org's road to recovery Posted Oct 13, 2011 8:23 UTC (Thu) by Klavs (subscriber, #10563) [Link]
Not to be annoying here - but fact remains, that upgrading to new versions of the kernel, also includes new features, which in turn may add more security problems, than the bugfixes solved.
There's a reason people pay RHEL to backport ONLY fixes (bugs, security etc.) - so the change becomes as little as possible - increasing the likelyhood of the amount of bugs with security impact going down, as time goes by and bugfixes are applied.
Kernel.org's road to recovery Posted Oct 11, 2011 15:48 UTC (Tue) by vonbrand (subscriber, #4458) [Link] You contend that my statement that for few bugs developers are aware of their security implications. Fine. Then prove me wrong by showing lots and lots of examples of patches where the developer did know the security impact. The "apply important fixes" angle is presumably well covered by the stable kernel series. If somebody wants to do their own work here, they'd better know what they are doing. Just grepping around for some keywords in the changelogs won't get them very far. Your second point is pure nonsense, ext* (and a lot of other classes of patches) are important. Nobody is advocating suppressing any class of patches, just flagging commits with potential miscreant atractors. Third, noone I heard of is trying to supress security information. Nobody is in any position to do so, in fact. What I do see is efforts to fix security bugs, and get the fixes out to anybody affected as soon as humanly possible, hopefully without alerting would-be miscreants beforehand. Sure, it's not perfect, you are wellcome to propose ways of making it more fluid. And yes, LWN's security errata page is a part of this effort. I never said exploits are written as you say, so this point is moot. Security through obscurity works as long as the attackers are in the dark, which will usually be for a limited time only. So it can have a short term beneficial effect, but won't normally work long term. That is all I said. If the net effect of posting in upstream changelogs that a patch fixes, say, an overflow is a net positive or negative is very much up for debate. AFAICS, there are clear negative effects (miscreants grepping, "only apply flagged security fixes" mindset) and few (if any) positive ones, so the net result would be a loss. You clearly see it otherwise, but haven't shown any positive result of your proposal. And the ones in charge of writing the kernel's commit messages are the ones in charge of the decision, not you or me.
Kernel.org's road to recovery Posted Oct 13, 2011 8:32 UTC (Thu) by Klavs (subscriber, #10563) [Link]
I don't understand why you keep believing that any "grep'able" info in changelogs as to some bugfix having a security impact, will in ANY way help someone who is smart enough to actually exploit it.
If you're smart enough to exploit a bug of a certain type - you'll be looking at the code(!) for any lines of C - that looks like the certain bug you want to exploit. I would like to hear of a person - smart enough to actually write an exploit, but dumb enough to be helped by information of security impact in changelogs? Pls. - just find me one, who can actually write an exploit, who thinks he/she is helped by such a msg in a changelog :)
As you may have gathered, I see no reason for actively excluding available information of bugs having a security impact.
Anyone dumb enough to think because it sometimes says along the lines of "security impact" in the changelog (which it actually already does some times AFAIK) - they should only upgrade when that's the case - is already doing their job horribly - and won't be worse off, if any relevant information was actually in the changelogs.
Kernel.org's road to recovery Posted Oct 13, 2011 8:47 UTC (Thu) by malor (subscriber, #2973) [Link]
And vonbrand is saying 'all bugs are security-related'. If that's true, which I do NOT grant, but if it is, then adding known security impacts to a bugfix note is irrelevant, because the bad guys are going to be looking at the code on EVERY patch.
Only good guys read changelogs, basically, so hiding security information only hurts good guys. And it makes the Linux kernel look more secure than it actually is, which is another form of lying by omission.
There is a reason why many many admins try to limit patches to known security-related issues... it's because they're constantly getting new features shoveled at them, with brand-new potential, unanalyzed security impacts. And programmers are very good at introducing weird and subtle regressions with their fixes. Architects and administrators only get shit when stuff breaks, so they try to change as little as possible with a setup, once they know it works. Even tiny tuning adjustments in the kernel code can throw a large-scale application out of kilter, so the people in charge of actually putting all that abstract code to real work in the world try to avoid running new code in a given application unless they either have to, or need new features.
The harder the programmers make it on the architects and administrators, the more appealing the BSDs, Solaris, and even Windows look. And hiding security impacts makes it much harder for them to do their jobs. Programmers just wave their hands and say "You should just run all the code we give you, no matter what", but they don't lose their jobs when the cluster dies.
Kernel.org's road to recovery Posted Oct 13, 2011 17:33 UTC (Thu) by dlang (✭ supporter ✭, #313) [Link]
and since the security impact of the patches is usually not known at the time the commit is written, if people only apply patches that say "this is a security patch" in the commit, they will skip installing a lot of security related fixes.
This can lead to worse security than not making such comments in the commit message.
In my opinion, this is a far bigger reason to not put such comments in the commit message than worries about bad guys reading them
Kernel.org's road to recovery Posted Oct 13, 2011 20:20 UTC (Thu) by raven667 (subscriber, #5198) [Link]
This is probably something that reasonable people can disagree on, whether it is better to be protected against _some_ attacks but have a system which runs stably or be protected against _more_ attacks (but not all, the kernel developers do not claim perfection but instead the opposite) and suffer stability issues due to unrelated changes.
Kernel.org's road to recovery Posted Oct 15, 2011 16:40 UTC (Sat) by PaXTeam (subscriber, #24616) [Link]
> if people only apply patches that say "this is a security patch" in the
> commit, they will skip installing a lot of security related fixes.
if security fixes are marked as such then how can they be missing 'security related fixes'?
> This can lead to worse security than not making such comments in the commit message.
how does fixing a security bug *decrease* security?
Kernel.org's road to recovery Posted Oct 15, 2011 21:19 UTC (Sat) by dlang (✭ supporter ✭, #313) [Link]
not all patches that have security implications are known at the time they are created
so if you only install fixes that were tagged as security fixes, you will miss other fixes that have security implications because those implications were not known at the time they were written, and so they were not tagged.
it's not that fixing a security bug decreases security. what decreases security is the attitude that if it's not tagged as being a security fix, then it doesn't have security implications.
tagged as a security fix guarantees security implications
not tagged as a security fix does not guarantee that there are no security implications.
And if even you are making the mistake that tagging known security fixes means that other fixes don't need to be applied (on the basis that they don't have security implications), then you have just proven the case that many of the kernel developers are trying to make, that tagging some patches as security related will cause people to ignore the others and have less security than updating to a newer version with all of the fixes
Kernel.org's road to recovery Posted Oct 16, 2011 6:35 UTC (Sun) by malor (subscriber, #2973) [Link] But less stability, because patches have an annoying habit of introducing both regressions and new, unwanted features, which can themselves, of course, have all kinds of nasty security implications.
you have just proven the case that many of the kernel developers are trying to make, that tagging some patches as security related will cause people to ignore the others and have less security than updating to a newer version with all of the fixes That's up to them to decide. The guys running these huge, complex systems are pretty goddamn good at what they're doing, and you guys are forcing new, untested code down their throats. Let people have their own agency, and make their own decisions. Don't try to force them to do things the way YOU think they should, sitting there coding on your laptop. Let the guys (and gals) standing in those roaring data centers full of thousands of machines make those calls for themselves. Just be honest, and things will come out better for the people who choose to use your code. If you are not expert in large-scale systems management, you shouldn't try to substitute your judgement for those who are.
Kernel.org's road to recovery Posted Oct 16, 2011 21:37 UTC (Sun) by PaXTeam (subscriber, #24616) [Link]
> not all patches that have security implications are known at the time they are created
so far so good.
> so if you only install fixes that were tagged as security fixes, you
now, following this logic, noone will ever be able to apply all security fixes since the security impact of a given commit may reveal itself any time in the distant future. therefore everyone who applies anything (tagged or not) is in a constant state of 'not tagged as being a security fix, then it doesn't have security implications'. IOW, i don't see the usefulness of your statement, it looks like a tautology.
> what decreases security is the attitude that if it's not tagged as being
why does it decrease security?
and since you've just established that everyone can only do selective backporting, regardless of commits being tagged with whatever or not, this attitude is seemingly prevalent, even you suffer it yourself, so why does it matter again?
> And if even you are making the mistake that tagging known security fixes
actually, i don't make that mistake, in fact, i don't see it as a mistake and you have yet to explain *why* it is a mistake at all. for a start, your acknowledging that fixing a security bug doesn't decrease security means that you're already in contradiction.
> then you have just proven the case that many of the kernel developers
this one bleeds from several wounds, i'm afraid:
1. you haven't shown evidence that people are actually ignoring anything else but explicitly marked security fixes (i think i asked this one before ;).
2. you haven't shown evidence that ignoring anything but explicitly marked security fixes is a bad thing (you actually acknowledged that it's not, now what ;).
3. you haven't explained what 'all of the fixes' means. you and others already said that *everything* not proven otherwise is a security fix therefore the same everything must be backported by everyone who cares which in practice is possible only by following linus's git HEAD. i bet even you don't dare to do that to your company's servers (i actually wonder what you do given that you don't use -stable either).
4. you haven't shown evidence that *not* ignoring (i.e., backporting) random unmarked paches increases one's security/etc. you see, all those security and other fixes are the result of some *earlier* change that *introduced* the problem, so you'd have to somehow prove that the net result of backporting everything under the sun (i.e., following git HEAD) is positive, not negative.
Kernel.org's road to recovery Posted Oct 17, 2011 1:09 UTC (Mon) by raven667 (subscriber, #5198) [Link] 3. you haven't explained what 'all of the fixes' means. you and others already said that *everything* not proven otherwise is a security fix therefore the same everything must be backported by everyone who cares which in practice is possible only by following linus's git HEAD. i bet even you don't dare to do that to your company's servers (i actually wonder what you do given that you don't use -stable either). I don't know if you pay attention to kernel development but from my understanding running the latest Linus kernel release is what is recommended to have all the fixes. I'm sure there are some people who _do_ run raw Linus kernels who want the latest fixes as soon as they are out of the oven. The current Linus kernel certainly has more security relevant fixes than any vendor kernel which only has backports as the very nature of cherry picking backports is going to miss security fixes which aren't known at the time the fix is made. That is what the kernel release announcements recommend. Many people think running the latest kernel.org release is potentially too disruptive due to other changes unrelated to bug and security fix work. Unfortunately trying to separate feature from fix work didn't work as a process from the kernel developers perspective which is why the development process was changed in the transition from 2.4 to 2.6 so that feature and architectural changes are fed right into the main line of development. I think that the major vendors (RedHat, Debian, SuSE, various embedded, etc.) should continuously re-evaluate how close they can run to the main line of kernel.org kernels rather than trying to cherry pick backports and maintain their own "stable" forks. Ideally the regular kernel releases would be equivalent in stability and superior in security than the current situation.
Kernel.org's road to recovery Posted Oct 17, 2011 6:53 UTC (Mon) by malor (subscriber, #2973) [Link] Unfortunately trying to separate feature from fix work didn't work as a process from the kernel developers perspective And that, right there, is the single core problem with Linux security. Security is hard. It means more pain during development. Separating fixes and features is a pain in the ass. But if it doesn't get done, you end up in the snarl they're in now. Even the developers themselves can't provide secure shared access to a single Linux kernel image. How can anyone else expect to?
Kernel.org's road to recovery Posted Oct 17, 2011 7:28 UTC (Mon) by dlang (✭ supporter ✭, #313) [Link]
the problem is that one person's 'bugfix' is another person's 'new feature'
especially when the bugfix can end up refactoring the code in the process.
yes, this is a big problem with Linux, but the rate of fixes (of all kinds) is the great strength of Linux. At this point nobody knows how to fix the weakness without giving up the strength. There are other OS groups (openBSD comes to mind) that seem like they follow the philosophy that you are advocating, but despite the fact that they had several years of a head start on Linux, their development models have caused them to be far less useful on current hardware. (and therefor any security benefits they may provide, far less useful)
I don't understand your comment about the kernel developers being unable to provide shared access to a single kernel image.
are you referring to the fact that there was a privilege escalation vulnerability on kernel.org? if so, any conclusions about what the problem was need to wait until we learn what happened. And in any case, the vast majority of the kernel developers were not involved in administering the systems (and note that it was several systems, not a single system)
Kernel.org's road to recovery Posted Oct 13, 2011 18:32 UTC (Thu) by raven667 (subscriber, #5198) [Link] And it makes the Linux kernel look more secure than it actually is, which is another form of lying by omission. I just have to respond to this one thing. The kernel announcements and general discussions have been pretty open about the belief that it is _less_ secure than many people would like. Just the other day in a discussion about containers and namespaces, major kernel developers comment was that it wasn't worth the effort to increase security separation between containers because there will always be local root exploits that will break separation. The kernel developers do not appear to be trying to claim more security by omission, they are explicitly claiming less.
Kernel.org's road to recovery Posted Oct 14, 2011 1:29 UTC (Fri) by malor (subscriber, #2973) [Link] major kernel developers comment was that it wasn't worth the effort to increase security separation between containers because there will always be local root exploits that will break separation. Well, that's good in the sense that they're admitting there's a big problem. But I would argue that if they can't keep user accounts secure from gaining root access, then there's really not much point to even HAVING user accounts. If your summary is accurate, there's no way you can safely use Linux to share access between potentially hostile accounts on one kernel. You can sorta do it through virtualization, but running an entire kernel per user is a hell of a lot of overhead to carry around. Security is probably the hardest problem in computing, and if they are indeed saying "there will always be root exploits", it sounds like they're giving up on the idea entirely. They want to make it go fast, and security be damned. This is something that people need to be very aware of; that wording makes it sound like they're throwing in the towel. If so, Linux is no longer appropriate for many use cases, particularly when lives are at risk.
Kernel.org's road to recovery Posted Oct 14, 2011 3:10 UTC (Fri) by dlang (✭ supporter ✭, #313) [Link]
they are working to fix any problems they find as fast as they can.
the kernel developers are not giving up.
there was one person who made the claim in the discussion on containers that containers were not good enough, but on the other hand, I'm one of the people who says that virtualisation isn't good enough isolation for some applications due to possible bugs in the hypervisor. It all depends on how much security you are going for.
This is part of the reason that SELinux is optional.
Kernel.org's road to recovery Posted Oct 14, 2011 7:14 UTC (Fri) by anselm (subscriber, #2796) [Link] Security is probably the hardest problem in computing, and if they are indeed saying "there will always be root exploits", it sounds like they're giving up on the idea entirely. Not necessarily. Maybe they're just being realistic while they're trying to fix problems as they are discovered (and prevent them where they can). With a program of the size and complexity of the Linux kernel, I would be very sceptical of anybody claiming the logical opposite, namely that »there will never be even a single root exploit«. Not even the OpenBSD folks subscribe to that kind of hubris ;^)
Kernel.org's road to recovery Posted Oct 15, 2011 16:31 UTC (Sat) by PaXTeam (subscriber, #24616) [Link]
> You contend that my statement that for few bugs developers are aware of
> their security implications.
actually no, what i contend is your assertion of there being 'very, very few' commits fixing a bug with a known security impact to be of relevance. have you got evidence for this assertion?
> Then prove me wrong by showing lots and lots of examples of patches
this is tangential, but i actually did provide examples in the past, here on LWN, in threads you also participated in so let google be your friend if you really want to see examples.
> The "apply important fixes" angle is presumably well covered by the
i guessed you'd bring this up but it means you also shoot yourself in the foot ;). you see, there's a contradiction in your statements. according to you:
> Count me in the camp with "any kernel bug that can't be shown to be
that implies that most of the bugfixes must be backported to -stable but we know that's not the case. therefore either -stable doesn't apply all 'important fixes' (including security fixes) or most bugfixes aren't security related as you claimed before. which is it?
> Your second point is pure nonsense, ext* (and a lot of other classes of
we aren't talking about suppressing patches per se (that'd be crazy), but important information in commit messages (security impact in general, file system corruption for the ext* case i brought up as comparison).
so you're admitting that there's a useful category of impact information that you would not advocate to suppress. that's a good step! now you'll have to explain why 'security impact' is different from 'filesystem corruption' in this regard. for that you'll have to explain how exploiting security bugs can never ever corrupt filesystems (else you'll have to conclude that at least some of the security fixes must be marked for filesystem corruption, which is enough to grep for, contradicting your other desire to make security fixes non-greppable), and also why helping miscreants to corrupt filesystems is a good thing (i.e., you can't use the same argument for contradicting purposes).
> Third, noone I heard of is trying to supress security information.
did you read the Linus mail (and the whole thread actually) i linked to? he admitted it.
> What I do see is efforts to fix security bugs, and get the fixes out to
how can they be fixing security bugs when they don't even know what bugs have a security impact? or are you now praising selective fixing of bugs?
> And yes, LWN's security errata page is a part of this effort.
so when kernel devs put security impact info into a commit it's a bad thing but when LWN points at the same commit it's a good thing. i think you want to try this one again.
> I never said exploits are written as you say, so this point is moot.
but you did, even in this latest response in yours:
> [...]hopefully without alerting would-be miscreants beforehand.[...]
this statement means that you assume that people can write exploits *because* they read about exploitable bugs in the commit message. that is, you're claiming that to exploit a kernel bug one has to read about the fact that a given commit fixes it, and magically the exploit appears out of thin air.
in the reality out there, people writing exploits couldn't care less about what the commit message says about the security impact, instead they'll look at the actual code and decide based on that. in other words, your justification to cover up security impact information in commit messages doesn't stand on any legs so far.
> Security through obscurity works as long as the attackers are in the
have you got evidence that attackers are in the dark when all they can rely on is the code in a patch (vs. the commit message)? as a sidenote, i'd like to hear your theory on how 0-day exploits are written 'cos they certainly can't be based on any security related information in the commits.
> AFAICS, there are clear negative effects (miscreants grepping,
you haven't shown any evidence for this.
> "only apply flagged security fixes" mindset)
you haven't shown any evidence for this. (see a theme here? repeating the same statement without evidence doesn't make it any more true)
> and few (if any) positive ones,
you must be out of your mind if you think that making security fixes public has no positive outcome. what else on earth would allow people to fix their systems?
> so the net result would be a loss.
since all the premises for this conclusion have yet to be shown to be true, the jury is still out on this one.
> You clearly see it otherwise, but haven't shown any positive result of your proposal.
it's not my proposal, it's what most of the rest of the world does (heck, even the linux world, just ask any distro maintainer how much they appreciate that they have to reverse engineer security impact information from kernel commits).
Kernel.org's road to recovery Posted Oct 15, 2011 21:11 UTC (Sat) by dlang (✭ supporter ✭, #313) [Link]
I actually don't think that all important fixes get backported to -stable, which is part of the reason why I tend not to rely on them as a long-term measure.
I see the -stable branch as useful for fixing any functional bugs that slipped through, but I don't rely on them for fixing security bugs
|
|||||||||||||