Quotes of the week [LWN.net]

Bugs are like mushrooms - found one, look around for more...

Maximizing security is hard: whether a bug has security implications is highly usecase and bug dependent, and the true security impact of bugs is not discovered in the majority of cases. I estimate that in *ANY* OS there's probably at least 10 times more bugs with some potential security impact than ever get a CVE number...

So putting CVEs into the changelog is harmful, pointless, misleading and would just create a fake "scare users" and "gain attention" industry (coupled with a "delay bug fixes for a long time" aspect, if paid well enough) that operates based on issuing CVEs and 'solving' them - which disincentivises the *real* bugfixes and the non-self-selected bug fixers.

I'd like to strengthen the natural 'bug fixing' industry, not the security circus industry.

-- Ingo Molnar

to post comments

Read the followup by pagexec

Posted Jun 9, 2011 1:42 UTC (Thu) by jamesmrh2 (guest, #31680) [Link] (43 responses)

Especially:

let me tell you now a real distadvantage of your coverup: you're hurting the good guys (the defenders) a lot more than you're hurting the bad guys (the attackers). why? because of the usual asymmetry of the situation we often face in security. an attacker needs to find only a single commit silently fixing a security bug (never mind finding the earlier commit that introduced it) whereas the defenders would have to find all of them.

(link)

Read the followup by pagexec

Posted Jun 9, 2011 5:55 UTC (Thu) by dlang (guest, #313) [Link] (41 responses)

this cuts both ways. If people think "I only need to apply the patches marked as being security fixes to be safe", then any of the many other patches that were not recognized as being security risks at the time become attack vectors.

and by making a practice of trying to label the security implications of every patch you encourage this behavior.

Read the followup by pagexec

Posted Jun 9, 2011 8:52 UTC (Thu) by ballombe (subscriber, #9523) [Link] (40 responses)

A simple solution is to ask for CVE numbers for all commit in Linus tree and all commit in the stable tree.
At the very least, this would enlighten the security industry.

Read the followup by pagexec

Posted Jun 9, 2011 15:59 UTC (Thu) by martinfick (subscriber, #4455) [Link] (21 responses)

It would also be playing their game. But, seeing how the whole point is to not play their game...

Read the followup by pagexec

Posted Jun 10, 2011 10:30 UTC (Fri) by mingo (subscriber, #31122) [Link] (20 responses)

Also, his argument is rather loaded in a "when did you stop beating your wife" sense: there's no "cover-up" - there's only a refusal to use a fundamentally flawed concept.

Asymmetry between attackers and defenders is a physical property of the universe: it is much easier to destroy structure than to create structure. (the second law of thermodynamics)

So an attacker will always have to find only one weak spot while the defence has to know about all weak spots.

No amount of kernel code or logistics will be able to counter the second law of thermodynamics.

We defenders should concentrate on what we are good at, i.e. we should concentrate on creating stuff: more fixes, more robust code, layered defences and making sure that we win the numbers game: having more useful software and more people on our side. That will put a natural cap on parasitic activity.

Read the followup by pagexec

Posted Jun 11, 2011 12:10 UTC (Sat) by spender (guest, #23067) [Link] (19 responses)

The same old, tired, unrealistic argument. It sounded familiar to me, and my response still has yet to be me with a realistic solution. Here's the thread from over a year ago:
http://lwn.net/Articles/374224/

"Bugs are bugs" yet virtually everyone but the kernel developers themselves are using either a distro kernel or a -stable kernel that both involve selective backports of bugfixes. -stable even often lags months behind (or misses completely) some security backports. I know this to be fact because I backport these myself and then see months later when -stable tries to apply the very same patch, or I see that my patch still remains months later.

Are you holding on to some belief that -stable can be perfected? Do you realize that every time a CVE gets posted by Eugene or Kees or anyone else on oss-sec for a silent upstream fix it undoes your hard obfuscation work and that you're essentially just creating more work for everyone?

Even the people on this site who have been lured into the smooth talk of the "bugs are bugs" tautology I'm sure aren't running the latest -git kernel on their production systems, so I still don't understand why this stillborn idea is treated as some pinnacle of intellectual achievement.

Backports are inevitable and are here to stay. Stop complaining about the very real separation between vulnerabilities and bugs; no amount of hand-waving (as much as you think it can) can remove that separation. You're not even capable of backporting fixes for all bugs, so the whole cry of "you're missing out on bugfixes!" is just as applicable.

The "solution" supported by Linus and yourself isn't even a solution. You have no solution. Show me the users implementing your ideals on a production server. If you believe they exist, I've got a nice bridge to sell you.

"Bugs are bugs" -- it's succinct, it's clever, and syntactically persuasive. But I'm sorry, it's just ridiculous and has no place in the real world. If you believe otherwise, again, show me these users and these production machines. If you can't, then maybe you should rethink the pointless exercise of active obfuscation.

-Brad

Read the followup by pagexec

Posted Jun 11, 2011 16:45 UTC (Sat) by mingo (subscriber, #31122) [Link] (18 responses)

"Bugs are bugs" yet virtually everyone but the kernel developers themselves are using either a distro kernel or a -stable kernel that both involve selective backports of bugfixes.

You are misrepresenting our position - nobody suggested to not do the -stable kernel, why are you claiming that? The -stable kernel is a back-port of all fixes that matter to users: data corruption bugs, boot hangs, crashes, mis-behavior and all sorts of other bugs - regardless of whether they have a known security impact or not.

Our argument and policy is to not treat security fixes differently from other bug fixes, because their identification is so imperfect in practice and because their special handling creates a false sense of security (and due to a host of other reasons - see the various discussions for a complete argument). Instead our argument is that if you want to be secure, stay updated with all bugfixes, not just the CVE list.

We are not alone in that view: see this MIT research paper on the topic.

Here's the abstract of their paper:

In this paper, we question the common practice of as- signing security impact ratings to OS updates. Specifi- cally, we present evidence that ranking updates by their perceived security importance, in order to defer applying some updates, exposes systems to significant risk. We argue that OS vendors and security groups should not focus on security updates to the detriment of other updates, but should instead seek update technologies that make it feasible to distribute updates for all disclosed OS bugs in a timely manner.

Which is part of Linus's and my argument as well.

Your attempt to distort and ridicule our position is disingenuous.

Read the followup by pagexec

Posted Jun 11, 2011 17:54 UTC (Sat) by spender (guest, #23067) [Link] (17 responses)

I'm not distorting at all. You're apparently incapable of reading what I wrote. You claim that -stable is a backport of all fixes (added qualifier here from you that it's only fixes that matter to users). I said -stable was selective backports, and gave evidence for my claim -- that even some security fixes take months to arrive in -stable, or don't appear there at all. So it's clearly *not* all bugfixes, and not even all security-relevant bugfixes.

Are you saying I'm wrong? That the latest -stable (and which -stable would that be? does -longterm count? do they all have *all* the bugfixes?) is being used by all users of Linux?

Are you saying a bug is only a bug if you deem it something that matters to users? I thought you said earlier on LKML that any bug in the kernel could be a kernel vulnerability! This is very troublesome.

If you concede (as you must) that even the very newest -stable users still aren't receiving *all* the bugfixes, then you'll agree that making it easier to spot things that need to be backported will help improve the entire process: like, say, by not intentionally obfuscating the commit message of something you know to be a security vulnerability. I'm even being kind by taking the example most closest to your ideal: if we were talking about the distro kernels i'm guessing at least 90% of people run, your own policy does nothing to change everyone else's policies -- the only effect it can have is to further delay the fixing of the vulnerabilities and bugs the distros have chosen to fix.

It's nice to quote from a paper that demonstrates the detrimental effects of your obfuscation strategy and then use it to reaffirm the use of that strategy. I don't see that it reaffirms your position at all: it only shows that your strategy contributes to the problem. You don't solve the problem of distros taking too long to push out kernel updates. BTW you also forgot to mention that the paper you cited was written entirely by Ksplice employees. Since Ksplice sells a product and service entirely built around this concept of security-through-bug-elimination, their conclusion is not surprising to me at all.

Here comes the real irony: I met some of the ksplice guys some time ago, and they asked me what bugs they should be patching. I said to them: if you can, all of them, because the developers silently patch so many security vulnerabilities that you'll be missing out on a lot of vulnerabilities that will be obvious to an attacker. If you take a look at the list of patches against an example system from http://www.ksplice.com/uptrack/using you'll see how realistic it was for even the very people whose paper you quoted to live up to the ideals you wrongly believed they shared ;)

So the real question is, will you dig your feet in the sand on this untenable position, hoping to solve some nonexistent problem through "a bug is a bug" mantras, or will you join the rest of us in reality to work on solutions to the problems people actually face?

-Brad

Read the followup by pagexec

Posted Jun 11, 2011 18:51 UTC (Sat) by mingo (subscriber, #31122) [Link]

If you concede (as you must) that even the very newest -stable users still aren't receiving *all* the bugfixes, then you'll agree that making it easier to spot things that need to be backported will help improve the entire process:

There's nothing to 'concede', it's obvious that marking bugs for -stable backports is useful. You are banging on open doors here really, we are already using such a method to help bug-fix backports and have been using it for years: the "Cc: stable" tag in commits.

We just refuse to play the CVE attention-seeking, self-promotion, reality-distorting security circus you are pushing for, for all the reasons outlined.

Read the followup by pagexec

Posted Jun 11, 2011 20:54 UTC (Sat) by Julie (guest, #66693) [Link] (15 responses)

the problems people actually face

If by 'people' you mean users, I'm pretty sure that users in general can get just as upset by a bug that results in sporadically hanging their box, stopping it from ever booting, corrupting their filesystem, breaking essential drivers or something else that results in an unuseable machine as by a bug that results in, or _might_ result in a successful attack :-)
I do accept that most bugs in the wild being fixed on a daily basis have a far less dramatic impact than the above mentioned but still, all things considered, 'a bug is a bug' works for me.

Read the followup by pagexec

Posted Jun 11, 2011 21:40 UTC (Sat) by spender (guest, #23067) [Link] (14 responses)

That's not the same as 'a bug is a bug' -- you've actually implicitly demonstrated why it's so silly: you've categorized already a subset of all bugs into "serious" bugs, and then further categorized them into boot issues, filesystem corruption, driver problems, general crashes, and potential security issues. So for you, as with everyone, you're concerned about serious bugs, but you don't care what type they are. Well, to determine if a bug is serious, you need to do the very kind of classification 'a bug is a bug' rejects. Do you think upstream avoids mentioning filesystem corruption when they know it to be true? That would be rather unnecessarily malicious on their part, would it not? So why is it acceptable to avoid mentioning security impact when they know it to be true?

So I don't think you really mean 'a bug is a bug' really works for you. Are you using a distro kernel? How often are you updating?

-Brad

Read the followup by pagexec

Posted Jun 11, 2011 21:55 UTC (Sat) by dlang (guest, #313) [Link] (9 responses)

you are misunderstanding the classification they do.

they classify bugs into 'serious' or 'not serious', but no more

what you list as categories ('boot bugs', 'filesystem corruption', ets) are not categories that things are sorted into, they are just examples of types of bugs that fall into the 'serious' category.

Read the followup by pagexec

Posted Jun 11, 2011 22:11 UTC (Sat) by spender (guest, #23067) [Link] (8 responses)

Sorry, you're right. I must have imagined all this:
http://lkml.org/lkml/2008/7/16/17
http://lkml.org/lkml/2008/4/18/154
https://lkml.org/lkml/2011/6/1/294
http://lkml.org/lkml/2009/10/1/590
http://lkml.org/lkml/2002/11/7/137
just from a cursory search.

Come on, this is ridiculous. I'm not going to argue with someone so detached from reality. Prove to me 'a bug is a bug' and show me even one example where the developers have not gone out of their way to say that a particular bug caused filesystem corruption when they knew it to be the case. Just because you say they don't mention a bug fixes booting issues, or filesystem corruption, doesn't make it the case. If your reply doesn't contain a real example as proof, don't bother replying.

-Brad

Read the followup by pagexec

Posted Jun 11, 2011 22:33 UTC (Sat) by mingo (subscriber, #31122) [Link] (7 responses)

Just because you say they don't mention a bug fixes booting issues, or filesystem corruption, doesn't make it the case.

Well, in my 15+ years of kernel maintenance experience, developers are generally very bad at writing changelogs, so yes, even if developers are aware of some serious side-effect, they more often than not do not properly mention it in the changelog.

You mention boot crashes: i'm an arch maintainer so i look after lowlevel code which is literally a magnet for boot crashes, and the majority of boot crash fixes come to me with a change-log that does not mention this fact - often i have to reconstruct it myself.

If you ever were in the position to write features and fix bugs for a big and popular OSS project you'd realize that it's a pretty natural process: developers are happy that they fixed the bug, they describe how they fixed it and what the behaviour now is - then they move on, they don't live in the past trying to figure out what other side-effects a bug might have and they don't try to find ways to break into the system.

They just spent hours or days debugging some bug and want to move on, so writing the changelog is often only an afterthought.

Linux is also very international, so often there's a language barrier that reduces the willingness of developers to write good changelogs and which degrades the quality of changelogs.

There's exceptions, but it's rare.

Come on, this is ridiculous. I'm not going to argue with someone so detached from reality.

I think you are the one detached from reality - you clearly have absolutely no idea how Linux kernel developers think when they write fixes and in this discussion you have demonstrated that you have no idea how the Linux kernel development process works.

To compound your mistake you also insult almost everyone who happens to disagree with you. I hope you behave in a different way with your loved ones.

Read the followup by pagexec

Posted Jun 12, 2011 12:18 UTC (Sun) by nix (subscriber, #2304) [Link] (6 responses)

To compound your mistake you also insult almost everyone who happens to disagree with you.

This is particularly foolish, because it goes directly against his primary goal, which is to get developers writing code with security in mind (well, OK, his *other* primary goal is to get credit come what may, but that is endemic in the security field so can be disregarded).

Unfortunately he is so unpleasant when doing it that virtually everyone who might benefit from his instruction is so turned off by his supercilious and venomous interaction style that they rapidly disregard everything he says, if they don't just killfile him and leave it at that.

I'm not a security person, just someone writing low-level userspace code some of which is network-exposed, i.e. just the sort of person he might be interested in educating, you'd think. When he started posting here I had no idea who he was. Within about fifty posts I'd realised that he obviously knew his stuff, but his interaction style was so unpleasant and so predicated on never admitting error and never giving anyone the benefit of the doubt that by that point I wouldn't believe him if he said the sky was blue. And I don't think I'm particularly unusual.

Even people who can cope with the... robust interactions on the l-k list get turned off by the spender style: the vast number of good developers who don't go near l-k because it's too unpleasant are not even going to consider paying attention to spender, because listening to his security lessons is like wading through acid. Nobody wants their lessons interspersed with commentary on how stupid and foolish they are, and how they are engaging in coverups against the public good (though that last is more PaXTeam's shtick). There's a reason decent teachers don't attack their students.

I have an aunt who communicates in a similar style to spender. Or she would, if she could -- nobody talks to her. Nobody wants to.

Read the followup by pagexec

Posted Jun 12, 2011 12:56 UTC (Sun) by spender (guest, #23067) [Link] (5 responses)

Call it whatever you like, I don't suffer fools gladly. In my view, you're like a child that walks into an advanced mathematics course and wants to talk over the professor the entire time because you foolishly think you have something immensely important to say.

You have a lot of comments about my ability as a teacher; I have some advice for you as well: a wise student knows to do his/her homework and when to be quiet and listen.

BTW while you all were busy spouting nonsense, I just backported a couple security fixes that didn't make it into -stable. In the one case, it made it into .39-stable but was left out of .32-longterm. Oh, and the fixes were all months old. But hey, forget objective evidence, here talking out of your ass about how people are running kernels that have all the bugs fixed is king, so suit yourselves.

Times like these remind me of the cognitive dissonance demonstrated here: http://lwn.net/Articles/290964/ and this post from the PaX team: http://lwn.net/Articles/290968/

See you next year.

-Brad

Read the followup by pagexec

Posted Jun 12, 2011 13:14 UTC (Sun) by mingo (subscriber, #31122) [Link]

BTW while you all were busy spouting nonsense, I just backported a couple security fixes that didn't make it into -stable. In the one case, it made it into .39-stable but was left out of .32-longterm. Oh, and the fixes were all months old. But hey, forget objective evidence, here talking out of your ass about how people are running kernels that have all the bugs fixed is king, so suit yourselves.

You make a quite elementary mistake of logic here: why do you assume that -stable backports are perfect? It's a human process and human processes are never perfect.

Our argument is that adding CVEs to change-logs and treating security bugs differently from other bugs is counter-productive, for all the reasons we outlined. You never addressed that simple argument heads on in this thread - and i submit that you cannot.

Read the followup by pagexec

Posted Jun 12, 2011 13:30 UTC (Sun) by mingo (subscriber, #31122) [Link]

I have a very simple question to people like you who seem to suffer from excessive narcissism: please name three other persons who are smarter and more capable than you, in the field you work in. (In most cases they are utterly unable to answer that question honestly.)

Read the followup by pagexec

Posted Jun 12, 2011 13:55 UTC (Sun) by nix (subscriber, #2304) [Link] (2 responses)

Call it whatever you like, I don't suffer fools gladly. In my view, you're like a child that walks into an advanced mathematics course and wants to talk over the professor the entire time because you foolishly think you have something immensely important to say.

My apologies for daring to post in any thread you have chosen to grace with your mighty presence, sire. I should have known my place, at the bottom forever, and kept silent while the adults were talking, never daring to venture comment, since that's how the free software world works.

(btw, professors really do not act like you do.)

I will shut up now, because this is verging on a flame, even if a justified one.

Read the followup by pagexec

Posted Jun 12, 2011 14:05 UTC (Sun) by nix (subscriber, #2304) [Link] (1 responses)

Apologies to Jon for lowering the tone of the discussion on LWN possibly even more than spender already had. I've filtered spender and PaXTeam out on the grounds that though they may occasionally post worthwhile things, most of their comments have me shaking with anger in seconds, so it's probably bad for my health for me to read any more of them. I will not contribute further to lowering the S/N ratio when they appear.

Because driving students away forever shaking with anger is what a good professor does.

Malicious content detected

Posted Jun 21, 2011 5:07 UTC (Tue) by gabucino (guest, #72504) [Link]

WARNING: the comment above was posted from an insecure operating system, therefore considered potentially harmful.

Read the followup by pagexec

Posted Jun 12, 2011 0:38 UTC (Sun) by Julie (guest, #66693) [Link] (3 responses)

you've categorized already a subset of all bugs into "serious" bugs, and then further categorized them into boot issues, filesystem corruption, driver problems, general crashes, and potential security issues.

Not really, with respect, _you_ just did. I just thought I'd mentioned a varied collection of bug examples (whilst not being altogether serious in my choice of 'serious' ones)- bugs can result in more than one type of offending behaviour, and sometimes might even be discovered/fixed _by accident_ when the bugfixer is fixing up something else.

Which is why I think the most sensible thing is to find bugs regardless of their effect, which to me is what 'a bug is a bug' means - it's a philosophical _starting point_, not an attempt to blur together classifications that may be artificial anyway. Some observed misbehaviour might tell you where to start looking but even then you might be wrong with any assumptions you immediately jump to and it's a good idea to be humble enough to allow the possibility. At best, you avoid a very red face later.
At the simplest level you look at suspicious code or features, find things that shouldn't be there and are or should be there and aren't, and fix it (well, sometimes this turns into development...).

Argue about whether it's a bug by all means and definitely discuss its context and the best way of fixing it in light of the intended purpose and possible side effects of the code it's in, but come to a conclusion and sort it. That does seem to be what the mailing list is for.
And don't spend all day explaining to others exactly how and why you did it, why would you want to do that, there are plenty more to fix? ;-)

Clarity from upstream is great (and I've seen maintainers make contributors go back and re-write changelogs to make them clearer) but nobody that writes code cares for explaining piecemeal to everyone what they did. You'd spend all your time documenting your changes instead of making them.
The poor documentation/lack of helpful comments in code/unhelpful config 'help' that sometimes results from this attitude irritates or inconveniences me a lot more than shoddy changelogs. Users probably won't read the changelogs and reviewers probably ought to look at the code too in any case. But from what I've seen any idea of a deliberate campaign of obfuscation is a fantasy.

So for you, as with everyone, you're concerned about serious bugs, but you don't care what type they are.

Actually, I am concerned about all bugs. I want a perfect kernel :-) It's a quality issue and if Linux can't do it with its talent and development model then no-one can.
It might be a practical impossibility, but this is due to restrictions imposed by a shortage of bug fixers/time. Not to do with aspiration or potential or even method :-)

Are you using a distro kernel? How often are you updating?

I use a distro kernel until I've set up my git kernel tree on the machine and done a hand-stripped config, then I use -next-2011whatever or -rcN or whatever I just built (including patch testing kernels I'm too lazy to reboot from :-)). Having said that I do keep an eye on updates and occasionally use the distro kernels and I've never had problems. I know that means I'm not a typical 'user' and if I was a server admin or someone I might take a different attutude - but I did say it 'works for me'.

Read the followup by pagexec

Posted Jun 12, 2011 9:07 UTC (Sun) by mingo (subscriber, #31122) [Link] (2 responses)

Btw., the primary way we back-port fixes to -stable is to simply consider whether the bug fix is relevant to the -stable tree - i.e. we try to answer the "does this fix a bug in an earlier kernel as well?" question.

That is something that bug fixers are generally capable to determine and they are willing to classify it. It's basically the same task they already did when fixing the bug, just time shifted back 3 or 6 months. It's not perfect but it works reasonably well in practice.

The "could this bug possibly be used to be a parasite in the system" question is a lot less interesting and a lot less natural to the average bug fixer (they are not thinking like parasites) and hence this information is a lot less obvious to extract and it's thus also not part of the regular flow of fixing bugs. It is also a lot more complex mathematically, because the space of potential unintended interactions between kernel bugs and the rest of the kernel and thousands of applications is very, very large. It's hard enough for bug fixers to consider all the intended interactions.

So at this stage we do not pretend to be able to answer that question, and we refuse to participate in the CVE security circus, for all the reasons outlined above.

So yes, the upstream policy is that a bug is a bug and that is applied consistently across the board, to -stable as well.

Read the followup by pagexec

Posted Jun 12, 2011 10:09 UTC (Sun) by dlang (guest, #313) [Link] (1 responses)

I'll note that bug fixers tend to only look back a short while when considering -stable fixes (if you look at the number of changesets in the various long-term and -stable supported kernels, you will see that they tend to get a lot of fixes initially, but that the rate of fixes tapers off fairly rapidly)

Read the followup by pagexec

Posted Jun 12, 2011 12:54 UTC (Sun) by mingo (subscriber, #31122) [Link]

I'll note that bug fixers tend to only look back a short while when considering -stable fixes (if you look at the number of changesets in the various long-term and -stable supported kernels, you will see that they tend to get a lot of fixes initially, but that the rate of fixes tapers off fairly rapidly)

Well, that's an expected curve: current empirical studies show that the distribution / time evolution of software defects goes along a Simon-Yule distribution - which goes down sharper than a bell curve as time goes on.

It's also roughly what you get if you use a simple bug fixing model: take the set of bugs still unfixed, and let bug fixers (developers) random walk the code, then there's a fixed probability to find a bug, with every man-hour spent on looking at the code.

As time goes on, the probability increases that a bug will be found, with a 1-1/t probability, where 't' is the time spent on the code. It will be like a bell-curve for a given difficulty of bug. The 'difficulty' of bugs is a constant parameter, so the combined bug metric would show the integral of 1-d/t ('d': difficulty, 't': time), integrated over 'd' and 't', which would still roughly be a bell curve in practice.

Thus older kernels seeing fewer fixes is natural and expected and is not in itself a sign of less attention.

There's also another aspect: the Cc: stable tag does not typically limit how far a fix is getting ported back - that is decided by the -stable maintainers. They will generally port is back as far as it will still cherry-pick cleanly - and they'll notify the patch submitter if there's a conflict with older stable kernels.

But yes, the older a stable kernel is, the higher the chance that a bug is missed and is not backported - this is a classic trade-off.

Read the followup by pagexec

Posted Jun 9, 2011 18:50 UTC (Thu) by dlang (guest, #313) [Link] (17 responses)

but to get a CVE number you have to not only say you fixed a bug, but explain what that bug is and how it could be exploited.

frankly, if I'm fixing a bug I'm far less worried about figuring out ways that it could be exploited than I am in getting it fixed.

and I know that I would prefer to have the kernel hackers spend all their time fixing bugs (and implementing new features) rather than trying to figure out ways that bugs could be exploited.

Read the followup by pagexec

Posted Jun 10, 2011 10:49 UTC (Fri) by mingo (subscriber, #31122) [Link] (16 responses)

Yes, there's also a fundamental problem: fixing bugs requires very concentrated attention - which narrows any thinking about what potential side-effects of a bug can have in practice.

Bug fixers are, very fundamentally, concentrated on understanding *one* specific symptom and then finding the bug, then going to the 'big picture' of the intent of the code and changing the code (and related code) into correct code which implements the original 'big picture' intent.

Note what is missing from those thought patterns: figuring out *other* obscure symptomatic ways to use unintended side-effects of the bug to bring some application (or the kernel) out of its intended path of execution and use that to inflict harm on the system ...

The bug fixer is simply not interested in it because that information is *immaterial* to him: he is interested in *fixing* the bug and then reviewing the rest of the logic to make sure it implements intent correctly.

So bug fixes, even by the best bug fixers, do not carry full information about whether some obscure side-effect of the bug may have some security impact.

Attackers on the other hand only deal with specific bugs and know the various patterns that are dangerous in terms of bringing apps off their regular execution pathways. They also know the various obscure side-effects that bugs can cause and which can be used creatively to elevate privileges. In essence attackers know how to turn existing structure against its own intended purpose, i.e. they know how to be *parasites*.

So this is a fundamental (almost genetic) conflict between 'doers' and 'parasites', and we cannot expect doers to think like parasites, and i argued in my mail that putting CVEs into change-logs only makes the situation worse, not better.

There might be other schemes that improve security - so this is not a refusal to do sensible things at all, it is a refusal to do nonsensical things.

Read the followup by pagexec

Posted Jun 11, 2011 13:14 UTC (Sat) by zakalwe2 (guest, #50472) [Link] (15 responses)

Kernel developers have no interest in the security aspect of bugs and therefore have little understanding or care of the potential impact. When the devastating impact of these bugs is revealed to them in the form of harsh language or working exploits, they gain a transient, whimsical interest in the problem and implement half-baked general solutions that degrade over time through code erosion and neglect. This pervasive attitude towards bugs seriously underestimates their impact, and in turn ensures that no real severity limiting features (exploit mitigation) is ever valued or implemented properly.

Read the followup by pagexec

Posted Jun 11, 2011 13:54 UTC (Sat) by mingo (subscriber, #31122) [Link] (14 responses)

I disagree, if you look at actual evidence you'll see that we kernel developers deeply care about bugs (regardless of whether they have security relevance or not), and fix them as quickly as we can.

We also have various bug mitigation facilities and are extending them continuously. The actual lkml thread where the quote came from is about such a security bug mitigation effort which is being merged into the kernel.

Read the followup by pagexec

Posted Jun 11, 2011 15:10 UTC (Sat) by nix (subscriber, #2304) [Link] (13 responses)

And your reward? Attacks and insults from the usual suspects because the proposed name of the config variable for the new feature is in their eyes wrong.

sigh.

Read the followup by pagexec

Posted Jun 11, 2011 16:13 UTC (Sat) by spender (guest, #23067) [Link] (12 responses)

By usual suspects do you mean Ingo and Linus? You know, since they were the ones who had the problem with the already-existing name.

-Brad

Read the followup by pagexec

Posted Jun 11, 2011 16:55 UTC (Sat) by mingo (subscriber, #31122) [Link] (11 responses)

FYI, pageexec objected to calling it COMPAT/LEGACY_VTIME and called that naming a "lie" and a "coverup".

So yes, the usual suspects are attacking and insulting us - and then in other forums they are denying that it ever happened.

Read the followup by pagexec

Posted Jun 11, 2011 17:08 UTC (Sat) by spender (guest, #23067) [Link] (10 responses)

Perhaps you might like to read further back in that thread, as it already contains the subject of renaming it from the initial proposed name, to a new name. Why did the name need to be changed? This may refresh your memory:

https://lkml.org/lkml/2011/6/6/94
Linus ^

https://lkml.org/lkml/2011/6/6/141
and you! ^

You see, the patch already had a name. That is what we would call "the proposed name" as it was the name given by the author of the patch. But with your view of security as a taboo subject that can't be discussed clearly, wanted the name changed despite the fact that the only reason for the change is for security reasons, which would make the old method "unsafe."

-Brad

Read the followup by pagexec

Posted Jun 11, 2011 17:42 UTC (Sat) by mingo (subscriber, #31122) [Link] (9 responses)

FYI, there were two naming discussions in the thread: first Linus disliked the proposed CONFIG_UNSAFE_VSYSCALLS name. Then pageexec disliked the CONFIG_COMPAT/LEGACY_VTIME proposed name and started insulting me.

Linus was actually right that the first name was misleading.

Read the followup by pagexec

Posted Jun 11, 2011 18:13 UTC (Sat) by spender (guest, #23067) [Link] (8 responses)

It's your opinion that he's right. Linus didn't want a random person to be "scared" by the name and turn it off, because then they might be subject to improved security and lower performance until they update their glibc. Of course, he didn't consider the other side of the coin, the same person who chooses not to read configuration help and enables/disables options based on their config name alone, doesn't disable this feature, even long after their glibc is updated. It's not black/white, right/wrong -- you preferred performance over security in this case. To say the original name was fine is perfectly acceptable.

BTW is it misleading to all those embedded users to have any mention of security when they don't have MMUs? They might get scared and turn features off! I urge you then to remove all mentions of security from your commit messages so they aren't mislead. Oh wait..

-Brad

Read the followup by pagexec

Posted Jun 11, 2011 18:34 UTC (Sat) by mingo (subscriber, #31122) [Link] (2 responses)

It's your opinion that he's right.

That the static vsyscall page is not a security vulnerability in itself is not just my opinion, it is a rather common-sense statement and it is our editor's opinion as well:

One useful point from that discussion is that the static vsyscall page is not, in fact, a security vulnerability; it's simply a resource which can make it easier for an attacker to exploit a vulnerability elsewhere in the system.

So could you guys please stop attacking people just because they disagree with you?

Read the followup by pagexec

Posted Jun 11, 2011 19:00 UTC (Sat) by spender (guest, #23067) [Link] (1 responses)

I agree it's not a security vulnerability because I understand the definition of vulnerability. We weren't discussing that, though, you just brought it up. What we *were* discussing was the "UNSAFE" name. "UNSAFE" is a perfectly accurate name for the option. If it's not "unsafe" then are you saying fixed-address vsyscall is safe? No, of course you wouldn't say that, that's the entire reason for the patch (read: security)!

So no, I'm not wrong because you brought in a strawman and put words in my mouth ;) BTW, I notice from on here and on LKML you often use appeals to authority as a means of making an argument. As the MIT example should show, it's generally a poor approach to use :)

Anyway, my point has been sufficiently made for the other readers here (as I see you won't ever concede your position). It's very telling really that possibly misleading a hypothetical user into lowering their performance in some cases is taken seriously, whereas misleading them into reducing their security (or rather, maintaining it at the same poor level), is not even discussed/acknowledged ;)

-Brad

Read the followup by pagexec

Posted Jun 11, 2011 20:18 UTC (Sat) by mingo (subscriber, #31122) [Link]

Well, you are language lawyering now.

Linus's and my opinion is that calling something 'unsafe' suggests that it's ... unsafe - while in reality the vsyscall itself is not unsafe: it needs a vulnerability to have any security role.

Read the followup by pagexec

Posted Jun 11, 2011 20:23 UTC (Sat) by Julie (guest, #66693) [Link] (4 responses)

he didn't consider the other side of the coin, the same person who chooses not to read configuration help and enables/disables options based on their config name alone, doesn't disable this feature...

If he didn't consider this maybe it was because he thought anyone config'ing their own kernel without checking the ---help--- is being at least a little silly.

I always build my own kernels and when I config one I feel it's only common sense to be expected to take some responsibility for the outcome myself, which means being able to work out (if I don't know) what does what, which implies checking the config help for new or unfamiliar features and not just making assumptions based on the name of the feature about what it is or does.
So I always check new config options, and if I didn't do this and I ended up disabling something I was relying on using or having some other unpleasant experience resulting from it well, that would be my own fault, wouldn't it. Because I shouldn't be messing with it in the first place. A really useful lesson in humility, at least...

I'm pretty sure distro and kernel maintainers and other users who have more at stake than me, or more of a measure of responsibility, check new configs too - isn't it just sensible?

CONFIG_UNSAFE_VSYSCALLS as a config name does seem rather histrionic though.

Read the followup by pagexec

Posted Jun 11, 2011 20:29 UTC (Sat) by mingo (subscriber, #31122) [Link] (2 responses)

LOL, histrionic indeed :-)

Note that the config option is gone in the patches i've applied days ago and which are queued up for mainline integration.

Read the followup by pagexec

Posted Jun 12, 2011 10:36 UTC (Sun) by patrick_g (subscriber, #44470) [Link] (1 responses)

>>> the config option is gone in the patches i've applied days ago and which are queued up for mainline integration

Mainline integration in 3.0 or for the next merge window (ie 3.1) ?

Read the followup by pagexec

Posted Jun 12, 2011 12:27 UTC (Sun) by mingo (subscriber, #31122) [Link]

Next merge window for now - the patches are too big and complex to be included in 3.0.

You can try them on 3.0 a well, via the -tip tree.

The patches are available standalone as well, see the tip/x86/vdso branch.

Read the followup by pagexec

Posted Jun 11, 2011 20:36 UTC (Sat) by spender (guest, #23067) [Link]

I was trying to illustrate that the same person who disables the option because they're "scared" by "UNSAFE" in the config name itself, apparently without reading the following config help:
+ Legacy user code expects to be able to issue three syscalls
+ by calling fixed addresses in kernel space. If you say N,
+ then the kernel traps and emulates these calls. If you say
+ Y, then there is actual executable code at a fixed address
+ to implement time() efficiently.
+
+ On a system with recent enough glibc (probably 2.14 or
+ newer) and no static binaries, you can say N without a
+ performance penalty to improve security
+
+ If unsure, say Y.

is the same kind of foolish user who would leave it on because it says "COMPAT" in the name and they don't want to break anything (even though disabling it doesn't break anything). So we're not in disagreement, I just didn't state it as clearly as I should have apparently.

-Brad

Read the followup by pagexec

Posted Jun 21, 2011 5:19 UTC (Tue) by gabucino (guest, #72504) [Link]

To Spender+PaXTeam: since nobody who's serious about IT would actually consider using this sechole+lulz collection called linux, please drop it altogether and kindly develop for us FreeBSD admins instead. Beverages are on me.

Quotes of the week

Posted Jun 9, 2011 13:20 UTC (Thu) by iq-0 (subscriber, #36655) [Link]

CVE is an external tracking system and as such should be handled external to the project.

In our in-house development we often attribute our fixes to tickets in our ticketing system. But often as not the mapping doesn't hold up here either. The problem is that changelogs and commit messages are for all practical purposes immutable. And any relation from tickets to (possible) changes is dynamic: New ones are encountered afterwards, olders ones are invalidated and sometimes the tickets themselves are duplicates that alread had other (partial) fixes for that problem.
So sure, a bugfix might be based on a bug that was reported elsewhere (be it your ticketing system or someone else's like CVE) but that is purely the "inspiration" for the fix. A real bugfix fixes the code itself and any real explanations about what it fixes are it's own (not some CVE number, not a bug report). Ofcourse any examples given in the original CVE/bugreport you used for "inspiration" can be used in a commit message as evidence why the old behaviour was wrong and the new behaviour is correct.

So the only times you should consider linking back from your bugfix to a ticket system would be to thank it for the insights it gave you to get to get you started on the bug ;-)

Ps.
This is about bugfixes, for features a backreference to the original tracking ticket *is* useful as that information is not dynamic or subject to change afterwards.

Thanks, Ingo

Posted Jun 15, 2011 18:09 UTC (Wed) by CChittleborough (subscriber, #60775) [Link] (1 responses)

Ingo's comments here were interesting and informative. They would make a good start on a great LWN article.

(And, yes, a bug is a bug. When your software is going wrong, the important thing is that it is going wrong, not whether the bug falls into any particular category. In other words, the only useful way to categorize known bugs is "not fixed yet" or "fixed".)

Thanks, Ingo

Posted Jun 15, 2011 20:21 UTC (Wed) by Julie (guest, #66693) [Link]

Ingo's comments here were interesting and informative. They would make a good start on a great LWN article.

I second this. :-)