Not logged in
Log in now
Create an account
Subscribe to LWN
LWN.net Weekly Edition for May 16, 2013
A look at the PyPy 2.0 release
PostgreSQL 9.3 beta: Federated databases and more
LWN.net Weekly Edition for May 9, 2013
(Nearly) full tickless operation in 3.10
Posted Nov 4, 2010 19:07 UTC (Thu) by dlang (✭ supporter ✭, #313)
Doing this may not help security much without the change you mention below, but it would reduce the size of the GRsec patchset, which would then mean that those people would not have to maintain this sort of thing as a external patch.
Then with the read-only status clearly defined for things, the kernel developers may be open to a configurable option that would change how read-only memory works, if the performance impact could be clearly documented (and under some conditions, making things read-only may result in performance increases, if it means that the system can allow multiple copies of the data to exist in CPU caches without having to worry about keeping them in sync because they are known to never change)
the key thing is to not pitch this as 'this is a critical security fix, take it immediatly', but instead pitch it as 'This is a cleanup to make otherwise undocumented assumptions about how variables are used explicit, improving long-term maintainability It has the potential to have a minor performance improvement and security gain, but only if the read-only status gets enforced.'
this changes the reason for accepting the patches into something that they can see a direct benifit for.
Posted Nov 4, 2010 19:24 UTC (Thu) by mingo (subscriber, #31122)
Doing this may not help security much without the change you mention
below, but it would reduce the size of the GRsec patchset, which would
then mean that those people would not have to maintain this sort of
thing as a external patch.
Some authors may be fine with having a product which can always be claimed to be 'safer than upstream' - because it's a security fork. It also gives them a constant stream of exploits that hit mainline but are averted via a security patch.
So security enhancements generally have different marketing and economic constraints from other kernel enhancements, and merging things upstream can be undesirable/counterproductive to any 'security circus' aspect.
I have no way to tell whether that's the case in the grsecurity case though - but it's something to consider. If they are helping such efforts then all the better - but unlike other kernel forks it's not necessarily in their own best interest.
Same goes for the Coverity static checker project as well. That too is a project that would essentially die if we had a static checker in the upstream kernel, which would be tightly integrated with the kernel build (like Sparse) and which could thus avoid bugs in their infancy, right when they are introduced.
Security is a weird field.
Posted Nov 4, 2010 22:18 UTC (Thu) by spender (subscriber, #23067)
Our source is open for anyone to use. We have no "marketing and economic constraints." And sorry, but the only thing 'circus'-like is upstream's approach to security.
You were already linked by the parent poster to the initial split-out const patches I made up over a year ago. What happened after that is quite typical of upstream's handling of security; it's treated more as a nuisance, something to brush out of the way by means of token gifts of minor improvements. There's no overarching goal or dedication to completeness; the recent thread on /proc/kallsyms that you participated in is a perfect example of this cargo-cult security. So someone will merge a few small patches that are ultimately useless by themselves, and then the idea will be dropped again until someone like the parent poster points out the stupidity of not having merged all of them.
We'll also ignore the fact that all of the const patches were already split up and sent to LKML multiple times, but that only a tiny fraction of them were merged.
So don't regurgitate this inapplicable 'security circus' crap from Linus and act as if it's profound; it's not. There's a simple truth here: upstream had more than enough opportunity over several years to adopt the changes, the work having already been done multiple times by multiple people. Nobody's holding on to any code or trying to prevent it from being merged. If upstream wanted it merged all they had to do was accept the submitted patches.
Look and see for yourself what happened in the most recent attempt:
I think that entire effort only reduced our patch size by about 50kb (out of what is nearing 2MB).
As far as where our own interests lie, it's in reducing the amount of time required for the patches. The time spent maintaining the const patches on our end is next to nothing (just some trivial rejects from nearby changes). The time and frustration involved in getting them all merged upstream is ridiculous. Even when some do get merged, more get added in in the following release that need to be fixed because there's no enforced upstream policy on the const-ness of various structures. So what happens? Upstream doesn't bother fixing them. Emese, the author of the set submitted to LKML, now maintains them just for us as there's no red-tape and no hassle.
Your backhanded derision in the form of a disingenuous question, posed as a faux-naive outside observer, belies the fact that if upstream is looking for someone to blame as explanation for why the changes haven't been merged, it needs only to look in the mirror.
Posted Nov 5, 2010 9:00 UTC (Fri) by mingo (subscriber, #31122)
I'd apply your academic analysis to your own exec-shield work.
Actually, FYI, a fair portion of exec-shield is upstream so your comparison fails right there.
Secondly, the few bits that are not upstream are the ones that were rejected explicitly by the VM maintainers. I have submitted them multiple times and early on. There was even a fine LWN article about it.
Look and see for yourself what happened in the most recent attempt:
I did, and it showed me a lot of acks from maintainers and some bugreports even.
In fact a fair amount of constification patches went upstream after the time you submitted similar work:
756e64a0b106: net: constify some ppp/pptp structs
2e7a091833f0: kconfig: constify file name
52dfb8ac0ef4: ceph: constify dentry_operations
54e5bc020ce1: x86, olpc: Constify an olpc_ofw() arg
f6db25a87643: squashfs: constify xattr handlers
b7bb0a12913a: gfs: constify xattr_handler
537d81ca7c53: ocfs: constify xattr_handler
365f0cb9d2d5: jffs2: constify xattr_handler
46e58764f0c5: xfs: constify xattr_handler
94d09a98cdb1: reiserfs: constify xattr_handler
11e27528076e: ext4: constify xattr_handler
d1f21049f918: ext3: constify xattr handlers
749c72efa4bd: ext2: constify xattr_handler
f01cbd3f8148: btrfs: constify xattr_handler
5bac942db3d2: SH: constify multiple DMA related objects and references to them
52cf25d0ab7f: Driver core: Constify struct sysfs_ops in struct kobj_type
9cd43611ccfb: kobject: Constify struct kset_uevent_ops
42747d712de5: [WATCHDOG] watchdog_info constify
8838d2560a8c: Staging: rt2870sta: constify RTUSBMultiWrite(), RTUSBFirmwareWrite()
a4c1a148a0c4: OMAP: DSS2: OMAPFB: Constify some function parameters
af930d646251: Input: gamecon - constify some of the setup structures
e87a344d0eef: Input: wacom - constify product features data
739674fb7feb: netfilter: xtables: constify args in compat copying functions
3b9cfc0a99f8: x86, mtrr: Constify struct mtrr_ops
5833929cc2ad: net: constify MIB name tables
45d5d805988f: iwlwifi: Constify struct iwl_ops
7d4e9d0962cd: drbd: Constify struct file_operations
f65380c07b64: resource: constify arg to resource_size() and resource_type()
28dfef8febe4: const: constify remaining pipe_buf_operations
9905a43b2d56: backlight: Constify struct backlight_ops
7707e61c7099: ctype: constify read-only _ctype string
471452104b85: const: constify remaining dev_pm_ops
3428838d8e88: page-types: constify read only arrays
1cceefd3a28e: tty: const: constify remaining tty_operations
b62d99b5028b: PCMCIA: soc_common: constify soc_pcmcia_socket ops member
a9966b580a3e: ieee802154: constify struct net_device in some operations
So this looks like good stuff, and it's being done - you got the acks - so why didn't you push your bits upstream? Have you asked akpm to include it in -mm, or have you prepared a tree yourself and sent a pull request to Linus? It's been a year since you submitted those patches - what are you waiting for?
Really, are you interested in keeping grsecurity forked? In your long reply i have not seen a single offer from you to make a serious push to merge grsecurity upstream. Instead you complain about the upstream process:
What happened after that is quite typical of upstream's handling of security; it's treated more as a nuisance, something to brush out of the way by means of token gifts of minor improvements.
The thing is, i have the exact opposite experience - security work is handled seriously and bugs are handled swiftly. The upstream kernel is aflush with security work. Patches are being welcome - check the very link you provided!
Instead of working with us you are hurling such insults at Linus:
So don't regurgitate this inapplicable 'security circus' crap from Linus and act as if it's profound; it's not.
Posted Nov 5, 2010 9:37 UTC (Fri) by dlang (✭ supporter ✭, #313)
as someone who makes a living in the security field, I can assure you that there is plenty of security circus and the field has a huge number of parasitic poseurs in it. anyone who doesn't believe this, please explain current airport security in real cost/risk terms.
there are also a lot of us who are just trying to keep ahead of the bad guys and explain the problems to management, unfortunantly we have a hard task to do and it isn't nearly as appealing as the marketing security circus to management who doesn't understand the technology at all
Posted Nov 5, 2010 10:26 UTC (Fri) by mingo (subscriber, #31122)
there are also a lot of us who are just trying to keep ahead of the bad guys and explain the problems to management
Yeah. And i think you can consider Linus one of your best allies in that quest really. (He just refuses to play the circus - and IMHO he has rather good and consistent arguments for that.)
Posted Nov 5, 2010 11:24 UTC (Fri) by ortalo (subscriber, #4654)
More seriously, that "circus" issue is really an annoying one and very deeply rooted in the field, I second that. Furthermore, from my own experience (subjective of course), I have to admit that most people with some responsibility (the "bosses") tend to be much more attentive to legal or marketing reasons than to practical/technical ones, from the security-oriented point of view. I am sure even Linus (or myself) can also feel this natural bias. This seems to me the fundamental reason why such theatral activity can exist and persist. And while the circus plays and attract attention in the forefront, technical problems and vulnerabilities bury themselves deeply in the code where they can stay for years and users opportunistically take advantage of the low light to write their passwords on a sheet of paper they "hide" in their wallet.
I have yet to fully understand the reasons for the long lasting existence of all this useless drama. But to me, it seems that this security circus is indeed a reality that we will need to cope with for a long time (from an historical persepctive, it may be that it fades away behind actual action in war times, but that's not a good reason to drop general pacifism... ;-).
So that refusal from the top-level kernel maintainer to play in the security circus sounds as an overall good news to me; but then, how does he want to play the security game (to get practical and continuous improvements to the overall security level of the kernel)?
Note: I certainly have a lot of ideas to propose on this topic: I've just deleted a lot of lines in this comment (the most interesting ones were probably associated to applying Coccinelle to identify missing "consts").
However, the very only comment I want to make is: then, how does Linus want to improve security management of the kernel (assuming he thinks improvements are possible)?
Posted Nov 5, 2010 15:09 UTC (Fri) by dlang (✭ supporter ✭, #313)
and they think that they are doing a pretty good job, and don't see a reason to make significant changes.
Posted Nov 5, 2010 12:51 UTC (Fri) by spender (subscriber, #23067)
Posted Nov 5, 2010 14:10 UTC (Fri) by Lionel_Debroux (subscriber, #30014)
Posted Nov 5, 2010 15:20 UTC (Fri) by spender (subscriber, #23067)
The answer is: "nothing." So why is it brought up so much when it's completely irrelevant to any kind of discussion we're having here?
I can think of a million better questions, like "what is Linux doing to attract security talent for mainline work instead of pushing away potential contributors?" or "Linux is used on millions of systems; why isn't there a single person employed full-time to improve upstream kernel security?" or "How can we move away from the find bug/patch bug mindset/approach to security?"
But it says a lot about an individual and their view toward security when the most useful thing they can muster is some regurgitated crap about "security circus" as if they're saying something insightful or intelligent.
Posted Nov 5, 2010 12:24 UTC (Fri) by spender (subscriber, #23067)
Let me make this clear: upstream collectively doesn't even have the common sense to acknowledge the existence of security bugs. The 'security circus' is a nice red herring to divert the blame for why Linux security is so atrocious. Maybe seven years ago you could complain about the one or two companies that embargoed vulns to make a name for themselves, but even then it would have been a poor excuse (again that anyone but upstream itself is somehow responsible for upstream's security). I hate to tell you, but those companies are long gone, and there's now nobody in the industry embargoing bugs for fame or anything else you attribute to this 'security circus.' Only notable exception being ITL with the stack/heap gap issue, though you were already notified about that in a 2005 presentation and did nothing about it; so sorry, you lose the ability to complain. That neither Linus nor anyone else has noticed this just illustrates how out of touch with the real world you all are. All you have now is people *volunteering* to try to fix things, and upstream acting like a bunch of idiots. Do you want a medal or something because upstream accepts submitted patches for trivial vulnerabilities found by volunteers? That's hardly a measure of success.
Complain about the 'security circus' all you want, this mythical boogieman that somehow magically prevents you from making any security improvements of your own, but until upstream takes a serious approach to security, I'm afraid the joke is on you. Distros taking a month to release updated kernels for vulnerabilities that only require reusing some of my enlightenment code to exploit is ridiculous. Time and again you have vulnerabilities capable of weaponized public exploits on day one; how many more years will it take for upstream to realize this is a serious problem worthy of actual attention?
Until then, keep up your "de-pessimizing" strategy to security: https://patchwork.kernel.org/patch/276921/
BTW, if I wanted to check for the existence of parasitic security poseurs, I'd probably look for more posts from this guy: http://lkml.org/lkml/2010/11/4/335
Posted Nov 5, 2010 13:59 UTC (Fri) by Lionel_Debroux (subscriber, #30014)
* It's sad that several hundreds low-controversy one-/few-liners from the huge grsecurity patch, which can be imported to mainline more easily than lots of other patches, without demonstrable performance or size penalty, remains developed outside of mainline.
Dan Rosenberg gets dozens of small patches, mostly fixes for information leaks, committed in mainline and backported to stable. So mainline does pay at least _some_ attention to security, and does prefer smaller patches, too.
* Brad says that new non-const struct instances are added all the time to mainline. And this is despite checkpatch.pl, which is supposed to be applied by authors before submitting patches, checking more than 30 _ops types for constness. But again, mainline certainly isn't against one-liner-per-type improvements to checkpatch.pl ;-)
And, in case somebody forgets to use checkpatch.pl (we programmers are just humans !), is there a gateway that passes each commit that enters mainline through checkpatch.pl ? Or a gateway integrated with the linux-next infrastructure, so that patches other than staging get hopefully fixed in the individual trees before they enter mainline ?
* The RO/NX patch for modules, one of the most recent incarnations of which seems to be http://lwn.net/Articles/372256/ / http://thread.gmane.org/gmane.linux.kernel.lsm/10347 , seems to have vanished: in the discussion thread, Kees Cook posts that he cannot find the patch in any tree he looked at... but there's a tip-bot notice for that patch ?
* Multiple patchsets (e.g. -mmotm and -ck) are provided, between other formats, as broken-out series, for easier consumption and review. I've looked for a broken-out series corresponding to grsecurity, but I haven't been able to find it. I haven't found the scripts for regenerating e.g. the _ops constification either - it's not that it's really hard to re-create them, but it's stupid to duplicate such work. Do I fail at reading Web pages or using a search engine ?
Posted Nov 5, 2010 15:07 UTC (Fri) by spender (subscriber, #23067)
Indeed, the status quo is not good for users. But I don't see how that is the problem of anyone but upstream. I only feel a responsibility toward my own users. Upstream is a lost cause for security, as far as I'm concerned. It's a waste of my time to deal with them and their narrow-minded view of security. If in the future they grow up and deal with security properly, then we'll re-evaluate working with them. As it stands, they continue to have the mentality that security just involves fixing bugs (which fits with the 'a bug is a bug' tautological stupidity).
And as I said earlier, when you go looking for "successes," it's mostly just them accepting trivial one-line patches for security issues found en masse via grep. What happens is quite different when a patch doesn't accompany the vulnerability report. So I reject this entire business of "the grsecurity folks aren't doing much to reduce the gap between mainline and grsecurity" -- it's not our job or what we want to be spending our time on.
> * It's sad that several hundreds low-controversy one-/few-liners from the huge grsecurity patch, which can be imported to mainline more easily than lots of other patches, without demonstrable performance or size penalty, remains developed outside of mainline.
> Dan Rosenberg gets dozens of small patches, mostly fixes for information leaks, committed in mainline and backported to stable. So mainline does pay at least _some_ attention to security, and does prefer smaller patches, too.
They prefer smaller patches, except when they don't. Look at the const patchset. First it wasn't split up enough, they complained, then somehow even though it was split up exactly as they asked, they complained that it was now split up too much. Sorry, but I view it as a failure of Linux if security improvements are dependent upon benevolent individuals with unlimited amounts of time to waste and an iron will.
Posted Nov 5, 2010 15:16 UTC (Fri) by dlang (✭ supporter ✭, #313)
if you don't feel and responsibility for anyone but your users and have no intent to try and push the fixes upstream,then your tree just became pretty irrelavent to linux security for anyone except for your users.
some people may choose to try and parse things apart to get them upstream, but very few people will want to try, so the security fixes that get into the kernel will be developed independantly (even though you will make a lot of noise each time someone recreates your work)
if you don't care about normal linux users, only your users, please quiet down about upstreams's policies, you've just said you don't care about anyone that uses it, so why should you care about it's policies?
Posted Nov 5, 2010 15:35 UTC (Fri) by spender (subscriber, #23067)
Posted Nov 5, 2010 15:55 UTC (Fri) by PaXTeam (subscriber, #24616)
so first it's our fault that our code isn't accepted in vanilla for whatever reasons then it's also our fault that we still dare to make it available for anyone who cares. and to crown the absurdity, we're at fault for caring about these people. man, are you sure you have anything to do with security? i'm seriously worried about *your* users.
> [...]and have no intent to try and push the fixes upstream
maybe you're not a native speaker, but if you re-read spender's post carefully, you'll note that he made the cooperation conditional. it's the sentence that starts with the 'if' word, i'm sure you'll be able to find all one instance of it.
> [...]then your tree just became pretty irrelavent to linux security for anyone except for your users.
wait, are you saying grsec *was* relevant up to this very moment in time for anyone but grsec users? wow, i take it you won't elaborate but i smell magic here.
> even though you will make a lot of noise each time someone recreates your work
only if they 'forget' to credit the people for the ideas and/or the code or if they fuck up the implementation. just ask ingo ;).
Posted Nov 5, 2010 16:25 UTC (Fri) by PaXTeam (subscriber, #24616)
it's more work than you think, just talk to Emese about it. afaik, she still has the cocci scripts that she used for the initial patch series then she had to manually fix up things here and there (i.e., there's no substitute for a make allmodconfig ;). the current const patches are about 230k but they only represent a small fraction of 200 or so ops structures that could probably be all constified. good luck getting that work done!
Posted Nov 6, 2010 9:52 UTC (Sat) by Lionel_Debroux (subscriber, #30014)
I haven't seen any reply about the broken-out form, if any, of the large grsecurity patch ?
Posted Nov 6, 2010 11:19 UTC (Sat) by PaXTeam (subscriber, #24616)
multiple people worked on this over time with different methods and eventually there was some evolution in that coccinelle proved smarter than grep/sed ;). note that it's still not all roses as according to Emese coccinelle has (or used to have, i'm not following this nowadays) some limitation in how it parsed include files (no recursion was the big problem, iirc) so when you do not only want to generate patches for specific structures but also want coccinelle to determine which ones could be constified at all automatically, then you'll have some extra work to do (or find/write another tool). and you'll want this level of automated help as checking every usage of those 200 ops structure types by hand is anything but fun (and no, compiling allyesconfig/allmodfconfig doesn't necessarily give you 100% coverage).
> I haven't seen any reply about the broken-out form, if any, of the large grsecurity patch ?
because there's no such thing, at most the const patches exist standalone but not the rest.
Posted Nov 5, 2010 17:16 UTC (Fri) by kees (subscriber, #27264)
On the reactive security front, upstream does an okay job; small obvious fixes are taken quickly, though sometimes larger fixes take some time to stew, but are ultimately taken. I'll skip talking about how fast reactive security handling by upstream has almost nothing to do with actually protecting end-users from the window of vulnerability, though.
On the proactive front, things are not as good. There has not been much distinction made between protecting userspace and protecting the kernel itself. Nearly all the proactive security work has been to protect userspace from itself, rather than protecting the kernel from userspace.
Small simple changes are done (limiting VFS, and now network, data lengths to below MAX_INT, for example), but larger architectural changes are frequently rejected or forgotten. (For example, why is module NX/RO http://git.kernel.org/?p=linux/kernel/git/x86/linux-2.6-t... not in Linus's tree after 9 months? What happened to "mode 2 SECCOMP" http://lkml.org/lkml/2009/5/7/479 ?)
It seems that there is a pervasive conservatism when it comes to proactive security improvements (both userspace-defensive and kernel-defensive), and only core maintainers have the ability to change that.
Posted Nov 7, 2010 21:27 UTC (Sun) by foom (subscriber, #14868)
Posted Nov 8, 2010 6:32 UTC (Mon) by Lionel_Debroux (subscriber, #30014)
I posted three small constification patches to kernel-janitors and Cc LKML yesterday - but for users' protection, the authors of the patches getting in are much less important than the fact the patches _really_ get in :)
As noted by Brad above, and in my mails, non-const instances of constifiable structs have been added post-2.6.35, although these structs were already checked for by checkpatch.pl... not good.
Getting the constification patches in will reduce the number of files touched by the grsecurity patch: it's not unusual for the constification to be the only set of changes affecting a given file.
Posted Nov 4, 2010 20:15 UTC (Thu) by Lionel_Debroux (subscriber, #30014)
For proper credit attribution, it requires coordination with the grsecurity people. Which I probably won't have time to do until next week, so feel free to beat me :)
Copyright © 2013, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds