Weekly Edition
Return to the Kernel page
| |||||||||||||
Better kernels with GCC plugins
Last week's article on GCC plugins drew a comment from "PaXTeam" regarding the PaX
project's use of plugins. It seems they have developed a set of plugins to
help with the task of creating a more secure kernel. Said plugins can be
hard to find; your editor located them in the grsecurity "test" patch
set. There appear to be four of them, each of which tweaks the
compilation process in a different way:
Use of plugins in this way allows significant changes to be made to the kernel without actually having to change the code:
To give you some numbers, an allmodconfig 2.6.39 i386 kernel loses
over 7000 static (i.e., not runtime allocated) writable function
pointers (a reduction of about 16%). creating an equivalent source
patch would be thousands of lines of code and have virtually no
chance to be accepted in any reasonable amount of time.
On the other hand, plugins of this type can increase the distance between the code one sees and what is actually run in the kernel; it is easy to imagine that leading to some real developer confusion at some point. Still, says PaXTeam, "the cost/benefit ratio of the plugin approach is excellent and there's a lot more in the pipeline." It is not too hard to imagine other uses that are not necessarily tied to security. (Amusingly, the plugins are licensed under GPLv2, meaning that they do not qualify for the GCC runtime library exemption. The kernel does not need that library, though, so all is well.) (Log in to post comments)
Better kernels with GCC plugins Posted Oct 6, 2011 8:41 UTC (Thu) by liljencrantz (subscriber, #28458) [Link]
I actually prefer the GPLv2 to GPLv3, and I'm not fan of the way various people connected to the FSF have tried to force GPLv3 down people's throat, but I still wish that people would release their GCC plugins under GPLv3. That way, those plugins can be brushed up and bundled with GCC, significantly extending their reach. From the sound of it, some of these plugins would definitely be useful outside of the kernel.
Better kernels with GCC plugins Posted Oct 6, 2011 10:54 UTC (Thu) by PaXTeam (subscriber, #24616) [Link]
as i've just posted in a comment below, the licensing of plugins for use with the kernel is somewhat complex (but IANAL so maybe i'm overreacting ;). i'm personally open to relicensing my code to GPLv3 if someone really wants to use them outside of the kernel (even if the only one i can think of that'd be useful in userland is already GPLv2+).
const structures Posted Oct 6, 2011 9:24 UTC (Thu) by epa (subscriber, #39769) [Link]
Is there any reason the upstream kernel could not add some more 'const' declarations to remove the need for the first plugin?
const structures Posted Oct 6, 2011 11:11 UTC (Thu) by PaXTeam (subscriber, #24616) [Link]
did you read the second link in the article? ;) solving the "let's make all constifiable variables const" problem is not possible with patching due to the sheer amount of patching needed. you can read the related lkml posts 1-2 years ago and see how well that process went.
const structures Posted Oct 6, 2011 14:07 UTC (Thu) by epa (subscriber, #39769) [Link] solving the "let's make all constifiable variables const" problem is not possible with patching due to the sheer amount of patching needed.I remember reading about this on LWN but I never quite believed it. A patch to add the 'const' keyword to some code is not hard to merge manually, even if the code has diverged a lot in the meantime. But then, the sheer size of the Linux codebase may make even a simple change unmanageable - I will defer to your experience. I guess the answer would be to generate the patch automatically against a given source tree. Then even if only a fraction of the total patch can be applied at a time, it is possible to keep trying. Clearly if it takes a week of manual work to create the patch it's never going to be practical to get it in.
const structures Posted Oct 6, 2011 19:41 UTC (Thu) by PaXTeam (subscriber, #24616) [Link]
yes, the problem with const patching is not its complexity but the amount of changes needed. i forget the exact numbers now, but a recent kernel has say 200 ops structure types (i.e., candidate for constification), and many more actual variables of said types.
the first problem you'll hit with manual patching is that it takes a lot of time to check every variable instance and their use to determine whether the given type can be made read-only or not (or you'll have to settle on individual variables only), the plugin approach is already a godsend for that reason (in the past another route was tried with coccinelle but that didn't work out too well).
the second problem is patching a given tree *and* forward porting the changes to a newer kernel, including redoing the analysis in the first step since new uses of a given type can come and go any time, meaning that the constifiable property of a type can change either way over time. the plugin approach is again the most efficient way of tracking these kind of changes.
last but not least, once you have such a plugin that can determine when to do constification, it's a very natural step to actually do it from the same plugin (it's like one extra line of code to set TREE_READONLY on the type) at which point one's enthusiasm to maintain a huge patch quickly evaporates ;).
now this was the producer side, but the consumer side is not any better unfortunately. if you read those lkml threads when constification patches were submitted, you'll realize that different developers expected them in different (and conflicting) ways (broken up by subsystem or maintainer or directory or structure type, etc), putting a huge burden on the producer side as if creating the patches wasn't consuming enough time already. then there's an issue with enforcing policy as well, seemingly noone takes constification or checkpatch.pl seriously enough as i keep seeing patches go in all the time that simply don't bother to constify structures.
all in all, while patching the source code is surely a noble goal, my and other people's lives are too short for it...
const structures Posted Oct 7, 2011 6:26 UTC (Fri) by Lionel_Debroux (subscriber, #30014) [Link]
The constification part of the PaX patch used to represent more than 700 KB, i.e. more than one third of the raw size of the patch, which was below 2 MB. For many files touched by the patch, constification was the only type of change involved in the file.
Some constifications performed by the PaX patch were possible only due to other changes in PaX - for example, ata_port_operation, which is one of the most widely used function pointer structs in the kernel.
I myself gave up trying to push upstream the constification of snd_pcm_ops, which represented ~10% of the size of the constification part of the PaX patch, because there were so many instances to check...
Could the plugin be modified to print warnings about pointers to non-const instances being passed to functions that take pointers to const instances ? (if enabled by some argument to the plugin, because upon introduction of that feature and for quite some time afterwards, there will be thousands of occurrences...)
Finding functions that take pointers to mutable instances while they could take pointers to const instances would be harder, wouldn't it be ?
const structures Posted Oct 7, 2011 8:22 UTC (Fri) by PaXTeam (subscriber, #24616) [Link]
> Could the plugin be modified to print warnings about pointers to non-const
> instances being passed to functions that take pointers to const instances ?
it's surely possible but i'm not sure this is what you really want as such typecasts are allowed by C and extensively used everywhere. now i assume you'd really want this detection for ops structures only in which case the problem is how the plugin would know of them (i assume you wouldn't want to use the current 'constify by default' approach). you could make use of the do_const attribute (without calling constify_type) then in the FINISH_TYPE callback you can check whether the pointed to structure is already const or not (and has the do_const attr, although on second thought, i forget now whether attrs are carried with types or have to be acted upon in the attr callback).
with all that said, i would not go the 'check the pointer' route but rather i'd constify the variable instances instead (but not function arguments) and then rely on gcc's existing warning system (if you take the address of a const object to initialize a ptr to the non-const type then you'll get a warning).
const structures Posted Oct 7, 2011 15:24 UTC (Fri) by vonbrand (subscriber, #4458) [Link] That there are lots of places that would require patching, and that the patch covering all of the different variables is large, isn't reason enough to just don't send patches fixing it piecewise upstream. Presumably there are reasons for not accepting said patches (probably much lessened today by using git).
Better kernels with GCC plugins Posted Oct 6, 2011 10:46 UTC (Thu) by PaXTeam (subscriber, #24616) [Link]
i wish i had had time to blog about all this stuff already (and i will, promise ;), so here's a few comments on the article that'll hopefully clarify some points in the meantime.
1. the canonical source code is in PaX itself so you can get the plugins from where you'd get PaX normally (grsec gets updated too eventually, but given how new this route is for all of us, spender's more conservative and doesn't immediately take everything into grsec). note that the plugins are not readily usable outside of the PaX kernel as they rely on both the build infrastructure i added to the kernel and some configuration as well.
2. as of last night, there're actually 5 plugins released (must be the late wednesday release syndrome ;) and a few more in the making.
3. the plugins in order of appearance:
3.1 the 'stackleak' plugin augments a PaX feature of the same name in that it provides an estimate about how deep the current syscall used the kernel stack (so that before returning to userland the used portion can be cleared). this feature (along with a few other changes) was written in response to a particular exploit technique published earlier this year.
3.2 the constify plugin make ops structures and those marked with the (newly introduced) do_const attribute read-only at compile time and consequently at runtime as well (non-static allocations will be flagged by the compiler and the source has to be patched to use a writable type marked with the (newly introduced, again) no_const attribute).
3.3 the kallocstat plugin (must be enabled with 'make CONFIG_KALLOCSTAT_PLUGIN=y') will emit statistics about the size argument of *alloc* functions (see the plugin for the whole list, it's not just kmalloc) if the given size is a compile time constant. the reason i wrote this plugin is because i was interested in the actual allocation size distribution vs. that of the kmalloc-* slab sizes. this in turn would enable one to adjust both the slab sizes and to fix some allocation sites to reduce internal fragmentation and waste. some excerpts from the histogram of a 3.0.4-i386-allyesconfig-nodebug kernel ('allocation size' 'call site count'):
16 503
as you can see, there seems to be room for improvement.
3.4 the kernexec plugin augments an old feature of the same name in PaX. it's not AMD (CPU) specific, rather it's amd64 (arch) specific. the short story is that on the i386 arch KERNEXEC does not only enforce non-executable pages for the kernel's side of the address space but also for userland (as in, the kernel won't be able to execute code from executable userland pages, SMEP in future CPUs will achieve the same). this was possible due to the use of segmentation, something not available in 64 bit mode so in this regard the amd64 version of KERNEXEC was always weaker, at least until i implemented this missing sub-feature as part of UDEREF/amd64. the problem with that approach is its non-negligible performance impact so for those not wanting the whole UDEREF experience, i wrote this plugin. it forces function pointers to point into the kernel's part of the address space, therefore effectively preventing executable userland pages from actually being executed from kernel code (and all this at a much lower performance impact than UDEREF of course). btw, the 'fairly sophisticated tweak' is a simple 'btsq $63,(%rsp)' before every 'retq' and something equivalent before every indirect call ;).
3.5 the checker plugin is the latest addition and as its name says, it may one day cover some/many things that sparse and checkpatch do. for now it's a PoC to demonstrate the use of the new address space support in gcc 4.6+ (as with the kallocstat plugin, it must be explicitly enabled with 'make CONFIG_CHECKER_PLUGIN=y'). note that i didn't actually patch the kernel to add all the missing annotations (and only __user is enabled for now, __iomem and the rest are put into the generic address space), so expect the compiler to error out frequently until someone fixes everything properly.
3.6 the soon-to-be-released intoverflow plugin will instrument all call sites where one or more argument is used as some kind of size (think *alloc and copy*user, but since the plugin adds a new function attribute, anything can be instrumented) and whose computation could have suffered from integer overflow/truncation - the runtime checks will detect such issues and prevent the incorrectly computed size from being used. you can find a beta (and for now, somewhat buggy ;) version of it at http://grsecurity.net/~ephox/overflow_plugin/ (note its license, since we are aware of how this one could actually be useful in userland as well, the license is not GPLv2 only but v2+).
3.7 the to-do list of future plugins just keeps growing, so without detailed explanations: generic ret2libc prevention (think of an actually sophisticated version of the kernexec plugin ;), free'd ptr sanitization to detect use-after-free problems and also infoleaks, forced structure gap initialization to eliminate infoleaks to userland, etc.
4. about licensing: i'm not sure if the situation is as amusing as Jake seems to think :). consider that the GPLv2 and GPLv3 are not compatible licenses and the kernel as a whole is GPLv2 so it cannot have GPLv3 parts in it (and even if the plugins are userland code, they can't really be argued to be mere aggregation for distribution purposes, they and their build system actively integrate with the kernel). that leaves us with licenses that are compatible with both, including GPLv2+ except the plugin versions distributed with the kernel would have to choose the GPLv2 part of the license and therefore changes made outside of the kernel (whose authors may choose the GPLv3) could not be reincorporated into the kernel's versions of the plugin (at least not without going through the explicit relicensing dance). the few other options left are some versions of the LGPL and BSD/MIT, none of which seems appealing to the kernel itself, and certainly not to me so i went with GPLv2 for now (note that not all the above mentioned plugins are mine, i'm just speaking of my code here). with all that said, if/when compiler plugins become part of upstream, the kernel devs will have to make some policy decisions regarding the acceptable licenses.
Better kernels with GCC plugins Posted Oct 6, 2011 11:20 UTC (Thu) by PaXTeam (subscriber, #24616) [Link]
correction, i didn't mean Jake but Jonathan ;).
Better kernels with GCC plugins Posted Oct 6, 2011 13:13 UTC (Thu) by SEJeff (subscriber, #51588) [Link]
This is excellent work. Do you think any of the distributions will use these plugins to build their kernels? Have you gotten any feedback from distribution kernel maintainers?
Better kernels with GCC plugins Posted Oct 6, 2011 17:34 UTC (Thu) by mjw (subscriber, #16740) [Link]
> the canonical source code is in PaX itself so you can get the plugins from where you'd get PaX normally
What is the canonical download/repository URL of PaX? I found http://pax.grsecurity.net/ but that seems to not have been updated for some years.
Better kernels with GCC plugins Posted Oct 6, 2011 18:49 UTC (Thu) by Lionel_Debroux (subscriber, #30014) [Link]
https://grsecurity.net/test.php contains PaX patches for a number of kernel versions, e.g., at the time of this writing:
https://grsecurity.net/test/pax-linux-3.0.4-test32.patch https://grsecurity.net/test/pax-linux-2.6.32.46-test124.p...
Better kernels with GCC plugins Posted Oct 6, 2011 19:06 UTC (Thu) by PaXTeam (subscriber, #24616) [Link]
that same page leads to the test directory hosting the latest patches: http://www.grsecurity.net/~paxguy1/ (the reason for not updating the main page is due to historical reasons, in the past i hoped to have the same kind of 'stable' release as we had with the 2.2 and 2.4 series, but as we now know, the whole development process changed so PaX is in permanent test mode ;).
Better kernels with GCC plugins Posted Oct 6, 2011 13:24 UTC (Thu) by eliezert (subscriber, #35757) [Link]
such a plugin would also be a great way to sneak in the exploit code you want to compile into the kernel.
Better kernels with GCC plugins Posted Oct 6, 2011 19:01 UTC (Thu) by ortalo (subscriber, #4654) [Link]
even more perverse: combine with the compiler "idea" proposed by Ken Thompson in his 1984 award speech ("Reflections on trusting trust")...
Better kernels with GCC plugins Posted Oct 13, 2011 10:24 UTC (Thu) by callegar (guest, #16148) [Link]
Basically PaX appears to be introducing a C dialect which is syntactically compatible with C, but characterized by some semantic differences.
This may have different implications depending on whether kernel compilation with PaX plugins gets widespread or not.
If it gets widespread, the kernel might end up depending on PaX semantics, leading to a kernel written in PaX_C rather than in C.
If it does not get widespread, I guess there are going to be issues in managing bug reports about kernels compiled with PaX.
Better kernels with GCC plugins Posted Oct 13, 2011 18:55 UTC (Thu) by vonbrand (subscriber, #4458) [Link] What I worry about is that PaX_C ends up being an opaque language, in which all sort of magic happens behind the scenes ("a struct full of function pointers gets to be const, unless..."). The central virtue of C (instead of, say, C++) is its transparency, which has been derided as "high-level assembly language": What you write is translated rather straightforwardly into what gets executed. Sure, C has its grave faults, extra tools for checking what you write are a must (thankfully the days of
Better kernels with GCC plugins Posted Oct 14, 2011 21:51 UTC (Fri) by PaXTeam (subscriber, #24616) [Link]
what if you stopped spreading FUD? there isn't any magic behind the scenes, the source code of the plugins is freely available. if you don't understand the code then it's your own problem, but that doesn't justify the FUD you're trying to spread.
as for C being a "high-level assembly language" and "is translated rather straightforwardly into what gets executed", it's never been true, one needs to look no further than all the undefined behaviours the current linux code triggers.
Better kernels with GCC plugins Posted Oct 16, 2011 4:03 UTC (Sun) by quotemstr (subscriber, #45331) [Link]
With an attitude like yours, I can understand PaX team has a hard time getting code into the kernel. The OP is in no way spreading FUD, and claiming that he is reflects more on you than it does him. The OP's point, which is that semantics differing markedly from ordinary C make code less readable, is perfectly valid. When code becomes opaque, it's not "your own problem": it's a problem for the entire community, increasing barriers to entry and ongoing maintenance costs for everyone. It would behoove you to actually address this point.
The compiler plugin source being available is no defense: INTERCAL sources are available too, but this fact doesn't make INTERCAL comprehensible.
Better kernels with GCC plugins Posted Oct 16, 2011 11:06 UTC (Sun) by PaXTeam (subscriber, #24616) [Link]
> With an attitude like yours, I can understand PaX team has a hard time getting code into the kernel.
given that i don't even try (lately that is, in previous years i got many bugfixes in without much hassle), i can't see how you managed to draw that conclusion ;).
> The OP is in no way spreading FUD, and claiming that he is reflects more on you than it does him.
and claiming that he doesn't reflects more on you than it does on me? can we skip the silly rhetoric please?
> The OP's point, which is that semantics differing markedly from ordinary
which is what i called FUD. the only semantical change we can talk about in this context is the forced constification of certain types and variables, the exact details of which i described above (did you read them?) and can also be learned from the source code of the constify plugin. this change in semantics, surprise surprise, does not make *any* change to the kernel source code, therefore it is as readable as it is without using the plugin.
> When code becomes opaque, it's not "your own problem":
you're misquoting me. i made that comment on his unwillingness to take a look at the plugin source code where he could have learned what it does. similarly how you take a look at the kernel source code if you want to learn what it does. IOW, lazyness doesn't an argument make.
> it's a problem for the entire community, increasing barriers to entry
why does a constification plugin increase the barriers to entry?
> and ongoing maintenance costs for everyone.
and what's the maintenance cost for everyone by using a constification plugin? i know its cost on me (having to check the code regularly for violations of the assumptions the plugin's based on) but a cost for *everyone*? what would that be?
also when evaluating a change one has to look at both sides of the coin, a.k.a. cost/benefit analysis. why did neither of you address the benefit side? don't see any? too little to be worth? any reasoning one can discuss?
> It would behoove you to actually address this point.
the ball's on your court now ;).
Better kernels with GCC plugins Posted Oct 16, 2011 19:14 UTC (Sun) by vonbrand (subscriber, #4458) [Link] The OP's point, which is that semantics differing markedly from ordinary C make code less readable, is perfectly valid.which is what i called FUD. the only semantical change we can talk about in this context is the forced constification of certain types and variables, the exact details of which i described above (did you read them?) and can also be learned from the source code of the constify plugin. this change in semantics, surprise surprise, does not make *any* change to the kernel source code, therefore it is as readable as it is without using the plugin. There is a semantics change, and you say yourself you have to check regularly if any of the assumptions of your behind-the-secenes changes get violated and fix the plugin accordingly. See, that is exactly the kinds of changes that the random kernel hacker won't be able to do by herself (and she will probably left scratching her head when perfectly sane looking C doesn't compile, or Oopses inexplicably, or changes plainly stated in the source just don't work). The cost for you is probably much larger than for everybody else, but that doesn't mean that the cost for others doesn't exist. As also stated, this extends the source of the kernel from GCC-C to PaX-C + GCC-plugins, and that means there have to be people familiar enough with that combination to keep it running (What has been called "the bus test": What happens to the kernel if you get run over by a bus?). A cost/benefit analysis would say that the benefit is slim to none (if there was a huge benefit, the changes would presumably have been done or accepted by the regular kernel hackers; please do spare me your conspiracy theories); costs include a specially spiked compiler (every time a risk) plus the growingly complex and opaque PaX-C language and GCC plugins as source, run-of-the-mill competent C programming skills aren't enough to pick up kernel hacking anymore. Sounds like a net loss today, and getting worse as time passes. And at least for constification there are perfectly sane solutions using vanilla C, so there is no real need for this circuitous route, so there isn't much of a justification either.
Better kernels with GCC plugins Posted Oct 17, 2011 21:45 UTC (Mon) by PaXTeam (subscriber, #24616) [Link]
> There is a semantics change, and you say yourself you have to check
> regularly if any of the assumptions of your behind-the-secenes changes > get violated and fix the plugin accordingly.
actually no, we don't have to fix the plugin, we have to 'fix' the kernel source rather. we made a conscious design decision to not bury non-constifiable type/variable names in the plugin code and implemented two attributes to give explicit control instead (with the default being 'constify unless marked' in PaX, but it's easy to flip it around to become 'constify if marked').
> See, that is exactly the kinds of changes that the random kernel hacker
you mean, upon seeing a clear compiler error about assigning to a read-only variable one won't be able to add a simple __no_const attribute to the affected type? this is the same effort when one has to remove a const qualifier for the same error message or to add annotations for sparse, etc. it's not a particularly hard to learn skill, maybe you want to give it a try just to see it for yourself instead of taking my word for it?
> and she will probably left scratching her head when perfectly sane
the compiler error message is very clear about what goes wrong ;), i never had a problem with figuring out what to 'fix', this gcc behaviour induced by the plugin was a conscious choice actually so that we can let the compiler's existing infrastructure figure out where the 'bad' code is (actually, the quotes are not exactly justified, it did find bad code).
> or Oopses inexplicably,
we're getting into FUD territory. how would the constify plugin cause an oops?
> or changes plainly stated in the source just don't work
what do you mean here?
> The cost for you is probably much larger than for everybody else [...]
yes, it's an insane amount of time. 15 mins of compiling an allyesconfig/allmodconfig kernel and say another hour to filter out and fix the newly introduced problems. every 3 months. i wish all other kernel changes cost this much to fix/work around ;). and if the plugin was part of the normal development process then this cost would be a minute per problem per developer (which is seemingly an order of magnitude more effort than people invest into running and acting on checkpatch). and if the default behaviour were flipped, it'd be back to the kernel janitor folks to add do_const every now and then to the deserving types.
> As also stated, this extends the source of the kernel from GCC-C to
if only there was such a thing as GCC-C ;). but there isn't, the kernel isn't written to any particular C standard, not even any GNU C specific extension or implementation (ever tried -O0?), it's some mix of these.
constifying ops structures is janitorial work, it's something that a programmer should have done ("if you don't mean this variable to be actually writable then don't leave it writable") but didn't, nor did anyone else to bother with the checkpatch output, etc.
> and that means there have to be people familiar enough with that
as with any new thing, one needs to learn the details of how gcc plugins work, not unlike how people had to learn the intricacies of SMP, etc along the way. i trust that more and more people will find useful things to do with gcc plugins and this lack of knowledge will disappear as it did with many other things.
Better kernels with GCC plugins Posted Oct 26, 2011 23:54 UTC (Wed) by nix (subscriber, #2304) [Link]
Well, a lot of people seem to think C is a high-level assembly language. If you ask them to run all their code at -O0 they get rather unhappy though, so I guess they'd rather it was a language for an abstract machine, amenable to optimization. :)
Better kernels with GCC plugins Posted Oct 14, 2011 21:43 UTC (Fri) by PaXTeam (subscriber, #24616) [Link]
you're assuming that there's such a thing as "C". there isn't. what does exist, or rather, do exist are several different flavours of C, some defined by standards, some (most of them) defined by actual implementations of a given compiler. you see, there's a reason that gcc 4.6 as of today is still a masked package in gentoo for example. if the concept of a single C specification existed and everything was written to that specification, this situation would not exist, but it does. IOW, having a new flavour defined by PaX plugins is nothing out of the ordinary, it's the norm in fact.
now obviously not all flavours of C are created equal, so one has to carefully evaluate what's worth using and what isn't. for the PaX flavours the defining goal is always something to do with "make the generated code more secure, even if the original source code wasn't written with security in mind". whether it's something you or anyone else values is of course not up to me to decide, but i did decide that it was worth for me (and many others) to pursue this route.
as for bugreports, we've always handled PaX/grsec bugreports ourselves and directed users upstream only when we could determine that the given problem existed there as well.
|
|||||||||||||