Better kernels with GCC plugins

1. the canonical source code is in PaX itself so you can get the plugins from where you'd get PaX normally (grsec gets updated too eventually, but given how new this route is for all of us, spender's more conservative and doesn't immediately take everything into grsec). note that the plugins are not readily usable outside of the PaX kernel as they rely on both the build infrastructure i added to the kernel and some configuration as well.

2. as of last night, there're actually 5 plugins released (must be the late wednesday release syndrome ;) and a few more in the making.

3. the plugins in order of appearance:

3.1 the 'stackleak' plugin augments a PaX feature of the same name in that it provides an estimate about how deep the current syscall used the kernel stack (so that before returning to userland the used portion can be cleared). this feature (along with a few other changes) was written in response to a particular exploit technique published earlier this year.

3.2 the constify plugin make ops structures and those marked with the (newly introduced) do_const attribute read-only at compile time and consequently at runtime as well (non-static allocations will be flagged by the compiler and the source has to be patched to use a writable type marked with the (newly introduced, again) no_const attribute).

3.3 the kallocstat plugin (must be enabled with 'make CONFIG_KALLOCSTAT_PLUGIN=y') will emit statistics about the size argument of *alloc* functions (see the plugin for the whole list, it's not just kmalloc) if the given size is a compile time constant. the reason i wrote this plugin is because i was interested in the actual allocation size distribution vs. that of the kmalloc-* slab sizes. this in turn would enable one to adjust both the slab sizes and to fix some allocation sites to reduce internal fragmentation and waste. some excerpts from the histogram of a 3.0.4-i386-allyesconfig-nodebug kernel ('allocation size' 'call site count'):

16 503
17 2
18 10
32 351
33 4
34 1
36 293
64 255
65 3
66 12
68 123
128 123
129 4
130 4
132 62
133 1
256 211
257 1
258 2
259 3
260 9
512 200
513 4
514 2
516 16
517 4
1024 157
1025 13
1026 8
1028 13
1032 13
2048 122
2049 2
2052 6
2056 2
4096 295
4098 2
4100 3
4104 1
8192 44
8195 2
8196 2
8200 1
16384 32
16392 1

as you can see, there seems to be room for improvement.

3.4 the kernexec plugin augments an old feature of the same name in PaX. it's not AMD (CPU) specific, rather it's amd64 (arch) specific. the short story is that on the i386 arch KERNEXEC does not only enforce non-executable pages for the kernel's side of the address space but also for userland (as in, the kernel won't be able to execute code from executable userland pages, SMEP in future CPUs will achieve the same). this was possible due to the use of segmentation, something not available in 64 bit mode so in this regard the amd64 version of KERNEXEC was always weaker, at least until i implemented this missing sub-feature as part of UDEREF/amd64. the problem with that approach is its non-negligible performance impact so for those not wanting the whole UDEREF experience, i wrote this plugin. it forces function pointers to point into the kernel's part of the address space, therefore effectively preventing executable userland pages from actually being executed from kernel code (and all this at a much lower performance impact than UDEREF of course). btw, the 'fairly sophisticated tweak' is a simple 'btsq $63,(%rsp)' before every 'retq' and something equivalent before every indirect call ;).

3.5 the checker plugin is the latest addition and as its name says, it may one day cover some/many things that sparse and checkpatch do. for now it's a PoC to demonstrate the use of the new address space support in gcc 4.6+ (as with the kallocstat plugin, it must be explicitly enabled with 'make CONFIG_CHECKER_PLUGIN=y'). note that i didn't actually patch the kernel to add all the missing annotations (and only __user is enabled for now, __iomem and the rest are put into the generic address space), so expect the compiler to error out frequently until someone fixes everything properly.

3.6 the soon-to-be-released intoverflow plugin will instrument all call sites where one or more argument is used as some kind of size (think *alloc and copy*user, but since the plugin adds a new function attribute, anything can be instrumented) and whose computation could have suffered from integer overflow/truncation - the runtime checks will detect such issues and prevent the incorrectly computed size from being used. you can find a beta (and for now, somewhat buggy ;) version of it at http://grsecurity.net/~ephox/overflow_plugin/ (note its license, since we are aware of how this one could actually be useful in userland as well, the license is not GPLv2 only but v2+).

3.7 the to-do list of future plugins just keeps growing, so without detailed explanations: generic ret2libc prevention (think of an actually sophisticated version of the kernexec plugin ;), free'd ptr sanitization to detect use-after-free problems and also infoleaks, forced structure gap initialization to eliminate infoleaks to userland, etc.

4. about licensing: i'm not sure if the situation is as amusing as Jake seems to think :). consider that the GPLv2 and GPLv3 are not compatible licenses and the kernel as a whole is GPLv2 so it cannot have GPLv3 parts in it (and even if the plugins are userland code, they can't really be argued to be mere aggregation for distribution purposes, they and their build system actively integrate with the kernel). that leaves us with licenses that are compatible with both, including GPLv2+ except the plugin versions distributed with the kernel would have to choose the GPLv2 part of the license and therefore changes made outside of the kernel (whose authors may choose the GPLv3) could not be reincorporated into the kernel's versions of the plugin (at least not without going through the explicit relicensing dance). the few other options left are some versions of the LGPL and BSD/MIT, none of which seems appealing to the kernel itself, and certainly not to me so i went with GPLv2 for now (note that not all the above mentioned plugins are mine, i'm just speaking of my code here). with all that said, if/when compiler plugins become part of upstream, the kernel devs will have to make some policy decisions regarding the acceptable licenses.