LWN: Comments on "An unexpected perf feature"

An unexpected perf feature

schabi — Wed, 29 May 2013 05:41:53 +0000

I agree.

Interpreting unsigned 64 bit IDs as signed 32 bit is exactly what I'd expect to generate at least a compiler warning. Even in C, this should have been caught by good compilers or static analysis tools.

So, what went wrong?

An unexpected perf feature

Cyberax — Tue, 28 May 2013 09:06:46 +0000

You can certainly do it. You won't be able to do free() unless you can statically prove its correctness, but doing malloc() and using GC to collect unused objects is possible.

You also certainly can allocate buffers with special properties, and you can even do stuff like reading from within buffers by using fat pointers (i.e. a pointer with a length).

In fact, it's all been done in the past even for OS kernels. It's the question of practicality, not possibility.

An unexpected perf feature

etienne — Tue, 28 May 2013 08:59:28 +0000

> And if your language simply doesn't have a way to express pointer arithmetic then you can't use it to do memory damage.

IHMO you can't use that language to manage the memory neither, i.e. you can't write malloc()/free() nor what the kernel needs, for instance "allocate a contiguous DMA able buffer accessible with DMA32 PCI card and fail if a wait is needed and give back it's physical address" or "free memory blocks after DMA hardware has finished sending them", or any variation of it.
Obviously if you had perfect hardware and a processor with an "allocate_memory" and a "free_memory_after_both_software_and_DMA_have_finished_with_it" assembly instruction, then maybe...

An unexpected perf feature

Cyberax — Mon, 27 May 2013 23:51:02 +0000

> Anywhere it was not intended to be, but mostly strings, specially strings that are interpreted by the program.
That won't cause memory damage.

> It's not the language but the virtual _machine_ they run on what isolates misbehaving applications.
Nope. If your language allows unrestricted pointer arithmetic then it doesn't matter at all if you are running it inside the most secure VM.

And if your language simply doesn't have a way to express pointer arithmetic then you can't use it to do memory damage.

An unexpected perf feature

nix — Mon, 27 May 2013 22:06:13 +0000

As a guy who used to do stuff with Arcs, '30 years later with the advent of ARM' is bizarre. ARMs of a sort were around in the early 90s :) but, sure, they hadn't set the world on fire yet.

An unexpected perf feature

dgm — Mon, 27 May 2013 17:13:57 +0000

> Write a wrong value _where_?

Anywhere it was not intended to be, but mostly strings, specially strings that are interpreted by the program.

> And yes, it's totally possible to create a language (hint: Java, C#) that do NOT require hardware memory protection at all to isolate misbehaving applications.

It's not the language but the virtual _machine_ they run on what isolates misbehaving applications.

An unexpected perf feature

spender — Sun, 26 May 2013 13:03:08 +0000

I've uploaded the definitive exploit for the vulnerability here:
http://grsecurity.net/~spender/exploits/enlightenment.tgz

It should work on any distro, x86 or x64, with any combination (or lack of) CONFIG_MODULES and CONFIG_JUMP_LABEL. I've personally tested it on RHEL, Ubuntu, Debian, and Gentoo, custom kernels and distro kernels: 2.6.32 (RHEL), 2.6.38, 3.0, 3.2, 3.5, 3.8. It requires no System.map or /proc/kallsyms on x64 (even though a System.map could be trivially obtained, or the symbols extracted from the visible kernel image in /boot instead). Once it gains control in the kernel it resolves symbols internally. Its generic ring0 payload (reusable with any other kernel exploit where the attacker controls eip/rip) disables SELinux, AppArmor, IMA -- all LSMs. It breaks out of any chroot or mnt namespace. It breaks out of vserver and OpenVZ. It creates no logs and leaves the system in a consistent state.

The initial port was completed last week:
http://www.youtube.com/watch?v=WI0FXZUsLuI
http://www.youtube.com/watch?v=llqxbMgIztk

I delayed publication a week to give people more time to update, but this exploit should be considered a demonstration of the true risk of depending on patching individual bugs as a means to security or in using shared-kernel virtualization without any kind of kernel self-protection. The techniques in the exploit, some of which have never been published before, are the kinds of techniques that are used and sold in private.

-Brad

An unexpected perf feature

mpr22 — Sat, 25 May 2013 10:36:10 +0000

192k of data (point DS, ES, SS to different segments).

An unexpected perf feature

ras — Sat, 25 May 2013 00:21:13 +0000

> P.S. I disagree with you about the Itanium. It wasn't a good design. They took far more silicon than other processors and ended up being less productive with it.

I wasn't commenting whether it was a good design. I don't know, as I haven't used it. I was just saying Itanium marked the point when Intel went back to sticking to the knitting - in other words trying to design a new architecture whose sole goal was to run programs fast. If you say they failed despite having that explicit goal then that's a shame.

As I recall the Itanium tried to improve it's speed in a RISC like way - ie by keeping things simple on the hardware side and offloading decisions to the compiler. In the Itanium's case those decisions were about parallelism.

I think it's pretty clear now tuning the machine language to whatever hardware is available at the time is a mistake when you are going for speed. It might work when the hardware is first designed, but then the transistor budget doubles and all those neat optimisations don't much so much sense anymore, but you are welded to them because there are hardwired into the instruction set. Instead the route we have gone down is to implement a virtual CPU, rather like the JVM. The real CPU then compiles the instruction set on the fly into something that can be run fast with todays transistor budget. With tomorrows transistor budget it might be complied into something different.

The x86 instruction set is actually good pretty in this scenario - better than ARM. It's compact, and each instruction gives lots of opportunities to execute bits of it in parallel. If this is true, then Intel ending up with an architecture that can run fast is just dumb luck, as they weren't planning for it 30 years ago. Now that I think about it, the VAX instruction set would probably be even better again.

An unexpected perf feature

dlang — Fri, 24 May 2013 23:50:41 +0000

> 8086 actually. It was their way of extending the address space of the 8080 to 20 bits.

That's right, I forgot about that. And IIRC, with the 286 they only expanded it to 24 bits

but in any case, my point was that the dawn of the AMD64 chip is eons past the point where segmentation was introduced, failed, and was rejected

P.S. I disagree with you about the Itanium. It wasn't a good design. They took far more silicon than other processors and ended up being less productive with it.

In part, the speed increases were the cause of this failure. As the speed differences between memory and the CPU core get larger, having a compact instruction set where each instruction can do more, becomes more important, and for all it's warts, the x86 instructions do rate pretty well on the size/capability graph.

but in part the programmers really should NOT have to change their programs to work well on a new CPU where the hardware designers have implemented things differently. By hiding such changes in the microcode, instructions that get used more can be further optimized, and ones that aren't can be emulated so they take less silicon. It's a useful layer of abstraction.

An unexpected perf feature

ras — Fri, 24 May 2013 23:31:43 +0000

> segmentation on the x86 came about with the 80286 (or possibly even the 80186, I'm not sure) CPU.

8086 actually. It was their way of extending the address space of the 8080 to 20 bits.

It was just a slightly more sophisticated take on the way CPU designers have gone about extending the range of addressable memory beyond the natural size of internal registers for eons. It had the advantage you could use separate address spaces for both code and data, giving you 64K for each. So by using a small amount of extra silicon they effectively doubled the amount of address space you had over simple "page extension" hack everybody else was doing. Kudos to them.

So you are saying it was the 80286 is where the rot set it. In the 80286 they had enough silicon to give the programmer true isolation between processes. They could of gone the 32 bits + page table route everybody else did, but no we got segmentation (without page tables instead) and retained 16 bit registers. Why? Well this was also the time the hardware designers had enough silicon to implement microcode - so they could become programmers to! And they decided they could do task switching, ACL's and god know what else better than programmers, so they did. In other words somehow they managed to forget their job was to provide hardware that ran programs quickly, and instead thought their job was to do operating system design.

It was a right royal mess. Fortunately for them the transition from DOS to Windows / OS2 (which is where the extra protections and address space matters) took a while, and by then the i386 was released. It added 32 bits and paging, so we could ignore all that 16 bit segmentation rubbish and get on with life. It turned out the transition to multitasking operating systems wasn't waiting on programmers figuring out had to do it (who would have thunk it?), but rather the price of the RAM needed to hold several tasks at once had to come down.

People here defending the segmentation model should try writing for the 80286, which could only address 64K of code and 64K of data at any one time. There was no reason for it. The 68000 family had a much better memory model at the time, so it wasn't silicon constrains. Well there would have been enough silicon if they hadn't devoted so much of it to creating their own operating system on a chip.

Intel finally came to their senses with Itanium. With it they used all that extra silicon to do what hardware designers should be doing - make programs run fast. Sadly it came along too late.

Back to your point - the time line. The 80286 was released in 1982. The iAPX432 was meant to be released in 1981. The 80286 was the fall back position. As Wikipedia points out, this is a part of its history Intel strives to forget. You will find no mention of the iAPX432 on their web site, for instance.

An unexpected perf feature

dlang — Fri, 24 May 2013 13:40:45 +0000

you aren't going nearly far enough back

segmentation on the x86 came about with the 80286 (or possibly even the 80186, I'm not sure) CPU.

It was intended as a way to allow programs to continue to use 16 bit 8086 addressing but be able to use more than 64K of ram in a system (by setting a segment offset and then running the 16 bit code inside that segment)

It never was an effective security measure, because to avoid wasting tons of memory on programs that didn't need it, the segments overlap, allowing one program to access memory of another program.

When the 80386 came out and supported paging and real memory protection, everyone stopped using segments real fast

An unexpected perf feature

helge.bahmann — Fri, 24 May 2013 12:43:55 +0000

Sparc has instructions "load from foreign address space" and "store to foreign address space" (and even "compare and swap in foreign address space").

An unexpected perf feature

ras — Fri, 24 May 2013 12:00:39 +0000

As a "bearded guy" (I have no beard) who has written designed and written bios's and protected mode operating systems x86, and who had to look at the x86 architecture in detail again in mind numbing detail (as in reading the 4 Intel x86 "data sheets" several times in order to port linux-abi system to AMD64), I recall my thoughts at the time as being "my - AMD has cleaned this mess up".

For those of you defending Intel's decisions at this time - I lived through it. At the time Intel regarded all programmers as idiots, and decided to solve the problem with hardware. Thus we have the absurdly complex designs we see today, with x86 interrupts taking 2000 cycles (?!?!? - that was back when Intel was game enough to publish cycles) and it wasn't the slowest instruction. Can anyone remember a Task Gate Descriptor?

Yet that wasn't the worst of it. The worst of the worst died. It was a new Intel architecture called iAPX432. It caused more excitement than Haswell in it's day. I am sure it's forebears would prefer we forgot it entirely. It remains in my mind the ultimate testimony to the arrogance caused by ignorance, in this case the Electrical Engineers thinking they could tell Software Engineers how we should do our jobs. But I exaggerate. Back then we weren't allowed to call ourselves Engineers.

Still they got their revenge with x86. In it they made it plain we could not be trusted to swap between two tasks quickly. Only 30 years later with the advent of ARM has their folly been made plain to everyone.

An unexpected perf feature

helge.bahmann — Fri, 24 May 2013 09:16:39 +0000

Just as an addendum... Don't get me wrong, you are right to complain that a mechanism that was usable for security purposes was removed while nobody bothered to add a suitable substitute, and that all of the pretty architectural ideas that had already been present and demonstrated to be workable for 25+ years now had been ignored, but the "proper" way going forward is not to revive segmentation but implement comparable semantics with mechanisms that are performance-neutral. SMEP and SMAP are actually quite "easy" from a hardware conceputal point of view, so it is kind of annoying that it took so long.

An unexpected perf feature

helge.bahmann — Fri, 24 May 2013 08:53:31 +0000

1. Sure it's not a single 80s style alu anymore, rather it is a set of independently operating arithmetic units; typically two are adders, and whatever is needed is scheduled to them (addr gen or general arithmetic alike). Doing anything else is wasteful (which is to say that it is done nevertheless occasionally *if* it can speed up a fast-path, which for address calculations it cannot).

2. TLS is not a fast path, profiling shows around 1 in 1000 to 10000 instructions is TLS, so no one bothers paying a 1 cycle penalty for that.

3. ASIDs are cheaper because they just become part of the tag in the TLB. You do the TLB lookup (which you do anyways) and compare the tag for equality which is cheaper than a "greater than" comparison against an address space limit (which, incidentally, is just another adder), end of story.

4. Enforcing read-only is easily done at the page level, so what's the point?

5. What segmentation has been designed for? To map the concept of program object segments (the name may be a hint, right?) directly to hardware, facilitate sharing and relocatability this way. This is also where the security model for them originates from. It was conceived when people were somehow not yet certain paging was scalable, but I am too young to have been involved back then, you would have to ask the 'bearded guys'.

And to answer your last question: Paging is cheaper because hashed lookup (TLB) and equality comparison are cheaper than "less than" comparisons in hardware. Segmentation is a conceptual dead-end, live with it :)

An unexpected perf feature

dlang — Fri, 24 May 2013 07:48:47 +0000

if they were so fully separated, how could you pass data between them?

Even if true, it just shows that price and performance trump low probability security benefits once again, so what's new?

An unexpected perf feature

ballombe — Fri, 24 May 2013 07:35:04 +0000

From a security point of view amd64 _is_ bad, especially compared to older architecture like sparc which provide fully separated kernel and user address space.

An unexpected perf feature

PaXTeam — Thu, 23 May 2013 23:33:32 +0000

1. the ALU concept is so 80's ;). seriously, 'modern' CPUs are a tad bit more complex, Intel and AMD CPUs have all dedicated adders for address calculations (not just for the already mentioned purposes but also for rip-relative addressing). TLS not being a fast path is probably news to everyone spending their time on multithreaded applications, and Intel, in the grand conspiracy of schemes, must have added dedicated fs/gs base manipulating insns to their latest CPUs in order to slow these workloads down even more. as for the BTB, isn't it indexed by virtual and not logical addresses?

2. ASIDs cannot by definition be cheaper, paging related caches and checks will always require more circuits and cycles than a simple comparator. not sure what address space layout decisions have to do with this though, when you have ASIDs by definition you have full address spaces for each ASID. if you meant mixing different ASIDs in the same virtual address space (how?), then nobody does that.

conceptually ASIDs are indeed more generic except this fact is utterly irrelevant, there isn't a mainstream OS out there that would make use of this ability (i.e., mix user and kernel pages in arbitrary ways in the same address space). in practice everyone simply divides the virtual address space into two parts between userland and the kernel, so simple limit checking would do just fine (vs. checking access rights at every level of the paging hierarchy).

3. ASIDs do have their uses indeed, in fact i would love to have a better mechanism on Intel/AMD CPUs to implement some of my ideas but for simple user/kernel separation a segment limit check has no match.

4. to understand the difference in the security level provided by a segmentation and paging based non-exec/no-access kernel protection scheme we have to consider the attacker's assumed ability. against an arbitrary read and write capability they're equivalent. however this is the ideal attacker model only we use to evaluate theoretical boundaries of protection schemes, in practice we rarely get such bugs and that's exactly where the difference becomes important. in particular, the segmentation based approach can achieve a certain level of self-protection by simply ensuring that the top-level page tables enforce the read-only property on the GDTs whereas doing the same for page tables themselves is much harder - this is the attack surface difference. that said, KERNEXEC (on i386/amd64 so far) does attempt to minimize the exposure of top-level page tables but it's far from being a closed system yet (breaking up kernel large page mappings, tracking virtual aliases, etc have non-negligible performance impact).

5. what has segmentation been designed for then? surely there's only so much you can do with a data or code segment ;). why is it difficult to make it efficient in hardware? and which particular bit (there're many descriptor types)? why would paging related data structures be easier to handle in hw than segmentation ones? and how do you imagine beating a simple comparator? so far you haven't offered any facts to make me think otherwise.

An unexpected perf feature

Cyberax — Thu, 23 May 2013 12:46:04 +0000

Write a wrong value _where_?

And yes, it's totally possible to create a language (hint: Java, C#) that do NOT require hardware memory protection at all to isolate misbehaving applications.

An unexpected perf feature

renox — Thu, 23 May 2013 12:32:11 +0000

Uh? Your definition of memory protection is different from the normal definition which makes your post quite useless..

Bug class

error27 — Thu, 23 May 2013 10:41:49 +0000

Smatch theoretically can find buffer underflows.

In this case it missed for several reasons. 1) This wasn't getting marked as untrusted user data. 2) The cross function tracking wasn't working. 3) The underflow check was ignoring if we capped the upper value. *eyeroll*

I haven't pushed all the Smatch changes yet, but it's sort of working now to the point where it would have found this bug. It did find a couple problems in ATM network drivers as well.

An unexpected perf feature

dgm — Thu, 23 May 2013 09:17:24 +0000

There's no way to prevent memory corruption, as long as I can write to the wrong variable (ooops!). Memory protection is a feature of the underlaying machine, not the language.

An unexpected perf feature

Cyberax — Thu, 23 May 2013 04:39:48 +0000

Dependent types can be feasibly added and fat pointers are already supported in hardware (on a couple of exotic architectures, but still).

I don't see much problems with multithreading, though memory model formalization should be quite interesting.

Garbage collector of some sort seems inevitable in any case. Perhaps with some kind of region inference to help with short-lived allocations. In some limited cases it may be possible to use static proofs of correctness.

An unexpected perf feature

ebiederm — Thu, 23 May 2013 04:34:15 +0000

Talx86 can not accurately type programs encoded in assembly.

There are no dependent types for allowing the removal of bounds checks in array updates. Instead magic array macros must be used. (Not supporting general memory accesses is a signification failure in adding types to assembly language).

There is no support for multiprocessing.

There is no support for support for manual memory management. Talx86 requires a garbage collector.

Which means a large number of common kernel constructs can not be encoded in this assembler. We are unfortunately quite a ways from safe languages that can be used for kernel programming.

An unexpected perf feature

helge.bahmann — Wed, 22 May 2013 21:06:45 +0000

First, AFAICT fs/gs overrides (as well as lea) go through ALU and pay the access penalty exactly because there is no dedicated adder anymore, it would be a total waste of resources otherwise as segment overrides are in practice used just for TLS which is not that much of a fast path. (It is actually even more disturbing what happens with a base!=0 cs as it totally messes up the btb).

Second, TLBs tagged with address space identifiers are actually cheaper and more generic than segment limits: You can save the comparators (even though they don't add latency because they can operate in parallel), and address space layout can be done at will. Since for separation the ranges are inteded to be disjoint, both approaches will actually have the same TLB foot print, so no advantages here for segment limits either.

VT-x style ASIDs are only poor due to their tie-in with, err, VT-x, their equivalents work just fine on every architecture that has been designed with address space identifiers (under their various names, as context/thread/process/... identifiers) to begin with (and BTW have more general applications for fast thread switching etc.). As for the paging-related caching: You don't get rid of that using segment limits, you just pile another layer on top.

As for the attack surface regarding page tables: Why do you think it is easier to protect page tables mapping virtual address space [kern_start;kern_end) in the segment limit case, than it is to protect page tables mapping asid=kern in the asid case? (Rhetorical question, as there is no difference, so the whole argument regarding paging is a red herring).

And yes x86 segmentation is oddball in that it has never been designed for what it is now being used for, is difficult to make efficient in hardware, while address-space-based methods are both easy to make efficient and can more explicitly support the intended separation semantic. Considering segmentation to be the best solution to the problem is suffering quite a bit from Stockholm syndrome ;)

Really I don't question your accomplishments, but segmentation is a shallow local minimum, and while infinite effort can be spent trying to micro-optimize beside this minimum, there are far deeper local minima (and many of them outside x86).

An unexpected perf feature

Cyberax — Wed, 22 May 2013 20:51:27 +0000

>Show me a type system that I can accurately type functions written in assembly
Your wish is my command: http://www.cs.cornell.edu/talc/

An unexpected perf feature

Cyberax — Wed, 22 May 2013 20:50:21 +0000

"Safe language" is actually well-defined. It means a language where it's impossible to cause memory corruption, which is certainly possible.

An unexpected perf feature

PaXTeam — Wed, 22 May 2013 20:09:46 +0000

first, the cost of non-0 bases is independent of width, the CPU already has full 64 bit adders just for this purpose (think support for 'lea' and fs/gs overrides). however for my purposes what is important is the ability to define segment limits and flip the meaning of that limit (lower or upper, for expand-down segments), so non-0 bases are pretty much irrelevant.

second, segment limits are pretty much the best way to implement world separation, they require the least amount of hw resources: one parallelizable compare on the virtual address (i.e., it can be done before or in parallel to the virtual->physical translation) vs. tons of paging related caching while resolving the physical address. so no, i maintain that ditching all of segmentation was a design mistake and sparc or VT-x style ASIDs are not equivalent replacements (just ask Bromium for their performance numbers ;).

third, while SMEP/SMAP are useful performance enchancements for amd64, they require quite a bit more kernel infrastructure to make them robust. in particular, since they rely on paging, *all* of the paging related structures must be properly protected against memory corruption bugs which is quite a bit larger attack surface than the GDTs (and no kernel i know does this work except for PaX). so while i 'mourn the loss of segmentation' (which is not as oddball as you think, and it's never been an incomplete hack) i've been doing the extra work for a decade now to make paging based replacements actually secure as well ;).

An unexpected perf feature

dlang — Wed, 22 May 2013 20:02:08 +0000

Also, x96 is not the entire world, it never was on the high-end, and with ARM and MIPS appliances, it's increasingly less so on the low-end. Right now x86 is the middle ground, but it's getting squeezed from the bottom the same way that amd64 is squeezing out the former high-end

An unexpected perf feature

helge.bahmann — Wed, 22 May 2013 19:37:35 +0000

i386-style segmentation requires an add of the segment base for every address generation (or using a virtual cache which is a can of worms on its own), and while this is already not funny to do it on 32 bit without wrecking access latency (it is just barely manageable because it is possible to restrict the fast path to upper 20 bit adds), it becomes just very expensive going to 48 bit addresses (not even thinking of what would happen going to full 64 bit addresses). And yes, scrapping this add even on 32 bit is a saving significant enough to the point that current-gen CPUs short-circuit base=0 and pay a penalty otherwise.

Segmentation anyways is a poor sustitute for what you *really* want: a full "world separation" between kernel and user, and this has since ages been possible e.g. on Sparc without any segmentation and much more efficiently using address space identifiers. And while amd64 took away segmentation, it also brought with virtualisation the ASIDs (admittedly, both are unfortunately annoyingly closely tied together).

It might make more sense to look forward and maybe figure out if the new facilities can be used to do the isolation properly (and that may perhaps include talking to chip makers), rather than looking backward and mourning the loss of an oddball capability that enabled an incomplete hack.

An unexpected perf feature

ebiederm — Wed, 22 May 2013 18:07:45 +0000

Show me a type system that I can accurately type functions written in assembly, and I will believe there are safe languages that could be used.

Until we can stop escaping the type-system in a kernel there is no such thing as a safe language.

An unexpected perf feature

nix — Wed, 22 May 2013 16:43:08 +0000

Quite. It didn't even make the silicon appreciably simpler, because the CPU still has to drag it all around for 32-bit code, even while the CPU is in long mode (unlike vm86 which it can skip entirely and doesn't need to make particularly efficient in any case). The most they could do was drop optimizations for non-maximally-sized segments, and they did *that* before x86-64 was even thought of. It frees up some opcodes that they could reuse, is all.

An unexpected perf feature

nix — Wed, 22 May 2013 16:40:11 +0000

I'd blame the language *as well* as the API. C makes it very easy to write insecure and buggy code, but the API being ridiculously complex gives you a lot of corners to make undetected-until-too-late mistakes in. I don't see how anyone could be confident that anything of that complexity was secure :/

(This is really the same complaint I have about SELinux policies. I *like* complexity, but Schneier is right: it is the enemy of security -- and the entire programmer profession is helping out. Including me. Ah well.)

An unexpected perf feature

fuhchee — Wed, 22 May 2013 13:45:46 +0000

"At that time, it was not recognized as a security problem ..."

The passive voice underlines the uncertainty. One can certainly take Greg at his word that he didn't realize it. However, the absence of formal treatment at the security@ mailing list by others is just as consistent with unawareness as with sweeping-under-the-rug.

Bug class

cesarb — Wed, 22 May 2013 13:35:52 +0000

> and even if the value is not preserved during the narrowing cast it may not be a bug but intended behaviour and it's very hard to tell for a compiler.

If the narrowing losing the higher bits is intended behavior, the programmer should either use an explicit cast instead of an implicit one (x = (int)y instead of int x = y), or explicitly mask out the upper bits (x = y & 0xffffffffu). This would be similar to having to add an extra pair of parenthesis when using bitwise operators within an if/while/for, and would also make it more visible to readers of the code that the narrowing is taking place. So this class of false positives (intentionally masking by narrowing) is not a problem.

However, I do agree that the rest of the false positives would be a problem. What would be needed would be a way to somehow track the range of the variables, so the checker would know that for instance the variable being narrowed is in the range -1..-4095 (the error code case).

> however some of them look genuine bugs, albeit nothing security related so far (of those that i checked that is).

Will they be reported upstream? A bug is a bug, so even if they are not security related, they should be fixed.

Bug class

PaXTeam — Wed, 22 May 2013 12:28:39 +0000

> [...]to detect this class of bug (value being truncated by being assigned from a larger type to a smaller type)

it's not true that casting a value from a wider type to a narrower one is a bug in general (think how often long->int happens when error codes are returned in the kernel) so detecting that construct would produce immense amounts of false positives (based on our own experiments last year we're talking about many thousands of instances for allyesconfig/amd64). and even if the value is not preserved during the narrowing cast it may not be a bug but intended behaviour and it's very hard to tell for a compiler.

with that said, Emese Revfy wrote a gcc plugin for us that tries to detect this specific instance of unchecked signed array index usage. the results are somewhat more managable, there're about 200 instances on allmod/amd64, most of which are false positives (interprocedural analysis, etc would help eliminate most of them, but that's not a half an hour project as this one was), however some of them look genuine bugs, albeit nothing security related so far (of those that i checked that is).

> And the related sign confusion class of bug (assigning from unsigned to
> signed or vice versa without checking the range first)?

the size overflow plugin from Emese (http://forums.grsecurity.net/viewtopic.php?f=7&t=3043) does this but we had to scale it back due to the amount of false positives (even gcc creates these itself during canonicalization), as it's just not feasible to eliminate such constructs for now.

An unexpected perf feature

PaXTeam — Wed, 22 May 2013 12:09:45 +0000

i would not call the effective removal of segmentation from amd64 a cleanup, more like the proverbial case of the baby going with the bathwater. sure, conforming code segments and call gates could be called obsolete, but the ability to define windows on the virtual address space is very useful (AMD had to add some of it back temporarily for VMware before hw virtualization caught on) and is a real shame that it got almost completely removed. it's a design mistake, not something to be proud of.

An unexpected perf feature

iq-0 — Wed, 22 May 2013 11:46:49 +0000

There is a whole lot of ground between a hypothetical "bug-free language", a language that makes it harder to write problematic constructs, a language that warns about use of dubious constructs and a language in the land where all people carry BFGs (without a safety) and walking around with size 150 shoes.

An unexpected perf feature

cesarb — Wed, 22 May 2013 11:42:56 +0000

It is because it depends for performance on an obsolete feature of the 32-bit x86 ISA (segmentation), which was almost completely removed in the cleanup during the design of the x86-64 ISA.