Signed overflow optimization hazards in the kernel [LWN.net]

Signed overflow optimization hazards in the kernel

Posted Aug 20, 2012 8:59 UTC (Mon) by etienne (guest, #25256)
Parent article: Signed overflow optimization hazards in the kernel

> So it is not too early to start future-proofing the Linux kernel by removing its reliance on signed integer overflow!

Would be nice to have a GCC option for ia32 so that any "add" used for signed arithmetic is followed with "into" (exception if overflow).
It will not slow too much the execution ("into" will not be predicted as a taken jump), and could help locate potential problems...

to post comments

Signed overflow optimization hazards in the kernel

Posted Aug 22, 2012 8:51 UTC (Wed) by mpr22 (subscriber, #60784) [Link] (1 responses)

-ftrapv is an architecture-independent compiler option to GCC. I don't know whether it works.

Signed overflow optimization hazards in the kernel

Posted Aug 22, 2012 13:21 UTC (Wed) by etienne (guest, #25256) [Link]

Seems like -ftrapv can work, but it is far from just adding an "into" instruction after each signed "add" on ia32 - note that "into" has disappeared in amd64 instruction set.

$ cat test.c
int a, b, c;
void main (void) {
c = a + b;
}
$ gcc -m32 -O2 -ftrapv test.c
$ objdump -d a.out

a.out: file format elf32-i386
....
080483f0 <main>:
push %ebp
mov %esp,%ebp
and $0xfffffff0,%esp
sub $0x10,%esp
mov 0x804a01c,%eax
mov %eax,0x4(%esp)
mov 0x804a020,%eax
mov %eax,(%esp)
call 8048420 <__addvsi3>
mov %eax,0x804a024
leave
ret
....
08048420 <__addvsi3>:
push %ebp
mov %esp,%ebp
push %ebx
sub $0x4,%esp
mov 0xc(%ebp),%ecx
mov 0x8(%ebp),%edx
call 804845c <__i686.get_pc_thunk.bx>
add $0x1bc2,%ebx
test %ecx,%ecx
lea (%ecx,%edx,1),%eax
js 8048450 <__addvsi3+0x30>
cmp %edx,%eax
setl %dl
test %dl,%dl
jne 8048457 <__addvsi3+0x37>
add $0x4,%esp
pop %ebx
pop %ebp
ret
xchg %ax,%ax
cmp %edx,%eax
setg %dl
jmp 8048444 <__addvsi3+0x24>
call 80482fc <abort@plt>

Without ftrapv:
080483c0 <main>:
mov 0x804a01c,%eax
add 0x804a018,%eax
push %ebp
mov %esp,%ebp
mov %eax,0x804a020
pop %ebp
ret