Striking gold in binutils [LWN.net]

Striking gold in binutils

Posted Mar 29, 2008 10:52 UTC (Sat) by stock (guest, #5849)
Parent article: Striking gold in binutils

hehe what a gas, I have been told by the real old diehard UNIX men that  
gcc was a total shit of banana republic code, as its C++ part was of  
utter inferior quality. With the arrival of Ian's gold linker gcc as a  
C/C++ package has finally moved up in the ranks. 
 
And it was about time also, as the impact of open source and the 
obliged use of gcc made many people puke. Now instead of naming it   
gold, integrate it properly inside the gcc "compilersuite" and make it  
use by default when compiling c++ code. But that's where the problem  
is, as many source trees are hybrid code of .c and .cpp files. 
 
As to linker problem when compiling Linux vmlinux kernels, i like to 
suggest that Torvalds et.al. could solve this inside their "Makefile"  
of the Linux kernel source tree. Some time ago i wanted to checkout how  
a vmlinuz binary actually was created by doing every command by hand.  
It took me more as two days to distillate the commands by hand from the  
various Kbuild and related patch works and custom utils. 
 
Robert 
--  
Robert M. Stockmann - RHCE 
Network Engineer - UNIX/Linux Specialist 
crashrecovery.org  stock@stokkie.net

(Log in to post comments)

Striking gold in binutils

Posted Mar 29, 2008 13:44 UTC (Sat) by nix (subscriber, #2304) [Link]

Your comment belongs on slashdot, not here. Let's see how many falsehoods 
I can eliminate before ncm gets here and vapourises you.

Firstly, your 'old diehard UNIX men' are clueless. GCC relied on a lot of 
ad-hoc algorithms at one point: that did not make it a 'total shit of 
banana republic code' by any standard. G++ was one of the first C++ 
compilers written, I think the first that did not emit C code: so it 
certainly wasn't of 'utter inferior quality' then. GCC as a whole went 
through a bad patch in the early-to-mid-90s, when maintenance largely 
stalled: but once egcs started (has it been ten years already?) G++ was 
one of the first parts to improve.

It's definitely not 'utterly inferior' now, in fact it's one of the better 
compilers out there, with an internal architecture that's gone from being 
crufty as anything and full of hidden dependencies to being, well, much 
less crufty over the last few years (cleaning up code that old is an 
achievement).

Its only remaining hole is optimization: especially on machines with small 
register files, vendor compilers could often out-optimize it. This, too, 
is steadily being fixed.

Introducing a new linker, no matter how hotshot (and gold *is* a damn good 
piece of work) doesn't make the *compiler* better or worse in any way. I 
can't fathom the misunderstandings that could lead you to believe that 
switching linkers depending on the source language in use has any merit at 
all. The most integration that is likely to happen seems to be to have the 
linker running in parallel with the compiler, accepting object code and 
incrementally linking it as the compiler disgorges it. Anything more than 
that seems pointless.

Makefiles are the wrong place to solve linker script problems. The problem 
is that GNU ld was fundamentally driven by linker scripts, and gold is not 
(as well as that the Linux kernel constructs some seriously contorted ELF 
files: OS kernels don't have to stick to the rules: they are generally 
loaded by special-purpose code, not by ELF loaders).

The obvious way to see what commands are executed when running make is to 
use 'make -n', but if some of the commands are themselves makes you'll 
need to do something else. It's probably simplest just to add a couple of 
printf()s to make itself, or strace the whole thing with "-ff -e 
trace=process". Digging the commands out of the makefile by hand is 
madness.

Striking gold in binutils

Posted Apr 3, 2008 19:03 UTC (Thu) by stock (guest, #5849) [Link]

" Digging the commands out of the makefile by hand is madness."  
  
You bet, but IF you would do such a thing, you would understand that on a  
64-bit platform we are actually booting a 32bit binary :  
  
ld -m elf_i386  -Ttext 0x100000 |\  
        -e startup_32 -m elf_i386 |\  
        arch/x86_64/boot/compressed/head.o |\  
        arch/x86_64/boot/compressed/misc.o |\  
        arch/x86_64/boot/compressed/piggy.o |\  
        -o arch/x86_64/boot/compressed/vmlinux  
  
Apparently we are actually booting a 32bit binary on our x86_64 platform.  
But that doesn't matter as long as we call our real kernel piggy.o :  
  
ld -m elf_i386  -r --format binary --oformat elf32-i386 |\  
        -T arch/x86_64/boot/compressed/vmlinux.scr |\  
        arch/x86_64/boot/compressed/vmlinux.bin.gz |\  
        -o arch/x86_64/boot/compressed/piggy.o  
  
where vmlinux.bin.gz is created by the following command :  
  
gzip -f -9 < arch/x86_64/boot/compressed/vmlinux.bin > |\  
        arch/x86_64/boot/compressed/vmlinux.bin.gz  
  
where vmlinux.bin is created by running a custom tool called objcopy :  
  
objcopy -O binary -R .note -R .comment -S  vmlinux |\  
         arch/x86_64/boot/compressed/vmlinux.bin  
  
where finally vmlinux is that ELF 64-bit LSB executable :  
  
ld -m elf_x86_64 -e stext -T arch/x86_64/kernel/vmlinux.lds.s |\  
     arch/x86_64/kernel/head.o arch/x86_64/kernel/head64.o |\  
     arch/x86_64/kernel/init_task.o   init/built-in.o |\  
     --start-group  usr/built-in.o  arch/x86_64/kernel/built-in.o |\  
     arch/x86_64/mm/built-in.o  arch/x86_64/ia32/built-in.o |\  
     kernel/built-in.o  mm/built-in.o  fs/built-in.o  ipc/built-in.o |\  
     security/built-in.o  crypto/built-in.o  lib/lib.a |\  
     arch/x86_64/lib/lib.a  lib/built-in.o  arch/x86_64/lib/built-in.o |\  
     drivers/built-in.o  sound/built-in.o  arch/x86_64/pci/built-in.o |\  
     net/built-in.o --end-group .tmp_kallsyms2.o -o vmlinux  
  
   -rwxr-xr-x  1 root root 7478601 Nov 21 14:48 vmlinux  
  
   vmlinux: ELF 64-bit LSB executable, AMD x86-64, version 1 (SYSV),  
            statically linked, not stripped  
  
Cheers,  
  
Robert  
--   
Robert M. Stockmann - RHCE  
Network Engineer - UNIX/Linux Specialist  
crashrecovery.org  stock@stokkie.net

Striking gold in binutils

Posted Apr 3, 2008 20:31 UTC (Thu) by nix (subscriber, #2304) [Link]

IF you would do such a thing, you would understand that on a 64-bit platform we are actually booting a 32bit binary

Actually, the only things that differ between the elf_x86-64 and elf_i386 emulations are details of the GOT and PLT (which don't exist for the kernel normally), details of relocation processing (which isn't carried out for the kernel), and the start address (which the kernel overrides anyway in its custom linker script). So since they do the same thing as far as the kernel's concerned, let's pick the one that everyone's got because it's present even in older x86 binutils and always present in newer ones, and that's elf_i386.

(At least that's my understanding.)

where vmlinux.bin is created by running a custom tool called objcopy :

Not only is objcopy not a 'custom' tool, but part of the GNU binutils, but it predates Linux (although before about 1993 it was called 'copy'). (The renaming was done by, oh look, Ian Lance Taylor. He's been involved in this area for a long, long time. ;} )

(The objcopy run is anyway hardly crucial: like most of the stuff before the final link, it's just a size optimization.)