A new linker is not generally something that arouses much interest outside
of the hardcore development community—or even inside it—unless
it provides something especially eye-opening. A newly released linker,
called gold has just that kind of feature, though, because it runs
up to five times as fast as its competition. For developers who do a lot
of compile-link-test cycles, that kind of performance increase can
significantly increase their efficiency.
Linking is an integral part of code development, but it can be invisible,
as it is often invoked by the compiler. The sidebar accompanying this
article is meant for
non-developers or those in need of a refresher about linker operation.
For those who want to know even more, the author of gold, Ian Lance
Taylor, has a twenty-part series about linker internals on his weblog,
starting with this entry.
For Linux systems, the GNU Compiler
Collection (GCC) has been the workhorse by
providing a complete toolchain to build programs in a number of different
languages. It uses the ld linker from the binutils collection. With
that gold has been added to binutils, there are now two
choices for linking GCC-compiled programs.
A linker overview
For non-developers, a quick overview of the process that turns source code
into executable programs may be helpful.
Compilers are programs that turn C—or other high-level
languages—into object code. Linkers then collect up object
code and produce an executable. Usually the linker will not only operate
on object code created from a project's source, but will also reference
libraries of object code—the C runtime library libc for
example. From those objects, the linker creates an executable program that
a user can invoke from the command line.
The linker allows program code in one file
to refer to a code or data object in another file or library. It arranges
that those references are usable at run time by
substituting an address for
the reference to an object. This "links" the two properly in the executable.
Things get more complicated when
considering shared libraries, where the library code is shared by multiple
concurrent executables, but this gives a rough outline of the basics of
The intent is for gold to be a complete drop-in replacement for
ld—though it is not quite there yet. It is currently
lacking support for some command-line options and Linux kernels that are
linked with it do not boot, but those things will come. It also currently
only supports x86 and x86_64 targets, but for many linker
jobs, gold seems to be working well. The speed seems to be very
some developers, with Bryan O'Sullivan saying:
When I switched to using gold as the linker, I was at first a little
surprised to find that it actually works at all. This isn't especially
common for a complicated program that's just been committed to a source
tree. Better yet, it's as fast as Ian claims: my app now links in 2.6
seconds, almost 5.4 times faster than with the old binutils linker!
Performance was definitely the goal that Taylor set for gold
development. It supports ELF (Executable
and Linking Format) objects and runs on UNIX-like operating systems
only. Only supporting one object/executable format, along with a fresh
start and an explicit performance goal are some of the reasons that
gold outperforms ld.
Tom Tromey likes the
looks of the code:
I looked through the gold sources a bit. I wish everything in the GNU
toolchain were written this way. It is very clean code, nicely commented,
and easy to follow. It shows pretty clearly, I think, the ways in which C++
can be better than C when it is used well.
Because the implementation is geared for speed, Taylor used techniques that
may confuse some.
He has some concerns
about the maintainability of his implementation:
While I think this is a reasonable approach, I do not yet know how
maintainable it will be over time. State machine implementations can be
difficult for people to understand, and the high-level locking is
vulnerable to low-level errors. I know that one of my characteristic
programming errors is a tendency toward code that is overly complex, which
requires global information to understand in detail. I've tried to avoid it
here, but I won't know whether I succeeded for some time.
Overall, it seems to be getting a nice reception by the community, with
O'Sullivan commenting that he is "looking forward to the point where
gold entirely supplants the existing binutils linker. I expect that won't
take too long, once Mozilla and KDE developers find out about the
performance boost." Once gold gets to that point, Taylor
is already thinking about concurrent
linking—running compiler and linker at the same time—as
the next big step.
There are two other ongoing projects that are working with the greater GCC
ecosystem in interesting ways: quagmire and ggx. Quagmire is an effort to
replace the GNU configure and build system—consisting of autoconf,
automake, and libtool—with something that depends
solely on GNU make. Currently, that system uses
various combinations of the shell, m4, and portable makefiles to make the
building and installation of programs easy—the famous
"./configure; make" command line. The tools were written that way
to try and ensure that users did not need to install additional packages to
configure and build GNU tools.
Quagmire, which has roots in a
posting by Taylor
recognizes that GNU make is ubiquitous, so basing a
system around that makes a great deal of sense.
The ggx project is Anthony Green's step-by-step procedure to create an
entire toolchain that can build programs for a processor architecture that he is
creating as a thought
experiment. The basic idea is to design the instruction set based on
the needs of the compiler, in this case GCC, rather than the needs of the
hardware designers. He is using GCC's ability to be retargeted for new
architectures, along with its simulation capabilities to create a CPU that
he can write programs for. As of this writing, he has a "hello world"
program working, along with large chunks of the GCC test suite passing.
Well worth a look.
Comments (146 posted)
Last month, an article about
to free the proprietary Ryzom game expressed
frustration with the implied idea that the free software community could
not, on its own, create a game experience comparable to Ryzom. One of the
with (what was seen as) a dismissive attitude toward the Second Life client
and pointed out some of the work which is being done based on that client. So your
editor decided to take another look. The bottom line is this: the work
being done in this area is still in an early and unstable state, but it
does have the potential to open a new frontier for free software in the
area of virtual environments.
Life client for Linux is now in a beta release. "Beta," in this case,
means that all of the features have, in some way, been implemented; now
it's just a matter of making it all actually work. Your editor found the
client to be slow, unwieldy, crash-prone, and very fussy about its graphics
environment. Your editor's well-supported (in X) Intel-based desktop was
not adequate for this client, for example; the associated documentation
recommends a long list of cards which (for now) are only supported with
proprietary drivers. Still, on the right system, the
client is able to render three-dimensional worlds with the same quality
that, well, Second Life has on any platform.
An alternative is OpenViewer, a
C#/Mono-based, BSD-licensed viewer project. Your editor had little luck
getting this client going, but the screenshots are nice. The developers
appear to have made significant progress toward the creation of a
functional, three-dimensional client; this is a project to watch. Less far
along is the Aether project,
which is working on a OpenViewer-based client meant to run within Firefox;
thus far, it has a nice design diagram but not much else.
There is also RealXtend, a project
based on the Second Life client which is emphasizing performance and visual
quality. Unfortunately, it also seems to be emphasizing Windows support,
so your editor did not give it a try.
Free software clients are certainly an important tool to have; we will not
be able to access this kind of virtual environment without them. But it
would be a real shame if these clients simply facilitated a world where we
use free clients to access locked-down, proprietary virtual worlds on
somebody else's server. What would be much better would be the ability to
create our own virtual worlds - using free software, of course - and to
link those worlds into a larger virtual universe. That is the formula
which made the World Wide Web (and many other Internet services) work, and
it should certainly be applicable in this context as well.
The good news is that people are working in this area. One project, OpenSim, has the look of
something which is about to achieve much wider awareness as its features
mature. In short, OpenSim is a virtual world server which can be deployed
to create environments much like what one would find in Second Life. It
works with the Second Life client and with OpenViewer as well, and it
presents a very similar experience - at least, in the virtual worlds which
have been deployed so far. Since it's free software, it can be customized
toward the creation of different kinds of environments, including
role-playing games and such.
It is written with C# and Mono - seemingly a common choice for this kind of
software. The Mono environment, for all its faults and potential pitfalls,
may well make it easier to create a cross-platform application with the
What makes OpenSim really interesting, though, is its ability to connect
servers together in a "grid" mode. Once this is done, a virtual world is
not limited to a single entity's server (or imagination). Servers across
the net can be interconnected into a single, larger world. This is the
feature which has the potential to take OpenSim from another interesting
project into something which transforms the net.
There are a number of people organizing grids with OpenSim now; there is a list of public grids
on the OpenSim site. Some of them appear to be relatively proprietary
operations offering the opportunity to buy virtual land - though subprime
loans are unavailable. Others allow anybody connect their server
into the grid and become part of the whole. These grids appear, in
general, to be in a sort of early adopter state at the moment, but much of
the fundamental functionality is there. How hard could it be to make it
all work properly at this point?
The answer to that question, of course, is "quite hard." But the fact
remains that people are working on this very interesting problem, and they
are making significant progress toward solving it. These projects bear
watching; they may well be planting the seeds of the systems we will all be
using in the coming years.
Comments (14 posted)
The Open Source Initiative
formed almost ten years ago to safeguard the "Open Source" name. Over the
years it has approved licenses and attempted some other activities while,
generally, having little relevance to the wider community. It has often
been seen as a relatively closed and non-democratic organization. Now one
of OSI's founders is trying to get back into the organization and change
its direction; the outcome of the resulting discussion may (or may not)
change the direction of the OSI.
Bruce Perens has launched a bid to be elected to the OSI
board of directors, but this bid has
not been particularly well received by the current board. His on-line petition to collect community support
specifies a number of reasons that he wants to be on the board—those
reasons are ruffling some feathers. Outgoing board member Matt Asay has taken Perens to
task for some of his statements as has OSI president
Perens's reasons for wanting to be on the board are threefold: reducing the
over-representation of vendors, trying to ensure Microsoft does not get a
seat on the board, and reducing license proliferation. The idea of a Microsoft seat on
an open source organization's board is sure to rile a segment of the
community, which is undoubtedly part of what Perens is hoping for. The
likelihood of that happening is pretty small, though. Tiemann makes it
clear that the board doesn't elect companies at all:
The OSI nominates people to the board despite their corporate affiliations,
not because of them. The idea that the OSI would elect a "Microsoft" board
member is as absurd as the idea that we'd elect a "Google" board member or
an "IBM" board member. We elect people based on their own merits, not the
merits (or demerits) of the companies or organizations they are affiliated
Microsoft and its employees do not currently contribute to open source in
way, so there is little that would lead the board to nominate them. If that ever
changes, it would be pretty disingenuous to deny someone a seat because of
their employer's past—or even at that time,
current—misbehavior. In addition, it is hard
to see how one board member—Perens or someone "controlled" by
Microsoft—is going to make such a crucial difference in what the board
does anyway. In many ways, the Microsoft connection is
a red herring—one sure to rally the troops, though.
Reducing license proliferation is a noble goal, one that the OSI tried to
tackle a few years back without much in the way of tangible success.
Perens states that he would like to see OSI do more reduce the number of
licenses, but his claims about the number of licenses needed have raised
Another problem is the failure to reduce the number of different licenses
in general use. My own work in this area shows that only four licenses, all
compatible with each other, can satisfy all common business and
non-business purposes of Open Source development. Three of these licenses
have essentially the same text, and the fourth is very short. Life would be
easier if more projects used them. While it would be difficult to shut down
approval of new licenses, I think OSI could be more proactive at reducing
Part of the reason that Tiemann and others are skeptical is due to some
obvious bad blood between the board and Perens over the license
proliferation committee. LWN covered some of that "debate" in
August 2005. Perens clearly believes he should have been a
member just as strongly as others on the board seem to feel he should not
have been. When the board was formed without him as a member, Perens
refused to participate in the process in any way. It
seems to stick in the craw of some for Perens to now claim that he has the
solution. Russ Nelson, former OSI president and current board
member—as well as a member of the committee—sums up the
frustration in a comment on Tiemann's post:
I don't see how Bruce can claim to have a short list of four licenses. I
start with BSD, GPLv2, GPLv3, LGPLv2 and LGPLv3 and that's five. If he
thinks that people should simply agree with him that all GPLv2 should be
relicensed GPLv3, I invite him to spend some time with Linus Torvalds, who
notoriously and politely disagrees.
Having a solution is not the same as convincing people to adopt it.
It is rather interesting to see Perens trying to get back on the board that
he famously resigned from in 1999
after having founded the organization with Eric Raymond in 1998. This is
not the first time Perens has lost interest and/or resigned from some form of community
leadership position; Debian and UserLinux spring to mind. Though none of
the expressed concerns about his candidacy have mentioned it, some must be
wondering how long it would be before ideology or a shifting focus caused
Perens to move on from a board position if he were elected.
Perens has been an excellent advocate for free software and/or open source
over the years, but his tendency towards self-promotion
grates on some. It may not be an ego thing, as he claims, but it certainly
rubs some people the wrong way. The ego issue is one of the reasons that board observer Andrew Oliver does
not support Perens for the board:
A return to a very Amerocentric hacker culture voice with big egos is not
the answer to OSI's problems. I think OSI is on the path to real
fundamental change. I'd like to hear Bruce explain what he'd do differently
in collaboration with others who may not always agree with him.
Asay certainly doesn't see Perens as
having the right credentials either:
The OSI needs a vibrant membership of those currently shaping the open
source landscape. It's possible that its current make-up doesn't reflect
this. Point well taken. But it's equally possible - indeed, I'd say
probable - that Bruce's directorship wouldn't change this. I like Bruce but
aside from the occasional picketing he does, I can't point to anything
substantive he has done for open source in the past half-decade or so.
The petition drive came about because Tiemann encouraged Perens to show
that there was strong community support for him to be a part of the board.
As of this writing, the petition has garnered more than 1700 "signatures",
which Perens believes is enough:
Regarding my candidacy, OSI's board, through its president, asked me to
show an uprising of strong community support if the board was to to elect
me. I have. Now that I have done what you asked, are you going to hide
behind complaints about my campaign, which is really quite mild in its
criticism and is in no way the "scorched earth" that Matt refers to, or are
you going to do what you said? If you OSI can't handle a political opponent
on my laid-back scale, you'd only looking for yes-men.
The OSI board is "self-replacing" with current board members nominating and
electing candidates for empty slots. Each director serves for a three-year
term, with roughly one-third coming up for election each year—though
this year there are five slots to be filled. Three directors are standing
for re-election, leaving two slots open. Unfortunately, it's not clear
when the actual election will be held, nor is there likely to be any
advance notice of who has been nominated. Transparency, it seems, is not
one of the attributes of OSI.
Self-replacement and overlapping terms of office tend to give a certain
stability to a board, but it also creates a kind of inbreeding. It is
unlikely that a board will nominate people who think substantially
differently from themselves. This is one thing that Perens is trying to
circumvent with his very public candidacy. Whatever else can be said about
Perens's candidacy, it is clear that he would bring a different voice into
the OSI boardroom.
But, what is OSI really? Is it an organization that is somehow
supposed to represent all of the diverse voices in the community? At the moment it appears to exist for
the purpose of approving licenses and "protecting the Open Source Definition".
Perens thinks it could be more than that. OSI itself seems to agree as
they have been moving towards more relevance in the community. Oliver
describes that effort:
OSI is trying to solve its problems, by becoming more grassroots and less
bottom up. Meanwhile, it is trying to grow the movement by expanding its
international representation. Corporations do influence OSI, in that not
all of the board has a free hand to say what is on their mind
publicly. However, the solution is to make the OSI board what it should
be: a governance board.
OSI and its board are currently in a state of flux, trying to define a
role for themselves that is broader than just a license approval body. There
doesn't seem to be a lot of discontent within the board that might
lead to Perens or another controversial figure being added. Whether this
leads to continued stagnation or a more vibrant OSI remains to be seen. A
more interesting question might be: will anyone care?
If OSI starts to do visible things for the community, it will finally
acquire some relevance. Given the attitude towards his candidacy, it seems
unlikely that Perens will be able to lead the board in that direction.
Which leaves it up to the current board and the two new
members—neither of which are likely to be Perens—to find a way
to make the community care.
Comments (55 posted)
Page editor: Jonathan Corbet
Inside this week's LWN.net Weekly Edition
- Security: Voting machine integrity through transparency; New vulnerabilities in firefox, openssh, xine-lib, xwine,...
- Kernel: Kernel markers and binary-only modules; Predictive ELF bitmaps; Atomic context and kernel API design.
- Distributions: Distribution-friendly projects - Part 1; Ubuntu 8.04 LTS Beta; Fedora 9 beta; Fifteen years of NetBSD; 2 linux-libre kernels; SliTaz GNULinux
- Development: Introducing Sphinx, the Python documentation toolchain, Free Telephony Project ports to Blackfin, new versions of BusyBox, xsel, GNU Radio, GnuPG, Semeir, SQL-Ledger, Games for Eclipse, Fraqtive, Wine, HOgg, Chandler, PyKeylogger, JOSSO, Parrot, Bazaar, Mercurial.
- Press: Protecting the Internet Without Wrecking It, a look at the patent reform bill, India cans OOXML, the story of Meraki, reviews of KDE 4.1, Xenomai/SOLO, AlertMe.
- Announcements: Curl turns 10, Document Freedom Day, GSoC announcements, CrossOver Games, Microsoft open source lab tour, Groklaw gets FSF award, AMK on PyCon 2008, ELC 2008, LinuxWorld's Linux Garage.