When the plans for LWN.net were being laid, back toward the end of 1997, we
(Liz Coolbaugh and Jonathan Corbet) set out to create a high-quality news
source that would allow the Linux community to keep up with the incredible
pace of its development. Plus, of course, we wanted to draw attention to an
ill-advised support operation that we were launching at the same time. The
support offerings are long since forgotten, but LWN has grown with the
community it reports on. LWN, now approaching its fifth anniversary,
processes a great deal of news and requires several people for its
production. As readers of these pages have been told too many times,
advertising does not cover even a fraction of the costs of producing LWN,
and similar types of revenue, such as corporate sponsorships, are mostly
Back in July, we had come to the conclusion that LWN was not a sustainable
operation, and that it was time for us to move on to other endeavors. The
result was an amazing and unexpected show of support from our readers, in
the form of donations, that caused us to rethink things. An unexpected
show of, well, something else from our (former) credit card processor
slowed that rethink greatly, but the time has now come. LWN is now at a
definitive, and possibly final, crossroads.
We will now try to transition LWN into a subscription-based publication,
supported by the readers that benefit from it. If LWN is valuable enough
to its readers to earn that support, we will continue to produce it - and
try to make it better. If not, well, then we will search for some other
way to use our skills in the free software community.
Here's the deal: a basic LWN subscription will cost $5/month, with options
for those wanting to pay a little less or a little more. Starting
October 3, the LWN Weekly Edition will be available only to
subscribers for the week following its publication; thereafter it will be
freely available to all. Some other features, such as the ability to
receive news via email, will be available to subscribers only. Other new
features which we may introduce in the future will also be restricted to
subscribers, at least initially. The LWN front page and the news items
posted there will remain free, as will our security database, kernel patch page, and various
other features of the site.
What this means is that all news posted to LWN will continue to be freely
available - if you are sufficiently patient. But we strongly hope that
most of our regular readers will consider becoming subscribers in order to
have immediate access to our content, and, most importantly, to help keep
Those of you working in the software field may want to consider asking your
employers to fund an LWN subscription as a useful tool for your job. Or,
even better, have them talk to us about group subscriptions, which can
provide access to LWN's subscription
content for an entire group, company, or university at substantial savings
We have a lot of ideas for where we would like to take LWN. We would like
to cover many important development projects the way we cover Linux kernel
development now. We would like to have Linux in Business coverage that is
more than a pile of press releases. It would be great to be able to pay
for occasional articles by well-known authors. We will release our
site code as free software as soon as we find the time to do it.
It would be nice, even, to
have a search engine that truly works well for the first time ever.
But first we have to stabilize LWN and turn it into a sustainable
operation, and an important part of that process is now in the hands of our
readers. With sufficient support from you, we can take LWN forward and
make it better than ever: a reader-supported community news resource which
need not worry about keeping big advertisers happy. If LWN is worthwhile
to you, now is the time to act; please consider signing up for a subscription today.
Comments (60 posted)
Readers of the LWN Kernel Page have been aware of the intensive effort to
improve threading performance on Linux - at least from the kernel point of
view. Now, with the announcement
of version 0.1
of the Native POSIX Thread Library (NPTL), the user-space side of this
project has come into view. This article will take a look at the
technical and performance aspects of NPTL; the next will wander briefly
into the political issues.
Threads, of course, are processes (or something that looks like processes)
that share an address space and various other resources. Multi-threaded
applications can be tricky to write (they end up presenting the same sorts
of problems with race conditions that operating system programmers have to
deal with), but they can be a good solution to a number of programming
challenges. Your web browser, for example, likely keeps one thread around
to respond quickly to user events (mouse clicks and such) while another
thread downloads a web page and yet another one renders it onto the
screen. Java programs also tend to be highly threaded. Some applications can
create many thousands of threads; obviously, such applications can only be
reasonably run on systems with top-quality thread support.
Threading can be implemented entirely in user space, in kernel space, or a
combination of both. User-space threads are traditionally lighter weight,
since they do not require system calls and do not run in independent
processes. They can be tricky to make work in all situations, however, and
a pure user-space implementation can not make good use of multiprocessor
systems, since all threads run within a single process. So most operating
systems provide some degree of kernel support for threading. Linux has
long had this support via the flexible clone() system call, which
allows a great deal of control over which resources are to be shared with
the new thread, and which are to be private.
Pure kernel-based threads are often perceived as being slow, however, since
the kernel scheduler must be invoked to switch between threads. So
conventional wisdom has often said that the best way to get good thread
performance is with the "M:N model." M:N is a hybrid approach, where M
user space threads run on each of N kernel threads. The multiple kernel
threads allow the application to use all processors on the system, while
keeping the performance benefits of doing (most) switching between
user-space threads. Many people have said that the key to fixing the (not
great) performance of Linux threads is adopting the M:N approach.
So it is interesting to note that NPTL has, instead, stayed with the 1:1
pure kernel thread model. NPTL authors Ulrich Drepper and Ingo Molnar took
a close look at the problem, and came to the conclusion that 1:1 was, in
the end, the more promising approach. Their reasoning can be found in the
NPTL white paper (available in PDF format);
the main points are:
- The kernel problems which slow down thread performance can be
eliminated; that has been the focus of Ingo Molnar's work.
- An M:N threading implementation requires two schedulers: the usual
kernel scheduler, and a user-space implementation. Getting the two to
work together for best performance is difficult - especially if you do
not want to impact performance for the system as a whole. Rather than
duplicate the scheduling function, the NPTL implementers felt it was
better to use the (highly optimized) kernel scheduler exclusively.
- Signal handling is the bane of many threading implementations, and
M:N implementations have an even harder time of it. The 1:1 model
leaves signal handling in the kernel.
- User-space threading implementations have to go to great length to
ensure that one thread, when it performs a blocking operation, does
not block all threads running under that process; this can be a
complex task. Kernel-based threads naturally schedule (and block)
independently of each other.
Finally, the 1:1 implementation is generally simpler, since user space need
not duplicate functionality already found in the kernel.
Of course, all of that means little if the 1:1 model is unable to perform
up to expectations. The benchmarking process has just begun, but the
initial signs are encouraging. Ingo ran one
test where he started up and ran 100,000 concurrent threads - in less
than two seconds. This test would have taken about 15 minutes before
the threading improvements went into the kernel.
Ulrich Drepper has posted some other
benchmarks which mostly measure thread creation and shutdown time; some
of his results can be seen in the chart to the right..
Such a test should naturally favor the M:N model, since user-space thread
creation and destruction can be performed without any system calls. And,
in fact, the M:N Next Generation
POSIX Threading (NGPT) implementation beat standard Linux threads by
at least a factor of two in these tests. The NPTL library, however, beat
NGPT by about a factor of four. So the initial indications are that NPTL
can deliver the goods. And this is only the 0.1 release.
Comments (none posted)
At the first encounter, the Native POSIX Thread Library looks like ideal
grist for the Red Hat basher's mill. The library appears to have sprung
fully formed from the head of glibc maintainer (and Red Hat employee)
Ulrich Drepper, who has made his plans clear:
Unless major flaws in the design are found this code is intended to
become the standard POSIX thread library on Linux system and it
will be included in the GNU C library distribution.
Installing this library is not easy, since it requires some fairly
bleeding-edge software: a 2.5.36 kernel, gcc 3.2, and a current
glibc 2.3 snapshot. In fact, the "only environment known to date
which works" is an updated version of the "(Null)" Red Hat beta. And, of
course, this development has seemingly ignored the longstanding efforts of
the Next Generation
POSIX Threading project, which is at release 2.0.2.
A certain (relatively small) amount of grumbling along these lines has been
seen on the net. But it's uncalled for.
NPTL was developed independently from NGPT for a straightforward reason:
the NPTL developers wanted to try a very different approach. NGPT is
designed around the M:N model, which, as noted above, NPTL avoids. There
was no way to integrate the NPTL approach into NGPT without massive
changes. The NPTL developers did, however, work with the NGPT
hackers with regard to the kernel changes; those enhancements will benefit
both projects in the end.
A new library at version 0.1 can probably be forgiven for using
bleeding-edge tools; it is a bleeding-edge tool, after all. By the
time NPTL has stabilized, the environments available to users will have
caught up substantially. Ingo Molnar, author of the kernel side of the
NPTL work, tells us that he intends to backport the kernel changes to the
2.4 kernel once things have stabilized in 2.5. (Whether 2.4 maintainer
Marcelo Tosatti will accept them is a separate issue, of course).
And, in the end, this development has been going on for less than two
months - it is a very new initiative.
This development shows some of the best aspects of the free software
model. Two hackers with some good ideas have proved those ideas in the way
the community accepts best: with code. We will all benefit from this
Comments (none posted)
of its donation of an elliptic curve encryption implementation to the OpenSSL project
was generally well
received. After all, the donation of more open source code has got to be a
good thing. As it turns out, however, some people are looking at this gift
and wondering how free it really is.
If you look at the OpenSSL LICENSE file in
the current snapshot, nothing has changed; it's a fairly straightforward
BSD-style license. But the Sun-contributed code contains its own license
text which differs from the OpenSSL license. In particular, it contains
this rather impenetrable language:
In addition, Sun covenants to all licensees who provide a
reciprocal covenant with respect to their own patents if any, not
to sue under current and future patent claims necessarily infringed
by the making, using, practicing, selling, offering for sale and/or
otherwise disposing of the Contribution as delivered hereunder (or
One would that Sun, by virtue of having released the code under a free
license, would have given up the right to sue people for using that code.
This clause, however, seems to put a string on it: Sun explicitly says it
won't sue, but only if you don't sue them either. The limitation seems to
only apply to suits over the elliptical curve code itself, but it's hard to
say for sure; the language is not all that clear.
For those who object to this language, the distinction does not matter. If
you start attaching strings to free code, they say, it is no longer free.
The most vocal of the dissenters is, of course, OpenBSD hacker Theo de
Raadt, who states:
It means that OpenSSL is becoming a non-free software project,
because the code from Sun contains licenses which invoke patent
litigation; the licence on the new code basically builds a contract
that says "if you use this code, you cannot sue Sun". In such a
way, by means of the slippery slope, a free software project
becomes not as free, and eventually, less and less free.
Theo has gone as far as suggesting a fork in the OpenSSL project as a way
of maintaining a version that is, to his eyes, truly free.
Whether or not this particular bit of language bothers people,
there is an issue here: companies often have a hard time resisting the
temptation to attach their own language to free software licenses. The
tendency toward custom licenses for each company and project has subsided
somewhat, but it is not completely gone. It will always be necessary to
scrutinize software licenses carefully, whether they are presented as free
Comments (8 posted)
For those of you who are curious about what Bob Young has been up to since
he left Red Hat: the Lulu Tech
is happening in Raleigh, NC starting September 27. It has
the look of a fun event for people who like to play with free software, and
with cool technology in general. Wish we could be there...
Comments (none posted)
Page editor: Jonathan Corbet
Inside this week's LWN.net Weekly Edition
- Security: OpenSSL worm in the news; RATS 2.0 released; CanSecWest/core03 call for papers
- Kernel: Deadline disk I/O scheduler; Keeping disks busy; Doing things in the kernel
- Distributions: Mandrake Linux 9.0; SCO Linux 4.0 open beta; New- BanShee Linux/R, eLSD, Firegate Server, MoviX, and Serverdisk diskette distro
- Development: The Phoenix lighweight browser, PHP Audio Extension,
Heartbeat 0.4.9d, Full transaction support for MySQL, Knoda 0.5.4,
BusyBox 0.60.4, Native POSIX Thread Library 0.1, Sweep Sound Editor,
Gspoof 2.1.1, FLTK 1.1.0rc7, PerlQt-3, Tcl/Tk 8.4.0.
- Commerce: Campaign for Digital Rights responds to patent office paper; Regal deploying Linux in hundreds of theaters
- Press: EU researches open-source, Building the underground computer railroad,
A Bounty on Spammers, Red Hat 8.0 coming soon.
- Announcements: YAPC 2003 CFP, The Business Case for Open Source Software.
- Letters: Closed betas; Vice-president Gates