Weekly Edition
Return to the Kernel page
| |||||||||||||
The kernel and the C library as a single project
The kernel has historically been developed independently of anything that
runs in user space. The well-defined kernel ABI, built around the POSIX
standard, has allowed for a nearly absolute separation between the kernel
and the rest of the system. Linux is nearly unique, however, in its
division of kernel and user-space development. Proprietary operating systems
have always been managed as a single project encompassing both user and
kernel space; other free systems (the
BSDs, for example) are run that way as well. Might Linux ever take a
more integrated approach?
Christopher Yeoh's cross-memory attach patch was covered here last September. He recently sent out a new version of the patch, wondering, in the process, how he could get a response other than silence. Andrew Morton answered that new system calls are increasingly hard to get into the mainline:
We have a bit of a track record of adding cool-looking syscalls and
then regretting it a few years later. Few people use them, and
maybe they weren't so cool after all, and we have to maintain them
for ever. Bugs (sometimes security-relevant ones) remain
undiscovered for long periods because few people use (or care
about) the code.
Ingo Molnar jumped in with a claim that the C library (libc) is the real problem. Getting a new feature into the kernel and, eventually, out to users takes long enough. But getting support for new system calls into the C library seems to take much longer. In the meantime, those system calls languish, unused. It is possible for a suitably motivated developer to invoke an unsupported system call with syscall(), but that approach is fiddly, Linux-specific, and not portable across architectures (since system call numbers can change from one architecture to the next). So most real-world use of syscall() is probably due to kernel developers testing out new system calls. But, Ingo said, it doesn't have to be that way:
If we had tools/libc/ (mapped by the kernel automagically via the
vDSO), where people could add new syscall usage to actual,
existing, real-life libc functions, where the improvements could
thus propagate into thousands of apps immediately, without
requiring any rebuild of apps or even any touching of the
user-space installation, we'd probably have _much_ more lively
development in this area.
Ingo went on to describe some of the benefits that could come from a built-in libc. At the top of the list is the ability to make standard libc functions take advantage of new system calls as soon as they are available; applications would then get immediate access to the new calls. Instrumentation could be added, eventually integrating libc and kernel tracing. Perhaps something better could have been done with asynchronous I/O. And so on. He concluded by saying "Apple and Google/Android understands that single-project mentality helps big time. We don't yet." As of this writing, nobody has responded to this suggestion. Perhaps it seems too fantastical, or, perhaps, nobody is reading the cross-memory attach thread. But it is an interesting idea to ponder on. In the early days of Linux kernel development, the purpose was to create an implementation of a well-established standard for which a great deal of software had already been written. There was room for discussion about how a specific system call might be implemented between the C library and the kernel, but the basic nature of the task was well understood. At this point, Linux has left POSIX far behind; that standard is fully implemented and any new functionality goes beyond it. New system calls are necessarily outside of POSIX, so taking advantage of them will require user-space changes that, say, a better open() implementation would not. But new features are only really visible if and when libc responds by making use of them and by making them available to applications. The library most of us use (glibc) has not always been known for its quick action in that regard. Turning libc into an extension of the kernel itself would short out the current library middlemen. Kernel developers could connect up and make use of new system calls immediately; they would be available to applications at the same time that the kernel itself is. The two components would presumably, over time, work together better. A kernel-tied libc could also shed a lot of compatibility code which is required if it must work properly with a wide range of kernel versions. If all went well, we could have a more tightly integrated libc which offers more functionality and better performance. Such a move would also raise some interesting questions, naturally, starting with "which libc?" The obvious candidate would be glibc, but it's a large body of code which is not universally loved. The developers of whichever version of libc is chosen might want to have a say in the matter; they might not immediately welcome their new kernel overlords. One would hope that the ability to run the system with an alternative C library would not be compromised. Picking up the pace of libc development might bring interesting new capabilities, but there is also the ever-present possibility of introducing new regressions. Licensing could raise some issues of its own; an integrated libc would have to remain separate enough to carry a different license. And, one should ask, where would the process stop? Putting nethack into the kernel repository might just pass muster, but, one assumes, Emacs would encounter resistance and LibreOffice is probably out of the question. So a line needs to be drawn somewhere. This idea has come up in the past, and the result has been that the line has stayed where it always was: at the kernel/user-space boundary. Putting perf into the kernel repository has distorted that line somewhat, though. By most accounts, the perf experiment has been a success; perf has evolved from a rough utility to a powerful tool in a surprisingly short time. Perhaps an integrated C library would be an equally interesting experiment. Running that experiment would take a lot of work, though; until somebody shows up with a desire to do that work, it will continue to be no more than a thought experiment. (Log in to post comments)
The kernel and the C library as a single project Posted Nov 30, 2010 20:37 UTC (Tue) by JoeBuck (subscriber, #2330) [Link] Given the huge rate of change of the kernel, the fact that the separateness of glibc imposes a speed bump, and that cool new kernel interfaces (that might disappear with the next release) are tricky to access until glibc's developers have agreed to add support, might not really be such a problem and could be seen as a feature.
The kernel and the C library as a single project Posted Nov 30, 2010 20:55 UTC (Tue) by nix (subscriber, #2304) [Link]
Also, upgrading your C library is dangerous. It's just about the only thing in the system where if you mess up installing it, or build a broken version and don't run 'make check', you break your whole system at once with no way back without use of an emergency boot disk, a netboot, or a statically linked emergency busybox shell (which you must have prepared in advance, of course, as with no working libc you're not going to be able to compile or install it after the fact). None of that is true of the kernel, unless you're unlucky enough to install a kernel that scrags your filesystem.
I'm happy to install new kernels frequently. I'm much less happy to install new glibcs frequently: I'm on tenterhooks even when a distribution glibc upgrade comes down the pipe, because I have known *those* to scrag my whole system in the past as well.
The kernel and the C library as a single project Posted Nov 30, 2010 21:05 UTC (Tue) by sfeam (subscriber, #2841) [Link]
Actually, making libc a kernel module would have the opposite effect. Whereas now if you manage to install a bad libc your system is unusable, if it were instead a matter of installing a new kernel version with a bad libc then you could just boot into the previous kernel and your good libc would come back with it.
The kernel and the C library as a single project Posted Nov 30, 2010 21:12 UTC (Tue) by madscientist (subscriber, #16861) [Link]
"Making libc a kernel module"? Was that suggested somewhere? I don't even know what that means.
The kernel and the C library as a single project Posted Nov 30, 2010 22:27 UTC (Tue) by sfeam (subscriber, #2841) [Link]
It was suggested in an earlier response to this article (by pj). But yeah, I don't know exactly what that would mean either.
Libc as a module ... Posted Dec 1, 2010 12:06 UTC (Wed) by AnswerGuy (guest, #1256) [Link]
Concievably one could use a design similar to the VDSO "linuxgate" which was part of the Linux threads NG support a few years ago.
A default might simply implement the same mmap/linkage semantics that are in use already. However, any specific kernel image could over-ride that behavior.
I'm not advocating such an approach. I'm merely speculating on what someone might reasonably mean by saying such things.
The kernel and the C library as a single project Posted Dec 3, 2010 19:34 UTC (Fri) by Spudd86 (guest, #51683) [Link]
Well that wasn't the suggestion, it the suggestion was basically make linuxgate.so into libc. (OR at least make libc work like linuxgate.so, probably making it relocatable, unlike linuxgate)
The kernel and the C library as a single project Posted Dec 1, 2010 16:27 UTC (Wed) by nix (subscriber, #2304) [Link]
Uh, you can't make libc a kernel module. It is both a userspace program and contains a component (ld-linux.so.2, the dynamic loader) whose absolute filename is wired into every dynamically linked binary on the entire system.
I suppose you could perhaps have a scheme where the kernel reads a PT_INTERP of '/lib/ld-linux.so.2' and looks for /lib/lds/{kernel version}/ld-linux.so.2 first, but I suspect this would have trouble getting upstream, because one extra path lookup per exec() is quite a cost.
Pity Linux doesn't support virtual files or filesystem Posted Dec 1, 2010 19:07 UTC (Wed) by kmself (subscriber, #11565) [Link]
Oh wait, scratch that.
/proc, /sys, /lib-ld-linux.so.2, /usr/games/nethack ....
Actually, since /lib/ld-linux.so.2 is already a symlink, it'd be even easier.
ls -l /dev/std{in,out,err} f'rex.
Pity Linux doesn't support virtual files or filesystem Posted Dec 2, 2010 14:39 UTC (Thu) by nix (subscriber, #2304) [Link]
Yeah, that would work. Kinda weird to have your libc provided by the kernel but I don't see a fundamental problem with it... except that the machinery in the kernel to export things like the vdso to userspace would need a *lot* of retooling to be able to export things as big as a C library.
Pity Linux doesn't support virtual files or filesystem Posted Dec 3, 2010 5:09 UTC (Fri) by jzbiciak (✭ supporter ✭, #5246) [Link]
Could it be built / linked to a kernel similarly to an initrd? The main thing you'd lose is the ability to page it out, I imagine...
Pity Linux doesn't support virtual files or filesystem Posted Dec 4, 2010 23:05 UTC (Sat) by nix (subscriber, #2304) [Link]
ld-linux.so.2 is never really paged out anyway. It's only a dozen pages long and all of it is in use by *something* nearly all the time.
The kernel and the C library as a single project Posted Nov 30, 2010 22:51 UTC (Tue) by gerdesj (subscriber, #5446) [Link]
I don't know what sort of distro you are using but on Gentoo I have upgraded glibc a huge number of times with no problems on countless (x) machines.
{x < "have to take someone else's shoes off to count"}
I also have had no problems with a non trivial number of RedHat, CentOS, Man{drake|driva}, OpenSuSE, SLES/SLED and others.
Perhaps your distro's dependency checker or your use of it is at fault.
Cheers
The kernel and the C library as a single project Posted Nov 30, 2010 23:15 UTC (Tue) by madscientist (subscriber, #16861) [Link]
Most likely nix just has a longer memory than you :-). I, too, remember when upgrading libc was a major cause of hair lossage and nail-biting.
But, I've not had any problems in probably 10 years or so... and I never shy away from an upgrade when it's offered to me. Modern distros definitely do this right.
The kernel and the C library as a single project Posted Dec 1, 2010 9:44 UTC (Wed) by cate (subscriber, #1359) [Link]
glibc learned how to make stable ABI, so now (last 10-12 years) upgrades are easy. IIRC the big improvement was to reserve extra space for structures, to allow library improvement without breaking ABI. (but wasting some more memory)
Probably also the c compiler is more conservative between ABI and optimisation. Actually c++ ABI is not so stable, but this cause less problem: init will anyway start, as well the essential things, thus allowing restoring systems.
BTW: libc upgrades was a nightmare because LILO cannot boot an old library, and live CD were non existent.
The kernel and the C library as a single project Posted Dec 1, 2010 16:30 UTC (Wed) by nix (subscriber, #2304) [Link]
The worry here is not 'the ABI breaks'. The worry here is 'it compiled, but if you run it it is too buggy to run significant programs without crashing'. Anything from miscompilation to buggy toolchains to general build environment problems to installation problems to a buggy glibc can do this.
The reason why glibc works when distros package it is because they take a lot of care, because they run 'make check', and because they know from experience which longstanding test failures indicate real problems. 'make check' takes, oh, half an hour on a fast Nehalem. This is not something that would add only a small amount of time to a kernel build!
The kernel and the C library as a single project Posted Dec 3, 2010 21:07 UTC (Fri) by Darkmere (subscriber, #53695) [Link]
A few Gentoo users got bit by that by trying out the gold linker. When you end up with a 64bit linker that dumps core when linking something major, like gcc, you have a somewhat usable but ultimately badly broken system.
So yeah, infrastructure upgrades like bash, glibc, gcc, binutils, python make us all _very very_ concerned. It may not be noticable at first, but if it causes bad code generation or breaks on non-trivial things, you end up with a nightmare to recover.
The kernel and the C library as a single project Posted Dec 4, 2010 19:38 UTC (Sat) by nix (subscriber, #2304) [Link]
Indeed Fedora just got hit with a mass rebuild due to just such a codegen bug in GCC's 4.5 branch (never released, but Fedora update periodically to tip-of-branch rather than using releases, so they were hit, and had to rebuild everything built with the buggy compiler).
The kernel and the C library as a single project Posted Dec 1, 2010 11:17 UTC (Wed) by NAR (subscriber, #1313) [Link]
Hmm, I think it wasn't that long ago when glibc broke (OK, exposed an already exiting bug in) flash on Fedora...
The kernel and the C library as a single project Posted Dec 1, 2010 14:07 UTC (Wed) by madscientist (subscriber, #16861) [Link]
Meh. A non-working flash definitely does not qualify for my definition of hair pulling or nail-biting. Does your system boot without having to get out a boot disk, mount your root partition, and do major surgery? Then you're good.
The kernel and the C library as a single project Posted Dec 3, 2010 9:47 UTC (Fri) by job (guest, #670) [Link]
Indeed.
Another thing that causes hair pulling but should be fairly straightforward is executable file format migrations. _Every_time_ that has caused problems.
;-)
The kernel and the C library as a single project Posted Nov 30, 2010 20:46 UTC (Tue) by pj (subscriber, #4506) [Link]
How about some middle ground... making syscall() less fiddly and more architecture-portable somehow (export a map of name -> syscall numbers into /sys/kernel somewhere, for instance) so that it was an actual option might be enough.
Pulling all of libc into the kernel strikes me as... bad. Compare the size of glibc to uclibc or eglibc ... the mere existence of all of them means to me that options are desirable.
So... maybe a kernel module? To try out the idea, at least. Ship it with a stub userland libc to link against, or patch an existing libc to detect it and use it if found.
We had an http server kernel module for some years (is that still maintained, even out of tree?), why not a libc one?
The kernel and the C library as a single project Posted Nov 30, 2010 21:06 UTC (Tue) by signbit (subscriber, #71372) [Link]
I won't be libc in the kernel runtime but libc in the kernel source code repository. It will still go in /lib/libc.so.$WHATEVER.
What happened to klibc? Wasn't that supposed to stay close and true to the kernel?
The kernel and the C library as a single project Posted Nov 30, 2010 21:31 UTC (Tue) by johill (subscriber, #25196) [Link]
libc could be a VDSO I guess ...
The kernel and the C library as a single project Posted Dec 1, 2010 8:14 UTC (Wed) by smueller@atsec.com (guest, #68260) [Link]
Considering how vDSO is implemented, why not exporting all system calls as stubs in the vDSO part? This way, user space could simply link to that vDSO AND use the system calls like a normal API without going through the oddities of syscall()?
klibc is direct opposite Posted Nov 30, 2010 22:27 UTC (Tue) by khim (subscriber, #9252) [Link] Klibc is alive and well - but it's direct opposite to what's offered! Klibc is minimalistic libc - enough to run initscripts, certainly now enough to run heavy-duty apps! It offers access to very few syscalls (you don't need much to insmod some modules and create few /dev files) and does not follow "latest and greatest" changes in kernel at all.
The kernel and the C library as a single project Posted Dec 1, 2010 6:14 UTC (Wed) by jamesh (guest, #1159) [Link]
The quote from Ingo talks of making it a VDSO. That is, code mapped into the address space of every new process by the kernel, rather than libraries loaded from the file system by the dynamic linker.
License? Posted Nov 30, 2010 20:57 UTC (Tue) by kripkenstein (subscriber, #43281) [Link]
I'm not sure how this can work in terms of license.
The kernel is GPL2, and userspace code is not seen as being a derivative work. So libc can have whatever license it wants. But if the two are worked on as a single project, presumably that means libc would need to be GPL2 as well.
For one thing, that limits the libcs that can be used here. I believe LGPL code can be relicensed as GPL, so that might work for an LGPL libc, but only if the libc is not GPL3+ (not sure what GNU libc is). I guess a permissively licensed libc (e.g. newlib) would be ok.
However, the bigger matter is what this would mean for other userspace code - can they link against a GPLed libc if they are not GPLed? That was the whole reason for using the LGPL instead of the GPL for GNU libc, I believe?
License? Posted Nov 30, 2010 21:09 UTC (Tue) by madscientist (subscriber, #16861) [Link]
I don't think this is an issue. The only thing that matters from a licensing perspective is whether the libc is legally a derived work of the kernel (or vice versa). If they are not currently derived works being distributed on separate sites, then I don't see how changing them to be distributed on the same site causes them to become derived works. The interface between the kernel and libc will continue to be the syscall interface + header files, just as today.
Of course we cannot have a libc which is licensed under the GPL. It must be the LGPL, or else something weaker. Otherwise no one will ever use it.
License? Posted Nov 30, 2010 22:11 UTC (Tue) by gmaxwell (subscriber, #30048) [Link]
"The only thing that matters from a licensing perspective is whether the libc is legally a derived work of the kernel (or vice versa)"
Your message appears to embody a commonly repeated piece of misinformation about how copyleft licenses work. I'm responding in order to squelch its continued dissemination, not because I disagree with your conclusion.
"Is foo a legally a derived work of Bar" is almost never a important question with respect to license compliance of bundled software.
The relevant software license sets out conditions under which is can be lawfully reproduced. Absent the license you can not lawfully reproduce the software. A license for Bar might specify that you can only reproduce the software on a computer that doesn't contain a copy of nethack and the requirement would be perfectly enforceable. Not become nethack is somehow a derivative of Bar and is thus tainted by bar's license but simply because the license of bar says so, and you're distributing bar so you must abide by the license or infringe. If you don't like it then you can opt not to distribute Bar by doing so escape its requirements.
The legal criteria for how copyright encumbrances flow from place to place are relevant in some contexts, but usually not in the context of bundling because no 'flowing' is required: When you bundle you must obey _ALL_ relevant licenses.
The GPLv3 carefully avoids any mention of the word "derivative" in order to avoid this confusion with other uses of the heavily overloaded term.
So in the case kernel and libc you have to actually look at what the licenses require. You can't just depend on some extenal notion of a legal definition of a derived work to somehow bound things, because it wouldn't. Considering that the distributions bundle these things already I think it's safe to say that there currently isn't a problem. It may more complicated as each of the packages is intimately extended with code existing in the other.
Yup. You just omitted important fact. Posted Nov 30, 2010 22:47 UTC (Tue) by khim (subscriber, #9252) [Link] The legal criteria for how copyright encumbrances flow from place to place are relevant in some contexts, but usually not in the context of bundling because no 'flowing' is required: When you bundle you must obey _ALL_ relevant licenses. Yup. This is correct. But it's important to continue you explanation: the words cited above would imply that even today you can not actually distribute kernel and non-GPLed programs... yet people do it with impurity. Why it's not a problem today? This is important stuff. Considering that the distributions bundle these things already I think it's safe to say that there currently isn't a problem. Yup, but again: why? It's important. It may more complicated as each of the packages is intimately extended with code existing in the other. Not "more compilcated". Impossible. The reason why you can distribute kernel with GLibC, newlib or Android's bionic is simple: "mere aggregation" clause. All version of GPL include this "escape clause": if you are distributing GPLed product and non-GPLed product on the same CD or on the same NAND chip it's Ok as long and GPLed product and non-GPLed product are developed separately and not tied to each other. Today it's easy: Linux kernel exposes simple, documented interface, there are multiple independent implementations of it on both consumer and producer side (on the consumer side there are GLibC, uClibc, klibc, etc - and on producer side there are at least Linux kernel and emulation of said kernel in FreeBSD). It's additionally reinforced by explicit promose in kernel's COPYING file (NOTE! This copyright does *not* cover user programs that use kernel services by normal system calls). So everything is great. But if libc will be developed in tandem with kernel... the "mere aggreggation" defense evaporates and "user programs that use kernel services by normal system calls" defense evaporates too! At this point there are no separation line which will allow you to redistribute non-GPLed programs! No, the full libc distributed with kernel idea will not fly - and AFAICS this is not what is discussed at all.
Yup. You just omitted important fact. Posted Nov 30, 2010 23:12 UTC (Tue) by madscientist (subscriber, #16861) [Link]
> But if libc will be developed in tandem with kernel... the "mere
> aggreggation" defense evaporates
I don't believe you're correct here.
First, the "mere aggregation" clause is not a "get out of jail free" card, where any cross-derivation of two different pieces of software magically becomes just fine if they're aggregated together on a single CD. If two works are legally derived, then you have to deal with the GPL, REGARDLESS of whether they're aggregated or not. The "mere aggregation" clause just makes crystal clear something that is probably already true anyway: that bundling two completely unconnected things together on a single media does not magically make them legally derived from each other.
Second, the reason glibc, etc. and the kernel are allowed to use different licenses is not the "merge aggregation" clause. These are not considered to be derived works of the kernel and so they don't have to take the kernel's license into account.
So whether libc is developed by the kernel developers or by GNU, and how they are bundled (or not) together, is not relevant in terms of licensing. All that matters is derivation (without derivation there is no way to enforce) and the content of the license.
"Mere aggregation" is there for a reason. Posted Dec 1, 2010 1:01 UTC (Wed) by khim (subscriber, #9252) [Link] The "mere aggregation" clause just makes crystal clear something that is probably already true anyway: that bundling two completely unconnected things together on a single media does not magically make them legally derived from each other. To bind two completely unconnected things together on a single media you still need explicit permissions from both copyright owners. And if one hates the guts of other... well, that's it: you can not distribute these two things on the same media. People from Be, Inc tested your insane theory - and failed (with dire consequences). Copyright owners can and often do forbid to distribute things together under same cover. People are often surprised to hear that, but think about it: can you put Linux kernel and Adobe Flash player on the same media? No! But why "no"? Because standard Adobe Flash license forbids that. You can buy separate redistribution license - but this is different story (and said redistribution license often does impose limitations over what can you put on the same CD). If Adobe's license can allow you to run and use program but will stop the redistribution then why GPL can not do the same? It can and it does - unless you'll use GPL for the whole "work" you plan to redistribute or use "mere aggregation" escape. Second, the reason glibc, etc. and the kernel are allowed to use different licenses is not the "merge aggregation" clause. Ok, I'll bite. What other clause gives you this right? GPL extends to "any work based on the Program" - and of course compilation of kernel and some other library/program is "any work based on the Program". If you'll forget about "mere aggregation" clause, that is. These are not considered to be derived works of the kernel and so they don't have to take the kernel's license into account. If they are distributed separately? Sure. When they are combined to form bigger, larger, work like Android or Ubuntu? No. It's like when Alice hates Bob's guts (they had messy divorce or something): if you are lucky you can get a permission to publish a book of stories from two under some cover - buf if they'll demand to never put two such stories on one page... you must obey. Or not publish said book at all. All that matters is derivation (without derivation there is no way to enforce) and the content of the license. Right. So we need to check if Android or Ubuntu are derived work of kernel and if the license of kernel cover the whole "work". First part is simple: of course Android and Ubuntu are derived works! They quite literally include millions of bytes of code from kernel! Second part is equally clear (if you ignore "mere aggregation" clause): you can only ever redistribute "any work based on the Program" if you'll distribute the whole compilation under GPL terms.
"Mere aggregation" is there for a reason. Posted Dec 1, 2010 1:23 UTC (Wed) by sfeam (subscriber, #2841) [Link] you can not distribute these two things on the same media. People from Be, Inc tested your insane theory - and failed (with dire consequences).I may be misremembering, but the article you link to reinforces my recollection that BeOS ran up against the bulk licensing agreements between Microsoft and OEMs, which forbade to sell dual-boot machines. However unfortunate that may have been, it doesn't directly bear on the question of bundling linux software on distribution media.
Oh, but it does... Posted Dec 1, 2010 3:36 UTC (Wed) by khim (subscriber, #9252) [Link] I may be misremembering, but the article you link to reinforces my recollection that BeOS ran up against the bulk licensing agreements between Microsoft and OEMs, which forbade to sell dual-boot machines. Yup. However unfortunate that may have been, it doesn't directly bear on the question of bundling linux software on distribution media. How come? Microsoft's Windows license given for OEMs made it illegal to distribute totally unrelated piece of code on the same medium unless it was licensed in some quite specific way - the same as with Linux kernel and linux programs. In case of Windows license only allows programs written for Windows but not separate OSes, in case of Linux license allows not only Linux programs but also any "mere aggregation" software. If Microsoft's license can stop BeOS inclusion then Linux license can stop any other software inclusion. You can only redistribute proprietary programs (and FOSS programs with GPL-incompatible licenses) with Linux because GPL gave you explicit permission. Copyright alone does not give you such rights.
Oh, but it does... Posted Dec 1, 2010 4:22 UTC (Wed) by sfeam (subscriber, #2841) [Link] Microsoft's Windows license given for OEMs made it illegal to distribute totally unrelated piece of code on the same medium unless it was licensed in some quite specific wayThat's not what it says in the article you linked to. As I understand it, the OEM's were allowed to ship BeOS in addition to Windows (or at any rate they thought they were and continued to do so). But in order to get the bulk pricing on Windows they could not enable dual boot - they instead resorted to offering instructions on how to switch your disk to booting BeOS (only) rather than Windows (only). So from that description, the limitation was not "the copyright or license forbids joint distribution"; it was "we will charge you more if you do so". And even so it wasn't a direct restriction on distribution, only on the machine configuration. So again, I really don't think the BeOS story is a close enough parallel to be relevant. Anyhow, back to your earlier diagnosis that if some clib equivalent were distributed with the kernel there would no longer be a dividing line across which the license terms did not apply. That makes no sense to me. The line would be drawn at the API between clib and user programs. Not much difference to what we have now.
Oh, but it does... Posted Dec 1, 2010 11:54 UTC (Wed) by tialaramex (subscriber, #21167) [Link]
And Microsoft's position here even makes sense, even if it was in practice anti-competitive and therefore perhaps unacceptable.
Microsoft believes (correctly) that most of its users don't understand much about computers. If you ask them "Which OS are you booting?" the _better_ answers will be something like "Microsoft" or "Office" and most people will just stare uncomprehending.
OEMs have consistently choked systems with shovelware because doing so is profitable and the margins in the PC world are beyond razor thin. So Microsoft's OEM license includes limits on shovelware, one of them ensured that when you buy a Windows PC, it boots Windows.
Because the alternative is it boots "AdBrowse - the great new way to earn valuable prizes while using the Internet" for which the OEM was paid $$$. And only a tiny fraction of users know to hold down Shift F8, pick "Windows" from the menu and boot the actual OS instead. Everybody else just complains that now "the Microsoft" makes them watch adverts, just as they queued up to complain that "the Facebook" had changed when they typed "facebook login" into a browser and got some blog page not facebook.com
More recently we've seen what happens when Microsoft's OEM negotiators let the OEMs bully them - that's what led to the Vista debacle. Every top-selling PC in Vista's opening season was unsuitable to run Vista, but OEMs didn't want these PCs gathering dust as "XP only" so they moaned until Microsoft authorised Vista badges of some kind for these machines.
Oh, but it does... Posted Dec 2, 2010 0:57 UTC (Thu) by cmccabe (subscriber, #60281) [Link]
> And Microsoft's position here even makes sense, even if it was in practice
> anti-competitive and therefore perhaps unacceptable.
OEMs continue to choke systems with shovelware. Microsoft doesn't give a hoot as long as they get their cut. In fact, Microsoft has made the situation considerably worse by refusing to give out Windows install discs along with new computers. It used to be that you could at least re-install a clean version of Windows to get rid of the adware and spyware. Now, all they give you is essentially a restore-from-ISO tool that has all the crap on it.
The restriction on dual-boot computers was never about helping users; it was always about killing the competition. Enforcing trademark law would have been enough to ensure that users didn't blame Windows for Linux's (or BeOS's) failings.
> More recently we've seen what happens when Microsoft's OEM negotiators let
Bad project management led to the Vista debacle. If your new operating system can't run on the newest hardware that's coming off the production lines, maybe you ought to delay shipping it until it can.
Don't get me wrong-- there's a lot of things that Microsoft has done well (Xbox comes to mind). But Vista wasn't one of them.
Oh, but it does... Posted Dec 2, 2010 1:51 UTC (Thu) by tialaramex (subscriber, #21167) [Link]
The restore situation I can explain (the quality of the restore though, you should blame on your OEM).
Despite everybody's best efforts, none of today's PC operating systems actually work properly on a brand new PC. In say, Fedora this can be painfully obvious as your brand new HP laptop runs X with VESA graphics or refuses to acknowledge the existence of its built-in touchpad. Sure there'll be an update that fixes it, but that's later, and you've got the new laptop now.
In Windows this is hidden by pre-installing the necessary patches, new drivers, utilities etc. at the factory.
But if the user re-installs from a "clean" OS DVD they don't get any of that, and then end up with the experience familiar to bleeding edge Linux people. They will, of course, report this as a fault, which will cost someone a bunch of support money. Hence, there is no way to install a clean OS, because they know it won't work properly anyway.
You mention trademark law as a solution to the dual boot problem. This is charmingly naive - to sophisticates like us it's obvious if you're running OS/2 or Mac OS X, but to the average user it's all a muddle. My sister continues to be caught out by the need to buy different software for her "new computer" (a MacBook). Trademarks don't help because they're not aware of the trademarks, it's just more technical mumbo-jumbo.
Oh, but it does... Posted Dec 2, 2010 3:42 UTC (Thu) by cmccabe (subscriber, #60281) [Link]
> Despite everybody's best efforts, none of today's PC operating systems
> actually work properly on a brand new PC. In say, Fedora this can be > painfully obvious as your brand new HP laptop runs X with VESA graphics or > refuses to acknowledge the existence of its built-in touchpad. Sure > there'll be an update that fixes it, but that's later, and you've got the > new laptop now.
It's fine to offer a restore CD with the driver pre-loaded. However, Microsoft should also offer the ability to download a clean version of Windows from its website, if you input the correct product key.
Microsoft has spent a lot of effort tying product keys to specific configurations and models of computer. They created Windows Genuine Advantage and barred pirated copies of the OS from downloading most updates. Offering a copy of the original Windows CD would really not be challenging for them.
> You mention trademark law as a solution to the dual boot problem. This is
It's not "charmingly naive" to believe that users know the difference between a can that says "Coke" and one that says "Pepsi". It's also not naive to believe that users can't tell the difference between a big Apple logo plastered on the startup screen, and on the desktop afterwards, and the same situation with a Windows logo.
Frankly, I find your comments to be elitist. Just because a person is not a computer professional, doesn't make him stupid. There is a difference between being stupid and choosing not to invest time and energy in something.
Oh, but it does... Posted Dec 2, 2010 12:08 UTC (Thu) by tialaramex (subscriber, #21167) [Link]
It's not "charmingly naive" to believe that users know the difference between a can that says "Coke" and one that says "Pepsi".
And if that was the comparison we were actually discussing it might be relevant.
There is a difference between being stupid and choosing not to invest time and energy in something.
Indeed, and a great many computer users (which means many millions of Microsoft customers) choose not to invest time and energy into understanding how a computer works. Microsoft has no problem with this, but it does mean they need to ensure that when those users buy a "PC with Windows" they don't get "a PC that could run Windows, but right now it's booted into BeOS". Because Microsoft (not Be Inc, not the guy on Slashdot promoting BeOS) ends up with a lot of the support costs if that happens.
If it's "elitist" to believe that some people don't care about computers then I guess I'm an "elitist" by your definition, but I caution you that this is an unusual definition which is likely to cause you problems.
Oh, but it does... Posted Dec 19, 2010 11:37 UTC (Sun) by Jan_Zerebecki (guest, #70319) [Link]
AFAIK Microsoft does not offer free support with normal OEM versions, I would guess they make money with all their support offers, that need work per support incident, i.e. not including things like Documentation. (I just looked it up that a support Question for Microsoft Windows 7 Professional N costs 72.)
Oh, but it does... Posted Dec 19, 2010 16:22 UTC (Sun) by Trelane (subscriber, #56877) [Link] Absolutely right. There are a large number of people in violation of the OEM licensing terms. Read the FAQ for yourself: Some highlights:Q. My customer wants to purchase a "naked" PC from me and acquire the Windows license through a Volume Licensing agreement. Is this OK? A. No. Full Windows licenses are not available through any Microsoft Volume Licensing program, including academic volume licenses. The customer must first acquire a Windows operating system license via OEM software included with a new PC from an OEM or system builder, or via the retail channel. Q. Can my customers transfer or sell their OEM software licenses? A. After an OEM software license has been installed on a PC, the license may not be installed on or transferred to another PC. However, the entire PC may be transferred to another end user along with the software license rights. When transferring the PC to the new end user, the software media, manuals (if applicable), and Certificate of Authenticity label must be included. It is also advisable to include the original purchase invoice or receipt. The original end user cannot keep any copies of the software. Q. Can two or more users access and fully utilize OEM Windows operating systems concurrently on the same machine? A. No. The End User Software License Terms do not permit two or more users to concurrently use the full feature sets of Windows operating systems. However, the Windows End User Software License Terms do allow for a limited number of computers or other electronic devices to connect to the computer upon which the software is installed to utilize one or more of the following services: File Services, Print Services, Internet Information Services, and telephony services. The End User Software License Terms also permit limited concurrent use in connection with the Remote Assistance and NetMeeting technologies. Please refer to the applicable End User Software License Terms for detailed information regarding such limited concurrent uses. Q. How does a company qualify to become a direct Microsoft OEM? It seems that the larger companies currently have an unfair advantage compared with smaller OEMs. A. Direct OEM licensees do receive a discount compared to buying through the System Builder channel, but that discount is based on the licensees commitment to receive ongoing bulk shipments versus purchasing at will. Other elements of the direct licensing agreement require significant initial investment from the OEM. Furthermore, legal and technical requirements are placed on direct OEMs to protect Microsoft intellectual property, and these requirements can add other costs to the production of a PC. The primary difference between the two programs cannot be gauged merely by looking at prices and software licenses. Each program is designed to meet the specific needs of the partner. Q. Can a PC with an OEM Windows operating system have its motherboard upgraded and keep the same license? What if it was replaced because it was defective? A. Generally, an end user can upgrade or replace all of the hardware components on a computerexcept the motherboardand still retain the license for the original Microsoft OEM operating system software. If the motherboard is upgraded or replaced for reasons other than a defect, then a new computer has been created. Microsoft OEM operating system software cannot be transferred to the new computer, and the license of new operating system software is required. If the motherboard is replaced because it is defective, you do not need to acquire a new operating system license for the PC as long as the replacement motherboard is the same make/model or the same manufacturer's replacement/equivalent, as defined by the manufacturer's warranty. The reason for this licensing rule primarily relates to the End User Software License Terms and the support of the software covered by that End User Software License Terms. The End User Software License Terms is a set of usage rights granted to the end user by the PC manufacturer and relates only to rights for that software as installed on that particular PC. The system builder is required to support the software on the original PC. Understanding that end users, over time, upgrade their PCs with different components, Microsoft needed to have one base component "left standing" that would still define the original PC. Since the motherboard contains the CPU and is the "heart and soul" of the PC, when the motherboard is replaced (for reasons other than defect) a new PC is essentially created. The original system builder did not manufacture this new PC, and therefore cannot be expected to support it. Q. Can I create my own recovery disks and sell these with the computer systems that I build? I have heard that direct OEMs can do this, so why can't I? A. No. System builders may not offer a recovery solution with removable media (e.g., a recovery CD) because it is prohibited by the terms of the Microsoft OEM System Builder License. A full version of the Windows operating system is provided on a hologram CD in the Microsoft System Builder pack for each end user, and the CD must be transferred to the end user at the time of distribution. The hologram CD acts as the recovery media. However, system builders can offer a hard disk recovery solution in addition to, but not as a replacement for, the hologram CD. Third-party software companies can also help system builders do this. Learn more about the legal, licensing, and technical requirements for this type of hard disk-based recovery solution. System builders are bound by the Microsoft OEM System Builder License, affixed to the side of the System Builder packs, which is different than the direct agreements utilized by direct OEMs. The licensing terms for system builders and large OEMs are different because they are designed to address the specific needs of each community. The right to create recovery media is limited to the OEMs with direct agreements; however, these OEMs are also bound by other contractual obligations. The OEM System Builder License is designed to make it easy for system builders to acquire and distribute genuine Microsoft software, and accordingly, its terms are different. Q. Can I provide a computer system to my customer without an operating system (also referred to as a "naked PC")? A. Yes. There is nothing illegal about selling a computer system without an operating system. However, getting the operating system preinstalled is your customer's most cost-effective way to acquire a genuine Windows operating system license. A customer who subsequently wants to install a Microsoft Windows desktop operating system on that naked PC will need to acquire it through the retail (full packaged product) channel which is a more costly option. Full Windows operating systems are not available through any Microsoft Volume Licensing program, and an OEM operating system license cannot be transferred from an "old" PC to a new one. Q. What are the different ways in which Microsoft OEM System Builder Windows desktop operating system licenses can be distributed? A. The current OEM System Builder License allows system builders to distribute Windows desktop operating system licenses in the following ways: 1. Preinstalled on a new PC. 2. Unopened OEM System Builder packs (1-, 3-, or 30-packs) can be distributed to other system builders by themselves. Note that they must remain unopened so the receiving system builder can accept and be bound by the break-the-seal license agreement that is affixed to the pack.Additional information "For hobbyists": OEM System Builder Software[...]Must be preinstalled on a PC and sold to another unrelated party.(commonly misunderstood part bolded and partially italicized by me) OEM System Builder Software[...]System builder that preinstalled the software must provide support for the software. Q. I would like to build PCs for my company and use OEM System Builder software for the operating system. Can I do this? A. OEM System Builder software must be preinstalled and then resold to another party. If you are using the PC within your organization, this "resale" requirement will not be met. In addition, as a system builder preinstalling OEM System Builder software onto new PCs, this requires that you grant the end user license terms to the third party acquiring the PCs from you. If you are distributing the PCs within your organization, you cant grant the end user license terms to yourself.Volume licensing FAQ is also highly recommended: http://www.microsoft.com/licensing/resources/faq.mspx
"Mere aggregation" is there for a reason. Posted Dec 1, 2010 2:51 UTC (Wed) by madscientist (subscriber, #16861) [Link]
> To bind two completely unconnected things together on a single media you
> still need explicit permissions from both copyright owners.
Of course, to redistribute something you need permission because copyright restricts redistribution. And of course a license that grants you permission to redistribute one way MAY restrict redistribution in other ways: this is not uncommon even. As you point out, many licenses such as Adobe's allow you to use the software only if you obtain it directly from Adobe; you cannot give the software to someone else. THAT is why you cannot get a CD with both Linux and Flash bundled on it; it has NOTHING to do with the GPL. You can't distribute a CD that contains nothing but Flash, even... unless you've executed a different license agreement with Adobe.
I guess we can disagree about whether the "mere aggregation" clause is necessary; my belief is that it's there merely as a clarification and that even without it the GPL implicitly allows aggregation. Maybe not. Either way it's moot because the clause DOES exist and so both the GPL and LGPL make explicit that aggregation is allowed.
Your argument was that because the kernel and libc would be developed together, somehow they would not be able to take advantage of the "mere aggregation" clause any longer. In order for that to be the case, libc would have to become a "work based on the Program" (e.g., the kernel); as long as it's not a work based on the Program, it can still take advantage of "mere aggregation". Whether or not libc is a "work based on the Program" depends on whether it is a legally derived work or not; it has nothing to do with the development model.
In other words, as I mentioned before, if libc developed by GNU is not considered to be a derived work of the kernel (and hence can take advantage of "mere aggregation"), then why would libc developed by kernel developers be a derived work (and hence NOT be able to take advantage of "mere aggregation")?
"Mere aggregation" is there for a reason. Posted Dec 1, 2010 3:08 UTC (Wed) by Trelane (subscriber, #56877) [Link] Of course, to redistribute something you need permission because copyright restricts redistribution.No, copyright restricts, erm, copying. Hence the name.
"Mere aggregation" is there for a reason. Posted Dec 1, 2010 3:14 UTC (Wed) by Trelane (subscriber, #56877) [Link] Ha. And, of course, I'm wrong. Quoth the Copyright Office (http://www.copyright.gov/circs/circ1.pdf)Section 106 of the 1976 Copyright Act generally gives the owner of copyright the exclusive right to do and to authorize others to do the following:
"Mere aggregation" is there for a reason. Posted Dec 2, 2010 1:49 UTC (Thu) by nybble41 (subscriber, #55106) [Link]
That's true of section 106 the original Copyright Act, but section 109 ("Limitations on exclusive rights") and the Doctrine of First Sale pretty much nullify the third point in the case of simple redistribution or public exhibition of existing copies--apart from sound recordings or software, which were (idiotically) excluded from the limitations in section 109(a) by 109(b).
Copy, right? Posted Dec 6, 2010 0:24 UTC (Mon) by marcH (subscriber, #57642) [Link]
> Ha. And, of course, I'm wrong.
Err... you mean, you are right?
Every bullet point you quote is about reproducing the original work in one way or the other. That does not seem to stretch the meaning of "copy" a lot, or does it?
"Mere aggregation" is there for a reason. Posted Dec 1, 2010 10:07 UTC (Wed) by PO8 (guest, #41661) [Link]
The term "copyright" originally was used to describe rights upon "copy" in the older sense of written material. The term itself had nothing explicit to do with "copying". Of course, it's easy to see where the confusion could come from here.
"Mere aggregation" is there for a reason. Posted Dec 1, 2010 14:01 UTC (Wed) by marcH (subscriber, #57642) [Link]
> > These are not considered to be derived works of the kernel and so they don't have to take the kernel's license into account.
> If they are distributed separately? Sure. When they are combined to form bigger, larger, work like Android or Ubuntu? No.
You are right but *in the case of the GPL* it does not make much difference since the GPL does not care about mere aggregation.
> It's like when Alice hates Bob's guts (they had messy divorce or something)
The GPL is all love!
> First part is simple: of course Android and Ubuntu are derived works! They quite literally include millions of bytes of code from kernel!
Ubuntu has no licence but a large *collection* of them, so I really doubt such an example will help make anything clearer.
Otherwise nice post, thanks.
"Mere aggregation" is there for a reason. Posted Dec 2, 2010 18:25 UTC (Thu) by vonbrand (subscriber, #4458) [Link] There must be a license (i.e., a permission to do something you normally aren't allowed to do) on the collection as such, else you can't redistribute it either... Red Hat (the collection) used to go under GPL, I suppose Fedora does the same (plus trademark restrictions). Ubuntu does restrict (via trademark) what derivative can (and can't) be called Ubuntu.
"Mere aggregation" is there for a reason. Posted Dec 2, 2010 18:52 UTC (Thu) by sfeam (subscriber, #2841) [Link]
It is possible to claim copyright on a compilation (== "collection"), yes. But I am not aware of any requirement to do so. If there is no additional license/copyright on the compilation as a whole, then redistribution is only limited by the terms of the individual items within the collection.
In fact you can only claim additional copyright on the collection if your preparation of the compiled materials involves sufficient additional work to constitute "an original work of authorship" in its own right. Just stuffing a bunch of existing packages on a disk would be unlikely to merit a new copyright as a compilation. Preparing a full linux distro's worth of packages selected and possibly modified for interoperability, together with meta-information, installation scripts, yadda, yadda, is clearly a different story.
"Mere aggregation" is there for a reason. Posted Dec 2, 2010 1:10 UTC (Thu) by cmccabe (subscriber, #60281) [Link]
> Right. So we need to check if Android or Ubuntu are derived work of kernel
> and if the license of kernel cover the whole "work". First part is simple: > of course Android and Ubuntu are derived works! They quite literally > include millions of bytes of code from kernel! Second part is equally > clear (if you ignore "mere aggregation" clause): you can only ever > redistribute "any work based on the Program" if you'll distribute the > whole compilation under GPL terms.
Android as a whole is, indeed, a derived work of the Linux kernel. But that doesn't mean that all of the component parts of Android are derived works of the Linux kernel. You can, and companies do, make changes to the BSD part of Android and ship them, without making the code publicly available.
Here is another example. If you're a book seller, you can put several books on the shelf-- even ones with the same subject-- that have different copyrights. You do not need to get permission from the authors to do so. Just putting two things side by side does not make one a derived work of the other.
And we haven't even talked about "fair use"-- the quantum physics of copyright law. What a confusing topic!
License? Posted Nov 30, 2010 22:49 UTC (Tue) by madscientist (subscriber, #16861) [Link] I should have been more clear and quoted the parent of my message. The statement was "But if the two are worked on as a single project, presumably that means libc would need to be GPL2 as well."My comment was "The only thing that matters from a licensing perspective is whether the libc is legally a derived work of the kernel (or vice versa)." I meant this in the context of the parent: the only thing that matters when deciding if libc would have to adopt the same license as the kernel is derived work status. I wasn't talking at all about whether it could be distributed, etc.
License? Posted Dec 1, 2010 1:10 UTC (Wed) by cmccabe (subscriber, #60281) [Link]
> Your message appears to embody a commonly repeated piece of misinformation
> about how copyleft licenses work. I'm responding in order to squelch its > continued dissemination, not because I disagree with your conclusion. > > "Is foo a legally a derived work of Bar" is almost never a important > question with respect to license compliance of bundled software. > > The relevant software license sets out conditions under which is can be > lawfully reproduced. Absent the license you can not lawfully reproduce the > software. A license for Bar might specify that you can only reproduce the > software on a computer that doesn't contain a copy of nethack and the > requirement would be perfectly enforceable. Not become nethack is somehow > a derivative of Bar and is thus tainted by bar's license but simply > because the license of bar says so, and you're distributing bar so you > must abide by the license or infringe. If you don't like it then you can > opt not to distribute Bar by doing so escape its requirements.
Did you read this part of the GPLv2?
> In addition, mere aggregation of another work not based on the Program
Since both the kernel and glibc are under the GPLv2 here, your hypothetical considerations about licenses that restrict bundling are irrelevant. The OP is correct.
P.S. Not everything that companies try to throw into EULAs and software licenses is legally binding. I think a judge would probably disagree with you that requiring the user to uninstall an unrelated piece of software to use your software was "perfectly reasonable." Otherwise, Microsoft, IBM, and others would have used this tactic already.
Yes, they do use this tatic. Posted Dec 1, 2010 4:00 UTC (Wed) by khim (subscriber, #9252) [Link] I think a judge would probably disagree with you that requiring the user to uninstall an unrelated piece of software to use your software was "perfectly reasonable." Otherwise, Microsoft, IBM, and others would have used this tactic already. We are not talking about uninstallation. We are talking about bundling. And yes, restrictions here are quite legal and at least Microsoft actually uses this tatic - and judges accept it. It's the same as with book: if you buy two books from authors which hate each other and then you can use scissors and glue to create "private compilation" of their works - noone can prevent this. But if you want to publish such compilation you'll need permission from both authors.
Not aggregation Posted Dec 1, 2010 12:09 UTC (Wed) by tialaramex (subscriber, #21167) [Link]
Again, the Bootloader restriction doesn't say anything about aggregation.
The rule was, if you want OEM discounts for a Windows license the PC must actually boot Windows, using the Windows bootloader as supplied by Microsoft. Pre-install Linux, OpenBSD, BeOS and OS/2 on them if you like, but they must boot Windows, using the Windows bootloader.
Microsoft also separately asserted that OEMs mustn't sell non-Windows PCs if they want the discount. That's a much more obviously anti-competitive rule, and they've been beaten up for it on several occasions. But again it wasn't about aggregation - in this case the OEMs weren't providing Windows (on those PCs) at all.
Yes, they do use this tatic. Posted Dec 2, 2010 0:42 UTC (Thu) by cmccabe (subscriber, #60281) [Link]
> We are not talking about uninstallation. We are talking about bundling.
> And yes, restrictions here are quite legal and at least Microsoft actually > uses this tatic - and judges accept it.
As far as I know, the legality of those contracts between Microsoft and the OEMs was never tested in court. So you can't say "judges accept it," because they were never asked.
It would be incredibly self-destructive for any OEM to sue Microsoft over this matter, because Microsoft has the ability to make or break them by offering or witholding discounts on Windows. To put it another way, Microsoft is the sole supplier for an essential part-- the Windows license.
> It's the same as with book: if you buy two books from authors which hate
That's a clear, obvious case of creating a derived work. It has nothing to do with bundling.
On a somewhat related note, I know that Microsoft claims (in their EULA) that you can't run certain versions of Windows inside a virtual machine. I'd really like to see the legality of that restriction tested in court.
If it's legal for them to enforce this restriction, then logically it should be legal for any program to do the same-- so we'd end up with a bunch of programs charging extra for the "prviliege" of using them inside a virtual machine.
License? Posted Dec 1, 2010 9:07 UTC (Wed) by lethal (subscriber, #36121) [Link]
The linking clauses don't really apply in the context of a vDSO. The vDSO is mapped in to every process space by the kernel, and can be called by anybody. Processes as such have no direct link map knowledge of the vDSO mapping, and must in turn grovel the auxvt for the entry point (especially as these days the page itself is stashed in a vma in order to randomize its location). One can also interrogate the memory map for the running process through procfs to find the mapping. In any event, restricting use of the vDSO would be akin to restricting visibility of system calls to userspace processes depending on licensing, which we don't really do today either. Where the licensing comes in to play is for libraries that encapsulate the system calls that applications in turn link against. If for example your inotify library contains licensing you don't care for, nothing stops you from constructing your own syscall() call-in that stubs in the bits you care about and never touch the library at all. If someone were to create a library (or extend one, in the case of glibc) to wrap in to the vDSO, then those wrappers within that context would be subjected to the licensing of the library, but the vDSO remains wholly insular.
Obeying licenses Posted Dec 5, 2010 18:48 UTC (Sun) by giraffedata (subscriber, #1954) [Link] When you bundle you must obey _ALL_ relevant licenses. A license isn't something you obey. It doesn't command you to do anything. Copyright law is something you have to obey. When you bundle, you must meet the conditions of all relevant licenses (i.e. the licenses you use).
License? Posted Nov 30, 2010 22:47 UTC (Tue) by marcH (subscriber, #57642) [Link]
> Of course we cannot have a libc which is licensed under the GPL. It must be the LGPL, or else something weaker. Otherwise no one will ever use it.
Damn, this would even prevent anyone from writing any POSIX program at all!
License? Posted Dec 1, 2010 3:06 UTC (Wed) by vonbrand (subscriber, #4458) [Link] This is all nonsense, IMVHO. The author(s) decide which licenses each single source file carries. If a bunch of "libc" files are distributed together with "linux" files, each maintains its own license (just look over the files in the kernel, some are specifically under BSD so they can be shared with other systems). Sure, it can get messy if Alice takes a file written by Bob for "linux" and placed under GPLv2, on which Charlie to Zoe then hacked over the years, and wants to reuse part of it and place it under LGPL as part of "libc". But that doesn't mean BSD/GPLv2/LGPL files can't be distributed in the same tarball, that is mere aggregation.
License? Posted Nov 30, 2010 21:48 UTC (Tue) by dlang (✭ supporter ✭, #313) [Link]
not everything in the kernel tree is under the GPL license.
the only requirement is that all the source code used to create the kernel binary must be under a GPL compatible license.
there is no reason that another binary that's produced couldn't be under a different license, but there would need to be an appropriate audit of any files that are shared between the two binaries to make sure that they are under a license compatible with both (which can include multiple licenses on a single file)
ABI backward-compatibility is the trick... Posted Nov 30, 2010 21:02 UTC (Tue) by madscientist (subscriber, #16861) [Link]
The important, and difficult, thing about libc is that is HAS to be completely backward-compatible at an ABI level. Today I can take programs that were compiled against libc's shipped with distributions 10 years or more old, and run them on my up-to-date systems with no problems caused by libc. Making this work takes a lot of detailed effort in weak linkage, linker symbol versioning, multiple versions maintained, etc. etc. Alternatives such as just providing lots of versions of libc are simply not reasonable/sustainable.
Libraries that are not careful about this type of thing are a constant source of pain and torment (I'm looking at you, ssl/crypto!)
ABI backward-compatibility is the trick... Posted Nov 30, 2010 22:15 UTC (Tue) by lindahl (subscriber, #15266) [Link]
You mean, other than the syscalls which aren't backward compatible, e.g. sched_setaffinity changed the number of args from 3 to 2 to 3.
ABI backward-compatibility is the trick... Posted Nov 30, 2010 22:42 UTC (Tue) by madscientist (subscriber, #16861) [Link]
I'm not sure what your comment is referring to. Users of libc don't care whether syscalls are backward compatible. It's up to libc to make all those changes invisible to userspace... that's what I mean by backward-compatible ABI: the ABI between libc and the users of libc, not the ABI between libc and the kernel. In fact the advantage of moving libc into the kernel is that we would NOT need to be so worried about the ABI between libc and the kernel since they were modified in sync.
Obviously if you invoke system calls directly then you have to deal with this, but that's not relevant to whether libc comes with the kernel or not.
He meant GLibC function, not syscall... Posted Nov 30, 2010 22:56 UTC (Tue) by khim (subscriber, #9252) [Link] This is good example: sched_setaffinity(2) had two arguments before 2.3.4. If you'll take the old program which uses this version - you'll be unable to compile it today. BUT! If you'll take binary of old version of program compiled with old version of GLibC - it'll run with new one just fine because today GLibC offers two interfaces of sched_setaffinity(2) and selects them automatically depending on what the calling program expects. GLibC maintains stable ABI - and it's very good at that (documented "undefined" behavior is exception, of course). Yet it does not maintain stable API! This experience is so alien compared to what other libraries are doing that it takes some time to wrap up the mind around the concept.
ABI == important; API == not so much... Posted Nov 30, 2010 23:21 UTC (Tue) by madscientist (subscriber, #16861) [Link]
Yes, I agree... exactly why my comments were all discussing ABI backward-compatibility. Your example of the sched_setaffinity(2) call is, as you say, a great example of my point... ensuring that a backward-compatible ABI is available involves a lot of tricky footsie-playing with the linker etc.
API compatibility is a completely different animal: if you're going to use Linux-specific APIs then I suppose you just have to fix your code if they change in the future. If you don't want to do that, restrict yourself to POSIX and other standard APIs. I don't really worry about this, to be honest.
He meant GLibC function, not syscall... Posted Dec 1, 2010 10:57 UTC (Wed) by Yorick (subscriber, #19241) [Link] As you note, the ABI stability only goes in one direction. Someone who wishes to distribute a binary that will work on systems of varying age will have to chose between:
He meant GLibC function, not syscall... Posted Dec 1, 2010 14:14 UTC (Wed) by paulj (subscriber, #341) [Link]
It's not alien, it's solid engineering. Further, it takes attention to detail and work to maintain ABIs, which can be boring and unsexy - which is not what a lot of evening/weekend hackers are interested in. So likely most of this work is being done by commercial distros in response to customer demand.
Further, even for the API, there are ways to offer multiple versions of those. You can categorise your APIs in various, including with levels. Applications can then *choose* to explicitly depend on certain APIs or they can choose not to and use whatever the default API is - some applications will do that through laziness or ignorance.
All boring and hard engineering work, but it's what widely deployed (in space and, especially, time) systems tend to have to acquire.
ABI backward-compatibility is the trick... Posted Dec 1, 2010 7:32 UTC (Wed) by steven676 (subscriber, #41893) [Link]
> In fact the advantage of moving libc into the kernel is that we would NOT need to be so worried about the ABI between libc and the kernel since they were modified in sync.
Not unless you want to eliminate static linking and make upgrading your kernel a nightmare. (If the ABI the kernel exposes to libc isn't stable, then libc has to be updated with the kernel -- except that the kernel update isn't effective until reboot, while the libc update will be effective for newly launched programs immediately.)
Solaris has this problem, and "solves" it via a horrible hack called "deferred activation patching" -- basically bind mounting the old libraries over the new ones until reboot. (It's also why there's no static linking on Solaris >= 10.)
ABI backward-compatibility is the trick... Posted Dec 1, 2010 14:13 UTC (Wed) by madscientist (subscriber, #16861) [Link]
This is very true, and I don't actually believe that the kernel devs would use this as an excuse to give up on their current rules regarding userspace-visible changes. It would have all sorts of evil repercussions.
On the other hand, it's pretty darn hard to link many types of programs statically even today.
ABI backward-compatibility is the trick... Posted Dec 3, 2010 19:50 UTC (Fri) by Spudd86 (guest, #51683) [Link]
I don't think anyone was talking about letting the kernel break libc<->kernel ABI, the talk was about making new API available sooner and having new implementations of the (old) ABI in the libc sooner. Statically linked stuff would still use the old KERNEL ABI, and it would still work, but dynamically linked stuff would use then new libc which would use new kernel ABI, hopefully offering improved performance in the process.
Basically API/ABI-wise nothing changes (at least not intentionally), except the speed at which new parts become useable.
ABI backward-compatibility is the trick... Posted Dec 3, 2010 19:43 UTC (Fri) by Spudd86 (guest, #51683) [Link]
... not if they're linked against libc5 (OK that's slightly more than 10 years ago... but still, you can't run those binaries on a modern distro)
Also the kernel developers ARE aware of this.
The kernel and the C library as a single project Posted Nov 30, 2010 21:19 UTC (Tue) by Trelane (subscriber, #56877) [Link]
Is there a particular reason new syscalls need to be in libc immediately? Why not have a different library?
The kernel and the C library as a single project Posted Nov 30, 2010 21:21 UTC (Tue) by Trelane (subscriber, #56877) [Link]
(as an intermediate; clearly, syscalls should eventually reside in libc)
The kernel and the C library as a single project Posted Nov 30, 2010 21:46 UTC (Tue) by xtifr (subscriber, #143) [Link]
Should they? When I'm writing C, I'd rather write C, not some OS-specific variant. Granted, that goal's not quite achievable today, but I'd rather things get better than worse. I'd like to see the OS-specific features, like weird, non-standard syscalls, be kept in a separate library and headers so I can use them if I need them, but I'll always be aware that they are OS-specific and not portable.
Of course, in my ideal world, we all have a choice of OSes, and can pick the one that suits us best, and have the apps we need with little more than a simple recompile.
The kernel and the C library as a single project Posted Nov 30, 2010 22:04 UTC (Tue) by Trelane (subscriber, #56877) [Link]
I'm pretty sure the proposed new syscalls are by their nature non-portable. Hence adding a new library would not be any worse in this regard (rather, it'd be even clearer that you're doing something non-portable).
The kernel and the C library as a single project Posted Nov 30, 2010 22:09 UTC (Tue) by skitching (subscriber, #36856) [Link]
Sounds like a good idea to me - a "liblinux" providing linux-specific functionality.
The kernel and the C library as a single project Posted Nov 30, 2010 22:37 UTC (Tue) by nteon (subscriber, #53899) [Link]
using the syscall itself would be non-portable. but libc functions can be rewritten to use a new syscall, which would give your program/lib performance improvements 'for free'. Ingo mentioned the POSIX AIO functions as an example.
The kernel and the C library as a single project Posted Nov 30, 2010 23:15 UTC (Tue) by Trelane (subscriber, #56877) [Link]
Sure, and you could use the lib until it gets integrated into libc.
The kernel and the C library as a single project Posted Dec 1, 2010 16:42 UTC (Wed) by busterb (subscriber, #560) [Link]
There are libraries that do this today. A couple of examples:
libpcap abstracts the OS-specific calls to setup and capture from a raw socket. It includes several variations on a theme for Linux specifically, ranging from select/read semantics to mapping a shared-memory ring buffer.
libevent abstracts OS-specific replacement syscalls for select/poll, using whatever is fastest for a particular OS.
libaio is similar, but for asynchronous IO
These libraries all present a stable ABI for apps to use, similar to libc. Having a library like this seems like the perfect incubator for new syscalls as well.
The kernel and the C library as a single project Posted Dec 1, 2010 16:37 UTC (Wed) by nix (subscriber, #2304) [Link]
Yep! And the aio functions are maintained in a separate library (libaio), not in glibc. This has really helped libaio develop fast, oh, wait, no it hasn't: it languished unmaintained for years.
The kernel and the C library as a single project Posted Dec 1, 2010 16:45 UTC (Wed) by busterb (subscriber, #560) [Link]
I don't think libaio failing indicates a general problem with the approach. libpcap and libevent seem to be doing well.
The kernel and the C library as a single project Posted Dec 2, 2010 14:41 UTC (Thu) by nix (subscriber, #2304) [Link]
libpcap was also near-dead for a long, long time.
It's not a kiss of death, but it's not a magic bullet to make things instantly be well-maintained, either. :)
The kernel and the C library as a single project Posted Jan 2, 2011 13:27 UTC (Sun) by gvy (guest, #11981) [Link]
Did it really need attention?
The kernel and the C library as a single project Posted Nov 30, 2010 21:54 UTC (Tue) by vmauery (subscriber, #22649) [Link]
Sounds like a good idea for linux-2.7. Didn't Linus say that if there was that was really big then he might open up a 2.7 branch? This sounds like a pretty invasive change; one that would have to be tested out for a while before it was accepted.
Does it *have* to be the C library? Posted Nov 30, 2010 22:04 UTC (Tue) by cesarb (subscriber, #6266) [Link]
Does it really have to be the C library? If I create a cool new syscall coolnewsyscall(), why not also distribute a libcoolnewsyscall? There is no reason why libc should be the only library allowed to use syscalls, and even libc itself is split in several libraries (libc, libm, libpthread, libdl, librt...).
Does it *have* to be the C library? Posted Nov 30, 2010 22:20 UTC (Tue) by gmaxwell (subscriber, #30048) [Link]
Because sometimes, even often, coolnewsyscall() is really just a function that enables a better, faster, or more flexible implementation of boringlibcfunction().
Does it *have* to be the C library? Posted Dec 1, 2010 14:14 UTC (Wed) by marcH (subscriber, #57642) [Link]
Isn't LD_PRELOAD=liblinux.so able to sneak new code into glibc and solve this problem?
Does it *have* to be the C library? Posted Dec 2, 2010 14:42 UTC (Thu) by nix (subscriber, #2304) [Link]
No. Internal glibc->glibc calls do not go via the PLT and are not subject to interposition, thus LD_PRELOAD has no effect on them. (This is a big speed boost.)
The kernel and the C library as a single project Posted Nov 30, 2010 22:04 UTC (Tue) by rleigh (subscriber, #14622) [Link]
Historically, the C library on Linux has provided the low-level system call interface, the POSIX functions (which wrap the system calls for the most part) and the standard C functions. Why do we need all these disparate parts in a single library?
Could the system call interface be wrapped and placed in a completely separate library, e.g. libsystem, which the POSIX and/or C libraries could then use?
If there was ever a library which could use some simplification, it's glibc. Its build system is impenetrable and its sources often incomprehensible (to a newcomer).
The kernel and the C library as a single project Posted Dec 1, 2010 11:21 UTC (Wed) by michaeljt (subscriber, #39183) [Link]
> Historically, the C library on Linux has provided the low-level system call interface, the POSIX functions (which wrap the system calls for the most part) and the standard C functions. Why do we need all these disparate parts in a single library?
I was just about to voice those very thoughts.
The kernel and the C library as a single project Posted Dec 1, 2010 11:58 UTC (Wed) by jlayton (subscriber, #31672) [Link]
Hear hear!
This is the sanest idea yet. Sounds simple (in principle) to break the syscall interfaces out of libc and into a separate lib. libc could then be built with dependencies on the new library, thereby maintaining backward compatability.
I'm sure there are all sorts of nasty details I haven't considered, but it sounds like a step in the right direction.
The kernel and the C library as a single project Posted Dec 4, 2010 0:24 UTC (Sat) by oak (guest, #2786) [Link]
And if it would be a real dynamic library (instead a kernel emulated one), intercepting file system calls would be much easier. Currently libc functions can internally do all kinds of interesting things, but with syscalls being in a separate library they could simply be cought with an LD_PRELOAD...
(And the other alternative for interception, ptrace(), changes signaling semantics and race-free interception of calls in threaded code with ptrace() needs architecture specific code.)
The kernel and the C library as a single project Posted Nov 30, 2010 22:38 UTC (Tue) by Frej (subscriber, #4165) [Link]
Sure single project mentality helps big time in actually designing the right solution. But why does that require a single 'module'? Is it impossible to post two different to two different patches to two different projects, in this case linux/glibc ?
It seem possible in gnome where each module is seperate git repo, or some lower system parts has added a feature that should be propogated upwards. But people manage to 'cross-patch'.
Or does current tools suck when handling two seperate projects?
Too late Posted Nov 30, 2010 22:59 UTC (Tue) by marcH (subscriber, #57642) [Link]
> Putting nethack into the kernel repository might just pass muster, but, one assumes, Emacs would encounter resistance...
Resistance is futile; besides a few other minor modules the next version of Emacs has already subsumed the kernel.
The kernel and the C library as a single project Posted Nov 30, 2010 23:35 UTC (Tue) by cyd (guest, #4153) [Link]
What, no one remembers "Linux libc" and the long messy transition from "libc5" (Linux libc) to "libc6" (upstream GNU libc)? Here's the relevant excerpt from Rick Moen's classic essay on free software forking, at [http://linuxmafia.com/faq/Licensing_and_Law/forking.html]:
> When Linus and his fellow programmers started work on the GNU/Linux system (using Linus's "Linux" kernel), they looked around for free-software C libraries, and chose Stallman's. However, they decided that FSF's library (then at version 1-point-something) could/should best be adapted for the Linux kernel as a separately-maintained project, and so decided to fork off their own version, dubbed "Linux libc"... but in 1997-98 they noticed something disconcerting: FSF's glibc, although it was still in 1-point-something version numbers, had developed some amazing advantages... even though their fork seemed a good idea at the time, it had been a strategic mistake. Adding all of FSF's improvements to their mutant version would be possible, but it was easier just to re-standardise onto glibc.
The kernel and the C library as a single project Posted Dec 1, 2010 2:56 UTC (Wed) by vonbrand (subscriber, #4458) [Link] I remember libc5 as an a.out library (and rather non-portable, it was intimately tied to the Linux kernel; the GNU libc just didn't work on Linux, as it was meant to be built on Solaris and such), the move to libc6 was due mostly to migrating to ELF (and meant large scale changes to glibc to make it work with Linux). Yes, the switchover was a veritable mess.
The kernel and the C library as a single project Posted Dec 1, 2010 6:11 UTC (Wed) by jamesh (guest, #1159) [Link]
The a.out to ELF transition occurred at the upgrade from libc4 to libc5. We were already using ELF when moving to glibc.
The kernel and the C library as a single project Posted Nov 30, 2010 23:52 UTC (Tue) by hmh (subscriber, #3838) [Link]
I think it would be more productive to have a "Linux glibc liason".
A proactive person which would:
1. Maintain a fast-merge branch of glibc for new Linux functionality
2. Pester the hell out of any kernel developer that does not ALSO
3. Work with glibc to merge the fast-merge topic branches as soon as
That liason needs to be someone with "very good taste" for ABI and
And people could use the fast-merge branches to widely test the new
The kernel and the C library as a single project Posted Dec 1, 2010 10:44 UTC (Wed) by dgm (subscriber, #49227) [Link]
This sounds just about right for me.
The only problem being that, as any interface, this person has to have a pretty thick skin and be resistant to constant flak from both sides. You must add that it has to be someone that both sides can trust and work comfortably. Where to find such a hero?
The kernel and the C library as a single project Posted Dec 1, 2010 14:48 UTC (Wed) by hmh (subscriber, #3838) [Link]
Hmm? The onus is, and has always been, with whomever is proposing the new interfaces.
The liaison might get some flak from whomever is proposing the new interfaces and being told to go back to the drawing board. But the liaison will be in a position of power in that case (as long as he is someone that is respected by Linus for his good taste in ABIs/syscalls).
The way these things have worked in LKML lately is that you either do _something_ productive when people decide to oppose your new syscall/ABI, even if it happens at the very last moment, or you are at a reasonable risk of all your effort being declared "not for mainline".
But yes, it HAS to be someone respected by both sides, or someone who is respected in LKML (he has to have power to reject bad interfaces since day one), and who has a large potential to become respected within the glibc community.
The kernel and the C library as a single project Posted Dec 1, 2010 16:40 UTC (Wed) by nix (subscriber, #2304) [Link]
Paging Roland McGrath... (who also has about as much pull on the glibc side of things as it is possible to have... of course he's also very busy and probably doesn't have time for this.)
The kernel and the C library as a single project Posted Dec 1, 2010 2:49 UTC (Wed) by RogerOdle (subscriber, #60791) [Link]
This is a bad idea. It amounts to creating a fork of glibc that will forever cause compatibility problems. It would expand the responsibility of the kernel maintainers to also include the maintenance of what is essentially not kernel code. They have enough to do already.
What problem are you trying to solve? glibc developers are not accepting changes to the API fast enough? They shouldn't allow the core interfaces for Linux applications to change quickly or easily as that would undermine the stability of the Linux environment and the confidence of the users. Why is it important for some particular function to be added to the c library? It is not that difficult to build a library for Linux and install it.
One of the main problems with glibc now is its size. For some embedded projects, glibc is just too big already. This issue is addressed by using reduced glibc alternatives or custom modifications of glibc itself. Neither solution is entirely satisfactory.
What is needed is to concentrate of sound engineering principals; particularly the one that Microsoft has never learned. That reliability and maintainability requires that things be kept as simple as practical. Do not add to the kernel developers the responsibility for the user space libc. The glibc maintainers have been doing an adequate job while not chasing every fad that comes along.
When Microsoft extends its operating system with Office code and Outlook code, it compromises reliability and security. Microsoft does not do this for any good engineering reason, they do it for a business reason. They do it to control the market.
The reasons for not combining libc with the kernel far out weigh the benefits. Please do not do this.
The kernel and the C library as a single project Posted Dec 1, 2010 6:29 UTC (Wed) by eru (subscriber, #2753) [Link]
Agreed! The original problem can be solved much better by improving the libc development, not by mashing it with the kernel. Maybe there could be some Linux-specific glibc fork that is updated very frequently to take advantage of the latest kernel features, but it must not be a part of the kernel in any sense, and must be replaceable by a totally different library if needed (this demands the kernel interface stays well-defined).
The kernel and the C library as a single project Posted Dec 2, 2010 13:43 UTC (Thu) by i3839 (guest, #31386) [Link]
Agreed. It's a bloody bad idea.
More problems would be solved if the kernel would drop bad system calls before they are in widespread use, instead of the stupid "stable ABI forever, even new bad ABI" mindset. The sooner crap is removed, the better. Only once it's in widespread use it's bad to break the ABI. But that can only happen if you didn't remove the bad ABI early enough. I hope one day Linus gets a clue about this.
The kernel and the C library as a single project Posted Dec 1, 2010 8:50 UTC (Wed) by jakub@redhat.com (subscriber, #31780) [Link]
AFAIK new syscalls are added to glibc quite fast, sometimes even faster than the kernel people actually make up their minds on the syscall ABI, so there are various issues when syscalls keep changing ABI or are dropped altogether. It doesn't help much that the kernel people are so creative
on finding out how to pass arguments to the syscalls (e.g. for simple stuff like passing 64-bit integral arguments to syscalls there are at least dozen different ways how syscalls do it - the issues related to say passing long long arguments in 32-bit ppc ABI on non-even positions which forces a padding argument register are known for years, but the approaches how to pass such arguments keep constantly changing). For ABI stability the only thing glibc needs is assurance of the syscall ABI and that it is not going to be removed the next day.
I'd also say that understanding libc as just a set of syscall wrappers is great misunderstanding of libc complexity.
The kernel and the C library as a single project Posted Dec 1, 2010 10:51 UTC (Wed) by dgm (subscriber, #49227) [Link]
> I'd also say that understanding libc as just a set of syscall wrappers is great misunderstanding of libc complexity.
Indeed, there are reasons for glibc being so big in the first place. Also, many functions of the libc have to be optimized by hand for each target platform, often writing them in assembler. Do the kernel developers really want to put such a burden over their own shoulders?
The kernel and the C library as a single project Posted Dec 1, 2010 14:05 UTC (Wed) by jwarnica (subscriber, #27492) [Link]
Not being a developer, I suspect I'm just talking gibberish... But why is it an all or nothing proposition?
Wouldn't it be possible to maintain a reference implementation of the Linux specific calls, which otherwise "should" be in libc? Since application developers would need to do something anyway (writing code to use the new calls), having +1 runtime .so seems to be all but insignificant.
Some libc implementors might ignore this, some may copy individual functions verbatim into their system, some might link to it, and put a thin layer over top. Some (say, those who disagree how memory copying should work....) could just use it for reference, to build compatible interfaces with entirely different internals.
It might also be possible, with this layer, to give post implementation stability, additional time for interface stability. That is, while the kernel syscall() might "never" change, the interface provided by the reference implementation has 12 months, or 2 releases, or whatever, to stabilize.
The kernel and the C library as a single project Posted Dec 1, 2010 15:25 UTC (Wed) by deater (subscriber, #11746) [Link]
I guess when you're a kernel developer you think any userspace problem can be solved by importing it into the kernel. It makes me weep.
So far Ingo has proposed moving performance counter utilities and qemu into the kernel and now libc. I can't wait until the web-browser gets merged too, I hear everyone else is doing it.
The kernel and the C library as a single project Posted Dec 1, 2010 16:33 UTC (Wed) by butlerm (subscriber, #13312) [Link]
No one is proposing importing the C library into the kernel, but rather the kernel source tree. Big difference.
The kernel and the C library as a single project Posted Dec 1, 2010 17:53 UTC (Wed) by roblucid (subscriber, #48964) [Link]
Ooh may be Ingo could intergrate an http daemon into kernel.. oh hang on.. been dun already???
|
|||||||||||||