The case for the /usr merge [LWN.net]

The case for the /usr merge

Posted Jan 27, 2012 21:39 UTC (Fri) by misc (subscriber, #73730)
In reply to: The case for the /usr merge by gb
Parent article: The case for the /usr merge

First, I should remind to people that Linux based systems are not the only free software unix out there. The various BSD are alive and kicking, and maybe they would be more what you want than some linux distributions. That's also the freedom of free software.

Now, for the point you make "initramfs should not customizable" is rather wrong. One, you cannot prevent people from messing with it, this is free software. Two, most people do not have to mess or do not know how to do, because most of the time ( and at least for me ), it simply work.

Now, gconf backend was just file based, editable with any editor. Maybe you speak of dconf, and even in this case, there is less complexity than with registry. Microsoft switched to a more featureful system to be able to offers stuff like audit, permissions and stuff like that. While there is maybe othes ways to achieve this, having a opaque storage and forcing to use a API was not more horrible than what a SGBD do. Most problem comes from usage of pseudo-uuid, and the constant need to tweak the internal to change something.

For dbus, i can perfectly run most program as other user. Maybe you mean "I cannot run some graphical program that requires to access to shared service and some kind of IPC for that", in which case this is correct, but I would ask how you would design the system differently. If there is the concept of session, something need to start the session, and in this case, this is dbus-launch.

Network manager is working without gui, there is cnetwork-manager thanks to dbus. And network-manager is not really mandatory, you can always do stuff by hand. Of course, that's annoying for wifi, but for a server, this work fine, setup once and that's all.

Video driver in kernel space is agan rather unavoidable. If you want the kernel to manage the graphical card ( and if you want to have a text console, it has to be done by the kernel ), then you need to have part of the driver in kernel. That's what kms is about. But maybe you would prefer to have no console, or weird lock and problem because both X and the kernel use the same memory without talking to each others.

Gnome 3 is rather controversial yes, but you may notice that there is also lots of others desktop environment and choice if it doesn't please you. I found that most of the reasons listed on gnome.org to be valid, and I can understand that despites being reported twice, gnome-shell can still be improved based on various feedbacks.

Finally, you seems to not like pulseaudio, and either you have issues with it ( and maybe that's not pulseaudio ), or maybe you think there is no use to the feature ( like saner mixer defaults ). It is hard to answer without you being more specific, but if you didn't liked having video driver in the kernel, you surely would prefer to have all the features of pulseaudio ( per application mixer, bluetooth support, streaming support, ease of use to change sound output for video card with hdmi output, music being stopped when you get a soft-phone call ), being out of the kernel ( and alsa ) as well. Maybe you don't care about the features, but I am rather happy to have my hardware working ( bluetooth ).

So if you do not want any changes and if your system worked fine, you can simply not upgrade.

to post comments

The case for the /usr merge

Posted Jan 27, 2012 22:32 UTC (Fri) by gb (subscriber, #58328) [Link] (4 responses)

Oh, seem i touched topic which is very heavy, and largely off-topic for the article...

initramfs: without it, a *lot* is impossible. No heavily modular kernels. No rootfs on LVM. No rootfs on md/RAID. No rootfs on any other filesystem requiring userspace involvement to mount. No rootfs over the network (sounds silly for most machines until you have a disk failure and can revive the system, temporarily mounting everything over the network, with a one-line change). The list goes on and on. And the cost of the initramfs? These days, nearly nil.

Sure. It's not avoidable for it's purpose, i see no other way than loading two files with bootloader, it just seems pity for me that where is no simplier way to solve problem of unaccessible /. Sidenote, you CAN have / on mdraid without initrd, kernel can autodetect / array and run init from mounted lets say /dev/md0, modularity of kernel is almost not impacted too because you can put all modules on /, only modules you really need in initrd are modules to mount your /, ie fs module, hdd device module and partition style support.

But i meant a bit different thing talking about / contents in initrd. i meant that it's so less natural to edit it than normal files on / that such rescue tools location seems inappropriate.

The case for the /usr merge

Posted Jan 27, 2012 22:59 UTC (Fri) by nix (subscriber, #2304) [Link] (2 responses)

You don't have to load two files with the bootloader to use an initramfs. They can be automatically assembled by the kernel build process and merged into the kernel image, whereupon the bootloader need not know of it at all. Most distros don't do this purely because they ship the binary over to the user but want to assemble the initramfs on the user's machine. For those of us building our own kernels, this is not a concern. In this situation you never need to edit it, either, just as you never need to edit the binary kernel image: just change its source and rebuild the kernel.

You can only have / on md/raid with in-kernel mounting if you want to use a somewhat-horrible and somewhat-deprecated v0.90 array. v1.x arrays are not supported and may never be.

The case for the /usr merge

Posted Jan 27, 2012 23:19 UTC (Fri) by gb (subscriber, #58328) [Link] (1 responses)

Embedding mostly for embedded systems i guess, i didn't mentioned all this details just to keep post in reasonable size :)

The case for the /usr merge

Posted Jan 28, 2012 14:56 UTC (Sat) by nix (subscriber, #2304) [Link]

I find embedding really useful even for non-embedded systems, and *particularly* for heavily modular kernels in which you must load modules from the initramfs just to mount root.

The problem with a non-embedded initramfs is that it can get out of synch with the kernel it goes with, especially if it has modules in it. So you might find that you preserve your kernel for future use in case of emergency but, oops, it's useless, it has the wrong initramfs and won't boot. With an initramfs linked into the kernel image, that is completely impossible: the kernel is a single self-contained file again. And that's a very valuable property.

initramfs

Posted Jan 28, 2012 12:22 UTC (Sat) by bjartur (guest, #67801) [Link]

You should be able to mess with initramfs gzipped CPIO archives just fine.

IMO initrd/initramfs is a great thing for portable operating systems such as live CDs and installers, because there is no way they can know in advance what hardware they might need to support before they are booted. And having an emergency kernel and rootfs populated with recovery tools could easily come in handy.

On regular, fairly static systems, where you don't replace your MBR with a GPT or a BSD disklabel after every other reboot, you can compile the stuff you always need into the kernel image and load additional modules lazily. But if disks and video cards vary from boot to boot, initramfs might actually be considered an optimization, allowing you to disable module unloading and use userspace tools to aid boot.

But none of this is why mainstream distros use initramfs. The actual reason is that they insist on distributing kernel binaries, but are OK with users assembling initramfs images. And who can blame them, compiling kernels takes CPU time.

The case for the /usr merge

Posted Jan 27, 2012 23:08 UTC (Fri) by gb (subscriber, #58328) [Link]

>pulseaudio: Finally, you seems to not like pulseaudio, and either you have issues with it ( and maybe that's not pulseaudio ), or maybe you think there is no use to the feature ( like saner mixer defaults ). It is hard to answer without you being more specific, but if you didn't liked having video driver in the kernel, you surely would prefer to have all the features of pulseaudio ( per application mixer, bluetooth support, streaming support, ease of use to change sound output for video card with hdmi output, music being stopped when you get a soft-phone call ), being out of the kernel ( and alsa ) as well. Maybe you don't care about the features, but I am rather happy to have my hardware working ( bluetooth ).

No, exactly opposite, i enjoy pulseaudio. Especially for it's original purpose - transferring sound over the network. It's best tool available for this task, and i am using it for to centralize my home sound.

Ah, dynamic changing of output card. You right, anyway some entity needed to implement this kind of functionality. Thanks for description, didn't thought about it.

The case for the /usr merge

Posted Jan 27, 2012 23:18 UTC (Fri) by gb (subscriber, #58328) [Link] (5 responses)

> Video driver in kernel space is again rather unavoidable. If you want the kernel to manage the graphical card ( and if you want to have a text console, it has to be done by the kernel ), then you need to have part of the driver in kernel. That's what kms is about. But maybe you would prefer to have no console, or weird lock and problem because both X and the kernel use the same memory without talking to each others.

Right again, but i can recall that early Linux feature: hey, our video processing is in userspace, so you have much less chances of lockup guys! I slowly fades out. I want to add more about kernel in general, former Torvalds told 'move as much as possible out to userspace!' now it seem for me that everything moves in opposite direction and kernel is bigger and bigger. Yes, it's unavoidable again. But pity that some ideals lost.

The case for the /usr merge

Posted Jan 27, 2012 23:21 UTC (Fri) by dlang (guest, #313) [Link] (1 responses)

the thing is that with video, more was moved out than was possible while still keeping the system reliable.

note that most of the driver is still in userspace, it's only a tiny portion that's been moved in to the kernel, just enough to be able to track (and set) the video mode.

to butcher a quote
things should be as simple as possible, but no simpler

this is just a small correction

The case for the /usr merge

Posted Jan 28, 2012 14:58 UTC (Sat) by nix (subscriber, #2304) [Link]

Memory management is also in the kernel (again, because it has to be, you can't share that sort of thing between multiple instances of Mesa, and you have multiple instances of Mesa all the time because the apps often have their own instances rather than going through the X server, for performance reasons). The command stream checker also obviously has to be in the kernel, to protect against a malicious userspace.

The case for the /usr merge

Posted Jan 28, 2012 3:43 UTC (Sat) by HelloWorld (guest, #56129) [Link]

> I want to add more about kernel in general, former Torvalds told 'move as much as possible out to userspace!'
Torvalds never said anything remotely like that.

The case for the /usr merge

Posted Jan 28, 2012 13:56 UTC (Sat) by farnz (subscriber, #17727) [Link]

And with KMS, the architecture is still that most of the video work is done in userspace. The kernel manages memory (both graphics card memory, if any, and shared system memory), dispatches command streams from userspace to the GPU in a safe fashion (stopping apps disrupting each other or accessing each other's memory space), handles interrupts, does modesetting, and resets the GPU when userspace apps crash it.

Everything else (all the complex command stream creation) is done in userspace still.

The case for the /usr merge

Posted Jan 29, 2012 21:45 UTC (Sun) by sbergman27 (guest, #10767) [Link]

"I want to add more about kernel in general, former Torvalds told 'move as much as possible out to userspace!'"

Linus has never taken that position. All other things being equal, he favors that things stay in user space. But if there is good argument for moving it into the kernel, he's always been OK with that. In fact, he has been very explicit on the point that the goal should not be to keep the kernel small, but to make the right decisions as to what goes in and what does not.

The GGI guys wanted to move the whole graphics driver nightmare into the kernel. Linus put his foot down on that idea. But DRM, memory management, and kernel mode-setting were the peices of Linux graphics which were always fundamentally broken under the old "all graphics in userspace" model. Do you know that switching from X to a text console and back has never been guaranteed not to hard lock your machine? Surely, if you've been around long enough, you've encountered such console crashes. And when that happened, it was likely not a bug, per se. It was the result of a fatally flawed design. Only with the inclusion of KMS is your distro's graphics system not fatally flawed at a fundamental level.

Regarding your concerns about the size of the kernel, what does it matter? The vast majority of kernel code expansion is in additional drivers. And those are all modules. Modules whose code-bases are orthogonal to each other.

In comparison, DRM, KMS, and GEM are not particularly large. I'd never want to go back to the bad old days of our former "Green Acres" graphics architecture.

The case for the /usr merge

Posted Jan 27, 2012 23:59 UTC (Fri) by gb (subscriber, #58328) [Link] (8 responses)

Network manager and dbus guys: One of aspects of this tools that they keep integrating into my favorite applications. In 2005, my mail application didn't need a 'X session' and export $(dbus-launch) and somehow worked in ssh -X session. It also didn't need to know about network manager connection state which magically wrong too often.

dbus: My ideal were fine systems without "system wide message bus" and related program interdepedency (now all programs are on single bus which one need other? who knows... you mailer doesn't work? check network-manager!). I am not a big specialist in it, but it seems that d-bus has very limited applicability (low-bandwidth transfers), but seem used as panacea everywhere. Dbus services seem special harm. I need to learn it othervise it would be security concern (something starts somewhere at random time - is it really so nice?). As i see it, system should have single point for starting services, not separate init.d and some black d-box.

Of course 'not upgrading' is never an option because i want and must have new features of rest of system - support for new hardware, new editors, new secutiry updates, new architectures, new videos, new internet browsers. Once i saw in openmoko list hardcore guy who really not upgrading, and using some VAX system with original software, his problems with ftp were nice example of 'non-upgrade' path.

The case for the /usr merge

Posted Jan 28, 2012 0:34 UTC (Sat) by Kit (guest, #55925) [Link] (6 responses)

Applications adding stupid restrictions/requirements is the application's fault. Applications have been doing stupid things based on what they ASSUME is the state of the network since long before Network Manager (and really, a NM like service is the only reasonable solution for any mobile system).

>As i see it, system should have single point for starting services,
>not separate init.d and some black d-box.

So now you're wanting systemd? The old init systems are most certainly not capable of that!

>Dbus services seem special harm. I need to learn it othervise it would
>be security concern (something starts somewhere at random time - is
>it really so nice?).
You can fairly easily figure out all the things that could possibly be started on demand... so how is that different than having them _always_ running? Of course, you'd still need to connect to them via an IPC system like dbus...

>I am not a big specialist in it, but it seems that d-bus has very
>limited applicability (low-bandwidth transfers), but seem used as
>panacea everywhere.
IPC has lots of uses in a desktop environment. Everything from an applet on your panel that controls your music player to administrative tools that don't expose root access to your whole X session (if you launch a graphical app as root on your regular X session, you better trust every program running as your user with root privileges, because they could have them if they wanted!). There are tools (including graphical ones) that let you explore the dbus user bus and system bus, to see what's accessible. You might be surprised at the large amount of functionality easily accessible.

The case for the /usr merge

Posted Jan 29, 2012 11:44 UTC (Sun) by k8to (guest, #15413) [Link] (5 responses)

Like it or not, the common programming patterns and libraries around dbus are pretty fragile. It's very common for dbus programs to fail to operate correctly if dbus isn't already running when they start up. It's common for them to fail to establish communication with launched subprocesses.

Essentially, the filesystem, pipes, sockets are pretty safe to assume to be always present, and you can develop against them without fear (though please handle the errors). With dbus, you have to handle a number of additional cases, and it seems more difficult than real programmers are going to do in reality.

For that problem alone, I question the value of having dbus at all.

Then there are the security concerns.. but that's the topic for another day.

The case for the /usr merge

Posted Jan 29, 2012 21:48 UTC (Sun) by paulj (subscriber, #341) [Link] (4 responses)

A good example of fragile DBus programming patterns is systemd. Guess what happens if early boot goes wrong (e.g. cryptsetup fails on some not-very-important filesystem) and you get dumped to a rescue shell and you try use systemctl? A nice "Can't connect to message-bus" type of error.

Why on earth is systemctl not talking *directly* to systemd??

The case for the /usr merge

Posted Jan 30, 2012 0:11 UTC (Mon) by mezcalero (subscriber, #45103) [Link] (3 responses)

This is simply not true. systemd speaks the D-Bus protocol without relying on the D-Bus daemon. One of the nice things about systemd is that is is fully introspectable even if the only processes running are systemd itself and a bash. For example, boot into emergency mode by passing "emergency" on the kernel cmdline, and you'll see that "systemctl" works just fine even though no processes between PID 1 and bash are around.

Don't spread obvious falsehoods, please!

The case for the /usr merge

Posted Jan 30, 2012 1:35 UTC (Mon) by slashdot (guest, #22014) [Link]

paulj seemed to speak from experience though, so maybe (some past version of?) systemctl first tries to talk via dbus, but doesn't always properly fallback to direct communication when that fails?

The case for the /usr merge

Posted Jan 30, 2012 12:20 UTC (Mon) by paulj (subscriber, #341) [Link] (1 responses)

Systemctl gives DBus connection errors in certain situations. And, other than to those quite familiar with systemd, it's NOT an obvious falsehood. I know you get a lot of uncalled-for personal crap from snipers, but does it help for you to respond to other people in a bordering-on-insulting fashion?

Here's what I was going by:

https://picasaweb.google.com/114169232705420738100/System...

I'm not sure, but there's probably some combination of a couple of silly little bugs here:

a) If emergency mode had root mounted RO (I'm not sure if that happened at shutdown or at beginning of emergency mode): Going to emergency mode with root RO when some ancillary fs has failed to mount (I don't remember what old initscripts did, but I think they first mounted / RW before mounting rest?)

b) Shutdown having become fragile in the face of a RO /. I've had old init + initscripts in a similar situation, & services would fail to stop, but init would eventually reboot anyway. A fragile reboot can be a huge problem if you're remote and have no physical access to the power/reset buttons.

c) A slightly misleading or confusing error message from systemctl.

d) Unrelated to my original post, but also evident in those photos: Parallel jobs stomp over the tty with output. Particularly confusing if you've missed the fact you're supposed to type in a passphrase. (This possibly isn't a little bug, in terms of fixing it).

Bugs, which will be fixed no doubt, I'm sure. Also, just because a messenger doesn't have a perfectly set-out message, doesn't mean the messenger should then be shot.

The case for the /usr merge

Posted Jan 30, 2012 16:55 UTC (Mon) by michich (guest, #17902) [Link]

If you can find a sequence of steps to reproduce the problem reliably, please file a bug in Bugzilla.

But... the second screenshot shows 2.6.34.7-66.fc13. Are you really running Fedora 16 on a kernel from Fedora 13?

The case for the /usr merge

Posted Jan 29, 2012 21:24 UTC (Sun) by speedster1 (guest, #8143) [Link]

> Of course 'not upgrading' is never an option because i want and must have new features of rest of system - support for new hardware, new editors, new secutiry updates, new architectures, new videos, new internet browsers.

It sounds like you are not using one of the Linux distributions best suited to your needs. Distributions such as debian and gentoo are well suited for an experienced user, such as yourself, who does not want to trade extra integration and automation for additional complexity. With one of those more flexible distros, it is easy to set up a useful desktop system without gconf, network manager or dbus, and without getting stuck on old versions of software. At this moment, I happen to be doing a gentoo install without any of those packages.