|
|
Log in / Subscribe / Register

Quotes of the week

[Posted March 2, 2011 by corbet]

Then, we have how the user interaction with the server is designed. Let me get this straight: It's 2011, it's about time to stop editing complex, error prone, plain text configuration files, don't you think? Now that we all are running fancy desktops full of powerful applications.. what sense does it make to have to open a terminal window, become root, and edit a text file, and type a new command to reload the service in order to make a little change on how your web server behaves. I'd say that none, none at all.
-- Alvaro Lopez Ortega 

+ log_warning("/usr appears to be on a different file system than /. "
+             "This is not supported anymore. "
+             "Some things will probably break (sometimes even silently) "
+             "in mysterious ways.");
-- Lennart Poettering
to post comments

Systemd incompatible with mounted /usr

Posted Mar 3, 2011 6:39 UTC (Thu) by ncm (guest, #165) [Link] (52 responses)

Wow, and just when I was beginning to think it was time to start looking into systemd. Thanks for the blast of sanity!

Systemd incompatible with mounted /usr

Posted Mar 3, 2011 8:14 UTC (Thu) by peter-b (guest, #66996) [Link] (9 responses)

The PCI and USB databases used by udev live in /usr on almost all distros, as does a lot of other pre-existing infrastructure that systemd uses.

This patch just makes explicit what was already the case.

You're arguing one of three things: systemd shoud reinvent a large selection of wheels; systemd should fix almost every mainstream distro to put *a lot* more stuff in /; or you wouldn't ever use systemd anyway, in which case you needn't of bothered wasting electrons with your comment. Which?

Systemd incompatible with mounted /usr

Posted Mar 3, 2011 9:48 UTC (Thu) by russell (guest, #10458) [Link] (7 responses)

Really? I just took a look on Fedora 13 and only found documentation under /usr

Can you be more specific? Have distros fixed this problem already?

Systemd incompatible with mounted /usr

Posted Mar 3, 2011 10:02 UTC (Thu) by peter-b (guest, #66996) [Link] (6 responses)

I'm just going by this thread on the systemd mailing list. Says Lennart:

It's just a statement of fact, as a warning. Thinks like locale, certain udev rules, udisks, SMART, the pci db, the usb id, and a lot more have been broken since about always on seperate /usr.

The only thing that is new here is that we now print a warning that things are broken. The fact that it is broken didn't change. And hence I see little reason to add this to the release notes.

Also, it's just a *warning*. If you ignore it then things will continue to work as well or badly as they have been doing before.

Systemd incompatible with mounted /usr

Posted Mar 3, 2011 10:27 UTC (Thu) by roblucid (guest, #48964) [Link] (5 responses)

Well mentioning those problems, would be much more sensible than saying, "not supported" and warn of "things breaking in mysterious ways". It's spreading FUD by being mysterious and non-specific.

If you have to have /usr in same FS as /, then there's no point anymore in /usr/{bin,sbin,lib}, in single user mode /usr is generally mounted.

If things like "locale, certain udev rules, udisks, SMART, the pci db, the usb id" are required for init(8) before /usr is mounted then that stuff should be under / instead. If someone fixes this error, how would someone see that systemd warning is no longer required?

Traditionally /usr was mountable read only, except during OS updates and being able to put / & /usr on different disks could be nice system optimisation.

With SSDs or flash based embedded systems, there's more reason to support multiple partitions and more than one system mount point, even if desktop developers don't see a need for it.

Systemd incompatible with mounted /usr

Posted Mar 3, 2011 20:12 UTC (Thu) by mezcalero (subscriber, #45103) [Link] (4 responses)

Oh, right. The usual complaint about "there's no documentation". Well, it's kinda unfounded, since the README explains the background of the warning:

http://cgit.freedesktop.org/systemd/tree/README

Systemd incompatible with mounted /usr

Posted Mar 4, 2011 10:53 UTC (Fri) by rvfh (guest, #31018) [Link] (3 responses)

The warning is visible against your will. The documentation needs to be searched for.

Please stop the RTFM stupidity and start giving proper information where and when it is needed.

Systemd incompatible with mounted /usr

Posted Mar 4, 2011 11:27 UTC (Fri) by rahulsundaram (subscriber, #21946) [Link] (2 responses)

As per demand,

http://cgit.freedesktop.org/systemd/commit/?id=aa1671320d...

Systemd incompatible with mounted /usr

Posted Mar 4, 2011 12:27 UTC (Fri) by rvfh (guest, #31018) [Link] (1 responses)

Thanks! And BTW, I can't wait to put my hands on systemd. Looks like the way to go IMVHO. Are we sure it will get in Fedora this time?

Systemd incompatible with mounted /usr

Posted Mar 4, 2011 12:52 UTC (Fri) by rahulsundaram (subscriber, #21946) [Link]

Your questions makes a assumption which I am not clear is true. Fedora 14 includes systemd. It is not the default but it is easy enough to try (yum install systemd, boot with init=/bin/systemd). Fedora 15 branch includes systemd as default and I don't see any reason it wouldn't be released that way. If you want to try it out, Fedora 15 alpha release is coming out shortly

http://alt.fedoraproject.org/pub/alt/stage/15-Alpha.RC2/

You can grab a nightly build from

http://alt.fedoraproject.org/pub/alt/nightly-composes/

Systemd incompatible with mounted /usr

Posted Mar 3, 2011 15:38 UTC (Thu) by kilpatds (subscriber, #29339) [Link]

You're arguing one of three things: systemd shoud reinvent a large selection of wheels; systemd should fix almost every mainstream distro to put *a lot* more stuff in /; or you wouldn't ever use systemd anyway, in which case you needn't of bothered wasting electrons with your comment. Which?

That a distribution that uses systemd should move the relevant bits to /.

Now ... uh ... which bits are those?

Systemd incompatible with mounted /usr

Posted Mar 3, 2011 16:08 UTC (Thu) by nix (subscriber, #2304) [Link] (40 responses)

Precisely. Not a hope of my using systemd now, not if it requires massive fs reorganization in order to install it.

Sheesh. What idiocy.

Systemd incompatible with mounted /usr

Posted Mar 3, 2011 16:11 UTC (Thu) by nix (subscriber, #2304) [Link] (26 responses)

To make my comment constructive, I suppose I should point out how to fix this. Have a reduced-functionality mode in which systemd automatically starts if pieces are missing, and have it watch /proc/mounts for signs of the addition of /usr. When that is there, it can turn on all the stuff that requires things like USB databases. (No system is going to need that sort of stuff in the very early boot phase before /usr has been located, but booting with /usr on NFS is not remotely unknown in large installations, even today. Perhaps it is unknown to Fedora users, but Linux is more than just Fedora.)

Systemd incompatible with mounted /usr

Posted Mar 3, 2011 20:06 UTC (Thu) by mezcalero (subscriber, #45103) [Link] (19 responses)

But why? Just *why* all this work?

I can only keep repeating myself: separate /usr is not a benefit in itself. If you want to do that to mount it read-only, then you are simply aiming too low, and you should rather work on making / read-only. And you know what? We actually have been working hard on fixing the remaining problems to make this possible. And even enable it by default one of those days.

My recommendation: try to investigate how things really are, and think about them for a while. And then make a smart comment, instead of just calling us idiots. Thank you.

separate /usr isn't just about r/o or NFS

Posted Mar 21, 2011 16:08 UTC (Mon) by gvy (guest, #11981) [Link] (2 responses)

Ever got filesystem corruption (doesn't really need rw, a lucky bad block might suffice)?

My recommendation: try to understand why smart people have been doing things that way before, and think about this for a while. And then try to fix the broken attitude of declaring working things useful for others broken only because you are working hard on solving entirely different thing.

The better folks I know do work on corner cases. "Mainstream" like McDonald's usually just don't bother. If Fedora is all about GNOME desktop only (as discussed already) then this attitude fits pretty well, of course.

separate /usr isn't just about r/o or NFS

Posted Mar 21, 2011 16:32 UTC (Mon) by rahulsundaram (subscriber, #21946) [Link] (1 responses)

If you want to fix the corner cases, you are welcome to. The current issues are no Fedora specific in any way. Your favor distro has it too. As has been pointed out, systemd is merely the messenger.

smells Richard Hughes, btw

Posted Mar 30, 2011 21:00 UTC (Wed) by gvy (guest, #11981) [Link]

> If you want to fix the corner cases, you are welcome to.
Last summer it was a filesystem deadlock. Hope you won't face it now :)

> Your favor distro has it too.
At least not the "udev under /usr" part. The thing is,
- I actually run it with separate /usr and see no problems;
- nothing bothers me about a non-issue for no reason.

> As has been pointed out, systemd is merely the messenger.
It's the messenger which was instructed to, well, lie.

And the instructor seems to have a stance that dumping that all on a user is more sane than going after those who are ignorant enough to introduce that crap in the first place (remember RH#534047?).

Now that's weird. I wouldn't waste time on Mr. Hughes but Lennart is, or at least was for many years (judging as a packager), much different.

r/o root and /etc/udev/rules.d/*persistent*rules

Posted Apr 1, 2011 11:22 UTC (Fri) by gvy (guest, #11981) [Link]

> If you want to do that to mount it read-only, then you are simply aiming
> too low, and you should rather work on making / read-only.
(diggin' within /lib/udev) BTW it's interesting how you're going to do with persistent rules which are essentially the cache (thus belonging to /var, not /etc) but may be crucial to bringing up the stuff to mount said /var, be it a network interface or FC-connected storage (a tape library over here used to perform a streamer dance until fixated that way).

Not a nitpick, it's quite interesting, actually.

Systemd incompatible with...

Posted Oct 3, 2012 9:20 UTC (Wed) by gvy (guest, #11981) [Link] (14 responses)

Seems that the mantra has changed, and now Linux kernel is barely compatible with udev: https://lkml.org/lkml/2012/10/2/303

Lennart, would you please admit that being deaf to users is what hurted KDE4/GNOME3 efforts *big* time and stop advocating Kay no matter what? You're trying to break system layer which isn't easy enough to work around (like switching to XFCE or staying with WindowMaker).

I do understand that Red Hat might have some vision on future init but pushing the result down the throat won't help down the road.

Thank you for all the effort and good code, once upon a time many of your project were my favourite upstreams due to concise info and clean maintenance.

Systemd incompatible with...

Posted Oct 4, 2012 0:33 UTC (Thu) by nix (subscriber, #2304) [Link] (1 responses)

Earlier link: <http://www.spinics.net/lists/netdev/msg185742.html>.

I boggle. Lots and lots of modules need firmware when they initialize. They always have, they always will. Decreeing that 'they should not' doesn't change the reality that they *do*.

Systemd incompatible with...

Posted Oct 4, 2012 15:39 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link]

No, that's a sane requirement. Requesting firmware in module_init() can lead to deadlocks, if userspace can't process the request. It happened to work before because most of time userspace usually just reads firmware from a disk/initramfs.

So it makes sense to do something about it, however definitely not in the way udev developers do it.

Systemd incompatible with...

Posted Oct 4, 2012 0:39 UTC (Thu) by bronson (subscriber, #4806) [Link] (11 responses)

Oof, I'm hoping Kay didn't actually release somewhat broken code without an idea of how he would fix it, but that seems to be what he's saying...?

This looks like an interesting story.

Systemd incompatible with...

Posted Oct 4, 2012 1:14 UTC (Thu) by nix (subscriber, #2304) [Link] (10 responses)

So it appears.

I'm damn glad that Linus and Al appear to agree with the rest of us who value stability in re the ongoing udev trainwreck. I don't *want* a new init system, I don't want to have to make invasive changes to my system on every udev upgrade, I just want something that manages a hotplug /dev for simple cases like usb mass storage plugged in after boot, lets me customize it by adding symlinks, tweaking permissions and the like, and otherwise gets out of the way.

udev used to do this when gregkh maintained it and for a while after, but currently is so far from this that is just not true. The last udev release that added a feature that I actually valued was about 130, but since then almost every release has been terrifying and introduced heaps of incompatible changes for no reason other than that it can, many releases have required invasive changes to the way my system is booted and break backward compatibility so that if I try to downgrade after a successful upgrade *that* will fail to boot, it's halfway to mandating systemd usage... no no no, bring back a udev with maintainership that understands how boot-critical software should be maintained, please. (That is to say, carefully and with great attention paid to backward compatibility and break-free upgrades without the poor distributors and/or upgraders having to whip themselves into a frenzy to do it.)

(Yes, I know I can just run an old udev. That's what I'm doing. But running unmaintained older versions of critical software required for boot because you can't trust the maintainers not to break your boot on every release is not good. Not good at all. Among other things, eventually programs will presumably start requiring a newer libudev...)

Systemd incompatible with...

Posted Oct 4, 2012 9:46 UTC (Thu) by Eckhart (guest, #74500) [Link] (7 responses)

> I just want something that manages a hotplug /dev for simple cases like usb mass storage plugged in after boot

… except that it does not make sense to differentiate between "before boot" and "after boot" at all

Systemd incompatible with...

Posted Oct 11, 2012 11:00 UTC (Thu) by nix (subscriber, #2304) [Link] (6 responses)

Really? People keep saying this but I really don't think it's true. 90% of the devices on every system I own, including all the 'difficult' ones like the ones that are part of RAID arrays and the networking and the framebuffers and the like, are non-hot-pluggable or hotplugged so rarely that I don't care if I have to do something special to make the system pick them up (much less than once a year).

Just about the only hotplugged stuff is USB devices and the block devices that generally hang off them... and udev has worked perfectly well in *that* case for many years. So the cause of the continuing disruptive churn is completely opaque to me. It certainly doesn't benefit me one bit.

Systemd incompatible with...

Posted Oct 11, 2012 14:20 UTC (Thu) by Jonno (guest, #49613) [Link] (5 responses)

Technically all PCIe devices are "hot plugged", at some indeterminate period after it gets power, which it get simultaneously with the main system. If the main OS assumed that all PCIe devices was available at start, that would lead to all kinds of race condition.

The same goes for USB (of course) and SATA, and likely a host other systems...

As an anecdote, I had an oldish SATA device that suddenly stopped being bootable, but worked perfectly once booted from CD.

It turned out that the drive firmware had a boot time of O(n²) where n was the number of bad block relocations. Thus as the drive got old, it began to turn up after BIOS handed over to GRUB, and GRUB didn't have hot plug support :(.

I "solved" the problem by configuring BIOS to always do a full memory check at boot, slowing it down by 5-10 seconds, making booting work 9 cases out of 10. I have since replaced the disc, but it goes to showing that even devices you think are static aren't really...

Systemd incompatible with...

Posted Oct 11, 2012 19:03 UTC (Thu) by nix (subscriber, #2304) [Link] (4 responses)

See, there's a major difference between 'technically' and 'in practice' here. Maybe my PCIe and SATA buses are theoretically hotplugged, but not once in the entire history of any of my systems with any of these buses has any device ever appeared while the OS was running, nor has one ever gone away except on one occasion when a disk failed (so hardly a hotplug event). Nor have I ever heard of systems other than really really high-end boxes (that I could not possibly afford) that can hotplug PCIe (though I have seen machines with external SATA connectors -- always unoccupied in my experience, but possibly useful in emergencies or to connect external drive enclosures, I suppose).

So any recomplication of the system to allow for those possibilities is wasted, from my perspective; and any administrative overhead imposed on me as a system administrator due to changes required by such recomplication is a cost sans benefit, that I would rather not pay.

Systemd incompatible with...

Posted Oct 11, 2012 19:42 UTC (Thu) by raven667 (guest, #5198) [Link] (3 responses)

Maybe I just don't understand but doesn't the fact that these busses are possible to hot-plug and the fact that when it comes to enumeration and probing they behave like hot-plug in that they show up on the bus whenever they become ready mean that the driver systems that handle these devices have to handle the hot-plug case. The infrastructure has to be there to support it so I don't see how you can get rid of the complexity of dealing with these devices as hot-pluggable. It seems a simplification to just handle everything as a hot-plug case, rather than trying to maintain separate static and hot device initialization. infrastructure

Systemd incompatible with...

Posted Oct 11, 2012 19:55 UTC (Thu) by raven667 (guest, #5198) [Link] (2 responses)

That was kind of a crappy comment. Maybe it would be better to say that "in practice" the hardware treats power-on enumeration like a series of hot-plug events which is why it makes sense for the kernel to handle it that way too. "in theory" maybe you could close your eyes and treat the system as static but there would be cases where that would cause you to get wedged that hot-plug wouldn't.

Systemd incompatible with...

Posted Oct 12, 2012 15:31 UTC (Fri) by nix (subscriber, #2304) [Link] (1 responses)

I suppose that's true, if it is common to have PCIe devices that take so long to come up that they haven't started by the time POST has finished. That's not true of any system I own, but since all my systems are desktops and desktops are famously slow to POST...

(Do laptops use PCIe yet? Presumably they do: it's got more power-saving features in it than PCI has.)

I don't actually object to the hotplug-everywhere world, as long as it doesn't break anything much it seems like it might add generality without cost. Unfortunately over the last forty or fifty udev releases (as measured by version numbers) this has not generally been true: every upgrade has, as mentioned elsewhere, been frightening and well above 50% would have broken booting if I hadn't done other things to prevent it. (Those breakages that were due to outright bugs in udev did get fixed fast, but most of them were due to explicit incompatibilities that required sysadmin action. And every one of those was a cost without a benefit. Classic example: udev 174, without advance warning or stated rationale of any kind, moved udevd from /sbin into /lib/udev without leaving behind a compatibility symlink, breaking booting uless you fixed your boot scripts. Because, you know, so many Unix daemons live in subdirectories of /lib or /usr/lib. Oh, wait, none whatsoever do, except for udev. Then udev 175 moved it again (into /usr/lib), and moved udevadm as well, breaking the boot *again* unless you compensated for it. The udev maintainers' motto appears to be "backward compatibility is for other people".)

Systemd incompatible with...

Posted Oct 12, 2012 15:54 UTC (Fri) by raven667 (guest, #5198) [Link]

> (Do laptops use PCIe yet? Presumably they do: it's got more power-saving features in it than PCI has.)

Oh I'm sure that all the internal busses are PCIe and if there is an external ExpressCard slot, that's hot-plug PCIe. When it comes to power management I imagine that some devices drop out and show up on the bus in a hot-plug fashion when they are powered off/on.

> moved udevd [...] without leaving behind a compatibility symlink, breaking booting ...

That's pretty screwed up in any event. I find the logic and evidence compelling for collapsing / into /usr but that is no excuse for not putting in appropriate compatibility measures, measures which should be totally straightforward and transparent.

> The udev maintainers' motto appears to be "backward compatibility is for other people"

That's an unfortunate motto of much (though thankfully not all!) of userspace, it's one area where having source available has lead people to believe that rebuilding and recompiling the world is acceptable so API/ABI compatibility don't matter.

Systemd incompatible with...

Posted Oct 6, 2012 1:03 UTC (Sat) by jackb (guest, #41909) [Link] (1 responses)

I remember the transition from static device nodes to devfs because there were problems with static devices, then shortly afterwards we moved to udev because devfs was fatally, unfixably flawed, then devtmpfs + udev for some reason which isn't particularly clear, now there's a mention in LKML of moving udev into the kernel source tree.

Now wondering if we're ever going back to static device nodes, just to really come back full circle.

Systemd incompatible with...

Posted Oct 6, 2012 1:08 UTC (Sat) by dlang (guest, #313) [Link]

many people never stopped using static device nodes. For systems that don't have the hardware changing on them, a static /dev works very well.

not everything is a laptop with random USB device of the day getting hot-plugged into it.

Systemd incompatible with mounted /usr

Posted Mar 4, 2011 1:06 UTC (Fri) by jmorris42 (guest, #2203) [Link] (5 responses)

> Have a reduced-functionality mode in which systemd
> automatically starts if pieces are missing..

It is worse, they just don't care. Because what is systemd? It is a replacement init that is based on dependencies. All they would have to do is make mounts a resource (And I think they already are) and declare /usr being mounted as one of the resources that a subsystem depends on to start. If that makes a circular dependency loop that won't allow /usr to be mounted it means some files should be in / and the package that supplies them in /usr is broken.

But on the other hand, if the goal is to migrate away from / and /usr there is a debate to be had on that subject, after all it isn't a core UNIX Way thing, it was a side effect from a popular system from the long ago prehistory days that typically had a small fast drive and a larger but slower one.

Systemd incompatible with mounted /usr

Posted Mar 4, 2011 3:50 UTC (Fri) by mezcalero (subscriber, #45103) [Link] (3 responses)

Well, such an automatism is pointless and wouldn't work anyway, since to mount things in the right order at the right time you need udev around, but the udev rules are the ones needed /usr. Hence you would want to mount /usr both before and after starting udev. Which logic shows is not possible.

Systemd incompatible with mounted /usr

Posted Mar 21, 2011 16:22 UTC (Mon) by gvy (guest, #11981) [Link] (2 responses)

# find /usr -name '*udev*'
[.../usr/share/{vim,doc,man}...]
/usr/libexec/ConsoleKit/run-seat.d/udev-acl.ck

And the rest is under /{etc,lib}/udev here on ALT Linux.

Could you please elaborate on "the udev rules are the ones needed /usr", or perhaps just file a bug proper?

> Hence you would want to mount /usr both before and after starting udev.
> Which logic shows is not possible.
Heh. There's a theory for inventor's problems, you might enjoy reading up on that, as well as on formal logic; see at least http://en.wikipedia.org/wiki/TRIZ -- in this particular case, the "visible" controversy is solved by "doing beforehand", that is, moving udev rules from /usr into root filesystem. No rocket science at all.

Systemd incompatible with mounted /usr

Posted Apr 18, 2011 20:24 UTC (Mon) by nix (subscriber, #2304) [Link] (1 responses)

Some rules (a very few, but perhaps more in time) want the PCI and USB ID databases, as well. Of course this should be fixed by eliminating /usr and annoying everyone who has to reinstall their entire systems, rather than by, I dunno, introducing /share, moving the ID databases there, and leaving a symlink where they came from.

As you know, "do things the hugely inconvenient way" is the new Linux slogan.

Systemd incompatible with mounted /usr

Posted Apr 19, 2011 1:06 UTC (Tue) by jrn (subscriber, #64214) [Link]

I thought the drive to make a /usr -> / symlink possible came from GNU Hurd developers in the hope of making PATH=/bin a viable configuration (and thus simplifying the concept of path resolution for users)?

Systemd incompatible with mounted /usr

Posted Mar 21, 2011 16:11 UTC (Mon) by gvy (guest, #11981) [Link]

> a small fast drive and a larger but slower one.
Did you just describe modern choices like SSD/HDD, SLC/MLC, especially in embedded/portable hardware, as prehistoric? ;-)

Systemd incompatible with mounted /usr

Posted Mar 3, 2011 19:59 UTC (Thu) by mezcalero (subscriber, #45103) [Link] (12 responses)

Wow, if you call me an idiot, then I'll call you an ignoramus: Ignoramus!

This has nothing to do with systemd. systemd is just the messenger, telling you what has been the status quo since quite some time. Don't shoot the messenger!

systemd itself has no issues with running with a separate /usr. We have been very careful in distributing our stuff among / and /usr to make boots with separate /usr possible. But uh, systemd does not exist in a vacuum, and it won't magically fix all other software for you.

If anything at all you should congratulate us systemd developers for the fact that we spent so much time on detail work to actually make systemd itself work fine with separate /usr. And then we added the extra cherry on top in that we point you to the fact that in general this probably won't work well for you due to other programs.

And we did all this "separate-/usr-support" work for you even though we personally think splitting off /usr is a useless and pointless exercise.

I am waiting for your excuse,

Love,

Lennart

Systemd incompatible with mounted /usr

Posted Mar 3, 2011 20:09 UTC (Thu) by mezcalero (subscriber, #45103) [Link]

s/excuse/apology/

Systemd incompatible with mounted /usr

Posted Mar 3, 2011 20:54 UTC (Thu) by DOT (subscriber, #58786) [Link] (4 responses)

Are you at all surprised that people are ignorant of software they didn't write themselves? Yes, you know that your warning is not what it appears to be, but that's because you are the systemd developer. With the caveat that nobody owes anybody anything in open source land, it's your job to make us understand the software you wrote.

Systemd incompatible with mounted /usr

Posted Mar 3, 2011 21:38 UTC (Thu) by drag (guest, #31333) [Link]

Yes. The warning is confusing and misleading. :)

Systemd incompatible with mounted /usr

Posted Mar 3, 2011 22:52 UTC (Thu) by AndreE (guest, #60148) [Link] (2 responses)

Is it your job to also understand the problems of udev? Or is that the job of systemd developers to point that out to you? Did the systemd developers write udev?

Yes, the message could have been worded better, but it seems that the underlying problem would be known to anyone who is already using udev and having a separate /usr/ partition.

Systemd incompatible with mounted /usr

Posted Mar 3, 2011 23:23 UTC (Thu) by DOT (subscriber, #58786) [Link] (1 responses)

Systemd doesn't have to warn in the first place; that's a nice extra. But whatever it says, should be clear. The current message reads like systemd will blow up or silently kidnap your first-born when using a separate /usr. I admit that I was pleasantly surprised to be pointed to an explanation in the README. I also tried to avoid passing judgment. I simply observe that people are understandably ignorant about software they don't have an intimate relationship with. Said ignorant people should obviously not call developers idiots either.

I think I've now managed to piss off all participants in this thread. Sorry about that.

Systemd incompatible with mounted /usr

Posted Mar 3, 2011 23:24 UTC (Thu) by nix (subscriber, #2304) [Link]

That's what I thought, at first. I knew udev has certain limited problems with a separate /usr (no such problems for me: I don't need any of the udev extras that rely on things like the PCI db). But the failure message never mentioned udev at all, so I assumed systemd had *more*, unique problems.

And if there's one thing that I don't like to experiment wildly with, it's init(8).

Systemd incompatible with mounted /usr

Posted Mar 3, 2011 23:08 UTC (Thu) by nix (subscriber, #2304) [Link] (5 responses)

I'm an ignoramus, agreed :)

If systemd has no problems and all the problems are with stuff it relies on breaking randomly because /usr is separate, that's fine. I thought systemd was explicitly relying itself on stuff in /usr, which for an init(8) replacement of all things seemed terribly, terribly unwise. If there's anything which has to work when the system is not fully up, it's init(8)!

(As for why separate /usr is necessary, I've had several reasons in the past. Lots of diskless systems with fast networking and very little storage is one of them: / on the limited available storage speeds things up a *lot* because /lib is local, but there's nowhere near enough room for /usr to go on there, it's huge. Not putting all your eggs in one basket is another reason: more than once I've had FS damage blow away /usr, but / and things like /etc were unaffected so reconstruction was easy. If /usr and / had been integrated, I'd have been visiting backup land.)

Systemd incompatible with mounted /usr

Posted Mar 4, 2011 11:15 UTC (Fri) by jospoortvliet (guest, #33164) [Link] (4 responses)

http://freedesktop.org/wiki/Software/systemd/separate-usr...

Systemd incompatible with mounted /usr

Posted Mar 4, 2011 12:43 UTC (Fri) by nix (subscriber, #2304) [Link] (3 responses)

I'm looking at that page and trying to figure out why anyone would want to run CUPS, PulseAudio, or gnome-color-manager when /usr wasn't mounted (i.e., in very early boot or emergency recovery mode). I can't think of a single reason :)
/ usually requires a similar amount of disk space as /usr does these days
What? 
Filesystem            Size  Used Avail Use% Mounted on
/dev/main/root       1008M  120M  838M  13% /
/dev/mapper/main-usr  197G   31G  157G  17% /usr
I suspect that this is only true if you have a massively modular kernel or half the world in /lib. It is most definitely comprehensively extremely not true for any system I have access to... other than Fedora and Ubuntu systems, and even for them, the relative space requirements are far from parity.

A look on a random Fedora 14 system shows 581Mb in /, more than half in kernel modules, and 4.26Gb on /usr. An eightfold difference, still pretty significant: a sixteenfold difference if it wasn't for the modular kernel.

harddisks are substantially bigger these days
Not all machines are personal desktops. (Assuming that all machines are desktops is a reasonable assumption for PulseAudio: it certainly is not for an init replacement). That Fedora install I just mentioned: its /usr would blow my firewall's total storage four times over. (It runs off an SD card. They are not large. Mind you, that's one machine where I didn't bother with a / versus /usr split: there was simply no point.)

It seems very odd to add this message to systemd if systemd is not at fault. If things exist on /, they shouldn't rely on things in /usr for some degree of function (sure, localization might break, but the program still works-in-emergencies without that.)

Systemd incompatible with mounted /usr

Posted Mar 4, 2011 15:02 UTC (Fri) by mezcalero (subscriber, #45103) [Link] (2 responses)

Well, I am not sure I can be more explicit than in the wiki page. But here's another try to explain this to you in simple words:

1) applications install udev rules

2) these are executed at early boot when the hw is detected

3) these rules fail, since /usr is not around

4) later on the apps won't see the devices properly, or can't see some of their features since the rules didn't get executed, and the udev devices lack the properties that should normally have been set had the rules been executed properly.

Systemd incompatible with mounted /usr

Posted Mar 4, 2011 16:06 UTC (Fri) by nix (subscriber, #2304) [Link]

Yep, that's exactly what I thought. All the applications which might fail because usr/ is not around are various udev extras, principally usb_db and pci_db. The default rules use these for something to do with Dell Bluetooth devices and to dig vendor descriptions out of the USB/PCI database. In fact, looking at the udev 164 source code, it looks like the stuff derived from the USB/PCI database lands in device properties then is never used for anything...

... ah. I see. PulseAudio uses it, nice example of an unadvertised coupling between programs, that. What does it use it for?

... to populate ZeroConf / Bonjour data, and to set vendor name properties on sources and sinks. Is that *all*? You're emitting a scary warning and triggering flamewars over *that*? Small wonder I never noticed it wasn't working. I hope you're planning to use it for something else in future, or this has really been a tempest in a teapot.

(I note that mapping USB/PCI IDs to vendors is something perfectly doable at runtime, when the system is likely to be up and /usr will be mounted: there's no need to do it at coldplug time. It's not like the USB or PCI databases are root-readable-only or anything like that, and the underlying IDs are stuck in the udev database regardless. However, it might require enhancements to libudev to do that, which is more coupling than the existing 'run usb_id/pci_id' approach.)

Systemd incompatible with mounted /usr

Posted Mar 21, 2011 17:29 UTC (Mon) by gvy (guest, #11981) [Link]

Here in ALT Linux, there's "early udev" in initramfs which is supplied with limited rules enough to bring up rootfs, and it's only used to populate initial /dev then stopped.

"Real" init can start proper udev with full blown rules later, and I can testify that *not* starting it -- or, again, starting to further populate /dev and then stopping it -- did make it possible for ALTSP5 thin client to fit into 16M RAM on a terminal (thus being useful).

Sure /usr is integral to / on a diskless thin client but hope that a use case like that might help to understand that there is awful lot more of uses for init(8) and udev(8) out there than can be imagined in a hurry.

As for initrd machinery, one is invited to have a look at http://en.altlinux.org/Make-initrd (original page in Russian at http://www.altlinux.org/Make-initrd).

PS: thanks for ifplugd :)

Systemd incompatible with mounted /usr

Posted Mar 3, 2011 20:30 UTC (Thu) by drag (guest, #31333) [Link]

> Wow, and just when I was beginning to think it was time to start looking into systemd. Thanks for the blast of sanity!

Yes.

Because when Systemd hints to you how your system works and has been working for years now it's Systemd's fault for shattering your illusions.

Quotes of the week

Posted Mar 3, 2011 8:02 UTC (Thu) by Wummel (guest, #7591) [Link] (11 responses)

The /usr warning has no information in it whatsoever.
It would have been nice to explain the problems with mounted /usr and systemd.
This way people would have a chance to fix it.

Now the remaining choice for distributions seems to be to avoid systemd.

Quotes of the week

Posted Mar 3, 2011 19:48 UTC (Thu) by mezcalero (subscriber, #45103) [Link] (10 responses)

Amazing, this ignorance!

systemd doesn't change anything here. There's no reason to avoid it. systemd itself will work fine if /usr is separate. However, a lot of stuff that we ship by default on most Linux distributions these days won't. We just warn about that *fact*.

So, let me repeat this and make this clear:

A) systemd itself works fine with seperate /usr

B) if you want to fix this, then fix the specific programs, which are not systemd

C) this is a statement of fact, on the status quo. It is not a change in any way. It isn't news in any way.

D) this is just a warning. You may ignore it if you wish.

Quotes of the week

Posted Mar 4, 2011 11:19 UTC (Fri) by jospoortvliet (guest, #33164) [Link] (7 responses)

You know, Lennart, I love you ;-)

Of course in a very platonic way as my girlfriend would probably kill you if things were otherwise.

You combine a lovely straightforward (should I say German?) communication style with being annoyingly (to many) right at least most of the time...

Oh and the page on FD.o http://freedesktop.org/wiki/Software/systemd/separate-usr... is awesome in this regard :D

As far as I am concerned, you can keep this up. It amuses me to see ppl get railed up over nothing all the time and you telling them (rightly so) what idiots they are, hehe...

Quotes of the week

Posted Mar 4, 2011 12:45 UTC (Fri) by nix (subscriber, #2304) [Link] (2 responses)

"I upgraded to systemd and now my system is throwing disturbing warnings" is hardly "nothing". The wiki link fixes everything: now we can see what the real problem is, and determine whether it applies to us.

Quotes of the week

Posted Mar 4, 2011 15:22 UTC (Fri) by Wummel (guest, #7591) [Link] (1 responses)

Yes, the Wiki link clarifies the warning message. The misleading warning text was the reason for the misunderstanding that most people (including me) had.

So the real problem occurs when
a) systemd queries udev information which executes a script in /usr and
b) /usr is not yet mounted.

One suggested solution is to move scripts from /usr/bin to /bin, another solution is to wait until /usr is mounted.

Hopefully, there will soon be concrete bug reports with the offending udev rules so that those problems can be fixed.

Quotes of the week

Posted Mar 4, 2011 19:32 UTC (Fri) by nix (subscriber, #2304) [Link]

No, systemd is executing a program in /lib/udev. The problem is that a few programs in /lib/udev depend on glib (which is often in /usr/lib) and a few depend on the USB or PCI databases under /usr/share. Without /usr, those databases are inaccessible, so everything which needs them will fail during coldplugging.

Quotes of the week

Posted Mar 4, 2011 14:38 UTC (Fri) by mezcalero (subscriber, #45103) [Link] (3 responses)

I never publicly called anybody an idiot.

Quotes of the week

Posted Mar 4, 2011 15:28 UTC (Fri) by Wummel (guest, #7591) [Link]

And I agree. You explained the warning message and the misunderstanding is cleared up.
Let's move on to fixing the technical problems that lead to the warning message.

Quotes of the week

Posted Mar 4, 2011 15:51 UTC (Fri) by nix (subscriber, #2304) [Link]

No, that was me!

Moral: never post after four hours of commuting. (Of course, if I'd followed that advice for the last four years I'd never have posted at all.)

Quotes of the week

Posted Mar 10, 2011 17:59 UTC (Thu) by welinder (guest, #4699) [Link]

> I never publicly called anybody an idiot.

That statement is not true. It's true for this thread, but not
true as written. Try googling yourself a bit.

For example:

http://www.mail-archive.com/pulseaudio-discuss@mail.0poin...

Quotes of the week

Posted Mar 4, 2011 12:31 UTC (Fri) by NAR (subscriber, #1313) [Link] (1 responses)

this is just a warning. You may ignore it if you wish.

That is a bad user interface design. Shove some stuff to the user's face that he can (and in most cases should) ignore. This leads to users ignoring all warnings in the future. If it's not relevant - don't show it. If systemd itself works fine, but the interaction with udev leads to an error - produce a meaningful error message then.

Quotes of the week

Posted Mar 4, 2011 12:36 UTC (Fri) by Darkmere (subscriber, #53695) [Link]

The issue is rather that the failure mode is hard to decipher and debug. Due to fallback and "robustness" of the stack, rather than breaking horribly when things are not where expected, much of this software will shut up and move on, doing it's best to avoid your system being unusable.

However, when you end up with a net result that mostly does not function, you have to get your warning from somewhere, and getting it from the init process is as good as any.

Also, remember that the kind of user who'd put /usr on it's own partition is not likely the user who'd ignore the errors or click through them. It's a user who's technically savvy, has enough knowhow of the oldschool things to commit to such a task, and as such, are well served by this warning.

Alvaro Ortega : Now that we all are running fancy desktops full of powerful applications

Posted Mar 3, 2011 10:31 UTC (Thu) by roblucid (guest, #48964) [Link] (5 responses)

"Now that we all are running fancy desktops full of powerful applications.. what sense does it make to have to open a terminal window, become root, and edit a text file, and type a new command to reload the service in order to make a little change on how your web server behaves."

What about the guys, installing and supporting this powerfull software applications across 100's of desktops, or wanting reliable replicated servers?

It makes no sense to diddle about in a GUI on 100's of machines, when you can write a script once to edit a config file, and reload the service remotely and have it automatically distributed.

Alvaro Ortega : Now that we all are running fancy desktops full of powerful applications

Posted Mar 3, 2011 10:57 UTC (Thu) by Los__D (guest, #15263) [Link] (1 responses)

As I read it: It is not the configuration files that is his problem: It is when no GUI frontend for updating them (and restart the proper services) exist.

- Which I agree with, up to a point. For some things, text editing is simpler.

Alvaro Ortega : Now that we all are running fancy desktops full of powerful applications

Posted Mar 3, 2011 21:35 UTC (Thu) by drag (guest, #31333) [Link]

It probably has less to do with GUI-ness then automation, security, and sanity checking. GUI is part of it usually, but not necessarily.

For example....

With Libvirt I can use GUI front ends, but I can also use 'virsh' to control it.

I use the GUI for common everything stuff, but for advanced features or unusual configurations I need to edit configurations by hand:

virsh dumpxml vm-name > vm-name.xml
vi vm-name.xml
virsh define vm-name.xml

I don't need to be on the local machine either. I can use ssh or SSL/TLS to provide remote access to that particular service. I don't need to be root, nor does it really depend on me being able to know the IP address or DNS name or whatever. It can be configured ahead of time by some other person and it can just be my job to manage the VMs.

The libvirtd does sanity checking on the xml files that get uploaded and, ideally, prevents typos and mis configurations from taking down important virtual machines or screwing up other machines.

This sort of thing is good. It works on a local standalone node, or it can scale across hundreds of machines effectively. Makes scripting easier. It can be used to delegate responsibilities to individuals, control access, help keep audit trails, and all sorts of other fun stuff. Delegation and specialization is important part of managing systems in a large organization.

The GUI is just part of it.

Flat text files are indeed very simple and relatively easy to deal with as long as they are documented well. The problem is that they don't work very well in complex environments and dynamic setups. At least not by themselves. You need other mechanisms to be able to solve problems like race conditions (as a example). The other problem is lack of uniformity. It's really difficult to write a script or a program that can deal with all the differences methods and formats that different Linux services use.

Makes automation very difficult and automation is the key to scalability and correctness. Without it your stuck with a very labor intensive and error prone processes. Going "cd /etc/ ; sudo vi foo/blah.cfg; service restart foo" is simple, but it is not terribly useful when dealing with 400-500 nodes... nor is it pleasant when its a configuration change that commonly happens on your desktop or laptop (such as giving a WPA password to a new network when your travelling).

Alvaro Ortega : Now that we all are running fancy desktops full of powerful applications

Posted Mar 5, 2011 20:36 UTC (Sat) by kleptog (subscriber, #1183) [Link]

The part I find best about editing files is the permanence. When you edit a configuration is a file you know it'll be there next boot.

On the other hand, most GUI tools don't even try to explain. This setting you're changing, is it just for this session or forever? When I change the volume on my computer it seems to remember this across boots. There's no distinction between defaults after booting and what you want the settings to be now.

That said, I'm not sure if there's a nice solution for this. Which brings you right back to: when you edit a file, you knows it's forever.

Alvaro Ortega : Now that we all are running fancy desktops full of powerful applications

Posted Mar 21, 2011 17:41 UTC (Mon) by gvy (guest, #11981) [Link] (1 responses)

Reliable you say... while posting a comment at the linked blog, got:

500 Internal Server Error
Cherokee web server 1.2, Port 80

I'd expect a bit better of ex-"Technical Lead en Sun Microsystems, High Performance / High Availability Engineer en Espanix", frankly. And FWIW the comment was:

---
"I’d say that none, none at all" -- if that would be the case, then typing all those letterses into scripts making up those very webapps would make no sense either. Folks might as well just do everything by hand in those pesky desktop applications not suited for any sane automation... but you were just trolling cheap, right? ;-)

BTW there is already apsstandard.org -- hope it's not news to folks over there.
---

Groundless Hype?

Posted Apr 6, 2011 11:53 UTC (Wed) by Pierre@G-WAN (guest, #74111) [Link]

And it's not like if Cherokee resolved the problem: it STILL uses opaque configuration files.

By contrast, G-WAN is much faster (without *any* configuration file):

http://gwan.ch/imgs/gwan_nginx_cherokee.png

Cherokee might rock... but maybe in another area:

http://forum.gwan.com/index.php?p=/discussion/182/cheroke...

Oh, and before we are (again) censored by Cherokee fan boys (like on Wikipedia*), here are the "Cherokee, the fastest Web server" facts checked by an independent source:

http://nbonvin.wordpress.com/2011/03/14/apache-vs-nginx-v...

Pierre.

(*): http://forum.gwan.com/index.php?p=/discussion/comment/150...

Quotes of the week

Posted Mar 3, 2011 17:19 UTC (Thu) by foom (subscriber, #14868) [Link] (2 responses)

Wow, such negativity for systemd here! It's warning you about a bug. Oh no, end of the world.

Quotes of the week

Posted Mar 3, 2011 18:09 UTC (Thu) by nix (subscriber, #2304) [Link] (1 responses)

It's not warning you about a bug: it's telling you that bug will never be fixed. That's a bit different.

Quotes of the week

Posted Mar 3, 2011 19:13 UTC (Thu) by foom (subscriber, #14868) [Link]

Well, from the email thread, it sounds like it sorta-kind-pretty-much works, and that they'd be perfectly happy for *you* to fix any such issues...just that they're tired of tracking down such things.

E.g., Key wrote:
> The most prominent /usr failure in udev land is unconfigured audio and
> 3G network cards. And the usual response is: "I don't need that.". :)

/etc/mtab as a file not supported?

Posted Mar 3, 2011 18:26 UTC (Thu) by ebiederm (subscriber, #35028) [Link] (2 responses)

Blink.

/etc/mtab as a symlink to /proc/mounts only works in the simple cases.

I recently had to change /etc/mtab from a symlink back to a file, on my systems fedora 14 systems because things don't work if /etc/mtab is a symlink to /proc/mounts.

In particular I have bind mount /home from another partition and on top of that I have nfs mounts for a few users directories, and without out /etc/mtab being a file the directories were mounted in the wrong order.

When /etc/mtab is a symlink df shows my mounted filesystems multiple times (because of my bind mounts) but when /etc/mtab is a file I see each mounted filesystem only once.

/etc/mtab as a file not supported?

Posted Mar 3, 2011 19:41 UTC (Thu) by mezcalero (subscriber, #45103) [Link] (1 responses)

Please file bugs against util-linux regarding issues like this. mount has been modified to store the meta information it needs on top of what the kernel stores anway in /dev/.mount/. If there's something missing then please report this on bugzilla, not on LWN. Thanks!

/etc/mtab as a file not supported?

Posted Oct 5, 2012 12:34 UTC (Fri) by rleigh (guest, #14622) [Link]

It should be under /run/mount along with utab, surely? We moved all use of dotfiles in /dev to /run; is this an exception which needs addressing?

Quotes of the week

Posted Mar 4, 2011 14:30 UTC (Fri) by mezcalero (subscriber, #45103) [Link]

BTW, I modified the warning in systemd git now to point users to http://freedesktop.org/wiki/Software/systemd/separate-usr... where we try to explain the situation in more detail.

systemd

Posted Mar 10, 2011 7:28 UTC (Thu) by m.galabant (guest, #73106) [Link] (2 responses)

I've been living with perfectly functioning Linux installations without all that udev-systemd-messagebus-whattheyarecalledtoday -- crap comfortably.
My /dev is fully populated and I don't care about a few inodes and MB of filesystem used by that (good filesystem don't have an inode limit anyway). My machines do always boot even if random stuff is missing or corrupt (and they won't with udev etc.). I've converted all CentOS/RH machines back to the good old style /dev system and it's been a relief: no more initrd, just boot the damn kernel via whatever you like. The startup is faster, too. All in all the system is simply robust.

Separating system directories (like /usr and /var) is what has made Unix great. It even helps performance (because of lesser locking) and fragmentation (yes, this is a problem on a write-intensive FS), let alone robustness (one FS can corrupt easily, but not 3 at the same time).
Being told by a freaking little program and their developers (who don't "believe" in separation) that I cannot do that is just ridiculous.
They've had their head too long in that "always on" mentality; they've not grasped proper System V init; they've forgotten about system init (not yet all mounts and maybe R/O) vs. system running (all's there) phases, which is a base paradigma of all technical systems.

I express my shock that those people are allowed to change the future of how we will be running our Linuxes; they maybe should get back to working on Ubuntu, where the "en vogue" seems to trump robust working concepts, and leave the rest of the world alone, just alongside with the Gnome people for removing important window manager buttons from their console.
We don't like to play with our systems, we like to run them in whatever hell breaks loose upon us.

systemd

Posted Mar 10, 2011 17:02 UTC (Thu) by foom (subscriber, #14868) [Link] (1 responses)

I think maybe you should've maybe read the rest of the comments before posting your own rambling uninformed commentary.

systemd

Posted Mar 10, 2011 21:46 UTC (Thu) by nix (subscriber, #2304) [Link]

I was particularly impressed by this comment:
Separating system directories (like /usr and /var) is what has made Unix great.
Not even slightly. "Everything is a file", anyone?


Copyright © 2011, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds