LWN: Comments on "SELF: Anatomy of an (alleged) failure"

SELF: Anatomy of an (alleged) failure

ShannonG — Fri, 04 May 2012 19:10:11 +0000

This is why the kernel-mailing is hostile.

SELF: Anatomy of an (alleged) failure

nix — Sat, 10 Jul 2010 20:24:03 +0000

dlopen() doesn't search directories for you, does it? Programs generally want to look in a subdirectory of the libdir, anyway. Nonetheless they almost all look in the right place.

SELF: Anatomy of an (alleged) failure

makomk — Sat, 10 Jul 2010 12:31:20 +0000

Yeah, dlopen() problems with not finding libraries in /lib32 don't tend to happen, mostly because it's just easier to do it the right way from the start and let dlopen() search the appropriate directories. (Even on pure 32-bit systems, some libraries are in /lib on some systems, /usr/lib on others, and perhaps even in /usr/local/lib or $HOME/lib if they've been manually installed.)

Where to rejected patches go?

dlang — Tue, 06 Jul 2010 09:35:11 +0000

the biggest problem is that there is no one place where rejected patches are ever together today.

there are many ways patches can be sent, to many different people (and in some cases the patches themselves aren't sent, just a request to pull from a git repository that may disappear in a few days)

it would be a significant amount of work to gather such patches, let alone maintain them.

there are patches like this today, but they get hosted and maintained by the people who care about them.

Where to rejected patches go?

milki — Fri, 02 Jul 2010 07:33:50 +0000

Btw, I've wondered this for a while. Back in 2.0 days, there was a small repository of applyable patches (iBCS2, PC speaker sound card emulation, dual monitor support and Co.), which weren't included in the mainline kernel. The externel hosting was never an official feature, though.

But what about today? Where do patches go if they are rejected? Is everything just thrown away?? Shouldn't there be a public review and archival repository somewhere? Or even a separate GIT for incoming patches? (Can't believe it's all sent per email.)

If use contributed patches don't make it INTO the kernel, it would help if they were at least AVAILABLE somewhere. Otherwise there is little chance for user lobbying, or at least patching your own kernel.

SELF: Anatomy of an (alleged) failure

eduperez — Thu, 01 Jul 2010 07:45:02 +0000

> What would be interesting would be a bytecode architecture, like some sort of LLVM IR, compiled via a JIT at run time.
> This way the same "executable" would run on ARM, x86-32, x86-64...

You man, like... Java?

Disk space (was: SELF: Anatomy of an (alleged) failure)

peter-b — Tue, 29 Jun 2010 23:03:17 +0000

So does POSIX sh. The following command is equivalent to true:

The following command is equivalent to false:

! :

I regularly use both when writing shell scripts.

SELF: Anatomy of an (alleged) failure

salimma — Mon, 28 Jun 2010 16:13:50 +0000

GNUstep? also, the ROX (RISC OS on X) Desktop.

SELF: Anatomy of an (alleged) failure

rjw@sisk.pl — Mon, 28 Jun 2010 15:09:14 +0000

Well, in fact none of the examples given in the original article are about code submitted by a distribution. All of them, and also the ones from your previous comments, fall into the common category where someone, apparently from the "outside", brings us a feature (not being a device driver) complete with bells and whistles and wants us to take it. People generally don't react well to that, which is not surprising (to me at least).

Moreover, such "complete features" often do much more than is really necessary to address the particular problem their submitters had in mind when they started to work on them. In many cases this "extra stuff" makes them objectionable. In some other cases they attempt to address many different problems with one, supposedly universal, feature which confuses things. It also often happens that the feature submitters are not willing to drop anything or redesign, because of the amount of work it took them to develop their code, so the objections cause the entire feature to be rejected eventually.

Now, if you do something that people are not going to react well to and you give them good technical reasons to object to it, you shouldn't be surprised too much when it fails in the end, should you?

SELF: Anatomy of an (alleged) failure

nix — Mon, 28 Jun 2010 13:25:25 +0000

Well, that's quite different from my experience (it fired once and demanded I phone a number where a licensing goon tried to extract the cost of an entire Windows license from me despite my giving them a key: 'that key is no longer valid because WGA has fired', wtf?).

I suspect that WGA's behaviour (always ill-documented) has shifted over time, and that as soon as you hit humans on phone lines you become vulnerable to the varying behaviour of those humans. I suspect all the variability can be explained away that way.

Still, give me free software any day. No irritating license enforcer and hackability both.

SELF: Anatomy of an (alleged) failure

nix — Mon, 28 Jun 2010 13:22:27 +0000

The attitude appears to be 'distributors don't need them therefore they are useless'. This seems, to me, more than a little shortsighted...

SELF: Anatomy of an (alleged) failure

nye — Mon, 28 Jun 2010 10:03:54 +0000

For the record, my experience of WGA is as follows:

I've never actually *seen* WGA complain about a hardware change; the only times I've ever seen it are when reinstalling on exactly the same hardware (eg 3 times in a row because of a problem with slipstreaming drivers).

In principal though, if you change more than a few items of hardware at once (obviously this would include transplanting the disk into another machine) or whenever you reinstall then Windows will ask to be reactivated. If you reactivate too many times over a short period, it will demand that you call the phone number to use automated phone activation. At some point it will escalate to non-automated phone activation where you actually speak to a person. This is the furthest I've ever seen it go, though I believe there's a further level where you speak to the person and you have to give them a plausible reason for why you've installed the same copy of Windows two dozen times in the last week. If you then can't persuade them, this would be the point where you have to pay for a new license.

This is obnoxious and hateful, to be sure, but it is entirely unlike the behaviour described. The half-truths and outright untruths directed at Windows from some parts of the open source community make it hard to maintain credibility when describing legitimate grievances or technical problems, and this undermines us all.

SELF: Anatomy of an (alleged) failure

Tet — Mon, 28 Jun 2010 08:53:04 +0000

Oh agreed, and in this particular case, it's not necessary. However, there are other situations where full fat binaries might be a win. I just get extremely annoyed by people claiming that the whole concept of multi-arch binaries is useless just because they happen to not have a valid use for them, and are unable to see that others might have.

SELF: Anatomy of an (alleged) failure

nix — Sun, 27 Jun 2010 20:36:15 +0000

Yes, but if that's all you need to do, the kernel side of FatELF is superfluous.

SELF: Anatomy of an (alleged) failure

Tet — Sun, 27 Jun 2010 17:55:02 +0000

So we don't need completely arbitrary fat binaries at all: we need a 'fat dlopen()'

To solve this particular problem, yes. But then Ryan's FatELF release supported dlopen()ing fat shared libraries.

SELF: Anatomy of an (alleged) failure

nix — Sun, 27 Jun 2010 17:48:31 +0000

Well, DRI is a whole different kettle of worms. I suspect a problem with your OpenGL implemementation, unless Google Earth has a statically linked one (ew).

(Words cannot express how much I don't care about statically linked apps.)

SELF: Anatomy of an (alleged) failure

da4089 — Sun, 27 Jun 2010 17:34:01 +0000

> The reason why Apple invented FAT binaries is because they were interested
> in maintaining extensive binary compatibility with their old systems.

Actually, Apple inherited fat binaries in Mach-O from NeXT.

NeXT supported fat binaries because NeXTSTEP (and later OpenStep) was available for multiple CPU architectures (68k, x86, SPARC32, PARISC), and they wanted to enable ISVs to ship binary applications that worked on all platforms.

To make that work effectively, they
a) Maintain ABIs, use weak-linking, etc
b) Distribute applications as a bundle
c) Support fat binaries

This is a viable approach, as Apple has recently demonstrated.

But it's a very different model to the usual Linux distribution, which needs none of those things, and relies on the dependency resolution of the packaging system and rigorous testing of API compatibility when building the consistent package set.

I don't think attempting to move Linux towards the NeXT/Apple model is useful, but I also don't see why those that want to can't maintain an out-of-tree patch to make it work.

SELF: Anatomy of an (alleged) failure

nix — Sun, 27 Jun 2010 17:17:48 +0000

FatELF wouldn't help with OSX anyway: OSX doesn't use ELF.

SELF: Anatomy of an (alleged) failure

nix — Sun, 27 Jun 2010 17:15:11 +0000

Ah, right. So we don't need completely arbitrary fat binaries at all: we need a 'fat dlopen()'.

I suspect -- though it's a kludge -- you could do this with an LD_PRELOADed wrapper around dlopen() which tweaks the filename appropriately, and slight changes to the downloading parts of e.g. firefox to put its dynamically loaded stuff in per-arch subdirectories when autodownloaded. dlopen() is not a hidden symbol so should be vulnerable to interposition.

Disk space (was: SELF: Anatomy of an (alleged) failure)

nix — Sun, 27 Jun 2010 16:46:15 +0000

The size, btw, is probably because the gnulib folks have found bugs in printf which the glibc folks refuse to fix (they only cause buffer overruns or bus errors on invalid input, after all, how problematic could that be?) so GNU software that uses gnulib will automatically replace glibc's printf with gnulib's at configure time. (That this happens even for things like /bin/true, which will never print the class of things that triggers the glibc printf bugs, is a flaw, but not a huge one.)

And gnulib, because it has no stable API or ABI, is always statically linked to its users.

26Kb for a printf implementation isn't bad.

Disk space (was: SELF: Anatomy of an (alleged) failure)

nix — Sun, 27 Jun 2010 16:42:37 +0000

There are two separate binaries because the GNU Project thinks it is confusing to have single binaries whose behaviour changes depending on what name they are run as, even though this is ancient hoary Unix tradition. Apparently people might go renaming the binaries and then get confused when they don't work the same. Because we do that all the time, y'know.

(I think this rule makes more sense on non-GNU platforms, where it is common to rename *everything* via --program-prefix=g or something similar, to prevent conflicts with the native tools. But why should those of us using the GNU toolchain everywhere be penalized for this?)

SELF: Anatomy of an (alleged) failure

Tet — Sun, 27 Jun 2010 16:11:48 +0000

<paxman>
Answer the question!
</paxman>

Your "solution" doesn't solve the problem where application plugins are concerned. Firstly, the majority them are not installed using the system package manager in the first place, and secondly, it's utterly irrelevant anyway. You can't package the achitecture specific bits separately, because the application only looks for them in one place. As I said right at the start, it would be good to fix the applications, but there are a hell of a lot of them. Fat binaries would solve the problem. Your suggestions wouldn't, without first also patching the apps.

SELF: Anatomy of an (alleged) failure

nix — Sun, 27 Jun 2010 16:03:46 +0000

IIRC this is currently being done by ClamAV (using LLVM, natch).

SELF: Anatomy of an (alleged) failure

cortana — Sun, 27 Jun 2010 13:14:04 +0000

I'm very happy that you did not run into this problem, but I have. IIRC it was with Google Earth. strace clearly showed it trying to dlopen some DRI-related library, followed by it complaining about 'wrong ELF class' and quitting.

SELF: Anatomy of an (alleged) failure

nix — Sun, 27 Jun 2010 12:31:27 +0000

Even if Debian did have an automatic setup for compiling all library packages with both architectures, you are then screwed because they put the amd64 libraries in /lib (with a symlink at /lib64) and the i386 libraries in /lib32. So your proprietary i386 software that tries to dlopen files in /lib fails because they are of the wrong architecture.

I've run LFS systems with the /lib / /lib32 layout for many years (because I consider /lib64 inelegant on a principally 64-bit system). You know how many things I've had to fix because they had lib hardwired into them? *Three*. And two of those were -config scripts (which says how old they are right then and there, modern stuff would use pkg-config). Not one was a dlopen(): they all seem to be using $libdir as they should.

This simply is not a significant problem.

SELF: Anatomy of an (alleged) failure

nix — Sun, 27 Jun 2010 12:14:24 +0000

More like a sort of really crippled and inflexible FreeBSD, with all ports forced to update only when the OS has a major version number bump: if you want a bugfix you have to wait for another giant mass of features to land. Great idea, not.

SELF: Anatomy of an (alleged) failure

nix — Sun, 27 Jun 2010 12:12:38 +0000

Well, to be charitable, WGA is an appalling intentionally-user-hostile mess that MS keep very much underdocumented, so it is reasonable to believe that this is what WGA does without being a liar. One could simply be mistaken.

(Certainly when WGA fires, it does make it *appear* that you have to reinstall the OS, because it demands that you pay MS a sum of money equivalent to a new OS install. But, no, they don't give you a new OS for that. You pay piles of cash and get a key back instead, which makes your OS work again -- until you have the temerity to change too much hardware at once; the scoring system used to determine which hardware is 'too much' is documented, but not by Microsoft.)

SELF: Anatomy of an (alleged) failure

nix — Sun, 27 Jun 2010 12:08:41 +0000

* Currently; If you do not need 64bit compatibility now you will probably want to install only 32bit binaries. However if in the future you run into software that requires 64bit compatibility. With the status quo it would require you to re-install the OS

So, because some distribution's biarch support sucks enough that it can't install a bunch of 64-bit dependencies into /lib64 and /usr/lib64 when you install a 64-bit binary, we need a kernel hack?

Please. There are good arguments for FatELF, but this is not one of them.

SELF: Anatomy of an (alleged) failure

tialaramex — Sun, 27 Jun 2010 11:59:41 +0000

All of them, the package management software is aware of your preferred architecture and will install that package. Packages which provide dependencies (e.g. a library) can be multiply installed, so that both the 32-bit and 64-bit library are installed, each from its own package.

This stuff has all been working and in everyday use for some time.

SELF: Anatomy of an (alleged) failure

dlang — Sat, 26 Jun 2010 21:49:42 +0000

and fat binaries won't work in cases where you need different config file options for different architectures and the config file is on a shared drive

SELF: Anatomy of an (alleged) failure

tzafrir — Sat, 26 Jun 2010 18:20:10 +0000

It works for rpm when all those shared fiels are identical in every package.

But what you want to do here is that rpm will be able to merge files from different packages into a single file on disk. This won't work.

SELF: Anatomy of an (alleged) failure

tzafrir — Sat, 26 Jun 2010 18:13:53 +0000

So which package includes /usr/bin/hello ? The i386 one? The amd64 one? The i686-with-uclibc one? The armel (v4) one?

SELF: Anatomy of an (alleged) failure

vonbrand — Sat, 26 Jun 2010 15:08:04 +0000

Just use two binary packages, with non-architecure-dependent stuff exactly the same, and arrange for the package manager to manage files belonging to several packages. RPM does this, and it works.

No need to screw around with the kernel, no need to have 3 versions of the package (arch 1, arch 2, fat).

Disconnecting nautilus from gnome session

ncm — Sat, 26 Jun 2010 09:56:25 +0000

In gconf-editor, go to desktop/gnome/session/, and change required_components_list to "[windowmanager,panel]".

While we're way, way off topic, you might also want to go to desktop/gnome/interface and change gtk_key_theme to "Emacs" so that the text edit box keybindings (except in Epiphany, grr) are Emacs-style.

Contempt, thy name is Gnome.

Getting back on topic, fat binaries makes perfect sense for shared libraries, so they can all go in /lib and /usr/lib. However, there's no reason to think anybody would force them on you for an EEE install.

SELF: Anatomy of an (alleged) failure

Tet — Sat, 26 Jun 2010 07:33:02 +0000

I'm not claiming fat binaries solve any particular distribution problem, nor do I believe that their existence means that fat binaries must cover every possible combination. In fact, just shipping a combined ia32 and x86_64 binary would cover 99% of the real world machines. But even if you don't want to ship a fat binary, it's not hard to envisage tools that would allow an end user to create a fat binary from two (or more) slim ones.

I've outlined a case where it would be both useful and desirable to have them, and to date, I haven't seen any sensible alternatives being proposed.

SELF: Anatomy of an (alleged) failure

waucka — Sat, 26 Jun 2010 00:11:18 +0000

If you want to do JIT, you should do it on an intermediate representation (IR) designed for the purpose. Deliberately using x86 (or any native code, really) for that purpose is ridiculous. Besides, we wouldn't necessarily have to do JIT all the time. With a good IR, we could have live CDs and USB sticks use JIT and convert to native code at install-time.

SELF: Anatomy of an (alleged) failure

dlang — Fri, 25 Jun 2010 22:17:00 +0000

people supporting want a Fat binary that's only fat enough for your particular system, but claim that having fat binaries would solve distribution problems because there wouldn't need to be multiple copies.

these two conflict, if you are going to support FAT binaries for every possible combination of options the distribution problem is much larger. If you are going to want the fat binary to support every possible system in a single binary it's going to be substantially larger.

it's not that I am so opposed to the idea of fat binaries as it is I don't see them as being that useful/desirable. the problems they are trying to address seem to be solvable by other means pretty easily, and there is not much more than hand waving over the cost.

SELF: Anatomy of an (alleged) failure

Tet — Fri, 25 Jun 2010 21:40:51 +0000

Yes, I do know about the vulnerabilities with 64-bit flash. But like I said, this conversation isn't about flash. Despite your claim, I don't need fat libraries everywhere. On the 32-bit machines, I would already have the corresponding 32-bit libraries installed. And on the 64-bit machines, I'd have the 64-bit libraries installed. No, I don't use OS X, nor is it relevant to this discussion. If a particular approach solves a problem (as it will here), it's probably worthwhile, even if it doesn't solve every problem. I say again, why are you so anti fat binaries/libraries?

SELF: Anatomy of an (alleged) failure

dlang — Fri, 25 Jun 2010 21:27:19 +0000

re: 64 bit flash

you do realize that the version you have is vunerable to exploits that are being used in the wild. Adobe decided to discontinue the 64 bit version instead of fixing it.

a fat plugin is only useful if you also have fat libraries everywhere. This directly contridicts posts earlier that said not to worry about the bload as the distros would still ship non-fat distros.

by the way, do you expect plugs to work across different operating systems as well so that you can have your $HOME NFS mounted on MACOS as well? where do you draw the line at what yo insist is needed?

SELF: Anatomy of an (alleged) failure

Tet — Fri, 25 Jun 2010 21:16:47 +0000

Ye gods, does it really take much effort to see past the lack of a 64-bit flash plugin (which incidentally, I do have, even if it's been discontinued by Adobe)? The same applies to any plugin. Forget that I mentioned flash, and think instead about a java plugin or an acroread plugin, or any other plugin you care to think of.

the same thing goes for any application that uses plugins. the plugin binaries should be able to be installed outside of $HOME. If they can be, then the application can work if $HOME is shared.

Yes, but here in the real world, they're not. Even if you could find a suitable location that would be writeable by a non-privileged user, it would mean changing the applications, and as I mentioned, there are many, many of those. Simply making a fat shared library possible would be a much easier and cleaner solution, and would have negligible impact on those that didn't want or need to use it. I don't understand why so many are opposed to it.