LWN.net Logo

Small problem for Linux ? Sure. Big problem for Linux user? Of course.

Small problem for Linux ? Sure. Big problem for Linux user? Of course.

Posted Jul 9, 2009 3:10 UTC (Thu) by khim (subscriber, #9252)
Parent article: Google Chrome OS and the community

The availability and management of libraries works well enough that it's often possible to move complicated binaries between distributions and expect them to run.

Sorry, but this is not true at all. When you grab stuff from older distribution you need to install some kind of compat package. If you grab stuff from newer distribution - you often need new version of some vital library (like glibc). Sometimes you can use alien libraries and LD_LIBRARY_PATH instead (this is how I used 7z from Hardy in Dapper). Are these obstacles insurmountable? Not at all - there are a lot of people who can do it. Millions of them - 1% of PC users is a good astimate, I guess - and coincidently this is the size of Linux "desktop penetration"...

Fragmentation does not hurt Linux development... much, but... it sure as hell does hurt Linux adoption. Things are not working "out of the box" if you go beyond your distribution's repository - and this is a big deal for Joe Average...


(Log in to post comments)

Small problem for Linux ? Sure. Big problem for Linux user? Of course.

Posted Jul 9, 2009 7:03 UTC (Thu) by Frej (subscriber, #4165) [Link]

Almost agree ;) It's basicly imposible to distribute linux software as an ISV (i hate using that term). Sure if you like to manage software or run a server, packages are very nice. But it's all very centralized.

Small problem for Linux ? Sure. Big problem for Linux user? Of course.

Posted Jul 9, 2009 7:47 UTC (Thu) by boudewijn (subscriber, #14185) [Link]

We manage alright. We have a fairly complicated application that uses Qt and about a dozen other
third party libraries. Our binaries (http://www.hyves.nl/hyvesdesktop/download/) have been
reported to work successfully on many linux distributions.

Of course, we ran into trouble when we wanted to support sound. Qt's phonon uses GStreamer as a
backend and that's a mess. We came to the conclusion that we'd better use QSound, since the
alternative would have been to redistribute the right gstreamer with all plugins ourselves.
Something compiled against a platfrom gstreamer on one distribution will crash on all other
distributions (that we tried).

Sound

Posted Jul 9, 2009 12:52 UTC (Thu) by rfunk (subscriber, #4054) [Link]

I thought Phonon was a KDE thing, not a Qt thing.

I also thought that the whole point of Phonon was that you don't have to
deal with the underlying Gstreamer (or Xine or whatever) engine at all, so
that engine could be swapped out or upgraded at will, as long as your
Phonon can handle the varying engines. Thus the confusion from Gstreamer
people who thought (wrongly) that Phonon was intended to replace
Gstreamer.

Of course, if you have a Qt app rather than a KDE app, it would seem to
make sense to use Qsound rather than Phonon anyway.

Sound

Posted Jul 9, 2009 13:09 UTC (Thu) by johnflux (guest, #58833) [Link]

Phonon was originally KDE, but was moved from KDE into Qt.

Sound

Posted Jul 9, 2009 13:33 UTC (Thu) by boudewijn (subscriber, #14185) [Link]

It's a Qt thing. But at least in Qt 4.4.x it was quite problematic. On WIndows XP, it would play wav
files, which it wouldn't play on WIndow Vista, where phonon would be able to play mp3, which it
wasn't able to play on XP... On OSX everything was fine. And on Linux, an app compiled on Ubuntu,
crashed in gestreamer on OpenSUSE and Fedora -- and all other permutation gave the same result.

We'll have to try again with 4.5 -- 4.5 is in many ways a great series of releases with lots of fixes
and cool things.

Small problem for Linux ? Sure. Big problem for Linux user? Of course.

Posted Jul 9, 2009 13:04 UTC (Thu) by marcH (subscriber, #57642) [Link]

> Almost agree ;) It's basicly imposible to distribute linux software as an ISV

If your goal is to distribute only one "universal" binary, you are right this is seldom possible. If on the other hand you start from source then it often just works or with only a minor effort.

Porting your source code from one Unix vendor to the other was generally much more difficult than from one Linux distribution to the other. Simply because Linux distributions share most of their underlying source code.

Minor effor for WHO?

Posted Jul 9, 2009 16:16 UTC (Thu) by khim (subscriber, #9252) [Link]

If your goal is to distribute only one "universal" binary, you are right this is seldom possible.

Your goal is to give users something they can use.

If on the other hand you start from source then it often just works or with only a minor effort.

...if you know how to install stuff from source. I was surprised at first when I found few years ago that some of my friends who work as admins don't know C and don't know how to compile programs from source... but then - why should they? It's rarely needed and when it's the only possibility - they can contact me and I'll help them (at first it was for free, later when I've tried to say "enough is enough" they just offered me to pay for my skills and now everyone is happy). If even admins can not compile stuff from source - what chance do you think "Joe Average" will have?

Porting your source code from one Unix vendor to the other was generally much more difficult than from one Linux distribution to the other. Simply because Linux distributions share most of their underlying source code.

Sure - this was the point in article. And my point is that such effort is still required if you are talking about programs for "normal users"... and there are more Linux distributions then there were Unix vendors...

Minor effor for WHO?

Posted Jul 16, 2009 16:08 UTC (Thu) by Wol (guest, #4433) [Link]

May I suggest you look at lilypond (www.lilypond.org). They distribute ONE universal binary which is pretty much guaranteed to run on ALL linux distros. (Firefox and OOo do the same ...)

And the tools lilypond uses to do that are open-source...

Cheers,
Wol

Small problem for Linux ? Sure. Big problem for Linux user? Of course.

Posted Jul 9, 2009 9:28 UTC (Thu) by DonDiego (subscriber, #24141) [Link]

Static linking is your friend. I never understood why ISVs don't simply distribute statically linked binaries.

Small problem for Linux ? Sure. Big problem for Linux user? Of course.

Posted Jul 9, 2009 11:20 UTC (Thu) by da4089 (subscriber, #1195) [Link]

Often, they do. It's the least bad alternative. The major downside is that the vendor needs to deal
with their own security patches for the embedded dependencies.

Small problem for Linux ? Sure. Big problem for Linux user? Of course.

Posted Jul 10, 2009 5:53 UTC (Fri) by pabs (subscriber, #43278) [Link]

Surely most of the time they don't bother to care about the embedded code copies?

Small problem for Linux ? Sure. Big problem for Linux user? Of course.

Posted Jul 9, 2009 14:32 UTC (Thu) by elanthis (subscriber, #6227) [Link]

Except that doesn't really work well for real applications. Static linking GTK means that your application doesn't use the same theme as the rest of the desktop that's running a newer GTK. Static linking sound libraries means that codec plugins don't work, and you need to statically link those too, which can be a legal nightmare. And static linking libc is a horrifically bad idea.

Linux compatibility at the ABI level is an absolute joke. Source compatibility can even be an issue now and then, because so many projects just change the damn API with every release, and distributions generally only ship the latest version of most projects (few distributions ship every 2.x release of Python for example, even though each release has a minor API and ABI breakages).

That's all fairly irrelevant though, since Linux distributions go out of their way to impose entirely artificial barriers to compatibility. Even if you make a solid portable Linux binary, there's no way to make that binary installable in a cross-platform way that doesn't rely on the user opening a shell and having wasted weeks/months/years of their life learning how to use a shell instead of spending their time doing something more important (like spending time with real people, instead of enslaving themselves to babysitting and hand-holding their "time saving" computational apparatus).

Until Linux distributions either agree on a common package manager (and standardize package names, virtual provides, etc.) or agree on shipping a second cross-platform installation tool (there are a ton of these, some of which I believe can even integrate with RPM/DPKG, but if these tools are not installed on the system then installing packages still requires shell magic to install the damn installation tool).

The various framework developers don't put a lot of effort into testing binary compatibility I believe, and that's largely because few people ask for binary compatibility, because binary compatibility is useless on an OS where getting the binaries installed is a nightmare. In turn, companies don't bother trying to make universal binaries because they know it's entirely pointless, and companies that flat out _can't_ release source just don't bother with Linux... which is why a great deal of us still have Windows installations around. The only thing stopping a great deal of those applications being ported to Linux is the fact that they'd be absolutely impossible to install on Linux.

Take a modern game for example. Even assuming the game source was released, you can't package those things in an RPM. They come on DVDs packed to the brim with textures, sounds, musics, meshes, maps, scripts, videos, and so on. Are we supposed to install a single 4.4GB RPM? And then every time there's a minor update to a few models, we're supposed to download a new 4.4GB RPM because there's no standard delta-RPM mechanism shared by all the RPM distros? That doesn't even include Debian/Ubuntu of course.

There needs to be a cross-platform way of installing software -- and I don't even care if it's a graphical frontend to a compilation script to make the GPL fans happy, so long as it can figure out how to install dependencies on its own -- including a way of updating that software in a realistic fashion given all of today's applications' needs, binary compatibility isn't worth testing and developing for on Linux. It's there mostly to make a nice bullet point for a few enterprise distros that don't really need it, and that's it.

In turn, Linux is still just an "appliance" OS and anybody who needs to do more than run a web browser and email client and word processor (which is a far, far greater percentage of users than the Linux desktop advocates continually claim -- I can't name a single person, even my 80+ year old grandparents, who limit themselves to just those three things) simply can't use Linux because the repositories don't include the software they want (be it Bejeweled 2 or their local geneaology club's favorite software package) and there's no possible way they could ever figure out how to install that software even if there was a Linux version.

Linux's future in the mass consumer market, assuming these things don't change (and I'm 100% convinced that they never will), is going to be handhelds and other appliance-like devices... assuming Linux developers can ever manage to beat the popularity of Apple's competing devices, anyway. Which, as of yet and for the forseeable future, they can't. Android and Pre have nothing on the iPhone's sales. And if you care about Linux from an Open Source/Free Software perspective, those Linux devices don't even matter to you because they rely on proprietary software to get full functionality!

I can't stress it enough though. Software installation is Linux's Achilles Heel. Until that's fixed, Linux is a just niche nerd OS in the desktop space.

Small problem for Linux ? Sure. Big problem for Linux user? Of course.

Posted Jul 9, 2009 15:48 UTC (Thu) by marcH (subscriber, #57642) [Link]

> anybody who needs to do more than run a web browser and email client and word processor (which is a far, far greater percentage of users than the Linux desktop advocates continually claim [...]) simply can't use Linux because the repositories don't include the software they want

So a typical Linux repository holds only web browsers, email clients and word processors, that is all? I agree with most of your (very good) post, except for this exaggeration above.

You are right that anything happening outside of repositories is a nightmare. So maybe the future of Linux software distribution is just "more repositories". See for instance this: http://www.virtualbox.org/wiki/Linux_Downloads or this: http://rpmfusion.org/Configuration

Performing such automated re-compilation and packaging is possibly easier than ensuring portability across multiple Windows versions.

Small problem for Linux ? Sure. Big problem for Linux user? Of course.

Posted Jul 16, 2009 19:15 UTC (Thu) by oak (guest, #2786) [Link]

> Are we supposed to install a single 4.4GB RPM? And then every time
there's a minor update to a few models, we're supposed to download a new
4.4GB RPM because there's no standard delta-RPM mechanism shared by all
the RPM distros?

Any competent packager would of course split the thing into suitable
packages.

In case of a game and related data files, the game itself would be in one
package and all the huge data files could be split e.g. so that there's
one level/scenerio/campain per additional package (depending how huge they
are).

I think this suits nicely to the model how shareware & ID software worked
earlier. You get (open sourced) game and some single player tutorial/demo
level as package(s) (maybe included into the distributor's repository) and
based on how much user likes this, s/he can then buy the full game. When
the demo level is finished, user would see something like: "Demo ended,
buy the full game? [enter <publisher>'s show]" which would open Browser
to the shop.

The non-code level data could be packages in publisher's own repo which
provides them in general packaging formats (RPM/DEB/tar.gz). As data files
don't have dependencies, these can be installed to any distro version.
Only game binary package needs to be recompiled & separate for each
distros' distro versions and that could be done by the distros themselves
if the game source is open and this is agreed with the publisher.

I'm not sure how one could get copy protection to this. DRM doesn't work
very well and isn't approved by anybody. Maybe this kind of games should
have some kind of network element where user needs to log in with his/her
game registration?

Static linking

Posted Jul 29, 2009 3:44 UTC (Wed) by jeff@uclinux.org (guest, #8024) [Link]

"Are we supposed to install a single 4.4GB RPM?" [when installing a static linked distribution-
portable game]

Well, game? Maybe 400M. Other things, sure. Doesn't make it right, but the incompatibility
problem is really quite serious, and disk space is cheap. Just for example... Download Xilinx
WebPack 11.1:

Jeff$ ls -l Xilinx_11.1_WebPack_SFD.tar
-rw-r--r-- 1 jeff users 2868316160 Apr 30 13:39 Xilinx_11.1_WebPack_SFD.tar

It's on that order of magnitude even with compressed installer files inside the tar. Mostly static
linked. Yes, I think for serious programs, this is a serious option. These programs have to work
today, next month, next year... if they achieve that, we'll see. If I need to distribute tools
binaries, --enable-static --disable-shared.

I think Bionic libc is a stellar library for a cell phone. I have never built any embedded system
with glibc, if you're doing that, you're not making the right engineering choices.

Small problem for Linux ? Sure. Big problem for Linux user? Of course.

Posted Jul 9, 2009 10:27 UTC (Thu) by nye (guest, #51576) [Link]

Well, I was pleasantly surprised the other week when I tried running an old copy of Mosaic. The Linux binary from something like 1995 ran without my having to do anything more than extract and execute. Of course, this is because it wasn't built to rely on shared libraries.

(Sadly it turns out that NCSA's own website was the only one I could find which still uses HTTP/1.0, so it was all for naught.)

You were surprised? Why?

Posted Jul 9, 2009 16:24 UTC (Thu) by khim (subscriber, #9252) [Link]

Well, I was pleasantly surprised the other week when I tried running an old copy of Mosaic.

I run old DOS/Windows programs quite often - and I expect that they'll work on new version of Windows (the fact that Windows Vista x64 dropped support for DOS/Win16 was unpleasant surprise but AFICS it's problem with AMD64 mode, not the result of Microsoft's negligence). For you the fact that old copy of Mosaic can run is pleasant surprise. Nuff said.

The really strange this here is that GNU/Linux core is 99% bullet-proof: linux kernel and glibc go the huge pains to guarantee backward compatibility. But the further you go up in the software stack the flakier ABI stability becomes - by the time you've reached layers where you can do something usefull it's gone almost completely...

You were surprised? Why?

Posted Jul 9, 2009 16:30 UTC (Thu) by martinfick (subscriber, #4455) [Link]

There are plenty of old dos programs that do not run on windows. Anything that polled a mouse will chew up CPU. Anything that did out of the ordinary graphics will not work, games...

Yeah, but then Linux's task is simpler from the start...

Posted Jul 9, 2009 19:07 UTC (Thu) by khim (subscriber, #9252) [Link]

There are plenty of old dos programs that do not run on windows. Anything that polled a mouse will chew up CPU. Anything that did out of the ordinary graphics will not work, games...

Sure. DOS programs were written as if they own the computer. Which they did. So it's not easy to containerize them without huge overhead. The comparable thing in Linux world are OSS programs - they also like to hog the part of computer (/dev/dsp device). And like MS DOS games these old programs worked poorly with new distributions. And like with Windows the idea that you can just rewrite all programs (Windows old took of with version 3.0 which was the first version with decent support for MS DOS programs) didn't fly. Why the hell linux developers must repeat all Microsoft's mistakes?

Yeah, but then Linux's task is simpler from the start...

Posted Jul 11, 2009 9:55 UTC (Sat) by nix (subscriber, #2304) [Link]

Because they're not "Microsoft"'s mistakes, per se: they're mistakes made
by software developers in general. MS just tripped over them, and now so
are we.

Actually, this one isn't even a mistake: it's an inevitable consequence of
what happens when a stable foundation grows that everything relies on,
when that foundation is then shown to be faulty by design. At least we
*can* rip it out: when biology does the same thing, we get stuck with the
same unfixable faults for hundreds of millions of years...

You were surprised? Why?

Posted Jul 11, 2009 9:46 UTC (Sat) by nix (subscriber, #2304) [Link]

But the further you go up in the software stack the flakier ABI stability becomes - by the time you've reached layers where you can do something usefull it's gone almost completely...
Is it? I see only one or two small ABI breakages a year (much less if you ignore libdb, OpenSSL, ffmpeg, libperl and libpython, which all break ABI at the drop of a hat), and the only specific complaints I've seen on this thread have been people trying to run things that expect new ABIs of old libraries, which isn't going to work until we all have our time machines.

The major high-in-the-stack desktop libraries and the things they use go to great lengths to maintain back-compatibility, and it seems to work. What they do instead is introduce new libraries with better APIs every so often (e.g. gvfs replacing gnome-vfs), and, sure, if you don't have those and you install something that needs them, you'll have to install them. But, again, this has nothing to do with ABI breakages, which pretty much aren't happening.

Binary compatibility

Posted Jul 10, 2009 18:31 UTC (Fri) by anton (subscriber, #25547) [Link]

Yes, my binary of Mosaic 2.7b5 still works on my Debian Lenny AMD64 system, and it also understands current HTTP (but often likes to download instead of display HTML pages due to charset issues (IIRC)). The binary is from 1998 and is statically linked.

A Mosaic binary from 1994 just segfaults, as well as all the other ZMAGIC binaries (1994-1995) I have lying around.

The QMAGIC binaries (1995-1997) all report "can't load dynamic linker '/lib/ld.so nor /usr/i486-linux/lib/ld.so'" and could probably be made to work by copying the appropriate file to the right place (plus any libraries needed).

Then we get into the ELF era (since 1998), and the binaries (e.g., ssh 1.2.25) work if (compatible versions of) the libraries they use are present; for that ssh binary the libraries are libnsl.so.1 libcrypt.so.1 libutil.so.1 libc.so.6 /lib/ld-linux.so.2, which came with the libc6-i386 package on Lenny. Not bad.

I usually preserve old distributions I used to use, so I can easily copy the old libraries to my new distribution if I need them (or maybe just include the old library directories in ld.so.conf). I have not needed to do that for quite some time, though.

Binary compatibility

Posted Jul 11, 2009 10:04 UTC (Sat) by nix (subscriber, #2304) [Link]

I think you have to turn off address space randomization to get old ZMAGIC
binaries to work (but I'd not be surprised if they'd rotted completely:
does anyone other than Alan Cox run them anymore? :) )

Binary compatibility

Posted Jul 13, 2009 13:26 UTC (Mon) by anton (subscriber, #25547) [Link]

Address-space randomization was turned off in my experiments, so that alone is not enough. Apart from such experiments I don't run them anymore.

Small problem for Linux ? Sure. Big problem for Linux user? Of course.

Posted Jul 9, 2009 15:10 UTC (Thu) by NAR (subscriber, #1313) [Link]

Oh, yeah. I've just had a user last week who didn't care to read the release notes and tried to run the software on SuSE 9 instead of SuSE 10. Of course, it didn't work due to glibc version differences.

Small problem for Linux ? Sure. Big problem for Linux user? Of course.

Posted Jul 9, 2009 22:34 UTC (Thu) by Baylink (subscriber, #755) [Link]

And this happens on many levels.

I just bought a Sierra 598 USB "Internet on a Rope" dongle from Sprint.

Sprint's shelf-talker card in the store -- offset color printed -- says it's
compatible with XP, Vista, Mac *and Linux* (without specifying a kernel version or distro).

Alas, though unsurprisingly, Sprint's support doesn't know from Linux, and Sierra "doesn't support" Linux -- though they do have a linux@ address which is ticket-tracked (with custhelp.com's absolutely *miserable* system, but don't get me started on that) -- the best thing Sierra has for me is "it works on Fedora and Ubuntu".

The sticking point is apparently that the USB devices that expose the modem as ttyUSB *are mode switched* from the USB devices that expose the onboard "TRU-install" PRAM and MicroSD card slot; the driver is supposed to switch from one mode to the other... and hald or udev may be what's getting in its way, which brings me back on topic: how portable your software {is,can be} depends on *how low level stuff it has to do*.

SysVinit is fairly well disseminated across distros now, so large packages can make reasonable assumptions about how they'll have to set their daemons up to run and suchlike, but while we used to consider that "low-level" stuff, there are lots of new middleware layers in the average Linux distro these days, and it's (again) not so easy...

(If anyone has any pointers on the USB thing; the gory story is at: http://www.evdoforums.com/thread12302.html)

Copyright © 2012, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds