LWN: Comments on "Some unreliable predictions for 2015"

A bit of off-topic

raven667 — Tue, 03 Feb 2015 15:55:15 +0000

I agree this is off topic but power management does interact with the console and requires communication with whatever is driving the console output, power management changes depending on whether the keyboard is being used or a video is being played or other factors that the UI program knows about and there should be UI elements which allow the user present on the console to suspend or power off the device. There may be a backend daemon which coordinates these changes but they are driven by policy encoded with the console UI itself.

Once you need to have software running to interact with the console present user then you need a context to run that software in, having a user session dedicated to the login screen allows it to have the same protections as any user and make it less of a special case.

I dunno, makes sense to me anyway.

A bit of off-topic

dgm — Tue, 03 Feb 2015 08:43:30 +0000

I think Mathew missed it big time with this post. Firstly, LightDM is *still* Ubuntu's dm of choice (and working great, if you ask me). But more importantly, the main reason for GDM to start a full session is profoundly flawed. In fact, the idea that power management and accessibility belong to the user's session is flawed. And funnily, it's Mathew himself that gives the reason:

> Closing the lid of my laptop should suspend the system regardless of whether it's logged in or not.

Absolutely. That's why power policy should *never* be related to the user's desktop session.

Some unreliable predictions for 2015

raven667 — Mon, 02 Feb 2015 22:41:24 +0000

Is the lesson that those who don't understand Multics (and Plan9) are doomed to reinvent it badly and that for all the "bloat" that was removed from the Multics design to make a "simple" UNIX, every bit of it was eventually added back because it was needed but organically without any coherent design, leading to the most deployed, standard system which largely destroyed all others being a creeping inconsistent horror, that we all love dearly 8-)

Reminds me of http://mjg59.livejournal.com/136274.html

Some unreliable predictions for 2015

nix — Mon, 02 Feb 2015 20:34:27 +0000

There is FAR too much reliance on theory in the computer space in general, and linux (the OS, not kernel) is no exception.

This may be true in the database realm you're singlemindedly focussed on (and I suspect in that respect it is only true with respect to one single theory which you happen to dislike and which, to be honest, RDBMS's implement about as closely as they implement flying to Mars), but it's very far from true everywhere else. GCC gained hugely from its switch to a representation that allowed it to actually use algorithms from the published research. The developers of most things other than compilers and Mesa aren't looking at research of any kind. In many cases, there is no research of any kind to look at.

Some unreliable predictions for 2015

nix — Mon, 02 Feb 2015 20:30:23 +0000

I dunno. In the world after the horrible but astonishingly still not dead BSD networking so-called "API" and the Unix wars, the claim could be made that being a haphazard mess with no discernible design philosophy *is* the Unix design philosophy. It's just not the one envisaged by Ken Thompson and the early designers, alas :( systemd is definitely a step back towards that early vision.

Some unreliable predictions for 2015

judas_iscariote — Wed, 21 Jan 2015 15:31:01 +0000

Truth is a bitter pill .. matter of fact is that the old
stuff barely has any man power to keep it in a working state. the way it is integrated is also beyond terrible.

Some unreliable predictions for 2015

anselm — Tue, 20 Jan 2015 23:02:02 +0000

It's not the distribution maintainers' fault that the traditional setup is so terrible. It isn't even the upstream software authors' fault. They all had an itch and they scratched it. The problem is that the various bits and pieces arose over a period of 20 years or so, and that nobody really talked to anybody else. For quite some time, the traditional setup used to be the best available solution to the problem at hand, and the distribution maintainers did great and mostly thankless work integrating and improving it. That does not detract from the fact that seen as a complete software system the traditional setup leaves a lot to be desired, especially compared to modern approaches like systemd.

Many systemd detractors do not seem to appreciate that one reason why so many distributions fell over themselves to adopt systemd is that systemd actually makes a distribution maintainer's life a lot easier. It basically saves a distribution from having to implement and maintain a lot of (often distro-specific) basic plumbing that is a hassle to design, build, and keep going. People who instead prefer their Linux distribution to be untainted by systemd are free to do that work themselves, or (as the distribution maintainers have mostly done it for them already) at least shoulder the burden of maintaining it going forward once the original distribution maintainers have decided – as is their privilege – that systemd is a more worthwhile use of their time. In that sense something like Devuan is a great idea because it will hopefully soak up all those folks who would otherwise keep hassling the Debian maintainers about sysvinit.

Some unreliable predictions for 2015

flussence — Tue, 20 Jan 2015 19:31:27 +0000

> Which is of course a proposition that is entirely different from your deciding that the distribution managers' time is yours to allocate, by requiring them to keep the existing hodgepodge of accidentally-combined components on life support forever.

What a tactless insult to the people whose work you've been freeloading off of for years before systemd.

Some unreliable predictions for 2015

anselm — Mon, 19 Jan 2015 22:18:43 +0000

The problem with systemd, as much as any other problem, is that Lennart and the distro managers who've drunk his FlavorAID decided that my time was theirs to allocate

Which is of course a proposition that is entirely different from your deciding that the distribution managers' time is yours to allocate, by requiring them to keep the existing hodgepodge of accidentally-combined components on life support forever.

I had other more important things to do than to spend time learning an entirely new core system component that gives me no measurable advantage.

Great, so go use Slackware.

That's *aside* from how thoroughly it violates nearly every precept of three decades of Unix design philosophy.

That claim has been debunked so often it isn't funny anymore. The main distinguishing characteristic of the existing haphazard setup is that, unlike systemd, it has no discernible design philosophy whatsoever – it is a thrown-together mixture of components from a variety of unrelated sources, with lots of almost-duplication, a disparate zoo of configuration file formats, and dismal documentation.

As far as the famous and much-maligned “Unix design philosophy” is concerned, there can be no doubt that in the traditional setup there are lots of bits and pieces that each do one thing – but it requires quite a fanciful imagination to claim that they do that thing well.

Some unreliable predictions for 2015

Baylink — Mon, 19 Jan 2015 17:45:22 +0000

The problem with systemd, as much as any other problem, is that Lennart and the distro managers who've drunk his FlavorAID decided that my time was theirs to allocate; I had other more important things to do than to spend time learning an entirely new core system component that gives me no measurable advantage.

That's *aside* from how thoroughly it violates nearly every precept of three decades of Unix design philosophy.

JPEG / JFIF

tialaramex — Sat, 17 Jan 2015 10:32:12 +0000

I am, since the name probably doesn't ring a bell, responsible for GIMP's TIFF loader, and in another previous life as a PhD research student I read and wrote a great many complex tiled TIFFs created by art historians studying/ preserving various great European works.

So, I'm not saying it's garbage because I don't understand how to use it, I'm saying it's garbage because I do understand and I don't sympathise.

Some unreliable predictions for 2015

mathstuf — Fri, 16 Jan 2015 21:22:16 +0000

Either bundle or ask for that version to be packaged. Or provide it in your repo (since you're not likely to have such a package shipped by Debian itself without them handing back patches to update it).

Poppycock!

mathstuf — Fri, 16 Jan 2015 20:41:06 +0000

Those sound like things which should be filed as bugs. Also, finally a concrete lost of things broken in the systemd transition (versus previous complaints of "all kinds of things *hand wave*").

Some unreliable predictions for 2015

cortana — Fri, 16 Jan 2015 18:56:31 +0000

BTW, what happens if I actually need libreadline.so.4?

Some unreliable predictions for 2015

cortana — Fri, 16 Jan 2015 18:55:29 +0000

No, we have an inadequate system that is only useful as long as upstream developers, downstream developers and distributors never make mistakes.

Some unreliable predictions for 2015

peter-b — Fri, 16 Jan 2015 17:45:11 +0000

So we have a perfectly adequate system that some people don't use properly, in fact. This is news?

Poppycock!

raven667 — Fri, 16 Jan 2015 16:32:34 +0000

> this narrative is being pushed by journalists with financial ties to Red Hat.

> propagandists will sacrifice truth and journalistic integrity for an appearance of pious conformity as their patrons require.

hahahahahahahahahahaha, I'm _sure_ that is what is going on (sarcasm), no one actually has real opinions, the entire world is a conspiracy of paymasters against you and that's not even a little bit paranoid.

Some unreliable predictions for 2015

raven667 — Fri, 16 Jan 2015 16:28:28 +0000

> Implicit in this prediction appears to be the assumption that "systemd will win", and sysv, upstart, and even those Linux distributions using a bsd-style init will all move to systemd.

systemd has won, the big three PC distros Debian, Redhat, SuSE are using it, mobile Tizen, Jolla are using it, thats a sufficient de-facto standard. It doesn't matter if Gentoo or Slackware uses it, it'd be better if they didn't so that there is always room around the edges for people to get together and try different things, bsd or sysv or whatever.

Some unreliable predictions for 2015

raven667 — Fri, 16 Jan 2015 16:19:30 +0000

I think it is true that there are a ton of library ABI breaks where the soname isn't changed because many library authors don't know when the changes they make break it, they just recompile which hides a lot of issues. Distros like Gentoo and tools like OBS rebuild applications when a library dependency changes even if the soname is the same for exactly this reason, they wouldn't bother to do this if the soname was a reliable indicator of compatibility.

Some unreliable predictions for 2015

mathstuf — Fri, 16 Jan 2015 15:56:48 +0000

Then someone goofed. Either you're either depending on unspecified behavior (which should be un-exported if possible), or a soname bump was missed upstream between 5.1 and 5.2.

Some unreliable predictions for 2015

cortana — Fri, 16 Jan 2015 15:44:17 +0000

This doesn't do what you think it does.

$ ls -l /lib/x86_64-linux-gnu/libreadline.so.{5,6}
lrwxrwxrwx 1 root root 18 Apr 27  2013 /lib/x86_64-linux-gnu/libreadline.so.5 -> libreadline.so.5.2
lrwxrwxrwx 1 root root 18 Jan 13 03:25 /lib/x86_64-linux-gnu/libreadline.so.6 -> libreadline.so.6.3

Looks promising at first glance, but what if my application wants libreadline 5.1?

The SONAME is not strongly tied to the version of the library, but to the compatibility level.

Poppycock!

ksandstr — Fri, 16 Jan 2015 14:22:22 +0000

This so-called "settling down" in the ongoing systemd war is a misconception stemming from the end-of-year holidays around the world, and this narrative is being pushed by journalists with financial ties to Red Hat. Unsurprisingly, examples of the latter will go out of their way to present systemd's brave new world as a foregone conclusion. According to this narrative we're to accept that everything pre-systemd will now break, and that the only way to fix it is to lube up and bend over; or become a ridiculous dinosaur for life.

In the mean time, power management in Debian testing and unstable continues to remain broken unless systemd or systemd-equivalent components are installed. Relatedly cryptsetup will still interact incorrectly with sysvinit boot scripts and the boot console, preventing it from setting up encrypted block devices at boot even when the correct passphrase is given. Since power management no longer detects that the computer is attached to a power supply, anacron only runs its jobs at fresh reboot and only if that is done while leashed; this keeps automatic backups from happening as specified.

The situation where power management failure prevents automatic suspend and automatic switch to "lose at most 30 seconds' work instead of 60 minutes" in low-battery conditions is a recipe for catastrophic data loss. The failing of regular automated backups aggravates such a catastrophe to outright disaster by preventing recovery from any backup besides those that were created manually, or pre-date Debian's systemd madness.

The constellation of outright egregious systemd breakage persists, so the war is not over. However, as in any war, propagandists will sacrifice truth and journalistic integrity for an appearance of pious conformity as their patrons require.

Some unreliable predictions for 2015

cesarb — Fri, 16 Jan 2015 13:55:15 +0000

> Much of the problem goes away if just the package managers could tolerate to have multiple versions of the same lib installed. versioned sonames exist for a reason...

It's not that simple. Suppose a program is linked against library A which in turn is linked against libpng2, and that program is also linked against library B which in turn is linked against libpng3.

Now imagine the program gets from library A a pointer to a libpng structure, which it then passes to library B.

JPEG / JFIF

rleigh — Fri, 16 Jan 2015 13:27:33 +0000

TIFF is certainly complex, but that's made up for by its unmatched power and sophistication. I've recently been working on TIFF reading/writing of microscopy imaging data, and testing all the different combinations of PhotometricInterpretation with and without LUTs, pixel type and depth (including complex floating point types), orientation, large numbers of channels, all sorts of combinations of tile and strip sizes, bigtiff, etc. It's quite surprising how many programs and graphics libraries get it wrong. The worst I found was the Microsoft TIFF support on Windows, e.g. for thumbnailing and viewing, which was incorrect for most cases, and apparently it's been much improved! Support on FreeBSD and Linux with free software viewers was better, but still not perfect for many cases.

I think this is primarily due to most authors staying well inside the 8-bit grey/RGB "comfort zone". Sometimes this extends to 12/16-bit or maybe float, and not testing with more sophisticated data.

Most of that is simply the author screwing up. For example, when dealing with strips and tiles, it's amazing how many people mess up the image data by failing to deal with the strip/tile overlapping the image bounds when it's not a multiple of the image size, sometimes for particular pixel types e.g. 1 or 2 bit data. Just a simple miscalulation or failure to check particular tiff tags.

I'm not sure what the solution is here. A collection of images which exercise usage of all the baseline tags with all the special cases (and their interactions) would be a good start. I currently generate a test set of around 4000 64×64 TIFF images for the set of tags I care about, but it's still far from comprehensive. I know it works for the most part, but even then it's going to fail for tags I don't currently code for.

Some unreliable predictions for 2015

anselm — Fri, 16 Jan 2015 12:41:49 +0000

I don't see why that would be a problem. On my Debian system I have multiple versions of, say, libreadline, libprocps and libtcl installed at the same time, in each case from separate packages, so the support seems to be there already.

JPEG / JFIF

peter-b — Fri, 16 Jan 2015 12:35:57 +0000

In TIFF's defence, I used TIFF files and libtiff extensively during my PhD, since they're the only sane way of storing and communicating remote sensing datasets (complex 32-bit fixed point pixel values on 6 image planes? no problem).

I didn't experience any problems that weren't due to my own incompetence.

Some unreliable predictions for 2015

hitmark — Fri, 16 Jan 2015 11:51:16 +0000

Much of the problem goes away if just the package managers could tolerate to have multiple versions of the same lib installed. versioned sonames exist for a reason...

Some unreliable predictions for 2015

hitmark — Fri, 16 Jan 2015 11:47:29 +0000

There is a certain difference though. Most of those are either end programs (they don't interact with other programs, only the user) or have strongly defined interfaces between themselves and the rest of the ecosystem.

JPEG / JFIF

tialaramex — Fri, 16 Jan 2015 11:24:12 +0000

Well, the deal is a bit more complicated than you've suggested. You've focused on the IJG's library, but there's much more.

JPEG per se isn't a file format. The committee weren't focused on storing the data from their compression algorithm as files, they were thinking you'd transmit it to somewhere and it'd get decompressed and then used. So the actual international standard is completely mute about files on disk.

Early on people who did think we should store data in files wrote JPEG compressed data to the pseudo-standard TIFF. But TIFF is a complete mess, conceived as the minimal way to store output from a drum or flatbed scanner on a computer and thus permitting absolutely everything but making nothing easy - and its attempt to handle JPEG led to incompatible (literally, as in "I sent you the file" "No, my program says it's corrupt" "OK, try this" "Did you mean for it to be black and white?") implementations. There were then a series of Adobe "technical notes" for TIFF that try to fix things, several times attempting a fresh start with little success.

JFIF is the "real" name for the file format we all use today, and it's basically where the IJG comes into the picture. Instead of TIFF's mess of irrelevant or nonsensical parameters you've got the exact parameters needed for the codec being used, and then you've got all this raw data to pass into the decoder. And there's this handy free library of code to read and write the files, so everybody just uses that.

So initially the IJG are great unifiers - instead of three or four incompatible attempts to store JPEG data in a TIFF you get these smaller and obviously non-TIFF JPG files and either the recipient can read them or they can't, no confusion as to what they mean. But then they proved (and libpng followed them for a while) incapable of grasping what an ABI is.

Some unreliable predictions for 2015

vonbrand — Fri, 16 Jan 2015 01:43:03 +0000

Add that distributions (or users) select different packages for the same functionality: different web servers, C/C++ compilers, editors, document/image viewers, ...

Some unreliable predictions for 2015

raven667 — Thu, 15 Jan 2015 21:40:36 +0000

The point is that there is more than one interoperable implementation, not that everyone has forked the reference implementation to get interoperability, a JPEG made on an Ubuntu 12.04 will work on an Ubuntu 10.10 and Ubuntu 14.10 and Fedora 21 and Fedora 15 without "recompiling" or converting, whereas software built on one can't run on the other (while using OS provided shared libraries) because they are too different in the tiny details even though they are broadly running the same software.

Some unreliable predictions for 2015

flussence — Thu, 15 Jan 2015 20:22:30 +0000

> Standardization has had some success in network protocols like IP or kernel interfaces like POSIX or file formats like JPEG

JPEG is a bad example to use there... everyone dropped the official reference implementation after its maintainer went off the rails and started changing the format in backwards-incompatible ways: http://www.libjpeg-turbo.org/About/Jpeg-9

Some unreliable predictions for 2015

dlang — Thu, 15 Jan 2015 18:32:37 +0000

The problem has never been that it's not technically solvable.

The problem is that the software (app and library) authors don't do what everyone thinks they should (which _is_ technically impossible, because people have conflicting opinions about what they should do)

Let's talk about older, but well maintained versions of packages.

Who is doing that maintenance? In many cases, software developers only really support the current version of an application, they may support one or two versions back, but any more than that is really unusual.

It's usually the distro packagers/maintainers that do a lot of the work of maintaining the older versions that they ship. And the maintinance of the old versions has the same 'include all changes' vs 'only include what's needed (with the problem of defining what's needed)' issue that the distros have in what versions they ship in the first place.

Some unreliable predictions for 2015

HIGHGuY — Thu, 15 Jan 2015 17:50:58 +0000

Actually, you haven't made any point that I think is not technically solvable. Let's say we build this system:
- app/lib-developers do as they've always done: write software
- packagers do as they've always done, with the exception that their output is a container that contains the app and any necessary libraries. They can even choose to build a small number of different containers, each with slightly different options.
- There's S/W available to aid in creating new and updating/patching existing containers. Much like Docker allows you to modify part of an existing container and call it a new one, you can apply a patched (yet backwards compatible) library or application in a container and ship it as an update to the old one.
- the few "normal-use" (i.e. sorry Gentoo, you're not it ;) )distro's that are left then pick from the existing packages and compose according to wishes. Fedora would likely be a spawning ground for latest versions of all packages, while Red Hat might pick some older (but well-maintained) package with all the patching that it has seen. This also means that Red Hat could reuse packages that originally spawned in Fedora or elsewhere.
- Those that care enough can still build and publish a new container for a package with whatever options they like.

In this scheme, a package with a particular set of options gets built just once. Users and distro's get to pick and match as much as they like. Distro's can reuse work done by other distro's/users and differentiate only where they need. Much of the redundant work is gone.

Some unreliable predictions for 2015

phred14 — Thu, 15 Jan 2015 16:44:51 +0000

Implicit in this prediction appears to be the assumption that "systemd will win", and sysv, upstart, and even those Linux distributions using a bsd-style init will all move to systemd.

NOTHING in Unix or Linux has been like this - ever. Both emacs and vi are still with us, decades later. We still have sendmail, postfix, courier, exim, etc. We still have DJB versions of various programs, along with the non-DJB versions. About the closest thing we have to a monoculture is XOrg, but that wasn't by Xorg trying to stamp out SVGALIB, and even now XOrg isn't trying to stamp out Mir or Wayland.

I would like to see the systemd wars die down, too. I would just like to see it die down with several distributions using it, several distributions using other init systems, and just have the flaming stop.

Some unreliable predictions for 2015

Wol — Thu, 15 Jan 2015 09:28:46 +0000

> why can't userspace library developers and distros have the same level of quality control that the kernel has?

Maybe because too much userspace software is written by Computer Science guys, and the kernel is run by an irrascible engineer?

There is FAR too much reliance on theory in the computer space in general, and linux (the OS, not kernel) is no exception. Indeed, I would go as far as to say that the database realm in particular has been seriously harmed by this ... :-)

There are far too few engineers out there - people who say "I want it to work in practice, not in theory".

Cheers,
Wol

Some unreliable predictions for 2015

raven667 — Thu, 15 Jan 2015 04:39:38 +0000

All interesting points to be sure but I'd be more interested in imaging what would need to happen to make this work, a Linux Foundation Standard ABI would look a lot like an enterprise distro, maybe shipping new versions of leaf software (like Firefox) but never breaking core software, or a proprietary system like Mac OS X. Right now every distro claims to be suitable for end-user deployment and a target for third party development, that would be true if there were comprehensive compatibility standards, or only one dominant vendor to support, but there are not so every distros advertising is false on this point.

These deployment and compatibility problems have been solved on the proprietary platforms, some of which are even based on Linux, and the Linux kernel team provides the same or better ABI compatibility that many proprietary systems offer, why can't userspace library developers and distros have the same level of quality control that the kernel has?

Some unreliable predictions for 2015

dlang — Wed, 14 Jan 2015 23:51:20 +0000

RHEL wants to run extremely well tested versions of software, even if they are old

Fedora wants to run the latest version of software, even if they aren't well tested

how are these two going to pick the same version?

other differences between distros, do you compile it to depend on Gnome, KDE, etc. Do you compile it to put it's data in SQLite, MySQL or PostgreSQL?

Some distros won't compile some options in because they provide support for (or depend on) proprietary software.

Is an option useful or bloat? different distros will decide differently for the same option.

Gentoo says "whatever the admin wants" for all of these things, other distros pick a set of options, or a few sets of options and make packages based on this.

As an example, Ubuntu has multiple versions of the gnuplot package, one that depends on X (and requires all the X libraries be installed), one that requires QT, and one that requires neither.

Some unreliable predictions for 2015

raven667 — Wed, 14 Jan 2015 22:35:30 +0000

> I didn't think that is what we were talking about.

That confusion may be my fault as well, reading back in the thread. You can blame me, it scales well 8-)

> the same package of a given application isn't suitable for all distros
> they opt for different options when they compile

I'd be interested in taking a serious look at what kinds of options are more likely to change and why to see if there are any broad categories which could be standardized if some other underlying problem (like ABI instability) was sufficiently resolved. My gut feeling (not data) is that the vast majority of the differences are not questions of functionality but of integrating with the base OS. Of course distros like Gentoo will continue to stick around for those who want to easily build whatever they want however they want but those distros aren't leading the pack or defining industry standards now and I don't expect that to change. From my Ameri-centric view this would seem to require RHEL/Fedora, Debian/Ubuntu and (Open)SuSE to get together and homogenize as much as possible so that upstream developers effectively have a single target (Linux-ABI-2020) ecosystem and could distribute binaries along with code for the default version of the application. I guess I just don't care about the technical differences between these distros, it all seems like a wash to me, pointless differentiation for marketing purposes. It's not too much to ask developers to package once, if the process is straightforward enough.

Some unreliable predictions for 2015

dlang — Wed, 14 Jan 2015 21:48:26 +0000

> That's kind of moving the goal posts isn't it? Weren't we talking about having a better defined standard ABI of system libraries that applications could depend on

I didn't think that is what we were talking about.

We were talking about distro packaging and why the same package of a given application isn't suitable for all distros (the reason being that they opt for different options when they compile the application)

As far as each distro having a different ABI for applications, the only reason that some aren't a subset of others (assuming they have the same libraries installed) is that the library authors don't maintain backwards compatibility. There's nothing that a distro can do to solve this problem except all ship exactly the same version and never upgrade it.

And since some distros want the latest, up-to-the-minute version of that library, while other distros want to use a version that's been tested more, you aren't going to have the distros all ship the same version, even for distro releases that happen at the same time (and if one distro releases in April, and another in June, how would they decide which version of the library to run?)