Pipe dreams... [LWN.net]

Pipe dreams...

Posted Aug 15, 2008 21:22 UTC (Fri) by drag (subscriber, #31333)
In reply to: Pipe dreams... by khim
Parent article: Something going on with Fedora

> Sure. It'll create thriving support industry: since RedHat will point fingers to LSB, LSB to
Adobe and Adobe back to RedHat we'll have complex scripts to install/uninstall packages, clean
up stuff after bad packages and so on. And NOBODY will be able to help you without access to
your system - since every system will be broken in it's own unique way. Now the only question
arises: does creation of such industry is worthy goal or not? To me answer is simple: thnx,
but no, thnx.

We already have Install/Uninstall scripts for packages. You invoke them when you do a yum
install or a apt-get install to install the software then apt-get remove and yum remove when
you remove the software.

> Nope. They'll have yet another sector of work: scripts and subsystems designed to cope with
broken installation/uninstallation programs and malware removal tools.

Ya. They already do that. It's called dpkg and rpm. )Maybe your a slackware user or something?
I'm suprised you never heard of this stuff! (just kidding))

And they'll probably do the same thing that Microsoft does to uninstall malicious software.
That is: "Nothing At All" Because there is nothing you can do, and nothing you should do. You
make your system 'correct' and you make it strong and if a administrator decides to install a
rootkit into their system, however mistaken, there is fundamentally nothing possible you can
stop that from happenning besides using some draconian method of locking out the system using
TPM or something bizzare like that.

Or go with a IPhone model of software delivery. Which is kinda sorta what we have with
apt-get.

> LSB is DOA - it tries to solve problem even more complex then Microsoft's one and even
Microsoft's problem is unsolvable. Simple one-binary programs without external dependencies
work without LSB just fine and it's useless for complex programs. It all was discussed
many-many times already: it does not work IRL and it'll not work in Linux too.

It's solved for Microsoft's customers. I don't know about you, but whenever I install software
on Microsoft Windows it works. It may not work well, but it works.

This is far more then what I can say about installing any reasonably complex piece of Linux
software that isn't pre-packaged for my distribution by my distribution. It usually takes a
lot of effort, requires significant skill (relative to installing software on Windows), and
takes upwards of hours to complete with only about a 70% success rate.

I am talking about compiling software and it's dependencies from scratch and trying to get it
work on Debian Testing/Sid.

And this has _nothing_to_do_ with closed source software. Everything I am personally talking
about up to this point is open source software.

--------------------

What I would like to see is uniformity and standardization between Linux distros. Not from the
top down like LSB, but from the ground up.

They already do things like ship all similar versions of GCC, libc, Linux kernel, Gnome, KDE,
etc etc. Every modern Linux desktop oriented distribution that I've used even uses the same
programs for managing the networks.

Right now we solve problems through a brute force and highly labor intensive approach.. each
Linux distro is responsible for packaging software, debugging those packages, and all that
sort of thing to compensate for relatively little differences between them. All this huge
duplication of work for just minor differences.

So there has to be a more elegant way to deal with this stuff. There is no reason on earth
does it make sense to have 8 different groups of people working independently on packaging 8
different versions of the same exact piece software for the same exact hardware platform on,
fundamentally, the same exact software platform.

The most obvious way is to just make everything identical. Maybe that would work, maybe it
won't. But I doubt it's the only solution. If that won't work then there _has_ to be a
different way.

Maybe something along the line of the new trend of integrating package making into using
revision control systems and standards on publishing packages. Something has to be possible.

(Log in to post comments)

Pipe dreams...

Posted Aug 16, 2008 2:11 UTC (Sat) by vonbrand (subscriber, #4458) [Link]

Right now we solve problems through a brute force and highly labor intensive approach.. each Linux distro is responsible for packaging software, debugging those packages, and all that sort of thing to compensate for relatively little differences between them. All this huge duplication of work for just minor differences.

Reasonable distros do track upstream software as closely as possible, and are careful to ship bug reports (even with proposed fixes) upstream where relevant, so this "huge duplication of work" just isn't there. Distributions do share patches and setups (or swizzle them from each other, that is what open source is for), there are even cases where an upstream developer is the packager for a distribution, or somebody packages for several distributions.

Besides, I just don't see a terrible amount of work when installing something from source... unless the package is very badly done software, in which case the installation troubles are probably just the very beginning of an extremely painful experience. A useful rule of thumb is that if installation is confusing or badly done, the rest of the stuff probably matches, and should be avoided.

Yes, I lived in the pre-Linux days, when there were lots of different Unixy systems around, with real differences among them and noone packaging "extraofficial" software. That was real pain. The current situation is tame in comparison.

Sorry, but it's just lies at this point

Posted Aug 16, 2008 7:24 UTC (Sat) by khim (subscriber, #9252) [Link]

And they'll probably do the same thing that Microsoft does to uninstall malicious software. That is: "Nothing At All".

Sorry, but this does not look like nothing. And programs like this or this are not like dpkg or rpm at all. Windows model is failing - and requires a lot of crutches to work. Will it be with us for much longer? Who knows? If people will stop installing every dancing sexy screensaver they can find it'll survive - but then it'll lose a lot of appeal: you'll be forced to use very limited set of software not because there are nothing else, but because you are afraid to break the system. A lot of people are in this situation already.

It's solved for Microsoft's customers. I don't know about you, but whenever I install software on Microsoft Windows it works. It may not work well, but it works.

If you install it on freshly installed Windows - yes, sure. But if your system is few years old... chances are it'll not only not work once installed it can even kill some programs already installed! Thus there are Windows File Protection, System Restore and other related crap.

I am talking about compiling software and it's dependencies from scratch and trying to get it work on Debian Testing/Sid.

Have you tried to do the same (compile software and it's dependencies from scratch) under Windows? Try it some time. You'll probably need to find few abandoned packages, buy some tools - and in the end get some non-working piece of software because your system included wrong set of headers...

So there has to be a more elegant way to deal with this stuff. There is no reason on earth does it make sense to have 8 different groups of people working independently on packaging 8 different versions of the same exact piece software for the same exact hardware platform on, fundamentally, the same exact software platform.

They don't do this independently. Patches are fying right and left and the only truly duplicated thing is testing - in Windows world it's the same. Situations where program works fine under XP and fails on Vista (or vice versa) are common...

Sorry, but it's just lies at this point

Posted Aug 16, 2008 21:10 UTC (Sat) by drag (subscriber, #31333) [Link]

> Have you tried to do the same (compile software and it's dependencies from scratch) under
Windows? Try it some time. You'll probably need to find few abandoned packages, buy some tools
- and in the end get some non-working piece of software because your system included wrong set
of headers...

They point is _I_Don't_have_to_. Not when I just want to run the software.

Every major open source project I've seen that has Windows support supplies Windows users with
executables and supplies Linux users with source tarballs, with only a few exceptions on the
Linux side.

-------------------------------------------

I think your misunderstanding me a bit:

What I want to see is a video game like Planeshift (It's a decent Linux MMORPG which follows
the ID Software approach were the engine is GPL, but the game itself is open, but non-free)
simply supply two DEB (or whatever) packages, themselves, that outline the dependencies and
versions they need.

Then the various distributions just pull down that package. If there is something wrong then
they figure it out and send a patch back to upstream.

Right now the package makers working for the various distributions are a huge bottleneck in
the distribution of new Linux software. This is because it requires such a huge manual
brute-force approach. For some software this is the only way it's going to work, but for the
vast majority.. something I expect like 80% don't need that level of dedication from the
distributions.

Even if they supply it in deb-src format, and distributions use their servers to compile it,
then it's still worlds better then what we have now.

Get it now?

Going back to video games, in Linux there is no way for the average Linux user to compile
them. The makers of them are forced to use all those nasty scripts and install dialogs to
package them for Linux users. That's they only way they can get it to work.

And like you said they are full of nastiness like having to use scripts with ld_library_path
and statically compiled binaries and weirdness. These make the games much larger then they
need to be and make it impossible for Linux users to use anything but the very latest games on
the very latest distribution releases.

And beta testing? _FORGET_IT_. It's impossible. Even for the latest and greatest from Debian
Unstable I need to do things like setup a chroot environment because trying to install the
software I need will obliterate any hope of having apt-get not break my system. And with that
your looking at _days_ of effort.

And on top of that, because the way Linux users are utterly dependent on their
distributions-supplied packages, there is no way the average Linux users will ever know about
90% of the games that are available for Linux unless they subscribe religiously to something
like happypenguin.org,

So what we have in Linux distributions is a handful of fairly decent FPS or some smaller
'gnome' or 'kde' games and older stuff that are packaged by the distribution maintainers, but
they don't have the time or desire to track down every new release of every game out there.

And that's just games.

I could go on and on for any sort of type of software you want.

How about Host-based Intrusion detection systems? I am working on evaluating that stuff from
work and due to various thingsthat are 100% out of the control of myself, my bosses, and my
entire company, having to compile things from source is not fun. It's certainly possible, but
it adds lots of hoops. And entirely besides that I don't like to have to install a large
number of developer's tools on production machines.

How many of these are packaged by your distribution?
Samhain
OSSEC
Integrit
AIDE
Tripwire (OSS version)
Tiger

They all seem to be good, solid, and stable pieces of software. And they don't have very
complex dependencies or anything like that. Some are packaged for certain distributions, some
are packaged for others, and some are packaged for none. There is no reason why they can't
simply supply a couple binaries in their own packages so I can install them and test them out,
except that I can't. Because distributions are the gatekeepers to what is installable on my
system and they don't have the manpower to manage all of that on their own.

(OOOHHH.. that's right. I am using Linux so I don't have to care about malware or rootkits
because the packages supplied by my distribution is the perfect way to provide my system with
unbreakable security. Well that solves that problem!)

Sorry, but it's just lies at this point

Posted Aug 16, 2008 23:23 UTC (Sat) by njs (subscriber, #40338) [Link]

I'm sorry you feel so much frustration.

> How many of these are packaged by your distribution? Samhain, OSSEC, Integrit, AIDE,
Tripwire (OSS version), Tiger

I did take a quick look at this, though, and it looks like for Debian and Ubuntu the answer
is, all of them except OSSEC.  Additionally, the Tiger package appears to contain extensive
enhancements to let it make use of the dpkg database to better validate installed files.  A
quick google suggests[0] that the hold-up on integrating OSSEC is a combination of manpower,
the fact that the upstream package is garbage (seriously, /var/ossec/etc, /var/ossec/bin?),
and the fact that OSSEC is *not legal to redistribute*, because the authors don't understand
that the GPL and OpenSSL licenses are incompatible.

This is a rather nice example of how expertise in coding does not imply expertise in
distribution.  They're different skill-sets.

I see two changes you might be arguing for.  The first is that upstream authors should
habitually make their own packages.  As we see in the case of OSSEC -- and this is pretty much
the universal opinion of anyone whose dealt with any sort of vendor-produced packages ever --
this is an AWFUL IDEA because a huge percentage of upstream will give you garbage.  So as a
user, I insist on having some technical and legal gatekeeper between upstream and my machine.
In fact, the possibility of getting such a gatekeeper is generally considered to be one of the
major advantages of Linux over Windows.

The other thing you seem to argue is that okay, if we need a gatekeeper, there should still
only be one of them -- systems should be similar enough that once one person has done this
work, everyone can make use of it.  Roughly, this comes down to saying "there should only be
one distribution".  Which, well, I guess I can see the argument... but frankly it doesn't
matter how good the argument is, because as soon as you successfully got things down to one
distribution, some jerk would ignore all your hard work and start another one, and there we go
again.  But maybe it helps to reflect that having multiple distributions also creates a lot of
good to justify the bad -- it creates competition to drive development, it provides space for
many different approaches to be explored (look at e.g. all the different init systems) before
any single one is picked, etc.

Hope that helps.

[0] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=361954