Carrez: The real problem with Java in Linux distros
Carrez: The real problem with Java in Linux distros
Posted Sep 25, 2010 2:39 UTC (Sat) by steffen780 (guest, #68142)In reply to: Carrez: The real problem with Java in Linux distros by alankila
Parent article: Carrez: The real problem with Java in Linux distros
Independent of that, there's a point in updating libraries - that fixes things. Using up to date libs means there's hundreds of bugs you'll never run into. And just because a developer's use of a program works just fine with ancient library version X, doesn't mean that users don't benefit from or even require e.g. unicode-related fixes in the newer library version X+10. The java model merely delays the hassle of ensuring your code works with newer dependencies, it doesn't avoid it. And nobody serious is demanding that every app developer always follows the latest commits of all his dependencies.
There's also no real reason not to release a source tarball alongside your binary release. It almost seems spiteful "well, on windows a source tarball is useless, so if you use Linux, BSD or OSX with MacPorts then just go away". Seriously, how hard is it to make a source tarball of a project? 2 seconds typing plus a few seconds for the compression to take place? How much effort is that compared to having to contend with random library versions that you have to manually update, and to create a giant mess called "a simple .jar"? Why would any upstream be _wanting_ to have to deal with libraries in this fashion, it's just baffling.
My own (python-)project releases:
- a source .tbz
- packages for Gentoo and Debian/Ubuntu that do not include dependencies, but merely specify them (e.g. >=python-2.6)
- a self-extracting .exe archive for Windows that does include all the dependencies, from an appropriate python version to GTK and PyGTK.
Java projects could release .jar (which, I want to note, is less handy then a self-extracting exe, because running a jar requires the user to install java herself) instead of the exe. What's the big deal?
And I don't care how easy anyone says it is to bundle your project's dependencies into a jar - it simply cannot be easier than simply making a list like this:
dev-python/matplotlib
>dev-python/pygtk-2.16
(...)
So whilst I admit that the Java way has advantages in some situations, in others the Linux way has many advantages. For a start, being available by the distribution mechanism the user expects is important for a consistent user experience - Windows users expect to download some executable from a random website and run it - not my thing, but hey, to each his own. But Linux users expect to simply type the name of the program into synaptic, or apt-get, or yum, or emerge, and then have the system take care of everything. Why not let them?
And from a project's point of view:
Best case: You get free publicity (being in the repos and their search engines), free testing of various combinations of libraries, pre-filtered bug reports, and users of the distros you're in won't ask you an install question ever again.
Worst case: You lose 5 seconds on each release from creating and uploading a tarball.
Seems crazy not to take this deal.
Posted Sep 25, 2010 8:35 UTC (Sat)
by nim-nim (subscriber, #34454)
[Link] (1 responses)
Linux tools have grown in the wild and focus on making good practices (regular updates, security, adapting to a changing environment, portability across compilers...) easier. Because if you don't play well with others, the FLOSS community marginalizes you.
Java tools (Apache included) have grown in the enterprise and focus on hiding the problems generated by the bad practices PHBs mandate for various reasons (Usually, short-term gain for someone else's long term pain. Except everyone is someone else's someone else, so everyone loses).
And it didn't help that this kind of practice ran SUN in the wall and SUN focused those last years on playing licensing games, not annoying its partners with correct, but unpopular changes, re-implementing other's stuff that already worked instead of fixing its own problems, and lastly tried to pimp itself up by promising more than it could deliver (Witness the crazy JVM release tree, with countless branches that all promised something different, none of which was ever finished. The first thing Oracle announced for Java was stopping this madness and merging all the branches, plus the bits it inherited from BEA, in a single JVM release). So the bad example came from the very top of the Java ecosystem.
Posted Sep 25, 2010 20:41 UTC (Sat)
by khc (guest, #45209)
[Link]
Posted Sep 25, 2010 20:50 UTC (Sat)
by NAR (subscriber, #1313)
[Link] (10 responses)
Unfortunately that's not true. Updating libraries (or any software) usually means replacing a set of bugs with a new set of bugs. Even old bugs get reintroduced from time to time, see the Linux kernel for current example.
Linux users expect to simply type the name of the program into synaptic, or apt-get, or yum, or emerge, and then have the system take care of everything. Why not let them?
The problem is that the Linux users can only get the version that is in that version of the distribution, they can't have an older or newer one. See example about Evolution in this page. I also presume there's no Linux distribution which doesn't have pulseaudio but has Firefox 3.5+ - you can't have this in Linux easily, but I can install Firefox 3.5 on the 9 years old Windows XP.
Posted Sep 25, 2010 21:40 UTC (Sat)
by sfeam (subscriber, #2841)
[Link]
That is simply not true. I can't speak to all the various distros, but certainly in mandriva there are back-compatibility packages for previous versions of common libraries going back quite a way. And you can often pull a newer version from Cooker.
Posted Sep 25, 2010 21:48 UTC (Sat)
by nicooo (guest, #69134)
[Link] (1 responses)
Gentoo, Arch, CrunchBang, Slackware, ...
Posted Sep 25, 2010 22:01 UTC (Sat)
by foom (subscriber, #14868)
[Link]
Posted Sep 26, 2010 6:18 UTC (Sun)
by drag (guest, #31333)
[Link] (2 responses)
Try installing Win98 audio subsystem to replace Windows XP's and see how far you get.
I know that in Linux-think it's hard to separate between applications and operating systems. We all have a tendency to want to think of them in a unified item... which is good in one way, but bad in another.
-------------------------------
The problem is that in Linux there is no real separation between what is 'applications' and what is the 'OS'.
I am sure that we all agree to the concept of 'do one thing right and do it well' and the concept of 'layers'.
That a complex system should have layers were you can work on improvements at one layer without disturbing the upper layers very much.
Like: Oh, I can upgrade my wireless drivers without breaking the browser.
Right? That makes sense.
That is why with the Linux kernel they make a clear distinction between 'No Internal API or ABI!!!!' and 'We support external API/ABI compatibility as a high priority!!!'.
The idea is that:
* 'Well we want the freedom to hack the kernel anyway we want. If we depend on internal ABI/API compatibility then that will hold us ransom to popular proprietary-only companies that will use their popularity to control us while contributing nothing!'
* 'Well, but we still want people to us our software. So we need to make sure that we can upgrade and change the kernel but not piss all the users off and break everything.'.
So, right: LAYERS.
So the API and ABI for Linux kernel takes the form of various IOCTLs, Posix file systems, Posix IPC, sockets, semephores, /dev directory, /proc, sysfs, etc etc etc. That is Linux kernel's API and they generally do a good job at preserving compatibility between different versions. Not 100%, but pretty decent all things considered.
But here is how things look from a distro perspective right now:
{Linux kernel}(Linux ext. API) --- > {OMG USERSPACE!!!!!1111oneoneoneo}
Everything above the kernel is a mush. A mixture of a hundred different things that people simply slap together and get working in whatever fashion pleases them.
For the most part I can build my own kernel and slap a Linux distro on top of that and it'll probably function decently. Sometimes I can break stuff, but it's going to be relatively minor. I can even quite happily use a Redhat kernel with Debian and probably things will work fine.
But compared to that installing a Debian packaged software on Redhat then all hell breaks loose and it's going to be just about as shitastic as it can get.
I can compile my own software and it'll work with pretty much complete disregard to the package system. I can also do binary downloads of many pieces of software... like Firefox or Chrome or Blender3D and as long as I do not try to work with the package management software then it'll usually work out very well regardless of the distro.
Even with proprietary software like Opera that uses C++ and QT, which I am told is going to be shit because of ABI issues and whatnot.... the binaries they provide are compiled the same completely regardless of what distro they are installed on.
Go and look at the different packages that Opera offers... They have a dozen different packages to work with a dozen different distros, but they only have a couple actually different binaries they provide. Last time I looked they would provide the same exact binaries for fedora, debian, ubuntu, etc etc... with the only significant difference being that they supported a old version of Ubuntu that was built using GCC 3.xx or something like that were (I am guessing) they did actually run into C++ ABI issues.
All the incompatibility exists in the package management systems. And it's just stupid stuff like 'were to stuff the documentation' or crap like that.
So Linux and distros are already doing something right. They just are not aware of it on a higher level or something like that.
Posted Sep 26, 2010 19:28 UTC (Sun)
by skybrian (guest, #365)
[Link] (1 responses)
I think the main problem is that some people want Linux distros to do too much. They don't have the resources and need to step back a bit. There's no reason to provide an all-encompassing solution like in C.
Linux distributions should provide a solid JDK and fix security bugs in it, and provide a standard way to install applications written in Java (which can bundle their own libraries). Then security holes in applications are officially Not The Distribution's Problem. Let application-level projects do that.
Posted Sep 27, 2010 3:38 UTC (Mon)
by cmccabe (guest, #60281)
[Link]
That's basically the situation as it exists today. You can easily use yum and apt to get a JDK, and then install your own jars with wget.
I think Thierry Carrez feels that this situation makes Java on the Linux desktop a "second-class citizen." I can easily install C, C++, python, or perl programs with apt-get. I'll get security updates and bugfixes each time I do apt-get upgrade. It's not just "someone else's problem." I can't do that with Java apps today.
Posted Sep 26, 2010 19:26 UTC (Sun)
by mfedyk (guest, #55303)
[Link]
some distributions like fedora/rhel have policies that specify only one version of a lib, but there is nothing in the package manager that keeps you from having multiple separate packages with different versions of a lib. I know that debian does this a lot with libfoo and libfoo12, etc. it just has more maintenance overhead to have another package to keep updated.
Posted Sep 27, 2010 13:07 UTC (Mon)
by buchanmilne (guest, #42315)
[Link] (1 responses)
So, let's not presume any further.
Posted Sep 29, 2010 18:29 UTC (Wed)
by nevyn (guest, #33129)
[Link]
yum -C list firefox pulseaudio
Posted Sep 29, 2010 18:22 UTC (Wed)
by nevyn (guest, #33129)
[Link]
I guess you've never paid for Linux then? RHEL (and I'd assume all distros. people actually pay for) has every version ever released available, just a simple "yum downgrade" (on RHEL-5) away. There are also the minor releases too, like 5.3.z, where you'll just never even see some newer versions of packages.
Posted Sep 26, 2010 2:47 UTC (Sun)
by mikov (guest, #33179)
[Link] (1 responses)
No, Linux users don't expect it - many of them simply have no other choice as installing software outside of the distro repository is simply very difficult.
I have said this before but it bears repeating - the notion that all software that users could possibly need can sit in a distribution repository is laughable.
Most of the software I use on day to day basis on my Linux desktop didn't come from the distribution repository. It couldn't possibly and even if the distro did package some of it, I still wouldn't use the packaged version because I can't rely on the distribution to keep up with the upstream changes.
Posted Sep 27, 2010 14:37 UTC (Mon)
by buchanmilne (guest, #42315)
[Link]
That doesn't mean that there is no value in fixing the broken tools that are the only stumbling block for software that could otherwise be supplied in the distribution repos.
Posted Sep 26, 2010 19:21 UTC (Sun)
by alankila (guest, #47141)
[Link] (5 responses)
It also breaks things. Frequent breakage of everything is a commonplace thing for me between each update of Ubuntu, as I tend to follow the devlopment branches out of interest. So no. Updating stuff breaks stuff, if done willy-nilly, you have to admit that much.
> With the java way a user of 100 java apps will have 100 versions
But every one of those 100 apps probably works with respect to the features that the app developer tested. Nobody else has gone in and changed something and screwed it up.
See, this is what I had to do in my old working place. At first we tried to package our dependencies into nice system-wide installs, but before long we ran into needing to make changes, break APIs, etc. and realized that we pretty much have to either update all the apps, or install new libraries with different names. (Perl did not support any versioning internally.) So we went for application-specific installs: apps come with wrappers that push a bunch of app-local package paths into the library search path, and then we went our merry way with never a problem. Seems fairly similar to the ".jar contains dependencies to me", at least in contextual level. The reason it is done is always the same: to have application still work even if the surrounding system changes.
I will mostly skip the discussion about source tarballs, it doesn't seem to really apply to me. In case of a .jar, no compiling is necessary, so Linux or OS X look just the same as Windows with respect to source tarball being equally useless. Right?
> Java projects could release .jar (which, I want to note, is less handy then a self-extracting exe
To run a Linux application, you need to download Linux first, too. My point being, we all want to make some assumptions about the platform we are running on. It would be nice to have JDK on the OS out of the box, or have it contained in the application binary itself, as some people would almost certainly benefit from this design. However, such a design is somewhat uncomfortably too close to crazy, and the only real reason you might really want to is because you have an application that in fact requires a very specific JDK to work. (Boo to such, etc.)
> For a start, being available by the distribution mechanism the user expects is important for a consistent user experience
Now we get to an interesting point. See, if distributors could give up on this mode of thought where every application must be chewed into tiny bits and spit out as part of the OS, we could already get all the java apps without anybody complaining. The complaints are not solely leveled at java, though, I saw firefox and chrome get their share of ire for forking libraries into their source tree not long ago.
To me, an OS's primary function should be just to contain and manage applications as blobs. When distros try to do more, they easily cause just the problems that application writers wanted to avoid in the first place!
For instance, Debian very nearly had to give up on packaging Eclipse at all because they simply could not successfully package Eclipse since 3.2.3 or so. This apparently activated a number of people, and Eclipse updated to version 3.5.2 at some point. Thinking to myself "Great! I can now stop downloading the tarball myself!", I tried it out but found quickly that code completion didn't work. Back to .tar.gz world. Working software is juts so far more important than the other arguments which I have heard about.
Posted Sep 27, 2010 3:24 UTC (Mon)
by ringerc (subscriber, #3071)
[Link] (2 responses)
The problem is that when a security problem arises, you have to find every copy of every library in every app, all different versions, and make sure they're properly patched.
Maybe if there was a reasonable, nearly universal way to declare that your app bundles these versions of these libraries, with tools that automatically manage the bundled sources, it'd be ok. But what happens instead is that each bundling app integrates the bundled libraries into its own funky build system and often starts patching them in odd ways. you can almost never just drop in fresh upstream sources.
At least Maven helps with this - you may declare dependencies on specific versions, but you don't pull them into your source tree. They're only bundled (optionally) at build-time.
Posted Sep 28, 2010 11:39 UTC (Tue)
by alankila (guest, #47141)
[Link] (1 responses)
Posted Sep 29, 2010 8:56 UTC (Wed)
by ringerc (subscriber, #3071)
[Link]
You can't have a buffer overflow that writes to arbitrary bits of memory when working with a String object - but you can still suffer from string length/truncation bugs. For example, if the program does a validity check on input, then truncates it for storage/use (perhaps in a database with a length-limited field and a JDBC driver that silently truncates long input) you can possibly trick the app into accepting invalid input. That could be a security issue.
You might find a code path that permits the bypassing of certain authorization checks in a webapp. Or an SQL injection attack caused by improper string interpolation. Or a way to trick the app into doing improper file system access for you. A way to modify parts of a data structure that you're not meant to be able to, because a JavaScript<->Java binding in a webapp exposes whole objects to JS, not just authorization-checked individual members.
Java tries to avoid making you walk through a security minefield as part of using the core language, but you can still shoot your own foot of in any language.
The SecurityManager can help protect apps from bugs in user/app code, but it has to be enabled and in use. It's used by default in applets and Java Web Start apps, but not for locally-executed apps. Glassfish v3 enables it too. Many other things don't, so there's no sandbox acting as a second layer if the app's built-in checks fail to prevent something. OTOH, if the security issue is with data/actions *within* the app, the SecurityManager won't help you anyway.
Java can help with security issues, but it's no magic bullet.
Posted Sep 27, 2010 4:38 UTC (Mon)
by nicooo (guest, #69134)
[Link]
After the libpng bug in June, I don't blame the distros for being unhappy.
Posted Sep 30, 2010 9:15 UTC (Thu)
by nim-nim (subscriber, #34454)
[Link]
> But every one of those 100 apps probably works with respect to the
And what happens when the app needs to be updated to add new customer-requested features? It can't because the local fork of Java deps is missing the features that were added in the real upstream version. So everything needs to be re-done.
The Java way does not solve anything it just delays the payback (That's one reason big Java apps cost astronomical prices, and have slow release cycles. The language is nice and should enable cheaper/faster development, but the abysmal release practices add up in multiple ways in the long term)
Posted Sep 27, 2010 3:18 UTC (Mon)
by ringerc (subscriber, #3071)
[Link]
For anyone else frustrated by this, there's a non-password protected mirror at http://svn-mirror.glassfish.org/ .
That said, most other projects I've dealt with are pretty good about releasing sources. They're usually not very good at pushing them to maven along with their binary artifacts, but they tend to publish them on their website.
Personally I think a big part of the trouble is release management. Ant is bloody awful; everyone has their own weird and wonderful recipe for disaster. Maven *should* help, but the bad documentation and lack of guidance on how to use it tend to lead to pom files full of fixed versions, with hard-coded repositories in the pom, dodgy settings for package deployment, etc. People write Maven poms by waving the chicken feet around and muttering their preferred arcane incantataions, because few people actually understand how the darn thing works or how it's *meant* to work.
Unlike rpm/deb, there's very little good guidance on writing Maven poms, or ant builds, that're friendly to users of your code, not just to the developers of it.
Carrez: The real problem with Java in Linux distros
Carrez: The real problem with Java in Linux distros
Independent of that, there's a point in updating libraries - that fixes things. Using up to date libs means there's hundreds of bugs you'll never run into.
Carrez: The real problem with Java in Linux distros
The problem is that the Linux users can only get the version that is in that version of the distribution, they can't have an older or newer one.
Carrez: The real problem with Java in Linux distros
Carrez: The real problem with Java in Linux distros
Carrez: The real problem with Java in Linux distros
Carrez: The real problem with Java in Linux distros
Carrez: The real problem with Java in Linux distros
Carrez: The real problem with Java in Linux distros
> it, and provide a standard way to install applications written in Java
> (which can bundle their own libraries). Then security holes in
> applications are officially Not The Distribution's Problem. Let
> application-level projects do that.
Carrez: The real problem with Java in Linux distros
Carrez: The real problem with Java in Linux distros
Unfortunately that's not true. Updating libraries (or any software) usually means replacing a set of bugs with a new set of bugs. Even old bugs get reintroduced from time to time, see the Linux kernel for current example.
By your logic, when re-introduced bugs are re-fixed, user shouldn't update, in case other old bugs are re-introduced ...
The problem is that the Linux users can only get the version that is in that version of the distribution, they can't have an older or newer one.
s/Linux/Fedora/
I also presume there's no Linux distribution which doesn't have pulseaudio but has Firefox 3.5+
Pulseaudio has been around for some time (many distros started shipping it in 2007), so you have to choose distros which have > 2 year lifetimes, IOW, not fedora, but not counting EPEL, CentOS5/RHEL5 counts:
# yum -C list 2>/dev/null|grep -E "(^pulseaudio\.|^firefox)"
firefox.i386 3.6.7-3.el5.centos updates
pulseaudio.i386 0.9.10-1.el5.3 epel
Carrez: The real problem with Java in Linux distros
Carrez: The real problem with Java in Linux distros
> that version of the distribution, they can't have an older or newer one.
Carrez: The real problem with Java in Linux distros
But Linux users expect to simply type the name of the program into synaptic, or apt-get, or yum, or emerge, and then have the system take care of everything. Why not let them?
Carrez: The real problem with Java in Linux distros
the notion that all software that users could possibly need can sit in a distribution repository is laughable.
Carrez: The real problem with Java in Linux distros
Bundling libraries is a security issue
Bundling libraries is a security issue
Bundling libraries is a security issue
Carrez: The real problem with Java in Linux distros
Carrez: The real problem with Java in Linux distros
> features that the app developer tested. Nobody else has gone in and
> changed something and screwed it up.
Source releases