| |||||||||||||
Wheeler: Insecure open source software libraries?Wheeler: Insecure open source software libraries?Posted Apr 6, 2012 22:44 UTC (Fri) by khim (subscriber, #9252)In reply to: Wheeler: Insecure open source software libraries? by vonbrand Parent article: Wheeler: Insecure open source software libraries?
Quite the contrary, distributions usually push new library versions out aggressively. Rilly? Let's check.
More often then not distribution includes some kind of obsolete version. This is not necessarily distribution's fault: if you'll just blindly update libraries then you'll probably disrupt tons of things, but this also means that if you don't bundle libraries with your application then you don't really know what version of library you are getting. And since there are tons of distros around the problem becomes quite severe.
(Log in to post comments)
Wheeler: Insecure open source software libraries? Posted Apr 6, 2012 23:04 UTC (Fri) by dlang (✭ supporter ✭, #313) [Link]
at the risk of feeding the troll that you are acting like
what distro are you looking at for these versions.
what is the timeframe between the release of the shipped version and the current version?
Linux distros tend to update the libraries with each release, but not update the library versions during a release (although they may backport fixes, which makes the version number mean a little less)
Wheeler: Insecure open source software libraries? Posted Apr 7, 2012 12:19 UTC (Sat) by khim (subscriber, #9252) [Link] at the risk of feeding the troll that you are acting like I've included links. It's the latest version of Ubuntu LTS. Linux distros tend to update the libraries with each release, but not update the library versions during a release (although they may backport fixes, which makes the version number mean a little less) Yup. And that means that if you want to support LTS distros like RHEL/CentOS or Ubuntu LTS then you must support old versions of libraries then it's much easier to bundle then - in this case you know you always have the version you've included.
Wheeler: Insecure open source software libraries? Posted Apr 7, 2012 15:45 UTC (Sat) by Del- (guest, #72641) [Link]
"Ubuntu LTS then you must support old versions of libraries then it's much easier to bundle then - in this case you know you always have the version you've included."
If you want bleeding edge libs, you need to bundle them, and that is quite trivial to do. However, your conclusion is way off. With Ubuntu LTS you know exactly what version you are getting, and you know that you will be getting bug-fixes and security fixes too, without you having to do anything. Quite contrary to when you bundle, when you (as application distributor) will need to track and update libs with all bug-fixes and security-fixes.
On one hand you criticize Ubuntu LTS for not being rolling (i.e., keeping API and major versions of libraries fixed), and on the other hand you complain about not knowing which library it ships. You seem very hard to please.
Wheeler: Insecure open source software libraries? Posted Apr 7, 2012 17:36 UTC (Sat) by khim (subscriber, #9252) [Link] With Ubuntu LTS you know exactly what version you are getting Rilly? Ubuntu Lucid Lynx includes freetype 2.3.11, Ubuntu Precise Pangolin includes freetype 2.4.8. And Ubuntu LTS is just one flavor of Linux among many! Just Ubuntu gives as four different versions at the same time and then there are Debian, Fedora, OpenSUSE, etc. They all include different versions of libraries and most of them include different local patches. This Q&A nightmare! you know that you will be getting bug-fixes and security fixes too, without you having to do anything. Well, this is nice “feel good” advantage from user's POV, but from developer's POV it's not an advantage at all: instead of tracking the upstream fixes now I must track and redo all the Q&A work every time some library gets the security update and Linux distributions stampede with fixes. On one hand you criticize Ubuntu LTS for not being rolling (i.e., keeping API and major versions of libraries fixed), and on the other hand you complain about not knowing which library it ships. You seem very hard to please. Sure. But that's because there are intrinsic conflict: it's easier to develop something if you always deal with latest version of the software but if you want to deploy something then you need stable foundation. The usual solution outside of Linux is to provide stable foundation and allow developer to bundle newer or forked version of libraries if s/he wants/needs them. Linux provides unstable foundations (dozens of libraries must be supported/tested at the same time) and tries to forbid to bundle libraries to solve this problem, too. IOW: it combines the worst properties of both worlds. No wonder developers are not happy.
Wheeler: Insecure open source software libraries? Posted Apr 7, 2012 18:57 UTC (Sat) by Wol (guest, #4433) [Link]
But you miss. LTS stands for "Long Term Support". That means it doesn't change! You're tilting at windmills.
The libraries will have any relevant bug fixes applied - that's what LTS effectively means. After all, doesn't RHEL (the equivalent of Ubuntu LTS) still actively support kernel 2.4 in at least one guise !!! ???
Cheers,
Wheeler: Insecure open source software libraries? Posted Apr 8, 2012 13:06 UTC (Sun) by nix (subscriber, #2304) [Link]
Wasn't he also suggesting that the ideal was a system where all the libraries never change? This makes complaining about LTS systems where exactly that property is true somewhat peculiar.
Wheeler: Insecure open source software libraries? Posted Apr 8, 2012 14:48 UTC (Sun) by khim (subscriber, #9252) [Link] Wasn't he also suggesting that the ideal was a system where all the libraries never change? Not exactly, but close enough. This makes complaining about LTS systems where exactly that property is true somewhat peculiar. Except I'm not complaining. I was just pointing out that vonbrand's statement (distributions usually push new library versions out aggressively) is a lie. Distributions most definitely don't push new library versions out aggressively - and that means that quite often the best alternative is to bundle library with the application. The alternative will be a way to supply applications with required versions of a library which bypass the distributions - and I'm not sure if such system is feasible or even possible.
Wheeler: Insecure open source software libraries? Posted Apr 9, 2012 8:48 UTC (Mon) by nix (subscriber, #2304) [Link]
Distributions *do* push new library versions out aggressively. Specifically, they push out security fixes, and they push out publically released ABI/API changes when and only when there are things in the distro themselves that depend on them.
If you don't do the latter, you end up being trapped by things like the short-lived libjpeg7, which broke soname and required a massive relink only to change soname *again* and require another huge relink when you'd just finished the last one.
Wheeler: Insecure open source software libraries? Posted Apr 9, 2012 9:23 UTC (Mon) by khim (subscriber, #9252) [Link] Distributions *do* push new library versions out aggressively. No, they don't. Specifically, they push out security fixes, Usually, but not always. YMMV, as usual. and they push out publically released ABI/API changes when and only when there are things in the distro themselves that depend on them. IOW: they most definitely don't “push push new library versions out aggressively”. They do it only when it's convenient for them. Note: I don't see this as anything “wrong” or as “something which should be fixed”. But since they don't usually push new library versions out aggressively it's often easier to bundle new version of the library with your program rather then try to persuade distributions to do that for you. If you don't do the latter, you end up being trapped by things like the short-lived libjpeg7, which broke soname and required a massive relink only to change soname *again* and require another huge relink when you'd just finished the last one. Yup. This may be valid justification for distributions but what about poor applications which needed/wanted to use JPEG7 when JPEG8 was not yet available?
Wheeler: Insecure open source software libraries? Posted Apr 9, 2012 14:56 UTC (Mon) by nix (subscriber, #2304) [Link] Yup. This may be valid justification for distributions but what about poor applications which needed/wanted to use JPEG7 when JPEG8 was not yet available?Were there any? Major libraries with significant improvements (rather than the tiny ones in libjpeg7) will get packaged. Minor ones might not. In that situation, yes, you'll probably have to bundle them if unavailable on the distro -- but look at icculus's porting efforts to see how much work this isn't. This is really not as big a problem as you're making it out to be. (For games, at least, the state of Linux 3D support was much more of a problem until recently.)
Wheeler: Insecure open source software libraries? Posted Apr 10, 2012 9:43 UTC (Tue) by khim (subscriber, #9252) [Link] For games, at least, the state of Linux 3D support was much more of a problem until recently. That's because you can not bundle it with a game. The things which you can bundle (some kind of ffmpeg, lua or mono, etc) are just bundled without any considerations for the distribution's plight. You don't hear cries about games from the distribution packagers not because they don't bundle as much as Chromium is bundling, but because games mostly are from the “proprietary commercial software” category and as such is ignored by distributions totally. One example: World Of Goo. List of bundled libraries includes libSDL and libogg, zlib and libpng… Everything is bundled. Only three libraries are not bundled: GLibC, libstdc++ and libGL. And that's because you can not really bundle them: libGL is hardware dependent and for technical reasons usually needs shared GLibC and libstdc++… Some game developers try to play by distribution's rules and usually are burned: it's much harder to install these games because the libraries they require are just not there.
Wheeler: Insecure open source software libraries? Posted Apr 10, 2012 10:41 UTC (Tue) by Cato (subscriber, #7643) [Link]
Good point about some games that don't bundle libs - this might be what happened to me with Crayon Physics Deluxe (in one of the Humble Indie Bundles), as I couldn't easily get it working on Linux and ended up using Windows.
Wheeler: Insecure open source software libraries? Posted Apr 12, 2012 15:26 UTC (Thu) by nye (guest, #51576) [Link]
>Only three libraries are not bundled: GLibC, libstdc++ and libGL. And that's because you can not really bundle them: libGL is hardware dependent and for technical reasons usually needs shared GLibC and libstdc++…
Probably not co-incidentally, that also appears to be a more-or-less exhaustive list of libs which care deeply about long-term ABI compatibility. And I'm not so sure about libstdc++.
(I'm semi-joking, but mostly not)
Wheeler: Insecure open source software libraries? Posted Apr 7, 2012 19:12 UTC (Sat) by Del- (guest, #72641) [Link]
Ubuntu Lucid Lynx is the current LTS. Precise Pangolin is not released, and hence not relevant. Ubuntu is one flavour among four relevant flavours. Let me repeat, support one or a couple of the four, and you will find little resistance. It is very common to only support one version of RHEL and SLES, as long at is is the latest version, we are happy. The rest is a problem for the community. I know, as an application developer you still get the heat. No, you don't, make your supported distro explicit, people are not that dense. When you explicitely tell them their distro is not supported, it is all up to their distro to provide compatibility. This works surprisingly well. Worst case scenario, Red Hat gets more paid customers. It is as easy as that. Not perfect, but the model works, and has for many years. The real problem is when app developers don't bother supporting a current version of _any_ distribution (do you have any experience on this?). OpenSuse even provides tools for packaging both debs and rpms, the real world problem is that proprietary developers don't know how to package at all, not that packaging is so damn difficult.
"I must track and redo all the Q&A work every time some library gets the security update and Linux distributions stampede with fixes."
No you don't. The only trouble you will find is your own damn bugs. And most developers are quite sloppy there. Please try to keep some decent perspective.
"it's easier to develop something if you always deal with latest version of the software"
Indeed. On this we agree. Developers have a religious tendency to use bleeding edge libraries. They simply must have these latest features. As I already said, then you bloody well have to bundle or compile static. Bundling libraries can be done quite easily in linux (given that some libraries like Qt have complexity that can be a nuisance). I can help you out in your next project if you like.
Wheeler: Insecure open source software libraries? Posted Apr 7, 2012 19:24 UTC (Sat) by rqosa (subscriber, #24136) [Link] Freetype isn't a good example for you to pick there — app developers seldom use it directly, but instead use it indirectly from a widget toolkit library. And I believe that the major toolkits try to have backwards compatibility throughout each major version (so apps developed for version x.y should run on x.(y+1), though not vice-versa — and sometimes C++ ABI changes have prevented this), and allow multiple major versions to be installed alongside each other. (And maybe freetype is like that too, I'm not sure…)
Wheeler: Insecure open source software libraries? Posted Apr 7, 2012 20:40 UTC (Sat) by khim (subscriber, #9252) [Link] Freetype isn't a good example for you to pick there — app developers seldom use it directly, but instead use it indirectly from a widget toolkit library. This is exactly why it's good example. Apps “don't use it directly” yet it still affects them. Surprisingly enough it's not even version difference but configuration difference. If you prerender something and then write text near prerender using freetype-based library then you can only achieve good result if freetype used has the same options (especially WRT bytecode interpreter) - and these options are unknown to you if you use system version of freetype!
Wheeler: Insecure open source software libraries? Posted Apr 8, 2012 7:57 UTC (Sun) by rqosa (subscriber, #24136) [Link] It seems like you're insisting that the upstream app developer must do "quality assurance" on every possible OS or distribution that the app can possibly run on. That's just fundamentally impossible in the absence of an OS monoculture, and by its very nature GNU/Linux cannot become a monoculture. (That would mean getting rid of all but one distro, and there's no way this is possible when all of the platform components are FLOSS.) So you just can't do that kind of QA on this platform. Linux users understand this is so and still keep using it, because the benefits of non-monoculture outweigh the downsides.
Wheeler: Insecure open source software libraries? Posted Apr 8, 2012 10:59 UTC (Sun) by khim (subscriber, #9252) [Link] It seems like you're insisting that the upstream app developer must do "quality assurance" on every possible OS or distribution that the app can possibly run on. No. If some platform uses programs designed for other platform - it's their choice, in this case OS designers should provide Q&A. This is very common case: it happens every time new version of OS is released (as usual Linux desktop developers shirk this responsibility, but other OS vendors are more serious about it). But yes, if developer releases program for some platform (especially if said program is sold for $$) then s/he must do Q&A - or else why release anything at all? That's just fundamentally impossible in the absence of an OS monoculture, and by its very nature GNU/Linux cannot become a monoculture. You don't need monoculture. Android releases include a lot of customizations - yet most of them don't affect app developers at all. The observed problems are mostly caused by changes in hardware (for example all the programs which assumed you can use trackball to move around are basically unusable on Galaxy Nexus S… yet they still run and use can use them with external mouse). Linux users understand this is so and still keep using it, because the benefits of non-monoculture outweigh the downsides. Sorry, but this is bullshit. This description may cover some small percentage of Linux users but most of them think that when something does not work on Fedora (usually because of SELinux) it's our fault even if the identical package works on Ubuntu. So no, they don't “understand this”. If you'll visit any FOSS conference you'll see how many former Linux users finally understood it… and decided that life is too short to play these games. Most of them are now MacOS users, but some returned back to Windows.
Wheeler: Insecure open source software libraries? Posted Apr 8, 2012 18:31 UTC (Sun) by rqosa (subscriber, #24136) [Link] > But yes, if developer releases program for some platform (especially if said program is sold for $$) then s/he must do Q&A - or else why release anything at all? The developer shouldn't try to do QA on every Linux distribution out there — doing it on just one major distro should be sufficient, and the user community will figure out how to get it to work on any other distro where there's enough demand for the app in question. > Android releases include a lot of customizations - yet most of them don't affect app developers at all. You could say the same thing about Windows — when it's preinstalled on PCs, it usually has customizations specific to that PC model and its manufacturer. As far as the platform APIs are concerned, though, it's still a monoculture. > If you'll visit any FOSS conference you'll see how many former Linux users finally understood it… and decided that life is too short to play these games. Yaaawwwnnnnn… We've heard a billion variations on the theme of "Linux is dying" (much of which was thinly-disguised propaganda FUD) for more than a decade now, and it's not even once been true. As far as I'm concerned, the Linux user experience just keeps getting better and better as time goes on, and that could hardly happen if users were abandoning the platform in droves. > Most of them are now MacOS users, but some returned back to Windows. Funny you should say that. 11 years ago, I thought that Mac OS X was great — it should run all the Unixy stuff I'm used to, plus most of the proprietary stuff that's not available on Linux, so what's not to like? Then I tried to actually use it, and quickly got frustrated by how many hoops I had to jump through to do things that on Linux would have needed little more than "apt-get install foo" or "./configure && make install". Since then I've been unwillingly dragged back to it at least once, and it was just as bad as ever. I dare you to try this: build the latest Git snapshot of FFmpeg on Mac OS X. I tried that once (well, except FFmpeg was still using SVN back then), and it went something like this: first I had to untar the Fink tarball into a directory, then figure out how to change sone config file to enable building source packages from the unstable branch of Fink, then run the command to build it — which made it download lots of stuff, compile said stuff, download lots more stuff, compile it, and on and on and on until I ran out of patience.
Wheeler: Insecure open source software libraries? Posted Apr 9, 2012 1:49 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link]
>Then I tried to actually use it, and quickly got frustrated by how many hoops I had to jump through to do things that on Linux would have needed little more than "apt-get install foo" or "./configure && make install"
So that's why you use fink ( http://www.finkproject.org/ ) or simply do "./configure && make install" on Mac OS X.
Oh, and there's that newfangled marketplace for consumer software.
Wheeler: Insecure open source software libraries? Posted Apr 9, 2012 21:48 UTC (Mon) by rqosa (subscriber, #24136) [Link] > So that's why you use fink ( http://www.finkproject.org/ ) or simply do "./configure && make install" on Mac OS X. Did you read the paragraph below that? Like I said there, I did use Fink — and even so, I found it very frustrating to do things that were easy to do with Linux distributions / package managers. For one thing, there seemed to be lots of packages which were available only as source packages in the "unstable branch" (I forget the exact terminology it used), which required me to edit a config file to switch to that branch (or else the package manager would say that the package in question doesn't exist), and I had a hard time finding the documentation that explained how to make that change. And then once I got that far, trying to build the package made it take a huge amount of time downloading and compiling dependencies and build-dependencies, to the point that I never even found out whether it actually finished or not. (This all happened during a group meeting for a class project, where I was trying to install FFmpeg on someone else's computer so he could use it to transcode video files, and it was still downloading/compiling things when I left for the day.) By comparison, building source packages on Linux distributions (such as AUR packages on Arch Linux) has always seemed a lot easier to me. I suppose it's partly because Mac OS X by default has next to none of the dependencies and build-dependencies installed (e.g. build tools aren't installed, and neither are lots of the libraries which generic-Unix programs like ffmpeg depend on and which are likely already installed on a machine running a Linux distro), and partly because the developer community for Fink is smaller than that of the major Linux distributions and/or unofficial package repositories like the AUR (and other Fink-like projects, such as MacPorts, seem to be even smaller) thus it has fewer packages.
Wheeler: Insecure open source software libraries? Posted Apr 9, 2012 23:17 UTC (Mon) by CycoJ (guest, #70454) [Link]
> You don't need monoculture. Android releases include a lot of customizations - yet most of them don't affect app developers at all. The observed problems are mostly caused by changes in hardware (for example all the programs which assumed you can use trackball to move around are basically unusable on Galaxy Nexus S… yet they still run and use can use them with external mouse).
Funny, have you actually looked at the review comments for apps on the Android market? It's full of reviews like "does not work on Samsung ..., HTC ...", "Please make this work on HTC..."
Wheeler: Insecure open source software libraries? Posted Apr 10, 2012 10:28 UTC (Tue) by khim (subscriber, #9252) [Link] I was on both sides of the fence… Do you know just why there are all these endless messages? 9 times out of 10 (if not 99 times out of 100) that's because said HTC or Samsung have older version of OS then the one clearly written in the app requirement. This means that complainers don't even think about OS upgrade (we have the same situation with NaCl which does not work with MacOS 10.6.7) - and you expect that they will track and install application dependencies? Preposterous.
Wheeler: Insecure open source software libraries? Posted Apr 26, 2012 9:51 UTC (Thu) by steffen780 (guest, #68142) [Link]
Small reality check: go and look how easy it is to upgrade the average Android phone. Once you found out you can come back and complain about lazy users failing to update their devices using official or easy updates that almost always don't exist; or using root access that generally requires reading pages upon pages upon pages to get. If it's possible at all.
And of course if you belong to the 1% or so of people who have the required skillset you have to give up your warranty (according to the manufacturers at least, I for one would love to see this tested in court).
The situation is slightly better on iOS, but even there updates are only available for a fraction of the useful life of these extremely expensive devices.
Wheeler: Insecure open source software libraries? Posted Apr 7, 2012 20:33 UTC (Sat) by gioele (subscriber, #61675) [Link]
> now I must track and redo all the Q&A work every time some library gets the security update and Linux distributions stampede with fixes.
And that is not a problem because you have an automated test suite, don't you?
Wheeler: Insecure open source software libraries? Posted Apr 7, 2012 20:48 UTC (Sat) by khim (subscriber, #9252) [Link] It's notoriously difficult task to write UX test suites. It's possible to do some automatic tests, but ultimately it's not a replacement for Q&A, sorry.
Wheeler: Insecure open source software libraries? Posted Apr 7, 2012 0:32 UTC (Sat) by smokeing (guest, #53685) [Link]
Debian-shipped libpng-1.2.xx is not only actively maintained upstream, but also comes with own set of patches on top of it to close any issues upstream hasn't had time to deal with. I would bet that at any time whenever a vulnerability is reported against libpng, debian will be the first to fix it and ship it safe, no matter what exact version it be and how 'old' it may look.
Concerning the main point, as a gentoo user, I ditched chrome even before it finished building as soon as I noticed the (huge!) source tarball contained a shitload of '3rd party' libraries that are *already* on my system.
Wheeler: Insecure open source software libraries? Posted Apr 7, 2012 12:27 UTC (Sat) by khim (subscriber, #9252) [Link] Debian-shipped libpng-1.2.xx is not only actively maintained upstream, but also comes with own set of patches on top of it to close any issues upstream hasn't had time to deal with. Well, this makes it even worse, isn't it? Now I have not only contend with old version of library, I must track differences between distributions, too. Concerning the main point, as a gentoo user, I ditched chrome even before it finished building as soon as I noticed the (huge!) source tarball contained a shitload of '3rd party' libraries that are *already* on my system. Well, that's your choice. I think Chrome developers know they are losing some users with “my way or the highway” mentality - and I think it's conscious choice.
Wheeler: Insecure open source software libraries? Posted Apr 7, 2012 12:39 UTC (Sat) by rqosa (subscriber, #24136) [Link] > Well, this makes it even worse, isn't it? Now I have not only contend with old version of library, I must track differences between distributions, too. No, you don't need to track differences between distributions. You just need to decide which version of the API to develop against, and then trust that the distributions will either ship a non-vulnerable and API-compatible variant of library or cease shipping any library with that API version. (In the latter case, that means they've stopped shipping your program too, until you port it to a currently-maintained version of the API.)
Wheeler: Insecure open source software libraries? Posted Apr 7, 2012 13:01 UTC (Sat) by khim (subscriber, #9252) [Link] This is fine if your goal is to “participate in advancement of FOSS ecosystem”. If your goal is to “create something for real users” then this approach does not work: distributions clog the communication between developer and user yet usually have no resources to fix my application anyway. The last bit (that distribution feel they have the right to decide for me if users deserve to see my app or not) is especially insulting - this is Apple-worthy level of arrogance. But at least Apple provides access to huge (and lucrative!) market in exchange. What do the distributions do to deserve such position?
Wheeler: Insecure open source software libraries? Posted Apr 7, 2012 14:59 UTC (Sat) by rqosa (subscriber, #24136) [Link] > If your goal is to “create something for real users” then this approach does not work It works for me — and don't you dare try to tell me I'm not a "real user". > The last bit (that distribution feel they have the right to decide for me if users deserve to see my app or not) is especially insulting - this is Apple-worthy level of arrogance. No it isn't. The distributions aren't actively preventing you from using non-distribution-provided software; some of them even encourage doing just that. Whereas Apple does actively prevent you from using unapproved software on iOS.
Wheeler: Insecure open source software libraries? Posted Apr 10, 2012 12:56 UTC (Tue) by michaeljt (subscriber, #39183) [Link]
> The distributions aren't actively preventing you from using non-distribution-provided software; some of them even encourage doing just that.
What percentage of users do you think are using a distribution which encourages the use of third-party software? My feeling was that the majority prefer you to either only use software they package (in the version they package) or software specifically targeted at and integrated with their distribution.
Wheeler: Insecure open source software libraries? Posted Apr 13, 2012 13:04 UTC (Fri) by nix (subscriber, #2304) [Link] What percentage of users do you think are using a distribution which encourages the use of third-party software?Ubuntu (via PPAs)? Lots.
Wheeler: Insecure open source software libraries? Posted Apr 13, 2012 16:51 UTC (Fri) by khim (subscriber, #9252) [Link] PPAs are right step politically, but awful step technically: its very easy to thoroughly hose the system with just a few PPAs. To make them really effective and safe to use you need the very same "stable ABI promise" which can make appstore-like model usable.
Wheeler: Insecure open source software libraries? Posted Apr 26, 2012 9:56 UTC (Thu) by steffen780 (guest, #68142) [Link]
It's very easy to hose your shining example Windows by installing random stuff off the internet. This is why we have distros, to reduce the need for installing random stuff off the Internet. That applies to any OS. Well, except iOS, where you're prevented from installing stuff that isn't blessed by Apple (unless you root it of course - and can hose your system by installing random stuff off the internet).
Wheeler: Insecure open source software libraries? Posted Apr 10, 2012 12:58 UTC (Tue) by michaeljt (subscriber, #39183) [Link]
khim wrote:
> The last bit (that distribution feel they have the right to decide for me if users deserve to see my app or not) is especially insulting - this is Apple-worthy level of arrogance. But at least Apple provides access to huge (and lucrative!) market in exchange. What do the distributions do to deserve such position?
Just out of interest, what software do you work on? You seem to have very strong feelings on software distribution issues.
Wheeler: Insecure open source software libraries? Posted Apr 10, 2012 14:12 UTC (Tue) by khim (subscriber, #9252) [Link] Well, it's not a secret. We are in somewhat unique position because we are providing packages for different OSes and, more importantly, we talk with developers and hope to attract them to our platform. This means that we see not only the distribution problems but also observe the developer's needs and wants WRT the new, nascent platform. There are many different observations (not all of them I can talk about publicly), but the key observation is: developers care about potential future users, they don't care about platform - and why should they? What they especially don't care about is the rules of said platform: for developers these rules are obvious obstacles which must be overcome somehow to reach the user. Necessary evil, nothing more. Traditionally Linux desktop just ignored Joe Average users and Joe Average developers. It was content to play with their existing rules and their existing developers. But now GNOME/KDE/etc are clearly trying to attract new Joe Average users - yet distributions refuse to talk with Joe Average developers. IOW: Linux desktop developers have said A but they still are refusing to say B. WTF? What's this hoopla is all about?
Wheeler: Insecure open source software libraries? Posted Apr 10, 2012 14:44 UTC (Tue) by anselm (subscriber, #2796) [Link] The last bit (that distribution feel they have the right to decide for me if users deserve to see my app or not) is especially insulting - this is Apple-worthy level of arrogance. That would depend not only on your app but also on the distribution. Debian, for example, seems to be happy to take anything that is (a) distributable by Debian and (b) supported by somebody who will see to packaging the software for Debian. So if you're a distribution and you don't package an app you're »arrogant«. If you do package as much as you can package, you get flak for overwhelming your users with choices. In any case you get booed for not offering 500.000 packages like the iOS app store does (even if 499.000 of those »apps« are either glorified bookmarks, trying to sell you something, or are otherwise uninteresting or useless). Seems there is no pleasing everybody all the time.
Wheeler: Insecure open source software libraries? Posted Apr 10, 2012 20:51 UTC (Tue) by khim (subscriber, #9252) [Link] That would depend not only on your app but also on the distribution. Debian, for example, seems to be happy to take anything that is (a) distributable by Debian. Yup. That's what I'm talking about. Apple is cursed for it's huge 30% cut while Debian demands price point of zero (and source code to boot). Seems there is no pleasing everybody all the time. It's impossible in principle. The problem is not that Debian manages to piss of somebody. The problem is that Debian pisses of 99% of users and 90% of developers (perhaps even more: there are more developers among Linux users but I'm not sure the difference is 10x).
Wheeler: Insecure open source software libraries? Posted Apr 10, 2012 23:05 UTC (Tue) by anselm (subscriber, #2796) [Link] Apple is cursed for it's huge 30% cut … Where's the problem? As a book author I would be enthusiastic if my publisher would let me keep 70% of the proceeds of selling my books. … while Debian demands price point of zero (and source code to boot). The whole point of Debian is providing a free (as in freedom) OS. While free (as in freedom) software does not need to be free as in beer, the Debian repositories do not have a coin slot, and that is generally considered a Good Thing™. On the other hand, a piece of software does not actually need to be inside the Debian repository to be usable on a Debian-based system. There is no reason whatsoever why somebody could not use their own repository to make their commercially-licensed software available for Debian, with dependencies on the Debian repository if required. With a steep price tag and no source code. The problem is that Debian pisses of 99% of users … This is probably not true. My company provides Linux instruction, among other things, and we get a constant stream of requests from people who run, or are interested in running, Debian-based installations. In fact, the proportion of Debian work we do is steadily rising while the proportion of SLES work (which used to be our bread-and-butter business) is going down. We certainly do more work based on Debian these days than we do based on RHEL or even CentOS.
Wheeler: Insecure open source software libraries? Posted Apr 10, 2012 23:16 UTC (Tue) by dlang (✭ supporter ✭, #313) [Link]
the Ubuntu PPA approach works very nicely for adding an extra repository for some special application, and the really nice thing is that once it's setup, updates for this application work just like updates for all the other software on the system.
Wheeler: Insecure open source software libraries? Posted Apr 7, 2012 15:55 UTC (Sat) by Del- (guest, #72641) [Link]
"Now I have not only contend with old version of library, I must track differences between distributions, too."
No, support the latest stable version of one or a couple of the major distributions (debian, ubuntu, rhel, sles). Those who run Arch will deal with it with few complaints. It is not hard to do and works wonders for proprietary software in the enterprise. You will need the occasional recompile every other year, that's basically it. It is not perfect, but it is certainly not the nightmare you make it out to be.
Wheeler: Insecure open source software libraries? Posted Apr 7, 2012 7:19 UTC (Sat) by scientes (guest, #83068) [Link]
libpng 1.2->1.5 transition in debian http://release.debian.org/transitions/html/libpng1.5.html
yes, 1.5 is in debian experimental pre-transition http://packages.debian.org/source/libpng
it so happens that the main change is that 1.5 drops support for an already-deprecated direct access to an internal structure, of which it has long provided a helper function. However, 1.0, 1.2, and 1.5 receive security updates (not as sure about 1.0 anymore)
Wheeler: Insecure open source software libraries? Posted Apr 7, 2012 11:26 UTC (Sat) by rqosa (subscriber, #24136) [Link] > More often then not distribution includes some kind of obsolete version. Obsolete != vulnerable. Those obsolete versions in Ubuntu might be "stable branches" with backported security fixes, maintained either by the upstream, or Debian, or Ubuntu. Oh, and in the other thread you were complaining that the library APIs change too often, but those "current" library versions you're saying here that Ubuntu ought to include probably have backwards-incompatible API changes. You can't have it both ways.
Wheeler: Insecure open source software libraries? Posted Apr 7, 2012 12:33 UTC (Sat) by khim (subscriber, #9252) [Link] Oh, and in the other thread you were complaining that the library APIs change too often, but those "current" library versions you're saying here that Ubuntu ought to include probably have backwards-incompatible API changes. You can't have it both ways. Of course not! The biggest problem with distro-supplied libraries is not even the fact that they provide obsolete version of libraries. This is similarly to what other, saner, platforms are doing. The biggest problem is that you have to work with unknown version of library. One distribution will have libpng 1.2, another one will have libpng 1.5, one will include some fixes backported from older version of library, another will have bugs. WTF? Why should I support all that zoo? It's often easier and simpler to bundle library with the application. OS should mostly provide facilities I can not easily bundle with the application, not provide bazillion of libraries with constantly changing ABI.
Wheeler: Insecure open source software libraries? Posted Apr 7, 2012 12:55 UTC (Sat) by rqosa (subscriber, #24136) [Link] > One distribution will have libpng 1.2, another one will have libpng 1.5, one will include some fixes backported from older version of library, another will have bugs. If a distro-provided library has bugs, then that's not the application developer's fault, and shouldn't be treated as such. > WTF? Why should I support all that zoo? You shouldn't. Instead, you should just pick an API-version of the library that's still maintained, and support only that one. The distributions will generally include any still-maintained version of a library as long as there is other software in the distribution that depends on it, and when they drop it, it's time for any remaining apps to either migrate away from the obsolete library or else get dropped by the distributions. > OS should mostly provide facilities I can not easily bundle with the application That sounds like Mac OS X, where you often have to update the whole OS to the next version in order to run the one program you want. No thanks!
Wheeler: Insecure open source software libraries? Posted Apr 7, 2012 13:20 UTC (Sat) by khim (subscriber, #9252) [Link] If a distro-provided library has bugs, then that's not the application developer's fault, and shouldn't be treated as such. As user I don't care if the root cause of problem lies with Intel, Debian or Google. If application does not work then it's author is to blame. Unless application explicitly says that my hardware or software is deficient I'll blame application author. You may say it's not fair, but this is how Joe Average thinks. That sounds like Mac OS X, where you often have to update the whole OS to the next version in order to run the one program you want. No thanks! Sorry, but no. In Mac OS X you need to update the whole version of OS to run newer version of application 1 time out of 10, on Linux it's more like 5 times out of 10 (and 4 times out of 10 you can try to run an application yet it'll exhibit some strange errors which will be blamed on the distribution and the solution offered will be “upgrade your OS, then try again”… 2 times out of 10 said solution will not help). Thnx, but no, thnx.
Wheeler: Insecure open source software libraries? Posted Apr 7, 2012 14:54 UTC (Sat) by rqosa (subscriber, #24136) [Link] > this is how Joe Average thinks. So what? Let them blame whoever they want to — it doesn't mean they're right. > on Linux it's more like 5 times out of 10 Not in my experience. On Linux, one can usually track down the dependencies for a particular program and install them in an out-of-the-way location (e.g. not in /usr) in order to get it to run. Good luck trying the same thing on Mac OS X…
Wheeler: Insecure open source software libraries? Posted Apr 7, 2012 21:40 UTC (Sat) by drag (subscriber, #31333) [Link]
> That sounds like Mac OS X, where you often have to update the whole OS to the next version in order to run the one program you want. No thanks!
The model to compare Linux package management solution is other, more effective, solutions like with Android and iOS.
The entire userland is versioned with a static API between minor versions. Not just individual libraries, but the application API is presented as a complete set.
Application developers are then free to drop their application, built the way they want to support it, into the ready-to-install pool of applications presented to the user via the package management system.
Then the package management system presents packages to users based on what versions of APIs that their OS supports. So you could have applications built for Linux versions years ago that are still installable as long as the API version they need is still supported.
With Linux desktop things need to be a bit more fined grained then that that, but I think it should be very possible if you present standard APIs in blocks.
For example a Gnome desktop may have these levels of API support:
Linux Kernel --> userland = 'set in stone API' (as per kernel current policy).
Core plumbing ---> The init system, low level libaries, system utilities, udev, server stuff. Distros can use whatever they want as long as it supports a 'common denominator' set of APIs that people can depend on. These would be free to update, break compatibility when necessary, every 3-4 years or so. Which is on par with competing proprietary desktops.
Generic desktop personality ---> Stuff that can be ignored if you are doing a server install. X libraries, Alsa, common media libraries, CUPS etc. Versioned in parallel with 'Core plumbing' with 3-4 years between breaking things.
Gnome personality ---> Ultimately up to project consideration how they want to support the API and what APIs need to be supported, but it must be versioned.
Ideally you want each distro manage the 'kernel', 'core plumbing', and 'generic desktop personality' how they see fit.
'Gnome/KDE/XFCE/etc personalities' should be managed and be compiled by upstream. That is Fedora and Debian would use the same binaries and same libraries built by Gnome.
No compiling and customization should be done on the distribution ideally. If distributions want branding and incorporate their trademarks into the desktop then they need to work with upstream on how to do that through themeing packages. That way the distributions can be kept honest.
That way we can go along way to making things a lot friendlier and would be a big first step into making Linux a lot easier for people to install their games and such.
Wheeler: Insecure open source software libraries? Posted Apr 8, 2012 8:18 UTC (Sun) by rqosa (subscriber, #24136) [Link] > The model to compare Linux package management solution is other, more effective, solutions like with Android and iOS. "More effective", yeah right. The insecure-bundled-libraries problem is endemic to those systems, just like it has traditionally been on that other "standard API" platform that's called Java SE. > I think it should be very possible if you present standard APIs in blocks. Every platform with "standard APIs" that's ever been tried has caused the exact sort of problem that this article describes. The platform APIs will never be enough for all apps, so the developers must turn to third-party libraries — and there's no way to use these without bundling them.
Wheeler: Insecure open source software libraries? Posted Apr 8, 2012 10:36 UTC (Sun) by rqosa (subscriber, #24136) [Link] > just like it has traditionally been on that other "standard API" platform that's called Java SE. And furthermore, more recently some Java developers are adopting something that's very similar to Linux-style package management (complete with transitive dependency resolution), because the standard API just isn't enough.
Wheeler: Insecure open source software libraries? Posted Apr 8, 2012 11:03 UTC (Sun) by alankila (subscriber, #47141) [Link]
I doubt this is actually very similar. Maven makes dependency management a breeze, but these libraries are versioned and people do specify the exact versions of the libraries they want in their pom.xml files.
Wheeler: Insecure open source software libraries? Posted Apr 8, 2012 19:00 UTC (Sun) by rqosa (subscriber, #24136) [Link] It's not all that different. In package management systems, packages also can specify a particular version (or minimum/maximum version) to depend on. (Though it's true that often they don't actually do this — likely because it's becoming common for library packages to include an "API major version number" right in the package name, e.g. "freetype2" or "kdelibs3" or "allegro4". This way allows for installing multiple major versions of a library in parallel, and also ensures that the most recent API-compatible versions of all dependencies for all installed apps are present.)
Wheeler: Insecure open source software libraries? Posted Apr 9, 2012 2:03 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link]
>It's not all that different. In package management systems, packages also can specify a particular version (or minimum/maximum version) to depend on.
In Maven you _must_ specify a particular version (or version range). Your POM won't be valid without it.
Wheeler: Insecure open source software libraries? Posted Apr 9, 2012 8:08 UTC (Mon) by micka (subscriber, #38720) [Link]
It's possible to specify a version range for a dependecy, like [1.0,1.5] or even [5.6,], though that's not exactly common practice.
Wheeler: Insecure open source software libraries? Posted Apr 9, 2012 8:12 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link]
I've seen it used with stuff like log4j or Spring where binary compatibility is kept between major versions.
Wheeler: Insecure open source software libraries? Posted Apr 9, 2012 2:01 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link]
Nope, it's most definitely NOT very similar.
Maven has two modes of operation for libraries: SNAPSHOT and fixed versions. In fixed version mode Maven simply downloads the referenced libraries. They are immutable, they are _not_ updated automatically and can be cached in a local repository (like http://www.jfrog.com/products.php ).
In the SNAPSHOT mode Maven tracks the most recent version of referenced libraries. This mode IS similar to what Linux does. However, this mode is used by all real project only for development, and quite explicitly NOT for releases.
I.e. you develop with SNAPSHOT versions, tracking the newest library versions. Then you fix the library versions (there's even a plugin for that: http://maven.apache.org/plugins/maven-release-plugin/inde... ) and push it for QA.
Wheeler: Insecure open source software libraries? Posted Apr 9, 2012 21:26 UTC (Mon) by cmccabe (guest, #60281) [Link]
In the real world, nobody uses Maven's ability to download the most recent versions of referenced libraries-- except maybe with libraries that are part of their own project. The problem is that Java has nothing like an API level. So if you tell Maven "just get the latest version" you don't know if you're getting something that's API-compatible with your code. So you have mysterious build breakages on random developer machines, and eventually someone edits the .pom file to use a fixed version. This is why in Hadoop, we use fixed versions for all external libraries.
This problem was solved with shared libraries in Linux a long time ago by introducing the concept of an API level. New versions of shared libraries can describe what previous revisions they are API-compatible with, This is the funny three-tuple you see at the end of shared library names. However, nothing similar to this exists in Java-- at least to my knowledge; I'd be happy to be corrected here.
This endemic "pinning" of dependencies to specific version numbers in Java can cause unexpected problems. For example, if your project tries to rely on version 1.23 of library A and version 4.56 of library B, but library B itself tries to pin version 7.89 of library A-- what do you do? Sometimes including two different versions of the same library in the project can cause problems at runtime. For an example of this, see https://issues.apache.org/jira/browse/HADOOP-8104
Wheeler: Insecure open source software libraries? Posted Apr 9, 2012 22:16 UTC (Mon) by rqosa (subscriber, #24136) [Link] > However, this mode is used by all real project only for development, and quite explicitly NOT for releases. I'm aware that when building a release with Maven, it makes a .jar which contains all dependencies. But that's beside the point — what I meant was that even though Java SE has a fairly large standard "platform API", it's still not enough for all apps, and that's why Java developers now use a package repository with a constantly-changing selection of libraries (unlike the set-in-stone "platform API") and also a tree of dependencies together with dependency resolution in the package manager (unlike the supposedly "more effective" Android Market and iOS App Store, which lack dependency resolution and therefore require all packages to depend on the "platform API" only).
Wheeler: Insecure open source software libraries? Posted Apr 9, 2012 22:27 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link]
>I'm aware that when building a release with Maven, it makes a .jar which contains all dependencies.
No it doesn't.
>But that's beside the point — what I meant was that even though Java SE has a fairly large standard "platform API", it's still not enough for all apps, and that's why Java developers now use a package repository with a constantly-changing selection of libraries (unlike the set-in-stone "platform API")
Most libraries in Java have excellent ABI compatibility. Besides, changes in Maven packages DO NOT affect dependent projects. I.e. right now my Clementine player on Ubuntu doesn't work because of some DBUS accessibility-related changes (and I'm too tired to check why).
Maven doesn't work that way. It emphasizes repeatable builds, i.e. I can checkout a release from 2 years ago from our repository and build a byte-perfect copy with all the dependencies preserved. In fact, Maven actually complains if you do something that breaks repeatable builds.
Debian packages provide nothing like this.
Wheeler: Insecure open source software libraries? Posted Apr 10, 2012 19:54 UTC (Tue) by rqosa (subscriber, #24136) [Link] > No it doesn't. According to here, > Debian packages provide nothing like this. That's because old package versions don't remain in the repository, and it doesn't allow installing multiple versions of the same package in parallel. (However, like I already said, different API-versions of the same dependency will often have different package names from each other, thus allowing them to be installed in parallel, and also allowing a package to depend on "the most recent compatible version" of each of its dependencies.) But again, this is all beside the point. drag was saying that there should be a "versioned […] static API" for the entire set of userland APIs on Linux; I responded that Java already tried this approach, which is what's to blame for the problem (insecure bundled libraries) that the original article is about, and the existence of Maven shows that Java developers are migrating away from the notion of a single "static API" to a system of multiple separately-versioned library APIs with a tree of dependencies among them (i.e. exactly what isn't done on iOS and Android, the OSes drag says have "more effective" package management).
Wheeler: Insecure open source software libraries? Posted Apr 10, 2012 20:05 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link]
>According to here, mvn package does bundle dependencies when building a package that has <packaging>war</packaging> in its POM.
Nope, maven-war-plugin does it. By itself Maven doesn't do any packaging.
Besides, even maven-war-plugin doesn't pack all dependencies into one JAR file, it simply copies them into WEB-INF/lib directory of a web application.
>That's because old package versions don't remain in the repository, and it doesn't allow installing multiple versions of the same package in parallel.
Yup. That's why it's different from Maven.
>I responded that Java already tried this approach, which is what's to blame for the problem (insecure bundled libraries) that the original article is about, and the existence of Maven shows that Java developers are migrating away from the notion of a single "static API" to a system of multiple separately-versioned library APIs with a tree of dependencies among them
Nope they don't. Maven is a freaking _build_ tool, it's NOT a package management tool. So it HAS to work with versions and dependencies.
You mean it's similar because it has a notion of dependency tree? Duh. Even 'make' has it.
Wheeler: Insecure open source software libraries? Posted Apr 10, 2012 21:17 UTC (Tue) by rqosa (subscriber, #24136) [Link] > Nope, maven-war-plugin does it. That fine distinction doesn't matter here. > Besides, even maven-war-plugin doesn't pack all dependencies into one JAR file, it simply copies them into WEB-INF/lib directory of a web application. But the WEB-INF/lib directory is in the WAR file — therefore the applications and its dependencies are bundled into a single archive file. > Maven is a freaking _build_ tool, it's NOT a package management tool. It's obviously more than just a build tool; there are plenty of build tools (probably most of them) that have neither transitive dependency resolution for external dependencies nor the ability to download said external dependencies from a central repository. > You mean it's similar because it has a notion of dependency tree? No. It's similar because it uses transitive dependency resolution to determine what packages from a central package repository (one that's hosted on a single public server on the Internet and used by default on each Maven installation, that is; not just something that's on the local HDD) are required by the project to be built, then downloads them from there and uses them to build and/or run the project. > Duh. Even 'make' has it. Make's dependency tree doesn't extend beyond the files that are part of the project to be built; it doesn't have any real notion of external dependencies. Higher-level build tools like cmake and autoconf/automake have a notion of immediate external dependencies, but no tree of external dependencies (and therefore no transitive dependency resolution for external dependencies) and no ability to retrieve dependencies from a central repository. (Though it's possible to build an external-dependency-resolution system on top of make, as the FreeBSD ports tree shows — but I consider the FreeBSD ports tree to be a package manager too.)
Wheeler: Insecure open source software libraries? Posted Apr 10, 2012 22:07 UTC (Tue) by khim (subscriber, #9252) [Link] I still don't understand what you are trying to discuss here. Yes, maven is similar to Linux distribution - so what? Linux distributions are quite good for development. They are abysmal for deplayment. And here Maven offers the solution: prepare package suitable for use with a given version of J2EE ABI. Administrator does not need to know what dependencies are there in a given package - s/he only need to know version of J2EE needed by a given WAR and version of J2EE supported by given server. Easy, simple, unambiguous. In a sense this approach nicely combines Linux distribution development advantages with Appstore-style deployment advantages. The only problem: it's tied to Java which by itself does not form suitable desktop environment.
Wheeler: Insecure open source software libraries? Posted Apr 10, 2012 23:02 UTC (Tue) by rqosa (subscriber, #24136) [Link] > I still don't understand what you are trying to discuss here. I've explained this twice already, here: even though Javaand here: drag was saying that there should be a "versioned […] static API" for the entire set of userland APIs on Linux; I responded that Java already tried this approach, which is what's to blame for the problem (insecure bundled libraries) that the original article is about, and the existence of Maven shows that Java developers are migrating away from the notion of a single "static API" to a system of multiple separately-versioned library APIs with a tree of dependencies among them (i.e. exactly what isn't done on iOS and Android, the OSes drag says have "more effective" package management).TL;DR version: Any "standard platform API" will never be enough for all apps on the platform; this is why developers using Java (which has a very large "standard platform API") now use a central package repository with an ever-increasing selection of libraries and a dependency tree. > In a sense this approach nicely combines Linux distribution development advantages with Appstore-style deployment advantages. I suppose it does help solve the problem of insecure-bundled-libraries, because it makes it easier to track the newest library versions. But still, it seems that the insecure-bundled-libraries problem persists, because Java library developers tend to follow the convention that "release version number == API version number" and break API compatibility often (leaving the old API versions unmaintained), which leads to "endemic "pinning" of dependencies to specific version numbers".
Wheeler: Insecure open source software libraries? Posted Apr 10, 2012 23:56 UTC (Tue) by rqosa (subscriber, #24136) [Link] And also, when using Maven to build apps as WARs (i.e. "Appstore-style deployment"), there's still the issue of multiple copies of the same library (leading to wasted storage space and RAM, and to a single library bugfix requiring an update for all apps using it).
Wheeler: Insecure open source software libraries? Posted Apr 11, 2012 9:56 UTC (Wed) by alankila (subscriber, #47141) [Link]
As an application developer these are acceptable tradeoffs. Disk space is very cheap, and jar files can not truly be shared between jvm processes anyway because the jar file is not useful per se, the class files are usually deflate-compressed, and they contain symbolic references that are resolved at class loading time to speed up interpretation. Then there's the whole jit compilation step whose result is affected by the surrounding code that uses the library.
|
|||||||||||||