Weekly Edition
Return to the Development page
| |||||||||||||
Quotes of the week
In my opinion, the problem with Linux on the Desktop is rooted in
the developer culture that was created around it.
— Miguel de Icaza
Linus, despite being a low-level kernel guy, set the tone for our community years ago when he dismissed binary compatibility for device drivers. The kernel people might have some valid reasons for it, and might have forced the industry to play by their rules, but the Desktop people did not have the power that the kernel people did. But we did keep the attitude.
What better to do on a Friday than test a @#%!-load of modems and phones? But hey, I know what’s better… drowning your incredible frustration with firmware and hardware in a kiddy-pool full of cheap booze. Isn’t hardware great?
— Dan Williams(Log in to post comments)
Quotes of the week Posted Aug 30, 2012 9:41 UTC (Thu) by khim (subscriber, #9252) [Link] Miguel's article is quite nice but the quote is probably the most misleading piece of it. For some reason Miguel tries to shift the blame on the innocent person. Yes, Linus rejected idea of binary drivers - and he was right: it's impossible to create stable ABI for drivers. The proof is in the pudding: all other OSes which claimed that they have stable ABI break it regularly and older devices are becoming useless. End of discussion. But Linus also supported the idea of stable ABI for programs - and here, too, he was absolutely right: this is possible, feasible and works. Linux, MacOS, Solaris, Windows and countless other systems do provide stable ABI for the userspace programs (for the ones who don't want to install their own kernel driver, that is) and they work. Linus is absolutely anal in application of it's other rule: we don't break user-space. If desktop developers have decided to notice one rule and ignored another one then they are to blame, not the Linus. In Linus's case these two rules nicely complement each other: system which has all boundaries frozen can not evolve. You need some layer where breakage is allowed - and Linus decided that for kernel it'll be the whole thing. Sensible approach for normal (non-"micro") kernel. But you also need some boundary where breakage is not allowed because otherwise your creation can only be ever used in isolation. And for kernel it's userspace ABI. Also sensible approach. Where is this boundary in GNOME? In KDE? In other desktops? Note: some people are trying to claim that this boundary exist. Yes, some projects are serious about such things - gstreamer is one example. But most projects just don't understand that stable ABI is not a checkpoint and it's not something you do for the PR. It provides the continuity: if you introduce new stuff (Android 4.0, for example, has brought new GPU-enhanced rendering pipeline to the mobile phones) but it continues to run the old programs - they are have the same platform which evolves over time (and when programs used by majority of users will not use the older ABI you can drop it even if the exact point is debatable). But if you introduce new stuff which breaks everything and forces the recompilation of the whole world… well, Microsoft tried that. It had 12% of smartphone market before his grandiose backward-incompatible revolution. Now it has less than 3% - and this is after billions spent on marketing and with sales of the biggest smartphone OS cannibalized for the sake of this new thing.
Quotes of the week Posted Aug 30, 2012 10:30 UTC (Thu) by hummassa (subscriber, #307) [Link]
KDE does the stable-abi thing, inside a major version of kdelibs. And it allows the user to run KDE3-libs and KDE4-libs apps simultaneously and Qt-only apps, too, and Gtk-apps, with theming etc.
Quotes of the week Posted Aug 30, 2012 12:17 UTC (Thu) by ebassi (subscriber, #54855) [Link] But you also need some boundary where breakage is not allowed because otherwise your creation can only be ever used in isolation. And for kernel it's userspace ABI. Also sensible approach. Where is this boundary in GNOME? In KDE? In other desktops? this whole approach is nice, and neatly applicable to the kernel: there's a pretty fundamental boundary between "internal user" and "external user", and that can be maintained indefinitely. the approach starts breaking down really quickly when applied to more encompassing projects like GNOME and KDE. what's an "internal user" of GNOME? what's an external user of GNOME? if we consider applications as "external users", it means that the libraries have to maintain two different API surfaces: one between them, and one for the applications to use. on the other hand, applications will most likely have to subclass widgets, or write custom ones. this means that GTK+ cannot have an ABI for widget users and one for widget implementors without creating a potential source of regressions for a class of applications that the GNOME project cannot (and won't) control. applications are not the only consumers, though; there are libraries based on other libraries, which can be used by non-core applications. these are usually developed without an API/ABI stability guarantee, until they get used everywhere and are frozen and included inside the core platform (happened various times in the history of GNOME). this happens because you cannot rewrite all the code all the time, and some functionality has to organically grow from the ecosystem. GStreamer, Telepathy, Clutter, just to mention a few, have been developed by third parties and are now part of the core platform - and they are all API/ABI frozen as part of this. at this point, the only option for third parties is to start bundling the parts of the platform whose ABI they have to rely on with their own applications - something which is part of the effort for GNOME OS, and that was discussed during GUADEC, but that has been (obviously) attacked by packagers and distribution people, as well as security-oriented people. the API surface size is also another difference between kernel and GNOME that makes the whole "GNOME didn't learn the lesson from the kernel": you cannot write a modern application just by defining a small set of syscalls and a bunch of files on a virtual file system. otherwise we'd all be using Plan9.
Quotes of the week Posted Aug 30, 2012 22:27 UTC (Thu) by dlang (✭ supporter ✭, #313) [Link]
> what's an external user of GNOME?
quoting myself from a comment on a different thread ( https://lwn.net/Articles/514179/ )
> Linux Desktop people need to figure out how to do less in their Desktop Environment, not more. Identify those things that really must be integrated, and then carve off all the other apps to let them develop (or fail) on their own. The fact that each DE has it's "official" browser is silly. Nobody uses it, they all use either Firefox or Chrome (or possibly Opera). E-mail clients are in the same category. They should not be part of the DE, they should be separate apps (possibly developed by some of the same people), and if they are good, people will use them, even on different desktops. If they aren't good, they will die and people will use good ones.
If you try and make "everything" (or even, "everything that a user interacts with") be part of your project, you are doomed to failure (because you cannot possible do "everything"
In the kernel space there are frequent debates over the question of if such-and-such functionality belongs in the kernel or should be external. There need to be similar debates in the DE camps. Just because you CAN make something depend on internal functionality doesn't mean that you should. Even if the internal API is 1000% faster than the external API, if it's something that is run once per user login, it's probably not a performance difference that matters in real life.
Quotes of the week Posted Aug 31, 2012 9:32 UTC (Fri) by ebassi (subscriber, #54855) [Link] quoting myself from a comment on a different thread that is definitely not what I was asking, and that's not even an answer in this specific context. I'm talking about ABI and conformance for third party software, not performance. given that I wasn't clear: let's assume for a minute that a fairy on night gave us an internal ABI for the core platform (libraries and applications) and an external ABI for the rest of the core platform consumers (i.e. developers); this split allows us to have the core platform moving without breaking third party applications ever: if something breaks in the internal ABI, it will be fixed on the gnome.org repository by the author of the change. mostly like how the kernel works. now, what are the side effects? we effectively have two libraries, one for the internal and one for the external ABI. do we load twice the amount of shared objects? I mean, apparently Enlightenment convinced everyone that it's fine to have a bajillion DSO, as long as you keep repeating that it's fast (hint: it's really not, and anyone with a minimal understanding of how kernel and linkers work knows this), but I don't think it'll scale. do we compile stable and unstable symbols in the same shared object? that works, but it requires re-linking the entire planet because you really want to bump the SONAME so that applications using the internal ABI can actually be updated; if you bump the SONAME, though, applications linking against the external ABI will also need to be relinked, otherwise they won't be able to find the shared object - look at all the apps linking against boost, and breaking after 12 months because nobody ships the old version of boost they use any more. so at this point third party apps are in the same situation as they are now, even if they use a supposedly stable ABI. what should they do? dlopen() the shared object? fun times - ask any GL developer how that works out - and it does not scale with a shared library of 5000+ symbols or more. ISVs tried to do that with gtk+ multiple times, it didn't work. but all of this assumes that the ABI is just a bunch of functions - which is not the case in an object oriented environment like GNOME. if we have two ABIs, where does subclassing a GtkWidget live? the internal ABI, that we can break, or the external one? we cannot really change the size of structures in an ABI-compatible way, so it'll have to be in the internal ABI, for core libraries and apps; though that means that external libraries and applications won't be able to create their own third party widgets unless they use the internal ABI - which is utterly broken, because applications need to define their own widgets, as GTK+ cannot possibly ship all possible widgets needed by all apps for all eternity, and do that even before the apps have been written; and in some cases third party apps are closed source, so the authors cannot put their widgets inside an open source toolkit. the whole "non derivative work" status of some kernel modules is still pretty fuzzy, I don't think people want that fuzziness to extend to user space as well. plus, GNOME is not just C. we do have another ABI in the form of the introspected data that is used to generate bindings for other languages, like Python, JavaScript, and Perl. that is a third ABI that would have to be split in two, and it would require jumping through various hoops, some of which are on fire. it's always easy to assume that people outside of your restricted area of interest are assholes that do not understand the problem space, especially if you spent about 15 minutes to think about it. I can assure you: we people working on the GNOME core platform discuss this shit all the time. there is simply no good solution, or good enough solution yet. after reading the comments in Miguel's blog post I have to say, though, that it's always cute how kernel developers think they can solve the user space problems using their limited perspective, like they are the second coming of Jesus Christ; it does take its toll, though, given that we have to explain all this basic software engineering and release management stuff all the time. you'd think they'd get it.
Quotes of the week Posted Aug 31, 2012 13:55 UTC (Fri) by khim (subscriber, #9252) [Link] I'm talking about ABI and conformance for third party software, not performance. What's your point? If that's a simple whine that creation of stable ABI is hard then I agree, it is hard. It's also necessary. Do we compile stable and unstable symbols in the same shared object? The same, of course. What's the point of splitting them? That works, but it requires re-linking the entire planet because you really want to bump the SONAME so that applications using the internal ABI can actually be updated. Absolutely not. ELF Symbol Versioning was proposed as a solution for this problem decade ago. Other solutions are possible, too (it's hard to use C++ with ELF Symbol Versioning, but there are it's own solution). You provide library with both internal and external symbols on the live OS and you provide shims with just public symbols for the external developers. This is how Windows does that, this is how MacOS does that, this is how LSB does that if you want an example closer to the Linux! if you bump the SONAME, though, applications linking against the external ABI will also need to be relinked, otherwise they won't be able to find the shared object - look at all the apps linking against boost, and breaking after 12 months because nobody ships the old version of boost they use any more. Well, the problem is obviously not boost. The problem is the stupid idea to include it in a "system code" without accepting the support burden. If you include library like boost in a system core then yes, you need to ship bazillion versions of the same library in a said code. It's a fee you pay for compatibility. Sane libraries (such as GTK+ or GLibC) bump versions very rarely which reduces said price. so at this point third party apps are in the same situation as they are now, even if they use a supposedly stable ABI. That's not a stable ABI. That's a perverse joke. Stable ABI means one simple thing: I can take binary compiled 10 years (or 5 years if 10 years is too much for you), start it - and it'll work. This is an ideal, 100% stable ABI is not really possible (even in Windows some programs stop working). But this is what you should strive for. Is it hard? Oh yes, there are countless stories about silly application developers and insane solution Microsoft comes with (for example programs called it's always easy to assume that people outside of your restricted area of interest are assholes that do not understand the problem space, especially if you spent about 15 minutes to think about it. I can assure you: we people working on the GNOME core platform discuss this shit all the time. there is simply no good solution, or good enough solution yet. Are you sure you are "discuss this shit" with a goal to find a solution and not with a goal to find an excuse? If I understand correctly you are discussing that "unsolvable problem" for a decade while other people come and solve it and then do it again and again. KDE was a good example till they decided to drop everything and create a new incompatible version. What was fine for the library (developers can always choose to use one version or another) is not fine for an OS: it must support old interfaces for a long time (years) till most applications (even obscure ones) will catch up to the new ABI. My favorite example is FCB: introduced in CP/M back in 1973, ported to MS-DOS in 1981, obsoleted in 1983 yet supported till Windows 2000. And even then people complained about their beloved WordStart! And it's not impossible to provide stable ABI even if you do radical redesign. Think MacOS: it provided stable ABI in all transitions. From m68k to PPC to x86-64, from MacOS classic to MacOS X. Every time you had the ability to run old programs. Yes, Apple is not Windows, it drops support for older ABIs regularly. But it only does that when they have persuaded most ISVs (at least the major ones) to switch to a new ABI and when people have mostly upgraded their applications to a new version. When other companies do the same idiotic radical ABI breakage GNOME and other Linux desktop are doing they suffer the same fate: witness Windows Phone 7.
Quotes of the week Posted Aug 31, 2012 19:13 UTC (Fri) by alankila (subscriber, #47141) [Link]
> You provide library with both internal and external symbols on the live OS and you provide shims with just public symbols for the external developers. This is how Windows does that, this is how MacOS does that, this is how LSB does that if you want an example closer to the Linux!
It's also how Android does it.
Another quote of the week Posted Aug 30, 2012 14:34 UTC (Thu) by pboddie (subscriber, #50784) [Link]
From the article, De Icaza talks about change and incompatibility:
"This killed the ecosystem for third party developers trying to target Linux on the desktop."
This intersects with assertions that monolithic distribution package management doesn't help people develop and deploy applications for Linux.
But first, the finger of blame: I used to read the blogs of the most prominent GNOME developers and it seemed like they were obsessed with "ISVs" and permissive licensing to bring them on board. Many of those people worked for major GNU/Linux distribution companies. Did this apparent lack of cross-distribution portability not obviously contradict their goals? Did they not try and push forward standards initiatives to remedy the situation? Couldn't they have tried to make tools available to help people target their employer's distribution if nothing else worked?
I would argue that a major pain for developers has been the way the big two desktop environments pushed dynamic languages away from the core and made it difficult to extend the functionality in a stable fashion in those languages (particularly with KDE 3 despite heroic efforts by various non-core contributors), thus eliminating any portability advantages those languages have. Belated efforts to embrace JavaScript - truly the MS-DOS of the dynamic language world - as an "extension" or "plugin" language instead of providing a robust foundation for dynamic language usage have done a lot more than a bunch of other things to show just how much those projects were interested in "third party developers".
Another quote of the week Posted Aug 31, 2012 1:56 UTC (Fri) by khim (subscriber, #9252) [Link] Did they not try and push forward standards initiatives to remedy the situation? Standards are only good if people use them. What standards you are talking about? LSB? For a desktop development??? It still does not even includes a way to play a sound. In 2012! While Windows and Mac supported video and audio twenty years ago. Couldn't they have tried to make tools available to help people target their employer's distribution if nothing else worked? They tried to do that, but it didn't work. The urge to create "perfect" API was irresistible. I would argue that a major pain for developers has been the way the big two desktop environments pushed dynamic languages away from the core and made it difficult to extend the functionality in a stable fashion in those languages (particularly with KDE 3 despite heroic efforts by various non-core contributors), thus eliminating any portability advantages those languages have. Somehow the same problem was never a big deal for MacOS and Windows. No, the real problem was that for years the only solution offered for the "deployment problem" was to expand distributions further and further. It took more than decade to even admit that binary distribution of stuff not included in the official repository is even desirable. Somehow all these talks about ISVs and permissive licenses were always concentrated on the way to create the package. And indeed in this regard Linux is often better than MacOS or Windows. But the fact that there are no way to reliably deliver said package to the end user made the whole exercise pointless. Server-side packages usually sidestep the issue because they don't really to automatically integrate with the rest of the system. And when they need such ability trained sysadmin is always there. But Joe Average does not know how to track programs dependencies and fix them. And he does not want to know. For him (or her) program which does not work because there are no libcrypto.so.0.9.8 is more-or-less indistinguishable from the one which does not work because some subsystem is just removed altogether. Somehow it took years till these people finally admitted that "recompile the world" is wrong answer to the API compatibility problem. Take a look on this discussion. The self-righteousness, the arrogance, the loftiness… People explained to the Linux community what's wrong with Linux years and years ago, but it was adamant: the diverse API model actually helps Linux--either the vendors will have to work extra-hard to get their apps working reliably in Linux, or they may (and sometimes do) get lazy and just release the specs to the Open Source community. Is it any wonder that ISVs and users did what they did? To abandon this nightmare and to embrace sane platforms (such as MacOS, Windows and now iOS with Android) looked like the only choice. Well, better late than never. It took a decade to admit the problem. We'll see if the fabled community will manage to invent a solution before it'll extinct.
Another quote of the week Posted Sep 7, 2012 9:21 UTC (Fri) by ovitters (subscriber, #27950) [Link]
Miguel is not involved with GNOME. He's only on Planet GNOME as a user and because he started it. We basically don't remove blogs.
It seems you're not aware of gobject introspection in your "instead of providing a robust foundation for dynamic language usage" bit.
Quotes of the week Posted Aug 31, 2012 10:21 UTC (Fri) by Uraeus (subscriber, #33755) [Link]
Miguels article and quote doesn't add up for me. Yes, better ABI stability is definitely a boon for 3rd party developers, but it wasn't like GNOME for instance didn't try to keep a stable ABI during the last 10 years of GNOME 2.x existence. And it is not like the APIs of Windows and Mac has been 100% still in those 10 years either. I am not saying that things could not have been better, but I think it is way to simplistic to primarily blame ABI stability for the lack of Linux desktop adoption.
Quotes of the week Posted Sep 1, 2012 15:50 UTC (Sat) by khim (subscriber, #9252) [Link] Yes, better ABI stability is definitely a boon for 3rd party developers, but it wasn't like GNOME for instance didn't try to keep a stable ABI during the last 10 years of GNOME 2.x existence. No. They tried to keep stable AP and minimize API breakage. It's was quite easy to recompile program with newer version of GNOME and the infamous "./configure; make; make install" incantation often worked well. But they never even considered the use-case where you build one binary and use it for years. This was fine approach 15 years when GNOME 1.x was planned but they kept this approach when GNOME 2.x was introduced, too. Sure, it was mostly distribution's fault (they usually refused to ship obsolete versions of libraries because "nothing in our distribution uses this old crap anymore" - let's remove it), but we are talking about the end result here. And when GNOME 3.x was introduced they did nothing to support programs designed to work with older versions of GNOME: some components are missing altogether, some are incompatible. You can drop some libraries from old version of GNOME and start most programs, but they often work incorrectly and GNOME developers don't consider this a problem. And it is not like the APIs of Windows and Mac has been 100% still in those 10 years either. Of course not! But they don't consider "just recompile your program" a valid answer to the problem. Sometimes APIs are removed without warning and you need to bring old, obsolete libraries from previous versions of OS to the latest and greatest (think DirectMusic), but these are notable exactly because they are rare. With Linux desktop and GNOME that's rule, not an exception. In effect they used wrong metric to measure success: instead of how many programs works out-of-the-box without additional efforts they used metric how many jumps through hoops you need to run old version of program. This is all about perception. If you use 10 programs, upgrade your OS and 9 of them work while one does not work (or require extensive tweaking to start it) then obviously it's fault of said program (because vast majority of them work, right?). If you use 10 programs, upgrade your OS and 5 or 6 don't work correctly after upgrade then it's obviously an OS developer's fault (even if the fix is simple and trivially googleable). Is it fair? No. But that's life. Deal with it.
Quotes of the week Posted Sep 7, 2012 9:29 UTC (Fri) by ovitters (subscriber, #27950) [Link] But they never even considered the use-case where you build one binary and use it for years. This was fine approach 15 years when GNOME 1.x was planned but they kept this approach when GNOME 2.x was introduced, too. Sure, it was mostly distribution's fault (they usually refused to ship obsolete versions of libraries because "nothing in our distribution uses this old crap anymore" - let's remove it), but we are talking about the end result here. I don't really get this bit. the ABI/API was the same for all those years. The distributions caused the breakage. For many years people have been trying to solve the binary bit as well. Meaning 'GNOME OS'. It has been around for way longer than you might assume if you just follow news sites. It seems you attribute 'no solution' as 'they never even considered'. GNOME also participated on LSB. And IIRC various libraries were added to that. So in short I disagree with the 'never even considered' as well as judging solely based on result.
|
|||||||||||||