LWN.net Logo

GNOME Platform Stormclouds

March 24, 2004

This article was contributed by Tom Dunstan

Some tensions that have been building in the GNOME community for a while came out this week. Havoc Pennington wrote about the language and platform options available to open source development generally, and GNOME in particular. This sparked a large debate on PlanetGnome, among other places.

Currently the GNOME platform, meaning primarily the libraries available to GNOME developers, is written in C. There are a number of historical reasons for this: C was considered to be the most portable language around, and it allowed for very easy bindings to be written for other languages. The only real alternative when GNOME was formed was C++, which, at the time, scored lower on both of the previous measures. Also, some members of the community generally didn't like C++ as a language.

Times have changed. The Mono project was launched in August 2001 after the folks at Ximian had decided that, having gone through the pain of developing a large, multi-component application, enough was enough:

There is a point in your life when you realize that you have written enough destructors, and have spent enough time tracking down a memory leak, and you have spend enough time tracking down memory corruption, and you have spent enough time using low-level insecure functions, and you have implemented way too many linked lists.

The obvious candidate for an object oriented, statically typed, garbage collected language is Java. Unfortunately, the most widely used, certified Java Virtual Machines (JVMs) are not free software. Sun Microsystems still maintains tight control over them, and even limits the ability to freely redistribute the free-as-in-beer JVM, even without modification. Currently, to the author's knowledge, the only Linux distributions that ship with either the Sun or IBM JVMs are a few commercial enterprise distributions. While there are some free software JVMs available, their performance is generally well below that offered by the proprietary ones.

You may be wondering why languages such as Python, Perl or Ruby aren't in consideration here. Havoc's paper doesn't do much to explain why languages such as these won't be used for writing system components, because the audience that he wrote it for probably take that as a given. Dave Camp's one-liner in this blog post probably explains the thoughs of most developers on this issue: I have a soft spot in my heart for Python (although I have a softer spot for static type checking).

In short:

  • The platforms currently associated with these languages are specific to the language, the sharing of components would be difficult. Parrot, at some point in the future, may unify these languages.
  • These languages are not statically type checked by a compiler. To developers who have been writing primarily in C, this seems dangerously like leaving certain types of error to happen at runtime, to be discovered by users. Automated testing can address some of these issues, but a certain type of C programmer might have trouble sleeping at night after shipping a large Python application, with the expectation of receiving bug reports containing NameErrors.
  • At least partially due to the dynamic nature of these languages, there are limits to some of the optimizations that can be done, either at compile time or by a Just-In-Time compiler at runtime, as used by the Java and .NET platforms. Thus performance, particularly in platform libraries which need to be fast, is a concern.

Ximian saw the standardized C# language, and the standardized Common Language Infrastructure (CLI) from Microsoft as the way out: they could write a GNOME platform with all to goodies of an object oriented, statically typed, garbage collected language using and extending the existing GNOME APIs. Since the language was standardized, there were seemingly no trademark issues, and the parts of the platform that were Windows specific would be replaced with GNOME components anyway.

So Ximian founded the Mono project, and things have continued with the GNOME platform itself continuing to be coded in C, with the Mono team progressing its C# compiler, CLI runtime and class libraries at a tremendous rate.

Two things have happened recently to make some in the community start to think about the strategic direction that GNOME should take. The first is that Mono is actually approaching a 1.0 release sometime this year. The second is that the Java compiler and class library that are part of the GNU Compiler Collection (GCC), GCJ, is now considered to be fairly mature.

GCJ treats Java essentially as a subset of C++ with garbage collection and a large class library. Java source code is compiled to native code and then linked against a modified Boehm garbage collection library. Most of the standard class libraries have been implemented, with the notable exception being the windowing packages AWT and SWING. The GCJ was able to natively compile Eclipse last year, and it is considered mature enough for Red Hat to ship a GCJ compiled Tomcat Java Servlet Container in their Application Server enterprise distribution.

All of these environments will work quite well together, until GNOME makes a decision to start writing platform libraries in a managed language, for a managed platform. Whichever platform is not chosen might have a much more difficult time integrating such components. Many feel that GNOME needs to make some decision about which platform to support officially, otherwise the community will continue to be tied to C for the platform libraries indefinitely, or worse, risk forking.

There are a number of reasons given (by Havoc and others) as to why some in the GNOME community are unhappy with using Mono as the platform of choice, including:

  • Patent concerns over parts of .NET
  • General distrust of Microsoft, or strategic reluctance to adopt a Microsoft technology
  • Not wanting to play "catch-up", since Mono will always be behind Microsoft's implementation
  • Not wanting to upset IBM or particularly Sun, who have contributed a lot to GNOME over the past few years
These complaints don't necessarily impress Mono advocates, however. Software patents, some say, may well be inside Java too, either by some third party, like in the Eolas case, or by Sun itself. Sun, it is claimed, has never released a general royalty free patent license in relation to its Java technologies.

While Havoc argued that using a C#/CLR combo might "speed up" the adoption of Microsoft's technology, proponents of Mono point out that it allows a migration path away from Windows in the future, something that won't be there if the Windows world is using CLR technologies, and the free software world has no CLR.

The catching-up-with-Microsoft argument is persuasive, but Java hardly presents a better alternative. Java, under the tight control of Sun, has moved much slower than the corresponding pace of many free software projects such as GNOME, Python, or indeed, Mono. The language has evolved very slowly, with features that developers have wanted for years, such as enums and generics (templates for C++ people), only just now being added to the language, after they came out as part of C# from Microsoft, a much younger piece of software.

Sun's reluctance to support a native widget based GUI toolkit, sticking with the sometimes sluggish, alien looking SWING, left developers with no real option for developing desktop applications. There's a reason why Java is used heavily on the server, with a devoted following, but very sparingly on the desktop. To use a contrasting example, Python has evolved as a language at a much faster rate, with incremental rather than groundbreaking speed improvements over time as well. The Benevolent Dictator model is used in Python as it is in Linux, with great effect.

Keeping Sun happy seems like a very valuable goal for some in the community in light of the major contributions that Sun has made to GNOME. It's ironic, however, that the community seemingly is in this position at least partly because of Sun's refusal to let go of Java, and more ironic that one proposed solution to keep Sun happy is to use a non-Sun-licensed, non-Sun-certified Java platform which doesn't support Sun's preferred GUI APIs. While it might keep Sun happier than if GNOME adopted Mono, it would seem unlikely that that's where they'd like to be. Even if Sun were to make an open source version of their JVM and class libraries available, as advocated by IBM and others, the interaction between two competing platforms may still be awkward. It would be a great step, in any case.

Meanwhile, discussions continue, and there have been some constructive suggestions as to how to interoperate and leave the community's options as open as possible. The debate is currently civil, intelligent and constructive. There are a number of parties with big stakes involved, however, so things may yet get heated. A fragmented platform helps nobody, so there's a big incentive to work out the most inclusive solution constructively. It's a debate which will be watched closely by many.


(Log in to post comments)

GNOME Platform Stormclouds

Posted Mar 25, 2004 3:32 UTC (Thu) by kyle_hayes (guest, #7904) [Link]

I think the title may be blowing this out of proportion a bit. I am
somewhat reminded of how there were many reports of the GNOME/KDE "war",
when the developers seemed to get along fine. It was the partisans of the
two projects that were at "war", not the people doing the coding!



GNOME Platform Stormclouds

Posted Mar 25, 2004 6:01 UTC (Thu) by raytd (guest, #4823) [Link]

I tend to agree. F/OSS developers will code in/on whatever language/platform suits them, not because a particular language/platform has been "blessed".

IMHO Havoc is more concerned about the immediate threat posed by the enemies of the F/OSS community rather than what language/platform the developers happen to be coding in/on. (yeah, yeah. x/y is getting on my nerves too, but I've had a couple and a very long day. sue me.)

Parrot - the way

Posted Mar 25, 2004 18:39 UTC (Thu) by johnjones (guest, #5462) [Link]

ok so

sun does not want to open its stack machine
Microsoft CLI is closed and only part of it submited

python and perl run VERY BADLY on both of these

parrot is a register based Virtual Machine that...

o In My Humble Opinion will run Perl Python Very well

o java compiler could target

o C# compiler could target

o portable

this is a no brainer if you thinking about architecture...

regards

John Jones

p.s. people who want to write things now should do so in python C or java
(mono is still under heavy dev and they just have what amounts to alot of demo programs and people who think they can get win32 coders into the gnome fold... really it would be nice if QT C++ AND GTK C could target parrot...)

Parrot - the way

Posted Mar 26, 2004 0:28 UTC (Fri) by hinoue (guest, #6949) [Link]

The operative word is "WILL". Parrot is very far from being finished and has yet to prove that it is going to replace the current Perl/Python/Ruby VMs. However, you are right that the fight over languages and libraries is a bit silly -- the fight should be over the VM.

The VM ultimately chosen should be decent target for any language. This precludes using a Java VM, IMO. CLR allows for a larger set of languages, but still isn't infinitely flexible. It has the advantage that the runtime is probably unencumbered by IP issues. The .NET libraries are a different story, but presumably people writing applications for GNOME will using GNOME bindings.

However, what is actually needed is really a subset of a VM - a portable intermediate representation along with a well-defined runtime model. This is not an new idea -- I recall that GCC people tried something similar a long time ago. It is an idea that is gaining ground among VM people today, though. VMs on modern systems are no longer seen an interpreters -- good VMs compile code, and the instruction sets of Java, CLR, and even Parrot aren't really great representations to do the kind of analysis for optimization that you would want. (Incidentally, there's only a tiny advantage in using a register based instruction set over a stack one. You've got to put the values in real registers at some point). Redefining the problem to this allows one to dismiss language support and library issues. It would also allow greater amount of modularity -- compilers would then be naturally split between the front end, which target the IR, and the backends, which target the native machines.

Unfortunately, I know of no suitable portable IR, so we are stuck with things that already exist -- Java or Mono.

GNOME Platform Stormclouds

Posted Mar 25, 2004 3:53 UTC (Thu) by jmalcolm (guest, #8876) [Link]

I am surprised that I have not heard more about the possibility of running IKVM/Classpath on top of Mono. It looks to me like this would allow the programer to choose either Java or C# while interacting with a common managed platform.

I realize that this still means using the CLR as a base but it seems to be more of a middle ground than having to choose or reject Java wholesale. It also seems like this would offer the largest possible library set.

Does anybody that better understands the issues have a comment?

GNOME Platform Stormclouds

Posted Mar 25, 2004 13:57 UTC (Thu) by cloose (subscriber, #5066) [Link]

You might want to take a look at Miguel de Icaza's blog
(http://primates.ximian.com/~miguel/activity-log.php). Especially at the
entry from March 18th.

Slackware has a JVM

Posted Mar 25, 2004 7:38 UTC (Thu) by remijnj (guest, #5838) [Link]

Currently, to the author's knowledge, the only Linux distributions that ship with either the Sun or IBM JVMs are a few commercial enterprise distributions.

My favorite distribution, Slackware, comes with a JVM installed by default. This is what i see in Slackware 9.1.
$ java -version
java version "1.4.2_03"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_03-b02)
Java HotSpot(TM) Client VM (build 1.4.2_03-b02, mixed mode)

SuSE has JVM, too

Posted Mar 25, 2004 13:47 UTC (Thu) by zorgan (guest, #4016) [Link]

and has always had it since quite a while ago. To me, the idea of a desktop
distribution without java support seems pretty strange. So many small OSS
application that a) want to have a GUI and b) want to be cross-platform decide to use
java.

Commercial SuSE?

Posted Mar 25, 2004 14:24 UTC (Thu) by tomd (subscriber, #881) [Link]

Yeah, the article probably should've just said commercial distributions, and left "enterprise" out. I'm running SuSE 9.0 installed from FTP, and no IBM or Sun JVM here that I can see (and I snarfed the entire free 9.0 mirror). One of the first things I did post-install was download and install one. Would I be right in assuming that you bought a boxed set?

Commercial SuSE?

Posted Mar 27, 2004 12:54 UTC (Sat) by rise (guest, #5045) [Link]

Retail boxed SuSE versions have been indeed been shipping with both Sun and IBM Java JREs/SDKs.  The FTP version could be missing the packages for a number of reasons, size being one non-license possiblity that springs to mind.

Slackware has a JVM

Posted Mar 25, 2004 14:37 UTC (Thu) by tomd (subscriber, #881) [Link]

This is interesting. I understood that there were licensing issues redistributing the Sun JDK, ie you can't except with explicit approval from Sun. That's certainly what the blackdown FAQ says:
http://www.blackdown.org/java-linux/docs/support/faq-release/FAQ-java-linux-8.html#licensing

While vendors can certainly ask Sun for permission, since I hadn't seen any free (meaning mirrored, in this case) distros carrying it, I believed that Sun wouldn't let unrestrained distribution take place in that way. Perhaps they decided that Slackware distribution included mirrors, but people couldn't redistribute the JDK separately? Hmm.

Slackware has a JVM

Posted Mar 25, 2004 15:56 UTC (Thu) by busterb (subscriber, #560) [Link]

Slackware carries a pre-packaged version of Sun Java on its mirrors, and has done so for several years. This comes from the Changelog for Slackware 8.0:

Mon Apr 2 15:30:09 PDT 2001
contrib/java/: Added Java(TM) 2 SDK, Standard Edition,
Version 1.3.0_02. Thanks to Sun Microsystems for
allowing us to include this with Slackware.

Prior to that, Slackware distributed the Blackdown JDK. I'm not sure what Patrick Volkerding did to get distribution rights, but there apparently has not been any issue getting the Sun version of Java included in the freely-downloadble version and even repackaged as a Slackware tarball.

Slackware has a JVM

Posted Mar 25, 2004 17:31 UTC (Thu) by remijnj (guest, #5838) [Link]

Yes, i see now. Thank god for Slackware and Patrick Volkerding, a bit of sanity in the world of linux distros.

Just to clarify though, the jvm is now in the normal install (not confined to contrib anymore). It resides in the "d" (development) set.

GNOME Platform Stormclouds

Posted Mar 25, 2004 10:14 UTC (Thu) by dale77 (guest, #1490) [Link]


One small point - C# has enums (yay! named ints!) but it doesn't have generics. This java feature does sound useful from a programming perspective, and Sun has implemented these in such as way as to run on an unchanged VM. Now, I don't think Microsoft would care about breaking the old VM, but I reckon end users and application developers do. The JVM is bound to be much more stable than .NET. Really we don't need wonderful new platform features every six months (which Microsoft only provides by the copy and paste method anyway) what we need is a stable platform to build wonderful new applications on!

Dale

Yes it does.

Posted Mar 25, 2004 19:26 UTC (Thu) by hummassa (subscriber, #307) [Link]

C# has generics. Uninformative.

Not today it doesn't

Posted Apr 1, 2004 9:00 UTC (Thu) by dale77 (guest, #1490) [Link]


When I said C# doesn't have generics I meant the C# you use today doesn't have it.

Today's C# doesn't have generics, so it seems a little off-beam to say that java is following C# in generics, when C# 2.0 isn't even released yet.

http://www.c-sharpcorner.com/Code/2004/March/NewCSharpFeaturesP1.asp

GNOME Platform Stormclouds

Posted Mar 25, 2004 12:51 UTC (Thu) by dcoutts (guest, #5387) [Link]

An incremental approach would be to include language bindings into the GNOME development platform (this is beginning to happen) and then bless certian languages/platforms as ok to develop GNOME *applications* that can then be included in the official GNOME desktop. The point being that the GNOME libs would still have to be written in C to allow easy bindings for other languages.

That introduces the issue of only one-way reusability; apps in any language can use the base libs but an app in one language cannot reuse an app/component written in another language. For that we use a bonobo/DCOP/DBUS style component protocol.

That still leaves the problem that reusable libraries would not be able to be written in a high level language; that is libs with a more fine grained interface than high level 'components' (which only really work well if they have coarse interfaces). However that would probably be an acceptable tradeoff for the next few years, since the pressure for high level languages is really due to the pain of writing complex *apps* in C.

Besides, integrating different languages at a finegrained level is still an unsolved problem, the CLR doesn't even do it except for C#-like languages due to the semantic mismatch between different languages. It'll be interesting to see if parrot can do it ok for Python/Perl/Ruby.

OCaml not mentioned then?

Posted Mar 25, 2004 14:41 UTC (Thu) by rwmj (subscriber, #5474) [Link]

I find it interesting that by developing Mono the free software community is actively and quickly embracing and extending a Microsoft standard. Way to go!

Nevertheless, my own personal choice for a language would be Objective CAML. It's as fast as C, and uses type inference, so you get static typing without all the hassle of having to defining every variable's type (as in Java, C, etc. ...). And of course it has garbage collection, LISP macros, data matching and all the features you'd want in a modern language (and won't find in Java or C#).

Rich.

GNOME Platform Stormclouds

Posted Mar 25, 2004 16:20 UTC (Thu) by jeremiah (subscriber, #1221) [Link]

I think we'll see sun open Java a little more the the near future, but I can understand where they are comming from. It's like letting your daughter go on her first date with a complete stranger who's a few years older. It will probably workout okay, but....

It's got to be nerve racking for them.

GNOME Platform Stormclouds

Posted Mar 25, 2004 16:38 UTC (Thu) by mly (guest, #2171) [Link]

I'm not involved in GNOME development, and I certainly feel that those who are must solve this in a way that they are happy with. I'm sure there are a number of Python programmers involved in GNOME who understand much more about the pros and cons of using Python in that particular context better than I do, maybe it's obvious that it can't be used, but I'm really surprised to find the lack of static typing to be an argument against Python. Sure, I misspell things in Python and get name errors, but they typically surface quicker than they would in C, and I mistype less in Python since I type less in Python.

It obviously depends on what kind of code I'm writing, and in what stage of development I'm in, but I think that I generally have an edit - test cycle in Python which is much shorter than my edit - compile cycle in C, and much, much shorter than the edit - compile - link - test cycle in C. The reason for this is that you typically have to write five or ten times more in C than in Python to achieve the same result. Testing as often in C as I do in Python would be impossible, particularly in a large system: I've been involved in C++ projects where an application took an hour to build... In Python, this is not an issue at all.

In general I think Guido van Rossum has been very successful in creating a language which is helpful to the programmer in a positive way, making it easy to do things right, rather than trying to make it hard to do bad things. In C, you must often do potentially dangerous things such as playing with pointers and type casting, because the safer way of working is too limited. There are also a number of well known gotchas in C syntax that have been solved in Python, e.g. "if x = 0" is a SyntaxError in Python.

You may be wondering why languages such as Python, Perl or Ruby aren't in consideration here. Havoc's paper doesn't do much to explain why languages such as these won't be used for writing system components, because the audience that he wrote it for probably take that as a given.

Might this possibly be a mistake?

Dave Camp's one-liner in this blog post probably explains the thoughs of most developers on this issue: I have a soft spot in my heart for Python (although I have a softer spot for static type checking).

I think this is mainly a matter of getting used to another way of programming. This might be a hurdle in itself, but it's maybe more a learning issue than a technical issue. Besides, there are tools such as PyChecker and PyLint that can take the role of the compiler. It's my impression after many years of using both C and Python that Python programs typically contain much fewer bugs than C programs, and that it's far easier to find and correct them.

Automated testing can address some of these issues, but a certain type of C programmer might have trouble sleeping at night after shipping a large Python application, with the expectation of receiving bug reports containing NameErrors.

I think most programmers sleep even worse when they have shipped C applications, with the expectation of receiving bug reports containing core dumps--and maybe even worse: security holes due to buffer overflows etc. There aren't a lot of reliable studies on the subject, but I've never seen any study suggesting that C programs contain less bugs than Python programs. I've seen some study that suggest the opposite. The problem with Python is not that Python programs are so buggy. Not at all.

Python certainly doesn't fit in all situations, but for a system like GNOME, I think the reliance on an external interpreter and the inability to build native machine code executables seem like small problems. Making source code or easilly decompiled byte code available to end users doesn't seem to be a problem in GNOME either. Considering the wide use of Python in the Linux community, and how easy it is to learn, I don't think that lack of skilled python programmers or lack of "support" are issues either.

The only non-religious problem I see with adapting something like Python for GNOME is performance. If Moore's law and novelties such as pyrex and psyco doesn't fix this, Python has always been able to cooperate nicely with C++ and C... That should also be helpful in a transition period. But I guess it's still possible that performance issues in Python would be a real problem for GNOME. I appreciate that performance is a big issue in GNOME.

Of course, if we'd switch from C to Python to get rid of C's problems with memory management etc, and end up writing parts in C anyway for performance reasons, you might feel that little is gained, but typically, the bottlenecks in programs are small. Instead of 10000 lines of C code for a particular program, you might end up with 2000 lines of Python and 500 lines of C. It's much easier to maintain 2500 lines than 10000, and all the risks of C, with memory leaks etc are roughly reduced by 95% if there is 95% less C code.

Perhaps a two language solution with Python and C is the easiest way to go forward from where GNOME stands today?

Maybe Java has improved a lot since I last looked at it, but it certainly used to be a major resource hog, and Java applications that I run regularly in Windows, such as IBM's DB2 ControlCenter are so slow and memory hungry that they are almost useless.

My experience is that Python programs are significantly shorter than Java programs, and much shorter than C programs, and at the same time clearer and easier to understand. A large part of the code that I would write myself in C, is already written by someone smarter in well tested C if I use Python. In a way, writing Python can be seen as a very clever way of reusing well written and tested C code. (After all, the classic Python implementation was started almost 15 years ago, and it's certainly not been allowed to rot.)

Somehow...I have problems envisioning something that Microsoft came up with as a solution for GNOME... I'm really ignorant on the issue, but I somehow suspect that C# for Linux might backfire in a nasty way. I'd just wait for some Microsoft C# patent to emerge as GNOME had turned into a real Windows contender. Of all aspects of Linux, it's the rise of a more polished desktop "experience" that is most likely to have an impact on Windows users. I'm sure MS would do everything in their power to stop Windows from being replaced by GNOME/Linux as the leading desktop OS, and Bill Gates have explained publically that he'll use the legal system to go after Linux if he can, and he's doing it right now by funding Caldera/SCO. Don't think it'll stop there. No, I don't think Mono will make the programmer sleep any better. :(

If you really think you need static typing and memory management, you should perhaps look at Eiffel or OCaml? Although I guess OCaml's inferred typing won't stop you from mistyping a variable name...

GNOME Platform Stormclouds

Posted Mar 25, 2004 17:21 UTC (Thu) by tjc (subscriber, #137) [Link]

I'm really surprised to find the lack of static typing to be an argument against Python. Sure, I misspell things in Python and get name errors, but they typically surface quicker than they would in C [snip]

They show up at compile time in C, which is as fast as it gets!

Dave Camp's one-liner in this blog post probably explains the thoughs of most developers on this issue: I have a soft spot in my heart for Python (although I have a softer spot for static type checking).

I think this is mainly a matter of getting used to another way of programming.

I never have gotten used to it. I've written a fair amount of code in both PHP and Tcl, and the lack of static type checking is a language "feature" that keeps on taking. Since my spelling is unlikely to improve significantly, I appreciate a language that points out my mistakes immediately rather than producing unexpected results at runtime.

GNOME Platform Stormclouds

Posted Mar 25, 2004 18:44 UTC (Thu) by mly (guest, #2171) [Link]

tjc wrote:
> They show up at compile time in C, which is as fast as it gets!

No it's not! Honestly! For my normal programming, I run my python programs more often than I compile my C or C++ programs during development. I typically write very modular code and change it in small steps, testing it often so that I notice whatever bugs I introduce as soon as I can. Of course, only a fraction of these bugs would have been caught by the compiler anyway.

I haven't programmed so much C or C++ lately, but even if I try to use short cycles there as well, they will still be longer, since I need to type so much more in these languages to produce the same amount of functionality.

But if you like the support of the compiler in C, just use PyChecker as your Python "compiler". I would still suggest proper unit tests though. Python has good unit test support in the standard library, and no compiler or code checker can replace real tests.

> I've written a fair amount of code in both PHP and Tcl

I don't think you can draw too many conclusions about Python from Tcl or PHP. That's a bit like deciding what to think about C based on experience from COBOL.

Strange as it may sound, I mistype much more in C or C++ than in Python. There are several reasons for that. One is naturally that there will be fewer mistypings if there is less typing. Another aspect is that Python's syntax is very simple and consistent. I don't have to remember so much, and
since the code typically looks clean and regular, I'm more likely to directly spot mistakes. Python fits my brain. My little experience of Tcl and PHP suggests that they don't offer the same benefits. I'll quote what Eric Raymond said about his first experiments with Python in Linux Journal:

I noticed (allowing for pauses needed to look up new features in Programming Python) I was generating working code nearly as fast as I could type. When I realized this, I was quite startled. ... it generally means you've achieved mastery of the language. But that didn't make sense, because it was still day one ...

This was my first clue that, in Python, I was actually dealing with an exceptionally good design. Most languages have so much friction and awkwardness built into their design that you learn most of their feature set long before your misstep rate drops anywhere near zero. Python was the first general-purpose language I'd ever used that reversed this process.

Pinched from http://www.linuxjournal.com/article.php?sid=3882

GNOME Platform Stormclouds

Posted Mar 25, 2004 20:25 UTC (Thu) by tjc (subscriber, #137) [Link]

I don't think you can draw too many conclusions about Python from Tcl or PHP.

All three lack static type checking, which is the *only* thing I was comparing with C. I'd probably have more to say about Python, but I've never had a contract that required it, and C is the only thing I use on my own time.

GNOME Platform Stormclouds

Posted Mar 26, 2004 18:08 UTC (Fri) by maney (subscriber, #12630) [Link]

All three lack static type checking, which is the *only* thing I was comparing with C.

Right, and that's why it's a useless comparison. There's a great deal more that differs among the four languages, and assuming that your experience with PHP and TCL are reliable guides to what Python is like is a bad assumption. I haven't used TCL, but I have worked with PHP at some length - indeed, I worked with it until it seemed to be unable to support the application, which was why I switched to Python. Nor was that the biased reaction of a fan of X who proves that Y isn't much good and finds reverting to X clearly better, as I had had nothing but superficial exposure to either language before beginning. And so I made the mistake of taking at face value all the hype that was going around then (still is, I guess) about how PHP is such a wonderful language for web apps with database backends. Maybe for Joe Hacker's Own CD Catalog it actually is; for the middling complex scheduling app I was working on it made some early, stripped down versions easy, but the cost rose quickly as I tried to add meat to those bones.

Rather more that 20 years after my first chance to use C (anyone else remember BDS C for CP/M?), I find that there are languages that work differently than C (or C++, which in some important ways is just a false front hung in front of C to disguise it), and for these languages the static type checking that is so essential in C - because the static checking, limited as it is, is the only sanity check you get - is simply irrelevant. I know I won't convince you: this is something that everyone has to learn for themselves. At least that has been my experience.

GNOME Platform Stormclouds

Posted Mar 26, 2004 22:57 UTC (Fri) by tjc (subscriber, #137) [Link]

All three lack static type checking, which is the *only* thing I was comparing with C.

Right, and that's why it's a useless comparison.

It was a useful comparison for my purposes, since I was able to determine from an early stage that quite of few languages would not be very useful to me. This is a lot better than spending a great deal of time learning a new language and then finding out further down the road that it's not going to work very well for a particular purpose or project.

GNOME Platform Stormclouds

Posted Mar 25, 2004 18:52 UTC (Thu) by angdraug (subscriber, #7487) [Link]

Unlike Ruby and Python, PHP and Tcl are not mentioned in the context of "making it easy to do things right", and there is a reason to that: they don't.

GNOME Platform Stormclouds

Posted Mar 25, 2004 20:34 UTC (Thu) by tjc (subscriber, #137) [Link]

My experience with Tcl is restricted to Vignette V/5, one of the most unproductive development environments that I've had the displeasure of using, so in this case I agree with you. Tcl on its own may be a much better experience.

I have little bad to say about PHP -- I like it better than Perl for server-side scripting. I've never used Python, other than working through the O'Reilly book a couple years ago to see what it was about. It seemed ok.

GNOME Platform Stormclouds

Posted Mar 27, 2004 7:42 UTC (Sat) by Cato (subscriber, #7643) [Link]

For (some) static checking in a dynamic language, try Perl - most code is written with 'use strict', which catches variable name typos although the type checking is still done at run time.

Or you can just use OCaml (Objective Caml), which is garbage collected and dynamic like Perl/PHP/Tcl/Ruby/Python, but also has strong inference-based type checking (somewhat like C/C++/Pascal, except that you never have to declare types - it just infers all types, working up from the constants and operators). OCaml is almost unique in having the ease of programming of a dynamic byte-code interpreted language (a la Perl) and the option to compile to truly efficient machine code (which is sometimes faster than C, due to more optimisation enabled by stronger type checking, see link above for details). Well worth a look.

GNOME Platform Stormclouds

Posted Mar 27, 2004 10:30 UTC (Sat) by mly (guest, #2171) [Link]

Or you can just use OCaml (Objective Caml), which is garbage collected and dynamic like Perl/PHP/Tcl/Ruby/Python, but also has strong inference-based type checking...

I realize that this helps improve performance, but does it prevent spelling errors?

GNOME Platform Stormclouds

Posted Mar 28, 2004 6:40 UTC (Sun) by Cato (subscriber, #7643) [Link]

Yes, OCaml prevents spelling mistakes in variable names and any type mis-matches, just like C/C++ type checking. In addition, it can also catch more subtle errors - my OCaml page (see original post) has a link to an example where a similar language's compiler detected an infinite loop at compile time

GNOME Platform Stormclouds

Posted Mar 25, 2004 18:59 UTC (Thu) by NAR (subscriber, #1313) [Link]

I generally have an edit - test cycle in Python which is much shorter than my edit - compile cycle in C

I generally have the opposite situation - the test runs slow (because the program communicates with a remote computer which I don't have any control over so I can't make it faster) but the C++ compilation is fast (it doesn't take that long to compile a file and link an executable). In this case it's really a pain in the a** if I mistyped a variable name in the last few statements of the Python code... This is the kind of error that could be found at compile time - I see no reason why we couldn't find it. The -c option of perl is really useful for this situation.

Bye,NAR

GNOME Platform Stormclouds

Posted Mar 25, 2004 19:56 UTC (Thu) by mly (guest, #2171) [Link]

> I generally have the opposite situation - the test runs slow
> (because the program communicates with a remote computer which
> I don't have any control over so I can't make it faster) but
> the C++ compilation is fast

This will probably not be an issue very often in GNOME, but you
are certainly right that running the complete program might
be slow. Nowadays I often write programs that access databases,
and I can tell you that accessing DB2 system tables on IBM
mainframes isn't very fast...

A solution to this is to either use proper unit tests for each
class, as prescribed by extreme programming, or to just make
a stub for the communication. I don't know what your program
does, but assuming that you will follow different paths in your
program depending on what data you recieve through your
communication, you could make a stub that will replace the actual
communication with test data that will cause your all of the
paths of your program to be explored. Perhaps you can make some
command line switch, or just a global variable TEST that will
let you read a file instead of reading from a socket. Then you
can just put yor test data in a file.

As I said, PyChecker or PyLint can also find many errors, but
test driven development can be particularly useful. When your
application gets big, you will be much less worried about
changing code if you can test each class and module separately.

Writing the tests first a la XP is both good as an exercise in
trying to flesh out in a proper and interface focused way what
the various parts of the program does, and you also get examples
of how each class and method can be used. In that way, the tests
become useful documentation. Always writing the tests first also
assures that the tests actually get written, and that you get a
good regression test suite.

GNOME Platform Stormclouds

Posted Mar 26, 2004 6:56 UTC (Fri) by bronson (subscriber, #4806) [Link]

"A solution to this is to either use proper unit tests for each
class, as prescribed by extreme programming, or to just make
a stub for the communication..."

or, just use a statically typed language. Languages are tools: use the right one for the job. mly, don't you think you're laying on the Python advocacy awfully thick?

GNOME Platform Stormclouds

Posted Mar 26, 2004 13:36 UTC (Fri) by mly (guest, #2171) [Link]

> or, just use a statically typed language. Languages are tools:
> use the right one for the job. mly, don't you think you're
> laying on the Python advocacy awfully thick?

I agree that we seem to have left the subject of GNOME rather far behind. :)

I'll just summarize and then I'll try to stay away from this thred. I hope you will see my point.

* Static typing is not a full substitue for tests. The compiler will only find particular types of errors. Never logical errors.

* Whether we use a language with static or dynamic typing, we *should* write automated unit tests when we develop systems of significant size. Automated regression tests are important for the long term maintenance of any software project.

* Proper unit tests will find all errors that the compiler would. If your unit tests fail to find typing errors in your code, you either have dead code that you should remove, or incomplete tests. This is not because you write extra tests to find spelling errors, but because spelling errors you make will have an effect on the actual function of the code, which you should test.

* With an appropriate language (such as Python) and good working habits, the code/test cycle is typically faster than the code/compile cycle in e.g. Java or C.

While this might sound good in theory, I won't deny that there are problems involved. It might be difficult to maintain a large set of tests if requirements change, and some things, such as GUIs (which are certainly relevant in the case of GNOME :) aren't always easy to unit test in an automated way with current free tools.

I think the solution to this is to make the tools and methods for testing even better, not to go back to old fashioned languages where you have to write several times more code than you need with a modern language.

GNOME Platform Stormclouds

Posted Mar 28, 2004 18:18 UTC (Sun) by bronson (subscriber, #4806) [Link]

Writing a comprehensive set of unit tests is realistic only for easily-definable portions of your program. Think of the amount of code required to fully test, say, a web browser. It would take years to write and you could never hope to make it complete. The problem space is just too complex. Unit tests are excellent, but they can't be applied universally.

"...not to go back to old fashioned languages where you have to write several times more code than you need with a modern language."

And yet you advocate writing comprehensive volumes of unit tests? Even if your surprising claim of "several times more code" were true, given the length of each of your replies to this story, I'm surprised that you should be so afraid of a few more keystrokes. :-)

mly, have you ever released a project that demonstrates your approach to unit testing? I'd like very much to see an example of what you advocate. For comparison, I wrote unit tests into Trestlemail 4 years ago. It took a very long time to write and debug those 150 tests, yet they were hopelessly insufficient. And Trestlemail has a simple problem space.

GNOME Platform Stormclouds

Posted Mar 28, 2004 22:57 UTC (Sun) by mly (guest, #2171) [Link]

Writing a comprehensive set of unit tests is realistic only for easily-definable portions of your program.

I've used the term Unit Test the way it's been used in Extreme Programming. These tests are not complete functional tests for the program in question. They test small units (typically classes) in isolation. Their aim is to assert that a piece of code does what the programmer inteded. They are rarely functionally complete, but often exercise every line of the unit under test, and then they will find everything that the compiler would, and many other things. See http://www.c2.com/cgi/wiki?ProgrammerTest

Functional testing on a system level is an interesting subject, but it has nothing to do with static typing or the lack thereof.

Even if your surprising claim of "several times more code" were true...

Is that controversial? Just a few examples:

  • Glyph Lefkowitz rewrote Twisted in Python and went from 20000 lines to 3000 lines.
  • Greg Stein reported that rewriting eShop in Python led to 90% fewer lines than C++.
  • Bruce Eckel, author of Thinking in C++ and Thinking in Java reports a tenfold productivity gain with Python.
I'm sure there are many situations where the improvement is much smaller, the study by Prechelt I referred to before had smaller differences, but it showed more than 50% reduction in lines of code and developer time, as well as fewer bugs.

...given the length of each of your replies to this story...

I wish I was able to explain things more briefly. It's not a skill I have I'm afraid. :( It impresses me when people can get the same message through in 50 words that I need 250 for, but at least I can do that stunt in code! :)

...I'm surprised that you should be so afraid of a few more keystrokes.

  • Fewer lines of code means fewer bugs. Many studies verify this.
  • Fewer lines of code means that it's easier to understand the program.
  • Fewer lines of code means shorter development time. This has also been verified many times. Less time spent coding means more time to test, rewrite bad parts and relax. I.e. better program, happier programmer.
All this naturally assumes that the fewer lines of code comes from using a language which lets you use a higher level of abstraction (or from finding a simpler solution to your problem). Every seasoned C or C++ programmer knows that squeezing too much code into one line will make debugging much harder, and that it's better to use a few lines more. That might make it hard to think that another language will allow you to reduce development time and bug count while making you type much less, but it works. Just like C was a big step forward from assembly language, Python was a big step forward from C.

Among the actively used and developed languages, Python stands out as a language where readability and ease of use and learning has a high priority in its design. It's not without warts, but it's a lot better than most, even if several others reach the same level of abstraction.

have you ever released a project that demonstrates your approach to unit testing?

SystemSpecifyer is publically available at sourceforge (see also www.systematik.se). I haven't been involved for a year, but there are few checkins since then. It's a rather odd piece of software though, and it has dependencies to particular versions ZODB and wxPython, but if you want have a look at it, go ahead. It's far from perfect, but the unit test were very helpful to me.

I wrote unit tests into Trestlemail 4 years ago.

Did you write the tests before you wrote the code to test? For me that makes a big difference. I've never managed to catch up with unit tests if I rushed ahead and wrote working code before the tests. This will then come back and haunt me when I need to change the code and don't have tests that show me at once when I break things. I'm not sure extreme programming works well in all situations, but I think they are right about writing the tests first. Among other things, it makes you write programs that are testable! Having the focus on testing from the start makes a difference.

There is much more to do on the subject of testing, and I think this is a very interesting subject, but we've strayed far off the issue of GNOME now.

Suffice to say that many project show that it's possible to write dependable software in Python. That Python is one of three permitted languages on the floor of the New York Stock Exchange and that it's used to plan space shuttle launches for NASA also suggest that you can write fairly reliable code with it.

GNOME Platform Stormclouds

Posted Mar 29, 2004 21:18 UTC (Mon) by bronson (subscriber, #4806) [Link]

If unit tests only test classes in isolation, then there's no way that they could be a good substitute for static type checking (as you claimed earlier). What about inter-unit errors?

FYI, the examples you posted are contrived. They either involve a significant amount of refactoring and redesign (eShop) or a heavy reliance on standard libraries (Twisted). Python is definitely less verbose than C++, but 90% is laughable. Remember that we're comparing langauge features here, not programming environments.

GNOME Platform Stormclouds

Posted Mar 29, 2004 23:31 UTC (Mon) by mly (guest, #2171) [Link]

If unit tests only test classes in isolation, then there's no way that they could be a good substitute for static type checking (as you claimed earlier). What about inter-unit errors?

I'm sorry if I was unclear. The purpose of the unit test is only to test the unit under test. That doesn't mean that you typically replace the calls to external units with stubs. You only use stubs when there is a good reason for it, e.g. when networking or database access is too slow, and then you can probably still have some simple test included that uses the interface.

In SystemSpecifyer for instance, I use ZODB's (non-persistent) dummy storage when I test my business logic. I access ZODB through the same module (kbase) to store things, but it's never saved to disk in these tests. On the other hand, the unit tests for the kbase module use filestorage, like my real application.

Another extreme programming practice that helps solve this problem is continous integration, but this is turning into a full XP tutorial... :)

Python is definitely less verbose than C++,

Great! We agree on something! :)

but 90% is laughable.

I never claimed than a ten-fold productivity gain was typical. I quoted a number of sources claiming from 2-3 and up to 10 times improvement. Your gain will depend on your use case. I think that several times more code in e.g. C than in Python is typical though, and that's what I wrote. That's my personal experience after seven years with Python and fifteen with C/C++.

Remember that we're comparing langauge features here, not programming environments.

I've just tried to explain why programming in Python will typically lead to fewer bugs, not more bugs, than e.g. C, even though Python lacks static typing. (But I guess that's easier to experience from programming than to convince others about in a comment thread on LWN...)

I'm not sure what you mean by "programming environment", but Python's standard library is certainly a part of Python, and a significant reason for Python's success.

GNOME Platform Stormclouds

Posted Mar 30, 2004 20:24 UTC (Tue) by bronson (subscriber, #4806) [Link]

mly, I really wish you would edit your replies to a more manageable size. I now have no idea what you're trying to say.

This thread started when you were advocating using unit tests instead of static type checking. I have been saying that in most real-world situtations, unit tests are a poor substitute for static type checking, and I maintain that position.

I don't understand why you're trying to sell Python so hard. It's a language, not a used car! And I'm not even in the market...

GNOME Platform Stormclouds

Posted Mar 29, 2004 23:56 UTC (Mon) by mly (guest, #2171) [Link]

I just stumbled over a blog by Bruce Eckel that might
explain what I've tried to say better than I can.
http://mindview.net/WebLog/log-0052

GNOME Platform Stormclouds

Posted Apr 1, 2004 13:27 UTC (Thu) by joib (guest, #8541) [Link]

That was a nice argument from Bruce. He has also argued similarly before, in a previous blog entry. He even spells it out in bold font: "If it's not tested, it's broken."

GNOME Platform Stormclouds

Posted Apr 2, 2004 4:10 UTC (Fri) by dvdeug (subscriber, #10998) [Link]

Proper unit tests will find all errors that the compiler would.

That's not at all true. Compilers may find errors that no amount of testing on your system would ever find, but the instant they run on a big-endian system or a system with a different libc or different size of integer they would have failed. They can find undefined variables that would have defaulted to a value that would usually work. One problem with unit tests is that programmers frequently have blind spots; while compilers have blind spots, they're much more consistent and usually different from the programmers.

If you create a variable for seconds in a minute, and handle every case from 0 to 59, you will be wrong once every leap-second. And if you use the standard time library in Ada, it will emit a warning on compilation, whereas a language that was interfacing two integers would have no warning, and no bug unless you specifically tested that case, which many programmers might not even know exists.

What about the cases where you don't what answers to expect? A book was published about the results from an expirement done in Fortran, and afterwards it was discovered that there was two different variables (one implicitly defined) where there should have been one. If he could have written a unit test to find this, he would have noticed it before the book was published.

Shouldn't GNOME libraries be portable?

Posted Mar 26, 2004 7:50 UTC (Fri) by skybrian (subscriber, #365) [Link]

I'm not a GNOME developer, but it seems to me that to be consistent with GNOME's goal of
portability, it should be possible to compile GNOME libraries to both Java .jar files and C#
assemblies. Just like Linux and the base GNOME libraries are portable to any reasonable real
machine, managed GNOME libraries should be portable to any reasonable virtual machine. Let
the end-users decide which virtual machine they want to run. Eventually the Parrot developers
might want to run these libraries too.

How could it be done? Since Java bytecodes are pretty much a subset of C# bytecodes, the
source language probably would look more like Java. But it would have to be a strictly defined
subset of both Java and C#. It would be best if there were a compiler specifically designed to
target multiple VM's and flag non-portable output, sort of a gcc for virtual machines.

GNOME Platform Stormclouds

Posted Mar 26, 2004 12:20 UTC (Fri) by massimiliano (subscriber, #3048) [Link]

Many of the previous comments are correct, but seem to miss one main point...

The issue here is not "let's write GNOME applications in Java/C#/ Perl/Python/Ruby". This is something that can be easily done now, with any of those languages.

The issue is "let's integrate a virtual machine in the GNOME system libraries, so that GNOME reusable components can be written in a language that is more abstract/expressive/safe/productive than C".

The issues about the type safety of languages, IMHO, should be looked at from this perspective.

In some cases it is perfectly appropriate developing an application in a language that is not statically type safe, while in other cases it is better to have a type safe language. Each application developer can pick his own choice, and be happy with it. It can be also perfectly acceptable to have different VMs supporting different languages for different applications.

On the other hand, for reusable components, the requirements are more stringent. Moreover, if you want to have true reusable components, ease of interoperability with other technologies is a must.

What Microsoft is doing internally, with .NET, is replacing COM and DCOM as a technologies for writing components with a more modern way of doing things (.NET assemblies and remoting). They will have the advantage of a very coherent system when the migration is completed (longhorn), and a very easy and incremental migration path for ISVs in the meanwhile.

If I got it correctly, it is because he was seeing this potential in the .NET standard that Miguel started the mono project; he tought that those benefits would apply to a free platform (GNU/Linux/GNOME) as well.

In my opinion, for individual applications the problem of "standardizing on one VM" is not an issue. Each application writer picks his own choice, and everything works anyway, just like we are doing now.

It is when you want to have a coherent system, and reap the benefits of a more modern platform also at the system component level, that you need to pick a choice globally to avoid fragmentation.

At this point, the only options are Parrot, Java and mono (for the sake of completeness, also portable .net, but this counts as a .NET reimplementation like mono). Everybody has his opinion on the subject... for me, Parrot is not ready, and Java is not free enough. Besides, .NET is very, very well tought out, and generally technically superior.

GNOME Platform Stormclouds

Posted Mar 27, 2004 14:48 UTC (Sat) by mly (guest, #2171) [Link]

Good points. My nagging worry with .NET is that Microsoft might get a chance to mess things up for us in the courts...

Just a few days ago, the European Union decided that Microsoft has to pay a big fine for having restricted competition and they are forced to open up their APIs. The coin has a backside though...

"Because Microsoft will be allowed to pursue royalty revenue from the APIs it publishes, Jeremy Allison says that the projects such as Samba, which he jointly leads, may face a prohibitive hurdle."

Quoted from http://theregister.co.uk/content/4/36520.html

Are we sure no such issues will pop up with .NET?

A related question:

Is it really clear that CORBA is no good as an infrastructure for GNOME? Why would .NET be so much better?

It seems to me that there are CORBA implementations for Linux that are much more mature than Mono, and they aren't burdenend by patents or Microsoft intellectual property. There are language bindings for all popular languages, and there are several alternative implementations.

GNOME Platform Stormclouds

Posted Mar 29, 2004 8:52 UTC (Mon) by massimiliano (subscriber, #3048) [Link]

"Because Microsoft will be allowed to pursue royalty revenue from the APIs it publishes, Jeremy Allison says that the projects such as Samba, which he jointly leads, may face a prohibitive hurdle."

Quoted from http://theregister.co.uk/content/4/36520.html

Are we sure no such issues will pop up with .NET?

The main reason why this is not an issue for GNOME is that .NET (and, more importantly, mono) is not a monolithic thing, kind of "all or nothing". Miguel tried hard to explain this, basically there are three large pieces in mono: the ECMA core, the Microsoft APIs reimplementation, and the Linux/Unix/GNOME APIs bindings. The only piece that can have some issue (in the sense you mention above) is the reimplementation of the Microsoft APIs, and it is totally unrelated with the other big chunk of APIs (the Linux/Posix/GNOME ones).

This is something very important: the only thing Microsoft can do is create difficulties in emulating the Windows APIs, which are needed only if you want to run .NET applications "conceived" for Windows.

In mono, we are creating a different "stack" of APIs, that give access to the Posix world, GTK+, Mozilla, Evolution, and ultimately all the GNOME libraries. On these, Microsoft has no rights at all.

Of course software patents are a different issue... but those patents would have to apply to the GNOME libraries themselves, and therefore would be a problem even today, and without mono (I hope you get the point, otherwise feel free to ask for a clarification).

Is it really clear that CORBA is no good as an infrastructure for GNOME? Why would .NET be so much better?

Hmmm... CORBA is just like a specific application of .NET remoting, and just that. .NET is much more powerful, it is a whole platform for software developing and execution. In simpler words, CORBA has no bytecode and virtual machine definition, and no real class libraries. It is just a way to define remote interfaces, with one protocol to have them talking each other, and the specification of language bindings to use these interfaces.

In fact, GNOME components (Bonobo objects) are based on CORBA, and the result is not so exiting. The KDE guys started with CORBA, too, but then sow how bad it was for in process desktop components, and threw it avay, going for a "propriatery" component system (KParts) which works very well ("propriatery" because they designed it themselves, it is not based on any existing standard, but of course it is free software).

So, .NET would be better because it would be a real platform to write code, not just a way to specify abstract interfaces that turns out to be overkill and cumbersome to use for in-process components.

And mind you, I have always liked CORBA a lot, it is simply not a tool for this job (and, by the way, you could also have CORBA objects in .NET if you really wanted so, at least in principle, but it seems to me that current IIOP implementations are not yet "carrier grade").

GNUStep/ObjC

Posted Mar 26, 2004 16:33 UTC (Fri) by tgriggs (guest, #1670) [Link]

Why not just use ObjC? Myself, I'd rewrite the world in Smalltalk derivatives, but if the Gnome guy has to have some sort of type checking and C familiarity, why not use ObjC and the GNUStep libraries? The only drawback I can see is the lack of garbage collection, and even that can be hacked fixed by using the Boehm Collector.

Copyright © 2004, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds