Leading items

Pyjamas: writing AJAX applications in Python

Maintaining a complex web application that uses a lot of Javascript for client-side, "AJAX"-style interactivity is rather difficult. The clumsiness of the Javascript language itself, as well as the various tricks needed to make an application work consistently across multiple browsers, all of which must be wrapped up inside HTML, makes for a jumble of issues for the application developer. Pyjamas is meant to ease that development, by allowing client-side applications to be written in Python, then translating that code to Javascript for use by the browser.

Pyjamas is a port of the Google Web Toolkit (GWT), which pioneered the technique, but GWT uses Java instead of Python. For developers who prefer Python, pyjamas provides an environment that almost completely insulates them from the code that actually runs in the user's browser. As described on the home page:

Essentially, a developer uses the pyjamas-supplied libraries that provide user interface widgets and Document Object Model (DOM) control from Python. This code is then combined with a library that implements various Javascript language features in Python—an impedance-matching layer between the two languages called pyjslib—and turned into Javascript and HTML that can be loaded into a browser. The pyjs program "translates Python code to Javascript by walking the Python abstract syntax tree and generating Javascript".

But, using pyjamas is not at all like "normal" web programming. As the web site and FAQ are quick to point out, pyjamas is not just another AJAX (Asynchronous Javascript and XML) framework, it is more akin to writing a desktop program that is translated into web application. In fact, Pyjamas-Desktop allows the same code that is developed for the web application to be run, unmodified, on the desktop. The same Python source that gets fed into the translator can, instead, be run, and, more importantly, debugged, on the developer's desktop.

The tool is best suited to writing one-page web applications that rely entirely on AJAX techniques to do their job—things like Gmail, Mozilla's Bespin, and many others. Each page load in a pyjamas application requires loading all of the Javascript that makes up the application. That includes code generated from the application as well as pyjamas libraries, so, ideally, that would only be done once. That is quite a contrast from the traditional, multi-page-oriented web application, but is certainly in keeping with the direction of web interaction.

The key to understanding pyjamas is to note that, unlike AJAX frameworks, it is not meant to add a bit of interactivity, or some desktop-like features, to an existing web application. Instead, the entire application is written in Python, likely debugged on the desktop, and then turned into a big blob (or, really, blobs) of Javascript for deployment. The application code will look very familiar to Python GUI developers. For example, the canonical "Hello World" program—which does a bit more than Kernighan and Ritchie's original C program—looks like:

    import pyjd # this is dummy in pyjs.
    from pyjamas.ui.RootPanel import RootPanel
    from pyjamas.ui.Button import Button
    from pyjamas.ui.HTML import HTML
    from pyjamas.ui.Label import Label
    from pyjamas import Window

    import pygwt

    def greet(fred):
	print "greet button"
	Window.alert("Hello, AJAX!")

    if __name__ == '__main__':
	pyjd.setup("public/Hello.html?fred=foo#me")
	b = Button("Click me", greet, StyleName='teststyle')
	h = HTML("<b>Hello World</b> (html)", StyleName='teststyle')
	l = Label("Hello World (label)", StyleName='teststyle')
	base = HTML("Hello from %s" % pygwt.getModuleBaseURL(),
				      StyleName='teststyle')
	RootPanel().add(b)
	RootPanel().add(h)
	RootPanel().add(l)
	RootPanel().add(base)
	pyjd.run()

Running the build.sh script on that example, as described on the "getting started" page, creates output and public directories that hold the generated code. There is a tiny bit of HTML and CSS, along with roughly 9,000 lines of Javascript that implement the example. Much of that is likely boilerplate code to implement pyjamas itself. A better example might be something like TimeSheet, which implements a more realistic application, and weighs in around 23,500 lines.

There are numerous examples on the pyjamas web page, many of which were ported from GWT. Both the source code and the running application are available, so one can get a sense for how much code it takes to create the examples—as well as use them as templates for other applications. There is quite a bit of documentation, though the FAQ would indicate that there have been complaints about that, on the site as well, including the in-progress Pyjamas Book (which is implemented as a pyjamas book reader application).

The project has just released version 0.6 of the tool, with many new features outlined in the CHANGELOG. There are some 140 changes from the version 0.5, including a rework of pyjs to make more Python features available (multiple inheritance and superclasses are specifically mentioned) and bringing Pyjamas-Desktop into the standard distribution. The pace of development is relatively quick; 0.5 was released in March, and 0.6 adds quite a bit of functionality on top of that.

Pyjamas is definitely worth a look for anyone considering building a new-style web application, and who would rather use Python than Java. Because GWT was released as free software, Pyjamas could leverage much of that work to give developers another language choice. Writing—and worse, debugging—complex Javascript applications is a major chore, so any tools that make that easier should be quite welcome. Those that just want a bit more interactivity in their existing web applications, though, might find the Pyjamas (and GWT) approach to be too heavy-handed for their needs.

Comments (26 posted)

Bespin adds collaborative features

Mozilla released version 0.4 of its online code editor Bespin last week, debuting integrated collaboration features that include project sharing and real-time live collaborative editing. Bespin is an in-browser web application, making it available from anywhere, but the "social coding" features are what really separates it from other collaborative editing solutions.

Welcome to Bespin

A Mozilla Labs project, Bespin is a free web service to which users can connect from any supported browser. The main Bespin server on which new official releases are rolled out is bespin.mozilla.com; to use it one must register a free Bespin server account. The client side of the editing environment runs entirely within the page: no extensions or other add-ons are required. Source code to the server is available under the Mozilla Public License version 1.1, and instructions for installing and running Bespin on a private server are provided on the Mozilla wiki.

Bespin was first unveiled in February of 2009, and is described as a code editor for web developers, built with web technologies. It supports line numbering, syntax highlighting (for HTML, CSS, and JavaScript by default), multiple projects, version control with Mercurial, cut-copy-paste with a clipboard, search, undo and redo, and previewing on the web. Code projects are stored on the server, but the editor can upload zipped or gzipped archives as well as clone existing Mercurial projects into the user's Bespin server account.

Once logged into their Bespin server account, users can begin using Bespin immediately. Bespin's editor takes up the entire window, save for a thin toolbar at the top that displays the current project and file and a few basic buttons. The other major mode provided by the program is the Bespin command line, which can be activated from the toolbar or by hitting Ctrl-j (which toggles between the editor and command line). The command line allows the user to create new projects, change interface settings, open files, and more. Command completion, history, and tooltip-style help assist new users in getting up to speed with the command set. For opening files and examining projects, a graphical file browser is also provided, which mimics the left-to-right navigation panes of Mac OS X's Finder rather than a more traditional Unix-like folder tree.

The Bespin editor is rendered inside of an HTML5 canvas element, so a canvas-supporting browser is required. Currently, this includes all recent versions of Firefox, Chrome/Chromium, Opera, and WebKit/Safari. The canvas is used to render the entire editing environment — window panes, menus, buttons, scroll bars, even the cursor — with JavaScript. Developer Ben Galbraith described the use of canvas as a superior alternative to building on top of the textarea tag or contentEditable property and native user interface widgets because canvas allows better flexibility to adapt the UI to users' personal preferences, better accessibility, and better performance and compatibility when editing large files. He also offered a caveat in his explanation, noting that "there may very well be a more effective way to do what we've done than with canvas. I'm not saying this is the best way to do it; I just want to explain why we did it."

One outcome from the decision to use canvas is that the Bespin team developed its own UI toolkit, which it plans to spin off into a reusable code project called Thunderhead. Likewise, the editing component itself could be easily reused in other Web applications, so it will eventually be separated out into a stand-alone, reusable component, too.

Bespin is designed to be adjustable to the individual tastes of the user, so it provides user-configurable settings. In addition to basic configuration options like auto-indentation and "strict lines" mode, each Bespin server account automatically includes a BespinSettings project that contains a config.js file for tweaking Bespin with JavaScript. Users can write their own commands, using the same syntax as Mozilla Labs' Ubiquity project, and Bespin has a plugin API which is still undergoing development.

Collaborate

The 0.4 release added some niceties to the existing feature set — such as argument completion for commands and the addition of Subversion as the second supported version control system — but the most exciting new addition is a built-in framework for sharing and live collaboration. Users must activate the collaborative features by executing set collaborate on from the Bespin command line.

The new "share" command allows users to mark projects as visible to other users on the Bespin server, as individuals, groups, or globally. Projects can be shared in read-only or read/write mode. Executing:

    share add projectname user1

shares the project in read-only mode, while

    share add projectname user1 edit

activates read-write permission. For sharing to work, the user on the receiving end of the shared project must explicitly accept shared code from the originating user with:

    follow username

This mechanism prevents spamming innocent users by sharing scores of unwanted projects with them. Substituting the string everyone for a username shares the project with everyone on the server who chooses to follow the project owner.

Other users' shared projects appear in the Bespin file browser just like locally-owned projects. If a project is shared in read-write mode, all users with access to it can edit a file simultaneously. Edits and text selection are visible in real time to all who have the file open in the editor. A collaboration window showing which users are currently editing the file appears in the upper right-hand corner of the editor window.

"Groups" in Bespin collaboration are strictly a private user-defined convenience, not a social feature; adding other people to a group simply saves keystrokes by permitting the user to type:

    share add myproject mygroup

instead of:

    share add myproject user1
    share add myproject user2
    ...

To define a group, just execute:

    group add groupname user1 user2 ... userN

Bespin uses Mobwrite to handle synchronization and conflict resolution between user edits. The system uses differential synchronization to keep copies of the file in sync. This algorithm is symmetric, repeatedly calculating the differences between each editing session's copy of the document and a previously-updated "shadow" copy, then relaying patches to every other session. Since every session receives every other session's updates, there is no "master" copy responsible for merging all of the changes and potentially creating bottlenecks. Unlike "event passing" synchronization, diffs automatically catch all changes, whether originating from typing, cut-and-paste, undo, or search-and-replace. Developer Dion Almaer said that the team has successfully tested Bespin with 30 concurrent users and 100,000-line files, making it significantly more robust than other online sharing tools.

Currently, sharing and collaborative editing are only possible between users on the same Bespin server, but Almaer said that the team very much wants to include support for federation in the future. "We want to create a compelling, fun, productive social editing experience for as many people as possible," he said. Joe Walker, who added the collaboration support to 0.4, commented that the Bespin team hopes to enable collaborative sessions between Bespin and other editors capable of collaboration, such as GEdit, Eclipse, and Emacs.

The future of Bespin will include more features to enhance this social dimension, Almaer explained:

Likewise, the Bespin team encourages users to add new features via JavaScript and the plugin API. Outside contributors have already added syntax highlighting support for Ruby and Arduino code. Almaer said that the initial feature set of the editor focuses on web technologies merely because the web is Mozilla's focus and the team has limited resources. If the Bespin development community wants to expand the scope of the project, it will.

Collaborative editing is possible in several other open source applications, from word processors like Abiword to text editors like Gobby, and via plugins for many more applications — even popular development tools like Emacs, gVim, and Eclipse. At first glance, Bespin seems to sport the same feature set as most of these other options, but it has two distinctive qualities that users may find attractive.

First, as pointed out in the initial release announcement, Bespin is available everywhere, on every platform with a modern web browser. The alternatives require installation, and in many cases are dependent on specific versions of the application for compatibility. Some are limited to specific operating systems because they rely on underlying packages like Telepathy.

Second and more importantly, Bespin is pushing the "social" nature of coding itself as the key feature, not just synchronization of files. The concepts of sharing projects and following individuals are the beginnings of a social graph implementation that could extend the concept beyond simple text changes. As Almaer hinted, and as shown in the project's roadmap and in various demo videos, more is on the way, such as a "heatmap" that graphically depicts where a codebase is changing and a dashboard that tracks what other users are doing.

Bespin 0.4 is stable and responsive enough that the Bespin team uses it for Bespin development. It should prove to be a valuable asset to remote development teams, particularly (in the short term) for those working in the core web technologies that remain its focus. Looking further out, the social coding concepts Bespin explores could significantly improve on the traditional distributed version control systems most developers use today.

Comments (4 posted)

SCO: not dead yet?

Back in 2007, it seemed like the SCO nightmare was done; the company had suffered a summary judgment depriving it of its claim to the Unix copyrights and it had gone into bankruptcy proceedings. In the latter half of 2009, though, SCO is still here. Now, an appeals court has ruled [PDF] that part of the 2007 judgment was erroneous and must be reconsidered; some worry that SCO could come back, zombie-like, to terrorize again. The real threat may not be SCO, though, but what comes after.

The agreement between Novell and the Santa Cruz Operation was a mess which never clearly spelled out what was being sold. It is far from surprising that Novell and the company now known as the SCO Group disagree on its particulars. The lawyers involved in making that agreement, quite simply, did not do their job. Even so, the district court, in 2007, was able to obtain enough clarity from this document to conclude that there was no question at all of whether the Unix copyrights had been transferred to SCO. The result was a summary judgment throwing out SCO's claims regarding those copyrights. That judgment was welcomed in the community, but there may be justice to SCO's claim that it was a little too hasty.

The appeals court took that view of the district court ruling, finding SCO's arguments sufficiently credible to create some doubt as to the facts in the case. As the ruling states:

It is worth noting that the appeals court did not rule that the copyrights do, in fact, belong to SCO. In fact, the ruling reads:

All the court has said is that there is enough doubt here that a full trial is needed to resolve the question. The end result could well be the same - Novell could still win - but SCO has created enough uncertainty to gain its full day in court.

SCO did not prevail in all of its appeals, though. The district court had ruled that SCO had converted ("stolen") $2.5 million in licensing revenue from Sun which, by the asset purchase agreement, truly belonged to Novell. The appeals court agreed with this part of the summary judgment and left SCO on the hook. This ruling has led some observers to believe that SCO may now head quickly into Chapter 7 bankruptcy, resulting in the liquidation of the company. Should that happen, it is said, the SCO zombie will truly be gone forever.

Unfortunately, the end of the SCO Group would not necessarily mean the end of the troubles it has created. Chapter 7 bankruptcy would result in the sale of SCO's remaining assets to pay the claims of the company's creditors. Those assets are likely to include office chairs belonging to both remaining employees, a storage locker full of unsold Caldera OpenLinux boxes, Darl McBride's bullhorn, a few SCO Mobile Server manuals - and the claimed ownership of the Unix copyrights. The bankruptcy trustee's job will be to sell all of these assets for the highest price possible.

The Unix copyright "asset" is an uncertain value, to say the least; it could evaporate entirely when SCO v. Novell runs its full course. But SCO v. Novell was always a sideshow; the real game is the multi-billion dollar claims against IBM. Who would be willing to bet that no trolls willing to try for that payoff exist? Instead, such trolls must certainly exist, and some of them will be well funded. One of them could come out of the bankruptcy process owning these "assets" and the related lawsuits.

What we could see then is a new push on these claims, with more money behind it and, possibly, less buffoonery as well. The new owner might just succeed in establishing ownership of the Unix copyrights - that agreement is, as was stated previously, a mess - and carry the IBM case forward. The fact that SCO's original claims against IBM are still without merit offers little comfort; we could be in for another extended period of FUD and bad press before the courts finally come to that conclusion.

Things need not go that way, of course. Should SCO go into chapter 7, we might see Novell and/or IBM hold their noses and buy the Unix claims themselves, putting an end to the entire affair. It could be the cheapest way for them to go. One assumes that lawyers in those companies are thinking about their options at this point; they, too, must be tired of this whole circus by now.

Comments (34 posted)

Page editor: Jonathan Corbet
Next page: Security>>