Pyjamas: writing AJAX applications in Python

By Jake Edge
August 26, 2009

Maintaining a complex web application that uses a lot of Javascript for client-side, "AJAX"-style interactivity is rather difficult. The clumsiness of the Javascript language itself, as well as the various tricks needed to make an application work consistently across multiple browsers, all of which must be wrapped up inside HTML, makes for a jumble of issues for the application developer. Pyjamas is meant to ease that development, by allowing client-side applications to be written in Python, then translating that code to Javascript for use by the browser.

Pyjamas is a port of the Google Web Toolkit (GWT), which pioneered the technique, but GWT uses Java instead of Python. For developers who prefer Python, pyjamas provides an environment that almost completely insulates them from the code that actually runs in the user's browser. As described on the home page:

You can write web applications in python - a readable programming language - instead of in HTML and Javascript, both of which become quickly unreadable for even medium-sized applications. Your application's design can benefit from encapsulating high level concepts into classes and modules (instead of trying to fit as much HTML as you can stand into one page); you can reuse - and import - classes and modules.

Also, the AJAX library takes care of all the browser interoperability issues on your behalf, leaving you free to focus on application development instead of learning all the "usual" browser incompatibilities.

Essentially, a developer uses the pyjamas-supplied libraries that provide user interface widgets and Document Object Model (DOM) control from Python. This code is then combined with a library that implements various Javascript language features in Python—an impedance-matching layer between the two languages called pyjslib—and turned into Javascript and HTML that can be loaded into a browser. The pyjs program "translates Python code to Javascript by walking the Python abstract syntax tree and generating Javascript".

But, using pyjamas is not at all like "normal" web programming. As the web site and FAQ are quick to point out, pyjamas is not just another AJAX (Asynchronous Javascript and XML) framework, it is more akin to writing a desktop program that is translated into web application. In fact, Pyjamas-Desktop allows the same code that is developed for the web application to be run, unmodified, on the desktop. The same Python source that gets fed into the translator can, instead, be run, and, more importantly, debugged, on the developer's desktop.

The tool is best suited to writing one-page web applications that rely entirely on AJAX techniques to do their job—things like Gmail, Mozilla's Bespin, and many others. Each page load in a pyjamas application requires loading all of the Javascript that makes up the application. That includes code generated from the application as well as pyjamas libraries, so, ideally, that would only be done once. That is quite a contrast from the traditional, multi-page-oriented web application, but is certainly in keeping with the direction of web interaction.

The key to understanding pyjamas is to note that, unlike AJAX frameworks, it is not meant to add a bit of interactivity, or some desktop-like features, to an existing web application. Instead, the entire application is written in Python, likely debugged on the desktop, and then turned into a big blob (or, really, blobs) of Javascript for deployment. The application code will look very familiar to Python GUI developers. For example, the canonical "Hello World" program—which does a bit more than Kernighan and Ritchie's original C program—looks like:

    import pyjd # this is dummy in pyjs.
    from pyjamas.ui.RootPanel import RootPanel
    from pyjamas.ui.Button import Button
    from pyjamas.ui.HTML import HTML
    from pyjamas.ui.Label import Label
    from pyjamas import Window

    import pygwt

    def greet(fred):
	print "greet button"
	Window.alert("Hello, AJAX!")

    if __name__ == '__main__':
	pyjd.setup("public/Hello.html?fred=foo#me")
	b = Button("Click me", greet, StyleName='teststyle')
	h = HTML("<b>Hello World</b> (html)", StyleName='teststyle')
	l = Label("Hello World (label)", StyleName='teststyle')
	base = HTML("Hello from %s" % pygwt.getModuleBaseURL(),
				      StyleName='teststyle')
	RootPanel().add(b)
	RootPanel().add(h)
	RootPanel().add(l)
	RootPanel().add(base)
	pyjd.run()

Running the build.sh script on that example, as described on the "getting started" page, creates output and public directories that hold the generated code. There is a tiny bit of HTML and CSS, along with roughly 9,000 lines of Javascript that implement the example. Much of that is likely boilerplate code to implement pyjamas itself. A better example might be something like TimeSheet, which implements a more realistic application, and weighs in around 23,500 lines.

There are numerous examples on the pyjamas web page, many of which were ported from GWT. Both the source code and the running application are available, so one can get a sense for how much code it takes to create the examples—as well as use them as templates for other applications. There is quite a bit of documentation, though the FAQ would indicate that there have been complaints about that, on the site as well, including the in-progress Pyjamas Book (which is implemented as a pyjamas book reader application).

The project has just released version 0.6 of the tool, with many new features outlined in the CHANGELOG. There are some 140 changes from the version 0.5, including a rework of pyjs to make more Python features available (multiple inheritance and superclasses are specifically mentioned) and bringing Pyjamas-Desktop into the standard distribution. The pace of development is relatively quick; 0.5 was released in March, and 0.6 adds quite a bit of functionality on top of that.

Pyjamas is definitely worth a look for anyone considering building a new-style web application, and who would rather use Python than Java. Because GWT was released as free software, Pyjamas could leverage much of that work to give developers another language choice. Writing—and worse, debugging—complex Javascript applications is a major chore, so any tools that make that easier should be quite welcome. Those that just want a bit more interactivity in their existing web applications, though, might find the Pyjamas (and GWT) approach to be too heavy-handed for their needs.

Pyjamas: writing AJAX applications in Python

Posted Aug 26, 2009 17:59 UTC (Wed) by nye (subscriber, #51576) [Link] (12 responses)

This sounds like an excellent project, mostly because it's a) retargetable (running the same code as a desktop application is a nice trick) and b) handles "the various tricks needed to make an application work consistently across multiple browsers, all of which must be wrapped up inside HTML".

What I don't get is why it's so fashionable to add bits like "the clumsiness of the Javascript language itself". Javascript is probably my second favourite programming language (after Lua, which I sadly find too little opportunity to actually use). Usually when people deride Javascript it turns out they are actually talking about, say, inconsistent event handling across browsers, or implementation bugs like IE's memory leaks when closures are used in particular ways - or other problems which really are in no way part of the 'language itself'.

I do write a fair bit in Python, because of the broad array of libraries available, but I can't imagine that automated compilation of arbitrary Python libraries into JS+HTML is feasible, so this seems to take away most of Python's appeal. Nevertheless I'm sure I'll give it a try some time...

Pyjamas: writing AJAX applications in Python

Posted Aug 26, 2009 19:30 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link] (9 responses)

JavaScript is extremely cludgy language. It has no namespaces, idiotic variable scoping rules, absolutely moronic automatic semicolon placing, etc.

See this presentation: http://googlecode.blogspot.com/2009/03/doug-crockford-jav...

Pyjamas: writing AJAX applications in Python

Posted Aug 26, 2009 21:14 UTC (Wed) by iabervon (subscriber, #722) [Link] (5 responses)

The scoping and line termination rules, combined with being interpreted at runtime, make a large portion of bugs difficult to detect without actually reaching every line of code.

Of course, Python has the same problem as Javascript here, but it's more than twice as bad if you're doing a project with both, because they not only both have error-encouraging scoping and line termination rules, they have different rules. So the Python you're writing encourages you to accidentally make your Javascript variables global and forget required semicolons.

Pyjamas: writing AJAX applications in Python

Posted Aug 26, 2009 22:07 UTC (Wed) by drag (guest, #31333) [Link] (3 responses)

> because they not only both have error-encouraging scoping and line
termination rules,

What?

I don't understand. The line termination rules for python is very simple..
A line of code ends with a newline. The only exceptions that I can think
of to this is that you can explicitly end a line of code with ";" and you
can extend a mathematical statement by enclosing it in ().

This, to me, is a hell of a lot better then requiring ;'s all over the
place.
I tend to forget those time to time and it's sometimes very irritating to
track that down. In python as long as I keep my code clean and line
lengths to under 80 characters (or so) then it's easy to know what a line
of code is just by looking at it. A line of code is a line of text.

And what is error prone about Python's scoping? I can't imagine that its
any worse then C++ or C or Java or anything like that.

Pyjamas: writing AJAX applications in Python

Posted Aug 28, 2009 7:12 UTC (Fri) by niner (subscriber, #26151) [Link]

Open brackets or braces can extend an expression to multiple lines. And that is a very
good thing, otherwise it would make code quite unreadable. The ''' or """ quotes do the
same.

But both mean that you cannot simply trust a statement to be a single line of code.

Pyjamas: writing AJAX applications in Python

Posted Aug 30, 2009 13:18 UTC (Sun) by kleptog (subscriber, #1183) [Link]

Actually, python's scoping is one of my biggest problems. The fact that the scoping of a variable can change in odd ways. Consider:

import time

def main():
   print time.clock()

   time = 1

main()

This program produces an error on the "print" line, yet if you comment out the "time = 1" line it works. This means you have to read a whole function before you can determine the scope of a variable. Being someone who controls scopes carefully so that typos are more easily found, it all feels weird.

Pyjamas: writing AJAX applications in Python

Posted Sep 3, 2009 4:28 UTC (Thu) by tjc (guest, #137) [Link]

This, to me, is a hell of a lot better then requiring ;'s all over the place. I tend to forget those time to time and it's sometimes very irritating to track that down.

Semicolons are optional in Javascript, so why bother to "track that down?" If you don't like them, then just leave them off.

Pyjamas: writing AJAX applications in Python

Posted Aug 29, 2009 5:44 UTC (Sat) by SteveAdept (guest, #5061) [Link]

Add me to the list of people who don't like Javascript. My primary peeve is how it represents null values, similar to the mess PHP makes of it. If I never initialized a variable before I start reading from it, I want to know about it. Ick. I find that it often makes debugging extremely frustrating for me.

And I realize this isn't specifically a language issue, but I hate the fact that we have to deal with so many different implementations. I'm spoiled by the ubiquity and consistency of CPython, I guess.

Pyjamas: writing AJAX applications in Python

Posted Aug 26, 2009 22:10 UTC (Wed) by quotemstr (subscriber, #45331) [Link]

Yet it has lexical closures, which somehow most languages seem to relax. Closures are Javascript's saving grace; with them, most of your other concerns can be addressed.

(Well, and never using the scope-destroying, mind-liquefying, optimization-prohibiting with facility.)

Pyjamas: writing AJAX applications in Python

Posted Aug 27, 2009 5:24 UTC (Thu) by flewellyn (subscriber, #5047) [Link] (1 responses)

I quite disagree: Javascript has a few warts, but it's quite elegant at its core, and very powerful.

Namespaces might be of use, but you can use objects to handle them easily enough; the variable scoping rules are only a problem if you don't understand closures (which are one of the things that gives JS such power), automatic semicolon placing...yeah, that sucks, so I don't rely on it. I always use semicolons.

The main thing I dislike about Javascript, the language, is the overloading of + to mean "addition" and "string concatenation". Operator overloading is never a good idea, especially in this case, because JS is dynamically typed, and numeric values may be misinterpreted as strings.

I think JS gets a bad rap because people are used to dealing with DOM incompatibilities, and blame those on the language.

Pyjamas: writing AJAX applications in Python

Posted Aug 29, 2009 11:24 UTC (Sat) by lkcl (guest, #60496) [Link]

"I quite disagree: Javascript has a few warts, but it's quite elegant at its core, and very powerful."

oo, absolutely! think about this: an implementation of python classes - dynamic multiple inheritance including superclasses and the (very rarely used and misunderstood _three_ arguments to the type() function: kls = type("ClassName", [Baseclass1, Baseclass2], {'method1': fn })...

... that can be implemented in UNDER 100 lines of javascript!

there's a little trick to do "object-like" inheritance which is typically deployed in javascript frameworks: it creates a new Object() and then sets up the prototypes copying them from the base object. that's typically implemented in about... 15 lines of javascript. all we did was expand that a little bit further.

voila. instant emulated multiple inheritance and python superclasses.

the warts: achhh. anonymous objects that don't carry around the "this" pointer properly. having to create a wrapper function which re-associates the .... never mind :)

Pyjamas: writing AJAX applications in Python

Posted Aug 26, 2009 23:43 UTC (Wed) by gouyou (guest, #30290) [Link]

I didn't like JavaScript up to the point I read "JavaScript: The Good Parts" from Douglas Crockford. This book was a revelation for me. (If all languages would have such a book, it would make learning new language for people with already a bit of programing experience much simpler.)

Pyjamas: writing AJAX applications in Python

Posted Aug 29, 2009 12:14 UTC (Sat) by lkcl (guest, #60496) [Link]

"but I can't imagine that automated compilation of arbitrary Python libraries into JS+HTML is feasible, "

into HTML? no. into javascript? yes. that's exactly what pyjamas does, and it's also what skulpt is aiming for, and it's exactly what the pypy project aimed for, except they gave up.

the tricky bit is doing it _efficiently_. we decided that that was a bad decision to enforce onto people, so we decided to offer two options: -O and --strict. they do exactly what you'd expect.

Pyjamas: writing AJAX applications in Python

Posted Aug 26, 2009 20:40 UTC (Wed) by days_of_ruin (guest, #58404) [Link] (1 responses)

What distro were you running this on?

Pyjamas: writing AJAX applications in Python

Posted Aug 26, 2009 21:09 UTC (Wed) by jake (editor, #205) [Link]

> What distro were you running this on?

Fedora 10, but that doesn't really matter as I just grabbed the 0.6 tarball and played with that.

jake

Pyjamas: writing AJAX applications in Python

Posted Aug 27, 2009 13:38 UTC (Thu) by NAR (subscriber, #1313) [Link] (3 responses)

My experience with generated code is "all is well until the first bugreport". Usually the bugreport mentions parts (stack traces, line numbers, etc.) from the generated code - how do you get back from that to the code you've written? Not to mention that the code generating can introduce errors on its own. Nowadays I spend most of my time chasing bugs in (open source) infrastructure code that should work (but doesn't), so I'd be cautious about adding any extra layers...

Pyjamas: writing AJAX applications in Python

Posted Aug 28, 2009 10:29 UTC (Fri) by lkcl (guest, #60496) [Link] (2 responses)

"My experience with generated code is "all is well until the first bugreport". Usually the bugreport mentions parts (stack traces, line numbers, etc.) from the generated code - how do you get back from that to the code you've written?"

that is exactly why:

1) we added a debugging system which keeps track of the current line number and even optionally includes the python source code line _in_ the javascript, so that during development you can get a full python stack trace - WITHOUT having to have any javascript debugger installed.

2) i went to all the trouble to create pyjamas-desktop, so that you would be able to run the exact same application under the standard python interpreter.

"Not to mention that the code generating can introduce errors on its own."

this is a matter of trust. you place your trust in the tool that it is going to do the job. just as you trust any other tool to do its job. to back up that trust, we always add a regression test whenever a new feature is added, and i've been pushing to have regression tests added whenever someone reports a bug, too, to demonstrate the missing feature.

lovelysystems are also working quite hard to make the python-to-javascript compiler pass the _standard_ python regression tests, from a command-line version that is combined with spidermonkey, and i've added experimental support for python-spidermonkey and pyv8, as well. this is a similar goal to what skulpt are aiming for.

"Nowadays I spend most of my time chasing bugs in (open source) infrastructure code that should work (but doesn't), so I'd be cautious about adding any extra layers..."

welcome to free software! did you pay money for any of that infrastructure? so - you got what you paid for.

pyjamas, like any free software project, does the job that the people who are *working on it* want it to do. everyone else who is NOT contributing is getting the [free] benefits of their efforts and expertise.

you should be grateful that google released GWT as free software; grateful that james tauber did the port to python, and grateful that the current developers continue to maintain it and release their contributions to the wider world.

Pyjamas: writing AJAX applications in Python

Posted Aug 28, 2009 13:36 UTC (Fri) by NAR (subscriber, #1313) [Link] (1 responses)

that is exactly why

This is the information that was missing from the article (at least for me).

welcome to free software! did you pay money for any of that infrastructure? so - you got what you paid for.

Well, actually quite a bit of money is spent on the support...

you should be grateful that google released GWT as free software

I don't feel like, because I hope I never have to use it, neither as a developer or a user. Web-based UIs (in my experience) suck, regardless of the technology behind them.

Pyjamas: writing AJAX applications in Python

Posted Aug 29, 2009 11:17 UTC (Sat) by lkcl (guest, #60496) [Link]

"This is the information that was missing from the article (at least for me)."

yehhh, it's tricky to cover a comprehensive topic, and often it's interactive discussion that works better for some (hellooo :)

"Well, actually quite a bit of money is spent on the support..."

oh dear! try http://siriusit.co.uk they actually employ or have contacts with many of the developers who work on products that their customers deploy, such as postgresql, kde, samba etc. unfortunately, with google's policy of employing many of the worlds' top free software developers (so that nobody else can) it's a quite a hard trick for any support company to pull off.

"you should be grateful that google released GWT as free software

I don't feel like, because I hope I never have to use it, neither as a developer or a user. Web-based UIs (in my experience) suck, regardless of the technology behind them. "

*rueful smile* you're not the only person to be utterly disillusioned with web-based UI development. this is one of the reasons why i risked creating and emphasising pyjamas-desktop, because by going "direct" to the DOM model and cutting out the javascript, one of the main painful psychological barriers (javascript) is lifted.

the skills list required to do decent web-based user interfaces is just absolutely horrendous. i listed them only last week: http://advogato.org/person/lkcl/diary/623.html - many people simply cannot cope with this, in order to create the level of UI interaction experience that web users expect these days.

anyway - it's particularly interesting to note that john resig, an experienced javascript developer, criticises pyjamas from a different angle: one of not being "direct" enough. but the thing is, the whole point of the pyjamas UI API is to solve and encode as many of the quirks and the bits of expertise as possible behind a common API so that _you_ don't have to know them. [that's the job of AJAX frameworks: it's just that pyjamas is written (mostly) in python, not javascript]

both GWT and Pyjamas do that: one for java developers, one for python developers. (there's also RWT (RubyJS) but its development stopped in 2007 unfortunately).

the point is: myself and many others have been just as exasperated and disillusioned as you, with web UI development (i vowed once that i would never do javascript programming, now i'm doing the complete opposite!) and both GWT and Pyjamas tackle UI development with less emphasis on "web", more emphasis on "desktop-widget-like", and thus allow us to carry on, keep abreast of current user expectations, not give up and not go bananas with stress either :)

Pyjamas: writing AJAX applications in Python

Posted Aug 27, 2009 15:15 UTC (Thu) by hppnq (guest, #14462) [Link] (3 responses)

That "Hello World" strikes me as very ugly, and a 23k line timesheet application makes me shiver. (A one page Gmail client?!) It looks to me that this is especially appealing to developers who simply want to develop in Python, and not in javascript and HTML. Fair enough.

But the main attraction of running code unadapted on the desktop and in a browser is the abstraction of the language. Technically these things are not extremely challenging; you can (already) develop, run and debug javascript on your desktop, with or without a browser. But being able to write things like

<window><button/></window>

and run it in a browser, or from the desktop, or in your head -- that is something completely different. Something like xul, what a relief.

Pyjamas: writing AJAX applications in Python

Posted Aug 28, 2009 10:33 UTC (Fri) by lkcl (guest, #60496) [Link]

yes i didn't know that this article was going to be written - i added some extra lines to the helloworld example and accidentally committed them, when testing some CSS styles on buttons and also testing the getModuleBaseURL function.

the original helloworld app was 8 lines long, and you can see it here:

http://pyjs.org/book/output/Bookreader.html#Getting%20Sta...

Pyjamas: writing AJAX applications in Python

Posted Aug 28, 2009 12:17 UTC (Fri) by lkcl (guest, #60496) [Link]

"Something like xul, what a relief."

yes - in fact, one of the ports of pyjamas to the desktop actually uses xulrunner. the xulrunner port is the most stable of the three: the MSHTML one has a couple of keyboard-event-related niggles, and the webkit one, one of the developers is being a bit of a twat. if you're interested in the pyxpcomext glue - in how pyjamas desktop joins up with xulrunner, or if you're interested in writing your own declarative pyxpcomext application, here's an article which explains it:

http://pyxpcomext.mozdev.org/no_wrap/tutorials/hulahop/xp...

this is a quite different - and i believe much simpler - approach than "standard" pyxcpomext apps, where you have to "register" your app with the mozilla xpcom infrastructure, then create a xulrunner shellscript wrapper (based on run-mozilla.sh) in order to start up the app - it's just such a pain (but has distinct advantages).

personally i much prefer to start from the python prompt, and the article above explains how to do exactly that.

Pyjamas: writing AJAX applications in Python

Posted Dec 8, 2011 7:16 UTC (Thu) by AvdN (guest, #81737) [Link]

I am not sure where the original 23K comes from. The example/timesheet directory has 1504 lines in .py files.
The resulting .js closer to 5K lines and the .html 42K for each of the supported browsers. But the latter two are IMHO non-important.

Pyjamas: writing AJAX applications in Python

Posted Nov 14, 2009 12:22 UTC (Sat) by pm101 (guest, #3011) [Link] (2 responses)

GWT is great, but I hate Java, so I spent a while messing with Pyjamas today. It is not ready for primetime. I'll give an example. Like GWT, the documentation is in a book reader ("bookreader" means it is an AJAX reimplementation of static HTML -- so you get static pages, but search engines can't index it, you can't easily download it for off-line viewing, and the interface to the web browser is off in little ways). I tried viewing the docs at work. It wasn't compatible with the old Firefox we had there. I tried viewing them in the latest Opera at home. No go on the compatibility front. Finally, I used a recent Firefox, and it worked.

It brags that it cut GWT's 80,000 lines to 8,000 lines, but it appears to have lost a lot of the functionality of GWT in the process. Part of this is that it is a thinner layer than GWT, but a big part of it is that it cut important things.

It's a pity. Pyjamas done right (which may happen in a year or two) would be sweet. I'd love to replace JavaScript with a language that doesn't suck.

Pyjamas: writing AJAX applications in Python

Posted Nov 15, 2009 19:02 UTC (Sun) by lkcl (guest, #60496) [Link] (1 responses)

"Like GWT, the documentation is in a book reader ("bookreader" means it is an AJAX reimplementation of static HTML -- so you get static pages, but search engines can't index it"

this is a standard well-known, well-understood problem associated with ALL ajax applications. as an AJAX developer, you will know this, and will have catered for it by providing HTML "alternative" pages, usually enclosed in < noscript > tags, along with a "redirect" to the AJAX equivalent so that any search engine people who see the "redirect" can see that the HTML "static" page looks identical to the AJAX equivalent, and don't cut your site off from the search engine. this problem has NOTHING to do with pyjamas, and EVERYTHING to do with AJAX. regardless of the AJAX framework chosen, be it GWT, DoJo, extjs, prototype - whatever: you WILL have to deal with this issue, period.

"you can't easily download it for off-line viewing"

it's a demo app, created by a free software developer (me), unpaid, in their free time - what do you expect, miracles? if you want me to create a special "PDF" button so you can download the same material for your convenience, *pay me*!

honestly, some people :)

"It wasn't compatible with the old Firefox we had there. I tried viewing them in the latest Opera at home. No go on the compatibility front"

where's the bugreport? did you raise a bugreport?

"It brags that it cut GWT's 80,000 lines to 8,000 lines, but it appears to have lost a lot of the functionality of GWT in the process. Part of this is that it is a thinner layer than GWT, but a big part of it is that it cut important things".

and? so? pyjamas does what the developers who CONTRIBUTE to it want it to do. if you don't like what it provides, *contribute*, don't complain. you can contribute in two ways to this unpaid and community-driven project: 1) code 2) money.

80,000 lines of code means that a team of engineers working round-the-clock, full-time, is required, and those engineers have to be paid money. a large corporation like google can sink large amounts of money into GWT.

by contrast, pyjamas is a community-driven effort.

"Pyjamas done right (which may happen in a year or two) would be sweet."

1) complaining and stating that pyjamas "isn't right" just because it doesn't do everything YOU want it do doesn't help, and is factually incorrect. pyjamas is quotes right quotes for those people who choose to work with it.

2) if it's "not done right", provide some code, or some money, to make that happen. otherwise, please do stop accusing the developers of having "done it wrong", and be grateful for what you get.

Pyjamas: writing AJAX applications in Python

Posted Nov 16, 2009 5:01 UTC (Mon) by foom (subscriber, #14868) [Link]

> this is a standard well-known, well-understood problem associated with ALL ajax applications

So, then, why isn't it well-known and well-understood that you should not use an AJAX
application where a standard HTML page is easier to create, more functional for users,
and better in every possible way (such as presenting static documentation)?

> if you want me to create a special "PDF" button so you can download the same material for
> your convenience, *pay me*!

No, I think he wanted a nice, simple, *non-ajax* normal webpage. Then he could've simply used
the browser's save menu option.