A look at Parrot 1.0 [LWN.net]

March 25, 2009

This article was contributed by Nathan Willis

The Parrot project released version 1.0 of its dynamic language interpreting virtual machine last week, marking the culmination of seven years of work. Project leader Allison Randal explains that although end users won't see the benefits yet, 1.0 does mean that Parrot is ready for serious work by language implementers. General developers can also begin to get a feel for what working with Parrot is like using popular languages like Ruby, Lua, Python, and, of course, Perl.

The evolution of Parrot

Parrot originated in 2001 as the planned interpreter for Perl 6, but soon expanded its scope to provide portable compilation and execution for Perl, Python, and any other dynamic language. In the intervening years, the structure of the project solidified — the Parrot team focused on implementing its virtual machine, refining the bytecode format, assembly language, instruction formats, and other core components, while separate teams focused on implementing the various languages, albeit working closely with the core Parrot developers.

The primary target for 1.0 was to have a stable platform ready for language implementers to write to, and a robust set of compiler tools suitable for any dynamic language. The 1.4 release, tentatively set for this July, will target general developers, and next January's 2.0 should be ready for production systems.

The promise of Parrot is tantalizing: rather than separate runtimes for Perl, Python, Ruby, and every other language, a single virtual machine that can compile each of them down to the same instruction set and run them. That opens the possibility of applications that incorporate code and call libraries written in multiple languages. "A big part of development these days isn't rolling everything from scratch, it's combining existing libraries to build your product or service," Randal said. "Access to multiple languages expands your available resources, without making you learn the syntax of a new language. It's also an advantage for new languages, because they can use the libraries from other existing languages and get a good jump-start."

The Parrot VM itself is register-based, which the project says better mirrors the design of underlying CPU hardware and thus permits compilation to more efficient native machine language than the stack-based VMs used for Java and .Net. It provides separate registers for integers, strings, floating-point numbers, and "polymorphic containers" (PMCs; an abstract type allowing language-specific custom use), and performs garbage collection. Parrot can directly execute code in its own native Parrot Bytecode (PBC) format, and uses just-in-time compilation to run programs written in higher-level host languages. In addition to PBC, developers and compilers can also generate two higher-level formats: Parrot Assembly (PASM) and Parrot Intermediate Representation (PIR). A fourth format, Parrot Abstract Syntax Tree (PAST), is designed specifically for compiler output. The differences between them, including the level of detail exposed, is documented at the Parrot web site.

Parrot includes a suite of core libraries that implement common data types like arrays, associative arrays, and complex numbers, as well as standard event, I/O, and exception handling. It also features a next-generation regular expression engine called Parser Grammar Engine (PGE). PGE is actually a fully-functional recursive descent parser, which Randal notes makes it a good deal more powerful than a standard regular expression engine, and a bit cleaner and easier to use.

The project plans to keep the core of Parrot light, however, and extend its functionality through libraries running on the dynamic languages that Parrot interprets. Keeping the core as small as possible will make Parrot usable on resource-constrained hardware like mobile devices and embedded systems.

Language experts wanted

The "getting started" documentation includes sample code written in PASM and PIR, but it is the high level language support that interests most developers. The project site maintains a list of active efforts to implement languages for the Parrot VM. As of today, there are 46 projects implementing 36 different languages. Three of the most prominent are Rakudo, the implementation of Perl 6 being developed by the Perl community, Cardinal, an implementation of Ruby, and Pynie, an implementation of Python. Among the rest there is serious work pursuing Lua and Lisp variants, as well as work on novelty languages such as Befunge and LOLCODE. Not all are complete, but Randal said development has accelerated in recent months after the 1.0 release date was announced, and she expects production ready releases of the key languages soon.

Language implementers come from within the Parrot project and from the language communities themselves. As Randal explained it, "we see it as our responsibility as a project to develop the core of the key language implementations, and to actively reach out to the language communities."

1.0 includes a set of parsing utilities called the Parrot Compiler Tools (PCT) to help implement dynamic languages on the Parrot VM. PCT includes the PGE parser, as well as classes to handle the lexical analyzer and compiler front-end, and to create the driver program that Parrot itself will call to run the compiler. Owing to its Perl heritage, PCT uses a subset of Perl 6 called Not Quite Perl (NQP). Developer documentation for NQP and all of the PCT components is available with Parrot 1.0 as well as on the Parrot Developer Wiki.

Parrot packages have been available for many Linux distributions and BSDs for much of its development cycle, but now that it has reached 1.0, Randal expects to see it ship by default in upcoming releases. For now, however, developers and language implementers interested in testing and running Parrot 1.0 can download source code releases from the project's web site or check out a copy from its Subversion repository. Building Parrot requires Perl, a C compiler, and a standard make utility.

Parrot has been a long time in coming, but now that 1.0 is out of the gate, the real work can begin, as the major language projects make their own stable releases and developers start to use the Parrot VM as a runtime environment. Although the technical work continues at full pace, Randal said the project is also pushing forward on the education and outreach front, with a book soon to be published through Onyx Neon Press, and Parrot sessions planned for upcoming open source conferences and workshops as well.

Index entries for this article
GuestArticles	Willis, Nathan

to post comments

A look at Parrot 1.0

Posted Mar 26, 2009 5:45 UTC (Thu) by flewellyn (subscriber, #5047) [Link] (3 responses)

Great!

Now comes the hard part: convincing the Perl, Python, Ruby, Lua, and other language developers to use it for their runtimes.

A look at Parrot 1.0

Posted Mar 26, 2009 17:23 UTC (Thu) by elanthis (guest, #6227) [Link] (1 responses)

Lua will never use it. The entire Parrot design goes against the design goals of the Lua authors -- the entire point of using Lua is less about the actual grammar/syntax and all about its very tiny, very fast, very easy to wrap, ANSI-C runtime.

If you're going to use a massive, powerful runtime, you might as well use a more capable syntax with it.

To be honest, I don't really see the point of mixing 20 different languages in a project anyway. It's enough of a nightmare when you have to mix 2, because finding "multi-lingual" developers these days is not easy (sadly). Most of the languages out there are really so close as to be redundant anyway. One of the things people love about Ruby, for example, is that it can pretty much entirely replace Perl.

On top of that, there's the issue of the totally different "API design" issues. A class/object in Perl might be called foo::bar, in Python it might be Foo.Bar, and in Ruby it might be Gluger. The methods might be named print, Print, put, write, display, etc. You end up having 50 implementations of the same damn thing to do things "the Python way" or "the Ruby way" or "the Perl way." This in turn destroys the possibility of actually mixing components written in the different languages because the HTTP component in Perl might produce and consume totally different objects than what the Python HTTP components are using.

Fixing that issue essentially requires that all these languages end up using a single shared standard library. Which in turn means _completely_ breaking compatibility with application written for the original runtimes.

Parrot as a low-level VM and runtime may be useful. It gives language implementors a set of pre-written and ready-to-use code to get a high performance runtime.

All this talk of mixing languages and sharing components and getting everyone to switch so rainbows can start shooting out of our butts is complete nonsense.

Mixing Languages in a Project

Posted Mar 26, 2009 18:11 UTC (Thu) by pkolloch (subscriber, #21709) [Link]

I agree that I would not by my own choice mix many languages in one project. I also agree that homogenizing naming conventions (so that the language transitions will be next to invisible) is probably not going to happen. Many language communities did not even achieve this kind of homogeneity in their own language ecosystem (e.g. C++).

Yet I strongly disagree that this is not a proper and important use case. Some special purpose libraries are just not written in your favorite language. Being able to integrate them easily is fantastic.

A look at Parrot 1.0

Posted Apr 5, 2009 12:15 UTC (Sun) by dlang (guest, #313) [Link]

all that it will take to accomplish that is to show that language X running on parrot is better (faster, less resources used, fewer bugs, etc)

if that is true you will see conversions (after they get over the NIH syndrome ;-)

for perl it will be a good check against the mainline (there isn't currently any second implementation of a language spec, so with only one implementation, the implementation is the spec)

Why is Parrot better?

Posted Mar 26, 2009 13:23 UTC (Thu) by sergey (guest, #31763) [Link] (3 responses)

The article seems silent on why or even if Parrot is
"better," neither for language developers nor for users.
I'd expect a more detailed comparison with other
"general purpose" VM as well as some indication of why
language X should choose to depend on Parrot.

For example:
* Is it more secure? How bullet-proof is the sandbox?
* Is it faster?
* How good is GC and memory management?
* Will it support object-oriented and functional
programming styles?
* If and how it deals with threads and multiple CPUs?
* (I'm sorry but I have to ask) does it run on certain
proprietary OS?
* Can every language team have a pony when they
switch :-)?

Why is Parrot better?

Posted Mar 27, 2009 20:16 UTC (Fri) by NAR (subscriber, #1313) [Link]

* Does it support hot code loading?
* Does it support runtime tracing?

Why is Parrot better?

Posted Mar 28, 2009 21:50 UTC (Sat) by mogul (subscriber, #3163) [Link]

In particular, I'd love to see a comparison of Parrot with the Mono CLR, which to my eye appears to have all the same benefits and be further along.

Why is Parrot better?

Posted Apr 3, 2009 12:21 UTC (Fri) by muwlgr (guest, #35359) [Link]

Yes, and will it implement tracing compiler similar to modern Javascript implementations (Tracemonkey, V8, SquirrelFish) as well as planned for LuaJIT ?

A look at Parrot 1.0

Posted Mar 26, 2009 13:40 UTC (Thu) by Frej (guest, #4165) [Link] (4 responses)

Remember to seperate the VM from the runtime when sharing between languages.

It's much easier to share the same VM, than runtime. A runtime always fits a particular style of programming and/or languge (JVM,.NET) better. (In this case OO).

But hasn't LLVM already partly delivered what parrot promised?

A look at Parrot 1.0

Posted Mar 27, 2009 4:15 UTC (Fri) by Ze (guest, #54182) [Link] (3 responses)

>>It's much easier to share the same VM, than runtime. A runtime always fits a particular style of programming and/or languge (JVM,.NET) better. (In this case OO).

>>But hasn't LLVM already partly delivered what parrot promised?
I would have preferred to see more work going into LLVM.

Parrot/Perl developers will argue that's it's for statically typed languages and not dynamically typed languages. I'd argue that type discovery (and particularly over the whole program) allows us to to infer the type in the vast majority of cases.

I'd also argue that dynamically typed languages are horrible hacks :p I'd much rather have a compiler automatically test for some errors than rely on them to be produced at runtime.

Static vs. dynamic

Posted Mar 27, 2009 12:17 UTC (Fri) by pboddie (guest, #50784) [Link] (2 responses)

I'd argue that type discovery (and particularly over the whole program) allows us to to infer the type in the vast majority of cases.

It's possible to do whole-program inference, but it's expensive. I think many people see functional programming languages employing type inference and consider that dynamic languages are as suitable for the application of similar techniques, but the scale of dynamicity undermines these techniques: assumptions that hold for languages like ML are gone in languages like Python.

I'd also argue that dynamically typed languages are horrible hacks :p I'd much rather have a compiler automatically test for some errors than rely on them to be produced at runtime.

Tests based on type declarations only cover a certain class of errors, as almost everyone should know by now. Dynamic languages obviously save the programmer from having to specify apparently obvious boilerplate, potentially permitting behaviour that would be awkward to define in many statically typed languages. However, restricting the dynamicity of such languages can lead to useful inference about program characteristics and can reveal errors, too.

I'd be happy to see a variant of Python with some of the more blatant dynamicity removed. For example, dynamic definition of classes has never really been very useful for me, so if that went away and enabled better program analysis, I'd be quite pleased with the compromise.

Static vs. dynamic

Posted Mar 28, 2009 23:43 UTC (Sat) by sergey (guest, #31763) [Link]

> For example, dynamic definition of classes has never really been very useful for me, so if that went away and enabled better program analysis, I'd be quite pleased with the compromise.

I just had to go through the exercise of choosing a Python SOAP client library, and most deserving ones all use dynamically defined classes. The morale is, these dynamic features are there for a reason, please don't touch them.

Static vs. dynamic

Posted Mar 29, 2009 14:41 UTC (Sun) by Ze (guest, #54182) [Link]

>>It's possible to do whole-program inference, but it's expensive. I think many people see functional programming languages employing type inference and consider that dynamic languages are as suitable for the application of similar techniques, but the scale of dynamicity undermines these techniques: assumptions that hold for languages like ML are gone in languages like Python.

A mix of type inference littered with bits of type specification where applicable works quite well in properly designed languages.

There are always going to be languages where it's unsuitable and I'd argue that's a pretty good sign (not a conclusive one though) that the programming language is shit.

Yes whole program compilation is expensive... but so is repeatedly calling an interpreter or a JIT.

>>Tests based on type declarations only cover a certain class of errors, as almost everyone should know by now. Dynamic languages obviously save the programmer from having to specify apparently obvious boilerplate, potentially permitting behaviour that would be awkward to define in many statically typed languages. However, restricting the dynamicity of such languages can lead to useful inference about program characteristics and can reveal errors, too.

Yes tests based upon type declarations only reveal certain classes of errors but they are errors that I as a programmer don't need to worry about. The compiler tests give far better coverage when it comes to those errors than any manual testing strategy that programmers practically implement.

This is where type-inference comes in , when you combine that with selective typing you can get the benefits of both. When you add templates and inheritance to the mix than very few things are hard to implement that are easy in other languages.

People will undoubtedly bring up C++ typing here. The c++0x standard addresses problems with the language standard to a large degree with the 'auto' specifier. A lot of the other errors and the difficulty with them rest at poorly implemented compilers. I've got high hopes here for LLVM.

A look at Parrot 1.0

Posted Apr 4, 2009 18:28 UTC (Sat) by mjcoder (guest, #54432) [Link]

I'd be very interested about a performance comparison between Pynie (Python on Parrot) and IronPython (Python on .NET/mono). But it seems that Pynie is in a very early stage (i.e. nothing useful yet on Google Code).