LWN: Comments on "The first Rakudo Star release"

Wow.

lindahl — Wed, 18 Aug 2010 05:21:36 +0000

Hint: Don't feed trolls. They just grow bigger.

Wow.

nix — Thu, 05 Aug 2010 15:42:18 +0000

In unoptimized systems it often is purely a matter of optimization, because code that's never been optimized often has some really expensive hot paths in it that slow things down a hell of a lot, because there was lots still to implement and getting that working at all was more important than getting the rest fast.

An example: I sped up the userspace daemon used by the entropy key hardware by more than 1/3rd (minimal figure) or more than 95% (maximal figure) last night. It took about an hour. This is not because the ekeyd authors were fools or lazy: it's because they had other things to do than optimization, so the first person to optimize it could get major speed improvements with very little effort and a few algorithmic changes to a couple of hotspots. (If the structure of the code is such that such changes are unnecessarily hard, that *is* an indictment of the original developers. This certainly wasn't true of the ekeyd and I doubt it's true of Rakudo.)

I would *expect* developer prereleases of software to be just like this more often than not.

Congratulations

bronson — Sun, 01 Aug 2010 20:35:30 +0000

You might want to hold your accolades until you actually start using it. :)

Wow.

rjbond3rd — Sun, 01 Aug 2010 06:49:39 +0000

Perl 6 is blowing my mind. It really goes to 11.

Over the years, my ideas about Perl became rigid and limited. But after reading the design docs (Synopses), it's easy to see the Perl 6 goals, the challenges, the trade-offs, the design decisions.

http://perlcabal.org/syn/

These docs really flesh out the rationale. The concepts from functional programming are fascinating.

It's not hard at all to imagine a lean and mean Rakudo down the road. Every major leap in computing has been accompanied by complaints about speed and memory :)

Congratulations

wruppert — Sun, 01 Aug 2010 03:40:16 +0000

Congratulations to the Parrot and Rakudo development teams for delivering a useful product. I've been eager to start playing with Perl6.

"Early Adaptors": R.I.P.

efexis — Sun, 01 Aug 2010 03:23:27 +0000

What's an 'early adaptor'? (please don't make me google it!!)

Wow.

efexis — Sun, 01 Aug 2010 03:21:17 +0000

I want the multi-threadedness that p6 offers (I am correct in assuming this is correct helpful multi-threadedness, rather than what p5 has?) and many other language enhancements of p6, BUT with the agreeable sigils of p5 that makes it a more natural language, eg, in p5, A $sigil on it's own or A $sigil[10] in A [group] IS agreeable. In p6, ARE A @sigil[10] agreeable? No, in p6, A @sigil[10] AREN'T agreeable. I'm assuming that if you can speak perl and especially if you've ever tried to learn another language where agreement is more obvious than it is in English that you can understand what I'm saying. And yes I am aware that it can be changed with grammars, but still... yacky! I love all the naturalnesses of the p5 language, it's what makes me love using it over any other language so much, I love expressive speech, it's sad to see much of that gone in p6 :'-( I do understand it makes it easier for the machine to parse tho.

I do >>love>> all that stuff tho. Not keen on tying ~ strings ~ together ~ with ~ this ~ though, that's a really awkward key to press, much slower than . or + which I would probably just have to overload if I can ever get over how un-agreeable all the @sigils is now... see, it just doesn't sound right! But then I don't like the term 'overload' either, it should be 'override', 'overloading' an operator sounds like a way of testing it to find out at what level it will fail, or have a scottish person show up and say "the operators captain, they cannae take it!" so maybe I'm just way too picky *lol* but no I can forgive that, I understand it means "to load over" not actually "overload", so that are one issue that I is happy to forgive... SEE! You need agreement in language! *lol*

I'll shut up now :-)

Wow.

chromatic — Sat, 31 Jul 2010 18:33:10 +0000

Exactly. Both Patrick and I believe that this is possible and desirable.

Even so, anyone who's happy with the features of Perl 5 is more than welcome to continue to use Perl 5. It's not going anyway any time soon.

For people who want use Perl 5 as if it were Perl 6, Rakudo Star is becoming a more and more useful and usable option, and it'll continue to do so.

Wow.

thepaul — Sat, 31 Jul 2010 17:13:52 +0000

NERDFIGHT! Woo!

Wow.

khc — Sat, 31 Jul 2010 16:39:50 +0000

I guess another way to look at it is, if they find a way to load those new features in perl6 on demand, then perl6 can be substantially smaller and faster (to start up).

Wow.

dskoll — Sat, 31 Jul 2010 14:16:48 +0000

Even your "accurate comparison" shows that Perl 5 is significantly smaller and faster than Rakudo Star.

But here's the thing: You had to add a lot of modules to Perl 5 to get it to be in the same order of magnitude of slowness and bloat as Rakudo Star. But Perl 5 without all those modules is still a very useful tool. It seems that with Perl 6, we have no choice: We get all the features whether or not we'll use them.

In 14+ years of Perl development, I haven't written a single project that uses even one of those Perl 5 modules you loaded for comparison purposes (though I suspect eventually I'll be forced to drag in Moose as more and more CPAN modules require it.) So I think it's you who is gaming the comparison. I'm pretty sure I could find a way to make Perl 5 10x slower and bigger than Rakudo Star, but that doesn't reflect the way people actually use Perl 5.

Wow.

chromatic — Sat, 31 Jul 2010 00:30:56 +0000

Your comparison of the size of an optimized binary produced by an optimizing C compiler to an unoptimized barely-a-binary-at-all produced by an unoptimizing Perl 6 comparison is silly. Ditto your comparison of the memory use. Likewise your comparison of the feature set.

Compare the optimized Perl 5 to the unoptimized Rakudo Star on feature set for a more accurate depiction of the situation.

Wow.

dskoll — Fri, 30 Jul 2010 23:30:29 +0000

I don't understand that last remark. Are you saying Rakudo Star is implemented in Perl 6? And that it's "just" a matter of rewriting it in something else to make it small and fast?

Wow.

chromatic — Fri, 30 Jul 2010 21:35:43 +0000

Perl 5 isn't written primarily in Perl 5.

Wow.

ariveira — Fri, 30 Jul 2010 21:06:29 +0000

I collected this somewhere:

"The most amazing achievement of the computer software industry is its
continuing cancellation of the steady and staggering gains made by the
computer hardware industry."
Henry Petroski

Wow.

dskoll — Fri, 30 Jul 2010 17:38:04 +0000

You are correct in assuming I haven't looked deeply at the code for Rakudo Star. I just downloaded it to "kick the tires".

I wish you luck in your optimization efforts, though I remain doubtful you'll succeed.

Rakudo's been in development since November 2007--to correct your "decade" number.

My "decade" reference refers to Perl 6 itself; the first design notes for Perl 6 came out in 2000, I believe. (Another sign of a project in trouble, IMO, is abandonment of a major line of development [PUGS] in favor of a completely different line.)

Wow.

chromatic — Fri, 30 Jul 2010 16:29:46 +0000

I can imagine an objective observer can tell the difference between the statements of two of the lead developers of Rakudo and Parrot and the statements of someone who, to my knowledge, has never looked at the code. (If you have, I apologize.)

Patrick and I know where time and memory go in Rakudo Star, and as such, we're confident that we can find major optimizations. For example, with some effort we can halve the amount of memory used for objects, which both reduces the memory footprint and the amount of time spent in the garbage collector. Other such optimizations exist; don't assume that we care nothing for them.

(Oh, and Rakudo's been in development since November 2007--to correct your "decade" number.)

Wow.

dskoll — Fri, 30 Jul 2010 15:34:57 +0000

Competitive on what grounds?

Performance and memory consumption.

On saving a couple of megabytes of memory?

Umm... a "couple of megabytes"? I'm talking about 93MB. Our commercial product uses many (sometimes hundreds) of concurrent Perl processes. Even though it's not embedded, 93MB extra overhead per process would kill us. (In perl 5, even though we fork after loading all our modules, it seems that the reference-counting GC method makes copy-on-write inefficient; most memory pages end up getting touched and we see poor memory sharing among processes.)

Me and many others just couldn't care less.

Yes, and that is obvious by the state of Perl 6: Buggy, feature-incomplete and bloated after a decade of development.

The notion that programmer time is far more valuable than performance or memory consumption is usually true, but it is not true when it comes to programming languages. The designers of programming languages must try to make them efficient, because the time they spend doing that is multiplied thousands or millions of times when people use the language. Something that makes Perl a bit faster or smaller has enormous dividends, far more than one particular developer making his or her particular Perl project smaller or faster.

So if your feelings about speed and memory ("couldn't care less") is indicative of the general attitude among Perl 6 developers, then I say it again: This is a doomed project. It may be unpopular to say that in the Perl community, but I bet most objective outsiders would agree with my assessment.

So one could easily turn that argument around: by requiring the use of countless fragile models to still not reach the same level of maintainability and robustness features that Perl 6 has, Perl 5 is just not competitive.

Except that's all in theory. A robust implementation of Perl 6 doesn't exist yet, and we have no idea whether or not the average Perl 6 project will be any more maintainable than the average Perl 5 project.

Wow.

niner — Fri, 30 Jul 2010 15:06:27 +0000

Competitive on what grounds? On saving a couple of megabytes of memory? That's probably very important to you and embedded applications. Me and many others just couldn't care less. We do not save time and money by not needing that 30MB more memory, but by having a language that makes development more productive and code more maintainable. And both will get a huge boost by the step Perl 6 is making. So one could easily turn that argument around: by requiring the use of countless fragile models to still not reach the same level of maintainability and robustness features that Perl 6 has, Perl 5 is just not competitive.

That's what you get by reducing the comparison of complex things like programming languages to one single metric.

Wow.

dskoll — Fri, 30 Jul 2010 14:49:19 +0000

Seriously: what exactly is the point of your statistics showing the surprising results that an early adopters pre-release is not as optimized as the predecessor that has been in used production for over 15 years?

Come on! Eliminating 95% of memory usage and reducing startup time by a factor of 500 is merely a matter of optimization?

I recently completely rewrote a C program we were using in-house to use less memory. I reduced the memory to 1/3rd of the original, and that was with a large amount of effort and creative hacks. There is simply no way to "optimize away" the amount of bloat needed to make Perl 6 competitive with Perl 5.

After ten years of development, if the best that can be achieved is a feature-incomplete, buggy, bloated alpha... that's a sign that the project has gone off the rails.

Wow.

dskoll — Fri, 30 Jul 2010 14:45:27 +0000

Moderately faster, I can believe, but certainly not a smaller memory footprint. My experience is that Perl's memory footprint has grown at a small rate with each new major release.

And Perl certainly hasn't shrunk by over 95% nor speeded up by a factor of 25, which is the kind of improvement needed for Perl 6 to be competitive.

Wow.

niner — Fri, 30 Jul 2010 12:23:00 +0000

Easy: Perl 5 has gotten faster over time and reduced memory footprint in many cases.

Wow.

niner — Fri, 30 Jul 2010 12:13:33 +0000

So now we are suddenly at the size of the binary?

Wow.

dskoll — Fri, 30 Jul 2010 12:01:37 +0000

Well the perl5 binary did have 17 (!!) years of time for optimization.

But in those 17 years, the perl5 binary has grown, not gotten smaller. (I'm not sure about speed. I suspect important parts such as the regex engine have been made faster over time.)

Wow.

dskoll — Fri, 30 Jul 2010 12:00:11 +0000

Just because these features are bloated today doesn't at all mean that it must always be so.

This is true in theory. Now please name one major software project that has become significantly smaller and faster over time. For bonus points, name one that did so while adding features (which is what Rakudo Star needs to do before it can go into production.)

I believe the improvements in speed and memory required to make Perl 6 competitive with Perl 5 are unprecedented in the history of computer science. I know the Perl 6 authors mean well and have been working hard, but I fear they've developed another Chandler.

Wow.

nix — Fri, 30 Jul 2010 10:02:06 +0000

We also have 17 years of Moore's Law to catch up with :)

"Early Adaptors": R.I.P.

exadon — Fri, 30 Jul 2010 07:37:47 +0000

Another term destroyed by reckless marketing abuse.

Wow.

niner — Fri, 30 Jul 2010 07:22:01 +0000

Well the perl5 binary did have 17 (!!) years of time for optimization. If Rakudo were even in the same category now it would seriously raise the question about what Perl 5 developers have been doing in that time.

Wow.

pmichaud — Fri, 30 Jul 2010 06:42:35 +0000

You've nailed it. In Perl 5, the bloat is optional. In Perl 6, it appears to be built-in.

...except that most of Rakudo's existing runtime startup and extra dynamic memory costs for these additional features can be eliminated altogether, whereas that will be much more difficult to do in Perl 5.

Just because these features are bloated today doesn't at all mean that it must always be so.

Wow.

chromatic — Fri, 30 Jul 2010 06:00:50 +0000

We're working on ways to avoid loading and initializing features you don't use. I have strong confidence that we can achieve that for deployable applications. I have some ideas on how to accomplish that for one-off applications.

Wow.

dskoll — Fri, 30 Jul 2010 03:45:16 +0000

To really have a fair comparison, one needs to compare the memory footprint and startup time of Rakudo versus Perl 5 *with* all of Moose, Devel::Declare, Regexp::Grammars, Contextual::Return, Data::Dumper, etc. loaded and available. Otherwise you're not really comparing equivalent capabilities.

You've nailed it. In Perl 5, the bloat is optional. In Perl 6, it appears to be built-in.

I've used Perl since 1996, and have been developing both free and commercial software with it intensively since 2002, and I'm utterly dismayed by the direction it is taking. IMO, Perl 6 is a textbook example of Second System Syndrome.

Even Perl 5 is suffering; CPAN is balkanizing into groups of modules with irreconcilable dependencies (eg. anything that wants Moose vs. Rose::DB that will never use Moose) that simply multiply the bloat.

Well, I'm hedging my bets and looking at other languages.

Wow.

mikov — Fri, 30 Jul 2010 03:45:16 +0000

Yes, but what if you don't need all those capabilities? No more cheap quick & dirty scripts that we all love so much.

It is starting to look like Java in terms of startup speed and footprint, and that is saying something...

Unless the situation changes drastically, no way you can use Perl 6 for lightweight system scripting, etc.

Wow.

pmichaud — Fri, 30 Jul 2010 03:20:48 +0000

All of chromatic's responses apply, but the comparisons above are flawed at even a more basic level.

I suspect Rakudo may still come out slower and fatter than an "equivalent Perl 5", but likely much less so than these naive comparisons seem to suggest. And we're still very early in the "optimization" aspects of the development process for Rakudo.

Wow.

chromatic — Fri, 30 Jul 2010 01:17:49 +0000

Those are difficult estimates. I could see worthwhile targets being 2.0 for each of them, though I'd like to get startup time to 1.0 or less. It's possible to get memory consumption below 1.0, but that'll require more work in Parrot.

Wow.

dskoll — Fri, 30 Jul 2010 00:43:07 +0000

Let's assume that the final Perl 6 release is N times larger than Perl 5, starts up M times as slowly, and uses P times as much memory after initialization. What's your best guess as to the values of N, M and P in the first production release? Huge bonus if they're less than 1.0 :)

Wow.

chromatic — Fri, 30 Jul 2010 00:27:03 +0000

That Perl 6 executable is a thin C wrapper around uncompressed (and unoptimized) Parrot bytecode. It doesn't compare directly to the Perl 5 binary at all.

Wow.

dskoll — Fri, 30 Jul 2010 00:16:53 +0000

OK, how about this:

$ size ./perl6 ./install/lib/libparrot.so.2.6.0 
   text    data     bss     dec   hex filename
14219892    400       8 1422030   d8fc0c ./perl6
1535297  196868     796 1732961   1a7161 .../libparrot.so.2.6.0

$ size /usr/bin/perl
   text    data     bss     dec     hex filename
1245399    6744     344 1252487  131c87 /usr/bin/perl

(I included libparrot.so because it's not a standard system library.) So perl5 is about 1.2MB of program text, whereas perl6 weighs in at 15MB and still needs to dynamically create things that couldn't be put directly in the binary?

I repeat the title. Wow.

Wow.

dskoll — Fri, 30 Jul 2010 00:09:54 +0000

All your points are taken, and yet... I have never heard of a piece of software whose memory consumption dropped by 96% and startup time by 98% from the first alpha release to the first production release. That's what it would take to match the perl5 binary.

I certainly hope the Perl 6 developers are smart enough to pull off such a feat.

Wow.

pmichaud — Thu, 29 Jul 2010 20:59:42 +0000

Parrot doesn't yet have a robust mechanism for keeping built-in class and data structures directly in the perl6 binary, so they end up being dynamically constructed at program startup.

This is the primary reason for Rakudo's slow startup; and like most things in Rakudo and Parrot, the development teams are actively working on fixes for it.

Wow.

b7j0c — Thu, 29 Jul 2010 20:13:09 +0000

its well understood that rakudo is not yet ready for deployment...it is slow and uses gobs of memory. all of this is disclaimed in the docs, no one is claiming this is ready to be dropped in where perl5 or python is used today

rakudo is about getting people using perl6...playing with the features, porting modules...

in that regard this is a huge success. its safe to say 99% of perl5 users probably don't know 1% of the cool features in the core perl6 language...its important that these people know that perl6 is no longer just a internet meme about vaporware...