User: Password:
|
|
Subscribe / Log in / New account

Vincent: Perl 5.16 and Beyond

From:  Jesse Vincent <jesse-AT-fsck.com>
To:  Perl5 Porters <perl5-porters-AT-perl.org>
Subject:  Perl 5.16 and Beyond.
Date:  Mon, 12 Sep 2011 12:28:47 -0400
Message-ID:  <09FF9706-C7B3-44E6-B108-E6790AEA6497@fsck.com>
Archive-link:  Article


[ What follows is the prose version of the talk I gave at YAPC::NA, OSCON
and YAPC::EU this summer.  I'll be giving a similar talk at YAPC::Asia. If
you prefer reading big sentence fragments on slides, 
http://www.slideshare.net/obrajesse/perl-516-and-beyond is my slide deck 
from YAPC::EU]

Porters,

Over the last two years, I've been thinking a lot about where Perl 5 is
headed. Well, that's not quite true. I've been thinking a lot about where
Perl 5 is headed since way back in 2005 or 2006 when I was the Perl 6
project manager and it become clear to me that Ponie wasn't likely a horse
I'd want to bet Perl 5 on.

As a community, we've done an absolutely stellar job of getting Perl 5 and
p5p back into a reasonable groove. We know how to do releases. More
individuals have released Perl 5 in the last 18 months as in the first 16
years Perl existed. It shouldn't really surprise anybody that knowing that
work you do will be released soon is a huge motivating factor. Releasing
frequently has dramatically increased Perl 5's rate of change. That's both
good and bad.

I'm thrilled that Perl development is incredibly vibrant. 5.14 is a
dramatically better Perl than 5.10 was, thanks to the work of approximately
260 porters. At the same time, I've made (or failed to make) some design
choices that, in retrospect, I somewhat regret. There are places where Perl
5 is now more baroque than it was before. There have been changes to Perl 5
that have broken backward compatibility where we might have been better
served not doing so. These mistakes are entirely my fault. If I had a
do-over, I'd make different mistakes..er. decisions.

Various members of the community have asked me about my vision for the
future of Perl 5. I was initially very, very hesitant to make any bold
statements about the language, since "we're going to put out a development
release every month and a new stable release every year" felt rather...bold
itself.

Putting out releases is hard work. At this point, it's hard work we have
well in hand.

Much of my thinking about the future of Perl 5 stems from the following
principles:

	* New versions of Perl 5 should not break your existing software
	* Backward compatibility must not stop Perl 5 from evolving

Pay particular attention to "should" and "must" there. It is critically
important that we not alienate the people, communities and companies who
have invested their time and money in Perl 5.  Pulling the rug out from
under them isn't good for them and isn't good for us. Wherever possible, we
need to preserve backward compatibility with earlier versions of Perl 5. At
the same time, it could be argued that _any_ change to Perl 5 breaks
backward compatibility. ("But I was depending on that segfault!") If Perl 5
is going to continue to flourish, we're going to need to be able to change
the language.

* Why we deprecate things

Perl 5 is not Latin. It is a living language, still borrowing liberally
from...just about everything. Sometimes we borrow the wrong
things. Sometimes we borrow things and use them wrong. Sometimes we invent
things and later wish we hadn't. Perl has always been something of a
packrat, but we're in danger of hitting the point of being diagnosed with a
pathological hoarding problem. We need to get better at fixing problems and
moving forward without hurting old code.

* How we handle language changes

We've always had a messy relationship with backward compatibility. Because
we don't have a spec for the language and we have a single implementation,
side-effects bugs and ill-considered accidents have become unchangeable
parts of the language.

For a project of Perl's age, scale and diversity, we've been astonishingly
conservative about how we make incompatible changes. It's not enough. Every
release breaks running code. The more the language evolves, the more legacy
code will break.

Standing still is not an option. Perl's internals, syntax and semantics
have seen some much-needed improvements in the past few years. There are
many additional changes we can't make because they may damage too much
legacy code.

To date, version declarations in code have been intended as a marker for
"minimum acceptable version of perl" -- They've also been used to enable
new language features or semantics changes.  Many _other_ new changes in
Perl are turned on by default without the need for any declaration.
Typically, this is because we don't believe those changes will harm earlier
code, though it's also the case that conditionally enabling certain changes
would be difficult.

Perl 5 doesn't require that a developer declare which version of Perl 5 a
program or library was written to run under. Historically, our default has
been "no declaration means latest, sort of."

That changes from here on in. Code that does not declare a 'use v5.16;' or
later will be presumed to have been written to target v5.14.

If there is no "use v5.xx" line at the top of the code, the runtime should
act as it did on v5.14 without a use v5.14 line.

We'll need a nice compact way to declare what version of Perl you want from
the commandline for one-liners too.

Along with this, there's a change to how we'll be removing deprecated
features from the core.

If a core feature (syntactic or semantic) is removed, a "use v5.xx"
declaration for a earlier version of Perl 5 should re-enable the feature.

This will be difficult. I fully expect people to start throwing pies,
tomatoes or bricks when I say this. I will not require that the
forward-ported implementation of the old feature have performance parity
with the old implementation. In certain circumstances, I _will_ make
exceptions to the mandatory-forward-port rule. Security fixes are one
obvious case. I know we'll run into other cases.

It is my strong preference that features granted R&R (removal and
reinstatement) be implemented as modules, so as not to bloat the runtime
when they're not needed. This isn't a pipe dream.  Classic::Perl already
does this for a few features removed in 5.10 and 5.12;

If it's not possible to reinstate a feature we've removed with existing
APIs, we'll need to look at the cost of removing the feature vs simply
disabling it in the presence of a new-enough v5.xx declaration.

* How quickly can we deprecate things

When I seized power in late 2009, I set us up with a new policy for
deprecation timing. Anything we deprecated needed to warn in version .x
that it would be removed in version .x+1. At the same time, I put us on a
track to ship a new release of Perl 5 each spring. I'm incredibly pleased
to see that multiple vendors have already incorporated Perl 5.12 into their
stable releases

There's a reasonable chance that some vendors on a slightly longer release
cycle may never ship a given major version of Perl, meaning that our
carefully-crafted deprecation warnings will never be seen.

If we manage to implement the R&R policy for deprecated features, we
have little to worry about.  Code that declares "use v5.14" should continue
to function just as it always has, but for code that declares 'use v5.16',
we can make changes without a deprecation cycle.

For cases where we _can't_ implement R&R, I think we need to move to a
two year deprecation cycle, so as to have as minimal an impact as possible
on users who are upgrading.

* Language Modularization

There's a lot of stuff that ships as part of "Perl 5 the distribution"
that's not part of "Perl 5 the language" - Over the years, we've done an
increasingly good job unwinding many dark corners of what we affectionately
refer to as "blead" into CPAN modules. Most of those items have been add-on
scripts and tools that have been part of the distribution but weren't part
of the language.  Most of those modules continue to ship as part of the
distribution and may well do so for many years to come. When I talk about
modularization, I'm not proposing that we stop shipping the newly
modularized code as part of the traditional core distribution.

Over the past few years, a number of people have spent significant effort
to make certain parts of the language runtime a good deal more pluggable. I
expect that we have a good amount of work to do before the core is flexible
enough for us to fully implement this plan.

It's time for us to start extracting parts of what has traditionally been
considered the "language" part of Perl 5 into CPANable modules. To do this
successfully, it is imperative that _nothing_ appear out of the ordinary to
code that expects to use those features. If code doesn't declare use v5.16,
it should still get the 5.14ish environment it would expect.

At the same time, it should be possible for a future Perl to declare that
certain previously-integral features are optional and run-time
loadable/unloadable. There are many situations where Perl is used that
don't need a whole set of the blades on our swiss-army chainsaw.  Forcing
their inclusion makes Perl a less viable candidate for a wide variety of
applications.

Once language features are modularized, it also becomes _possible_ to
maintain and improve them without requiring a full Perl upgrade.

Making this work for some things, like grammar changes, may take herculean
effort but will invariably result in a more robust and flexible Perl.

I don't know what we'll extract or when we'll extract it, but there are a
number of language features that seem like they might make sense to make
pluggable: Formats, SysV IPC functions, Socket IO functions, Unix user
information functions, Unix network information functions and Process and
process group functions. Jesse Luehrs has already built us a first version
of an extraction, modularization and replacement of smartmatch.

It should be clear from this list that I don't intend for us to kill,
banish or exile these functions from the traditional Perl
distribution. They're a useful and important part of our language and our
culture. But they're not all needed or even desired everywhere we'd want
Perl 5 to run. If we can modularize them in a way that doesn't negatively
impact performance, Perl 5's manipulexity gets better without damaging its
whipitupitude. The increased flexibility of the core should actually
improve our whipitupitude at the same time.

* What should ship in the Perl core

When I say "traditional Perl distribution", I'm talking about the
conglomeration of "language" and "toolkit" we currently ship. In 5.001,
that totalled up to 59 packages. In 5.005, it grew to 176.  By 5.008, it
was 338 packages. 5.10 saw 541 packages. 5.12 ballooned to 625
packages. 5.14 was the most modest growth we've had in many versions. It
ships 655 packages. Many of these are, of course, part of a smaller number
of distributions.

The bar for getting a distribution into blead has varied over time. The
"point" of the core distribution has often been up for debate. Is it an SDK
for software written in Perl 5? If so, then we ought to be shipping what we
currently consider to be the best-practice modules for building software in
Perl. Is it "stuff we've always shipped with the Perl core?"  If so, we're
doomed to a life in a house full of stacks of old line-printer paper and
TK50 cartridges.

Right now, what we include in the core distribution is a somewhat eclectic
set of modules. They're not a particularly good SDK. They're not a
particularly good sysadmin toolkit. They do, however, include an excellent
set of modules to bootstrap getting more modules.

During my tenure, the guiding principle for new distributions in the core
has been "does it help us bootstrap CPAN module installation or to test the
core?"

In this day and age, most users get Perl from their operating system vendor
or a third-party packager. Those vendors often look to us to decide what to
ship. And they often ignore us. That's 100% ok. There are vendors who
choose not to ship Perl's documentation as part of their "perl"
package. There are vendors who have considered removing Perl from their
default installation because it's "too big" -- They've talked about
splitting the distribution up according to their own logic or sometimes of
removing it entirely. We've done some amazing work paring down the
installed footprint. So far, I think we've avoided getting the axe.

And then there's the maintenance issue. It takes a lot of work for us to
keep up to date with all the modules we include in the core. We do a pretty
good job, but it takes a lot out of us.

It's time we made some more work for ourselves. We will continue to ship a
distribution of Perl that contains roughly what we've always shipped, but
I'd like it to be our secondary product.

The primary product will be "Perl the language" with the minimal set of
modules needed to test the core, offer all the language features we intend
to offer (Encode, for example) and to bootstrap the installation of new
modules from CPAN. While I intend for us to ship the R&R features in
the core distribution, it shouldn't actually be necessary.

Since I became involved in the Perl community, there have been constant
requests for various "SDKs" for various uses of Perl. The state of the art
in how to do such a thing is better than it was, but isn't really all that
useful at this point. It should be possible to take a "Perl the language"
distribution, drop some modules into a directory, tar/zip it up and hand it
off to your user community. And that's how we should be able to build the
"traditional" Perl distribution.

* Documentation changes

Perl's documentation is. Well, it is many things. Current, well-edited,
complete and completely accurate are four things it is not. We've been
getting a little better at splitting out "introduction for newcomers"
documentation from "canonical reference" documentation.

Our introduction documentation has always been opinionated. The particular
opinions have varied by author, document and phase of the moon. That's
ok. Introductory documentation _should_ be opinionated. Introductory
documentation should recommend "at least one good way to do it" and point
the curious reader who wants to know more to canonical reference
documentation.  Canonical reference documentation should try to actually
spell out how things work with a minimum of opinion.

As we modularize features, we'll need to consider how we reorganize the
docs so that they are maximally useful to users who should neither need nor
want to know about how Perl's internals work.

We need a new document, built from previous perldeltas that succinctly
describes what, other than bug fixes, came or went in each version of Perl
5. Porters should be discouraged from adding or removing a feature without
updating this document (and other documentation).

* Tests

In order to understand how well we're doing on back-compat as we move
forward, we're going to need to be able to run a new Perl against older
versions of the test suite.  One step in that direction would be to be able
to run the test suite against an installed Perl.

* TL;DR

	- New versions of Perl 5 should not break your existing software
	- Backward compatibility must not stop Perl 5 from evolving
	- From 'use v5.16' forward, Perl should start treating 'use v5.x'
	  statements as "try to give me a Perl that looks like v5.x" rather
	  than "give me at least v5.x" 
	- We're awesome at modules. Where possible, we should be
	  modularizing core features. 


Best,

Jesse

(Log in to post comments)

Vincent: Perl 5.16 and Beyond

Posted Sep 13, 2011 14:19 UTC (Tue) by imgx64 (guest, #78590) [Link]

Probably a silly question, but why is Perl 5.x getting backwards-incompatible changes? I thought this was the purpose of Perl 6.

Vincent: Perl 5.16 and Beyond

Posted Sep 13, 2011 14:21 UTC (Tue) by danieldk (subscriber, #27876) [Link]

Doesn't pretty much everyone agree now that Perl 5 and Perl 6 are separate languages?

Vincent: Perl 5.16 and Beyond

Posted Sep 13, 2011 15:39 UTC (Tue) by dskoll (subscriber, #1630) [Link]

Because Perl 5 is what everybody actually uses. It's what needs to evolve to stay relevant.

To paraphrase the original article: "Perl 6 is. Well, it is many things. Useful, fast, stable (in the API-stability sense) and a viable replacement for Perl 5 are four things it is not."

Vincent: Perl 5.16 and Beyond

Posted Sep 13, 2011 16:21 UTC (Tue) by b7j0c (guest, #27559) [Link]

right. perl6 will be all of those things at some point, or not. i'm not worried. if all perl6 accomplishes is to inform perl5 of cool new features, then i consider it a success.

Vincent: Perl 5.16 and Beyond

Posted Sep 13, 2011 18:22 UTC (Tue) by dskoll (subscriber, #1630) [Link]

I was worried about Perl 6 in the past because I thought it would suck developer time away from Perl 5 and lead the Perl community down a dead-end. I am no longer worried now that I see the excellent work still being done by the Perl 5 community.

I'm confident that Perl 5 will continue to thrive and be developed by excellent people like Jesse Vincent. As for Perl 6's future, I believe it's as bright as PHP 6's future, but that's just my opinion... we'll have to wait and see.

Vincent: Perl 5.16 and Beyond

Posted Sep 13, 2011 16:13 UTC (Tue) by b7j0c (guest, #27559) [Link]

i love the modern perl renaissance. they're making regular meaningful releases. cpan is as great as ever. new meta-packages like moose and mojolicious are keeping me interested.

perl6 will either arrive or not. i've kept up with whats happening in perl6 land and i'm treating it as a player-to-be-named-later. until then, i'm glad that the perl5 team is cherrypicking the best-of and migrating it in to perl5

Vincent: Perl 5.16 and Beyond

Posted Sep 13, 2011 16:23 UTC (Tue) by alankila (guest, #47141) [Link]

It will be rather comical if perl6 ends up abandoned without ever seeing any real use, and development moves back to perl5. I half expect this to happen, though. Still, perl6 will prove to have been a rather costly mistake^Wlearning experience.

What is good is that perl5 will finally have a strategy for future evolution. The backwards compatibility has been imposed way too harshly for way too long, and has concerned even features that are widely seen as mistakes to be corrected. To name an example, Perl thinks you refer to package variable $name::s when you type "$name's dog". (The :: package separator can be written as ' although nobody does it.)

Vincent: Perl 5.16 and Beyond

Posted Sep 13, 2011 18:11 UTC (Tue) by b7j0c (guest, #27559) [Link]

well i don't know about development moving "back" to perl5 since the communities have virtually no overlap. my guess is if perl6 ever terminated, the interested parties would probably move on to some other prototypical thing

in any case, perl6 is doing fine. both parrot and rakudo are making regular releases...they're well on their way toward their goal. the question for developers is if the perl6 team goals coincide with their own.

Vincent: Perl 5.16 and Beyond

Posted Sep 14, 2011 2:45 UTC (Wed) by HelloWorld (guest, #56129) [Link]

> To name an example, Perl thinks you refer to package variable $name::s when you type "$name's dog".
Well, for what it's worth, string interpolation is pointless to begin with as $name."'s dog" works perfectly fine.

Vincent: Perl 5.16 and Beyond

Posted Sep 14, 2011 3:15 UTC (Wed) by jzbiciak (subscriber, #5246) [Link]

Well, for what it's worth, string interpolation is pointless to begin with as $name."'s dog" works perfectly fine.

Now try that in a lengthy "here document." Sure, interpolation is a waste on short strings, but it's quite handy when you have a larger bit of boiler plate.

At least the syntax ${names}'s works around the "feature."

Vincent: Perl 5.16 and Beyond

Posted Sep 14, 2011 15:26 UTC (Wed) by rfunk (subscriber, #4054) [Link]

Why would I write $name."'s dog" when I can write the much more readable "${name}'s dog" ?
(And last I heard, Perl's concatenation was slightly slower than interpolation anyway, though I care less about that than about concatenation being less readable than interpolation.)

Vincent: Perl 5.16 and Beyond

Posted Sep 14, 2011 15:38 UTC (Wed) by HelloWorld (guest, #56129) [Link]

> Why would I write $name."'s dog" when I can write the much more readable "${name}'s dog" ?
Which of these is more readable is entirely subjective, so you don't have much of a point.

Vincent: Perl 5.16 and Beyond

Posted Sep 14, 2011 15:51 UTC (Wed) by rfunk (subscriber, #4054) [Link]

Oh yeah, I forgot, we're talking about Perl here, so nobody cares about readability.

Vincent: Perl 5.16 and Beyond

Posted Sep 14, 2011 18:43 UTC (Wed) by HelloWorld (guest, #56129) [Link]

Sorry, but you don't make any sense whatsoever.

Vincent: Perl 5.16 and Beyond

Posted Sep 14, 2011 17:36 UTC (Wed) by k8to (subscriber, #15413) [Link]

Readability is not subjective, because there are testable human norms which exist, and readability is generally concerned with the overall result.

Vincent: Perl 5.16 and Beyond

Posted Sep 14, 2011 23:40 UTC (Wed) by alankila (guest, #47141) [Link]

This line of argument misses the point. What I dislike is two competing ways to write the same thing, the other which is not even used but still isn't removed. I used this to illustrate the fact that Perl is backwards compatible to a fault.

Vincent: Perl 5.16 and Beyond

Posted Sep 15, 2011 0:19 UTC (Thu) by dlang (subscriber, #313) [Link]

yes perl maintains backwards compatibility and many ways to do the same thing.

for some people this is a problem

for other people this is part of the great power of perl and why they love to use it.

but in any case, this shouldn't be news to anyone.

Vincent: Perl 5.16 and Beyond

Posted Sep 13, 2011 17:49 UTC (Tue) by nix (subscriber, #2304) [Link]

* New versions of Perl 5 should not break your existing software
Only whenever even minor updates come out, I have to rebuild every single program that embeds Perl, because they embed the path to libperl.so into their RPATH, and that includes not only a version number but a minor version, e.g. /usr/lib/perl5/5.14.1/...

So in practice even minor upgrades involve hacking the source to fake the version number to whatever I was using before, and bigger upgrades (e.g. from 5.12 to 5.14) mean an enforced mass of rebuilding and pretty much prevent me from doing such updates if there is any hope that a distributor will get around to it in the foreseeable future.

Perhaps I'm missing something, but I don't think so.

Vincent: Perl 5.16 and Beyond

Posted Sep 13, 2011 19:52 UTC (Tue) by FranTaylor (guest, #80190) [Link]

Do you know about chrpath? Use it to edit the runpath information in an ELF binary.

Vincent: Perl 5.16 and Beyond

Posted Sep 14, 2011 1:36 UTC (Wed) by nix (subscriber, #2304) [Link]

Yes... but having to do that is disgusting.

Vincent: Perl 5.16 and Beyond

Posted Sep 14, 2011 11:28 UTC (Wed) by FranTaylor (guest, #80190) [Link]

And rebuilding from scratch isn't more disgusting? For goodness sake, you are just editing a file. It's no different from editing LD_LIBRARY_PATH in a wrapper script.

Seriously, put it in an upgrade script and move on! There are plenty of things more worthy of your disgust than this.

Vincent: Perl 5.16 and Beyond

Posted Sep 14, 2011 17:39 UTC (Wed) by k8to (subscriber, #15413) [Link]

He's pointing out that embedding perl is broken, essentially.

You're saying "but it's not *that* broken".

I don't find this argument very compelling.

Vincent: Perl 5.16 and Beyond

Posted Sep 14, 2011 21:52 UTC (Wed) by nix (subscriber, #2304) [Link]

Well, it's broken by default, but on further analysis virtually every distro sets -Darchlib=/usr/lib/perl5/something/without/a/version at configure time, ameliorating this problem somewhat (but making it impossible to install two different versions of Perl in parallel).

Vincent: Perl 5.16 and Beyond

Posted Sep 13, 2011 21:20 UTC (Tue) by lkundrak (subscriber, #43452) [Link]

My distribution (el6) has it in /usr/lib/perl5/CORE/libperl.so and I can't immediately spon any trickery in the SPEC file that would move it there, so maybe some standard Configure flag influences it?

Vincent: Perl 5.16 and Beyond

Posted Sep 14, 2011 1:38 UTC (Wed) by nix (subscriber, #2304) [Link]

None I've noticed. I'll have to look again.

It's in the archlibexpdir, which is on my system (using unmodified upstream perl) /usr/lib/perl5/5.14.1/x86_64-linux-thread-multi/CORE. Note that version number.

/usr/lib/perl5/CORE seems like a bad place for it: surely you need an architecture triplet of some kind in there?

Vincent: Perl 5.16 and Beyond

Posted Sep 14, 2011 3:35 UTC (Wed) by mattdm (subscriber, #18) [Link]

My distribution (el6) has it in /usr/lib/perl5/CORE/libperl.so and I can't immediately spon any trickery in the SPEC file that would move it there, so maybe some standard Configure flag influences it?

perl-USE_MM_LD_RUN_PATH.patch?

Backward compatibility

Posted Sep 13, 2011 20:14 UTC (Tue) by thomas.poulsen (subscriber, #22480) [Link]

The Lillypond language has long used an explicit version declaration in scripts.
But rather than supporting old versions indefinitely, old scripts can be converted using a utility function.

They also have very beautiful release notes

Vincent: Perl 5.16 and Beyond

Posted Sep 13, 2011 21:23 UTC (Tue) by lkundrak (subscriber, #43452) [Link]

A couple of pictures visualizing Perl 5 GIT history here: http://v3.sk/~lkundrak/perl5/

(There's no better way to share GoodData reports publicly other than exporting PNGs yet; the raw data and scripts used are attached.)

Vincent: Perl 5.16 and Beyond

Posted Sep 13, 2011 23:54 UTC (Tue) by fest3er (guest, #60379) [Link]

Documentation? I'm still using Wall & Schwartz's 1992 printing of 'Programming perl' and, on occasion, Srinivasan's 1997 edition of "Advanced Perl Programming. :)

Vincent: Perl 5.16 and Beyond

Posted Sep 14, 2011 15:31 UTC (Wed) by rfunk (subscriber, #4054) [Link]

My understanding is that many of the techniques promoted in those books are now considered passé if not deprecated, and there are much better ways of doing things now.

That said, when I write any Perl these days I still write it like I did a decade ago. Since it's no longer my primary language I can't be bothered to keep up with all the changes.

Perl 5.16 and Relevance

Posted Sep 14, 2011 9:11 UTC (Wed) by kragilkragil2 (guest, #76172) [Link]

Just out of curiousity: What was last big FOSS project that was started and uses Perl?
I hear about stuff written in C(++), Java, Javascript, Python and PHP all the time. Sometimes I hear about Rails projects, but Perl .. I nearly never read or hear about new well-known projects written in Perl (besides the occasional small Wiki engine).

Is Perl just for FOSS legacy applications?

Perl 5.16 and Relevance

Posted Sep 14, 2011 11:37 UTC (Wed) by FranTaylor (guest, #80190) [Link]

Perl is for all manner of internal plumbing for your site.

Use it to generate reports, to auto-configure your application when you reinstall it. Use it to fill in the missing bits of functionality when you smash together a bunch of existing programs.

Perl is duct tape for the system administrator. You wouldn't want to code up a word processor or a database in it, but it sure does come in handy. Even if your app is written in something else, Perl can be the fastest and most expedient way to add that missing functionality that pulls the whole room together.

Perl 5.16 and Relevance

Posted Sep 14, 2011 13:29 UTC (Wed) by renox (subscriber, #23785) [Link]

I don't think it counts as a big FOSS project, but many tools in Mandriva were coded in Perl.
Not sure that it is a glowing reference though..

Perl 5.16 and Relevance

Posted Sep 14, 2011 14:14 UTC (Wed) by dskoll (subscriber, #1630) [Link]

As others have mentioned, Perl is used with in lots of plumbing. Many Debian system admin scripts are written in Perl (though the quality of said scripts is uneven at best.)

High-profile Perl projects include SpamAssassin, Catalyst (on which a bunch of web sites run), RT (a ticket-tracking system) and Slashcode (that powers slashdot). Admittedly, these are not new projects, but they are actively maintained.

Perl is also used a lot in bioinformatics, I believe. Of most interest to me, my company's (non-FOSS) products are written mostly in Perl.

Perl 5.16 and Relevance

Posted Sep 14, 2011 17:42 UTC (Wed) by k8to (subscriber, #15413) [Link]

Debian has been moving deliberately towards python from perl for their administrative tools. They don't foolishly abandon stuff just for language purity but the drift is there.

I'm intrigued that bioinformatics is still solidly on the perl train. It always seemed a surprising choice to me. Perhaps it will end up like fortran for physics -- used forever in that space.

Perl 5.16 and Relevance

Posted Sep 14, 2011 17:57 UTC (Wed) by andrel (guest, #5166) [Link]

I know bioinformaticians who still use Perl. But there's a strong trend towards using Python or R.

Perl 5.16 and Relevance

Posted Sep 14, 2011 21:57 UTC (Wed) by dskoll (subscriber, #1630) [Link]

Totally useless and meaningless statistics from my Debian Squeeze system:

$ echo /usr/bin/* /bin/* /usr/sbin/* /sbin/* | wc -w          
4619

$ file /usr/bin/* /bin/* /usr/sbin/* /sbin/* | grep -c shell.*script      
621

$ file /usr/bin/* /bin/* /usr/sbin/* /sbin/* | grep -c perl.*script
420

$ file /usr/bin/* /bin/* /usr/sbin/* /sbin/* | grep -c python.*script
198

So shell (typically Bourne shell) is ubiquitous, Perl looks solidly entrenched and Python is indeed up and coming.

Perl 5.16 and Relevance

Posted Sep 15, 2011 3:46 UTC (Thu) by k8to (subscriber, #15413) [Link]

I was going on some wiki and email policy statements as well as seeing a few key tools move to c or python. python for example is the recommended interface these days to apt.

But there are plenty of debian developers who do not view a sort of vision statement like that as very important so yes some perl tools will remain forever as sprout anew.

Perl 5.16 and Relevance

Posted Sep 15, 2011 22:58 UTC (Thu) by BenHutchings (subscriber, #37955) [Link]

Debian has been moving deliberately towards python from perl for their administrative tools.

There has been no such policy decision. Further, a subset of Perl (the perl-base package) is 'essential' (guaranteed to be installed) while no such subset of Python has that status. (Which I regret, as I would much prefer to use Python for complex maintainer scripts.)

Some utilities accompanying dpkg were written in Perl but have been rewritten in C so that 'Emdebian' installations can leave out perl-base as well.

Perl and bioinformatics

Posted Sep 16, 2011 13:55 UTC (Fri) by Kluge (subscriber, #2881) [Link]

I think part of the reason Perl is used so widely in bioinformatics is historical. But I think it's also because much of bioinformatics is text processing, and what is better suited for that than Perl?

Perl and bioinformatics

Posted Sep 16, 2011 15:34 UTC (Fri) by rgmoore (✭ supporter ✭, #75) [Link]

I think this is essentially correct. The problem in Bioinformatics, as in many areas, is that much of the early work was done in widely separated groups who weren't coordinating much. The result was a bunch of really good tools that don't play nicely together because they speak different languages. Rather than try to rewrite everything to work together perfectly, they stuck it together with a glue layer written in Perl. And of course once you have a setup in Perl that's the only thing that understands all the different languages, it's the natural choice to build new tools.

The result is that Bioperl now has literally hundreds of modules. Many of them are rather special purpose, but taken together they give you a very nice toolbox for solving your Bioinformatics problems. They may be a bit heavyweight- for some simple problems I find it's easier to write my own, lightweight solution than to read through the reams of Bioperl documentation to find the tool that does what I want- but that's the price you pay for the power and flexibility.

Perl 5.16 and Relevance

Posted Sep 15, 2011 3:07 UTC (Thu) by sumanah (subscriber, #59891) [Link]

Just adding another actively maintained Perl project to your list: Dreamwidth. http://wiki.dwscoalition.org/notes/Dev_Getting_Started for more info.

Perl 5.16 and Relevance

Posted Sep 15, 2011 6:20 UTC (Thu) by szabgab (guest, #80245) [Link]

AFAIK Most of the open source Perl stuff consists of libraries allowing one to build large in-house applications. Both web applications and automation stuff. See CPAN for details.

For applications you could take a look at Perl Dancer web framework and at the dancefloor where they show a number of projects written using Dancer.

Another option is the Mojolicious web framework though I am not sure where do they list the projects written in it.

Finally let me point to Padre, the Perl IDE. While being an IDE for Perl is a bit self serving, it is a large open source projects written in Perl recently. (disclosure: I started this project)

Perl 5.16 and Relevance

Posted Sep 21, 2011 21:53 UTC (Wed) by sebastian (subscriber, #4910) [Link]

Perl's ease and speed of use are important. A lot of device vendors will ship sample developer kits with Java and C, but these take quite a bit of time to decipher. If I was developing a fancy expensive interface I would take the time to dig thru the developer kits. But a quick search on CPAN may reveal modules that will get you up and running in a couple of hours. Try that in Java. You could waste a week coding up a working example and have less functionality than in Perl.

FOSS Project in Perl

Posted Sep 17, 2011 22:04 UTC (Sat) by plaxx (subscriber, #53703) [Link]

Another FOSS Perl project: PacketFence (network access control). We announce our releases here on LWN (http://www.google.com/search?q=site%3Alwn.net+packetfence).

The choice of Perl was done before my time so I can't tell why it was decided that way but I can say that it has proven to be very productive (developer time wise), easy for users to do customization and easy for us to implement quick on-site fixes or apply patches to clients (because no recompilation required). Also, network administrators seems to be speaking Perl more than anything else so they are familiar with it.

I am not saying that Python doesn't apply to all the above (I believe it does) but so does Perl and a project with that history won't be reimplemented just because the cool kid around the block is python these days. It would not be economic to do so.

Full disclosure: I am PacketFence's lead developer living the dream of doing open source for a living at Inverse (http://inverse.ca).

Vincent: Perl 5.16 and Beyond

Posted Sep 15, 2011 8:44 UTC (Thu) by Auders (guest, #53318) [Link]

> Perl 5 is not Latin.

http://www.csse.monash.edu.au/~damian/papers/HTML/Perliga...

Couldn't resist :)

Vincent: Perl 5.16 and Beyond

Posted Sep 16, 2011 0:41 UTC (Fri) by dlang (subscriber, #313) [Link]

there should be some easy way of saying 'use the latest version of perl and let me deal with the breakage' rather than forcing every perl script to be modified to say that it works with that version.

Vincent: Perl 5.16 and Beyond

Posted Sep 16, 2011 6:49 UTC (Fri) by cladisch (✭ supporter ✭, #50193) [Link]

"Let me deal with the breakage" is appropriate only for throwaway scripts, or scripts that are being modified continuously so that the additional porting effort doesn't matter.

Preferring backward compatibility over new features by default is more appropriate when you have lots of 'legacy' scripts that you want to keep running without too much porting effort; and this is seen as the primary use case for Perl 5.

That said, I'd be suprised if there wasn't a "use v-infinity" or a command-line option or something like that.

Vincent: Perl 5.16 and Beyond

Posted Sep 16, 2011 17:52 UTC (Fri) by dlang (subscriber, #313) [Link]

the problem is that without an easy way to upgrade perl and then see if everything still works the process of testing the scripts becomes impractical (having to change each one to test it)

this will mean that when someone (years from now) finally does want to modify a script to use a new feature and they go from 5.14 (default with no use) to 5.20, a lot of things unrelated to what they are currently working on may break. In many cases it's far better to do the testing of new features and fixing breakage incrementally rather than let the changes pile up and have to deal with them all at once.

Vincent: Perl 5.16 and Beyond

Posted Sep 16, 2011 19:58 UTC (Fri) by rgmoore (✭ supporter ✭, #75) [Link]

A use latest; switch would be a good way of telling Perl to give native behavior rather than emulating an older version. It should be obvious what it means, and it doesn't require something like "infinity".

I expect maintaining backward compatibility to a specific version number will become an ever larger headache as the number of compatible versions grows. I wouldn't be surprised if it winds up being limited to some set number when the maintenance gets to be too much of a drain on the project.


Copyright © 2011, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds