It’s been over six weeks since I wrote my blog post on Perl usage. I really didn’t mean to leave it so long to write the follow-up. But real life intervened and I haven’t had time for much blogging. That’s still the case (I should be writing a talk right now) but I thought it was worth jotting down some quick notes about what I think is causing Perl’s decline.
Reputation
We have a lot to thank Matt Wright for. And I don’t mean that sarcastically. A lot of the popularity of Perl in the mid-90s stems directly from people like Matt and Selena Sol making their collections of CGI programs available really early on. The popularity of their programs made Perl the de-facto standard for CGI programming.
But that was a double-edged sword. People searching the web for examples of CGI programming found Matt or Selena’s code and assumed they represented best practice. Which, of course, they didn’t. While people were blithely copying Matt’s programming style, good Perl programmers were using CGI.pm to parse their incoming parameters and separating their HTML generation out into templates.
In my previous post, I mentioned that fifteen or twenty years ago Perl was the programming language of choice for internet start-ups. That’s true, but a lot of the code written at that time was in the Matt Wright style. Matt’s style just about works for a guestbook or a form mailer. But when you try to build a business on top of code like that, it quickly becomes obvious that it’s an unmaintainable mess.
Many of the technical architects and CTOs who are making decisions about technology in companies today are the programmers who spent too many late nights battling those balls of mud in the 1990s. They were never really Perl programmers, they were only using it because it was fashionable, and they haven’t been keeping up with recent advances in Perl so it’s not surprising that they often choose to avoid using Perl.
Complexity
A lot of Perl’s reputation as executable line noise is completely unwarranted. The people who were writing those 1990s balls of mud were under such pressure to deliver that they would have almost certainly delivered something just as unmaintainable whatever language they were using. But some of that reputation is fair. I’ve been teaching Perl for almost fifteen years and I know that there are some parts of Perl that people find confusing. Here are some examples:
Sigils – I can explain things like @array, $array[$key] and even @array[@keys] to people. And most of them get it. But it takes them a while. And then it all goes to pieces again when I have to explain the difference between $array[$key] and $array->[$key].
Context – Does any other programming language have the concept of context? Yes, when used correctly it’s a powerful tool. But it’s hard to explain and a good source of hard-to-find bugs. Can anyone honestly say that they haven’t been bitten by a context bug at some point in the last years?
Data Structures – Is the difference between arrays and array references really necessary? Think of all the complexity that is added because you can’t just pass arrays and hashes into subroutines without being bitten by list flattening. As experienced Perl programmers we know the problems and our brains are hard-wired to work around it. But other languages treat all aggregate data structures as references and it all becomes a lot easier.
I know that each of these features (and half a dozen other examples I could list) makes Perl a richer and more expressive language. But this comes at the cost of learnability and readability. Perhaps that trade-off once seemed like a good idea. When you’re trying to encourage people to look at your language then the advantages seem less obvious.
Of course, none of these features can be changed as they would break pretty much every existing Perl codebase. Which would be a terrible idea. But you can get away with a lot more breakage when you increase your major version number. Which Perl hasn’t been able to do for fourteen years.
Perl 6
I need to be clear here. I think that Perl 6 looks like a great language. I am really looking forward to using on production systems. And it looks like the current Perl 6 team are doing great work towards making that possible. In fact I think that our best approach to reviving Perl’s fortunes is to get a production-ready version of Perl 6 out and to make a big noise about that.
However, that name has been a big problem.
Looking from outside the Perl echo chamber, it’s easy to believe that Perl hasn’t had a major release for twenty years. And that can probably explain a lot of Perl’s current problems.
I know that people who believe that are wrong. The current version of Perl (5.20.1 as I write this) is a lot different to the version that was current when Perl 6 was first announced (which was 5.6.0, I think). Perl has gone through huge changes in the last fourteen years. But the version number hides that.
I also know that we no longer tell people that Perl 6 is the next version of Perl. The Wikipedia page makes it clear in its first sentence that “Perl 6 is a member of the Perl family of programming languages“. So why do people continue to think it’s the next version of Perl? Well, probably because people assume that they know how software version numbers work and don’t bother to check the web site to see it a particular project has changed the standard meaning that has worked well for decades.
So Perl 6 has been simultaneously both good and bad for Perl. Good because a lot of Perl 6 ideas have been backported into Perl 5. But bad because Perl 5 has been unable to change its major version number in order to advertise these improvements to the wider software-using world.
Nothing can be done about this now. The damage is done. As I said at the start of this section, it’s likely that the only thing we can do is to bet heavily on Perl 6 and get it out as soon as possible. Perl 5 will continue to exist. People will continue to maintain and improve it. Some companies will continue to use it. But it’s usage will continue to fall. I really think it’s too late to do anything about that.
The only problem is missing Applications…… it is as simple as that…..
Spot on. There is no problem within the language. Only a lack of polished, ready to use products, that people can put to use. One web framework that comes with a set of nice looking HTML/CSS templates and a decent tutorial is worth more than 20 state of the art blog engines that look like shit out of the box.
Front-end themes for ShinyCMS will be gratefully received. Documentation patches also. 🙂
I will take you up on that offer 🙂
I tend to disagree with Torsten on just about anything, but this time he gets it totally right.
Getty, the apps don’t exist because the developers don’t exist to write them.
I was really excited by Perl 6 when it was first announced. I thought this would finally clean up all the chaotic nonsense that makes Perl such a toxic swamp… but no, it’s got a whole new set of gratuitous Englishisms.
sigils….yeah, I think you just internalize them, shrug and move on
but context (wantarray etc?). to me, discovering this years ago was like discovering gold
Applications.
I’m with Torsten too on this one.
Since bugzilla and nagios basically nothing really happened in that department and I have to get my wordpress, forum, docker, jenkins, trello, buildbot, chef, puppet, you name it fix elsewhere.
Also, why do Haskell and Python have a versioning system and we don’t?
Oh, wait, I think we have a Wiki. 😉
(And yes, I’m aware of projects like Movable Type and WebGUI.)
Oh, yeah, c++ does now :D. (reference and rvalue reference modifiers on methods)
Hmm, Dave you seem to be blaming Perl’s failings on one guy’s sloppy code – I’m not sure that’s entirely fair! 😉 Also, when I think of Perl, I don’t things sigils are the issue are they? I actually quite like them as they give some differentiation. Likewise I always thought of context as a wonderful aspect of the language. Where I think Perl’s syntax falls down is when you try to scale or write more complex programs (which of course Perl was never really designed for). So for example, the references syntax is frankly horrific. Attempting to process complex data structures, such as parsing XML documents, results in a horrible “arrow/brace soup”, that makes my eyes bleed. It’s all just so much less painful in Python or PHP. And apologies for going off topic, but as I mention PHP, I think it has now been given a new lease of life through FB’s release of Hack/HHVM. Not so sure about Perl 6.
Well, Matt’s scripts are only one of the reasons I mention. And even then it’s not so much the fact that Matt wrote such bad code, but that so many people copied it. Also, I tried to make it clear that without people like Matt I’m not sure that Perl would ever have achieved the ubiquity it once had in Web programming.
And, yes, for experienced Perl programmers sigils and context are (usually) great. I’m just pointing out that they are often a big stumbling block to people coming to Perl from other languages.
I think that whoever first said we need to leapfrog Perl around version 6 and just go to version 7 was right. Instead of Perl 5.22, we should just declare victory and call it 7.0. Bring forward the stuff out of Perl 6 that makes sense, but keep the basic core 5.x stuff.
I have no reason to believe 6.x will ever become usable, production code with widespread adoption. I have every reason to believe that a Perl 7.x based on the advances made since the original version 5 came out would be at least a moderate success.
Will we ever return back to the glory days for Perl? No. That ship has sailed. There are too many viable competitors out there now that have gained mindshare, and too much inertia in the idea that Perl is in decline. We may be able to slow the perception of the decline, maybe even bring it to the point where it is stabilized, but it is not going to once again become the de facto glue of the Internet.
We stand a decent chance of losing momentum on Perl altogether. The revitalization of the 5.x stuff around 5.10 prevented the language from the fate of QBasic, but it won’t be enough unless we start generating some new buzz and creating some new mindshare as a result. If things continue along their present course and speed, we will lose the developers we regained. It’s important we don’t allow that to happen.
Version numbers are a silly thing to rally around, it is true. They are completely arbitrary and really are not technically important. But version numbers are what non-gearheads understand. They will “get it” if we move Perl to 7.x. The message will be clear: Perl remains a force to be reckoned with, a supported language that you can bet your next project on, a language that has a large feature set, the power of CPAN behind it, and thousands upon thousands of trained programmers.
It’s time to declare victory. It’s time for Perl 7.
Three things make Perl look archaic next to Ruby and Python – unrolling @_, the boilerplate required to get subroutine signatures working and the dearth of new books. Even when a book does appear it’s usually yet another general language guide. Compare that with applied programming titles such as: “RubyMotion”, “The RSPec Book” and “Sinatra Up and Running”. “Perl Cookbook” is now 11 years old which kind of sums up the state of Perl.
I think you may be confusing cause and effect when it comes to books. No-one is going to publish a book that they don’t think there is a market for.
Are you saying there’s no market for the Perl programming language?
I’m saying that publishers believe that there is no market for books about Perl.
(There are exceptions, of course, O’Reilly still update some of their Perl books regularly – but I can’t remember the last brand new Perl book they published.)
Boilerplate like: use feature ‘signatures’; ?
use 5.020;
use feature ‘signatures’;
no warnings ‘experimental::signatures’;
Even worse, you still have to tell Perl 6 to “use v6;”. I despair.
Erm, you don’t need to tell Perl 6 to “use v6;”. That’s just a best practice, like “use strict;” in Perl 5, so that if you try to run a Perl 6 script with Perl 5 it’ll fail quickly
use experimental ‘signatures’;
I agree with the idea of Perl 7. Things it need to include: a decent MOP (meta-object protocol) and a deep revision of the standard library shipped with the core, who has to be rewritten on top of the new mop. The main problem for Perl … that is what Ruby and Python offers us nowadays
Perl 7 sounds like a good idea. Until you realise that by skipping past Perl 6 you let people think that it was a failure.
(Preface: If I seem particularly angry or passionate here, it’s because I’ve seen that Perl can be used to make great things. There are some brilliant people in the Perl community pouring their hearts into excellent projects. Seeing the language struggle makes me sad.)
I am in complete agreement with your points. I’ve come to program professionally in Perl after having learned programming using Python and various lisp family languages. Perl’s reputation among non Perl programmers is horrible. Whether it’s “People still use Perl?” or “Perl is line noise” or “Perl violates every aspect of language design,” I am always on the defensive when explaining my use of it.
I have never been able to explain how context helps the programmer. Why not just create another function, instead of opening up the doors to all sorts of weird bugs? Automatic list flattening, as well, could have simply been a function and then dealing with nested data structures (which any web programmer is doing constantly, thanks to JSON) wouldn’t be a nightmare anymore. Yes, I’m at the point now where the @{}s and ->s happen automatically, but it was a big part of the learning curve and is completely unnecessary in competitor languages.
It would be one thing if Perl had some kind of killer app that would motivate people to take it up, but I think it’s telling that people are more willing to learn a completely different paradigm such as functional programming in Clojure, Scala, or even straight up Haskell than to learn yet another scripting language so that they can use Mojo instead of Flask — despite the seeming awesomeness of Mojo, PDL, etc.
Another barrier to entry for beginners is the lack of a featureful bundled REPL, which pretty much every other dynamic language (and even some static languages like Haskell) have. A REPL is critical for learning and experimentation. I know there are some modules that do this, but as I witnessed in IRC some of them have broken or unclear instructions that only resulted in the poor newbie getting flamed by many for not having “RTFM”.
In regards to Perl 6 as the future: in the same way that Rails made Ruby popular through making MVC easy, Perl 6 needs to seize the fact that many developers are struggling with async and concurrency and show them that it has the tools to handle those problems in a straightforward yet powerful way. As far as I am aware, this is something that is a fairly big problem in both Python and Ruby.
Unfortunately, I’m not certain if even this will be enough because P6 still has list flattening (albeit only in list context — ugh, context again) and syntax that rivals the Scala DSLs of old. But I do think that Perl 6 is the best chance for a language with Perl in the name to really gain back mindshare and thus to start seeing the kinds of high profile projects developed that can bring in new programmers. That being said, in the limited amount of free time I have to study and contribute and work on my own career, I have a very difficult time justifying putting effort into Perl 5 or Perl 6, given the uncertainty of their futures.
I gotta be honest, until I can build full API’s in perl faster than I can in Java (http://www.xenoterracide.com/2014/09/using-spring-to-create-full-rest-api-in.html)… Perl just isn’t going to cut it.
It doesn’t matter whether or not the language is ok, it matters that I feel that for the last 5 years, the language hasn’t made a lot of stable progress (the best new features are still experimental). The work on ORM’s and Web Frameworks has been insufficient, and unintegrated, PSGI is great, but not enough.
Also things like Padre are cool (not having really tried it) but major IDE support would be a good idea. Being able to hack up Perl 5/6 in IDEA, would be great.
If you’re worried that Perl needs a killer app then perhaps you should try to write one.
I’d say we know the problems, so that is good. One problem I don’t see listed is alienating Windows people. Windows is a bad operating system, but at least it installs correctly. And no matter how bad it is, you need to cater to it if you plan to be the biggest and best software language. To this day it is too hard to get a module off of CPAN on my Activestate system.
Mike,
I suspect that part of your problem is that you’re using Activestate. If you switched to using Strawberry Perl then I think you’d have a lot more luck. One of the main goals of the Strawberry Perl project was to make Perl on Windows look as much as possible like Perl on Unix.
Just a quick note:
I got and read Mastering Perl and Modern Perl and the two books got me interested in Perl again. They are pretty recent adds to the publishing space.
Case in point – these are both general Perl language books and only updates of existing publications at that. Where are the new, relevant applied Perl equivalents of “Mastering Clojure Data Analysis” or “Command Line Applications in Ruby”?