Perl APIs

For a lot of programmers out there, Perl has become largely invisible. They just never come across it. That might seem strange to you as you sit inside the Perl community echo chamber reading the Perl Ironman or p5p, but try this simple experiment.

Think of a web site that you use and that supplies an API. Now go to that API’s documentation and look at the example code. What languages are the examples written in? PHP? Almost certainly. Ruby? Probably. Python? Probably. C#? Quite possibly. Perl? Almost certainly not[1].

Perl has fallen so far off the radar of most people that when web sites write example code for these APIs, they very rarely consider Perl as a language worth including. And because they don’t bother including Perl then any random programmer coming to that API will assume that the API doesn’t support Perl, or (at the very least) that the lack of examples will make using Perl harder than it would be with plenty of examples to copy from.

This then becomes a self-fulfilling prophecy as Github fills with more and more projects using other languages to talk to these APIs. And the chance of anyone who isn’t already a Perl user ever trying to interact with these APIs using Perl falls and falls.

Of course, this is all completely wrong. These APIs are just going to be a series of HTTP requests using REST or XML-RPC (or, if you’re really unlucky, SOAP). Perl has good support for all of that. You might need to use something like OAuth to get access to the API – well Perl does that too.

Of course, in some cases good Perl support exists already – Net::Twitter is a good example. And to be fair to Twitter, their API documentation doesn’t seem to give any examples in specific languages – so Perl isn’t excluded here. But in many other cases, the Perl version languishes unnoticed on CPAN while other languages get mentioned on the API page.

I think that we can try to address this in 2014. And I’d like to ask you to help me. I’ve set up a mailing list called perl-api-squad where we can discuss this. In a nutshell, I think that the plan should be something like this:

  1. Identify useful APIs where there is either no Perl API or no Perl examples
  2. Write CPAN API wrappers where they are missing
  3. Approve API owners and offer them Perl examples to add to their web site

That doesn’t sound too complicated to me. And I think (or, perhaps, hope) that most API owners will be grateful to add more examples of API usage to their site – particularly if it involves next to no effort on their part.

I also expect that the Perl API Squad will produce a web site that lists Perl API support. We might even move towards producing a framework that makes it easy to write a basic Perl wrapper around any new API.

What do you think? Is this a worthwhile project? Who’s interested in joining in?

[1] Yes, I know there are exceptions. But they are just that – exceptions.

Send to Kindle

Perl in Banks

An email has flooded in:

I came across your presentation ‘Perl in the Enterprise’ and happen to have a burning concern closely related to one of the bullets ‘Banks see as a competitive advantage’.

I am consulting at a major bank in South Africa, and our team have been using Perl very productively to develop and maintain a customer loyalty rewards  application. This application has now suddenly caught the attention of the ‘Architecture Board’ who are questioning whether Perl (or any other scripting language) is appropriate for production use. A presentation will have to be made to this board to present a business case.

Whatever the basis for this concern, some quotable and referencable stories from other banks, such as the list included in your clients on the website, would be gold for us. I have searched extensively on the web and cannot find anything in this light which is also recent, as in the last 5 years. Other than Nvidia and I can only find anecdotal, 2nd hand evidence of Perl use for medium-to-large applications in well-known corporates.

Do you by any chance have anything or any contacts in those banks that would be willing to respond to an email to confirm their use of Perl and perhaps repeat the competitive advantage claim?

The presentation he’s talking about is Perl in the Enterprise, which I gave at Linux World Expo in 2005. The particular line he’s interested in is on slide 6 where I say that banks see using Perl as a competitive advantage. I’m pretty sure that I was paraphrasing Phillip Moore of Morgan Stanley who had said something similar in a keynote at OSCON in about 2001.

Obviously I know that banks in the City of London still use Perl. But it’s been several years since I worked at one, so my personal experience is slightly out of date. What my correspondent needs is people who are currently working in banks who are happy to go public and say “we use Perl in our production systems”. It would be even more helpful if they could add “and here’s why…”.

Can anyone out there help?

Send to Kindle

Perl Search Revisited

A couple of years ago I wrote a blog post about a Google Custom Search that I had set up to create a specialised search engine for Perl.

Recently I’ve revisited this idea. I’ve given the search engine its own subdomain and I’ve added some new sites to the list of sites that it covers. I’ve also given it a simplified look (thanks Bootstrap) and it’s now being hosted on Github pages.

It lives at Please give it a try and let me know what you think.

Send to Kindle

Just Build Something

The Political Web

About a month ago, JT Smith suggested that we should all stop talking about Perl and just build something. And, purely coincidentally, over the last few weeks I resurrected a project that I have been poking at for about five years and have finally turned it into something that I’m happy to show the world.

The Political Web is a site which aggregates all of the information I can find on the web about individual British MPs. I say “all of the information”, but that’s obviously a bit of a work in progress. But I think that what I already have is useful and interesting – well, for people who are interested in British politics. I have plans to bring in more information in the future.

Although I’ve been working on the site for five years, I pretty much rebuilt it from scratch when I recently returned to it. Actually getting something useful up and running took about four hours. That’s because I was building it using Perl and, more specifically, Dancer.

Send to Kindle

Removing Modules from Core

I was on holiday last week and missed most of the discussions about removing Module::Build and CGI from the Perl core in the next few years. I hope you won’t mind if I chime in a little late with my thoughts.


I’m a little bemused by the Module::Build story. Well, perhaps “bemused” isn’t quite the right word. Perhaps I should say “embarrassed”. Embarrassed because the discussions around Module::Build just show how out of touch I am with important parts of the Perl ecosystem.

I remember seeing Schwern’s MakeMaker Is DOOMED! talk ten years ago. What he said made a lot of sense to me and I switched all of my CPAN modules to Module::Build over the next few years. And then I rather took my eye off the ball. I was vaguely aware that people were complaining about Module::Build and were switching to things like Module::Install and Dist::Zilla. But I was happy with Module::Build and just stuck with it.

Now I’m hearing people saying that Module::Build is fundamentally flawed. Someone whose opinion I value mentioned that he lowers his estimate of the IQ of any CPAN author who uses Module::Build. That, presumably, includes me. But when I ask people what the problems are with Module::Build, I can’t get anyone to give me a detailed and dispassionate answer.

I’m quite prepared to believe that there are problems with Module::Build. Given the calibre of the people who are telling me that these problems exist, it would be astonishing if it wasn’t true. But it would be nice if there was a good explanation available somewhere of just what these problems are.

And I think that this is indicative of a deeper problem that the Perl community has. I like to think that I’m pretty much in touch with what is going on in the Perl community. I read the blogs. I’m subscribed to (too many) mailing lists. But it appears I’m not following discussions of the CPAN toolchain closely enough. I guess it’s something that I just expect to work. And I guess I expect that if there are important discussions going on, then I’ll hear about them through other channels – blog posts, for example. But that doesn’t seem to have happened here. Of course I can subscribe myself to another mailing list or two to correct that, but I don’t think I’m unusual in not trying to follow every single Perl discussion. There are surely many people like me who would like information like this to be disseminated better. This is part of the reason why Leo and I set up the Perl News site. But we have a tiny number of editors and we can’t know everything that is worth publishing. If you think we’ve missed an interesting story then please let us know about it.

So I’m not going to object to Module::Build leaving the core. I’m sure there are good reasons, I just wish I knew what they are. I am, however, slightly disappointed to find that Schwern was wrong ten years ago and that ExtUtils::MakeMaker wasn’t doomed.

CGI and I go back a long way. In fact I think was added to the Perl core at about the same time as I started using Perl.

It’s obvious why it was added to the core. Back in 1997, Perl and CGI were tightly bound together in many people’s minds. I had people telling me that Perl could only be used to write CGI programs and other people telling me that CGI programs could only be written in Perl. Both sets of people were, of course, wrong; but it’s easy to see how they reached both of those conclusions.

Before was added to the Perl core, people used to parse CGI parameters using horribly broken code copied from Matt Wright (code which he had originally copied from Reuven Lerner). Of course, adding the module to the Perl core didn’t magically rewrite all those horrible CGI programs to use CGI::params, but at least it meant that the option was there.

This became important to me when I started the nms project. This project rewrote Matt Wright’s scripts using a better standard of Perl code. The problem was that many people started by installing one of Matt’s scripts on their server and later, when they wanted to move on from using other people’s code, they would use Matt’s code as a template for their own first steps into Perl. This is partly what made Perl so popular. But it’s also what lead to the preponderance of terrible Perl code that you still find on the web today.

We wrote the nms programs to work, as far as possible, in the same environment as Matt’s scripts. Matt said that his scripts would work fine on Perl 4 (back then, there were still many cheap hosting plans that came with Perl 4). We decided not to put ourselves through that pain and to target Perl 5.004 – because that was the first version to include Remember, we were targeting Matt Wright’s users. And they were people who used cheap hosting plans with basic FTP access and little or no chance of installing anything from CPAN. We had to rely on core modules. If hadn’t been in the Perl core, then the nms programs would have been far harder to write and the project may have never been started.

The nms project was started over ten years ago. What is the situation like now? Those cheap hosting plans still exist. People still use them for simple web sites. Some of those people still use Matt’s scripts or the nms alternatives. But the numbers of people using those programs are far smaller than they used to be. These days, people wanting to add simple dynamic functionality to a web site are far more likely to use a PHP program. And those cheap web hosts are far more likely to want to support PHP.

Of course, these days, no serious Perl developer uses to write web programs. We’d be far more likely to use something based on PSGI. And even if we did want to use, we’d know how to get it installed from CPAN. The only people who are going to suffer from the removal of are the people using something like the nms programs on a cheap hosting plan. I worry a little about the influx of support email we will get when some cheap hosting company updates from Perl 5.18 to Perl 5.21 and suddenly all the nms programs stop working. It’s slightly galling to realise that Matt’s scripts will continue to work at that point. Of course, the kinds of companies that we’re talking about tend to lag well behind the bleeding edge of Perl versions, so this might not happen for eight or more years. So I think it might well be someone else’s problem.

All in all, I think it will be sad to see removed from the core. It will effectively see the end of even half-decent Perl code being used on bottom of the range hosting plans. But Perl has been losing that market for years and, speaking as someone who spent a lot of time helping out on support forums for basic Perl/CGI code, I’m far from convinced that I’ll miss those users. So my sadness at seeing CGI,pm go, will be purely historical.

But I’d really like to see some cut-down version of PSGI take its place!

Send to Kindle

What New(ish) Perl features Do You Use?

Over on LinkedIn, someone asked me “What core PERL[sic] features do you use regularly that are new since 95?” It’s hard to be sure as the perldelta files only seem to go back to 1997 (for example, when were qw(...), q(...) and qq(...) added?), but here’s a quick list off the top of my head.

  • my was, of course, added in 5.0. But 5.004 added the ability to use it in control expressions – while (my $foo = <>) – and in foreach loops – foreach my $foo (@foos)
  • use VERSION
  • Regex extensions – (?<=RE) and similar. Oh, and qr/.../
  • Data::Dumper (added in 5.005)
  • Unicode support – first added in 5.6.0 and improved in every release since
  • our
  • Three-argument open
  • Omission of intermediate arrows in data structure lookups – $foo[$x][$y] instead of $foo[$x]->[$y]
  • use warnings
  • Memoize
  • Test::More and Test::Simple
  • say
  • defined-or
  • use base (or, more recently, use parent)
  • yada-yada operator

Have I missed anything obvious? What new Perl features do you use most?

Send to Kindle

Training Cancelled

A few weeks ago, I mentioned that I had two days of public training coming up in Manchester in April. I’ve just heard that the organisers have decided to cancel this training as they have had no bookings for the classes. I’m sorry if anyone was planning to book closer to the date and has been inconvenienced by this.

Send to Kindle

Training Debrief

I’ve spent a lot of the last seven days running training courses. It might be interesting to share some thoughts about how they went.

Last Saturday was Perl School 4. A week before the course I was a little worried about ticket sales, but I did a bit of marketing early last week and managed to more than double sales in a few days. In the end I had 27 people signed up.

Perl School is always enjoyable. I think that people often turn up with quite low expectations as it’s so cheap. So it’s fun to overturn those expectations and give them a day of high quality training. People obviously recognise that as I’m getting a lot of repeat business – at least one person has come along to three of the four courses so far.

Many of the courses I give are overviews of Perl at various levels. This one was just about DBIx::Class so it was great to be able to go into a lot more depth on a single topic. Of course, DBIx::Class is a great subject to cover and it was fun explaining its more powerful corners to a room of people who don’t know much about it.

I thought it went well. But don’t just take my word for it. I’ve been asking attendees to fill in feedback forms about all the Perl School courses and I’ve published a page summarising that feedback.

Then this week has been two two-day courses for flossUK. Two day courses give us time to include practical sessions so that people go home having actually tried out the techniques that I’ve taught – which nicely reinforces the lessons. I really enjoy those sessions as you really see lightbulb moments as people see how easy it is to use these tools. This afternoon, for example, it was great to see people getting a simple Catalyst application up and running in less than an hour. An hour later people were really impressed as I introduced them to Plack::Middleware::Debug and showed them how I could get detailed DBIC_TRACE output on the web page by making tiny changes to the application code. At least one person went away determined to reimplement a number of key applications in Catalyst as soon as possible.

And that, to me, is the joy of running training courses. It’s great to open people’s eyes to the possibilities that these new tools give them. I love to see them leave filled with renewed enthusiasm for the language.

Send to Kindle

Perl Books

The Perl community on LinkedIn is fascinating. It’s a great way to see how Perl is perceived and used outside of the echo chamber. And that’s a real eye-opener.

Here’s an example. Every few weeks (it seems) someone asks for advice on Perl books.At that point a few people will jump in with sensible suggestions. But for every reasonable suggestion, you’ll get three or four people suggesting something from this list:

  • Something horribly out of date. A lot of publishers stopped updating their Perl books about ten years ago. The most recent version of Perl Black Book that I can find is from 2001. Teach Yourself Perl in 21 Days was last updated in 2002. Perl By Example dates from 2007. I haven’t read any of these books, so I can’t comment on their quality, but I’d be really wary of suggesting that someone learns Perl from a book that is so out of date. Before the days of Amazon, these books would quietly disappear from bookshops, but now they always seem to be available.
  • Something out of date, but that used to be great. There are plenty of books that I would have recommended when they were current, but that I wouldn’t really want to recommend now. A lot of these are O’Reilly books that haven’t been updated recently – Advanced Perl Programming and The Perl Cookbook, for example. I’m sure there’s lots of good stuff in those books. But I really wouldn’t want anyone to read them if they didn’t already have enough experience to update the Perl to current best practices. I sometimes recommend these books myself, but only in specific circumstances. Whenever I run a course on OO Perl I mention Damian’s book. But I also point out that there have been a lot of advances in the area since the book was published. And, I have to confess, my own Data Munging with Perl fits firmly in this category.
  • Something of dubious provenance.  Most of these discussions will, at some point, attract a link to some dubious web site in Eastern Europe that contains the full text of O’Reilly’s various CD bookshelves. I know that these sites exist and I know that there’s nothing that O’Reilly can do about them, But I’d rather not see them mentioned on a professional site like LinkedIn. And quite apart from the copyright issues, there’s the fact that most of the books on these sites fall firmly into the first category above. They’re all over ten years old.
  • Some great tutorial on the internet. I’ve talked about the problems of old and dodgy tutorials before. And things definitely seem to be looking up. We are getting more good tutorials out there. But people still insist on sharing links to the tutorial that they learned from. Even if it’s appalling.

In a recent discussion someone said (and I’m paraphrasing) “just go to Amazon and look for the highest rated books”. I think there are two problems with that:

  1. The people who are rating beginner’s programming books on Amazon are usually the least qualified people to do that. Sure, they can tell how easy the book was to read and how well they picked up what the author taught them. But they have noway of knowing whether what they learned was accurate or useful.
  2. Amazon ratings last forever. But, as noted above, the quality of a technical book in fast-moving subject like programming falls over time. Perhaps Amazon ratings on programming books should have a half-life.

Here are some examples of things that you’re going to miss out on by using outdated Perl references.

  • say – We all love say, don’t we?
  • Lexical filehandles – Storing filehandles in lexical variables is great. I no longer have to worry about bareword filehandles being reused elsewhere in the code.
  • Defined or operator – There’s now no excuse for the $val ||= $default bug.
  • given/when – Perl has a switch statement. And it’s better than anyone else’s :)
  • Unicode support – Perl’s Unicode support is second to none. Except, if you’re using an old version of Perl in which case it’s a bit rubbish. And if you’re reading about Perl in an old book, then you won’t know about it.
  • State variables – Ok, I don’t use those every day. But when I need them, they make my code cleaner.

And then there are all the great CPAN modules that aren’t covered in books that were written before they were released. Would you really want to introduce someone to OO Perl without mentioning Moose?

My rules for recommending books are pretty simple. They should be books that I’ve found useful and they should have been published in the last few years. And given the falling numbers of Perl books that are published each year, that’s now a rather small number of books. Perhaps a dozen or so.

Am I being too harsh? Can beginners get something useful out of older Perl books? How do you decide whether to recommend a book to a colleague?

Send to Kindle

Why Corporates Hate Perl

This is a reprint of an old blog post.

A few years ago I was writing blog posts (semi-)regularly for O’Reilly. This is the one that probably got the most feedback. I’m reprinting it now because a) it’s pretty hard to find on the O’Reilly site and b) it’s relevant to a couple of conversations that I’ve had over the last few days.

Last week I was in Copenhagen for YAPC::Europe. One of the announcements at the conference was the location of next year’s conference which will be in Lisbon. The theme of next year’s conference will be “Corporate Perl”. And that (along with a couple of conversations last night) got me thinking about a talk that I’ll submit to next year’s conference which might well be entitled “Why Corporates Hate Perl”.

It’s not true, of course. There are a still large number of large companies who love Perl. I could probably work through to my retirement enhancing and extending systems that are written in Perl at many of the big banks in the City of London. There are, however, also many companies who are moving away from Perl for a number of reasons. Here’s one of the reasons that will be included in my talk.

I was talking to people from one such company last night. The Powers That Be at this company have announced that Perl is no longer their language of choice for web systems and that over time (probably a lot of time) systems will be rewritten in a combination of Java and PHP. Management have started to refer to Perl-based systems as “legacy” and to generally disparage it. This attitude has seeped through to non-technical business users who have started to worry if developers mention a system that is written in Perl. Business users, of course, don’t want nasty old, broken Perl code. They want the shiny new technologies.

And so, in a matter of months, the technical managers at this company have created a business environment where Perl is seen as the cause of most of the problems with the current systems. It’s an impressive piece of social engineering.

It’s also, of course, completely unfair. I don’t deny at all that this company (like many others) has a large amount of badly written and hard to maintain Perl code. But I maintain that this isn’t directly due to the code being written in Perl. It’s because the Perl code has developed piecemeal over the last ten or so years in an environment where there was no design authority which encouraged developers to think beyond getting their immediate task done. Many of these systems date back to this company’s first steps onto the internet and were made by separate departments who had no interaction with each other. It’s not really a surprise that the systems don’t interact well and a lot of the code is hard to maintain.

There are, on the other hand, a number of newer systems which are also written in Perl which follow current best practices in Perl development and are far easier to to maintain and enhance – as easy, I would contend, as anything written in the new approved languages.

It’s certainly true that this company has a large number of systems that need to be rewritten over the next few years. But throwing away all of the company’s accumulated Perl expertise and moving to new languages seems to be a step too far. Management are blaming Perl for the problems when really they should be blaming the management and design procedures that were in place (or, more likely, weren’t in place) when the code was originally written.

Many organisations are in the same situation, with large amounts of unwieldy Perl code. Ten or twelve years ago everyone was writing web systems in Perl and we were all making mistakes. We all have to deal with those mistakes but we’ve  hopefully, learned from them and can rewrite our systems to take account of everything that we’ve learned in the last ten years.

It’s too late for the company I’ve been talking about in this article. The anti-Perl social engineering has probably insinuated itself too deeply into the culture. It’s unlikely that Perl’s reputation can be rescued.

But if you have similar problems in your own company, then please try to ensure that blame is apportioned correctly and that you don’t use Perl as a scapegoat.

A couple of updates to the post. I did propose the talk to the next YAPC, but the proposal wasn’t accepted. And the company I talk about in the article is still employing a lot of Perl programmers – four years after this post was written.


Send to Kindle