Removing Modules from Core

I was on holiday last week and missed most of the discussions about removing Module::Build and CGI from the Perl core in the next few years. I hope you won’t mind if I chime in a little late with my thoughts.

Module::Build

I’m a little bemused by the Module::Build story. Well, perhaps “bemused” isn’t quite the right word. Perhaps I should say “embarrassed”. Embarrassed because the discussions around Module::Build just show how out of touch I am with important parts of the Perl ecosystem.

I remember seeing Schwern’s MakeMaker Is DOOMED! talk ten years ago. What he said made a lot of sense to me and I switched all of my CPAN modules to Module::Build over the next few years. And then I rather took my eye off the ball. I was vaguely aware that people were complaining about Module::Build and were switching to things like Module::Install and Dist::Zilla. But I was happy with Module::Build and just stuck with it.

Now I’m hearing people saying that Module::Build is fundamentally flawed. Someone whose opinion I value mentioned that he lowers his estimate of the IQ of any CPAN author who uses Module::Build. That, presumably, includes me. But when I ask people what the problems are with Module::Build, I can’t get anyone to give me a detailed and dispassionate answer.

I’m quite prepared to believe that there are problems with Module::Build. Given the calibre of the people who are telling me that these problems exist, it would be astonishing if it wasn’t true. But it would be nice if there was a good explanation available somewhere of just what these problems are.

And I think that this is indicative of a deeper problem that the Perl community has. I like to think that I’m pretty much in touch with what is going on in the Perl community. I read the blogs. I’m subscribed to (too many) mailing lists. But it appears I’m not following discussions of the CPAN toolchain closely enough. I guess it’s something that I just expect to work. And I guess I expect that if there are important discussions going on, then I’ll hear about them through other channels – blog posts, for example. But that doesn’t seem to have happened here. Of course I can subscribe myself to another mailing list or two to correct that, but I don’t think I’m unusual in not trying to follow every single Perl discussion. There are surely many people like me who would like information like this to be disseminated better. This is part of the reason why Leo and I set up the Perl News site. But we have a tiny number of editors and we can’t know everything that is worth publishing. If you think we’ve missed an interesting story then please let us know about it.

So I’m not going to object to Module::Build leaving the core. I’m sure there are good reasons, I just wish I knew what they are. I am, however, slightly disappointed to find that Schwern was wrong ten years ago and that ExtUtils::MakeMaker wasn’t doomed.

CGI

CGI.pm and I go back a long way. In fact I think CGI.pm was added to the Perl core at about the same time as I started using Perl.

It’s obvious why it was added to the core. Back in 1997, Perl and CGI were tightly bound together in many people’s minds. I had people telling me that Perl could only be used to write CGI programs and other people telling me that CGI programs could only be written in Perl. Both sets of people were, of course, wrong; but it’s easy to see how they reached both of those conclusions.

Before CGI.pm was added to the Perl core, people used to parse CGI parameters using horribly broken code copied from Matt Wright (code which he had originally copied from Reuven Lerner). Of course, adding the module to the Perl core didn’t magically rewrite all those horrible CGI programs to use CGI::params, but at least it meant that the option was there.

This became important to me when I started the nms project. This project rewrote Matt Wright’s scripts using a better standard of Perl code. The problem was that many people started by installing one of Matt’s scripts on their server and later, when they wanted to move on from using other people’s code, they would use Matt’s code as a template for their own first steps into Perl. This is partly what made Perl so popular. But it’s also what lead to the preponderance of terrible Perl code that you still find on the web today.

We wrote the nms programs to work, as far as possible, in the same environment as Matt’s scripts. Matt said that his scripts would work fine on Perl 4 (back then, there were still many cheap hosting plans that came with Perl 4). We decided not to put ourselves through that pain and to target Perl 5.004 – because that was the first version to include CGI.pm. Remember, we were targeting Matt Wright’s users. And they were people who used cheap hosting plans with basic FTP access and little or no chance of installing anything from CPAN. We had to rely on core modules. If CGI.pm hadn’t been in the Perl core, then the nms programs would have been far harder to write and the project may have never been started.

The nms project was started over ten years ago. What is the situation like now? Those cheap hosting plans still exist. People still use them for simple web sites. Some of those people still use Matt’s scripts or the nms alternatives. But the numbers of people using those programs are far smaller than they used to be. These days, people wanting to add simple dynamic functionality to a web site are far more likely to use a PHP program. And those cheap web hosts are far more likely to want to support PHP.

Of course, these days, no serious Perl developer uses CGI.pm to write web programs. We’d be far more likely to use something based on PSGI. And even if we did want to use CGI.pm, we’d know how to get it installed from CPAN. The only people who are going to suffer from the removal of CGI.pm are the people using something like the nms programs on a cheap hosting plan. I worry a little about the influx of support email we will get when some cheap hosting company updates from Perl 5.18 to Perl 5.21 and suddenly all the nms programs stop working. It’s slightly galling to realise that Matt’s scripts will continue to work at that point. Of course, the kinds of companies that we’re talking about tend to lag well behind the bleeding edge of Perl versions, so this might not happen for eight or more years. So I think it might well be someone else’s problem.

All in all, I think it will be sad to see CGI.pm removed from the core. It will effectively see the end of even half-decent Perl code being used on bottom of the range hosting plans. But Perl has been losing that market for years and, speaking as someone who spent a lot of time helping out on support forums for basic Perl/CGI code, I’m far from convinced that I’ll miss those users. So my sadness at seeing CGI,pm go, will be purely historical.

But I’d really like to see some cut-down version of PSGI take its place!

What New(ish) Perl features Do You Use?

Over on LinkedIn, someone asked me “What core PERL[sic] features do you use regularly that are new since 95?” It’s hard to be sure as the perldelta files only seem to go back to 1997 (for example, when were qw(...), q(...) and qq(...) added?), but here’s a quick list off the top of my head.

  • my was, of course, added in 5.0. But 5.004 added the ability to use it in control expressions – while (my $foo = <>) – and in foreach loops – foreach my $foo (@foos)
  • use VERSION
  • Regex extensions – (?<=RE) and similar. Oh, and qr/.../
  • Data::Dumper (added in 5.005)
  • Unicode support – first added in 5.6.0 and improved in every release since
  • our
  • Three-argument open
  • Omission of intermediate arrows in data structure lookups – $foo[$x][$y] instead of $foo[$x]->[$y]
  • use warnings
  • Memoize
  • Test::More and Test::Simple
  • say
  • defined-or
  • use base (or, more recently, use parent)
  • yada-yada operator

Have I missed anything obvious? What new Perl features do you use most?

Training Debrief

I’ve spent a lot of the last seven days running training courses. It might be interesting to share some thoughts about how they went.

Last Saturday was Perl School 4. A week before the course I was a little worried about ticket sales, but I did a bit of marketing early last week and managed to more than double sales in a few days. In the end I had 27 people signed up.

Perl School is always enjoyable. I think that people often turn up with quite low expectations as it’s so cheap. So it’s fun to overturn those expectations and give them a day of high quality training. People obviously recognise that as I’m getting a lot of repeat business – at least one person has come along to three of the four courses so far.

Many of the courses I give are overviews of Perl at various levels. This one was just about DBIx::Class so it was great to be able to go into a lot more depth on a single topic. Of course, DBIx::Class is a great subject to cover and it was fun explaining its more powerful corners to a room of people who don’t know much about it.

I thought it went well. But don’t just take my word for it. I’ve been asking attendees to fill in feedback forms about all the Perl School courses and I’ve published a page summarising that feedback.

Then this week has been two two-day courses for flossUK. Two day courses give us time to include practical sessions so that people go home having actually tried out the techniques that I’ve taught – which nicely reinforces the lessons. I really enjoy those sessions as you really see lightbulb moments as people see how easy it is to use these tools. This afternoon, for example, it was great to see people getting a simple Catalyst application up and running in less than an hour. An hour later people were really impressed as I introduced them to Plack::Middleware::Debug and showed them how I could get detailed DBIC_TRACE output on the web page by making tiny changes to the application code. At least one person went away determined to reimplement a number of key applications in Catalyst as soon as possible.

And that, to me, is the joy of running training courses. It’s great to open people’s eyes to the possibilities that these new tools give them. I love to see them leave filled with renewed enthusiasm for the language.

Perl Books

The Perl community on LinkedIn is fascinating. It’s a great way to see how Perl is perceived and used outside of the echo chamber. And that’s a real eye-opener.

Here’s an example. Every few weeks (it seems) someone asks for advice on Perl books.At that point a few people will jump in with sensible suggestions. But for every reasonable suggestion, you’ll get three or four people suggesting something from this list:

  • Something horribly out of date. A lot of publishers stopped updating their Perl books about ten years ago. The most recent version of Perl Black Book that I can find is from 2001. Teach Yourself Perl in 21 Days was last updated in 2002. Perl By Example dates from 2007. I haven’t read any of these books, so I can’t comment on their quality, but I’d be really wary of suggesting that someone learns Perl from a book that is so out of date. Before the days of Amazon, these books would quietly disappear from bookshops, but now they always seem to be available.
  • Something out of date, but that used to be great. There are plenty of books that I would have recommended when they were current, but that I wouldn’t really want to recommend now. A lot of these are O’Reilly books that haven’t been updated recently – Advanced Perl Programming and The Perl Cookbook, for example. I’m sure there’s lots of good stuff in those books. But I really wouldn’t want anyone to read them if they didn’t already have enough experience to update the Perl to current best practices. I sometimes recommend these books myself, but only in specific circumstances. Whenever I run a course on OO Perl I mention Damian’s book. But I also point out that there have been a lot of advances in the area since the book was published. And, I have to confess, my own Data Munging with Perl fits firmly in this category.
  • Something of dubious provenance.  Most of these discussions will, at some point, attract a link to some dubious web site in Eastern Europe that contains the full text of O’Reilly’s various CD bookshelves. I know that these sites exist and I know that there’s nothing that O’Reilly can do about them, But I’d rather not see them mentioned on a professional site like LinkedIn. And quite apart from the copyright issues, there’s the fact that most of the books on these sites fall firmly into the first category above. They’re all over ten years old.
  • Some great tutorial on the internet. I’ve talked about the problems of old and dodgy tutorials before. And things definitely seem to be looking up. We are getting more good tutorials out there. But people still insist on sharing links to the tutorial that they learned from. Even if it’s appalling.

In a recent discussion someone said (and I’m paraphrasing) “just go to Amazon and look for the highest rated books”. I think there are two problems with that:

  1. The people who are rating beginner’s programming books on Amazon are usually the least qualified people to do that. Sure, they can tell how easy the book was to read and how well they picked up what the author taught them. But they have noway of knowing whether what they learned was accurate or useful.
  2. Amazon ratings last forever. But, as noted above, the quality of a technical book in fast-moving subject like programming falls over time. Perhaps Amazon ratings on programming books should have a half-life.

Here are some examples of things that you’re going to miss out on by using outdated Perl references.

  • say – We all love say, don’t we?
  • Lexical filehandles – Storing filehandles in lexical variables is great. I no longer have to worry about bareword filehandles being reused elsewhere in the code.
  • Defined or operator – There’s now no excuse for the $val ||= $default bug.
  • given/when – Perl has a switch statement. And it’s better than anyone else’s :)
  • Unicode support – Perl’s Unicode support is second to none. Except, if you’re using an old version of Perl in which case it’s a bit rubbish. And if you’re reading about Perl in an old book, then you won’t know about it.
  • State variables – Ok, I don’t use those every day. But when I need them, they make my code cleaner.

And then there are all the great CPAN modules that aren’t covered in books that were written before they were released. Would you really want to introduce someone to OO Perl without mentioning Moose?

My rules for recommending books are pretty simple. They should be books that I’ve found useful and they should have been published in the last few years. And given the falling numbers of Perl books that are published each year, that’s now a rather small number of books. Perhaps a dozen or so.

Am I being too harsh? Can beginners get something useful out of older Perl books? How do you decide whether to recommend a book to a colleague?

Why Corporates Hate Perl

This is a reprint of an old blog post.

A few years ago I was writing blog posts (semi-)regularly for O’Reilly. This is the one that probably got the most feedback. I’m reprinting it now because a) it’s pretty hard to find on the O’Reilly site and b) it’s relevant to a couple of conversations that I’ve had over the last few days.

Last week I was in Copenhagen for YAPC::Europe. One of the announcements at the conference was the location of next year’s conference which will be in Lisbon. The theme of next year’s conference will be “Corporate Perl”. And that (along with a couple of conversations last night) got me thinking about a talk that I’ll submit to next year’s conference which might well be entitled “Why Corporates Hate Perl”.

It’s not true, of course. There are a still large number of large companies who love Perl. I could probably work through to my retirement enhancing and extending systems that are written in Perl at many of the big banks in the City of London. There are, however, also many companies who are moving away from Perl for a number of reasons. Here’s one of the reasons that will be included in my talk.

I was talking to people from one such company last night. The Powers That Be at this company have announced that Perl is no longer their language of choice for web systems and that over time (probably a lot of time) systems will be rewritten in a combination of Java and PHP. Management have started to refer to Perl-based systems as “legacy” and to generally disparage it. This attitude has seeped through to non-technical business users who have started to worry if developers mention a system that is written in Perl. Business users, of course, don’t want nasty old, broken Perl code. They want the shiny new technologies.

And so, in a matter of months, the technical managers at this company have created a business environment where Perl is seen as the cause of most of the problems with the current systems. It’s an impressive piece of social engineering.

It’s also, of course, completely unfair. I don’t deny at all that this company (like many others) has a large amount of badly written and hard to maintain Perl code. But I maintain that this isn’t directly due to the code being written in Perl. It’s because the Perl code has developed piecemeal over the last ten or so years in an environment where there was no design authority which encouraged developers to think beyond getting their immediate task done. Many of these systems date back to this company’s first steps onto the internet and were made by separate departments who had no interaction with each other. It’s not really a surprise that the systems don’t interact well and a lot of the code is hard to maintain.

There are, on the other hand, a number of newer systems which are also written in Perl which follow current best practices in Perl development and are far easier to to maintain and enhance – as easy, I would contend, as anything written in the new approved languages.

It’s certainly true that this company has a large number of systems that need to be rewritten over the next few years. But throwing away all of the company’s accumulated Perl expertise and moving to new languages seems to be a step too far. Management are blaming Perl for the problems when really they should be blaming the management and design procedures that were in place (or, more likely, weren’t in place) when the code was originally written.

Many organisations are in the same situation, with large amounts of unwieldy Perl code. Ten or twelve years ago everyone was writing web systems in Perl and we were all making mistakes. We all have to deal with those mistakes but we’ve  hopefully, learned from them and can rewrite our systems to take account of everything that we’ve learned in the last ten years.

It’s too late for the company I’ve been talking about in this article. The anti-Perl social engineering has probably insinuated itself too deeply into the culture. It’s unlikely that Perl’s reputation can be rescued.

But if you have similar problems in your own company, then please try to ensure that blame is apportioned correctly and that you don’t use Perl as a scapegoat.

A couple of updates to the post. I did propose the talk to the next YAPC, but the proposal wasn’t accepted. And the company I talk about in the article is still employing a lot of Perl programmers – four years after this post was written.

A Cautionary Tale

I can never remember exactly how Time::Piece works. But that’s ok because I have documentation.

$ perldoc Time::Piece
No documentation found for "Time::Piece".

Huh?

$perl -v
This is perl 5, version 14, subversion 2 (v5.14.2) built for x86_64-linux-thread-multi
...

$ corelist Time::Piece
Time::Piece was first released with perl v5.9.5

$ perl -MTime::Piece -E'say $Time::Piece::VERSION'
Can't locate Time/Piece.pm in @INC (@INC contains: /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .).
BEGIN failed--compilation aborted.

So Time::Piece has been in the Perl core since 5.9.5. I’m running Perl 5.14.2 but I don’t have Time::Piece installed.

After ten minutes or so of head-scratching it came to me.

$ sudo yum install perl-core
Loaded plugins: langpacks, local, presto, refresh-packagekit
[ stuff snipped ]
---> Package perl-Time-Piece.x86_64 0:1.20.1-212.fc17 will be installed
[ more stuff snipped]

I’m running Fedora. The Fedora packagers have decided that they don’t need to install the whole standard Perl distribution as part of their standard installation. I don’t have a problem with that. I do have a problem with their naming conventions.

The minimal Perl installation that they include by default is in an RPM called “perl”. The full RPM that includes everything that a Perl developer would expect to see is called “perl-core”. Surely it’s obvious that those names are the wrong way round?

Isn’t there some way that the Perl 5 Porters can object to  this renaming of Perl?

I know I should be installing my own Perl with perlbrew. But I generally find that the system Perl works for everything that I need. There’s just this one thing that is guaranteed to trip me up every time I work on a new Fedora installation.

This is a public service blog post. Perhaps someone will come across it and be saved a couple of hours of confusion.

CGI.pm vs Templates

I’ve just been involved in a discussion on LinkedIn that I thought deserved a wider audience (I have no idea how well that link works if you’re not a member or or logged in to LinkedIn).

A couple of days ago, someone asked for advice on the best way to include HTML in a Perl program. They got a lot of good advice (basically – don’t do that, use a templating system instead). And then this morning, someone came in and said:

use CGI module, u don’t need to write a single HTML tag..just use CGI Methods…

I did a bit of a double take at that point. I thought we’d all stopped using CGI.pm’s HTML generation methods back in the last millennium. I replied saying that, but this chap was adamant that CGI.pm worked for him. He asked me to explain why I was so against his approach.

This was my reply:

I never said that it didn’t work. I just think that it’s a really limited way to build things and you’d be better off taking a more flexible approach right from the start.

My main objection is the separation of concerns. The logic of your application is separate to the display layer. It’s quite possible that the display of the application will need to change in the future. And that all becomes easier if the HTML is stored separately from code.

This leads to other problems too:

As I hinted at above, CGI.pm only has pretty basic support for HTML. It’s really hard to create a great-looking web application using CGI.pm. In fact, I wouldn’t be surprised if CGI.pm was largely responsible for the terrible web sites that the Perl community has been building for itself for most of the last twenty years.

Your project might well have HTML experts who know how to create good looking web pages. Expecting them to do it by editing Perl code is a bad idea. Front end developers need to know HTML, CSS and Javascript. Why force them to understand Perl as well?

Even if you’re the only developer on a project (as I often am for my personal projects) there are advantages to separating your work into “front-end task” and “back-end tasks”. I find it really helps me switch my mindset to the appropriate skills if I switch from editing a Perl source file to editing an HTML template.

It makes your Perl code more complicated than it needs to be. Seriously, try moving the HTML out of the code and into a template. See how much simpler your code becomes.

It’s unlikely that your CGI-generated pages will make up all of the web site. There will almost certainly be static pages too. If you use a tool like the Template Toolkit then you can generate static pages from the same set of templates that your CGI programs use. That way it’s easier to maintain a consistent look and feel across all of the pages on the site.

I’m not saying “don’t use CGI.pm” (well, I am a bit – but that’s a different story). The CGI bits of CGI.pm (for example the bits that give you access to the incoming parameters or let you set the outgoing headers) are great. It’s just the HTML generation stuff that I strongly recommend that you don’t use.

I’ve just remembered an article that I wrote in 2001 that might also help. The first half is about cookies, but the second half demonstrates using Templates instead of the HTML generation methods. I hope it shows how much simpler it makes things

See http://mag-sol.com/articles/cgi3.html

Does that help at all?

I think that pretty much sums up my objections. Did I miss anything obvious? Or am I completely wrong and many of you are still using the HTML generation methods in CGI.pm to do this?

The Perl community on LinkedIn is a fascinating place. I should really write a blog post about it.

What is Modern Perl?

I wrote an article for Josette called “What is Modern Perl?” In it, I talk about the different things that people might mean when they talk about Modern Perl and why it’s well worth buying a copy of the new edition of the camel book.

After a gap of twelve years, a new edition of Programming Perl (affectionately known as “the Camel book” to its many fans) was published earlier this year. That came as quite a surprise to many people who had given up on seeing a new edition.  What has changed in the new edition? Does the book now cover Modern Perl? The answer depends on what you mean by the term “Modern Perl”.

Of course it’s not a complete coincidence that the subject ties in nicely with my forthcoming Modern Perl for Non-Perl Programmers course.

Update: There’s a discussion of the article on Hacker News. It seems to be taking a predictable path.

Modern Perl for Non-Perl Programmers (Redux)

Six weeks ago I announced that I’d be running a free one-day course called “Modern Perl for Non-Perl Programmers” in August. Places on that course were fully booked in less than a day.

So I’ve decided to run to course again, two months later. It will still be at Google Campus, and the date is Saturday 6th October.

The original course was free, but I can’t afford to do that on a regular basis. I hope you’ll agree that £30 is still a bargain for a day of training.

As last time, most readers of this blog probably aren’t in the target audience for this course[1], but you might know people who would be interested in the course. Please pass on the link.

You can buy a ticket using at perlschool2.eventbrite.co.uk.

Eventbrite - Perl School: Modern Perl for Non-Perl Programmers

[1] Programmers of other languages who want to learn Perl.