Reviving WWW::Shorten

Last July I wrote a post threatening to cull some of my unused CPAN modules. Some people made sensible comments and I never got round to removing the modules (and I’m no longer planning to) but I marked the modules in question as “HANDOFF” and waited for the rush of volunteers.

As predicted in my previous post, I wasn’t overwhelmed by the interest, but this week I got an email from Chase Whitener offering to take over maintenance of my WWW::Shorten modules. I think the WWW::Shorten modules are a great idea, but (as I said in my previous post) the link-shortening industry moves very quickly and you would need to far more dedicated than I am in order to keep up with it. All too often I’d start getting failures for a module and, on investigation, discover that another link-shortening service had closed down.

So I was happy to hand over my modules to Chase. And in the week or so since he’s taken them over he’s been more proactive than I’ve been in the last five years.

  • He’s set up a p5-shorten organisation on Github to control the repos for all of the modules (all of my repos are now forks from those repos).
  • He has started cleaning up all of the individual modules so that they are all a lot more similar than they have previously been.
  • He has improved the documentation.
  • He has started to release new versions of some of the modules to CPAN.

All stuff that I’ve been promising myself that I’d get round to doing – for about the last five years. So I’m really grateful that the project seems to have been given the shot in the arm that I never had the time to give it. Thanks to Chase for taking it on.

Looking at CPAN while writing this post, I see that there are two pages of modules in the WWW::Shorten namespace – and only about 20% of them are modules that I wrote or inherited from SPOON. It’s great to see so many modules based on my work. However, I have no idea how many of them still work (services close down, HTML changes – breaking web scrapers). It would be great if the authors could all work together to share best practice on keeping up with this fast-moving industry. Perhaps the p5-shorten Github organisation would be a good place to do that.

Anyway, that’s seven fewer distributions that I own on CPAN. And that makes me happy. Many thanks to Chase for taking them on.

Now. Who wants Guardian::OpenPlatform::API, Net::Backpack or the AudioFile::Info modules?

Training Debrief

During the second week of February, I ran my (approximately) annual public Perl training courses in association with FlossUK. Things were organised slightly differently this year. Previously we’ve run two two-day “general purpose” courses – one on Intermediate Perl and one on Advanced Perl. This year we ran four courses, each of which were on a more specific technology. There were one-day courses on Moose, DBIx::Class and testing and a two-day course on web development.

Class numbers weren’t huge during the week. We had about six people on each of the days. That’s a large enough class to get plenty of discussion and interaction going but, to be honest, I’d be happier if we got a few more people in. The attendees were split pretty much down the middle between people who working for commercial organisations and people who worked for universities. I’m sorry to report that there were no women booked on any of the courses this year.

As is often the case on these courses most of the attendees had been using Perl for a long time and were pretty comfortable with some quite advanced Perl features. But, for various reasons, they simply hadn’t had the time to investigate some of the newer Perl tools that would have made their lives much easier. I often get people at these courses telling me that the best thing about the course is just having a day set aside where they can try out cool new technologies that they have heard of without the worry of someone calling them away to deal with some vital production issue (this, incidentally, is also why most trainers far prefer off-site training courses).

We started on day one with Moose. Most of them had used “classic” Perl OO techniques and were well aware of how baroque that can become. They were therefore very interested in the more declarative approach that Moose gave them. By the end of the day our Dalek class was using roles, traits, type coercion and many other useful Moose features.

Day two was DBIx::Class. Everyone was using a database of some kind and all of them were using DBI to interface to their database. I really enjoy introducing people like that to DBIx::Class. Once they’ve run dbicdump and have generated classes for their own databases most people’s eyes light up when they see ho much easier their code can become. As a bonus, this class contained no-one who needed to be persuaded of the benefits of defining foreign keys in your DDL and making use of referential integrity checks.

The third day was testing. I mean, it was about testing – not that it was particularly difficult. The class was full of people who knew the benefits of testing but who were maintaining large codebases with hardly any (in some cases no) tests. In the morning we looked at how simple the Perl testing framework is, did a quick survey of some useful testing modules on CPAN and even looked at writing our own testing modules. In the afternoon we expanded that to look at mocking objects and Test::Class. I think the most popular sections were when I introduced Devel::Cover and the concept of continuous integration. I encouraged them to write even just a few tests and to hook their test suite up to a highly visible Jenkins job. If you make your lack of test coverage obvious, then other people in the team can be encouraged to help improve it out of sheer embarrassment.

Thursday was the first day of the two-day course on web development. The first day concentrated on PSGI and Plack. We looked at what they are and how they make web development simpler. We also looked at ways to run non-PSGI applications in a PSGI environment in order to benefit from PSGI’s advantages. This seemed to really engage a couple of people in the class who used a practical session at the end of the day to start working to get their own legacy apps running under PSGI. I was particularly pleased when, the next morning, one of them told me that he had continued to work on the problem overnight and that he had got a huge system that used a combination of CGI and mod_perl working under PSGI. He was really happy too.

On the final day, we looked at web frameworks in Perl. The morning was all about Dancer2. We started by building a small web app and I showed them how simple it was to interface with a database and to add authentication to the system. Later on we added an API to the app so that it could return JSON or XML instead of web pages. Early in the afternoon, I took that a step further and demonstrated Web::Machine and WebAPI::DBIC. The rest of the afternoon was about Catalyst. We built another app (similar to the Dancer on from the morning) using the standard Catalyst tutorial as a basis. I’m not sure how well this went, to be honest. Following the simplicity of Dancer with the (relative) complexity of Catalyst wasn’t, perhaps, the best advert for Catalyst.

But, all in all, I think the week went really well. I sent a small but enthusiastic group of people back to their offices with a new interest in using Modern Perl tools in their day-to-day work. And, perhaps more usefully, I think that many of them will be getting more involved in the Perl community. A few of them said “see you at the LPW” as they left.

I’m running a half-day workshop on Modern Perl Web Development at  the FlossUK Spring Conference next month. Other than that, I don’t have any public courses planned. But if you think that your company would find my training useful, then please get in touch.

Why Learn Perl?

A couple of months ago I mentioned some public training courses that I’ll be running in London next month. The courses are being organised by FlossUK and since the courses have been announced the FlossUK crew have been running a marketing campaign to ensure that as many people as possible know about the courses. As part of that campaign they’ve run some sponsored tweets, so information about the courses will have been displayed to people who previously didn’t know about them (that is, after all, the point of marketing).

And, in a couple of cases, the tweet was shown to people who apparently weren’t that interested in the courses.

As you’ll see, both tweets are based on the idea that Perl training is pointless in 2016. Presumably because Perl has no place in the world of modern software development. This idea is, of course, wrong and I thought I’d take some time to explain why it is so wrong.

In order for training to be relevant, I think that two things need to be true. Firstly the training has to be in a technology that people use and secondly there needs to be an expectation that some people who use that technology aren’t as expert in as they would like to be (or as their managers would like them to be). Let’s look at those two propositions individually.

Do people still use Perl? Seems strange that I even have to dignify that question with a response. Of course people still use Perl. I’m a freelance programmer who specialises in Perl and I’m never short of people wanting me to work for them. I won’t deny that the pool of Perl-using companies has got smaller in the last ten years, but they are still out there. And they are still running successful businesses based on Perl.

So there’s no question that Perl satisfies the first of our two points. You just have to look at the size of the Perl groups on Facebook or LinkedIn to see that plenty of people are still using Perl. Or come along to a YAPC and see how many companies are desperate to employ Perl programmers.

I think it’s the second part of the question that is more interesting. Because I think that reveals what is really behind the negative attitude that some people have towards Perl. Are there people using Perl who don’t know all they need to know about it?

Think back to Perl’s heyday in the second half of the 1990s. A huge majority of dotcoms were using Perl to power their web sites. And because web technologies were so new, most of the Perl behind those sites was of a terrible standard. They were horrible monolithic CGI programs with hard-coded HTML within the Perl code (thereby making it almost impossible for designers to improve the look of the web site). When they talked to databases, they used raw SQL that was also hard-coded into the source. The CGI technology itself meant that as soon as your site became popular, your web server was spawning hundreds of Perl processes every minute and response times ballooned. So we switched to mod_perl which meant rewriting all of the code and in many cases the second version was even more unmaintainable than the first.

It’s not surprising that many people got a bad impression of Perl. But any technology that was being used back then had exactly the same problems. We were all learning on the job.

Many people turned their backs on Perl at that point. And, crucially, stopped caring what was going on in Perl development. And like British ex-pats who think the UK still works the way it did when they left in the 1960s, these people think the state of the art in Perl web development is those balls of mud they worked on fifteen or twenty years ago.

And it’s not like that at all. Perl has moved on. Perl has all of the tools that you’d expect to see in any modern programming language. Moose is as good as, if not better than, the OO support in any other language. DBIx::Class is as flexible an ORM as you’ll find anywhere. Plack and PSGI make writing web apps in Perl as easy as it is in any other language. Perl has always been the magpie language – it would be crazy to assume that it hasn’t stolen all the good ideas that have emerged in other languages over the last fifteen years. It has stolen those ideas and in many cases it has improved on them.

All of which brings us back to my second question. Are there people out there who need to learn more about Perl? Absolutely there are. The two people whose tweets I quoted above are good examples. They appear to have bought into the common misconception that Perl hasn’t changed since Perl 5 was released over twenty years ago.

That’s often what I find when I run these courses. There are people out there with ten or fifteen years of Perl experience who haven’t been exposed to all of the great Modern Perl tools that have been developed in the last ten years. They think they know Perl, but their eyes are opened after a couple of hours on the course. They go away with long lists of tools that they want to investigate further.

I’m not saying that everyone should use Perl. If you’re comfortable using other technologies to get your job done, then that’s fine, of course. But if you haven’t followed Perl development for over ten years, then please don’t assume that you know the current state of the language. And please try to resist making snarky comments about things that you know nothing about.

If, on the other hand, you are interesting in seeing how Perl has changed in recent years and getting an overview of the Modern Perl toolset, then we’d love to see you on the courses.


When I write replies to questions on StackOverflow and places like that recommending that people abandon CGI programs in favour of something that uses PSGI, I often get some push-back from people claiming that PSGI makes things far too complicated.

I don’t believe that’s true. But I think I know why they say it. I think they say it because most of the time when we say “you should really port that code to PSGI” we follow up with links to Dancer, Catalyst or Mojolicious tutorials.

I know why we do that. I know that a web framework is usually going to make writing a web app far simpler. And, yes, I know that in the Plack::Request documentation, Miyagawa explicitly says:

Note that this module is intended to be used by Plack middleware developers and web application framework developers rather than application developers (end users).

Writing your web application directly using Plack::Request is certainly possible but not recommended: it’s like doing so with mod_perl’s Apache::Request: yet too low level.

If you’re writing a web application, not a framework, then you’re encouraged to use one of the web application frameworks that support PSGI (, or see modules like HTTP::Engine to provide higher level Request and Response API on top of PSGI.

And, in general, I agree with him wholeheartedly. But I think that when we’re trying to persuade people to switch to PSGI, these suggestions can get in the way. People see switching their grungy old CGI programs to a web framework as a big job. I don’t think it’s as scary as they might think, but I agree it’s often a non-trivial task.

Even without using a web framework, I think that you can get benefits from moving software to PSGI. When I’m running training courses on PSGI, I emphasise three advantages that PSGI gives you over other Perl web development environments.

  1. PSGI applications are easier to debug and test.
  2. PSGI applications can be deployed in any environment you want without changing a line of code.
  3. Plack Middleware

And I think that you can benefit from all of these features pretty easily, without moving to a framework. I’ve been thinking about the best way to do this and I think I’ve come up with a simple plan:

  • Change your shebang line to /usr/bin/plackup (or equivalent)
  • Put all of your code inside my $app = sub { ... }
  • Switch to using Plack::Request to access all of your input parameters
  • Build up your response output in a variable
  • At the end of the code, create and return the required Plack response (either using Plack::Response or just creating the correct array reference).

That’s all you need. You can drop your new program into your cgi-bin directory and it will just start working. You can immediately benefit from easier testing and later on, you can easily deploy your application in a different environment or start adding in middleware.

As an experiment to find how easy this was, I’ve been porting some old CGI programs. Back in 2000, I wrote three articles introducing CGI programming for Linux Format. I’ve gone back to those articles and converted the CGI programs to PSGI (well, so far I’ve done the programs from the first two articles – I’ll finish the last one in the next day or so, I hope).

It’s not the nicest of code. I was still using the CGI’s HTML generation functions back then. I’ve replaced those calls with HTML::Tiny. And they aren’t very complicated programs at all (they were aimed at complete beginners). But I hope they’ll be a useful guide to how easy it is to start using PSGI.

My programs are on Github. Please let me know what you think.

If you’re interested in modern Perl Web Development Techniques, you might find it useful to attend my upcoming two-day course on the subject.

Update: On Twitter, Miyagawa reminds me that you can use CGI::Emulate::PSGI or CGI::PSGI to run CGI programs under PSGI without changing them at all (or, at least, changing them a lot less than I’m suggesting here). And that’s what I’d probably do if I had a large amount of CGI code that I wanted to to move to PSGI quickly. But I still think it’s worth showing people that simple PSGI programs really aren’t any more complicated than simple CGI programs.

London Perl Workshop Review

(Photo by Mark Keating)

Last Saturday was the annual London Perl Workshop. And, as always, it was a great opportunity to soak up the generosity, good humour and all-round-awesomeness of the European Perl community. I say “European” as the LPW doesn’t just get visitors from London or the UK. There are many people who attend regularly from all over Europe. And, actually, from further afield – there are usually two or three Americans there.

I arrived at about twenty to nine, which gave me just enough time to register and say hello to a couple of people before heading to the main room for Mark Keating’s welcome. Mark hinted that with next year’s workshop being the tenth that he will have organised, he is starting to wonder if it’s time for someone else to take over. More on that later.

I then had a quick dash back down to the basement where I was running a course on Modern Web Development with Perl. It seemed to go well, people seemed engaged and asked some interesting questions. Oh, and my timing was spot on to let my class out two minutes early so that they were at the front of the queue for the free cakes (courtesy of Exonetric). That’s just my little trick for getting slightly higher marks in the feedback survey.

After the coffee break I was in the smaller lecture theatre for three interesting talks – Neil Bowers on Boosting community engagement with CPAN‎ (and, yes, I’ve finally got round to signing up for the CPAN Pull Request Challenge), Smylers on Code Interface Mistakes to Avoid‎ and Neil Bowers (again) on ‎Dependencies and the River of CPAN‎ which was an interesting discussion on how the way you maintain a CPAN module should change as it becomes more important to more people.

Then it was lunch, which I spent in the University cafeteria catching up with friends.

After lunch, I saw Léon Brocard on Making your website seem faster, followed by Steve’s Man Publishing Pint, which turned out to be about publishing ebooks to Amazon easily – something which I’ve been very interested in recently.

The schedule was in a bit of a state of flux, so I missed Andrew Solomon’s talk on How to grow a Perl team‎ and instead saw Steve Mynott talking about Perl 6 Grammars. Following that, I gave my talk on Conference Driven Publishing (which is part apology for not writing the book I promised to write at the last LPW and part attempt to get more people writing and publishing ebooks about Perl).

Then there was another coffee break which I spent getting all the latest gossip from some former colleagues. We got so caught up in it that I was slightly late for Theo van Hoesel’s talk Dancer2 REST assured. I like Theo’s ideas but (as I’ve told him face to face) I would like to see a far simpler interface.

Next up was the keynote. Liz Mattijsen stood in for Jonathan Worthington (who had to cancel at the last minute) and she explained the history of her involvement in Perl and how she was drawn to working on Perl 6. She finished with a brief overview of some interesting Perl 6 features.

Then there were the lightning talks which were their usual mixture of useful, thought-provoking and insane.

Mark Keating closed the conference by thanking everyone for their work, their sponsorship and their attendance. He returned to the theme of perhaps passing on the organisation of the workshop to someone new. No-one, I think, can fail to be incredibly grateful for the effort that Mark has put into organising the last nine workshops and it makes complete sense to me that he can’t maintain that level of effort forever. So it makes sense to start looking for someone else to take over organising the workshop in the future. And, given the complexity of the task, it would be sensible if that person got involved as soon as possible so that we could have a smooth transition during the organisation of next year’s event.

If you’re interested in becoming a major hero to the European Perl community, then please get in touch with Mark.

There was no planned post-workshop event this year. So we broke up into smaller groups and probably colonised most of central London. Personally, I gathered a few friends and wandered off to my favourite restaurant in Chinatown.

I can only repeat what Mark said as he closed the workshop and give my thanks to all of the organisers, volunteers, speakers, sponsors and attendees. There’s little doubt in my mind that the LPW is, year after year, one of the best grass-roots-organised events in the European geek calendar. And this year’s was as good as any.