Modern Web Development with Perl and Dancer2

Here are some more details of the Modern Web Development with Perl and Dancer2 course that I’ll be running in Cluj-Napoca on the day before YAPC Europe.

The course runs a full day (that’s six hours – in four 90-minute sessions with breaks in between). It’s a hands-on course – you’ll need to bring a laptop and closer to the time I’ll email attendees with details of the software they will need to have installed. Like all of the pre-conference training, the course will take place at Cluj Hub on Tuesday 23rd August.

Over the course of twelve steps, we’ll build a simple Todo list program. We’ll be using a number of modern web development techniques (not just Perl) in order to make the app look really shiny and modern.

The twelve steps we will be taking are as follows:

  1. Set up a basic Dancer2 app
  2. Make it look nicer with the addition of the Bootstrap CSS framework
  3. Use Plack Middleware to serve static content more efficiently
  4. Display some data in our app
  5. Get the data from a database
  6. Return the data as JSON and display it using Mustache
  7. Use jQuery to show/hide completed items
  8. Mark items as completed
  9. Add new tasks to the app
  10. Add user login
  11. Edit and delete tasks
  12. Add tags to tasks and filter the display on those tags

If there’s time left at the end, we’ll discuss other useful enhancements that we might want to make to the app – and perhaps even try adding them.

We’ll be using the following Perl tools:

And the following non-Perl tools:

Usually, a course like this would cost around £300. But because it’s at YAPC and the sponsors are so generous, we can offer it for the heavily discounted price of 100€.

Cluj Hub sounds like a fabulous venue for the training courses and I’m sure that the day will be a lot of fun. Perhaps more importantly, I’m also sure that attendees will come away with some useful skills to add to their CVs.

Tickets are on sale now. Please buy quickly – before they sell out.

Updating Ogg::Vorbis::Header

Last night, I uploaded a new version of Ogg::Vorbis::Header – a CPAN module that hasn’t been updated since 2003 and which I strongly suspect no-one at all uses any more. You might be interested to hear what I did or why I did it.

About a year ago, I wrote about the dashboard I had written for my CPAN modules. It’s a simple page that pulls together information about all of my modules and, among other things, shows me how they are doing on Travis CI and Coveralls.

One of the aims of the dashboard was to encourage me to do more work ensuring that my CPAN modules were working well and had good test coverage. The idea was that if I’m constantly looking at a page which shows how rubbish the test coverage for a module is, then I’ll be more motivated to fix it. Of course, that only works if I’m constantly looking at the dashboard and, to be honest, over the year since I built it I really haven’t taken much notice of it.

But recently, I was reminded of its existence as I updated it to remove some modules that I’ve handed over to other people and to add a couple of new modules I had released. And, in doing that, I took a closer look at it and my attention was drawn to AudioFile::Info::Ogg::Vorbis::Header. This is the only one of my modules which doesn’t even build on Travis. Clearly, more investigation was needed. But, before we get into that, it’s probably worth making a brief digression to explain what the module does.

Some of you will be too young to remember this, but there was a time back in the early middle ages of internet history (so, perhaps, fifteen years ago) when not everyone listened to music as MP3s. Back then, one of the biggest sources of digital music for many people was ripping their existing CDs (ask your parents – they might still have a CD or two they can show you). And when ripping music from CDs we had a choice of formats. Most people (even then) were using MP3, but some of us took the ideologically superior option of ripping to Ogg Vorbis. The main reason was that MP3 format was patented, but Ogg Vorbis was completely free.

All of this meant that in the first five or six years of the 21st century, I ended up with hundreds (maybe thousands) of Ogg Vorbis files on my hard disk. This immediately gave me problems as it dramatically limited the devices that I could play my music on. For example, it probably explains why I’ve never owned an iPod.

But I also… er… acquired a number of MP3s over the same time. And, being a geek, at times I wanted to write programs that gathered information about all of my music, no matter what format it was stored in. There were modules on CPAN for dealing with MP3s and there were modules on CPAN for dealing with Ogg Vorbis files. But (as is so often the way with these things) all of these modules worked in completely different ways.

And that’s I wrote the AudioFile::Info set of modules. They acted as a wrapper around the various modules for dealing with the different audio formats and gave them all the same interface. It meant that I could write programs that got information from any of my audio files and I didn’t need to care what format they were in. Think of them a bit like a DBI for audio file formats.

Of course, no-one else ever had any use for them. And soon afterwards MP3 became the de-facto standard for digital audio and Ogg Vorbis was relegated to the same (virtual) drawer as Betamax. I’d say that no-one uses it any more – but I suspect there are actually about eight users left and they would all write comments telling me that I was wrong.

None of the AudioFile::Info modules have been updated for a very long time, because no-one uses them any more and no-one cares about them. I’d remove them from CPAN, but that goes against my pack-rat nature.

All of which leaves me annoyed by the failure of AudioFile::Info::Ogg::Vorbis::Header to build on Travis. So a couple of weeks ago, I investigated further. And, to my delight, I found that it wasn’t my fault. Actually, it was the underlying module (Ogg::Vorbis::Header) that had the problem. That module no longer built successfully on modern Perls. And that failure prevented my module from building on top of it.

The problem is described in this RT ticket. Basically, there was some very funky syntax in a test. Syntax that became a fatal error through some parser fixes in Perl 5.22. The test looked like this:

When it should have looked like this:

In the RT ticket, H. Merijn Brand gives a good explanation of how the test ever passed – but try working it out for yourself before looking.

So, anyway, I knew what the problem was and I knew how to fix it. My next step was to pass this information on to the author of the module. I emailed him a couple of weeks ago, offering to make the fixes myself if he was too busy (or too uninterested) to do it himself. I got no reply, so at the end of last week I emailed the CPAN Powers That Be explaining the situation and asking for co-maintenance rights on the module so that I could fix the problem. They granted my request – which is why the new version was released yesterday. I can already see that the tests for this version look a lot healthier than the ones for the previous version.

Healthier, but still not as healthy as I’d like them. Within an hour or so of my release hitting CPAN, this issue was raised. The Makefile.PL uses Inline::MakeMaker and I can’t work out how to make that work, given that the “use” statement is executed long before any of the configuration code that tells the build tools what modules are required. If you have any suggestions, please let me know (or send me a pull request). I’m a bit out of my depth when it comes to Inline-based modules.

There are a few other things that I might fix. It’s an old-style distribution where there are no /lib or /t directories. It’s all in the top-level directory. I’m very tempted to move all that stuff around.

But really, I should get back to ensuring that my module builds successfully now.

Update: On Sunday, I released another version of Ogg::Vorbis::Header which fixed the packaging problems. But it still hasn’t solved my Travis-CI woes and there are still a disappointing number of failures on CPAN testers (actually they are “unknown” results rather than real failures – because there are still cases where the module won’t even build).

The problem is the underlying C libraries. Ogg::Vorbis::Header relies on the libogg and libvorbis libraries. And a large number of people aren’t going to have those libraries installed (the Travis-CI environment certainly doesn’t). Trying to build the module on a system that doesn’t have those libraries is doomed to failure.

The solution is, I suspect, to build Alien modules for these two libraries. But that’s something that I know very little about. I doubt I’ll have the time to learn a whole new area of CPAN packaging until after YAPC Europe at the earliest. Of course, if some kind person who knows more about this area than me was to offer to help (or even to produce the Alien modules for me) then that would make me very happy 🙂

Dancing in Cluj-Napoca

Over the last couple of weeks I’ve been running a poll to decide which training course to run at YAPC Europe in August. Thank you you the people who voted in the poll.

I’ve just closed the poll and the results are pretty clear. In Cluj-Napoca I’ll be running a course on Modern Web Development with Perl and Dancer. That was the most popular choice with 31% of the vote. Moose and “Other” were the second most popular choice with 19% each. Here are the full results.

Title %
Modern Web Development with Perl and Dancer 31%
Object Oriented Programming with Perl and Moose 19%
Other 19%
Database Programming with Perl and DBIx::Class 16.7%
Testing Perl Programs 14.3%

The “other” responses were interesting. A couple of people asked for Perl 6 training (and I think their wish might be granted – but I don’t want to pre-empt announcements by other people). Someone wanted “Advanced Testing”. Someone wanted “nodejs”. Someone wanted the web training, but with Mojoicious instead of Dancer (I’ve never used Mojolicious so I’m not the right person to be running that course). Oh, and we had one vote each for “all of the above” and “none of the above”. Perhaps some suggestions there if someone else wants to run a training course at the conference.

I also asked about cost. And those answers were interesting too. I guess it’s no surprise that people gravitated towards the lower numbers (“how much do you want to pay?”; “as little as possible, obviously!”) but it wasn’t the lowest price that was most popular. The most popular choice (with 42.5%) was 100 €. We haven’t worked out the details of the pricing yet (we need to see what the venue will charge us) but I hope to get it as close to 100 € as possible.

Speaking of the venue, we do know where the training will be. It’s will be at Cluj Hub which is a really great-looking co-working and events space in Cluj. As I said above, we’re still working out the details (costs, catering, stuff like that) but there are some fabulous plans being discussed and I hope to be able to annouce full details soon.

And what about the class itself? Well, I’m glad you asked. It’ll be a hands-on course and over a day we’ll build a complete (and, hopefully) useful little web application using a number of modern web technologies. The back-end will (of course) be Perl (specifically Dancer2) but we’ll also be using Bootstrap, jQuery, Mustache and more.

It’s a course I’ve been working on intermittently for some time and I’m really pleased with how it’s shaping up. I think you’ll enjoy it too.

So when you’re planning your trip to Cluj-Napoca, please consider travelling a day early and coming to the training. It’ll be a lot of fun.

Once the last details have been worked out, we’ll add it to the YAPC web site so you can book it.

In summary:

Modern Web Development with Perl and Dancer
One-day, hands-on course
Cluj Hub, Cluj-Napoca, Romania
Tue 23 August 2016
Cost: TBA (but as close to 100 € as possible)

Taming QMail

I run my own email server. It uses QMail. I realise there are at least two problems there – all of the cool kids have been using Gmail for their email since approximately forever, and who the hell uses QMail anyway?

Like most of these situations, it’s that way for historical reasons. And, of course, I’d love to change the current situation but it works well enough most of the time that the occasional pain it gives me isn’t worth the pain of changing the set-up.

So there I am. One of about three people in the world who are still running an email server using QMail. And that occasionally gives me problems.

One problem is the management of the mailing queues. The tools that QMail gives you for checking the contents of the queues and deleting any unnecessary messages are rather primitive. It seems that everyone uses third-party tools. A couple of years ago, I had a WordPress installation that was compromised and was used to send thousands of email messages to random people. The outgoing queue was full of spam and the incoming queue was full of bounce messages. I stopped QMail and started looking for a way to cleanly delete all the bad messages while retaining the tiny number of legitimate ones.

I discovered a program called qmHandle which did exactly what I wanted. It enabled me to remove messages from both queues that matched various criteria and before very long at all, I was back in business (having also cleaned up the WordPress installation and tightening its security).

The qmHandle program was written in Perl. And I always had it in the back of my mind that at some point I’d revisit the program and give something back by fixing it or improving it in some way. A few months ago I found time to do that.

I started by looking at the code. And realised that it had pretty much been written to be as hard to maintain as possible. Ok, that’s probably not true, but it certainly wasn’t written to be particularly easy to understand. What the original author, Michele Beltrame, created was really useful, but it seemed to me that he had made it far harder for himself than he needed to.

So I had found my project. I wanted to hack on qmHandle. But before  I could do that, I needed to rewrite it so that it was easier to work on. That became something that I’d hack on in quite moments over a few weeks. The new version is on Github. I started by importing the original version, so it’s interesting to read the commit history to trace the changes that I made. I think there were three main areas where I improved things.

  1. Splitting most of the logic out into a module. I say “most”, but it’s actually all. The command-line program is now a pleasingly simple:
  2. Improving (by which I mainly mean simplifying) the logic and the syntax. I moved a few variable declarations around (so their scope was smaller) and renamed some so their meaning was more obvious. Oh, and I added a couple of useful CPAN modules – Term::ANSIColor and Getopt::Std.
  3. Using Moose. Switching to an OO approach was a big win in general and Moose made this far easier than it would otherwise have been. At some point in the future, I might consider moving from Moose to Moo, for performance reasons.

For a few weeks now, I’ve been using the revised version on my email server and it seems to be working pretty much the same as the original version. So it’s time to set it loose on the wider world. This afternoon, I released it to CPAN. I’ve said above that the number of people using QMail to handle their email is tiny. But if you’re in that group and you want a more powerful way to manage your mail queues, then the new version of qmHandle might well be useful to you.

There are a few things that I know I need to do.

  1. More tests. The main point of moving most of the code into a module was to make it easier to test. Now it’s time to prove that. The current test suite is tiny. I need to improve that.
  2. Configuration. Currently, the configuration is all hard-coded. And different systems might well need different configuration (for example, the queues might be stored in a different directory). There needs to be a simple way to configure that.
  3. Bug fixes and improvements. This was, after all, why I started doing this. I don’t know what those might be, but I’m sure there are ways to improve the program.

I hope at least someone finds this useful.

Training in Cluj – The Poll

A couple of weeks ago, I mentioned that I was planning to run a one-day training course the day before YAPC Europe in Cluj-Napoca this year. There have been a few discussions of my ideas in a various forums, so now it’s time for the next stage.

Below, you’ll see a simple questionnaire. Please use it to give your feedback on what course you would like me to run – and how much you think it should cost.

I’ll collate all of the responses in a couple of weeks and make an announcement about what I’m going to do.