Net::Songkick

Sometimes it’s good to just take a new idea and hack on it for a couple of hours to see what happens. That’s what I’ve done this evening.

I’ve been using Songkick for a while. Songkick is a web site that tracks users’ attendance at gigs. I’ve been tracking the gigs I’ve been going to as well as trying to fill in over thirty years of old gigs.

I’ve known for some time that Songkick has an API, but until yesterday the API has been for invited partners only. But yesterday on Get Satisfaction they announced that it was now public.

The public API doesn’t do that much yet. There are only four documented calls. But it’s already useful and I’m sure it will gain more functionality quickly.

And this evening I’ve spent some time writing a very simple Net::Songkick module. This is really just a proof of concept, but I’ve got big plans for improving it. And I can already write a useful program like this:

If you have any interest in gigs then I highly recommend Songkick to you. And if you like attending gigs and hacking Perl (although not necessarily at the same time) then why not give Net::Songkick a try. All you’ll need is an API key from Songkick.

There’s a Github repository too; if you feel like hacking on it…

Learning About Traits

I’ve been teaching basic Moose in my training courses for several years now. And, as I’ve mentioned before, I’ve been slowly converting some of my CPAN modules to use Moose. But there are still bits of Moose that I haven’t really needed to get to grips with.

One such area is Moose’s support for Traits. Oh, I knew vaguely what they were and I understood why you might use them. But I’d never implemented a system using traits, so my knowledge about how you’d actually use them was a bit shaky to say the least.

But over the last few days I’ve learned quite a lot about how to use traits. And I’ve had to learn it pretty quickly.

It all started a few weeks ago when I got a github pull request from Oliver Charles. Oliver had taken a fork of my Perlanet repository and had massively refactored the codebase so that all the clever bits were implemented as traits.

What this means is that the core Perlanet code is pretty dumb. In order to do anything really useful with it to need to add in some traits. There are traits to read the configuration from a YAML file, traits to carry out various kinds of cleaning of the input and a trait to produce the output using the Template Toolkit. There are also traits to handle caching and OPML generation. All in all it makes the code far nicer to work on.

Oh, and there’s a Perlanet::Simple module which uses all of the traits required to implement the Perlanet behaviour that users currently expect.

There were a few problems with Oliver’s initial version. Some of the dependencies weren’t quite right. But we soon fixed that and last week I finally released Perlanet 0.47 which implemented these changes.

Then I installed it on the server which hosts most of my planets. And everything broke.

So I’ve spent a lot of the weekend fixing these issues. Part of it was that Oliver’s changes assumed some configuration file changes that I hadn’t implemented. I changed his changes so that they worked with the existing configuration settings (we don’t want users having to change configuration files unnecessarily). Other changes were harder to track down. I particularly enjoyed one where no feeds were fetched unless the user turned on OPML support. I learned a lot about how traits worked by tracking that one down.

But it all seems to be fixed now. Perlanet version 0.51 is available on CPAN and it will hopefully be a lot easier to customise to your needs. I hope we’ll see a number of other Perlanet traits appearing over the next few months.

And, most importantly, I’ve learnt a lot about traits. I think I understand them now.

If you want to learn traits, I can highly recommend asking someone to implement traits in one of your projects. And it’s even better if they do it in a slightly broken way so that you need to debug it.

Ironman and XML::Feed

Sam Graham complains that since the Ironman feed switched to using Perlanet, the entries have been “mangled”. By that he means that in some cases any HTML in feed entries is lost.

I think they’re running up against this bug in XML::Feed (which is one of the modules that Perlanet uses to process the feeds it subscribes to). There’s a patch in the bug report that I’ve applied to my local installation of XML::Feed and it seems to have fixed the problem.

Until the author releases a new version of XML::Feed that includes this patch, I recommend that anyone using XML::Feed (and that includes everyone using Perlanet) applies the patch for themselves.

More RPM Stuff

It’s been a while since I wrote anything here. if anyone is keeping score I’ve probably failed the Iron Man challenge of posting something every ten days.

Don’t have much to add here either but I thought some of you might be interested in a quick tweak I made to my spreadsheet of CPAN RPMs available for Fedora. It now lists all of the RPMs available across all of the repositories that I use and shows you which version of the module is available. I’ve also added the current CPAN version for all of these modules.

This gives me the information I need to do a few things that I’ve wanted to do for a while. In particular I should be able to script the automatic removal of RPMs from my repository when the official Fedora repository catches up with the version I’m carrying. I can also easily identify CPAN modules where the latest Fedora version (from any of the repositories) is lagging behind the CPAN version.

As always, the code is available on Github and patches are very welcome.

Update: And here’s another spreadsheet covering CPAN RPMs available for Centos.

META.yml and Building RPMs

An email has flooded in. It was in response to my piece about Building RPMs from CPAN Distributions and it was from Andreas Koenig. Andreas runs PAUSE, which is the service CPAN authors use to upload stuff to CPAN, so he knows what he’s talking about when it comes to CPAN (and many other matters). He says this:

It’s not correct that the META.yml contains the exact list of dependencies. The META.yml is not the authoritative source for them. The reason behind is that dependencies do differ across architectures. Exceptions to this rule may declare dynamic_config=0. In order to obtain the real list of dependencies you must run your Makefile.PL or Build.PL. Recent Module::Build provides a MYMETA.yml after Build.PL has run. You could use that instead. MakeMaker always had the dependency as a comment in the Makefile.

He is, of course, right. My previous article skipped blithely over some of the more gnarly corners of this problem. I should point out that Gabor and I discussed some of these over the weekend but it’s almost certainly worthwhile going into a little more detail.

It’s true that a static META.yml file can’t deal with all of the possibilities. Here are a number of examples of areas that need to be looked at in more detail.

Environmental differences
This is the area that Andreas is talking about. And the Padre problem I mentioned on Monday is one example of this. Padre runs on several different platforms. And some dependencies will only be required on certain platforms. For example the Win32::API module is only required if the module is being installed on Windows.

But it’s not just different operating systems or architectures that cause problems like this. If you’re trying to use Plack on a server with Apache 2 installed, you’ll need Apache2::Request. If your server has Apache 1 installed you’ll need Apache::Request. In each case, you won’t need the request module for the Apache version that you aren’t using. As things stand, the META.yml for Plack doesn’t list either of the Apache request modules, but a more intelligent system could work out which one of them is required and add that one to the list of dependencies.

“Choose One” requirements
Some modules exist simply as a way of allowing the user to choose between one of a number of implementations of a feature. A good example is JSON::Any. There are (at least) three different JSON modules on CPAN – JSON, JSON::DWIW and JSON::XS). Different systems will have different ones installed. JSON::Any allows a program to use any JSON module and not care which of them is installed. But how do you model that dependency? If you make any (or all) of the supported modules a required dependency, you rather miss the whole point of the module. JSON::Any’s META.yml ignores the problem and leaves it to the Makefile.PL to work out what to do. The Fedora RPM for this module takes a weird approach and makes JSON::XS a required dependency. Even if META.yml could support this mode of working, RPM doesn’t have this feature.

Added features
Some modules have optional requirements. That is, if certain other modules are installed then the module gains more features. One example is the Template-XML distribution. Template-XML contain a plugin (Template::Plugin::XML) for the Template Toolkit. Template::Plugin::XML is a wrapper around a number of XML processing modules. If a particular module (for example, XML::DOM) is installed then Template::Plugin::XML allows the user to uses XML::DOM for XML processing. It works similarly for XML::LibXML, XML::Parser, XML::RSS, XML::Simple and XML::XPath. None of them are required, different functionality is turned on for each one that is installed. You don’t have to configure Template::Plugin::XML at all to work with these modules. It just works if a particular module is installed. If, at a later date, you remove that module then Template::Plugin::XML removes the features supported by that module.

This seems to be somewhere where I have philosophical differences with the Fedora RPM packaging team. I believe that all of these modules should be seen as optional and there for shouldn’t be listed as dependencies in the META.yml or the RPM. The Fedora team disagrees. They want each RPM to depend on all of the modules it needs in order to have as many features as possible, The Template-XML RPM therefore requires all of the XML processing modules I listed above. That seems wrong to me.

META.yml supports the concept of  “recommended modules”. I think that these optional modules should be listed there. But I don’t believe that RPM has a similar feature.

So there are a few problems that I see with the META.yml approach. In the face of these issues I should probably back down slightly from my previous position that META.yml is the definitive way to get a list of dependencies. What I now believe is that parsing META.yml will give you a better position to start from than parsing the Perl code and extracting all of the “use” statements.

But I hadn’t previously heard of the MYMETA.yml that Andreas mentioned in his email. That’s certainly a way to get round the environmental differences I listed above. I don’t think it solves the other two issues though.

Are there any other corner cases that I’ve missed. Does anyone else have any opinions on building RPMs from CPAN distributions?