Categories
CPAN

Replacing CPAN RT

Two weeks ago, we learned that the CPAN Request Tracker was closing down early next year. I proposed a plan that CPAN authors could follow to ensure that their users can still find somewhere to report bugs in modules (and, perhaps more importantly, to see what bugs have already been reported in modules).

But that’s only part of the problem. In fact, it’s probably a minor part of the problem. If you’re an active CPAN author, then you probably already knew about the impending closure and had already made plans to deal with it. It’s likely that you had already moved your bug tracking to a new system. At the very least, you now know what the problem is and are considering the best way to deal with it before next March.

The far larger problem is the thousands of distributions that aren’t owned by active CPAN authors. What are we going to do about those?

How big is the problem? Well, the site cpan.rocks displays stats about CPAN. One of the panels on its front page shows a summary of the bugtracker information in CPAN distribution. You’ll see that 24,873 distributions (that’s 66% of them) have no bugtracker information included in their metadata. That’ll be for a number of reasons. Some of them will be distributions that haven’t been updated since alternative bugtrackers were supported by CPAN (originally, it was just assumed that everyone used the CPAN RT); some of them will be because the authors don’t know how to add the required metadata; and some of them (including most of my distributions) are missing it because the author just hasn’t got round to adding it yet. There will, of course, be many more reasons.

Some people will have read the recent news and will be galvanised into doing something about it (I fall firmly into that category) but others (and I’d suggest that it’s a large majority) either won’t hear about the change or won’t care about it. And why should they care? They were generous enough to donate some of their code to CPAN at some point. They don’t have any obligation at all to carry on maintaining it after they’ve lost interest in whatever project led to them writing that software.

Take, for example, Mail-Alias. That was released by someone called Tom Zeltwanger twenty years ago. He released three versions over a period of two months and then stopped. Who knows why. When he last updated the module, the CPAN search engine didn’t support alternative bugtrackers, so he never considered adding one. And that meant that the CPAN page for his module linked to the default bugtracker set up for the distribution on the CPAN RT. In the last fifteen years, four bugs have been reported against that module. But as Tom has moved on, nothing has been done about any of them. There are a lot of modules on CPAN in a similar situation.

But that leaves MetaCPAN (the current CPAN search engine) with a problem. Where does it send people who want to report a bug against an inactive module?

You might think that it doesn’t matter. But I disagree. Maybe I think that Mail::Alias would be the perfect module for a project I’m working on. Even before I start using it, it’s useful to be able to browse any existing bugs to see how they might affect my use of the module. And if someone later comes along and wants to take over maintenance of the module, then it’s useful for them to see any bugs that have been raised during the hiatus when the module was unmaintained.

So, I’m a big fan of having a default bugtracker for CPAN modules – even for ones with inactive authors. Which leads us to the question of where should that be. And I have a suggestion.

A few years ago, Micheal Schwern and Olaf Alders set up Gitpan. It’s an organisation which has a Github repo for every distribution on CPAN. And those repos each have a commit for every release of those distributions. Here, for example, is the repo for Mail-Alias – and you can see the three commits for the three releases I mentioned above.

So I’d like to suggest Gitpan as a suitable place to use as a default bugtracker for CPAN distributions. There are a couple of problems:

  • It looks like the auto-population of the commits stopped a few years ago. We’d need to work out how that works and catch up on the recent uploads.
  • None of the repos has the issue tracker turned on. But I expect that can be done with a relatively simple program that uses the GitHub API.

Of course, we also have the problem that some people object to using GitHub since it was taken over by Microsoft. But that’s fine, they can just point their bugtracker metadata to their preferred system.

The problem with the CPAN RT was that it needed too much maintenance – and the Perl NOC team is really overworked. Any self-hosted alternative seems likely to have the same problem eventually. So I’m all in favour of using a third-party alternative. And if you’re taking that route, then it makes sense (to me, at least) to use a third-party system that already has all (ok, most) of the repos set up.

I haven’t spoken to Schwern or Olaf about this, so I don’t know if there was some major problem that would stop this plan from working. But I think it’s worth looking at.

Categories
Miscellaneous

RT – Action Plan for CPAN Authors

CPAN RT is going away. CPAN authors have until the beginning of March to extract any useful information from it.

RT is the “Request Tracker”, a bug tracking system that is written by Best Practical. For almost as long as I can remember, anyone who uploads a module to CPAN gets a free ticket queue for their module at rt.cpan.org. MetaCPAN assumes that’s where people should report bugs in your module and helpfully adds an “issues” link that goes to the appropriate page in RT.

But now that system is going away. It will be switched off on the 1st March 2021. The Perl NOC team is spread pretty thinly and they just don’t have the resources to keep it running.

Gabor has published a video talking about what this means and some of the potential problems. But I thought it would be useful to work on a list of things that CPAN authors should be thinking about over the next three months.

Firstly, and most importantly, you’ll need somewhere new for people to report problems with your modules. For most people, that’ll be simple enough. If you host your code repos somewhere like GitHub, then you could just use the issue trackers that most of those services provide. If you host your own code repos (or don’t have public code repos), then you’ll need to find an alternative solution.

Next you’ll need to tell people where to find your new bug tracker. You do this by adding it to the metadata for your CPAN distribution. If, like most people, you provide a Makefile.PL in your distribution, then you’ll want to add a snippet like this to your code:

It’s likely that you already have a “resources” key in your data structure (containing, for example, a link to your code repo), in which case you just need to add the “bugtracker” key inside it. When you release this new update to CPAN, the “issues” link will change to point to your new bug tracker.

You then have the problem of dealing with the tickets that are currently in your RT queues. I suggest one or more of the following strategies:

  1. Go through the list and fix any that can be easily fixed. I found two like that when looking through my list this morning. If you’re releasing new versions of the modules (to add the new bug tracker information) then you might as well fix a bug or two at the same time.
  2. Look for tickets that can be closed. My list contains some very old tickets. I mean like fifteen years old. If someone had a problem installing one of your modules fifteen years ago and hasn’t followed-up more recently, then there’s a good chance that they no longer care about the solution. What I’ve been doing is to check on CPAN Testers to see if anyone else has seen a similar problem. If I see other reports, I know that it’s something that needs to be fixed. If there’s just the one in RT, then I close it with a message saying (paraphrased) “if this is still a problem that you’d like me to investigate, then please open a new ticket at [link to new bug tracker]”.
  3. Then you’re left with the tickets that you’d still like to address at some point. The Perl NOC team say that they’ll probably make a static archive of the old RT tickets available. But it would be good to get those tickets over to your new bug tracker. As I’m using GitHub for my new bug trackers, and that’s currently the most popular solution other than the CPAN RT itself, I’m hoping that someone cleverer than me will write some code that will make moving the tickets easy. But if nothing happens before the end of January, I might have to look into that myself.

I’ve got a bit of work to do in this area myself. Although I’ve been using GitHub for all of my CPAN code for a long time, I haven’t been advertising the fact that I’d prefer people to use GitHub for bug reporting too. So I need to update all of my modules with the new bug tracker information included. I’ll do that over the next few weeks.

I have a couple of tools that might help in this process. Firstly, I’ve just added to CPAN Dashboard a column that links to the module’s bug tracker if one exists. I can use that to know which modules need to be updated.

You could add yourself to CPAN Dashboard if you wanted to get that information for your modules. But if you don’t want to do that, I’ve written a really short program that you can use to find your CPAN distributions that don’t include the bug tracker information. It’s available as a Gist.

Have I missed anything? I mean, yes, I know there are a large number of unmaintained CPAN distributions that no-one will get round to updating. But this post was aimed at active CPAN authors. If there’s anything else you think we should be doing, then please let me know in the comments.

Categories
Miscellaneous

Down the rabbit hole

Blog posts are like busses. You wait months for one and then two come along on consecutive days!

Yesterday I wrote about how we didn’t need a blogging platform for the Perl community – all we really needed was a good-looking feed aggregator. I mentioned Perlsphere as one such aggregator.

Then Matthew commented, saying that Perlsphere looked a bit broken as Dave Cantrell’s posts from a few years ago frequently pop up there as new posts. I had a quick look at the problem and couldn’t quite work out what was going on. His web feed seems valid, but Perlsphere didn’t seem to recognise the dates of the posts.

Perlsphere is implemented using Plagger, a feed aggregator written by Miyagawa a very long time ago (back when it seemed that web feeds were going to conquer the world). It’s a pretty complex beast and it seemed possible that somewhere deep in its code, it was mis-parsing Dave’s web feed. So I cloned the repo and tried to work out what was going on.

But Plagger hasn’t been updated for a very long time. As you can see from the test results, it stopped passing its tests a long while ago (I suspect when the current directory was removed from @INC). I spent a brief time trying to get it working but, ultimately, decided it was too hard a job.

So I took another look at Perlsphere. And I decided that if it’s based on bit-rotten software, it’s not going to be very easy to maintain (which gave me flashbacks to blogs.perl.org!)

But there’s another Perl tool for aggregating web feeds. And I wrote it. It’s called Perlanet (and, boy, do I regret that name now). Back in the first decade of this millennium, I was fascinated by web feeds and the idea of using them to build local “planets” – a web site that aggregated web feeds that had interesting information about your local area. Of course, that was back in the days when everything had a web feed and it was simple enough to pull them together and create really quite interesting sites. These days, of course, web feeds are rather unfashionable and almost no-one thinks to add them on to their web site.

However, they still cling to life in the world of blogging. Mainly, I suspect, because blogging platforms added them fifteen years ago and just haven’t bothered to remove them yet. And it’s blog posts that we’re interested in here – so we’re still in with a chance of building something useful.

And that’s what I did over lunch. I give you Planet Perl, a new site for aggregating Perl blogs.

Currently, it only has four blogs in the configuration. But the config file is on Github and I’ll be happy to get pull requests adding other blogs.

I hope you find it useful.

Categories
Miscellaneous

Blogging for Perl

I think it was at YAPC Copenhagen in 2008 that a small group of us first discussed the idea of building a shared blogging platform for the Perl community. It was over a year later that we launched blogs.perl.org.

I remember a lot of discussions over that time where we tried to thrash out exactly what we wanted to build. I know that one of my main drivers was that I wanted to replace the journals feature of use.perl. For those of you too young to remember, use.perl was a Perl community web site from the dawn of time. The site ran Perl news on its front page, but users could also have their own journal. For a few years, we had a nice little blogging community on the site. It used the same software as Slashdot and, like Slashdot, was looking pretty dated at the time. I remember being particularly disappointed that you couldn’t put images in a journal entry.

So that was certainly where I started from when planning blogs.perl.org. I wanted a modern blogging platform to replace use.perl. Other people wanted slightly different things. At least one person argued strongly that what we really needed was a nice-looking blog aggregator and that people could do their actual blogging on their own sites (or some other blog provider).

But, as you can see, that point of view lost out and we implemented a full blogging system using Movable Type.

Ten years on, I think that was a mistake. I think that having a community blogging platform for Perl is actively harmful to Perl.

I’ll explain why a bit later, but first let’s talk through a bit of more recent history of the site.

It’s well-known, I think, that the site is not is a good state. I’m happy to explain what happened there, but it’s a long and rather dull story. There have been a couple of attempts to build replacement systems but they both stalled before they were complete.

So we’re left in a situation where the site is broken and no-one really has the time or the expertise to fix it. And attempts to replace it seem doomed to failure.

Earlier this year I decided that the situation was untenable and that I wanted to close the site down. I spoke to Aaron and he pretty much agreed with me. We decided that we’d give people plenty of notice to take their blogging elsewhere and, rather than closing the site completely, we would make it read-only (so all the existing content would still be there). We ran the idea past the rest of the loose “management team” and Aristotle spoke up, saying that he would rather take the site over himself and try to improve matters. And that’s what happened. Over the last few months, Aaron and I have extricated ourselves from running the site and it’s all now handled by Aristotle.

I wish him all the best in those endeavours and really hope he manages to make the site better than it was (he’s already making great progress in removing loads of old spam blogs from the site).

But, really, I think that my original plan was a better idea. I don’t want people to blog about Perl on a Perl community site. I want people to blog about Perl on sites where people blog about other languages and technologies.

I’ve been giving talks about the need for the Perl community to break out of its echo chamber for almost as long as I’ve been part of the community. But I think it’s important. It’s a kind of low-key marketing. If you’re talking about your cool Perl project on a Perl community web site then only people who are looking for Perl articles will find it. But if you blog about it on a general programming web site then a) you’ll get a far bigger audience and b) some of that audience might say “oh! I never knew Perl could do that – perhaps I’ll give it a closer look.”

And that’s why I think blogs.perl.org (and, probably, use.perl before it) are harmful to Perl. It encourages Perl bloggers to blog for the Perl community – when it would be far better to get your blog posts in front of more people. Perl people will still read your Perl posts (through Perlsphere perhaps, or post the link to /r/perl or the Perl Community Facebook group) but other people will see them too. On blogs.perl.org, you’re pretty much guaranteed that only Perl programmers will see your posts.

To be honest, I don’t understand why people still use blogs.perl.org. Until Aristotle’s work bears fruit it’s painfully broken. What do you get that’s worth putting yourself through all of that pain? Why not just blog somewhere else and submit your web feed to Perlsphere? Or blog where other programmers blog – somewhere like dev.to, perhaps (I’ve started publishing my more technical blog posts there).

I think it will be great if Aristotle gets blogs.perl.org working well. I really want that to happen. But it would be even better if he didn’t have to. If people stopped using it; if they went off and started spreading their interesting Perl blog posts all over the web.

Break out of the echo chamber. Find yourself a bigger audience. Spread the word beyond the Perl community.

Or, alternatively, explain to me why it’s so vital that the Perl community has its own centralised blogging platform.

Categories
Miscellaneous

The Best of Perl Hacks

What do you do when you’re stuck inside because Coronavirus means that your country is in lockdown? Well, you write a book, of course. Or, to be more accurate, you cobble together fifty or so old blog posts into a book.

So that’s what I’ve done. Now you can read some of your favourite Perl Hacks blog posts in a handy Kindle book. Other ebook marketplaces are, of course, available – but I haven’t had the time to make a version that’s available from anywhere else yet. That might follow if enough people ask for it.

The book is, predictably, called The Best of Perl Hacks and it’s available from Amazon now (that link goes to the UK store, but it should be available on all Amazon sites).

Please buy it, read it and let me know what you think.