Categories
CPAN

Replacing CPAN RT

Two weeks ago, we learned that the CPAN Request Tracker was closing down early next year. I proposed a plan that CPAN authors could follow to ensure that their users can still find somewhere to report bugs in modules (and, perhaps more importantly, to see what bugs have already been reported in modules).

But that’s only part of the problem. In fact, it’s probably a minor part of the problem. If you’re an active CPAN author, then you probably already knew about the impending closure and had already made plans to deal with it. It’s likely that you had already moved your bug tracking to a new system. At the very least, you now know what the problem is and are considering the best way to deal with it before next March.

The far larger problem is the thousands of distributions that aren’t owned by active CPAN authors. What are we going to do about those?

How big is the problem? Well, the site cpan.rocks displays stats about CPAN. One of the panels on its front page shows a summary of the bugtracker information in CPAN distribution. You’ll see that 24,873 distributions (that’s 66% of them) have no bugtracker information included in their metadata. That’ll be for a number of reasons. Some of them will be distributions that haven’t been updated since alternative bugtrackers were supported by CPAN (originally, it was just assumed that everyone used the CPAN RT); some of them will be because the authors don’t know how to add the required metadata; and some of them (including most of my distributions) are missing it because the author just hasn’t got round to adding it yet. There will, of course, be many more reasons.

Some people will have read the recent news and will be galvanised into doing something about it (I fall firmly into that category) but others (and I’d suggest that it’s a large majority) either won’t hear about the change or won’t care about it. And why should they care? They were generous enough to donate some of their code to CPAN at some point. They don’t have any obligation at all to carry on maintaining it after they’ve lost interest in whatever project led to them writing that software.

Take, for example, Mail-Alias. That was released by someone called Tom Zeltwanger twenty years ago. He released three versions over a period of two months and then stopped. Who knows why. When he last updated the module, the CPAN search engine didn’t support alternative bugtrackers, so he never considered adding one. And that meant that the CPAN page for his module linked to the default bugtracker set up for the distribution on the CPAN RT. In the last fifteen years, four bugs have been reported against that module. But as Tom has moved on, nothing has been done about any of them. There are a lot of modules on CPAN in a similar situation.

But that leaves MetaCPAN (the current CPAN search engine) with a problem. Where does it send people who want to report a bug against an inactive module?

You might think that it doesn’t matter. But I disagree. Maybe I think that Mail::Alias would be the perfect module for a project I’m working on. Even before I start using it, it’s useful to be able to browse any existing bugs to see how they might affect my use of the module. And if someone later comes along and wants to take over maintenance of the module, then it’s useful for them to see any bugs that have been raised during the hiatus when the module was unmaintained.

So, I’m a big fan of having a default bugtracker for CPAN modules – even for ones with inactive authors. Which leads us to the question of where should that be. And I have a suggestion.

A few years ago, Micheal Schwern and Olaf Alders set up Gitpan. It’s an organisation which has a Github repo for every distribution on CPAN. And those repos each have a commit for every release of those distributions. Here, for example, is the repo for Mail-Alias – and you can see the three commits for the three releases I mentioned above.

So I’d like to suggest Gitpan as a suitable place to use as a default bugtracker for CPAN distributions. There are a couple of problems:

  • It looks like the auto-population of the commits stopped a few years ago. We’d need to work out how that works and catch up on the recent uploads.
  • None of the repos has the issue tracker turned on. But I expect that can be done with a relatively simple program that uses the GitHub API.

Of course, we also have the problem that some people object to using GitHub since it was taken over by Microsoft. But that’s fine, they can just point their bugtracker metadata to their preferred system.

The problem with the CPAN RT was that it needed too much maintenance – and the Perl NOC team is really overworked. Any self-hosted alternative seems likely to have the same problem eventually. So I’m all in favour of using a third-party alternative. And if you’re taking that route, then it makes sense (to me, at least) to use a third-party system that already has all (ok, most) of the repos set up.

I haven’t spoken to Schwern or Olaf about this, so I don’t know if there was some major problem that would stop this plan from working. But I think it’s worth looking at.

Categories
CPAN

Two New Modules

There was a London Perl Mongers meeting at ZPG about ten days ago. I gave a short talk explaining why (and how) a republican like me came to be running a site about the Line of Succession to the British Throne. The meeting was great (as they always are) and I think my talk went well (I’m told the videos are imminent). The photo shows the final slide from my talk. I left it up so it was a backdrop for a number of announcements that various people gave just before the break.

In order to write my talk, I revisited the source code for my site and, in doing so, I realised that there were a couple of chunks of logic that I could (and should) carve out into separate distributions that I could put on CPAN. I’ve done that over the last couple of days and the modules are now available.

MooX::Role::JSON_LD

The first is called MooX::Role::JSON_LD. It’s a Moo role that you can add to your classes in order to make it easy to generate JSON-LD from your objects. What’s JSON-LD? I hear you ask. Well, it’s JavaScript Object Notation for Linked Data. The most popular use for it is to add structured data to web sites. Adding structured data to your web site makes it easy for Google (other search engines are available) to understand what your web site is about and that, in turn, will hopefully persuade Google to list your site higher than it otherwise would. You can see JSON-LD in action on a couple of my recent projects – https://lineofsuccession.co.uk/ and https://towerbridge.dave.org.uk/ (you’ll need to look at the source to see it).

I’ve written the module as a Moo role, which means it should be usable in Moose classes too. To add JSON-LD to your class, you need to do three things:

  • Add the role to your class
  • Define a method called json_ld_type() which defines the type of JSON-LD object that you want to generate (see Schema.Org for a list of types)
  • Define a method called json_ld_fields() which defines the fields that you want to include in your JSON-LD. There are a few ways to define that which allow you to do things like mapping an output field to a method of a different name. The details are in the documentation.

Your class inherits two methods from the role – json_ld_data() returns the data structure which will be encoded into JSON (it’s provided in case you want to massage the data before encoding it) and json_ld() which returns the actual encoded JSON in a format that’s suitable for embedding in a web page.

Genealogy::Relationship

One of the most satisfying parts of the Line of Succession site to write was the code that shows the relationship between a person in the line and the current sovereign. Prince Charles (currently first in line) is the son of the sovereign and Tāne Lewis (currently thirtieth in line) is the first cousin twice removed of the sovereign.

That code might be useful to other people, so it’s now on CPAN as Genealogy::Relationship. To be honest, I’m not sure exactly how useful it will be. The Line of Succession is a rather specialised version of a family tree – because we’re tracing a bloodline, we’re only interested in one parent (which is unlike normal genealogy where we’d be interested in both). It also might be too closely tied to the data model I use on my site – but I have plans to fix that soon.

Currently, because of the requirements of my site, it only goes as far as third cousins (that’s people who share the same great, great grandparents). That’s five generations. But I have an idea to build a site which shows the relationship between any two English or British monarchs back to 1066. I think that’s around forty generations – so I’ll need to expand the coverage somewhat!

But anyway, it’s there and I’d be happy if you tried it and let me know whether it works for you. The documentation should explain all you need to know.

The Line of Succession site doesn’t get much traffic yet – I really need to do more marketing for it. So it’s satisfying to know that some of the code might, at least, be useful outside of the project.

Categories
CPAN

Yak Shaving with Aphra Behn

I have a few ideas for static web sites that I want to build. And, currently, the best place to host static web sites is, in my opinion, Github Pages.

And if you’re hosting a site on Github Pages, everyone knows that the best tool to use is Jekyll. Or is it?

I’ve tried to use Jekyll a couple of times and it just confused me. Something about the way it works just doesn’t fit into my head in some way. I’m not sure what it is, but every time I change something, it all breaks completely. I’m sure the problem is with me rather than the software. Everyone else seems to get on with it just fine.

So, anyway, when faced with a problem like that I did what any self-respecting geek would do. I wrote my own tool to solve the problem.

And, of course, I wrote it in Perl (because that’s what I know best) and I used the Template Toolkit (because, well, why wouldn’t I). And because I wrote it to reflect the way that I think about building static web sites, I understand how it works.

To be honest, how it works is pretty simple so far. It takes a bunch of files in an input directory, processes them using the Template Toolkit and writes them into a mirror directory structure under an output directory. So far, not so different to tttree (the tool that comes with the Template Toolkit), but there’s one little improvement that I’m finding very useful.

I like writing text using Markdown. And I thought that it would be great to write text in Markdown, but have it pre-processed to HTML before passing it through the Template Toolkit. A couple of months ago I released Template::Provider::Pandoc which does just that (actually, it does a lot more than that – it will convert between any two text formats that are supported by Pandoc.

And my new site builder software used Template::Provider::Pandoc to process all of the templates in the site. You don’t really want to be using Markdown for the main layout of your site – Markdown is rubbish for building navbars, footers or image carousels – but when I have a large amount of text, I can [% INCLUDE %] a template which includes that text in Markdown, knowing that it will be converted to HTML before being included in the page.

I’ve called the software aphra (for reasons that I’ll get to in a minute). There’s an early version on CPAN and the code is, of course, on Github too.

If you want to try it out, the best documentation, currently, is in the command line tool, aphra, that comes with the distribution.

What about that name?

Yes, it’s a strange name.

When I first realised I’d be writing something like Jekyll, I wanted to call it Hyde. I wanted to be able to say that it was uglier and more powerful than Jekyll. But there’s already a Python sitebuilder called that. Then I considered Utterson (he’s Henry Jekyll’s friend in the novel) but that had been taken too.

So I abandoned the idea of using the name of a character from The Strange Case of Dr Jekyll and Mr Hyde and started looking elsewhere.

I first came across Aphra Behn when I read Philip José Farmer‘s Riverworld books about thirty years ago and she has stuck with me ever since. [I should point out for people who haven’t read Farmer’s books that he takes real historical characters, like Behn, and drops them into a science fiction environment.]

Behn was a British writer who wrote novels, plays and poetry in the second half of the seventeenth century. At a time when women simply didn’t do those things, it just didn’t seem to occur to her that she shouldn’t. She was a great role model to many of the great women writers of the following centuries.

Oh, and she was a spy too, during the Second Anglo-Dutch War.

All in all, she was an inspirational woman who deserves wider recognition. And I hope that, in some small way, that my software will raise her profile.

What’s next?

So now I have my tool, it’s time to start creating the web sites that I wanted. I hope to have some news on those for you in a few weeks.

Or, perhaps, I’ll get bogged down creating a web site for Aphra. I’ve just registered a domain name…

Categories
CPAN

Genealogical Timelines in Perl and SVG

If you ever read my (mostly dead) more general blog, you might know that I’m a bit of an amateur genealogist. I’ve been tracing my family for over twenty-five years and I’ve got some branches of it back to the 1700s (actually, I have one branch back to the late 1600s).

One problem in genealogy is how to present data in a readable and easily-understandable way. Family trees are messy things. Both the roots and the branches can get very tangled. A good way to cut through all of that is to ignore unnecessary branches and just show the ancestors of a given person on the tree.

And that’s what the image at the top of this post shows. In the right-hand side of the image, halfway down, you will see Prince George of Cambridge (ok actually, you’ll see “Princ”, that’s a bug that I need to fix – it works when someone’s lifespan is long enough to fit their name in!) Above and below him (at a quarter and three-quarters of the way down the page) you’ll see his parents. And so on back through time until on the left of the page you’ll see his great, great, grandparents – most of whom were born back in the nineteenth century.

It’s all created with a Perl program, of course. I’ve just uploaded SVG::Timeline::Genealogy to CPAN (it should be there at some point later today) and that can be used to draw these diagrams.

The module is very similar to SVG::Timeline which I wrote about a couple of weeks ago. And that’s completely unsurprising as it’s a sub-class of that module. Interestingly, early drafts of this module pre-date SVG::Timeline, but I recently realised that it should be a sub-class so I spent yesterday re-implementing it (and making more than a few changes to SVG::Timeline as some idiot had made it hard to sub-class!)

There are two ways to use the module. The hard way involves writing your own code:

The easy way involves putting the information in a data file and using the treeline program that is included in the distribution.

The fields in the data are separated by tabs.

The important bit to get right is the “ahnen” attribute. “Ahnen” is short for “Ahnentafel Number” and it’s a concept that is common in genealogy. You take a person in your family tree (say you for example) and give that person a number of 1. Your father then has a number of 2 and your mother is 3. Carry on with that scheme through the generations. You paternal grandparents are 4 and 5, your maternal grandparents are 6 and 7… and so on.

These numbers have a couple of interesting properties. Firstly, if a person has an Ahnentafel Number of $x, then their parents are 2 * $x and 2 * $x + 1. Secondly, with the exception of person 1 (who can obviously be of either sex) all the men have even numbered Ahnentafel Numbers and the women all have odd numbers.

It is therefore these numbers that allow us to convert a flat data file into a tree structure. They tie the records together in the correct order. If you want to know more, I have a module called Genealogy::Ahnentafel which allows you to manipulate these numbers in various ways.

So that’s, SVG::Timeline::Genealogy. Hope you find it useful. Please share any interesting genealogies that you find.

Categories
CPAN

Timeline Diagrams with Perl

Two weeks ago, I introduced you to my new module SVG::TrafficLight and hinted that there were more SVG-based modules to follow. Today, I’d like to talk about the next one – SVG::Timeline.

It all started over a year ago when I was looking through some of the more ridiculous religious questions on Quora when I came across one asking why Adam wasn’t mentioned in the Bible after the first couple of books in Genesis. As part of my answer I wanted to illustrate just how long a time Genesis 5 covers.

I knew that SVG would be the best approach and it only took half an hour or so to whip up the image you can see in my answer (and there’s a newer version, generated with the current version of the code, at the top of this page). It’s important to note that  I didn’t hand-craft SVG that drew the diagram – I wrote code that generated the diagram from an input file.

I then realised that this could be a more generally useful tool, so I set about making the code more generic. It languished on Github for a year or so before I decided it was useful enough to clean it up and release it to CPAN. Let’s take a quick look at how it works.

As you can see from the example above, a timeline is made up of a number of events. An event has a start date, an end date and some text. So you can start a timeline diagram with code like this:

And once you have added all of your events, you can produce the timeline using:

That code writes the SVG document is written to STDOUT, so you’ll probably want to redirect that to a file.

That will draw a timeline of your events using all of the default settings (which, in most cases produce a useful diagram). There are plenty of options that you can pass to the object constructor to tweak things. The most useful are probably the aspect ratio (if your diagram is going to be particularly long or thing – the default is 16/9) and the number of years between gridlines in  the output (the default is ten and you might want to change that if your timeline covers a particularly large or small number of years – like the Genesis example above).

The default behaviour is to colour all of the events the same colour (which can be changed from the default in the constructor for the SVG::Timeline object). But you can also change it for each individual event by adding an optional “colour” parameter to the add_event() call.

But that’s all a lot of work for the simple case. So the distribution also includes a command line program called timeline which does all of that for you. It reads a datafile and produces a timeline diagram based on  the contents.

Each record in the input file has three or four fields separated by tabs. The fields are the parameters for the add_event() call in the order: text, start, end and (optionally) colour.

There are example data files in the distribution for producing some of the timelines I’ve talked about in this article – along with shell scripts showing how to produce timeline diagrams using the command line program.

There are a few things I’d like to add. Support for events with unknown dates (perhaps fading the colour towards the unknown end). Diagrams that go vertically instead of horizontally and support for events that begin and end in the same year (currently, they are zero size and just vanish – I discovered that when I added Paul McGann and Christopher Eccleston to the Doctor Who example).

I find the program… well, if not exactly useful, it’s still fun to play with.

Please let me know if you produce any interesting timelines with it.