Build RPMs of CPAN Modules

If you’ve been reading my blog for a while, you probably already know that I have an interest in building RPMs of CPAN modules. I run a small RPM repository where I make available all of the RPMs that I have built for myself. These will either be modules that aren’t available in other RPM repositories or modules where I wanted a newer version than the currently available one.

I’m happy to take requests for my repo, but I don’t often get any. That’s probably because most people very sensibly use the cpanminus/local::lib approach or something along those lines.

But earlier this week, I was sitting on IRC and Ilmari asked if I had a particular module available. When I said that I didn’t, he asked if I had a guide for building an RPM. I didn’t (well there are slides from YAPC 2008 – but they’re a bit dated) but I could see that it was a good suggestion. So here it is. Oh, and I built the missing RPM for him.

Setting Up

In order to build RPMs you’ll need a few things set up. This is stuff you’ll only need to do once. Firstly, you’ll need two new packages installed – cpanspec (which parses a CPAN distribution and produces a spec file) and rpm-build (which takes a spec file and a distribution and turns them into an RPM). They will be available in the standard repos for your distribution (assuming your distribution is something RPM-based like Fedora or Centos) so installing them is as simple as:

If you’re using Fedora 22 or later, “yum” has been replaced with “dnf”.

Next, you’ll need a directory structure in which to build your RPMs. I always have an “rpm” directory in my home directory, but it can be anywhere and called anything you like. Within that directory you will need subdirectories called BUILD, BUILDROOT, RPMS, SOURCES, SPECS and SRPMS. We’ll see what most of those are for a little later.

The final thing you’ll need is a file called “.rpmmacros” in your home directory. At a minimum, it should contain this:

The packager and vendor settings are just to stop you having to type in that information every time you build an RPM. The _topdir setting points to the “rpm” directory that you created a couple of paragraphs up.

I would highly recommend adding the following line as well:

This turns off the default behavior for adding “Requires” data to the RPM. The standard behaviour is to parse the module’s source code looking for every “use” statement. By turning that off, you instead trust the information in the META.yml to be correct. If you’re interesting in hearing more detail about why I think the default behaviour is broken, then ask me in a pub sometime.

Ok. Now we’re all set. We can build our first RPM.

Building an RPM

Building an RPM is simple. You use “cpanspec” to make the spec file and then “rpmbuild” to build the RPM. You can use “cpanspec” in a few different modes. If you have the module tarball, then you can pass that to “cpanspec”.

That will unwrap the tarball, parse the code and create the spec file.

But if you’re building an RPM for a CPAN module, you don’t need to download the tarball first, “cpanspec” will do that for you if you give it a distribution name.

That will connect to CPAN, find the latest version of the distribution, download the right tarball and then do all the unwrapping, parsing and spec creation.

But there’s another, even cleverer way to use “cpanspec” and that’s the one that I use. If you only know the module’s name and you’re not sure which distribution it’s in, then you can just pass the name of the module.

This is the mode that I always use it in.

No matter how you invoke “cpanspec”, you will end up with the distribution tarball and the spec file – which will be called “perl-Some-Module.spec”. You need to copy these files into the correct directories under your rpm building directory. The tarball goes into SOURCES and the spec goes into SPECS. It’s also probably easiest if you change directory into your rpm building directory.

You can now build the RPM with this command:

You’ll see a lot of output as “rpmbuild” goes through the whole CPAN module testing and building process. But hopefully eventually you’ll see some output saying that the build has succeeded and that an RPM has been written under your RPMS directory (in either the “noarch” or “x86_64” subdirectory). You can install that RPM with any of the following commands:

And that should be that. Of course there are a few things that can go wrong. And that’s what the next section is about.

Fixing Problems

There are a number of things that can go wrong when building RPMs. Here are some of the most common, along with suggested fixes.

Missing prerequisites

This is also known as “dependency hell”. The module you are building is likely to require other modules. And you will need to have those installed before “rpmbuild” will let you build the RPM (and, note, they’ll need to be installed as RPMS – the RPM database doesn’t know about modules you have installed with “cpan” or “cpanminus”).

If you have missing prerequisites, the first step is to try to install them using “yum” (or “dnf”). Sometimes you will get lucky, other times the prerequisites won’t exist in the repos that you’re using and you will have to build them yourself. This is the point at which building an RPM for a single module suddenly spirals into three hours of painstaking work as you struggle to keep track of how far down the rabbit-hole you have gone.

I keep thinking that I should build a tool which parses the prerequisites, works out which ones already exist and automatically tries to build the ones that are missing. It would need to work recursively of course. I haven’t summoned the courage yet.

Extra files

Sometimes at the end of an RPM build, you’ll get an error saying that files were found which weren’t listed in the spec file. This usually means that the distribution contains programs that “cpanspec” didn’t find and therefore didn’t add to the spec file. This is a simple fix. Open the spec file in an editor and look for the section labelled ‘%files’. Usually, it will look something like this:

This is a list of the files which will be added to the RPM. See the _mandir entry? That’s the man page for the module that is generated from the module’s Pod (section 3 is where library documentation goes). We just need to add two lines to the bottom of this section:

This says “add any files you find in the binaries directories (and also any man pages you find for those programs)”.

If you add these lines and re-run the “rpmbuild” command, the build should now succeed.

Missing header files

If you’re building an XS module that is a wrapped around a C library then you will also need the C header files for that library in order to compile the XS files. If you get errors about missing definitions, then this is probably the problem. In RedHat-land a C library called “mycoolthing” will live in an RPM called “libmycoolthing” and the headers will be in an RPM library called “libmycoolthing-devel”. You will need both of those installed.

Your users, however, will only need the C library (libmycoolthing) installed. It’s well worth telling the RPM system that this external library is required by adding the following line to the spec file:

That way, when people install your module using “yum” or “dnf”, it will pull in the correct C library too. “cpanspec” will automatically generate “Requires” lines for other Perl RPMs, but it can’t do it for libraries that aren’t declared in the META.yml file.

 

So that’s it. A basic guide to building RPMs from CPAN distributions. There’s a lot more detail that I could cover, but this should be enough to work for 80-90% of the modules that you will want to build.

If you have any questions, then please leave a comment below.

13 Replies to “Build RPMs of CPAN Modules”

  1. There are two circles of dependency hell. One is when the RPM has explicit dependencies which you need to install (with dnf, or by building your own package with cpanspec). But it can also happen that building requires some module not listed in the BuildRequires, so all seems OK until the build or the test suite fails part way through. Or worse, the module is built OK but fails at runtime with a missing dependency. This last case is quite rare though.

    I agree with you that rpm scanning all the source files looking for module uses is a kludge, but it does somewhat reduce the risk of not finding a dependency which exists. The best behaviour might be to combine both: use the META.yaml file (or Makefile.PL, etc etc) for the dependencies, but also do the source code scan and give a warning if any dependency is found that’s not mentioned in the metadata. If such a finding is a false positive, adding an explicit ‘not required’ entry in the metadata somehow would quieten the warning.

  2. I highly recommend FPM, which can quickly and easily build and convert packages from many different sources, including CPAN modules.

    Read more about it here: https://github.com/jordansissel/fpm/wiki/Perl-Packaging-With-FPM

    Building an RPM from a CPAN dist can be as simple as this: fpm -t rpm -s cpan DateTime # or whatever

    Installing fpm is easy, as it is packaged as a ruby gem, so you can gem install fpm, (and my favorite trick is to use fpm to package itself as an rpm)

    You don’t have to be root for any of this, unless the code you’re packaging needs it, but IMO you should never build test and package software as root, and IIRC the FPM docs also say so. 🙂

        1. I want a reproducible build so that I can rebuild the package (with local patches and customizations) when a new upstream version comes out, when needing to apply a new local patch, or when upgrading to a new release of my distribution. There are certainly tools which let you quickly generate a binary RPM from some files but they don’t address one of the main reasons for using RPM to maintain your installed software. So I will continue using cpanspec to generate a spec file, from which I can make a source package to rebuild whenever I need.

          1. While you can certainly do what you want, my personal best practice is to not touch upstream packages. If you require local modifications, they are best done by config management toolchain after the RPM is installed, not by patching SRPM and building custom RPMs.

  3. RPM building is one of the most frustrating and excruciating tasks I have the displeasure to undertake.

    cpanspec is a great boiler plate, but its “Requires:” entries tend to need manual intervention. Also building against el6 youre fighting with seriously old packages from the OS which need updating. RedHat doesnt do many favours with the way they package perl either – so its not elegant to update pieces they have bundled with perl itself.

    Epel can save you some time and there are a number of repositories around the internet that you might trust as-is and/or use their spec files (openfusion.com.au/labs/srpms for example). A lot of fedora srpms will compile, and even if the latest wont – you may find that fedora 17 or 18 will and its new enough for what you need.

    mock is a nice enough tool. koji is good in theory but messy in practice. if you can wade through its weirdness and get it running, you can have it serve you quite well doing massive builds and rebuilds. project-builder.org is also very interesting (and in perl) – i need to take a closer look at it.

    If you can on CentOS and RHEL – perlbrew and cpanm will save you serious serious time and grey hairs.

    RedHat should throw in the towel with rpms and use apt+dpkg – focusing their energy on adding value to their distro rather than maintaining yet another package management system.

    1. “If you can on CentOS and RHEL – perlbrew and cpanm will save you serious serious time and grey hairs.”

      ^^^ THIS x 1000

      My current task is to migrate from CentOS5 to CentOS7, and along with it our important perl project. The previous devs and sysadmins chose the RPM way of managing dependencies…and now, after the previously assigned dev left the company, in the middle of the migration, I have to pick it up and somehow make it work…

      Not only that my hair will be gray, but I will surely loose it in the process

  4. I second FPM, You can take generate the srpm from rpmbuild -s *.spec file that fpm generates.
    I have mostly ubuntu and some centos/rhel in my environment. Nice to use the same script.

    But do not create advanced dependencies. I got into hell because I tried to make one single
    package of everything.

    Prepare a base docker or vagrant image that holds everything you need except for your perl software, that way you can go back in seconds to retry if you get some errors.

  5. CPAN::Plugin::Specfile is an alternative specfilewriter. Disclaimer: I wrote it together with Branislav Zahradnik. It is far from being universal, but worth exploring. The main advantage is that it writes specfiles while you’re running the cpan shell and so the timestamps of the spec files already reflect the order that resolves the dependency hell.

Leave a Reply