CGI.pm has been removed from the core Perl distribution. From 5.22, it is no longer included in a standard Perl installation.
There are good technical reasons for this. CGI is a dying technology. In 2015, there are far better ways to write web applications in Perl. We don’t want to be seen to encourage the use of a technology which no-one should be using.
This does lead to a small problem for us though. There are plenty of web hosting providers out there who don’t have particularly strong Perl support. They will advertise that they support Perl, but that’s just because they know that Perl comes as a standard part of the operating system that they run on their servers. They won’t do anything to change their installation in any way. Neither you nor I would use a hosting company that works like that – but plenty of people do.
The problem comes when these companies start to deploy an operating system that includes Perl 5.22. All of a sudden, those companies will stop including CGI.pm on their servers. And while we don’t want to encourage people to use CGI.pm (or, indeed, the CGI protocol itself) we need to accept that there are thousands of sites out there that have been happily using software based on CGI.pm for years and the owners of these sites will at some point change hosting providers or upgrade their service plan and end up on a server that has Perl 5.22 and doesn’t have CGI.pm. And their software will break.
I’ve always assumed that this problem is some time in the future. As far as I can see, the only mainstream Linux distribution that currently includes Perl 5.22 is Fedora 23. And you’d need to be pretty stupid to run a web hosting business on any version of Fedora. Fedora is a cutting edge distribution with no long term support. Versions of Fedora are only supported for about a year after their release.
So the problem is in the future, but it is coming. At some point Perl 5.22 or one of its successors will make it into Red Hat Enterprise Linux. And at that point we have a problem.
Or so I thought. But that’s not the case. The problem is here already. Not because of Perl 5.22 (that’s still a year or two in the future for most of these web hosting companies) but because of Red Hat.
Red Hat, like pretty much everyone, include Perl in their standard installation. If you install any Linux distribution based on Red Hat, then the out of the box installation includes an RPM called “perl”. But it’s not really what you would recognise as Perl. It’s a cut down version of Perl. They have stripped out many parts of Perl that they consider non-essential. And those parts include CGI.pm.
This change in the way they package Perl started with RHEL 6 – which comes with Perl 5.10. And remember it’s not just RHEL that is affected. There are plenty of other distributions that use RHEL as a base – Centos, Scientific Linux, Cloud Linux and many, many more.
So if someone uses a server running RHEL 6 or greater (or another OS that is based on RHEL 6 or greater) and the hosting company have not taken appropriate action, then that server will not have CGI.pm installed.
What is the “appropriate action” you ask. Well it’s pretty simple. Red Hat also make another RPM available that contains the whole Perl distribution. So bringing the Perl up to scratch on a RHEL host is as simple as running:
yum install perl-core
That will work on a server running RHEL 6 (which has Perl 5.10) and RHEL 7 (which has Perl 5.16). On a future version of RHEL which includes Perl 5.22 or later, that obviously won’t work as CGI.pm won’t be part of the standard Perl installation and therefore won’t be included in “perl-core”. At that point it will still be a good idea to install “perl-core” (to get the rest of the installation that you are missing) but to get CGI.pm, you’ll need to run:
yum install perl-CGI
So this is a plea to people who are running web hosting services using Red Hat style Linux distributions. Please ensure that your servers are running a complete Perl installation by running the “yum” command above.
All of which brings me to this blog post that Marc Lehmann wrote a couple of days ago. Marc found a web site which no longer worked because it had been moved to a new server which had a newer version of Perl – one that didn’t include CGI.pm. Marc thinks that the Perl 5 Porters have adopted a cavalier approach to backward compatibility and that the removal of CGI.pm is a good example of the problems they are causing. He therefore chose to interpret the problems this site was having as being caused by p5p’s approach to backward compatibility and the removal of CGI.pm.
This sounded unlikely to me. As I said above, it would be surprising if any web hosting company was using 5.22 at this point. So, I did a little digging. I found that the site was hosted by BlackNight solutions and that their web says that their servers run Perl 5.8. At the same time, Lee Johnson, the current maintainer of CGI.pm, got in touch with the web site’s owner who confirmed what I had worked out was correct.
Later yesterday I had a conversation with @BlackNight on Twitter. They told me that their hosts all ran Cloud Linux (which is based on RHEL) and that new servers were being provisioned using Cloud Linux 6 (which is based on RHEL 6).
So it seems clear what has happened here. The site was running on an older server which was running Cloud Linux 5. That includes Perl 5.8 and predates Red Hat removing CGI.pm from the “perl” RPM. It then moved to a new host running Cloud Linux 6 which is based on RHEL 6 and doesn’t include CGI.pm in the default installation. So what the site’s owner said is true, he moved to a new host with a newer version of Perl (that new version of Perl was 5.10!) but it wasn’t the new version of Perl that caused the problems, it was the new version of the operating system or, more specifically, the change in the way that Red Hat (and its derivatives) packaged Perl.
Marc is right that when Perl 5.22 hits the web hosting industry we’ll lose CGI.pm from a lot a web servers. You can make your own mind up on how important that is and whether or not you share Marc’s other opinions on how p5p is steering Perl. But he’s wrong to assume that, in this instance, the problem was caused by anything that p5p have done. In this instance, the problem was caused by Red Hat’s Perl packaging policy and was compounded by a hosting company who didn’t know that upgrading their servers to Cloud Linux 6 would remove CGI.pm.
RHEL 6 was released five years ago. I suspect it’s pretty mainstream in the web hosting industry by now. So CGI.pm will already have disappeared from a large number of web servers. I wonder why we haven’t seen a tsunami of complaints?
Update: More discussion on Reddit and Hacker News.
The canonical way to install the perl module Foo on a Fedora-based system is to use the command: yum install “perl(Foo)”
This will install the package that contains the Foo module regardless of its name.
Dave
If we had received complaints about missing Perl modules that our clients were relying on we’d have addressed it. We have added various “extras” to our web server configurations based on what clients tell us and are using.
Michele
Hi Michele,
Thanks for the speedy comment!
I suspect that the owner of Gamebase64.com just didn’t complain as he wanted to rewrite the site in PHP anyway.
Would you consider ensuring that “perl-core” is installed on all of your systems to ensure that all users have access to a full Perl installation?
Dave
I’ve asked our technical team about this and will revert once I get a reply from them
Michele
The main problems with CGI.pm from a practical point of view in my experience are:
* the $cgi->param(‘…’) interface is baroque and hard to replicate.
* the magic flags in $cgi->header and $cgi->redirect
* STDOUT being magically tied for print-ing purposes (making for amusing debugging experiences of at least two different kinds)
* Horrible abuse of exit(0) as a proxy for $app->plack_response->finalize
If someone shimmed them then into a sane Plack::Request/Response API then 90% of CGI.pm problems would go away. This would of course leave the remaining 90% of problems, but they only arise in 11% of cases, so YMMV.
Picking a nit here, but I thought Marc handled his post well, aside from (as you say) misidentifying the source of the problem. He just called this one person choosing to rewrite in PHP when Perl (apparently, from their point of view) “stopped working”. Whether this was the start of a trend or any sort of a problem was left to the reader — as I read the article.
But thank you for writing about the topic, and I have opinions too. There are certainly technical reasons to get rid of CGI.pm (both the implementation and the basic design), but as Marc’s article hints and you acknowledge, this has the potential to break a lot of sites, and wide scale breakage isn’t something Perl has been known for. Understandably, large scale breakage has to happen when there is no other way to fix a security problem, but when you slide into the turf of breaking things for the sake of tidying things around the edge (things that don’t hold back larger cleanups), you run the risk of creating Python or Java style version fragility. So I’m sad to see CGI.pm go. I’d prefer it to remain forever deprecated but never actually removed.
It’s ironic that Java has an immense legal framework dedicated to preserving the integrity of their standard API (which caused them to sue Microsoft and then more recently Google over vendor changes to the API) but then create their own version fragility. That’s a worst of both worlds scenarios. Proactively contacting hosts about the problem and asking them to install perl-core and/or CGI may be the only sane resolution.
But it hasn’t gone. It’s just a simple installation command away.
As I understand the decision, people worried that keeping CGI.pm in the Perl core would be seen as a recommendation to use it. I’m sure no-one wants to see it seen that way. I can ‘t see any way to retain it in the core and mark it as “please don’t use this”.
And anyway, as this post discusses, what p5p puts in the core doesn’t mean anything if Linux distributions feel empowered to rip stuff out before passing it on to the end users.
So if I understand this correctly, I’ll get Perl if I install Perl.
Oops. Good point. Fixed :-/
For Dada Mail deployment, we do support Plack/PSGI, as well as running under CGI. I would say that the great majority of installs are done under CGI, and not Plack/PSGI – it’s just so much easier for a *non-developer* to set things up, that way. I would love that to change.
The target platform for CGI deployment is cPanel, which comes currently with v5.10.1 of Perl, and I don’t see that changing anytime soon, as this is the system Perl. If you dig around, (cPanel is a Perl shop, at least in part, yeah?) they do have Perl 5.14.x available as well.
https://documentation.cpanel.net/display/SDK/_PerlEnvironments
Thankfully, they’re system also has a web interface to CPAN and if you’re lucky, they’ll give you access to gcc.
So, that’s one example of a host, that supports running Perl apps under CGI, and probably one of the best. I’m still coming across hosts that only support Perl 5.8. It’s rare to find a host that doesn’t have CGI.pm installed, but it does happen. CGI.pm is an easy enough module to install without needing CPAN (in a pinch).
The other problem is that the current maintainer of CGI.pm is REMOVING features, which is also breaking things. Some of the features removed are the HTML generating methods, which for things like regular tags (h1, p, br, etc), doesn’t impact me, but anything exceptionally old will now just suddenly not work after 10+ years of deployment. I think the case could be made that these methods weren’t ever the greatest idea, but this may have been before using a templating system made a lot of sense (or maybe it was a performance thing? I dunno)
Some of those HTML generating methods are for HTML form widgets, which are actually very useful and I was happy that there were those methods to produce them, as to recreate these methods using your favorite template language (HTML::Template, Template-Toolkit) is not going to be as nice of a solution (IMHO). I’m thinking of popup menus and checkbox groups. At least for popup menus, there is HTML::Menu::Select but it isn’t a drop-in replacement.
So, yeah – lotsa hate for CGI.pm, the most maligned of the most used CPAN modules: it’s being removed from core on old Perls by distros, removed from the core in new Perls by p5p, and it’s maintainer is removing features at a pretty impressive rate, without any drop in replacement (no CGI::OutDatedMethods module)
The documentation for the current version of CGI.pm (version 4.23 – released yesterday) says this:
So, no, they haven’t been removed. Their documentation has been moved into another file and they are deprecated and unmaintained. But they are still there. And the documentation specifically says “They will, however, continue to exist in CGI.pm.”
If you look at the docs from previous version so CGI.pm (for example v4.21), you’ll see the verbiage has changed quite dramatically:
It was going to be deprecated in v5 (soft? whatever that means…), now it’s just deprecated outright. I’m getting a little weary thinking that they’ll be removed whenever the current maintainer decides too, as they keep changing their plan.
The other kinda cavalier gripe with CGI.pm has been the, “calling param() in list context can lead to vulnerabilities” issue. That seemed to have filled may error logs, as the error for it is printed many, many times per call. I can change my own code, but I can’t change code in other people’s releases, and this vulnerability is so old, it touches on many different modules. I’ve had to put out a lot of fires because of this sloppy error reporting. Perhaps once would have been enough?
I changed my plans because i had requests from users to change my plans.
Open source is a two (three, four) way conversation and i’m treading a fine line between tidying the module up enough to make it more maintainable, removing the real danger and cruft, documenting that CGI.pm really shouldn’t be used, and improving test coverage. Sometimes i’ll make mistakes or take decisions that others are unhappy with but, again, i’m treading a fine line and i’m only human.
Netanel Rubin called me stupid for *only* raising a warning to fix list context behaviour, but i’m experienced enough to know that i can’t just make breaking changes and mature enough to ignore the name calling. What i struggle to deal with is the criticism with no corresponding traffic on the github issues repo as it’s trivial to raise and issue or comment, or even send me an e-mail, and i will usually respond to github issues in a matter of minutes if not hours.
I will raise an issue for the warning as it *is* more sensible to raise the warning once, a good idea that i didn’t think of.
These are good efforts to fix the problem caused by omission of CGI.pm from core. It was a grave mistake to do so.
This has to be one of the dumbest things you can possibly do. What person decided that cgi is a dying technology? Who cares if it’s a dying technology? If it’s sufficient for getting a job done, it’s fine to continue to use.
The result of removing functionality for a language is KILLING the language. Guess why I won’t work under Windows anymore? Because Microsoft keeps modifying the language and requiring constant continue, expensive, updates to source code to allow it to work.
Apache still, and probably ALWAYS will support cgi.
History, I guess 🙂
I do, for one. But it’s fine if you don’t agree.
Of course. And I don’t think anyone has suggested otherwise. On the other hand, I think the Perl 5 Porters did the right thing by no longer encouraging its use.
But the functionality is all still there. Perl still supports the CGI protocol. And if you want the CGI module, well it’s just a single installation command away. And, for most Red Hat users, that’s been the case for many years.
Of course it does. And so does Perl.