Tag Archives: linkedin

Misunderstanding Context

Over the last few days I’ve been involved in a discussion on LinkedIn[1]. It has been interesting as it shows how many people still misunderstand many of the intricacies of context and, in particular, how it ties in with the values returned from subroutines.

The original question asked why these two pieces of code acted differently:

my ($index) = grep {
  $array[$_] == $variable
} 0 .. $#array;
my $index = grep {
  $array[$_] == $variable
} 0 .. $#array;

The first one gives the first index where the element equals $variable, the second one gives the number of indexes where the element equals $variable.

The answer, of course, comes down to context.  And most people who answered seemed to understand that, but many of their explanations were still way off the mark. Here’s the first answer:

grep returns an array – so the first one will return the value of the first match, but 2nd one in scalar context will return the size of the array, the number of matches. I believe.

The first statement here – “grep returns an array” – is wrong, so any explanation built on that fact is going to be fundamentally flawed.

After a couple of similar answers, I jumped in and pointed out that the only way to know how grep will work in different contexts is to read the documentation; which says:

returns the list value consisting of those elements for which the expression evaluated to true. In scalar context, returns the number of times the expression was true.

At that point it got a bit weird. People started telling me all sorts of strange things in order to show that the earlier answers were better than mine. I was told that lists and arrays are the same thing, that there was something called “array context” and that it was possible for function to return arrays.

I think I’ve worked out what most of the misconceptions in the discussion were. Here’s a list.

1. Arrays are not lists

I know this is a very common misunderstanding. I come across it all the time. People use the terms “list” and “array” interchangeably and end up thinking that they are the same thing. They aren’t. A list is a data value and an array is a variable. They act the same way a lot of the time but unless you understand the difference, you will make mistakes.

Mike Friedman wrote a great blog post that explains the difference in considerable detail. But the core difference is this – arrays are persistent (at least while they are in scope) data structures; lists are ephemeral.

On training courses, I tell people that if they understand the difference between an array and a list they’ll be in the top 20% of Perl programmers. In my experience, that’s pretty close to the truth.

2. Subroutines return lists

A subroutine can only ever return a list. Never an array. It can be an empty list. It can be a list with only one item. But it’s always a list.

There’s one small “gotcha” here. I know this bit me a few times in the past. If you read the documentation for return, it says this:

Evaluation of EXPR may be in list, scalar, or void context, depending on how the return value will be used

This means that when you have code like return @array, it’s @array that is evaluated in the context of the subroutine call. So, depending on  the context of the call, this will return either a list consisting of all the elements in @array, or a list with one element which is the number of elements in @array.

3. There is no “array context”

When you confuse lists and arrays, then it’s not surprising that you might also decide that you can also invent a new context called “array context”. There are only list context, scalar context and void context. Ok, actually there are a few more specialised contexts that you can use (see the Want module for details) but most of the time you’ll only be dealing with those three.

Of course, the name of the wantarray function doesn’t help at all here. But it’s worth noting that the documentation for wantarray ends by saying:

This function should have been named wantlist() instead.

4. You  can’t guess contextual behaviour

One common argument I got when pointing this out on LinkedIn ran along the lines of “but grep acts like it returns an array, so it’s a useful mental model – it helps people to understand context”.

The first part of this is true – grep does act like it returns an array. It returns a list (which you might often store in an array) in list context and it returns the length of that list in scalar context. I can see how you would mistake that for returning an array. But  see my point 2 above. Subroutines do not ever return arrays.

But is it a useful mental model? Does it help people understand how subroutines work in different contexts?

No. It helps people to understand how this particular function works. And there are several other Perl functions that work the same way (keys is one example). But there are plenty of other functions  that don’t work that way. The canonical example is localtime. In list context, it returns a list of nine values; in scalar context it returns a single value (which isn’t the number 9).  Another good example is caller. In list context, it returns a list of items (which can contain either three or eleven elements); in scalar context it returns the first item of that list. There are many more examples.

So the problem with a mental model that assumes that functions work as though they return arrays is that it only works for a subset of functions. And you’d need to waste effort remembering which functions it works for and which ones it doesn’t work for. You’d be far better off (in my opinion) realising that there is no rule that works for all functions and just remembering how each function works (or, more practically, remembering to check the documentation whenever you need to know).

 

If you’ve seen me giving a lightning talk at a conference this  year, you’ll know that I’ve been encouraging more people from the Perl community to get involved in discussions in places outside the echo chamber, places like LinkedIn. It’s interesting because you see how people outside of the community see Perl. You see the misunderstandings that they work with and you might see how we can improve the Perl documentation to help them get around those misunderstandings.

This morning, this comment was posted to the LinkedIn discussion that I’ve been talking about:

This stuff should be included in a default perl tutorial to avoid common mistake.

And, you know, I think he’s probably right.

[1] Because of the way LinkedIn works, you won’t be able to see this discussion unless you’re a member of LinkedIn and, probably, a member of the Perl group as well.

Perl Books

The Perl community on LinkedIn is fascinating. It’s a great way to see how Perl is perceived and used outside of the echo chamber. And that’s a real eye-opener.

Here’s an example. Every few weeks (it seems) someone asks for advice on Perl books.At that point a few people will jump in with sensible suggestions. But for every reasonable suggestion, you’ll get three or four people suggesting something from this list:

  • Something horribly out of date. A lot of publishers stopped updating their Perl books about ten years ago. The most recent version of Perl Black Book that I can find is from 2001. Teach Yourself Perl in 21 Days was last updated in 2002. Perl By Example dates from 2007. I haven’t read any of these books, so I can’t comment on their quality, but I’d be really wary of suggesting that someone learns Perl from a book that is so out of date. Before the days of Amazon, these books would quietly disappear from bookshops, but now they always seem to be available.
  • Something out of date, but that used to be great. There are plenty of books that I would have recommended when they were current, but that I wouldn’t really want to recommend now. A lot of these are O’Reilly books that haven’t been updated recently – Advanced Perl Programming and The Perl Cookbook, for example. I’m sure there’s lots of good stuff in those books. But I really wouldn’t want anyone to read them if they didn’t already have enough experience to update the Perl to current best practices. I sometimes recommend these books myself, but only in specific circumstances. Whenever I run a course on OO Perl I mention Damian’s book. But I also point out that there have been a lot of advances in the area since the book was published. And, I have to confess, my own Data Munging with Perl fits firmly in this category.
  • Something of dubious provenance.  Most of these discussions will, at some point, attract a link to some dubious web site in Eastern Europe that contains the full text of O’Reilly’s various CD bookshelves. I know that these sites exist and I know that there’s nothing that O’Reilly can do about them, But I’d rather not see them mentioned on a professional site like LinkedIn. And quite apart from the copyright issues, there’s the fact that most of the books on these sites fall firmly into the first category above. They’re all over ten years old.
  • Some great tutorial on the internet. I’ve talked about the problems of old and dodgy tutorials before. And things definitely seem to be looking up. We are getting more good tutorials out there. But people still insist on sharing links to the tutorial that they learned from. Even if it’s appalling.

In a recent discussion someone said (and I’m paraphrasing) “just go to Amazon and look for the highest rated books”. I think there are two problems with that:

  1. The people who are rating beginner’s programming books on Amazon are usually the least qualified people to do that. Sure, they can tell how easy the book was to read and how well they picked up what the author taught them. But they have noway of knowing whether what they learned was accurate or useful.
  2. Amazon ratings last forever. But, as noted above, the quality of a technical book in fast-moving subject like programming falls over time. Perhaps Amazon ratings on programming books should have a half-life.

Here are some examples of things that you’re going to miss out on by using outdated Perl references.

  • say – We all love say, don’t we?
  • Lexical filehandles – Storing filehandles in lexical variables is great. I no longer have to worry about bareword filehandles being reused elsewhere in the code.
  • Defined or operator – There’s now no excuse for the $val ||= $default bug.
  • given/when – Perl has a switch statement. And it’s better than anyone else’s :)
  • Unicode support – Perl’s Unicode support is second to none. Except, if you’re using an old version of Perl in which case it’s a bit rubbish. And if you’re reading about Perl in an old book, then you won’t know about it.
  • State variables – Ok, I don’t use those every day. But when I need them, they make my code cleaner.

And then there are all the great CPAN modules that aren’t covered in books that were written before they were released. Would you really want to introduce someone to OO Perl without mentioning Moose?

My rules for recommending books are pretty simple. They should be books that I’ve found useful and they should have been published in the last few years. And given the falling numbers of Perl books that are published each year, that’s now a rather small number of books. Perhaps a dozen or so.

Am I being too harsh? Can beginners get something useful out of older Perl books? How do you decide whether to recommend a book to a colleague?