Writing a TOON Module for Perl

Writing a TOON Module for Perl

Every so often, a new data serialisation format appears and people get excited about it. Recently, one of those formats is TOON — Token-Oriented Object Notation. As the name suggests, it’s another way of representing the same kinds of data structures that you’d normally store in JSON or YAML: hashes, arrays, strings, numbers, booleans and nulls.

So the obvious Perl question is: “Ok, where’s the CPAN module?”

This post explains what TOON is, why some people think it’s useful, and why I decided to write a Perl module for it — with an interface that should feel very familiar to anyone who has used JSON.pm.

I should point out that I knew about Data::Toon but I wanted something with an interface that was more like JSON.pm.


What TOON Is

TOON stands for Token-Oriented Object Notation. It’s a textual format for representing structured data — the same data model as JSON:

  • Objects (hashes)
  • Arrays
  • Strings
  • Numbers
  • Booleans
  • Null

The idea behind TOON is that it is designed to be easy for both humans and language models to read and write. It tries to reduce punctuation noise and make the structure of data clearer.

If you think of the landscape like this:

Format Human-friendly Machine-friendly Very common
JSON Medium Very Yes
YAML High Medium Yes
TOON High High Not yet

TOON is trying to sit in the middle: simpler than YAML, more readable than JSON.

Whether it succeeds at that is a matter of taste — but it’s an interesting idea.


TOON vs JSON vs YAML

It’s probably easiest to understand TOON by comparing it to JSON and YAML. Here’s the same “person” record written in all three formats.

JSON

{
  "name": "Arthur Dent",
  "age": 42,
  "email": "arthur@example.com",
  "alive": true,
  "address": {
    "street": "High Street",
    "city": "Guildford"
  },
  "phones": [
    "01234 567890",
    "07700 900123"
  ]
}

YAML

name: Arthur Dent
age: 42
email: arthur@example.com
alive: true
address:
  street: High Street
  city: Guildford
phones:
  - 01234 567890
  - 07700 900123

TOON

{
  name: "Arthur Dent",
  age: 42,
  email: "arthur@example.com",
  alive: true,
  address: {
    street: "High Street",
    city: "Guildford"
  },
  phones: [
    "01234 567890",
    "07700 900123"
  ]
}

You can see that TOON sits somewhere between JSON and YAML:

  • Less punctuation and quoting than JSON
  • More explicit structure than YAML
  • Still very easy to parse
  • Still clearly structured for machines

That’s the idea, anyway.


Why People Think TOON Is Useful

The current interest in TOON is largely driven by AI/LLM workflows.

People are using it because:

  1. It is easier for humans to read than JSON.
  2. It is less ambiguous and complex than YAML.
  3. It maps cleanly to the JSON data model.
  4. It is relatively easy to parse.
  5. It works well in prompts and generated output.

In other words, it’s not trying to replace JSON for APIs, and it’s not trying to replace YAML for configuration files. It’s aiming at the space where humans and machines are collaborating on structured data.

You may or may not buy that argument — but it’s an interesting niche.


Why I Wrote a Perl Module

I don’t have particularly strong opinions about TOON as a format. It might take off, it might not. We’ve seen plenty of “next big data format” ideas over the years.

But what I do have a strong opinion about is this:

If a data format exists, then Perl should have a CPAN module for it that works the way Perl programmers expect.

Perl already has very good, very consistent interfaces for data serialisation:

  • JSON
  • YAML
  • Storable
  • Sereal

They all tend to follow the same pattern, particularly the object-oriented interface:

use JSON;
my $json = JSON->new->pretty->canonical;
my $text = $json->encode($data);
my $data = $json->decode($text);

So I wanted a TOON module that worked the same way.


Design Goals

When designing the module, I had a few simple goals.

1. Familiar OO Interface

The primary interface should be object-oriented and feel like JSON.pm:

use TOON;
my $toon = TOON->new
               ->pretty
               ->canonical
               ->indent(2);
my $text = $toon->encode($data);
my $data = $toon->decode($text);

If you already know JSON, you already know how to use TOON.

There are also convenience functions, but the OO interface is the main one.

2. Pure Perl Implementation

Version 0.001 is pure Perl. That means:

  • Easy to install
  • No compiler required
  • Works everywhere Perl works

If TOON becomes popular and performance matters, someone can always write an XS backend later.

3. Clean Separation of Components

Internally, the module is split into:

  • Tokenizer – turns text into tokens
  • Parser – turns tokens into Perl data structures
  • Emitter – turns Perl data structures into TOON text
  • Error handling – reports line/column errors cleanly

This makes it easier to test and maintain.

4. Do the Simple Things Well First

Version 0.001 supports:

  • Scalars
  • Arrayrefs
  • Hashrefs
  • undef → null
  • Pretty printing
  • Canonical key ordering

It does not (yet) try to serialise blessed objects or do anything clever. That can come later if people actually want it.


Example Usage (OO Style)

Here’s a simple Perl data structure:

my $data = {
  name   => "Arthur Dent",
  age    => 42,
  drinks => [ "tea", "coffee" ],
  alive  => 1,
};

Encoding

use TOON;
my $toon = TOON->new->pretty->canonical;
my $text = $toon->encode($data);
print $text;

Decoding

use TOON;
my $toon = TOON->new;
my $data = $toon->decode($text);
print $data->{name};

Convenience Functions

use TOON qw(encode_toon decode_toon);
my $text = encode_toon($data);
my $data = decode_toon($text);

But the OO interface is where most of the flexibility lives.


Command Line Tool

There’s also a command-line tool, toon_pp, similar to json_pp:

cat data.toon | toon_pp

Which will pretty-print TOON data.


Final Thoughts

I don’t know whether TOON will become widely used. Predicting the success of data formats is a fool’s game. But the cost of supporting it in Perl is low, and the potential usefulness is high enough to make it worth doing.

And fundamentally, this is how CPAN has always worked:

See a problem. Write a module. Upload it. See if anyone else finds it useful.

So now Perl has a TOON module. And if you already know how to use JSON.pm, you already know how to use it.

That was the goal.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.