If you’ve been building with Large Language Models (LLMs) for more than five minutes, you’ve likely hit the wall. You know the one: you have a beautiful, structured dataset you need the model to analyze - maybe a history of user transactions, a list of flight options, or a catalog of products—and you reach for your old reliable friend, JSON.
But as soon as you paste that JSON into your prompt, you hear a distinct *cha-ching* sound. You aren’t just paying for your data; you’re paying for an army of curly braces, quotation marks, repeated keys, and whitespace.
It turns out, JSON was built for machines to talk to machines, not for humans to talk to LLMs.
Enter TOON (Token-Oriented Object Notation). It’s a new serialization format that is quietly revolutionizing how we feed structured data to AI. It’s cleaner, cheaper, and—surprisingly—makes models smarter.
Here is your expert guide to what TOON is, how it works, and why your next AI agent should probably be using it.
What is TOON?
TOON is a "lossless, drop-in representation of JSON" designed specifically for token efficiency.
Think of it as the love child of YAML and CSV. It takes the clean, indentation-based hierarchy of YAML and fuses it with the dense, header-based layout of CSV.
The philosophy is simple: don't repeat yourself.
In a standard JSON array of objects, you repeat the field names (keys) for every single item.
{"id": 1, "name": "Alice"},
{"id": 2, "name": "Bob"}
That’s a lot of wasted tokens. TOON defines the keys once, and then just streams the values. The result is a format that is human-readable, machine-parsable, and drastically smaller.
A Visual Comparison
Let’s look at a real-world example. Imagine we are feeding an LLM a list of hiking trails.
JSON
This is what we usually send. It’s verbose and full of syntactic sugar.
{
"context": {
"location": "Boulder",
"season": "summer"
},
"hikes": [
{
"id": 1,
"name": "Blue Lake Trail",
"distance": 7.5,
"difficulty": "hard"
},
{
"id": 2,
"name": "Ridge Overlook",
"distance": 9.2,
"difficulty": "medium"
}
]
}
TOON
Here is the exact same data in TOON:
context:
location: Boulder
season: summer
hikes[2]{id,name,distance,difficulty}:
1,Blue Lake Trail,7.5,hard
2,Ridge Overlook,9.2,medium
Notice the difference?
1. No Quotes: syntactic noise is gone.
2. Tabular Arrays: the `hikes` array is defined once with a header: `hikes[2]{id,name...}`. The numbers [2] explicitly tell the LLM, "Expect 2 rows."
3. Dense Data: The actual data follows in a CSV-like format.
Why Should You Care?
You might be thinking - why learn a new format just to save a few characters? But TOON offers more than just compression.
Massive Token Savings
Benchmarks show that TOON uses 30% to 60% fewer tokens than pretty-printed JSON for uniform arrays.
If you are running a high-volume AI application, cutting your input token costs in half is not an optimization; it’s a business model upgrade.
Better Needle in a Haystack Retrieval
This is the most surprising advantage. You might assume that compressing data makes it harder for an LLM to understand. The opposite is true.
According to benchmarks involving models like Claude and Gemini, TOON actually improves retrieval accuracy (reaching ~74% accuracy vs JSON's ~70% in mixed-structure tests). Why? Because the structure is explicit. By telling the model users[5]{id,name}, you are giving it a schema upfront. The model doesn't have to parse endless brackets to understand the data structure; it just reads the table.
Context Window Optimization
Every token you save on data is a token you can spend on *reasoning*. By shrinking your data payload, you leave more room in the context window for few-shot examples, detailed instructions, or maintaining a longer conversation history.
How to Use TOON
The beauty of TOON is that you don't need to manually format strings. There are libraries available that handle the heavy lifting.
The Syntax in a Nutshell
Objects: use indentation (like YAML).
user:
name: Neo
role: The One
Arrays: use the header syntax key[count]{columns} followed by rows.
matrix[2]{color,pill}:
red,true
blue,false
Prompting Strategy
When asking an LLM to output TOON, you don't need to explain the whole spec. LLMs are pattern matchers. Just show them the format.
Prompt example:
Here is the user data in TOON format. Please analyze the spending patterns:
transactions[3]{id,amount,category}:
1,50.00,groceries
2,12.50,transport
3,100.00,entertainment
If you want the LLM to respond in TOON (to save output tokens), simply provide the header in your system prompt:
Reply using this format: "analysis[N]{insight,confidence}:"
When NOT to Use TOON
TOON is amazing, but it isn't a silver bullet for everything.
Deeply Nested, Irregular Data: if your data looks like a complex configuration tree where every object has different fields, TOON loses its tabular advantage. In those cases, it just looks like YAML.
APIs and Storage: DO NOT replace your database storage or REST APIs with TOON. JSON is still the king of compatibility. TOON is strictly a "last-mile" format for the interface between your code and the LLM.
Final Thoughts
We are moving from an era of "Human-to-Machine" communication to "Human-to-LLM" communication. Our tools need to evolve. JSON was built for JavaScript parsers; TOON is built for Neural Networks.
It respects the model's attention mechanism, it respects your wallet, and it’s surprisingly readable for human debugging.
Give it a try. Your token budget will thank you.