Uncategorized

Token-Oriented Object Notation

Token-Oriented Object Notation

Token-Oriented Object Notation ( TOON ), is a data serialization format designed to be a more compact and LLM – friendly alternative to JSON. It reduces the token count by eliminating redundant syntax like quotes and braces, using indentation for structure and a tabular format for arrays of objects. This makes data cheaper and faster for large language models to process while still being human-readable.

Features of TOON

  • Token efficiency: 

    It minimizes tokens by removing characters like quotes, braces, and commas, which are often repeated in JSON. 

  • LLM-friendly structure: 

    TOON adds structural metadata that helps LLMs parse and understand the data more reliably, reducing parsing errors. 

  • Tabular arrays: 

    Uniform arrays of objects, a common data pattern, are represented in a compact, CSV-like tabular format with keys declared once for the entire block. 

  • Human-readable: 

    The use of indentation and a clean layout makes the data easy for humans to read, similar to YAML. 

  • JSON compatibility: 

    It’s designed to be a lossless, drop-in replacement for JSON when sending data to LLMs. You can use JSON for your application code and convert it to TOON specifically for LLM prompts. 

  • Purpose: 
    It acts as a translation layer between traditional JSON-based systems and LLM prompts, making AI interactions more efficient and cost-effective.Â