README

The fastest TOON (Token-Oriented Object Notation) encoder and decoder for PHP, with full support for PHP 7.0 through 8.4.

TOON is a data serialization format optimized for LLM (Large Language Model) input, providing approximately 40% token reduction compared to JSON while maintaining full semantic equivalence.

⚡ Performance Highlights

This library is specifically optimized for speed and broad compatibility:

Benchmark Results

Performance comparison against helgesverre/toon (the reference implementation):

Benchmarks run on PHP 8.3 with 1000 iterations (100 for large data). Results may vary based on hardware and PHP version.

Installation

You can install the package via composer:

composer require mfonte/fast-toon

Usage

use Mfonte\FastToon\Toon;

$data = [
    'users' => [
        ['id' => 1, 'name' => 'Alice', 'active' => true],
        ['id' => 2, 'name' => 'Bob', 'active' => false],
        ['id' => 3, 'name' => 'Charlie', 'active' => true],
    ]
];

$toon = toon_encode($data);

// Output:
// users[3]{id,name,active}:
// 1|Alice|T
// 2|Bob|F
// 3|Charlie|T

$decoded = toon_decode($toon);

Fluent API

Instead of using helper functions, you may also use the fluent API:

use Mfonte\FastToon\Toon;

$toon = Toon::encode($data)
    ->withFlags(TOON_THROW_ON_ERROR | TOON_KEY_FOLDING)
    ->withDepth(100)
    ->toString();

$decoded = Toon::decode($toon)
    ->withOptions(['strict' => true])
    ->toArray();

Encoding flags

Like json_encode(), TOON supports bitmask flags for controlling encoding behavior:

// Combine flags with bitwise OR
$toon = toon_encode($data, TOON_THROW_ON_ERROR | TOON_KEY_FOLDING);

// Use tab delimiter for tabular output
$toon = toon_encode($data, TOON_TAB_DELIMITER);

Decoding flags

$data = toon_decode($toon, TOON_STRICT | TOON_THROW_ON_ERROR);

Error handling

TOON provides error handling similar to JSON:

$result = toon_encode($data);

if ($result === false) {
    echo toon_last_error_msg();
}

When using TOON_THROW_ON_ERROR, exceptions are thrown instead:

try {
    $toon = toon_encode($data, TOON_THROW_ON_ERROR);
} catch (\Mfonte\FastToon\Exceptions\EncodingException $e) {
    // Handle exception
}

TOON format overview

TOON uses a line-based syntax that eliminates redundant characters found in JSON.

Objects are represented as key-value pairs:

name:John Doe
age:30
active:T

Arrays of objects with the same structure use a highly efficient tabular format:

users[3]{id,name,email}:
1|Alice|alice@example.com
2|Bob|bob@example.com
3|Charlie|charlie@example.com

This provides approximately 50% token savings compared to the equivalent JSON for tabular data.

Testing

This package supports PHP 7.0 through 8.4 and provides PHPUnit configurations for each version range.

# Run tests on PHP 8.1+ (PHPUnit 10+)
composer test

# Run tests on PHP 7.0-7.1 (PHPUnit 6-7)
composer test:php70

# Run tests on PHP 7.2-8.0 (PHPUnit 8-9)
composer test:php72

# Run tests with coverage
composer test:coverage

The test suite is compatible with:

Code Style

This package uses PHP-CS-Fixer for code style enforcement. Two configurations are provided for compatibility across PHP versions.

# Check code style (PHP 7.4+, php-cs-fixer 3.x)
composer cs-check

# Fix code style (PHP 7.4+, php-cs-fixer 3.x)
composer cs-fix

# Check code style (PHP 7.0-7.3, php-cs-fixer 2.x)
composer cs-check:v2

# Fix code style (PHP 7.0-7.3, php-cs-fixer 2.x)
composer cs-fix:v2

PHP version compatibility

This library is designed to work across PHP 7.0 through 8.4:

• PHP 7.0-7.4: Full support with polyfills for missing functions
• PHP 8.0: Stringable interface support
• PHP 8.1+: Enum support, readonly properties

Type support

TOON supports all PHP types that json_encode() supports:

Benchmarks

TOON vs JSON

Compared to JSON for typical API response data:

Token counts measured with OpenAI's tiktoken (cl100k_base)

Running Benchmarks

This library includes comprehensive benchmarks to measure performance:

# Compare against helgesverre/toon (requires both libraries installed)
composer benchmark:compare

# Run phpbench microbenchmarks
composer benchmark

# Run phpbench with summary report
composer benchmark:summary

# Run profiling benchmark
composer benchmark:profile

The comparative benchmark (composer benchmark:compare) will output:

• Encoding/decoding times for small, medium, and large datasets
• Operations per second
• Memory usage
• Speedup factors vs helgesverre/toon

Why is mfonte/fast-toon faster?

Key optimizations that make this library significantly faster:

1. Inline Type Checks: Direct type comparisons instead of function calls
2. Pre-computed Values: Indent strings, delimiter symbols cached at construction
3. Optimized String Building: Direct array appending instead of buffered writing
4. Fast Path Detection: Early returns for common primitive types
5. Minimal Memory Allocation: Reuse of internal buffers

Why Choose mfonte/fast-toon?

Changelog

Please see CHANGELOG for more information on what has changed recently.

Contributing

Please see CONTRIBUTING for details.

License

The MIT License (MIT). Please see License File for more information.

Credits

• Maurizio Fonte
• TOON specification: github.com/toon-format