README

FerryAI — native AI inference for PHP

Run ONNX, GGUF, and RubixML models directly in PHP — no Python, no HTTP microservices, no Docker sidecars. One API, full FFI bridge to native engines. Inference-only. PHP 8.3+.

Status: early release (v0.1.1). The public API is stabilizing and may change before 1.0 — pin a version and skim the CHANGELOG when upgrading. Code quality is production-grade (PHPStan level 8, Psalm level 3, 793 tests green on Windows + Linux).

Contents

• Quick example
• Why FerryAI
• Backends
• Vector store
• Observability & model pool
• Install
• Dependencies
• Capabilities
• Packages
• Testing
• Examples
• Documentation
• Contributing & license

Quick example

Embeddings & vector search — semantic RAG in 8 lines:

use FerryAI\AI;

AI::config([
    'backend' => 'onnx',
    'backends' => ['embedding' => ['model_path' => '/models/all-MiniLM-L6-v2-onnx']],
]);

// embed → 384d vector, then store and search
$vec = AI::embed('Hello world');
$store = AI::vector('docs');
$store->add('doc1', $vec->vector, ['title' => 'Getting Started']);
$hits = $store->search(AI::embed('semantic query')->vector, k: 5);

// similarity between any two texts
echo AI::similarity('cat', 'kitten');   // 0.79

// compose a processing pipeline
$results = AI::pipeline()
    ->pipe(new TransformStage(strtoupper(...)))
    ->pipe(new FilterStage(fn($x) => strlen($x) > 3))
    ->run(['hi', 'hello', 'hey']);

Chat & streaming — local LLM in 3 lines:

AI::config(['backend' => 'llama', 'backends' => ['llama' => ['model_path' => '/models/qwen.gguf']]]);

echo AI::chat('Explain PHP FFI in one sentence.');        // full reply
foreach (AI::stream('Write a haiku about ferries.') as $token) { echo $token; }

// structured output via JSON Schema → GBNF grammar
$json = AI::chat('List 3 famous bridges with year and city.', [
    'grammar' => [
        'type' => 'object',
        'properties' => ['bridges' => [
            'type' => 'array',
            'items' => ['type' => 'object', 'properties' => [
                'name' => ['type' => 'string'],
                'year' => ['type' => 'integer'],
                'city' => ['type' => 'string'],
            ]],
        ]],
    ],
]);

// HTTP streaming response (PSR-7 SSE/NDJSON) for web apps
return AI::streamResponse([['role' => 'user', 'content' => $prompt]]);

Why FerryAI

FerryAI loads native shared libraries (onnxruntime.dll, llama.dll) directly via PHP FFI — the same C APIs that Python uses. No subprocess, no shell_exec, no Python. Tokenizers, vector search and tensor math all run in pure PHP when native equivalents are unavailable.

Backends

LLM in detail

AI::config([
    'backend'  => 'llama',
    'device'   => 'cuda',   // or 'cpu'
    'backends' => ['llama' => ['model_path' => '/models/model.gguf', 'n_gpu_layers' => 35]],
]);

echo AI::chat('Summarize FFI in PHP.');

Configure the wrapper via FERRY_AI_LLAMA_WRAPPER (or FERRY_AI_LLAMA_LIB), add that dir to PATH. Sampling is per-request: temperature: 0 → greedy, > 0 → top-p; force one with ['sampler' => 'top_k'] or supply a ['grammar' => '<gbnf>'] / JSON Schema. Build steps: docs/DOCUMENTATION.md · native/llama-wrapper/README.md. Run: examples/03-chat.php · examples/04-streaming.php · examples/09-grammar.php.

Vector store

Two interchangeable backends behind the same VectorStore contract — pick per environment:

AI::config(['vector' => [
    'driver' => 'pgsql',                                     // or omit for SQLite
    'dsn' => 'pgsql:host=127.0.0.1;port=5432',
    'user' => 'postgres', 'password' => 'postgres',
]]);

$store = AI::vector('docs');
$store->add('doc1', $vec->vector, ['lang' => 'en']);
$hits = $store->search($query, k: 5, filter: ['lang' => ['eq' => 'en']]);

SQLite transparently uses sqlite-vec (vec0 virtual tables) for native KNN on PHP 8.4+, and falls back to pure-PHP brute-force otherwise — filters always work. examples/21-postgres-vector.php · examples/23-sqlite-vec.php.

Observability & model pool

Instrumentation lives at the facade layer (backends stay isolated). Off by default — zero overhead when disabled:

AI::config(['observability' => ['metrics' => true, 'profiling' => true, 'logging' => true]]);

AI::embed('hello');                 // automatically timed, counted and logged
print_r(FerryAI\Metrics::report()); // counters + timing histograms per operation
print_r(FerryAI\Profiler::report());// per-operation count / avg / min / max ms

AI::warmup([...]) preloads models into a memory-bounded LRU ModelPool; classify() / moderate() / predict() / chat() reuse pooled instances. Opt into cross-worker weight sharing via ext-shmop. Downloads retry transient failures. examples/22-observability.php.

Install

composer require ferry-ai/php-inference

Base requirements: PHP 8.3+, ext-ffi, ext-json, ext-hash, ext-fileinfo.

After install — run the diagnostic to see what's available:

vendor/bin/ferry-ai check              # PHP, extensions, backends, cache — full report
vendor/bin/ferry-ai check --json       # machine-readable

# Download models from HuggingFace and start using them immediately
vendor/bin/ferry-ai models:download sentence-transformers/all-MiniLM-L6-v2
vendor/bin/ferry-ai chat "Explain FFI in one sentence."
vendor/bin/ferry-ai chat "Hello" --stream --max=100

Everything else is optional and on-demand — install only what a feature needs. FerryAI degrades gracefully (pure-PHP fallback or a clear "not available" message) when a native library or model is missing.

Dependencies

What you need for each capability. Full source list with versions: docs/SOURCES.md.

GPU setup guide (CUDA/cuDNN/curand/cufft + llama.cpp build): docs/DOCUMENTATION.md → Quick Start → GPU setup. GPU→CPU fallback is automatic and silent.

Capabilities

Inference

Structured generation

Vector store

Model management

Concurrency

Developer experience

Platform

Packages

packages/
├── core/          Contracts, enums, value objects, exceptions, AIConfig
├── tensor/        ArrayTensor (pure PHP), BackedTensor, TensorFactory
├── onnx-backend/  ONNX Runtime via ankane/onnxruntime FFI
├── llama-backend/ llama.cpp FFI, samplers (greedy/top-k/top-p/grammar),
│                  GBNF grammar, JSON Schema→GBNF, ChatFormatter (5 templates)
├── tokenizer/     Pure PHP BPE + WordPiece (round-tripping, chunking)
├── embedding/     Mean/CLS/EOS/Max pooling, 4 built-in models
├── vector/        SQLite + PostgreSQL/pgvector store, brute-force & native ANN, metadata filtering
├── model-hub/     HF download, LRU cache, SHA-256+Ed25519, format detection
├── pipeline/      Generator-based stages (8 types)
├── cpu-backend/   Pure-PHP tensor math + optional RubixML (.rbm) tabular inference
├── dataframe/     Tabular data: typed columns, CSV/JSON I/O, Tensor conversion
├── ai/            Facade (AI::), backend registry, model pool, metrics, profiler
├── laravel/       Service provider + facade (env-based config)
└── symfony/       Bundle + DI extension

Testing

composer test                # Unit tests — 793 pure-PHP tests
composer test-integration    # Integration — needs ONNX Runtime / llama.cpp / PostgreSQL
composer check               # Lint (CS + PHPStan lvl8 + Psalm lvl3) + unit tests — gate

Examples

examples/ — 26 standalone scripts covering every capability: embedding, tokenizer, chat, streaming, RAG, pipeline, SQLite + sqlite-vec & PostgreSQL/pgvector, grammar-constrained generation, model hub, profiling, async fibers, model pool, observability, retry, CPU tensor math + RubixML, benchmarks, Laravel, Symfony.

set FERRY_AI_MODEL_DIR=C:\models\all-MiniLM-L6-v2-onnx
php examples/01-hello-embedding.php

Documentation

Start here: docs/DOCUMENTATION.md — definitive single-file reference (architecture, facade API, contracts, GPU setup).

Guides: getting-started · configuration · ONNX / llama.cpp · embedding · vector store · pipeline · model hub · safetensors → GGUF · tokenizer · streaming · security · deployment · Laravel / Symfony · troubleshooting · API reference · CHANGELOG

Contributing & license

• Contributing: guidelines and workflow in .github/CONTRIBUTING.md.
• Security: report vulnerabilities via .github/SECURITY.md — please do not open public issues for them.
• Code of Conduct: .github/CODE_OF_CONDUCT.md.
• License: MIT — see LICENSE.md.