README

Core AI abstraction layer for Laravel. Owns the contract, the retry trait, the response cache, the token estimator, and the conversation builder that every provider package in the ubxty family (AWS Bedrock, Azure OpenAI, …) extends. Provider-specific logic — SDK wiring, bearer/IAM auth, wire-format caching markers — lives in the provider packages.

This package ships:

• The AiManagerContract interface every provider manager implements.
• The AbstractAiManager base class with response-cache, max-tokens clamp + fits gate, idempotency-key, cost-limit enforcement, event dispatching, and invocation logging baked into invoke() / converse() / converseStream().
• The HasRetryLogic trait with key rotation, exponential backoff, and Retry-After honouring.
• The ConversationBuilder fluent API for multi-turn, multimodal conversations.
• The TokenEstimator, ModelSpecResolver, ModelAliasResolver helpers.
• The InvocationLogger for structured invocation telemetry.
• Three events (AiInvoked, AiKeyRotated, AiRateLimited) and four exceptions (AiException, ConfigurationException, RateLimitException, CostLimitExceededException).
• Abstract command bases so every provider package's CLI has the same UX.

Provider packages built on top:

• ubxty/bedrock-ai — AWS Bedrock (IAM + bearer/ABSK auth, multi-key failover, prompt-cache checkpoints).
• ubxty/azure-ai — Azure OpenAI (Microsoft Foundry v1 + traditional endpoints).

Contents

1. Installation
2. Why a core package?
3. Quickstart
4. The contract at a glance
5. AbstractAiManager reference
6. Cost Optimisations (v2.1.0+)
7. HasRetryLogic trait
8. ConversationBuilder
9. TokenEstimator
10. ModelSpecResolver
11. ModelAliasResolver
12. InvocationLogger
13. Events
14. Exceptions
15. Configuration
16. Testing
17. Extending core-ai
18. Contributing
19. Security
20. Changelog
21. License

Installation

composer require ubxty/core-ai

The service provider is auto-discovered through Laravel's package discovery. PHP 8.2+ and Laravel 11 or 12 are required.

To publish the config (optional — the defaults are usable out of the box):

php artisan vendor:publish --tag=core-ai-config

This copies config/core-ai.php into your app, where you can customise defaults and add provider blocks (bedrock, azure_ai, …).

Why a core package?

Writing an AI provider package sounds simple — call the SDK, get a string back. The hard parts show up a week later.

If you're building your own provider package, extending AbstractAiManager and HasRetryLogic gets all of the above with ~200 lines of glue code (see the /docs/extending-core-ai.md cookbook).

Quickstart

use Ubxty\BedrockAi\BedrockManager;
// — or —
// use Ubxty\AzureAi\AzureManager;

$result = app(BedrockManager::class)->invoke(
    modelId: 'claude-sonnet-4-20250514',  // or your 'default' model
    systemPrompt: 'You are a helpful assistant.',
    userMessage: 'Summarise this in one sentence: …',
    maxTokens: 1024,
    temperature: 0.3,
);

echo $result['response'];
// [
//     'response'      => 'The package adds 6 cost-saving hooks across …',
//     'input_tokens'  => 128,
//     'output_tokens' => 47,
//     'total_tokens'  => 175,
//     'cost'          => 0.0017,
//     'latency_ms'    => 812,
//     'status'        => 'ok',
//     'key_used'      => 'Primary',
//     'model_id'      => 'anthropic.claude-sonnet-4-20250514-v1:0',
// ]

Both BedrockManager and AzureManager extend this package's AbstractAiManager. The invoke(), converse(), and conversation() calls are defined here; providers implement the abstract perform* hooks.

For a longer walkthrough see docs/getting-started.md.

The contract at a glance

Every provider manager implements Ubxty\CoreAi\Contracts\AiManagerContract.

All 19 methods are documented in /docs/real-world-patterns.md and /docs/extending-core-ai.md.

AbstractAiManager reference

AbstractAiManager implements the contract and adds the cost-optimisation pipeline. Concrete provider classes (Bedrock, Azure) extend this.

Public methods added by the base class

Abstract perform* hooks (must be implemented by providers)

Protected helpers available to subclasses

Cost Optimisations (v2.1.0+)

The base manager runs the following pre-flight + cache hooks before any provider call. Each is independently opt-in via config.

1. Max-tokens clamp + fits gate

invoke() and converse() call ModelSpecResolver::resolve($modelId) to fetch the model's context_window and max_tokens ceiling, then clamp the requested max_tokens to the lowest of:

• the model's max_tokens cap,
• (context_window − estimated_input_tokens),

so callers can pass generous maxTokens values without provoking upstream 400s. Multi-turn calls use TokenEstimator::estimateMultimodal($messages, $systemPrompt) for the input estimate.

$result = $manager->invoke(
    'claude-3-haiku-20240307',
    $system,
    $longUserMessage,
    maxTokens: 16384,           // ← Haiku caps at 4096 → silently clamped to 4096
);

2. Response cache (cache.response_ttl)

When core-ai.cache.response_ttl > 0, identical (modelId, systemPrompt, userMessage, maxTokens, temperature) calls return the previous result without hitting the provider. Multi-turn converse() hashes the canonical messages JSON array.

// config/core-ai.php
'cache' => [
    'response_ttl' => 3600, // memoise for 1 hour
],

Cache key: sha256("{platform}_response_{model}|{system}|{user}|{maxTokens}|{temp}") (single-turn) and the SHA256 of the canonical messages JSON (multi-turn).

Cached responses carry:

[
    'response'    => '…',
    'cached'      => true,
    'latency_ms'  => 0,
    // …everything the original result had
]

Important: enable only when the prompt is deterministic for a given set of inputs. Chat UIs with rapidly-changing conversation history should leave response_ttl = 0.

3. Idempotency-Key

AbstractAiManager::idempotencyKey($modelId, $content) returns a deterministic hash suitable for upstream Idempotency-Key HTTP headers. Provider packages inject this on the Bearer-mode HTTP path so a network-blip retry returns the same cached upstream response instead of double-billing.

$key = $manager->idempotencyKey($modelId, $systemPrompt.$userMessage);
// "aws_bedrock_ai-<sha256 hash>"

4. Embedding cache (cache.embedding_ttl)

cache.embedding_ttl (default 604800 = 7 days) is consumed by BedrockManager::embed() and AzureManager::embed() to memoise per-text embeddings. Embeddings are deterministic for a fixed model ID + dimensions, so caching eliminates redundant ingestion.

5. Cost-limit enforcement

Set core-ai.{bedrock,azure_ai}.limits.daily and/or .monthly (in USD). When the cumulative spend for the day or month exceeds the limit, CostLimitExceededException fires before any provider call.

try {
    $manager->invoke(…);
} catch (CostLimitExceededException $e) {
    // $e->getLimitType()  → 'daily' / 'monthly'
    // $e->getLimit()      → 10.0
    // $e->getCurrentSpend() → 10.42
}

6. Prompt caching (provider-specific)

The Bedrock provider package implements wire-format caching on top of core-ai:

• Bedrock — cachePoint: { type: 'default' } markers injected on Converse content arrays. Configured via core-ai.bedrock.prompt_caching.points.

See /docs/caching-strategy.md for the full deep-dive including a worked "before/after v2.1.0" cost comparison.

HasRetryLogic trait

Ubxty\CoreAi\Client\HasRetryLogic is the shared retry + key-rotation trait. Provider-specific Client classes (use HasRetryLogic;) get the same behaviour for free.

Public API

trait HasRetryLogic {
    public function setPromptCachePoints(array $points): static;   // v2.1.1
    public function setRetryAfterSeconds(?int $seconds): static;  // v2.1.1
}

setPromptCachePoints() accepts ['system', 'last_user'] (other values are silently filtered out).

setRetryAfterSeconds() is for the HTTP path to record the upstream Retry-After hint before throwing on 429. The trait's withRetry() loop prefers the hint over the exponential backoff when set, then consumes one hint per iteration.

Override hooks (in subclasses)

HasRetryLogic defines the retry control flow, but its onKeyRotated() and onRateLimitExhausted() hooks are empty in core-ai. Provider packages override these hooks when they need to dispatch provider-specific lifecycle events.

| Hook | Override to | |---|---|---|---| | withRetry(string $modelId, callable $callback): array | The main retry+rotation loop. Most providers don't override this directly — base behaviour is correct. | | resolveModelId(string $modelId, array $key): string | Apply per-key/per-region model transformations (e.g. Bedrock inference profiles). | | isRateLimitError(string $message): bool | Add platform-specific rate-limit fingerprints. | | extractFriendlyError(string $errorMessage): string | Map raw provider errors to user-friendly text. | | resetPlatformClient(): void | Drop a cached SDK client (e.g. when rotating IAM creds). | | onKeyRotated(array $fromKey, array $toKey, string $reason, string $modelId): void | Dispatch an AiKeyRotated event (core-ai's default hook is empty). | | onRateLimitExhausted(string $modelId, array $key, int $retryAttempt): void | Dispatch an AiRateLimited event (core-ai's default hook is empty). | | calculateCost(int $in, int $out, ?array $pricing): float | Provider-specific cost math. |

Default behaviour

withRetry():

for each key in credentials (N keys):
    for attempt = 0..maxRetries:
        try:
            return callback(currentKey)
        except exception e:
            if isRateLimitError(e) and attempt < maxRetries:
                if retryAfterSeconds is set:
                    sleep(retryAfterSeconds)   # honour upstream hint
                else:
                    sleep(baseDelay ** attempt) # exponential
                continue
            resetPlatformClient()
            if credentials.next():
                onKeyRotated(from, to, reason, modelId)
                break  # try next key
            if isRateLimitError(e):
                onRateLimitExhausted(modelId, key, attempt)
                throw RateLimitException
            throw AiException
throw AiException('All credential keys exhausted.')

ConversationBuilder

Ubxty\CoreAi\Conversation\ConversationBuilder is a fluent wrapper around converse() for multi-turn, multimodal flows.

use App\Ai\MyManager;

$result = app(MyManager::class)
    ->conversation('claude-sonnet-4')
    ->system('You are a careful reader.')
    ->user('Here is a contract. List the unusual clauses.')
    ->userWithDocument('Anything you missed?', '/tmp/contract.pdf')
    ->assistant('…prior assistant turn…')
    ->maxTokens(2048)
    ->temperature(0.1)
    ->withPricing(['input_price_per_1k' => 0.003, 'output_price_per_1k' => 0.015])
    ->connection('us')
    ->send();

echo $result['response'];

Fluent methods

send() returns the same array{response, input_tokens, output_tokens, total_tokens, stop_reason, latency_ms, model_id, key_used, cost} as converse(). The builder appends the assistant reply to its in-memory history; if you ->send() twice, the second call sees the first reply as a prior assistant turn.

For image and document files larger than 15 MB, the builder throws AiException before any HTTP call. Resize / compress first.

TokenEstimator

Ubxty\CoreAi\Support\TokenEstimator is a static helper with rough-but-cheap heuristics.

use Ubxty\CoreAi\Support\TokenEstimator;

$tokens = TokenEstimator::estimateMultimodal(
    [
        ['role' => 'user', 'content' => [
            ['type' => 'image', 'format' => 'jpeg', 'data' => $b64],
            ['type' => 'text', 'text' => 'What is in this image?'],
        ]],
    ],
    'You are a vision model.'
);

// int(1700) — 1600 for image + ~100 for system.

Note — the heuristic is intentionally character-based for cross-lingual robustness. Replacing it with a real BPE tokenizer is on the v2.2 roadmap.

ModelSpecResolver

Ubxty\CoreAi\Models\ModelSpecResolver is a static catalog of context_window + max_tokens for every supported model. The manager calls it during max-tokens clamp.

Supported families: claude-3.5, claude-4, gpt-4o, gpt-4, gpt-3.5, nova, titan, llama-3, llama-4, mistral, cohere, jamba. See source for the per-model specs.

Override per-model defaults by adding entries to core-ai.bedrock.models or core-ai.azure_ai.models in your published config — the resolver still wins when the catalog has no match.

ModelAliasResolver

Ubxty\CoreAi\Client\ModelAliasResolver is a tiny alias registry.

$resolver = new ModelAliasResolver([
    'sonnet' => 'anthropic.claude-sonnet-4-20250514-v1:0',
    'mini'   => 'amazon.nova-micro-v1:0',
]);

$resolver->resolve('sonnet');         // 'anthropic.claude-sonnet-4-20250514-v1:0'
$resolver->resolve('mystery-model');  // 'mystery-model' (pass-through)
$resolver->isAlias('sonnet');         // true
$resolver->register('haiku', 'anthropic.claude-3-5-haiku-20241022-v1:0');
$resolver->all();                     // ['sonnet' => '…', 'mini' => '…', 'haiku' => '…']

Aliases are seeded from core-ai.{bedrock,azure_ai}.aliases config. The manager exposes the resolver via aliases() and auto-resolves through resolveAlias() on every invocation.

InvocationLogger

Ubxty\CoreAi\Logging\InvocationLogger writes structured records to a Laravel log channel.

Configure via core-ai.logging:

'logging' => [
    'enabled' => true,
    'channel' => 'stack', // or 'ai', 'daily', …
],

The logger never logs the response body — only counters and metadata. Pair with a custom channel that writes to your audit DB or SIEM for full observability.

Events

Core-ai defines three Dispatchable event payloads. AbstractAiManager dispatches AiInvoked after successful invocations. The default HasRetryLogic hooks for key rotation and rate-limit exhaustion are empty; provider packages decide whether to dispatch AiKeyRotated and AiRateLimited from their overrides.

AiInvoked

new AiInvoked(
    modelId: 'anthropic.claude-sonnet-4-20250514-v1:0',
    inputTokens: 128,
    outputTokens: 47,
    cost: 0.0017,
    latencyMs: 812,
    keyUsed: 'Primary',
    connection: 'default',   // optional
    platform: 'AWS Bedrock', // optional
);

Fires after every successful invocation — including cached hits. Pair with a listener that writes to a usage table for tenant-level dashboards.

AiKeyRotated

new AiKeyRotated(
    fromKeyLabel: 'Primary',
    toKeyLabel:   'Secondary',
    reason:       '429: Too many requests',
    modelId:      'anthropic.claude-sonnet-4-20250514-v1:0',
    platform:     'AWS Bedrock',
);

Provider packages may dispatch this from an onKeyRotated() override. Core-ai's default hook is empty.

AiRateLimited

new AiRateLimited(
    modelId:      'anthropic.claude-sonnet-4-20250514-v1:0',
    keyLabel:     'Primary',
    retryAttempt: 3,
    waitSeconds:  17,
    platform:     'AWS Bedrock',
);

Provider packages may dispatch this from an onRateLimitExhausted() override. Core-ai's retry loop does not dispatch it before retry sleeps, and the default exhaustion hook is empty.

Exceptions

use Ubxty\CoreAi\Exceptions\RateLimitException;
use Ubxty\CoreAi\Exceptions\CostLimitExceededException;

try {
    return $manager->invoke(…);
} catch (CostLimitExceededException $e) {
    return response()->json([
        'error'  => 'daily_limit',
        'limit'  => $e->getLimit(),
        'spent'  => $e->getCurrentSpend(),
    ], 402);
} catch (RateLimitException $e) {
    return response()->json(['error' => 'temporarily_busy'], 503);
}

Configuration

The full config/core-ai.php ships with these top-level keys. Provider packages add their own top-level blocks (bedrock, azure_ai); see the provider docs for those.

Both provider blocks (core-ai.bedrock.*, core-ai.azure_ai.*) include default, connections (with nested keys), retry, limits, cache, providers.disabled_providers, providers.chat.disabled_providers, providers.image.disabled_providers, defaults, aliases, models, logging, and health_check. Bedrock additionally includes pricing, usage, and prompt_caching; those keys are not part of the Azure block. See each provider's README for the full reference.

Don't dump the config file in your README. Read ubxty/bedrock-ai README § Configuration for the canonical shape; copying it here would diverge on the next bump.

Testing

The abstract manager exists for testability. Bind a fake manager in the container and the rest of your test suite does not need to know which provider is wired:

use Ubxty\CoreAi\Contracts\AiManagerContract;
use Ubxty\CoreAi\Manager\AbstractAiManager;

class FakeManager extends AbstractAiManager
{
    public array $calls = [];
    public array $stub = [
        'response' => 'stub',
        'input_tokens' => 0, 'output_tokens' => 0,
        'cost' => 0, 'latency_ms' => 0,
        'status' => 'ok', 'key_used' => 'fake',
        'model_id' => 'fake-model',
    ];

    public function invoke(…): array
    {
        $this->calls[] = func_get_args();
        return $this->stub;
    }

    // … implement the other abstract hooks …
    protected function performInvoke(…) { return $this->stub; }
    protected function performConverse(…) { return $this->stub; }
    protected function performConverseStream(…) { return $this->stub; }

    public function testConnection(?string $c = null): array { return ['success' => true, 'message' => 'fake', 'response_time' => 0]; }
    public function listModels(?string $c = null): array { return []; }
    public function fetchModels(?string $c = null): array { return []; }
    public function syncModels(?string $c = null): int { return 0; }
    public function isConfigured(?string $c = null): bool { return true; }
    public function supportsStreaming(?string $c = null): bool { return false; }
    public function getCredentialInfo(?string $c = null): array { return []; }
    public function platformName(): string { return 'Fake'; }
}

beforeEach(function () {
    app()->instance(AiManagerContract::class, new FakeManager(['defaults' => ['model' => 'fake']]));
});

Extending core-ai

For a complete walkthrough — building your own provider from scratch — see docs/extending-core-ai.md. The cookbook covers:

1. Why AbstractAiManager is the right base.
2. The 11 abstract hooks, with stubs.
3. Implementing AbstractCredentialManager for your auth model.
4. Wiring HasRetryLogic into your client classes.
5. Mapping your SDK's response into the standard array.
6. Registering the service provider, merging config, optional facade.
7. An 80-line "hello provider" template (no SDK).

For real-world patterns — embedding pipelines, multi-tenant providers, RAG pipelines, streaming responses, cost-cap listeners — see docs/real-world-patterns.md.

Contributing

Open a PR on the main branch. Match existing code style:

• PHP 8.2+ syntax (readonly, match, named args).
• Strict-typing on all new public methods — no mixed returns unless documented.
• Tests are out of scope for this project (see the host-app convention). The maintainer verifies regressions by hand against the rubric report.

Security

• Provider keys are never logged, never serialised through configuration dumps. getCredentialInfo() returns {index, label} only.
• Bearer tokens are loaded from env at request time, not committed to config/core-ai.php.
• The ConversationBuilder 15 MB cap on image/document payloads prevents accidental OOM at the SDK layer — bigger inputs go to the SDK but never blow past that cap.

Report vulnerabilities via info.ubxty@gmail.com. PGP key on request.

Changelog

See CHANGELOG.md. Past minor versions:

• 2.1.3 — ConversationBuilder helpers plus corrections that bring the v2.1.2 docs in line with the shipped config and event hooks.
• 2.1.2 — Documentation overhaul. No API changes.
• 2.1.1 — setPromptCachePoints() + setRetryAfterSeconds() on the canonical HasRetryLogic trait; Retry-After honouring.
• 2.1.0 — clampMaxTokens(), response-cache layer, idempotencyKey(). New cache TTL knobs (response_ttl, embedding_ttl). Bedrock prompt_caching config block.
• 2.0.0 — Unified core-ai namespace. bedrock/azure_ai config blocks now live under one file.
• 1.0.0 — Initial release extracted from ubxty/bedrock-ai.

License

MIT — see LICENSE.