README

Test Output Formatter

Isolates failing test files and under-coverage classes for rapid triage.

Benefits for AI and Triage

• Token Efficiency: Minimizes output to only what's necessary, saving tokens and reducing noise in AI-driven workflows.
• Rapid Triage: Focuses attention on the 1% of files that actually need fixing, speeding up the development cycle.

• Test Output Formatter
  • Benefits for AI and Triage
  • Installation
    • Auto-Registration
  • Features
    • PHPStan
    • Pest
• Test Output Formatter Skill
  • Pest Plugin Flags
    • --format=json
    • --cache-dir=<dir>
    • --rerun-failures
    • --changed
    • --since=<ref>
    • --under=<percentage>
    • --slowest=<count>
    • --over=<ms>
  • PHPStan Error Formatters
  • AI Agent Integration
    • ErrorFormatter
      • Public methods of ErrorFormatter
    • CompactErrorFormatter
      • Public methods of CompactErrorFormatter
    • JsonErrorFormatter
      • Public methods of JsonErrorFormatter

Installation

composer require --dev schenke-io/test-output-formatter

Auto-Registration

The package uses standard discovery mechanisms to integrate with your tools:

• PHPStan: The extension is automatically registered via the phpstan-extension type in composer.json and the extension.neon file. For this to work seamlessly, it is highly recommended to install the extension installer:

composer require --dev phpstan/extension-installer

• Pest: The plugin is automatically registered through the extra.pest.plugins configuration in composer.json. No additional configuration is required.

Features

PHPStan

• Error Formatter: Output only file paths with errors for quick consumption by other tools.
• Usage:
  • File paths only:

    vendor/bin/phpstan analyse --error-format=testOutput

  • Compact format: (file:line message)

    vendor/bin/phpstan analyse --error-format=testOutputCompact

  • JSON format:

    vendor/bin/phpstan analyse --error-format=testOutputJson

Pest

• Integration: Plugin to assist in isolating failing tests and checking coverage.
• Usage:
  • Failures only: Just gives you the list of failing test files.

    vendor/bin/pest --parallel --failed-files-only

  • Coverage check: Reports classes with line coverage below the specified threshold.

    vendor/bin/pest --parallel --under=80

  • Slow tests: Identifies tests that take longer than a threshold or the N slowest tests.

    vendor/bin/pest --parallel --over=100 --slowest=5

  • JSON Output: Returns results as a JSON object (includes exit code, failures, timing, and coverage).

    vendor/bin/pest --format=json

  • Caching: Stores results (failures, timing, coverage map) to speed up subsequent runs.

    vendor/bin/pest --cache-dir=.pest-cache --rerun-failures

  • Git-based Selection: Runs only tests affected by changes since a specific ref or in the current working tree. Requires --cache-dir for best results (to use the coverage map).

    vendor/bin/pest --changed
    vendor/bin/pest --since=main

Test Output Formatter Skill

This package provides a Pest plugin and PHPStan error formatters to isolate failing tests, check coverage, and identify slow tests.

Pest Plugin Flags

--format=json

Returns results as a JSON object. Useful for CI and AI agents.

• exitCode: The exit code of the Pest run.
• failedFiles: List of failing test files.
• underCovered: List of files below coverage threshold, including uncovered lines.
• timing: List of tests and their execution time.
• coverageMap: Mapping from source files to the tests that cover them.

--cache-dir=<dir>

Specifies a directory to store test results. Required for --rerun-failures, --changed, and --since.

--rerun-failures

Only runs tests that failed in the previous run (requires --cache-dir).

--changed

Runs tests affected by changes in the current Git working tree.

• Uses coverageMap from cache to identify which tests to run for changed source files.
• Fails open (runs all tests) if no coverage map is found or if a source file is not in the map.

--since=<ref>

Runs tests affected by changes since the specified Git reference (e.g., main).

• Same logic as --changed.

--under=<percentage>

Reports files with coverage below the specified percentage. Triggers --coverage automatically if not present.

--slowest=<count>

Reports the N slowest tests.

--over=<ms>

Reports tests taking longer than the specified milliseconds.

PHPStan Error Formatters

• testOutput: File paths only.
• testOutputCompact: File, line, and message.
• testOutputJson: Full error details in JSON.

AI Agent Integration

When working with an AI agent:

1. Use --format=json to get a machine-readable summary of failures.
2. Use --cache-dir to maintain state across multiple agent steps.
3. Use --rerun-failures to quickly verify fixes.
4. Use --changed or --since to minimize the test suite for the current task.

ErrorFormatter

Public methods of ErrorFormatter

method	summary
formatErrors	Formats the errors and outputs them to the console.

CompactErrorFormatter

Custom PHPStan error formatter that outputs one line per error

Public methods of CompactErrorFormatter

method	summary
formatErrors	-

JsonErrorFormatter

Public methods of JsonErrorFormatter

method	summary
formatErrors	-

Markdown file generated by schenke-io/packaging-tools