[Extension]: Evaluator Contract

[tbitcs]

Extension Submission

Extension Metadata

• ID: evaluator
• Name: Evaluator Contract
• Version: 1.0.0
• Description: Standard evaluator result contract for evidence, provenance, uncertainty, and recovery — a provider-neutral protocol for extensions that evaluate artifact quality between Spec-Driven Development phases
• Author: ElectroHire
• License: MIT

Links

• Repository: https://github.com/electrohire/spec-kit-evaluator
• Download URL: https://github.com/electrohire/spec-kit-evaluator/archive/refs/tags/v1.0.0.zip
• Documentation: https://github.com/electrohire/spec-kit-evaluator#readme
• Changelog: https://github.com/electrohire/spec-kit-evaluator/blob/main/CHANGELOG.md

Requirements

• Spec Kit Version: >=1.0.0
• Tools: None required

Provides

• Commands: 4 (speckit.evaluator.run, .compose, .report, .route)
• Hooks: 4 (after_specify, after_plan, after_tasks, after_implement)

Tags

evaluator, evidence, provenance, quality, governance, compliance, workflow, model-routing, portfolio

Key Features

• JSON Schema for evaluator results (6 outcomes, 14 finding kinds, 5 evidence kinds)
• Deterministic composition with contradiction detection (strict/majority/optimistic)
• Model routing: recommends budget/standard/premium tier per phase
• All 5 report formats: terminal, markdown, JSON, CI annotations, gate
• 3 parity scripts: Python, Bash, PowerShell
• Quick-start demo: python examples/demo.py
• 70 tests with 0 regressions against spec-kit test suite
• Token-economic benchmark suite with Monte Carlo simulation

Testing Confirmation

• Tested locally with uv run specify --help
• Ran existing tests with uv sync && uv run pytest
• Tested with a sample project (install → verify → uninstall)

Refs: #4290, #4390

[github-actions]

cc @github/spec-kit-maintainers — new catalog submission for review.

[github-actions]

Validation failed for this community extension submission:

• Proposed Catalog Entry: The required Proposed Catalog Entry field is missing from the issue body. Add the complete proposed catalog JSON entry.
• Testing Checklist: The required Testing Checklist section is missing. The body includes a differently named Testing Confirmation checklist, so the required checklist cannot be confirmed.
• Submission Requirements: The required Submission Requirements section and its checkboxes are missing.

The extension ID (evaluator), version (1.0.0), repository and required repository files (extension.yml, README.md, LICENSE), release tag (v1.0.0), and download URL pattern were otherwise validated successfully. Please update the issue with the missing required fields and checklists, then submit it again for validation.

Generated by 🧩 Add Community Extension from Issue Submission for issue #4403 · 26.9 AIC · ⊞ 29.2K · ◷

[tbitcs]

Superseded by #4414, which was resubmitted through the current Extension Submission form with the required field headings, Testing Checklist, Submission Requirements, and proposed catalog JSON. Please use #4414 for review.

AI-assistance disclosure: This administrative comment was drafted with ChatGPT assistance and reviewed by me.