Skip to content

chore: retire src/eval.ts and npm run eval#24

Merged
dhruva-reddy merged 1 commit into
mainfrom
chore/retire-eval-runner
May 11, 2026
Merged

chore: retire src/eval.ts and npm run eval#24
dhruva-reddy merged 1 commit into
mainfrom
chore/retire-eval-runner

Conversation

@dhruva-reddy
Copy link
Copy Markdown
Contributor

Summary

  • Delete src/eval.ts and the npm run eval script. Both have been dead-code-walking since Stack E shipped npm run sim (improvements.md entry feat: push --dry-run preview mode #16, resolved 2026-04-30). The two runners coexisting under similar names actively misled engineers and customer forks.
  • Scrub all doc references: README.md (commands table row, Direct-Mode example block, the ### Running Evals section, the src/ directory-tree comment) and AGENTS.md (testing-commands block).
  • Rewrite the two back-reference comments in src/sim.ts and src/sim-cmd.ts so they no longer cite the deleted file. Technical nuance ("env-loading is inlined to avoid triggering config.ts CLI arg parsing when imported from tests") is preserved.

The Vapi /eval REST endpoint, the evals resource type, the /eval/simulation/* simulation API, and docs/learnings/simulations.md are all unaffected — only the local CLI runner is gone.

Customer-fork callers: if any of your automation still invokes npm run eval, swap to npm run sim -- <org> --suite <name> --target <name>. See AGENTS.md for syntax.

Out of scope (pre-existing, not introduced here):

  • Pre-existing unresolved git merge-conflict markers in README.md around lines 520-530. Fix separately.
  • Stale back-references to src/eval.ts inside improvements.md and requested improvements.md. Those are user-curated trackers and per repo convention are not modified in feature/chore PRs.
  • The canonical repo has no biome.json/prettier config — running biome defaults reformats files spaces→tabs. Caught in this PR and reverted; worth a separate hygiene PR to pick a convention and lock it in.

Skipped pipeline stages:

  • test-writer skipped — pure deletion of unreachable code with no new behavior to specify. The existing tests/sim.test.ts already covers the surviving runner.
  • code-reviewer ran and approved with zero blocking findings.

Test plan

  • `npm run build` clean (tsc --noEmit, 0 errors)
  • `npm run test` — 115/115 tests pass
  • `npm run eval -- foo -s bar` errors with `Missing script: "eval"`
  • `npm run sim` (no args) still prints its usage banner
  • `grep -rn 'eval.ts' --include='.ts' --include='.json' .` returns zero hits in source/scripts (remaining mentions only in user-curated improvements.md / requested improvements.md, intentionally untouched)

`src/eval.ts` and the `npm run eval` script have been dead-code-walking
since Stack E shipped `npm run sim` (improvements.md #16, resolved
2026-04-30). The two runners coexisting under similar names actively
misled engineers and customer forks.

This change deletes the legacy runner and scrubs all doc references.
The Vapi `/eval` REST endpoint, the `evals` resource type, the
`/eval/simulation/*` simulation API, and `docs/learnings/simulations.md`
are all unaffected — only the local CLI runner is gone.

If customer-fork automation still invokes `npm run eval`, swap to
`npm run sim -- <org> --suite <name> --target <name>`.
@dhruva-reddy dhruva-reddy merged commit e86cd34 into main May 11, 2026
@dhruva-reddy dhruva-reddy deleted the chore/retire-eval-runner branch May 11, 2026 00:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant