Skip to content

test(ai): require explicit opt-in for Ollama live tests - #559

Merged
code-yeongyu merged 5 commits into
code-yeongyu:mainfrom
ChickenBreast-ky:fix/ollama-live-test-gate
Aug 3, 2026
Merged

test(ai): require explicit opt-in for Ollama live tests#559
code-yeongyu merged 5 commits into
code-yeongyu:mainfrom
ChickenBreast-ky:fix/ollama-live-test-gate

Conversation

@ChickenBreast-ky

@ChickenBreast-ky ChickenBreast-ky commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Why

A normal npm test run treated the presence of the ollama executable as permission to enter the live Ollama suite. That path can pull gpt-oss:20b, start a local server, and load a large model without explicit consent.

What changed

  • Gate Ollama discovery behind PI_ENABLE_LOCAL_LLM=1 or PI_ENABLE_LIVE_API_TESTS=1.
  • Keep the actual which ollama command inside the shared gate and mock that command boundary in regression tests.
  • Verify the default path does not probe Ollama and both explicit opt-in paths do.
  • Update test.sh to clear the opt-in flags instead of exporting the retired PI_NO_LOCAL_LLM opt-out flag.
  • Record the fork-specific change and expected conflict zones in packages/ai/changes.md.

Validation

  • npx vitest run packages/ai/test/live-api-gates.test.ts packages/ai/test/stream.test.ts
  • npx biome check --error-on-warnings packages/ai/test/live-api-gates.ts packages/ai/test/live-api-gates.test.ts packages/ai/test/stream.test.ts
  • npx tsgo --noEmit
  • bash -n test.sh
  • npm run build
  • npm test

The default-env runs did not start an Ollama process, open TCP 11434, or modify the Ollama server log. The positive live Ollama suite was intentionally not executed; its opt-in boundary is covered with a mocked command.


Summary by cubic

Require explicit opt-in to run Ollama live tests so npm test no longer probes or starts local models by default. The probe is now portable across Unix and Windows.

  • Bug Fixes

    • Gate Ollama discovery behind PI_ENABLE_LOCAL_LLM=1 or PI_ENABLE_LIVE_API_TESTS=1 and short-circuit before probing; use which on Unix and where on Windows.
    • Replace the legacy PI_NO_LOCAL_LLM opt-out; update tests to mock child_process.execSync and update test.sh to unset the new opt-in flags.
    • Update stream.test.ts to use the shared availability check and skip by default.
  • Migration

    • To run the live Ollama suite, set PI_ENABLE_LOCAL_LLM=1 or PI_ENABLE_LIVE_API_TESTS=1.

Written for commit 6393b5b. Summary will update on new commits.

Review in cubic

@code-yeongyu
code-yeongyu merged commit 88bf6e5 into code-yeongyu:main Aug 3, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants