Skip to content

Replace JavaScript client with Go binary and integration test runner#221

Merged
data-douser merged 5 commits intodd/ql-mcp-client/2from
copilot/remove-javascript-client
Apr 6, 2026
Merged

Replace JavaScript client with Go binary and integration test runner#221
data-douser merged 5 commits intodd/ql-mcp-client/2from
copilot/remove-javascript-client

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 6, 2026

Integration Testing Improvement PR: MCP Server Primitives

Use this template for PRs that implement an accepted issue created from: .github/ISSUE_TEMPLATE/mcp-integration-testing-improvement.yml.

Related Issue:

Scope Guard: This PR modifies files under client/ to replace the JavaScript-based integration test client with an equivalent Go binary. The integration test fixtures under client/integration-tests/** are preserved; only two test-config.json paths were adjusted.

Summary

Phase 2 of the ql-mcp-client migration: remove the JS-based ql-mcp-client.js and all supporting modules, replace with an equivalent Go CLI (gh-ql-mcp-client) built on Cobra + mcp-go. The integration test runner is ported from JavaScript to Go, preserving all existing test fixtures and their before/after structure.

Removed

  • client/src/ — all 18 JS files (entry point, 14 lib modules, 3 command modules)
  • client/package.json, client/eslint.config.mjs

Added — Go CLI framework

  • client/main.gocmd.Execute()
  • client/cmd/root.go — root command with --mode, --host, --port, --format flags
  • client/cmd/helpers.goparseRepo() utility
  • client/cmd/integration_tests.gointegration-tests subcommand wiring MCP client to test runner

Added — MCP client library (internal/mcp/)

  • Stdio transport (spawns node server/dist/... as child process) and HTTP transport
  • Per-tool timeout escalation (60s default, 5m for codeql_* CLI tools)
  • Graceful close with 3s timeout for stuck Node.js stdio servers

Added — Integration test runner (internal/testing/)

Go port of the JS integration-test-runner.js:

  • runner.go — discovers fixtures in client/integration-tests/primitives/tools/, enforces tool availability, handles deprecated session tools, priority-based ordering
  • params.go — parameter resolution chain: test-config.jsonmonitoring-state.json embedded params → tool-specific defaults (30+ tools)
  • {{tmpdir}} placeholder resolution to <repoRoot>/.tmp/

Modified

  • client/Makefile — removed @if [ -f go.mod ] guards; added VERSION extraction from cmd/root.go
  • client/scripts/run-integration-tests.sh — builds Go binary via make build, invokes gh-ql-mcp-client integration-tests instead of node src/ql-mcp-client.js
  • test-config.json (2 files) — logDir paths updated from client/.../after to server/.tmp/query-logs/custom-test-dir

Primitive Metadata

  • Primitive name(s): All MCP server tools covered by existing integration test fixtures (30+ tools)
  • Type(s): Tool
  • Related resources or prompts (if any): N/A — this PR replaces the test runner infrastructure, not the test fixtures themselves

Test Additions / Modifications

  • client/cmd/root_test.go : Unit tests for root command help output, version constant, default flag values, subcommand visibility
  • client/cmd/helpers_test.go : Unit tests for parseRepo() with valid placeholder values and invalid inputs
  • client/internal/mcp/client_test.go : Unit tests for tool timeout selection, config defaults, unconnected client close, constant values
  • client/internal/testing/runner_test.go : Unit tests for {{tmpdir}} placeholder resolution, tool priority ordering, string truncation, mock caller integration
  • client/internal/testing/params_test.go : Unit tests for test-config.json parsing, monitoring-state.json parameter extraction, tool-specific defaults (validate_codeql_query, codeql_resolve_languages), unknown tool error, file extension filtering
  • client/integration-tests/primitives/tools/codeql_query_run/custom_log_directory/test-config.json : Updated logDir path
  • client/integration-tests/primitives/tools/codeql_test_run/custom_log_directory/test-config.json : Updated logDir path

Before vs After File States

file: client/integration-tests/primitives/tools/codeql_query_run/custom_log_directory/test-config.json
  logDir: "client/.../after" -> "server/.tmp/query-logs/custom-test-dir"

file: client/integration-tests/primitives/tools/codeql_test_run/custom_log_directory/test-config.json
  logDir: "client/.../after" -> "server/.tmp/query-logs/custom-test-dir"

Assertions Added

  • Tool result validation (success, error handling)
  • Error path / negative case
  • Other: CLI flag parsing, tool timeout selection, parameter resolution chain, mock MCP caller

Edge & Negative Cases

  • Unknown tool names return errors from buildToolParams()
  • Invalid owner/repo strings (empty, no slash, missing owner, missing repo) return errors from parseRepo()
  • Deprecated session_*/sessions_* tools are skipped during integration test discovery
  • Unconnected MCP client Close() does not error
  • {{tmpdir}} placeholders in non-string values are left unchanged

Non-Goals / Deferred Follow-ups

  • Code Scanning and SARIF-specific CLI commands (Phase 3)
  • HTTP transport integration tests (requires running server)
  • CHANGELOG.md updates (deferred to release preparation)

Local Verification

# Build Go binary
make -C client build

# Run Go unit tests (16 tests)
make -C client test-unit

# Run linter
make -C client lint

# Full server test suite
npm run test:server

Checklist

  • Scope limited to client/integration-tests/**
  • All new tests pass locally
  • Does not introduce unrelated refactors
  • Documentation/comments updated where helpful
  • Re-ran tests to confirm deterministic results
  • Added negative test (or documented why not applicable)
  • Followed directory structure (primitives/tools/<tool>/<test>/before|after)
  • Each file in before has matching file in after
  • No binary or non-diffable files added
  • Tests focus on a single primitive concern
  • Ran npm run lint:fix and npm run build-and-test successfully
  • Checked for accidental changes outside allowed path
  • Unit tests use only placeholder/dummy values (no real repository references)

Screenshots / Logs (Optional)

All 16 Go unit tests pass:

ok  github.com/advanced-security/codeql-development-mcp-server/client/cmd
ok  github.com/advanced-security/codeql-development-mcp-server/client/internal/mcp
ok  github.com/advanced-security/codeql-development-mcp-server/client/internal/testing

Additional Notes

  • The Go binary default --mode flag is stdio (matching the most common local development workflow), unlike the reference branch which defaults to http.
  • CodeQL security scan: 0 alerts found.
  • Go test fixtures use t.TempDir() for isolated test directories.

Copilot AI and others added 3 commits April 6, 2026 16:20
- Remove client/src/ directory with all JS files (ql-mcp-client.js and 14 library modules)
- Remove client/package.json and client/eslint.config.mjs
- Add Go module (go.mod, go.sum) with cobra and mcp-go dependencies
- Add CLI entry point (main.go) and root Cobra command (cmd/root.go)
- Add CLI helpers (cmd/helpers.go) and integration test command (cmd/integration_tests.go)
- Add MCP client library (internal/mcp/client.go) with stdio and HTTP transport
- Add integration test runner (internal/testing/runner.go, params.go)
- Add comprehensive unit tests for all packages (16 tests, all passing)
- Update Makefile to remove go.mod guards (Go source now available)
- Update run-integration-tests.sh to build and use Go binary
- Update test-config.json logDir paths for custom_log_directory tests

Agent-Logs-Url: https://github.com/advanced-security/codeql-development-mcp-server/sessions/8c006672-cf7e-4045-9488-f6d97fafe2f2

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
Agent-Logs-Url: https://github.com/advanced-security/codeql-development-mcp-server/sessions/8c006672-cf7e-4045-9488-f6d97fafe2f2

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
Copilot AI changed the title [WIP] Remove JavaScript client and supporting library Replace JavaScript client with Go binary and integration test runner Apr 6, 2026
Comment thread client/cmd/helpers_test.go Outdated
@data-douser data-douser marked this pull request as ready for review April 6, 2026 16:42
@data-douser data-douser requested review from a team and enyil as code owners April 6, 2026 16:42
@data-douser data-douser merged commit 2c88aeb into dd/ql-mcp-client/2 Apr 6, 2026
1 check passed
@data-douser data-douser deleted the copilot/remove-javascript-client branch April 6, 2026 16:45
data-douser added a commit that referenced this pull request Apr 7, 2026
…221)

* Initial plan

* Replace JavaScript client with Go binary and integration test runner

- Remove client/src/ directory with all JS files (ql-mcp-client.js and 14 library modules)
- Remove client/package.json and client/eslint.config.mjs
- Add Go module (go.mod, go.sum) with cobra and mcp-go dependencies
- Add CLI entry point (main.go) and root Cobra command (cmd/root.go)
- Add CLI helpers (cmd/helpers.go) and integration test command (cmd/integration_tests.go)
- Add MCP client library (internal/mcp/client.go) with stdio and HTTP transport
- Add integration test runner (internal/testing/runner.go, params.go)
- Add comprehensive unit tests for all packages (16 tests, all passing)
- Update Makefile to remove go.mod guards (Go source now available)
- Update run-integration-tests.sh to build and use Go binary
- Update test-config.json logDir paths for custom_log_directory tests

Agent-Logs-Url: https://github.com/advanced-security/codeql-development-mcp-server/sessions/8c006672-cf7e-4045-9488-f6d97fafe2f2

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>

* Final validation complete

Agent-Logs-Url: https://github.com/advanced-security/codeql-development-mcp-server/sessions/8c006672-cf7e-4045-9488-f6d97fafe2f2

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>

* Revert unrelated go.mod changes in server/ql/go/tools/test/

Agent-Logs-Url: https://github.com/advanced-security/codeql-development-mcp-server/sessions/8c006672-cf7e-4045-9488-f6d97fafe2f2

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>

* Replace actual repo reference with placeholder in helpers_test.go

Agent-Logs-Url: https://github.com/advanced-security/codeql-development-mcp-server/sessions/86c1cb26-2977-409f-ace0-aabc4fc9cee7

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
data-douser added a commit that referenced this pull request Apr 7, 2026
…221)

* Initial plan

* Replace JavaScript client with Go binary and integration test runner

- Remove client/src/ directory with all JS files (ql-mcp-client.js and 14 library modules)
- Remove client/package.json and client/eslint.config.mjs
- Add Go module (go.mod, go.sum) with cobra and mcp-go dependencies
- Add CLI entry point (main.go) and root Cobra command (cmd/root.go)
- Add CLI helpers (cmd/helpers.go) and integration test command (cmd/integration_tests.go)
- Add MCP client library (internal/mcp/client.go) with stdio and HTTP transport
- Add integration test runner (internal/testing/runner.go, params.go)
- Add comprehensive unit tests for all packages (16 tests, all passing)
- Update Makefile to remove go.mod guards (Go source now available)
- Update run-integration-tests.sh to build and use Go binary
- Update test-config.json logDir paths for custom_log_directory tests

Agent-Logs-Url: https://github.com/advanced-security/codeql-development-mcp-server/sessions/8c006672-cf7e-4045-9488-f6d97fafe2f2

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>

* Final validation complete

Agent-Logs-Url: https://github.com/advanced-security/codeql-development-mcp-server/sessions/8c006672-cf7e-4045-9488-f6d97fafe2f2

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>

* Revert unrelated go.mod changes in server/ql/go/tools/test/

Agent-Logs-Url: https://github.com/advanced-security/codeql-development-mcp-server/sessions/8c006672-cf7e-4045-9488-f6d97fafe2f2

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>

* Replace actual repo reference with placeholder in helpers_test.go

Agent-Logs-Url: https://github.com/advanced-security/codeql-development-mcp-server/sessions/86c1cb26-2977-409f-ace0-aabc4fc9cee7

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
data-douser added a commit that referenced this pull request Apr 7, 2026
…th Go implementation (#223)

* Initial plan

* Phase 1: Update documentation and GitHub config for Go-based client

Update all .github config files, documentation, workflows, and package.json
to reflect the intended Go-based ql-mcp-client architecture. No Go source
code is introduced — only documentation, config, and a stub Makefile.

- Create .github/instructions/client_go.instructions.md for Go conventions
- Update client_src_js.instructions.md to Go guidance
- Update client_integration_tests.instructions.md for Go test runner
- Update agent files to reference Go client instead of JS client
- Update ql-mcp-server-fix-build-and-test.prompt.md with make commands
- Add actions/setup-go to CI workflows
- Rewrite client/README.md and CLI-USAGE.md for gh-ql-mcp-client
- Update client/integration-tests/README.md for Go test runner
- Update docs/testing.md to describe Go MCP client
- Update package.json to use make -C client targets, remove client workspace
- Add stub client/Makefile with go.mod-conditional targets

Agent-Logs-Url: https://github.com/advanced-security/codeql-development-mcp-server/sessions/f5595942-48af-4847-8d28-fba34a17ba76

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>

* Update client/.gitignore for go ql-mcp-client

* Replace JavaScript client with Go binary and integration test runner (#221)

* Initial plan

* Replace JavaScript client with Go binary and integration test runner

- Remove client/src/ directory with all JS files (ql-mcp-client.js and 14 library modules)
- Remove client/package.json and client/eslint.config.mjs
- Add Go module (go.mod, go.sum) with cobra and mcp-go dependencies
- Add CLI entry point (main.go) and root Cobra command (cmd/root.go)
- Add CLI helpers (cmd/helpers.go) and integration test command (cmd/integration_tests.go)
- Add MCP client library (internal/mcp/client.go) with stdio and HTTP transport
- Add integration test runner (internal/testing/runner.go, params.go)
- Add comprehensive unit tests for all packages (16 tests, all passing)
- Update Makefile to remove go.mod guards (Go source now available)
- Update run-integration-tests.sh to build and use Go binary
- Update test-config.json logDir paths for custom_log_directory tests

Agent-Logs-Url: https://github.com/advanced-security/codeql-development-mcp-server/sessions/8c006672-cf7e-4045-9488-f6d97fafe2f2

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>

* Final validation complete

Agent-Logs-Url: https://github.com/advanced-security/codeql-development-mcp-server/sessions/8c006672-cf7e-4045-9488-f6d97fafe2f2

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>

* Revert unrelated go.mod changes in server/ql/go/tools/test/

Agent-Logs-Url: https://github.com/advanced-security/codeql-development-mcp-server/sessions/8c006672-cf7e-4045-9488-f6d97fafe2f2

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>

* Replace actual repo reference with placeholder in helpers_test.go

Agent-Logs-Url: https://github.com/advanced-security/codeql-development-mcp-server/sessions/86c1cb26-2977-409f-ace0-aabc4fc9cee7

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Nathan Randall <70299490+data-douser@users.noreply.github.com>

* Remove client_src_js.instructions.md

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Nathan Randall <70299490+data-douser@users.noreply.github.com>

* Address PR review feedback

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Nathan Randall <70299490+data-douser@users.noreply.github.com>

* Improve ql-mcp-client from PR feedback

* Fix "--no-install-packs" in client integration test runner

* Fixes for failing client integration tests

* Fix extension mcp-tool-e2e.integration.test.ts

* More fixes for PR review feedback

* Fix client-integration-tests.yml Windows job

* More fixes for PR review feedback

* Updates for compatibility with main-merged PRs

* Fix client-integration-tests for extract dbs

* Address PR review feedback: fix Close() kill, remove enableAnnotationTools setting, record skipped results, fix schema default, clarify changelog

Agent-Logs-Url: https://github.com/advanced-security/codeql-development-mcp-server/sessions/a90a53a5-b2ad-4775-8b61-f11d16b33749

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>

* Revert test-integration to skip pack install by default, add test-integration-with-packs for explicit pack setup

Agent-Logs-Url: https://github.com/advanced-security/codeql-development-mcp-server/sessions/a90a53a5-b2ad-4775-8b61-f11d16b33749

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>

* Address review thread 4071470992: path check, changelog, timeout help, error message, cross-platform test, output path sandboxing

Agent-Logs-Url: https://github.com/advanced-security/codeql-development-mcp-server/sessions/0666a5dd-b460-4e6e-9749-f729c46f0b62

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>

---------

Signed-off-by: Nathan Randall <70299490+data-douser@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
data-douser added a commit that referenced this pull request Apr 16, 2026
…th Go implementation (#223)

* Initial plan

* Phase 1: Update documentation and GitHub config for Go-based client

Update all .github config files, documentation, workflows, and package.json
to reflect the intended Go-based ql-mcp-client architecture. No Go source
code is introduced — only documentation, config, and a stub Makefile.

- Create .github/instructions/client_go.instructions.md for Go conventions
- Update client_src_js.instructions.md to Go guidance
- Update client_integration_tests.instructions.md for Go test runner
- Update agent files to reference Go client instead of JS client
- Update ql-mcp-server-fix-build-and-test.prompt.md with make commands
- Add actions/setup-go to CI workflows
- Rewrite client/README.md and CLI-USAGE.md for gh-ql-mcp-client
- Update client/integration-tests/README.md for Go test runner
- Update docs/testing.md to describe Go MCP client
- Update package.json to use make -C client targets, remove client workspace
- Add stub client/Makefile with go.mod-conditional targets

Agent-Logs-Url: https://github.com/advanced-security/codeql-development-mcp-server/sessions/f5595942-48af-4847-8d28-fba34a17ba76

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>

* Update client/.gitignore for go ql-mcp-client

* Replace JavaScript client with Go binary and integration test runner (#221)

* Initial plan

* Replace JavaScript client with Go binary and integration test runner

- Remove client/src/ directory with all JS files (ql-mcp-client.js and 14 library modules)
- Remove client/package.json and client/eslint.config.mjs
- Add Go module (go.mod, go.sum) with cobra and mcp-go dependencies
- Add CLI entry point (main.go) and root Cobra command (cmd/root.go)
- Add CLI helpers (cmd/helpers.go) and integration test command (cmd/integration_tests.go)
- Add MCP client library (internal/mcp/client.go) with stdio and HTTP transport
- Add integration test runner (internal/testing/runner.go, params.go)
- Add comprehensive unit tests for all packages (16 tests, all passing)
- Update Makefile to remove go.mod guards (Go source now available)
- Update run-integration-tests.sh to build and use Go binary
- Update test-config.json logDir paths for custom_log_directory tests

Agent-Logs-Url: https://github.com/advanced-security/codeql-development-mcp-server/sessions/8c006672-cf7e-4045-9488-f6d97fafe2f2

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>

* Final validation complete

Agent-Logs-Url: https://github.com/advanced-security/codeql-development-mcp-server/sessions/8c006672-cf7e-4045-9488-f6d97fafe2f2

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>

* Revert unrelated go.mod changes in server/ql/go/tools/test/

Agent-Logs-Url: https://github.com/advanced-security/codeql-development-mcp-server/sessions/8c006672-cf7e-4045-9488-f6d97fafe2f2

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>

* Replace actual repo reference with placeholder in helpers_test.go

Agent-Logs-Url: https://github.com/advanced-security/codeql-development-mcp-server/sessions/86c1cb26-2977-409f-ace0-aabc4fc9cee7

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Nathan Randall <70299490+data-douser@users.noreply.github.com>

* Remove client_src_js.instructions.md

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Nathan Randall <70299490+data-douser@users.noreply.github.com>

* Address PR review feedback

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Nathan Randall <70299490+data-douser@users.noreply.github.com>

* Improve ql-mcp-client from PR feedback

* Fix "--no-install-packs" in client integration test runner

* Fixes for failing client integration tests

* Fix extension mcp-tool-e2e.integration.test.ts

* More fixes for PR review feedback

* Fix client-integration-tests.yml Windows job

* More fixes for PR review feedback

* Updates for compatibility with main-merged PRs

* Fix client-integration-tests for extract dbs

* Address PR review feedback: fix Close() kill, remove enableAnnotationTools setting, record skipped results, fix schema default, clarify changelog

Agent-Logs-Url: https://github.com/advanced-security/codeql-development-mcp-server/sessions/a90a53a5-b2ad-4775-8b61-f11d16b33749

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>

* Revert test-integration to skip pack install by default, add test-integration-with-packs for explicit pack setup

Agent-Logs-Url: https://github.com/advanced-security/codeql-development-mcp-server/sessions/a90a53a5-b2ad-4775-8b61-f11d16b33749

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>

* Address review thread 4071470992: path check, changelog, timeout help, error message, cross-platform test, output path sandboxing

Agent-Logs-Url: https://github.com/advanced-security/codeql-development-mcp-server/sessions/0666a5dd-b460-4e6e-9749-f729c46f0b62

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>

---------

Signed-off-by: Nathan Randall <70299490+data-douser@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ql-mcp-client Phase 2: Replace JavaScript client with Go binary and integration test runner

2 participants