[UPDATE PRIMITIVE] Phase 3(B): Code Scanning lifecycle, SARIF enhancements, gh extension packaging

[Copilot]

📝 Update Information

Primitive Details

• Type: Tool and CLI
• Name: sarif_compare_alerts, sarif_store, sarif_deduplicate_rules, gh-ql-mcp-client
• Update Category: Feature Enhancement

⚠️ CRITICAL: PR SCOPE VALIDATION

• ONLY server implementation files are included
• NO temporary or output files are included
• NO unrelated configuration files are included
• ALL existing tests continue to pass
• NEW functionality is properly tested

---

• Impact Scope: Extensive — new Go client subcommands + server SARIF tool enhancements

Update Metadata

• Breaking Changes: No
• API Compatibility: Enhanced — new tools and overlap modes added, existing signatures preserved
• Performance Impact: Improved — sarif_deduplicate_rules now precomputes per-rule data before pairwise comparison

🎯 Changes Description

Adds Code Scanning alert lifecycle management to gh-ql-mcp-client via GitHub REST API (go-gh), and enhances server-side SARIF tools with fingerprint-based overlap analysis, session cache ingest, and cross-file rule deduplication.

Current Behavior

• gh-ql-mcp-client had no Code Scanning API integration
• sarif_compare_alerts supported 4 overlap modes: sink, source, any-location, full-path
• No way to ingest SARIF into the session cache or deduplicate rules across files
• loadSarif only accepted positional sarifPath/cacheKey args

Updated Behavior

• gh-ql-mcp-client code-scanning list-analyses|list-alerts|download-analysis — full Code Scanning REST API integration via go-gh auth
• sarif_compare_alerts gains fingerprint mode using partialFingerprints with automatic fallback to full-path; response now includes fingerprintMatch and matchedFingerprints fields when present
• sarif_store — ingest SARIF into session cache with accurate cache metadata (databasePath stores the actual SARIF file path or 'inline', queryName uses stable 'sarif_store'), returns deterministic cacheKey
• sarif_deduplicate_rules — pairwise rule comparison across two SARIF files using fingerprint + full-path overlap, with precomputed per-rule extraction for performance
• loadSarif refactored to options object with inlineContent support and source-agnostic error messaging

Motivation

Required by the Code Scanning alert lifecycle workflow: download existing analyses → compare against new codeql_database_analyze results → identify churn/duplication before replacement.

🔄 Before vs. After Comparison

API Changes

// sarif_compare_alerts: new 'fingerprint' overlap mode
overlapMode: z.enum(['sink', 'source', 'any-location', 'full-path', 'fingerprint'])

// New OverlapResult fields for fingerprint mode
interface OverlapResult {
  fingerprintMatch?: boolean;
  matchedFingerprints?: Record<string, string>;
  // ... existing fields
}

// sarif_compare_alerts response now includes fingerprint fields when present:
// { overlaps, overlapMode, sharedLocations, fingerprintMatch?, matchedFingerprints?, pathSimilarity? }

New Tools

// sarif_store — ingest SARIF, get cache key
{ sarifPath?: string, sarifContent?: string, label?: string }
// Returns: { cacheKey, resultCount, ruleCount, toolName, source }
// Cache metadata: databasePath = sarifPath ?? 'inline', queryName = 'sarif_store'

// sarif_deduplicate_rules — cross-file dedup
{ sarifPathA, sarifPathB, overlapThreshold?: number }
// Returns: { duplicateGroups: [...], summary: { ... } }

CLI Additions

gh-ql-mcp-client code-scanning list-analyses --repo owner/repo
gh-ql-mcp-client code-scanning list-alerts --repo owner/repo --state open
gh-ql-mcp-client code-scanning download-analysis --repo owner/repo --analysis-id 201
gh-ql-mcp-client sarif  # parent for future SARIF delegation subcommands

🧪 Testing & Validation

Test Coverage Updates

• Existing Tests: All 1313 server tests pass, all Go tests pass
• New Test Cases: computeFingerprintOverlap (5 cases), fingerprint mode overlap (2 cases), sarif_store (5 cases), sarif_deduplicate_rules handler (3 cases), GitHub API types (JSON round-trip, buildQuery, intToStr)
• Regression Tests: Tool registration count updated (5 → 7)
• Edge Case Tests: Empty fingerprints, absent fingerprints, invalid SARIF content, no-overlap dedup, missing input errors

Test Results

• Unit Tests: All pass (1313 server, 5 Go packages)
• Manual Testing: CLI help verified for all new subcommands
• Note: parallel_validation could not run due to known SIGPIPE with ~10MB dist files in diff

📋 Implementation Details

Files Modified

• Core Implementation: server/src/tools/sarif-tools.ts — 2 new tools, fingerprint mode, loadSarif refactor, fingerprint response fields, dedup precomputation, sarif_store cache metadata fix
• Supporting Libraries: server/src/lib/sarif-utils.ts — computeFingerprintOverlap, OverlapMode/OverlapResult updates
• Tests: server/test/src/lib/sarif-utils.test.ts, server/test/src/tools/sarif-tools.test.ts
• Go Client: client/cmd/code_scanning*.go, client/cmd/sarif.go, client/internal/github/*
• Documentation: CHANGELOG.md
• Build Artifacts: server/dist/*

Code Changes Summary

• Algorithm Improvements: Jaccard-like overlap scoring for sarif_deduplicate_rules with documented formula
• Performance Optimization: sarif_deduplicate_rules precomputes per-rule extracted results into Map<string, RuleData> before the pairwise loop, eliminating redundant extractRuleFromSarif calls in the O(rulesA × rulesB) comparison
• Error Handling: loadSarif error message updated to source-agnostic "No SARIF source provided" (was "Either sarifPath or cacheKey is required") to correctly reflect all three input sources (sarifPath, cacheKey, inlineContent)
• Type Safety: OverlapMode union type extended with 'fingerprint', OverlapResult interface extended
• Input Validation: sarif_store validates SARIF structure before caching
• Cache Metadata: sarif_store now stores sarifPath ?? 'inline' in databasePath and uses stable 'sarif_store' for queryName, instead of overloading both with the user-provided label
• Thread Safety: sarifClient() uses sync.Once to prevent data races from concurrent goroutines
• Response Completeness: sarif_compare_alerts response now includes fingerprintMatch and matchedFingerprints fields when present in the overlap result (fingerprint mode), so callers can see which fingerprints matched

Dependencies

• New Dependencies: github.com/cli/go-gh/v2 v2.13.0 (Go client — GitHub REST API auth). No advisories.

🔗 References

Related Issues/PRs

• Related PRs: Rewrite ql-mcp-client in Go as a gh CLI extension for Code Scanning SARIF management #207 (parent), ql-mcp-client Phase 2: Replace JavaScript client with Go binary and integration test runner #217 (Phase 2), MCP tool usability improvements: validation errors, parameter naming, and SARIF enhancements #208 (SARIF improvements), sarif_list_rules: add per-rule resultCount field #214 (sarif_list_rules resultCount)
• Reference branch: origin/dd/ql-mcp-client/1

🚀 Compatibility & Migration

Backward Compatibility

• Fully Compatible: No breaking changes — new tools and CLI commands only

API Evolution

• Enhanced Parameters: fingerprint overlap mode added to sarif_compare_alerts
• Improved Responses: OverlapResult gains fingerprintMatch/matchedFingerprints; sarif_compare_alerts response now surfaces these fields
• Maintained Contracts: All existing tool signatures preserved

👥 Review Guidelines

For Reviewers

• ⚠️ SCOPE COMPLIANCE: Server implementation + Go client files only
• ⚠️ NO UNRELATED FILES: Clean diff
• ⚠️ BACKWARD COMPATIBILITY: All existing functionality preserved

Testing Instructions

# Server tests
cd server && npm test

# Go client tests
make -C client test-unit

# Build verification
make -C client build
./client/gh-ql-mcp-client code-scanning --help

# Lint
cd server && npx eslint src test --ext .ts
cd client && go vet ./...

📊 Impact Assessment

Server Impact

• Startup Time: Negligible — 2 additional tool registrations
• Runtime Stability: No impact
• Concurrent Usage: Safe — all new tools are stateless per-request; Go client sarifClient() uses sync.Once for thread-safe lazy initialization

Review Feedback Addressed

• sarifClient() data race (comment Extended support for CodeQL Language Server (LSP) features #1): Replaced nil-check with sync.Once to guarantee thread-safe lazy initialization of the SARIF REST client.
• Overlap score formula clarity (comment Extend FindRelatedPrimitives tool queries for all languages #4): Added a 5-line comment block explaining the Jaccard-like scoring strategy: takes the higher of location-based and fingerprint-based matching counts, then applies matchedAlerts / (totalA + totalB - matchedAlerts) with capping to prevent negative unmatched counts.
• sarif_deduplicate_rules handler tests (comment Validate MCP server compatibility across CodeQL CLI versions #5): Added 3 handler-level tests exercising the full deduplication workflow: overlap detection with matching codeFlow locations, empty groups when no rules overlap, and error handling for missing input.
• loadSarif error message (automated review): Updated to source-agnostic "No SARIF source provided" since inlineContent is now also a valid input source.
• Fingerprint response fields (automated review): sarif_compare_alerts response now includes fingerprintMatch and matchedFingerprints when those fields are present in the overlap result.
• sarif_deduplicate_rules performance (automated review): Precomputes per-rule extracted results into Map<string, RuleData> before the pairwise comparison loop, eliminating redundant extractRuleFromSarif calls.
• CHANGELOG corrections (automated review): Fixed overlap mode name from partialFingerprints to fingerprint and added ([#234](...)) PR links to all entries missing them.
• sarif_store cache metadata (automated review): databasePath now stores sarifPath ?? 'inline' and queryName uses stable 'sarif_store', instead of overloading both fields with the user-provided label. Server dist rebuilt.

[data-douser]

Review of Phase 3(B): Code Scanning lifecycle, SARIF enhancements, gh extension packaging

Windows stdio transport hang (blocking)

The client-integration-tests workflow hangs on windows-latest + stdio mode because the Go client never drains the Node.js server's stderr pipe. The mcp-go Stdio transport creates a stderr pipe via cmd.StderrPipe() but never reads from it — it exposes Stderr() for callers to optionally consume. On Windows, pipe buffers are ~4 KB (vs 64 KB on Linux). Once the server's log output exceeds the buffer, process.stderr.write() blocks the entire Node.js event loop, halting all JSON-RPC responses. This causes every subsequent tool call to timeout.

Fix (applied locally): In connectStdio(), after creating the client, drain stderr in a background goroutine:

if stderr, ok := mcpclient.GetStderr(client); ok {
    go func() { _, _ = io.Copy(io.Discard, stderr) }()
}

This is the only change needed to unblock the CI job. The fix is already committed to the branch.

---

Go client issues

1. client/internal/github/client.go:43-50 — sarifClient() has a data race: concurrent goroutines can both pass the nil check and create independent HTTP clients, with the second silently overwriting the first. Guard with sync.Once or sync.Mutex.

2. client/cmd/use_tool.go:71 — outputToolResult() prints tool content to stdout then returns a generic fmt.Errorf("tool returned error"). The content is printed, so it's not lost, but the generic error message propagated to cobra is uninformative. Consider including a summary from result.Content in the error.

3. client/cmd/list.go:90,120 — String truncation desc[:77] operates on byte offsets, which can split multi-byte UTF-8 sequences (emoji, CJK, etc.) producing invalid output. Use []rune conversion.

Server-side notes

4. server/src/tools/sarif-tools.ts ~L508-516 — The overlap score formula in sarif_deduplicate_rules uses Math.max(overlaps.length, matchedAIndices.size) as matchedAlerts, which conflates two different matching metrics. The intent appears to be a Jaccard-like similarity but the numerator/denominator don't consistently represent the same sets. Worth adding a comment or test clarifying the expected behavior for asymmetric result counts.

5. sarif_deduplicate_rules handler — No handler-level tests exist in sarif-tools.test.ts. The computeFingerprintOverlap unit tests cover the utility, but there are no tests exercising the full deduplication workflow (multi-rule SARIF → pairwise comparison → grouped output).

VS Code extension test coverage regression

6. The PR deletes cli-resolver.test.ts and pack-installer.test.ts without replacement tests. CliResolver has complex multi-strategy resolution logic (CODEQL_PATH, PATH lookup, vscode-codeql distribution.json parsing with version folding) and PackInstaller has CLI-version-to-pack-version mapping logic. Both are critical paths that should retain unit test coverage.

[Copilot]

Review of Phase 3(B): Code Scanning lifecycle, SARIF enhancements, gh extension packaging

Windows stdio transport hang (blocking)...

Addressed items 1, 4, and 5 in a9df508:

#1 — sarifClient() data race: Replaced the nil-check with sync.Once to guarantee thread-safe lazy initialization. The sarifInitErr is now stored alongside the client so errors are also returned consistently on subsequent calls.

#4 — Overlap score formula: Added a 5-line comment block explaining the scoring strategy: it takes the higher of location-based and fingerprint-based matching counts, then applies a Jaccard-like formula with capping to prevent negative unmatched counts.

#5 — sarif_deduplicate_rules handler tests: Added 3 handler-level tests: (a) overlap detection with matching codeFlow locations, (b) empty groups when no rules overlap, (c) error when sarifPathA is missing.

Items 2, 3, and 6 are in pre-existing code not modified by this PR (use_tool.go, list.go, and VS Code extension test files are not in my diff).

[Copilot]

Pull request overview

This PR extends the MCP server’s SARIF tooling to support fingerprint-based overlap and session-cache SARIF ingestion, and expands the Go gh-ql-mcp-client with GitHub Code Scanning REST API subcommands for alert lifecycle workflows.

Changes:

• Add fingerprint overlap support (via partialFingerprints) and new SARIF tools: sarif_store and sarif_deduplicate_rules.
• Add Go GitHub API client/types for Code Scanning plus new code-scanning and sarif CLI command groups.
• Update tests, changelog, and generated server dist artifact to reflect new functionality.

Show a summary per file

File	Description
server/src/lib/sarif-utils.ts	Adds fingerprint overlap computation and extends overlap modes/results.
server/src/tools/sarif-tools.ts	Registers new SARIF tools, refactors SARIF loading, and adds new tool handlers.
server/test/src/lib/sarif-utils.test.ts	Adds unit tests for fingerprint overlap and fingerprint mode behavior.
server/test/src/tools/sarif-tools.test.ts	Updates tool-registration count and adds handler-level tests for sarif_store/sarif_deduplicate_rules.
server/dist/codeql-development-mcp-server.js	Updates bundled output to include new SARIF tool functionality.
client/internal/mcp/client.go	Prevents stdio subprocess hangs by draining server stderr in background.
client/internal/github/client.go	Adds typed Code Scanning REST API wrapper using go-gh auth.
client/internal/github/types.go	Introduces Go types for analyses/alerts/instances payloads.
client/internal/github/client_test.go	Adds JSON round-trip and small helper tests for the GitHub client/types.
client/go.mod / client/go.sum	Adds github.com/cli/go-gh/v2 and related indirect deps.
client/cmd/root.go	Updates CLI description to focus on Code Scanning lifecycle management.
client/cmd/code_scanning*.go	Adds code-scanning subcommands: list analyses/alerts and download analysis SARIF.
client/cmd/sarif.go	Adds sarif parent command group for future SARIF delegation workflows.
CHANGELOG.md	Documents new Code Scanning CLI features and SARIF tool enhancements.

Copilot's findings

Comments suppressed due to low confidence (2)

CHANGELOG.md:40

• The new tools row is missing the required PR link (entries in the tools table should link the introducing PR). Also consider describing the enhancement to sarif_compare_alerts as “fingerprint overlap mode using partialFingerprints” to match the actual parameter name.

| `query_results_cache_lookup`, `query_results_cache_retrieve`, `query_results_cache_clear`, `query_results_cache_compare` | Query result cache tools for lookup, subset retrieval, cache clearing, and cross-database comparison. ([#169](https://github.com/advanced-security/codeql-development-mcp-server/pull/169))                                                     |
| `sarif_list_rules`, `sarif_extract_rule`, `sarif_rule_to_markdown`, `sarif_compare_alerts`, `sarif_diff_runs`            | SARIF analysis tools for rule discovery, per-rule extraction, Mermaid dataflow visualization, alert overlap comparison, and cross-run behavioral diffing. ([#204](https://github.com/advanced-security/codeql-development-mcp-server/pull/204)) |
| `sarif_store`, `sarif_deduplicate_rules`                                                                                 | SARIF session cache ingest and cross-file rule deduplication tools. `sarif_compare_alerts` enhanced with `partialFingerprints` overlap mode with automatic fallback to full-path comparison.                                                    |

CHANGELOG.md:64

• These new Infrastructure & CI/CD bullets are missing the required PR links. Please append ([#NNN](...)) for this PR to each new bullet (and avoid leaving changelog entries unlinked).

- Added Rust coverage to CI and release workflows, including query unit tests and VSIX bundling. ([#195](https://github.com/advanced-security/codeql-development-mcp-server/pull/195))
- Added client integration tests for the new Rust queries and for the annotation, audit, and cache tool suites, including an MRVA triage workflow end-to-end test. ([#169](https://github.com/advanced-security/codeql-development-mcp-server/pull/169), [#195](https://github.com/advanced-security/codeql-development-mcp-server/pull/195))
- Added `code-scanning` and `sarif` subcommand groups to `gh-ql-mcp-client` with GitHub REST API client integration via `go-gh` for Code Scanning alert lifecycle management.
- Added `gh` extension packaging support with cross-compilation targets for `darwin/amd64`, `darwin/arm64`, `linux/amd64`, `linux/arm64`, `windows/amd64`.

• Files reviewed: 16/19 changed files
• Comments generated: 4

[data-douser]

@copilot apply changes based on the comments in this thread

[Copilot]

@copilot apply changes based on the comments in this thread

Applied all 4 changes from the review thread in 0157993:

1. loadSarif error message updated to source-agnostic "No SARIF source provided" (was "Either sarifPath or cacheKey is required")
2. sarif_compare_alerts response now includes fingerprintMatch and matchedFingerprints fields when present in fingerprint mode
3. sarif_deduplicate_rules precomputes per-rule extracted results into Map<string, RuleData> before the pairwise loop, eliminating redundant extractRuleFromSarif calls
4. CHANGELOG entries corrected: partialFingerprints → fingerprint overlap mode name, added ([#234](...)) PR links to all 4 missing locations (lines 23, 39, 63, 64)

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• https://api.github.com/graphql
  • Triggering command: /usr/bin/gh gh pr view --json number (http block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

[Copilot]

Pull request overview

Adds Code Scanning lifecycle commands to the Go-based gh-ql-mcp-client and expands the MCP server’s SARIF tooling to support fingerprint-based overlap detection, SARIF ingestion into the session cache, and cross-file rule deduplication.

Changes:

• Added sarif_store and sarif_deduplicate_rules MCP tools, and introduced fingerprint as a new sarif_compare_alerts overlap mode.
• Added code-scanning subcommands (list-analyses, list-alerts, download-analysis) plus a sarif command group to the Go CLI, backed by a typed GitHub Code Scanning REST client (go-gh).
• Updated unit tests and CHANGELOG.md, and regenerated the bundled server dist output.

Show a summary per file

File	Description
server/src/lib/sarif-utils.ts	Adds fingerprint overlap support and extends overlap mode/type definitions.
server/src/tools/sarif-tools.ts	Registers new SARIF tools (sarif_store, sarif_deduplicate_rules), refactors SARIF loading, and surfaces fingerprint fields in responses.
server/test/src/lib/sarif-utils.test.ts	Adds unit tests for fingerprint overlap and fingerprint-mode behavior.
server/test/src/tools/sarif-tools.test.ts	Updates tool registration expectations and adds handler tests for sarif_store / sarif_deduplicate_rules.
server/dist/codeql-development-mcp-server.js	Regenerated build artifact reflecting the server-side changes.
client/internal/mcp/client.go	Prevents stdio server hangs by draining server stderr in the background.
client/internal/github/client.go	Introduces typed GitHub Code Scanning REST API client (analyses, alerts, SARIF download, update).
client/internal/github/types.go	Adds Go structs for Code Scanning API payloads.
client/internal/github/client_test.go	Adds JSON round-trip and helper tests for the GitHub client/types.
client/go.mod	Adds github.com/cli/go-gh/v2 dependency for GitHub API auth/client.
client/go.sum	Updates sums for newly added Go dependencies.
client/cmd/root.go	Updates CLI messaging to focus on Code Scanning lifecycle management.
client/cmd/sarif.go	Adds sarif parent command group for future SARIF workflows.
client/cmd/code_scanning.go	Adds code-scanning parent command group (with cs alias).
client/cmd/code_scanning_list_analyses.go	Implements code-scanning list-analyses.
client/cmd/code_scanning_list_alerts.go	Implements code-scanning list-alerts.
client/cmd/code_scanning_download_analysis.go	Implements code-scanning download-analysis (writes SARIF to disk).
CHANGELOG.md	Documents the new CLI subcommands and SARIF tool enhancements under [Unreleased].

Copilot's findings

• Files reviewed: 16/19 changed files
• Comments generated: 2