Skip to content

test: add NotApplicable aggregation regression tests - #819

Merged
yvonnedevlinrh merged 1 commit into
complytime:mainfrom
yvonnedevlinrh:chore/go-gemara-aggregation-tests
Aug 24, 2026
Merged

yvonnedevlinrh merged 1 commit into
complytime:mainfrom
yvonnedevlinrh:chore/go-gemara-aggregation-tests

Conversation

@yvonnedevlinrh

Copy link
Copy Markdown
Contributor

Summary

Adds regression test coverage and documentation for theUpdateAggregateResult behavioral change introduced in
go-gemara v0.9.0 (merged via #813, originally proposed in #801).

Changes

  1. New tests in evaluator_test.go:
    • All-skipped steps → NotApplicable (was Passed pre-v0.9.0)
    • Mixed passed + skipped → Passed (dominance preserved)
    • Multi-control all-skipped → overall NotApplicable
  2. Fixed stale comment in scan_summary_test.go:300-302
  3. CHANGELOG entries for behavioral change and CVE-2026-50163

Context

The go-gemara v0.9.0+ UpdateAggregateResult fix correctly returns NotApplicable instead of Passed when all inputs areNotApplicable. This affects evaluator.go:91,106 and sarif.go:61. These tests lock down the new semantics to
prevent silent regressions if upstream behavior changes again.

@yvonnedevlinrh yvonnedevlinrh self-assigned this Aug 18, 2026
@yvonnedevlinrh
yvonnedevlinrh requested a review from a team as a code owner August 18, 2026 15:17
@yvonnedevlinrh yvonnedevlinrh added dependencies Pull requests that update a dependency file llm_assisted Filed or drafted with LLM assistance labels Aug 18, 2026
@yvonnedevlinrh
yvonnedevlinrh force-pushed the chore/go-gemara-aggregation-tests branch from ffd667c to f8587ab Compare August 18, 2026 16:34
@yvonnedevlinrh
yvonnedevlinrh requested review from em-redhat and trevor-vaughan and removed request for cdaniels255 and sedonnel August 20, 2026 13:04
Add regression test coverage and documentation for the
UpdateAggregateResult behavioral change introduced in
go-gemara v0.9.0 (merged via complytime#813, originally proposed in complytime#801).

- New tests in evaluator_test.go:
  - All-skipped steps yields NotApplicable (was Passed pre-v0.9.0)
  - Mixed passed and skipped yields Passed (dominance preserved)
  - Multi-control all-skipped yields overall NotApplicable
- Fix stale comment in scan_summary_test.go describing pre-v0.9.0
  aggregation behavior
- CHANGELOG entries for behavioral change and CVE-2026-50163

Assisted-by: OpenCode (claude-opus-4-6)
Signed-off-by: Yvonne Devlin <ydevlin@redhat.com>
@yvonnedevlinrh
yvonnedevlinrh force-pushed the chore/go-gemara-aggregation-tests branch from f8587ab to ef8d084 Compare August 20, 2026 13:44

@em-redhat em-redhat left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR #819 Review: test: add NotApplicable aggregation regression tests

Verdict: APPROVE

Summary

This PR adds regression test coverage for the UpdateAggregateResult behavioral change introduced in go-gemara v0.9.0 (pulled in via #813). The tests lock down the corrected NotApplicable semantics and fix a stale comment. CHANGELOG entries document the behavioral change and a CVE fix. Clean, focused, well-structured.

What Changed (3 files, +43/-3)

File Change
CHANGELOG.md Added ### Fixed (NotApplicable aggregation) and ### Security (CVE-2026-50163) under ## Unreleased
internal/output/evaluator_test.go 2 new table-driven cases + 1 standalone multi-control test
internal/output/scan_summary_test.go Fixed stale comment (was "aggregates to Passed", now "preserves NotApplicable")

Convention Compliance

Check Status
TC-001 (success + edge cases) PASS
TC-003 (meaningful assertions) PASS
TC-006 (regression test for bug fix) PASS
TC-007 (descriptive test names) PASS
CR-001 (testify assert/require) PASS
CR-002 (assert for non-fatal) PASS
SPDX headers PASS
Conventional Commits PASS
Signed-off-by trailer PASS
Spec requirement N/A (test-only, exempt)

Source Verification

Verified against vendored UpdateAggregateResult in go-gemara@v0.9.1 (vendor/github.com/gemaraproj/go-gemara/enums.go:994-1026). All three test scenarios match the actual switch logic:

  • NotApplicable + NotApplicable -> NotApplicable (line 1025)
  • Passed + NotApplicable -> Passed (line 1022)
  • NotRun -> returns previous unchanged (line 999)

CI

All 29 checks PASS (or NEUTRAL/SKIPPED). Zero failures.

Issues Found

None. No findings at any severity level.

Notes

  • The llm_assisted label and Assisted-by: OpenCode trailer are appropriately present.
  • No website issue needed (test-only change, exempt per AGENTS.md).
  • CHANGELOG entries are correctly placed and well-formatted.

@yvonnedevlinrh
yvonnedevlinrh merged commit 3164c4f into complytime:main Aug 24, 2026
29 checks passed
@yvonnedevlinrh
yvonnedevlinrh deleted the chore/go-gemara-aggregation-tests branch August 24, 2026 06:25
em-redhat pushed a commit to em-redhat/complyctl that referenced this pull request Sep 1, 2026
Add regression test coverage and documentation for the
UpdateAggregateResult behavioral change introduced in
go-gemara v0.9.0 (merged via complytime#813, originally proposed in complytime#801).

- New tests in evaluator_test.go:
  - All-skipped steps yields NotApplicable (was Passed pre-v0.9.0)
  - Mixed passed and skipped yields Passed (dominance preserved)
  - Multi-control all-skipped yields overall NotApplicable
- Fix stale comment in scan_summary_test.go describing pre-v0.9.0
  aggregation behavior
- CHANGELOG entries for behavioral change and CVE-2026-50163

Assisted-by: OpenCode (claude-opus-4-6)

Signed-off-by: Yvonne Devlin <ydevlin@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file llm_assisted Filed or drafted with LLM assistance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants