Repository navigation
fix(cve-tool-vulnogram): let the first push after allocation replace the placeholder product - #2194
ChickenBenny wants to merge 3 commits into
Conversation
Kaap10
left a comment
There was a problem hiding this comment.
@ChickenBenny Thanks for picking this up and welcome!
For the clean tests and spec updates! The approach of treating the allocation placeholder as first-time population is the right direction, but there is one safety gap regarding record state gating and an open question on the live placeholder shape that we should address before merge:
correctness: exemption is state-unconstrained (tools/cve-tool-vulnogram/oauth-api/src/vulnogram_api/merge_mode.py:213)
No source rule covers this. It is a safety-boundary issue in the product-change guard.
In #1736 the issue explicitly proposed:
"detect the untouched allocation placeholder (state RESERVED, placeholder product) and allow the product to be set there automatically"
In the current diff, _is_placeholder_entry is evaluated purely inside _diff_affected_products(current, new) without knowledge of the record's state (CNA_private.state or cveMetadata.state). The summary notes "the fix doesn't need the record's state, so it doesn't read it", but merge_mode exists specifically as a safety net against accidental regressions on published or reviewed CVEs.
If a record has already transitioned to REVIEW, READY, or PUBLIC while still carrying an unpopulated placeholder, silently allowing a product overwrite without --allow-product-change bypasses that safety net.
The exemption should be gated so it only applies when the current record is in its initial unsubmitted state (RESERVED / DRAFT):
# Pass current_state into _diff_affected_products, or gate the placeholder exemption:
is_untouched_state = current_state in ("RESERVED", "DRAFT", None)
current_sigs = {
_product_signature(entry)
for entry in current
if isinstance(entry, dict) and not (is_untouched_state and _is_placeholder_entry(entry))
}correctness: unverified placeholder payload format (tools/cve-tool-vulnogram/oauth-api/src/vulnogram_api/merge_mode.py:183)
No source rule covers this. It is a live-service payload assumption.
You rightly noted in the PR description:
"The placeholder shape comes from the #1736 description; I don't have Vulnogram access to fetch an allocated record. Could someone with access paste the redacted affected[] of one? If it differs I'll adjust the match."
In CVE 5.0 JSON schema, Vulnogram records have both vendor and product keys in containers.cna.affected[]. If Vulnogram initializes a fresh allocation as:
{
"vendor": "[VENDOR]",
"product": "[PRODUCT]"
}or has whitespace variations, then entry.get("product") == "[VENDOR] [PRODUCT]" will evaluate to False and the push will continue to be refused in production.
We should either check with someone holding live Vulnogram credentials (or check if entry.get("product") in ("[VENDOR] [PRODUCT]", "[PRODUCT]")) before locking this in.
Smaller observations
general— Eval journal commit SHA: Intools/skill-evals/evals/security-issue-sync/runs.jsonl(line 14) andREADME.md, the recorded ref is57480dcf, but the branch commit is28cddb28. Please update the journal entry so the ref matches the commit on the PR branch.- See inline comments on
tools/cve-tool-vulnogram/oauth-api/src/vulnogram_api/merge_mode.py:183andtools/cve-tool-vulnogram/oauth-api/src/vulnogram_api/merge_mode.py:213.
Worth a second look from
Folks with context on Vulnogram allocation payloads and merge mode:
@potiuk— author of #1736
This review was drafted by an AI-assisted tool and
confirmed by an Apache Magpie maintainer. The findings
below are observations, not blockers; an Apache Magpie
maintainer — a real person — will take the next look at the
PR. If you think a finding is mis-applied, please reply on
the PR and a maintainer will weigh in.More on how Apache Magpie handles maintainer review:
Contributing guide.
There was a problem hiding this comment.
@ChickenBenny Thanks for the PR. Matching only a placeholder entry with no package keeps the exemption narrow, the tests show the old code refusing and the new code allowing, and the description openly flags what you couldn't check against live Vulnogram, which is exactly what a reviewer needs to see. I ran tests/test_merge_mode.py on 28cddb28 and it passes.
Kaap10's review covers the main points; a few notes building on it:
- Placeholder shape: agreed this is the open question. #1736 (filed by @potiuk) quotes the live product as
[VENDOR] [PRODUCT], so I'd keep the exact match rather than widen it to[PRODUCT]on a guess, since widening loosens a safety guard. @potiuk, could you confirm theaffected[]of a fresh allocation? - State gating: #1736 does mention RESERVED. As I read it, the guard protects products someone has written, and the placeholder hasn't been written by anyone in any state, so I don't see a rename it would let through. Gating on RESERVED/DRAFT would be extra caution; happy to go with whatever @potiuk prefers.
- Eval journal ref: the journal is written by the runner and a commit can't contain its own SHA, so
57480dcf(the commit the eval ran on) looks expected to me; I'd leave the record as it is rather than edit it by hand. - Small suggestion:
_allocated()in the tests defaults toDRAFT, while #1736 says a fresh allocation isRESERVED. Defaulting toRESERVEDwould mirror the real record.
This review was drafted by an AI-assisted tool and
confirmed by an Apache Magpie maintainer. The findings
below are observations, not blockers; an Apache Magpie
maintainer — a real person — will take the next look at the
PR. If you think a finding is mis-applied, please reply on
the PR and a maintainer will weigh in.More on how Apache Magpie handles maintainer review:
Contributing guide.
|
@ChickenBenny, a gentle check-in on this one. The main open point is the state gating Kaap10 raised on |
Only skip the placeholder in the product guard while the record is RESERVED or DRAFT.
Re-run step-5b-judged-gates after the review changes: 2/2 passed.
|
Thanks @Kaap10 and @KatalKavya96 for the reviews. I've pushed two commits:
|
KatalKavya96
left a comment
There was a problem hiding this comment.
@ChickenBenny, thanks for the PR.
The state gate in 93050ec is exactly the narrow version: the placeholder is skipped only while CNA_private.state is RESERVED or DRAFT, and leaving a missing state out of PLACEHOLDER_EXEMPT_STATES keeps the guard on whenever the record can't be shown to be untouched.
The parametrised test_placeholder_past_draft_still_refuses covers REVIEW, READY, PUBLIC and a missing state, and you checked those cases fail on the previous commit, which is what makes the test worth having.
Defaulting _allocated() to RESERVED, re-running the eval on 93050ec and naming the two states in the spec and in apply-and-push.md cover the rest of the earlier comments.
The one open item is the placeholder shape on Kaap10's thread, which needs someone with Vulnogram access (@potiuk has been asked) to confirm the affected[] of a fresh allocation. Keeping the exact [VENDOR] [PRODUCT] match until then is the safe choice: if the live shape differs, the guard simply keeps refusing, as it does today. Nothing else from my side.
Thanks again for working through both reviews.
This review was drafted by an AI-assisted tool and
confirmed by an Apache Magpie maintainer. The findings
below are observations, not blockers; an Apache Magpie
maintainer — a real person — will take the next look at the
PR. If you think a finding is mis-applied, please reply on
the PR and a maintainer will weigh in.More on how Apache Magpie handles maintainer review:
Contributing guide.
Summary
vulnogram-api-allocate, the firstvulnogram-api-record-updategets refused by the product guard.Vulnogram seeds a fresh record with a
[VENDOR] [PRODUCT]placeholder inaffected[], and the guard reads "placeholder → real product" as a rename.The vetted-ops
cve-record-updatenever passes--allow-product-change, so the agent path is just stuck.CNA_private.stateisRESERVEDorDRAFT, the product guard leaves the placeholder entry (product[VENDOR] [PRODUCT], no package) out of the current record before comparing.A record with only the placeholder then falls into the existing "empty
affected[]is first-time population" rule.REVIEW,READYorPUBLIC, or has no state, the placeholder is compared like any product and still needs--allow-product-change.vendorisn't read: it isn't part of the product signature.Type of change
issue-sync/apply-and-push.mdnow says the first push after allocation needs no override, so the agent no longer asks for one theretools/cve-tool-vulnogram/oauth-api)tools/spec-loop/specs/cve-tooling.mdrecords the exemptionTest plan
TestAllocationPlaceholdercovers each allowed / refused case above, including the state gateREVIEW/READY/PUBLIC/ no state) fail on the previous commituv run pytest(oauth-api): all passedprek run --all-files --skip workspace-pytestprek run workspace-pytest --from-ref origin/main --to-ref HEADsecurity-issue-sync/step-5b-judged-gateseval (reads the changedapply-and-push.mdsection): 2/2 passed withclaude -p, journaled inruns.jsonl(re-run after the review changes, 2/2 again)check-family-plugins --fixsynced the copies underplugins/magpie-security-toolsandplugins/magpie-vetted-opsRFC-AI-0004 compliance
Linked issues
Closes #1736
Notes for reviewers
Could someone with access paste the redacted
affected[]of one? If it differs I'll adjust the match.[VENDOR] [PRODUCT], nothing is exempted and the guard refuses as before; it never lets a real rename through.apply_merge_mode_guardsis unchanged; the refusal message and--allow-product-changebehave exactly as before.Generative AI disclosure: written with help from Claude Code (Opus 5.5).
I've read every line of the diff, run the tests, and can answer review questions myself.