Skip to content

fix(cve-tool-vulnogram): let the first push after allocation replace the placeholder product - #2194

Open
ChickenBenny wants to merge 3 commits into
apache:mainfrom
ChickenBenny:fix/1736-allocation-placeholder
Open

ChickenBenny wants to merge 3 commits into
apache:mainfrom
ChickenBenny:fix/1736-allocation-placeholder

Conversation

@ChickenBenny

@ChickenBenny ChickenBenny commented Nov 18, 2026 •

Copy link
Copy Markdown

Summary

  • Right after vulnogram-api-allocate, the first vulnogram-api-record-update gets refused by the product guard.
    Vulnogram seeds a fresh record with a [VENDOR] [PRODUCT] placeholder in affected[], and the guard reads "placeholder → real product" as a rename.
    The vetted-ops cve-record-update never passes --allow-product-change, so the agent path is just stuck.
  • While the record's CNA_private.state is RESERVED or DRAFT, the product guard leaves the placeholder entry (product [VENDOR] [PRODUCT], no package) out of the current record before comparing.
    A record with only the placeholder then falls into the existing "empty affected[] is first-time population" rule.
  • Real products are still compared as before: renaming one, even next to a placeholder, is still refused, and so is a placeholder that carries a package.
  • Once the record has reached REVIEW, READY or PUBLIC, or has no state, the placeholder is compared like any product and still needs --allow-product-change.
  • vendor isn't read: it isn't part of the product signature.

Type of change

  • Skill change — issue-sync/apply-and-push.md now says the first push after allocation needs no override, so the agent no longer asks for one there
  • Python package (tools/cve-tool-vulnogram/oauth-api)
  • Other: spec — tools/spec-loop/specs/cve-tooling.md records the exemption

Test plan

  • New TestAllocationPlaceholder covers each allowed / refused case above, including the state gate
  • The "allowed" tests fail on the old code and pass with the fix
  • The new refused-state tests (REVIEW / READY / PUBLIC / no state) fail on the previous commit
  • uv run pytest (oauth-api): all passed
  • prek run --all-files --skip workspace-pytest
  • prek run workspace-pytest --from-ref origin/main --to-ref HEAD
  • security-issue-sync/step-5b-judged-gates eval (reads the changed apply-and-push.md section): 2/2 passed with claude -p, journaled in runs.jsonl (re-run after the review changes, 2/2 again)
  • check-family-plugins --fix synced the copies under plugins/magpie-security-tools and plugins/magpie-vetted-ops

RFC-AI-0004 compliance

  • HITL — no new mutation; this only stops a false refusal

Linked issues

Closes #1736

Notes for reviewers

  • The placeholder shape comes from the security-cve-allocate: first push after API allocation is refused (placeholder product) #1736 description; I don't have Vulnogram access to fetch an allocated record.
    Could someone with access paste the redacted affected[] of one? If it differs I'll adjust the match.
  • If a record's product isn't exactly [VENDOR] [PRODUCT], nothing is exempted and the guard refuses as before; it never lets a real rename through.
  • apply_merge_mode_guards is unchanged; the refusal message and --allow-product-change behave exactly as before.
  • Thanks to @choksi2212 for the earlier analysis on the issue, which this builds on.

Generative AI disclosure: written with help from Claude Code (Opus 5.5).
I've read every line of the diff, run the tests, and can answer review questions myself.

@Kaap10 Kaap10 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ChickenBenny Thanks for picking this up and welcome!
For the clean tests and spec updates! The approach of treating the allocation placeholder as first-time population is the right direction, but there is one safety gap regarding record state gating and an open question on the live placeholder shape that we should address before merge:

correctness: exemption is state-unconstrained (tools/cve-tool-vulnogram/oauth-api/src/vulnogram_api/merge_mode.py:213)

No source rule covers this. It is a safety-boundary issue in the product-change guard.

In #1736 the issue explicitly proposed:

"detect the untouched allocation placeholder (state RESERVED, placeholder product) and allow the product to be set there automatically"

In the current diff, _is_placeholder_entry is evaluated purely inside _diff_affected_products(current, new) without knowledge of the record's state (CNA_private.state or cveMetadata.state). The summary notes "the fix doesn't need the record's state, so it doesn't read it", but merge_mode exists specifically as a safety net against accidental regressions on published or reviewed CVEs.

If a record has already transitioned to REVIEW, READY, or PUBLIC while still carrying an unpopulated placeholder, silently allowing a product overwrite without --allow-product-change bypasses that safety net.

The exemption should be gated so it only applies when the current record is in its initial unsubmitted state (RESERVED / DRAFT):

# Pass current_state into _diff_affected_products, or gate the placeholder exemption:
is_untouched_state = current_state in ("RESERVED", "DRAFT", None)
current_sigs = {
    _product_signature(entry)
    for entry in current
    if isinstance(entry, dict) and not (is_untouched_state and _is_placeholder_entry(entry))
}

correctness: unverified placeholder payload format (tools/cve-tool-vulnogram/oauth-api/src/vulnogram_api/merge_mode.py:183)

No source rule covers this. It is a live-service payload assumption.

You rightly noted in the PR description:

"The placeholder shape comes from the #1736 description; I don't have Vulnogram access to fetch an allocated record. Could someone with access paste the redacted affected[] of one? If it differs I'll adjust the match."

In CVE 5.0 JSON schema, Vulnogram records have both vendor and product keys in containers.cna.affected[]. If Vulnogram initializes a fresh allocation as:

{
  "vendor": "[VENDOR]",
  "product": "[PRODUCT]"
}

or has whitespace variations, then entry.get("product") == "[VENDOR] [PRODUCT]" will evaluate to False and the push will continue to be refused in production.

We should either check with someone holding live Vulnogram credentials (or check if entry.get("product") in ("[VENDOR] [PRODUCT]", "[PRODUCT]")) before locking this in.

Smaller observations

  • general — Eval journal commit SHA: In tools/skill-evals/evals/security-issue-sync/runs.jsonl (line 14) and README.md, the recorded ref is 57480dcf, but the branch commit is 28cddb28. Please update the journal entry so the ref matches the commit on the PR branch.
  • See inline comments on tools/cve-tool-vulnogram/oauth-api/src/vulnogram_api/merge_mode.py:183 and tools/cve-tool-vulnogram/oauth-api/src/vulnogram_api/merge_mode.py:213.

Worth a second look from

Folks with context on Vulnogram allocation payloads and merge mode:

  • @potiuk — author of #1736

This review was drafted by an AI-assisted tool and
confirmed by an Apache Magpie maintainer. The findings
below are observations, not blockers; an Apache Magpie
maintainer — a real person — will take the next look at the
PR. If you think a finding is mis-applied, please reply on
the PR and a maintainer will weigh in.

More on how Apache Magpie handles maintainer review:
Contributing guide.

@KatalKavya96 KatalKavya96 left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ChickenBenny Thanks for the PR. Matching only a placeholder entry with no package keeps the exemption narrow, the tests show the old code refusing and the new code allowing, and the description openly flags what you couldn't check against live Vulnogram, which is exactly what a reviewer needs to see. I ran tests/test_merge_mode.py on 28cddb28 and it passes.

Kaap10's review covers the main points; a few notes building on it:

  • Placeholder shape: agreed this is the open question. #1736 (filed by @potiuk) quotes the live product as [VENDOR] [PRODUCT], so I'd keep the exact match rather than widen it to [PRODUCT] on a guess, since widening loosens a safety guard. @potiuk, could you confirm the affected[] of a fresh allocation?
  • State gating: #1736 does mention RESERVED. As I read it, the guard protects products someone has written, and the placeholder hasn't been written by anyone in any state, so I don't see a rename it would let through. Gating on RESERVED/DRAFT would be extra caution; happy to go with whatever @potiuk prefers.
  • Eval journal ref: the journal is written by the runner and a commit can't contain its own SHA, so 57480dcf (the commit the eval ran on) looks expected to me; I'd leave the record as it is rather than edit it by hand.
  • Small suggestion: _allocated() in the tests defaults to DRAFT, while #1736 says a fresh allocation is RESERVED. Defaulting to RESERVED would mirror the real record.

This review was drafted by an AI-assisted tool and
confirmed by an Apache Magpie maintainer. The findings
below are observations, not blockers; an Apache Magpie
maintainer — a real person — will take the next look at the
PR. If you think a finding is mis-applied, please reply on
the PR and a maintainer will weigh in.

More on how Apache Magpie handles maintainer review:
Contributing guide.

@github-actions github-actions Bot added family:security security-* skills family:tools tools/* family:docs Docs, MISSION.md, READMEs capability:intake Import external signal + sync tracker state contract:cve-authority Tool capability: CVE allocation / record management / publication family:dev Repo dev: .asf.yaml, spec-loop specs, Node manifests, .claude/settings.json labels Nov 18, 2026
@KatalKavya96

Copy link
Copy Markdown
Member

@ChickenBenny, a gentle check-in on this one. The main open point is the state gating Kaap10 raised on merge_mode.py:213, plus the placeholder-shape question. If anything in the reviews is unclear, just ask here and we can help.

Only skip the placeholder in the product guard while the record is RESERVED or DRAFT.
Re-run step-5b-judged-gates after the review changes: 2/2 passed.
@ChickenBenny

Copy link
Copy Markdown
Author

Thanks @Kaap10 and @KatalKavya96 for the reviews. I've pushed two commits:

  • State gate (93050ec): the placeholder is exempt only while CNA_private.state is RESERVED or DRAFT. Any other state, or a missing one, gets the same product guard as before.
  • Placeholder shape: the exact [VENDOR] [PRODUCT] match stays as it is until @potiuk confirms the live payload.
  • Tests: _allocated() now defaults to RESERVED; added DRAFT (allowed) and REVIEW / READY / PUBLIC / no state (refused). The new refused-state tests fail on the previous commit.
  • Docs: the spec, the record_update docstring and apply-and-push.md now name the two states.
  • Eval: step-5b-judged-gates re-run on 93050ec, 2/2 passed, journaled in eaa3ea4.

uv run pytest (oauth-api) passes, as do prek run --all-files --skip workspace-pytest and the workspace pytest for the changed members.

@KatalKavya96 KatalKavya96 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ChickenBenny, thanks for the PR.
The state gate in 93050ec is exactly the narrow version: the placeholder is skipped only while CNA_private.state is RESERVED or DRAFT, and leaving a missing state out of PLACEHOLDER_EXEMPT_STATES keeps the guard on whenever the record can't be shown to be untouched.
The parametrised test_placeholder_past_draft_still_refuses covers REVIEW, READY, PUBLIC and a missing state, and you checked those cases fail on the previous commit, which is what makes the test worth having.
Defaulting _allocated() to RESERVED, re-running the eval on 93050ec and naming the two states in the spec and in apply-and-push.md cover the rest of the earlier comments.

The one open item is the placeholder shape on Kaap10's thread, which needs someone with Vulnogram access (@potiuk has been asked) to confirm the affected[] of a fresh allocation. Keeping the exact [VENDOR] [PRODUCT] match until then is the safe choice: if the live shape differs, the guard simply keeps refusing, as it does today. Nothing else from my side.

Thanks again for working through both reviews.


This review was drafted by an AI-assisted tool and
confirmed by an Apache Magpie maintainer. The findings
below are observations, not blockers; an Apache Magpie
maintainer — a real person — will take the next look at the
PR. If you think a finding is mis-applied, please reply on
the PR and a maintainer will weigh in.

More on how Apache Magpie handles maintainer review:
Contributing guide.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

capability:intake Import external signal + sync tracker state contract:cve-authority Tool capability: CVE allocation / record management / publication family:dev Repo dev: .asf.yaml, spec-loop specs, Node manifests, .claude/settings.json family:docs Docs, MISSION.md, READMEs family:security security-* skills family:tools tools/*

Projects

None yet

Development

Successfully merging this pull request may close these issues.

security-cve-allocate: first push after API allocation is refused (placeholder product)

3 participants