Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 13 additions & 3 deletions plugins/magpie-release-management/skills/verify-rc/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ argument-hint: "<version>-rcN [--post-to <planning-issue-url>] [--skip-repro] [-
capability: capability:triage
surface_hash: sha256:594e276d2b0a7906
license: Apache-2.0
measured_tokens: 6951
measured_tokens: 7210
---

<!-- SPDX-License-Identifier: Apache-2.0
Expand Down Expand Up @@ -268,6 +268,13 @@ uv run --project <framework>/tools/release-management release-management verify-
`commands` holds one resolved [`release-verify`](../../../../tools/release-verify/README.md) command per step (`all` runs Steps 1–3 and 5–8 at once),
built from `§ Expected artefact list` (`--expect` / `--expect-optional` by its `required` / `optional` marking, in order),
`§ Digest set`, `§ Binary-exclude list`, `§ Source-tree validators` and `version_manifest_files`.
`args` adds `--podling` by itself when `project_stage` in `release-management-config.md` is `incubating`:
read the stage from that key or from `args`' own `project_stage` (never from a disclaimer file —
the missing file is what the podling check must detect), and carry its `podling` value into the report.
When the key is unset for an ASF project, `args` warns and no disclaimer check runs:
ask the RM once whether the project is incubating (as `committer-onboarding` asks for organization facts),
record `project_stage: incubating` or `top-level` in `<project-config>/release-management-config.md`,
and re-run `release-config load` — the stage is a config fact, not a per-RC decision.
Run them as printed; surface `warnings`.

`<verify-rc>` below is `uv run --project <framework>/tools/release-management release-management verify-rc <command> --load <scratch>/verify-rc-load.json`:
Expand Down Expand Up @@ -355,6 +362,9 @@ The tool's `status` is `FAIL` when either file is absent from the root of the cu
That is a defect of this RC, whatever any previous release contains:
the diff counts are `null` because there is nothing to diff, not because a previous release is missing.
It is `PASS` when both are present and there is no previous release or no change.
When `args` added `--podling` (a recorded `project_stage: incubating`), `disclaimer_present` reports whether `DISCLAIMER` or `DISCLAIMER-WIP` is a file at the RC root.
If neither exists, the tool returns `FAIL`; a file in a previous release does not satisfy this check.
Presence alone does not validate the disclaimer wording or approve a release.
`REVIEW` means a diff exists and the call is yours:
read `notice_diff` / `license_diff`, surface them to the RM, and decide.

Expand All @@ -365,9 +375,9 @@ read `notice_diff` / `license_diff`, surface them to the RM, and decide.

Return ONLY valid JSON: the tool's `step`, `status` (with `REVIEW` resolved to `PASS` or `WARN`),
`notice_present`, `license_present`, `notice_diff_lines`, `license_diff_lines`,
plus `diff_summary` — a one-line description of the changes,
plus `disclaimer_present` when the tool returned it, and `diff_summary` — a one-line description of the changes,
or `"no diff — no previous release found"`, or `"no changes"`;
on `FAIL`, which file the RC artefact lacks and that there is nothing to diff (e.g. `"NOTICE absent from the RC artefact root; nothing to diff"`).
on `FAIL`, name the missing file; only say there is nothing to diff for a missing `NOTICE` or `LICENSE`, not for a missing disclaimer.

---

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,8 @@ a placeholder companion is the Maven-Central-sanctioned pattern, never a defect;

It prints `SKIP` with `skip_reason` when the gate fails;
otherwise `paste_recipe`, the `maven-artifact-verify` run with `--digests` from `jvm_digest_set` (else the § Digest set, default `sha512`)
and `--podling` **only** when the unpacked source artefact ships a `DISCLAIMER` or `DISCLAIMER-WIP` at its root.
and `--podling` when the config's `project_stage` is `incubating` — the same signal Step 5 uses, so the two steps never disagree.
Only when `project_stage` is unset does it fall back to a `DISCLAIMER` / `DISCLAIMER-WIP` at the unpacked source root.
Run it (offline, read-only: any voter may), save the JSON report, and re-run the command with `--report <report.json>`:
the output is this step's JSON, the recipe now carrying the companion `gpg --verify` lines.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,9 @@ No command here invokes `git`, `gpg`, `svn`, `gh`, `aws` or `atr`, reads a key,
A render that would emit something a skill must refuse (a `dist/release/` staging target, an MD5 or SHA-1 digest, an unconfigured backend)
exits `2` with `{"ok": false, "refused": "<why>"}` instead of printing a block.

For an ASF project that records `project_stage: incubating` in `release-management-config.md`, `verify-rc args` adds the explicit source-disclaimer check (`--podling`) to the generated `notice-license` and `all` commands, and `verify-rc jvm` uses the same stage; other commands and the default behavior are unchanged.
`--podling` remains an explicit override for a run whose config has not recorded the stage yet.

## Prerequisites

- **Runtime:** Python 3.11+, stdlib only; run with `uv run --project <framework>/tools/release-management`.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ def invocations(
keys_file: str = "<keys-file>",
unpacked_dir: str | None = None,
previous_dir: str | None = None,
podling: bool | None = None,
) -> dict[str, Any]:
"""Every `release-verify` command of Steps 1-3 and 5-8, plus `all`, resolved from the config."""
tree = unpacked_dir or v.unpacked_dir
Expand All @@ -82,12 +83,18 @@ def invocations(
validators = [f"--validator {dq(c)}" for c in v.meta.get("source_tree_validators") or []]
manifests = [f"--manifest {q(str(m))}" for m in v.meta.get("version_manifest_files") or []]
previous = [f"--previous {q(previous_dir)}"] if previous_dir else []
# The stage is a config fact (`project_stage`), never an inference from a staged file:
# `--podling` stays an explicit override for a run whose config has not recorded it yet.
is_podling = v.is_podling if podling is None else podling
disclaimer = ["--podling"] if is_podling else []
staged, unpacked = q(staging_copy), q(tree)
commands = {
"inventory": _join(f"{RELEASE_VERIFY} inventory", [f"--dir {staged}", *expect, *digest]),
"signatures": _join(f"{RELEASE_VERIFY} signatures", [f"--dir {staged}", *expect, *keys]),
"checksums": _join(f"{RELEASE_VERIFY} checksums", [f"--dir {staged}", *expect, *digest]),
"notice-license": _join(f"{RELEASE_VERIFY} notice-license", [f"--tree {unpacked}", *previous]),
"notice-license": _join(
f"{RELEASE_VERIFY} notice-license", [f"--tree {unpacked}", *previous, *disclaimer]
),
"binaries": _join(f"{RELEASE_VERIFY} binaries", [f"--tree {unpacked}", *prohibit, *accept]),
"symlinks": _join(f"{RELEASE_VERIFY} symlinks", [f"--tree {unpacked}", *validators]),
"version": _join(
Expand All @@ -102,6 +109,7 @@ def invocations(
*digest,
*keys,
*previous,
*disclaimer,
*prohibit,
*accept,
*validators,
Expand All @@ -111,6 +119,11 @@ def invocations(
),
}
warnings = []
if v.is_asf and v.project_stage is None:
warnings.append(
"project_stage is unset: record `incubating` or `top-level` in release-management-config.md "
"so the podling checks are decided by the config; no disclaimer check is emitted"
)
if not v.artefacts:
warnings.append("release-build.md § Expected artefact list names no artefact: fill it before Step 1")
if not manifests:
Expand All @@ -123,6 +136,8 @@ def invocations(
return {
"rc_tag": v.rc_tag,
"unpacked_dir": tree,
"project_stage": v.project_stage,
"podling": is_podling,
"required": [e["name"] for e in v.artefacts if e.get("required", True)],
"optional": [e["name"] for e in v.artefacts if not e.get("required", True)],
"digests": checked,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,12 @@ def with_load(name: str, help_text: str) -> argparse.ArgumentParser:
"--unpacked-dir", help="unpacked source artefact (default: its filename without extension)"
)
p.add_argument("--previous-dir", help="directory holding the previous release's NOTICE and LICENSE")
p.add_argument(
"--podling",
action="store_true",
default=None,
help="force the ASF podling disclaimer check (default: the config's project_stage decides)",
)

p = with_load("rat", "Step 4: RAT exit code and report → PASS / FAIL; no RAT config → SKIP + WARN")
p.add_argument("--exit-code", type=int, help="RAT's exit code")
Expand Down Expand Up @@ -104,7 +110,8 @@ def with_load(name: str, help_text: str) -> argparse.ArgumentParser:
p.add_argument("--staged-dir", help="local staging copy (listed for .jar / .pom)")
if name == "jvm":
p.add_argument(
"--tree", help="unpacked source artefact (DISCLAIMER / DISCLAIMER-WIP → --podling)"
"--tree",
help="unpacked source artefact (fallback podling signal when project_stage is unset)",
)
p.add_argument("--report", help="file holding maven-artifact-verify's JSON report")
else:
Expand Down Expand Up @@ -166,6 +173,7 @@ def _run(args: argparse.Namespace) -> dict[str, Any]:
keys_file=args.keys_file,
unpacked_dir=args.unpacked_dir,
previous_dir=args.previous_dir,
podling=args.podling,
)
if args.command == "rat":
return checks.rat(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@
)

ARCHIVE_SUFFIXES = (".tar.gz", ".tgz", ".tar.bz2", ".tar.xz", ".tar", ".zip")
#: The lifecycle stages `release-management-config.md` may record in `project_stage`.
PROJECT_STAGES = ("incubating", "top-level")
_SAFE = re.compile(r"[\w@%+=:,./-]+")


Expand Down Expand Up @@ -80,6 +82,30 @@ def rc_tag(self) -> str:
def is_asf(self) -> bool:
return self.organization.upper() == "ASF"

@property
def project_stage(self) -> str | None:
"""`project_stage` from the config: the lifecycle stage this project records for itself.

The organization manifest only publishes the vocabulary (`project_stage_vocab`);
which of its values applies to *this* project is an adopter fact, so it has to be
declared here rather than guessed from an artefact. `None` when unset.
"""
raw = self.config.get("project_stage")
stage = str(raw if raw is not None else "").strip().lower()
if stage in ("", "<project-stage>"):
return None
if stage not in PROJECT_STAGES:
raise VerifyError(
f"project_stage is {raw!r}: release-management-config.md takes "
f"{' or '.join(PROJECT_STAGES)} (the ASF organization's project_stage_vocab)"
)
return stage

@property
def is_podling(self) -> bool:
"""A recorded `project_stage: incubating`; an unset stage is never a podling."""
return self.project_stage == "incubating"

@property
def ci_automated(self) -> bool:
"""`automated_release_signing: enabled` under an organization that offers it."""
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -222,13 +222,19 @@ def jvm(
raise VerifyError(
"--report is not maven-artifact-verify's JSON report (no PASS / WARN / FAIL / SKIP status)"
)
# The podling signal is read from the unpacked tree; without one, the report's own flag stands.
is_podling = podling(tree) if tree else bool(report and report.get("podling"))
# One signal per run: a recorded `project_stage` decides; the source-root file is only the
# fallback for a config that has not recorded one, so Step 5 and Step 6b cannot disagree here.
if v.project_stage is not None:
is_podling = v.is_podling
else:
is_podling = podling(tree) if tree else bool(report and report.get("podling"))
if report is None or report_text is None:
pending = ["run the paste_recipe and pass --report with maven-artifact-verify's JSON"]
if not tree:
if v.project_stage is None and not tree:
pending.insert(
0, "pass --tree <unpacked-dir> so --podling is decided from DISCLAIMER / DISCLAIMER-WIP"
0,
"record project_stage in release-management-config.md (or pass --tree <unpacked-dir>) "
"so --podling is decided from the configured stage, else DISCLAIMER / DISCLAIMER-WIP",
)
return base | {
"paste_recipe": recipe(v, staged, is_podling=is_podling, companions=[]),
Expand Down
10 changes: 9 additions & 1 deletion plugins/magpie-security-tools/tools/release-verify/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,14 @@ Judgement stays with the reader: whether a NOTICE / LICENSE diff is material, an

Guarantees: nothing is written outside a throwaway `GNUPGHOME`; the user's keyring is never read or changed; nothing is signed; a `KEYS` file containing private-key material is refused; nothing touches the network — the caller downloads the staged files and `KEYS` first.

For an ASF incubating project, add `--podling` to `notice-license` or `all` to require
a `DISCLAIMER` or `DISCLAIMER-WIP` file alongside `NOTICE` and `LICENSE`.
The JSON includes `disclaimer_present`; a missing file makes this check `FAIL`,
even when a previous release contains one.
Without the flag the existing output and checks are unchanged.
This checks file presence only, not the disclaimer wording or approval of a release.
The [Incubator release guide](https://incubator.apache.org/guides/releasemanagement.html#choice-of-disclaimers) describes the two disclaimer choices.

## Prerequisites

- **Runtime:** Python 3.11+ (stdlib only); run via `uv run --project tools/release-verify` or as a plain `python3 <framework>/tools/release-verify/src/release_verify/__init__.py`.
Expand All @@ -46,7 +54,7 @@ RV="uv run --project <framework>/tools/release-verify release-verify"
$RV inventory --dir <staging> --expect '<pattern>'... [--expect-optional '<pattern>']... --digest sha512 [--digest sha256]
$RV signatures --dir <staging> --expect '<pattern>'... [--expect-optional '<pattern>']... --keys KEYS --keys-url <keys-url> [--extra-key <signer.pub>]
$RV checksums --dir <staging> --expect '<pattern>'... [--expect-optional '<pattern>']... --digest sha512 [--digest sha256]
$RV notice-license --tree <tree> [--previous <dir with the previous release's NOTICE and LICENSE>]
$RV notice-license --tree <tree> [--podling] [--previous <dir with the previous release's NOTICE and LICENSE>]
$RV binaries --tree <tree> [--prohibit '<glob>']... [--accept '<glob>']...
$RV symlinks --tree <tree> [--validator '<command>']...
$RV version --tree <tree> --rc-tag <version>-rcN --manifest <path>[=<regex>]...
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -566,7 +566,7 @@ def _diff(current: Path, previous: Path, name: str) -> tuple[int, str]:
return changed, "".join(lines)


def notice_license(tree: Path, previous: Path | None) -> dict[str, Any]:
def notice_license(tree: Path, previous: Path | None, *, podling: bool = False) -> dict[str, Any]:
if not tree.is_dir():
raise InputError(f"not a directory: {tree}")
out: dict[str, Any] = {"step": "notice-license", "status": None}
Expand Down Expand Up @@ -595,11 +595,17 @@ def notice_license(tree: Path, previous: Path | None) -> dict[str, Any]:
out["status"] = "PASS"
else:
out["status"] = "REVIEW"
if podling:
out["disclaimer_present"] = any((tree / name).is_file() for name in ("DISCLAIMER", "DISCLAIMER-WIP"))
if not out["disclaimer_present"]:
out["status"] = "FAIL"
detail = "DISCLAIMER or DISCLAIMER-WIP absent from the root of the podling RC artefact."
out["detail"] = f"{out['detail']} {detail}" if "detail" in out else detail
return out


def cmd_notice_license(args: argparse.Namespace) -> dict[str, Any]:
return notice_license(Path(args.tree), Path(args.previous) if args.previous else None)
return notice_license(Path(args.tree), Path(args.previous) if args.previous else None, podling=args.podling)


# ---------------------------------------------------------------------------
Expand Down Expand Up @@ -883,7 +889,7 @@ def cmd_all(args: argparse.Namespace) -> dict[str, Any]:
inventory(_dir_names(staging), args.expect, args.digest, str(staging), args.expect_optional or []),
signatures(staging, _patterns(args), args.keys, args.keys_url, args.extra_key or []),
checksums(staging, _patterns(args), args.digest),
notice_license(tree, Path(args.previous) if args.previous else None),
notice_license(tree, Path(args.previous) if args.previous else None, podling=args.podling),
binaries(tree, args.prohibit or [], args.accept or [], args.recipe_dir),
symlinks(tree, args.validator or [], args.recipe_dir),
version_consistency(tree, args.expected or expected_version(args.rc_tag), args.manifest),
Expand Down Expand Up @@ -964,6 +970,7 @@ def build_parser() -> argparse.ArgumentParser:
p = sub.add_parser("notice-license", help="Step 5: NOTICE / LICENSE presence and diff")
p.add_argument("--tree", required=True, help="unpacked source artefact")
p.add_argument("--previous", help="directory holding the previous release's NOTICE and LICENSE")
p.add_argument("--podling", action="store_true", help="require DISCLAIMER or DISCLAIMER-WIP in the source artefact root")
p.set_defaults(func=cmd_notice_license)

p = sub.add_parser("binaries", help="Step 6: prohibited binaries in the unpacked source")
Expand Down Expand Up @@ -996,6 +1003,7 @@ def build_parser() -> argparse.ArgumentParser:
_add_digest(p)
_add_keys(p)
p.add_argument("--previous", help="directory holding the previous release's NOTICE and LICENSE")
p.add_argument("--podling", action="store_true", help="require DISCLAIMER or DISCLAIMER-WIP in the source artefact root")
p.add_argument("--prohibit", action="append", metavar="GLOB")
p.add_argument("--accept", action="append", metavar="GLOB")
p.add_argument("--validator", action="append", metavar="CMD")
Expand Down
14 changes: 14 additions & 0 deletions plugins/magpie-setup/templates/release-management-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ ship in the same adopter directory and are referenced from here:
| `release_planning_issue_template` | `<project-config>/release-planning-issue.md` |
| `release_branch_base` | `main` |
| `version_manifest_files` | `setup.cfg`, `foo/__init__.py` |
| `project_stage` | *(optional, ASF only)* `incubating` or `top-level` |
| `changelog_label_categories` | *(optional)* e.g. `kind/feature=Features`, `kind/bug-fix=Bug fixes`, `area/docs=Documentation` |

> `product_name` (optional) is the human-readable name a release is
Expand All @@ -73,6 +74,19 @@ ship in the same adopter directory and are referenced from here:
> It is display text only; artefact names and dist paths use
> `project_dist_name`.

> `project_stage` (optional, ASF only) records which lifecycle stage
> *this project* is in — `incubating` for a podling, `top-level` for a
> TLP. The organization manifest publishes only the vocabulary
> (`governance_vocabulary.project_stage_vocab`); nothing there says
> which value is yours, and a disclaimer file in the release is what
> the podling check exists to detect, so the key is the only place the
> stage can be read from. `verify-rc args` adds `--podling` for
> `incubating` on its own: do not pass that flag by hand, and do not
> edit this key per candidate. Unset, the podling checks do not run
> (`verify-rc args` warns) and the JVM recipe falls back to the
> `DISCLAIMER` / `DISCLAIMER-WIP` file at the source root. Non-ASF
> adopters leave it unset.

> `changelog_label_categories` (optional) maps PR labels to the
> `release-prepare prep` changelog sections (*Features*, *Bug fixes*,
> *Documentation*, *Other changes*), as `LABEL=SECTION` pairs. Unset, a
Expand Down
1 change: 1 addition & 0 deletions projects/magpie/release-management-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ mandatory ASF approval + announce mechanisms (`dev-list-vote`,
|---|---|
| `project_dist_name` | `magpie` |
| `product_name` | `Apache Magpie` |
| `project_stage` | `top-level` |
| `upstream` | `apache/magpie` |
| `git_upstream_remote` | `upstream` |
| `release_planning_issue_template` | *(none — uses the `release-prepare` default template)* |
Expand Down
Loading