CI and validator cover manifests only: skills, hooks, mcp.json, agents and the script test suites are unchecked #531

Summary

The validator and the workflow that runs it leave most of what plugins ship unchecked. Collected while going through the repo last week; filing as one issue so the pieces can be picked off separately.

What validate-plugins.yml covers today

• Trigger paths: is .cursor-plugin/marketplace.json, **/plugin.json, schemas/**. A PR that only touches SKILL.md, hooks.json, mcp.json, agents or scripts runs no CI at all.
• Dependencies are npm install --no-save ajv ajv-formats with no lockfile, so every run resolves whatever Ajv major is current.
• scripts/validate-plugins.mjs checks the two manifests against the schemas and that name matches the marketplace entry. It does not check that skills / agents / rules / hooks / mcpServers / logo paths exist, that SKILL.md or agent frontmatter parses, that ${VAR} placeholders in mcp.json are declared in variables, or that every directory with a plugin.json is listed in the marketplace.

What that has let through

• agent-compatibility/skills/check-agent-compatibility/SKILL.md frontmatter is invalid YAML (unquoted : in description); npx skills skips it with a parse warning (check-agent-compatibility: SKILL.md description has an unquoted colon, YAML parse fails #381, PRs Fix invalid YAML frontmatter in check-agent-compatibility skill #391 and fix(agent-compatibility): quote check-agent-compatibility description #465).
• ralph-loop/hooks/hooks.json addressed its scripts as ./hooks/*.sh, the layout fix(advisor): address hook scripts via CURSOR_PLUGIN_ROOT #315 identified as public-invalid for the registry (fix(ralph-loop): address hook scripts via CURSOR_PLUGIN_ROOT #524).
• cursor-sdk skill description over the 1024-char truncation limit (chore: keep the cursor-sdk skill description under the 1024-char limit; label agent-compatibility changelog 1.0.0 #530).
• orchestrate/skills/orchestrate/scripts (28 *.test.ts, bun test + tsc) and pstack/skills/poteto-mode/scripts (bun test orch watch-pr) have test suites that CI never runs, so regressions in those scripts are only caught by contributors who run them locally.

Suggested shape

1. Broaden paths: to the whole tree (or drop the filter); the job takes seconds.
2. Add a root package.json + lockfile with ajv/ajv-formats pinned and use npm ci.
3. Extend the validator: path existence for component fields, frontmatter parse + required name/description for SKILL.md and agents/*.md, mcp.json placeholders vs variables, unlisted plugin directories. Validate SKILL.md frontmatter in CI #440 already proposes the SKILL.md part with a skill.schema.json; it is conflicting now but the approach fits.
4. A second job running bun test / tsc in the two script packages.