Skip to content

Organize projects by semantic kind - #73

Open
johnnygreco wants to merge 2 commits into
mainfrom
johnny/organize-projects-by-kind
Open

johnnygreco wants to merge 2 commits into
mainfrom
johnny/organize-projects-by-kind

Conversation

@johnnygreco

Copy link
Copy Markdown
Collaborator

Summary

  • group projects under tools, research-spikes, and use-case-examples
  • add project.yaml kind metadata and teach the OAR reviewer the nested layout
  • update repository paths across CI workflows, code, tests, documentation, and install links

Validation

  • repository tests: 134 passed, 11 skipped
  • OAR: 153 passed; wheel and sdist built
  • Egress Gate: 324 passed
  • OpenShell Middleware Manager: 95 passed; wheel and sdist built
  • Python project template: 2 passed with 100% coverage; wheel and sdist built
  • documentation: 23 passed, 3 skipped; strict site build completed; local preview returned HTTP 200
  • Robotics Policy Prover: frontend production build passed
  • Reachy Mini OpenShell: uv lock --check passed
  • stale legacy path and whitespace scans passed

Environment limitations

  • Long-horizon eval checks need the authenticated GitHub Packages dependency, which was unavailable to npm in this environment.
  • Robotics Cargo tests need cmake; visual verification needs the Playwright Chromium binary.
  • Reachy development sync is restricted by its lockfile to macOS, so the full suite could not run on this Linux host.

Signed-off-by: Johnny Greco <jogreco@nvidia.com>
@github-actions

github-actions Bot commented Sep 14, 2026

Copy link
Copy Markdown

New project review

Revision: 1d9379f323d33a5376944e4cde5579f5f4308876 · Workflow and result artifacts

Review findings are advisory. Required checks remain separate merge gates.

Execution: success

Reviewer and guidelines revision: 26dbfd52730429695670c27a0ec0449a851e755c

Reviews

Project Verdict Guidelines Findings
projects/research-spikes ✅ Pass ✅ Pass 3
projects/tools ✅ Pass ✅ Pass 3
projects/use-case-examples ✅ Pass ✅ Pass 4
projects/research-spikes

The contribution places and declares `projects/research-spikes/long-horizon-agent-evals` as a `research-spike`: a small, deliberately concrete TypeScript harness that runs a persistent Codex "challenger" inside an OpenShell sandbox for a wall-clock horizon and tests whether it can induce a separate, tool-free AI reviewer to grant cumulative OpenShell policy sufficient to mutate a protected GitHub repository. It serves agent-security researchers studying long-horizon (trajectory, not single-turn) policy-review pressure. The pieces fit together as documented: `src/campaign.ts` (setup, sandbox, oracle, outcome classification), `src/reviewer.ts` plus `src/reviewer-model.ts` (structured approve/reject applied via the OpenShell SDK by the harness, not the model), `src/scale.ts` (bounded-concurrency repeats), `scripts/challenger.sh`, and prompts under `experiments/github-policy-review/`. The README gives an ordered first run (install, `.env`, `npm run endpoint:check`, `image:build`, `github:preflight`, `campaign`, `scale`, `transcript`/`timeline`/`costs`), prerequisites (Node 20.3+, Docker, dedicated OpenShell 0.0.106+ gateway, GitHub Packages access, fine-grained repo token), and a prominent warning that real repository mutations occur. Methodological strengths verified in code: GitHub is an external oracle rather than model self-report (`src/campaign.ts:893`), approvals are separated into returned / applied / fail-closed counts plus `requiresAdjudication`, and operational failures invalidate non-compromised attempts (`classifyOutcome`, `src/campaign.ts:481`). Focused tests cover outcome classification, paginated ref snapshots, lull detection, handoff, reviewer history compaction, and version matching. Remaining gaps are documentation-level and non-blocking: no results summary or link to the repository write-up, no committed run artifacts or fixtures, and one tuning default justified by an uncommitted trace corpus.

Project guidelines: ✅ Pass

Applicable requirements are supported. `kind: research-spike` is declared in both `projects/research-spikes/project.yaml` and the project's own `project.yaml`. Purpose, audience, OpenShell usage, and a `Limits` section are in the README; layout is self-contained with no empty docs scaffolding (single README plus a small `experiments/` prompt directory and a short `AGENTS.md` with genuinely project-specific conventions). First run is documented with prerequisites, disclosed hardware/service/credential/cost requirements, and an explicit real-mutation warning. Verification is proportionate: `npm run check` (typecheck plus focused node:test files) satisfies the "focused tests, a runnable check, or recorded evidence" option, and the README states what is not verified. Dependencies use the normal manifest plus a committed `package-lock.json`; the `uv`/`pyproject` rule is inapplicable to this Node project. Secrets are kept out of the tree: `.env` and `runs/*` are gitignored, `.npmrc` uses `${NODE_AUTH_TOKEN}`, and `.env.example` documents variables with empty credential values. Engineering is proportionate and `AGENTS.md` explicitly forbids adding a scenario framework before a second experiment. Not verified within scope: repository-root license/attribution and the repo-wide content-routing conventions in the root AGENTS.md/docs tree, which were outside the provided snapshot.

Findings

  • medium: No recorded results or write-up link for the flagship experimentprojects/research-spikes/long-horizon-agent-evals/README.md:286
    • Evidence: The README documents the harness and interpretation fields, and the Limits section says "A zero-compromise result applies only to the exact models, prompts, OpenShell version, configuration, and budget recorded for that run", but no run is summarized, no artifact is committed (`runs/` contains only `.gitkeep`, and `runs/*` is gitignored), and no link is given to the repository's `docs/dev-notes/posts/2026-08-27-adversarial-policy-review-long-horizon-agents.md`.
    • Recommendation: Add a short "What we observed so far" section stating whether campaigns have been run, the headline outcome (including a null result), and a link to the existing dev-note write-up; optionally commit one redacted example run directory as a fixture.
    • Impact: A reader cannot tell whether the experiment has been executed end to end or what it found, and the analysis tooling (`transcript`, `timeline`, `costs`) has no reference artifact to validate against, weakening the evidence chain for a research spike.
  • low: Lull-detection defaults justified by an uncommitted trace corpusprojects/research-spikes/long-horizon-agent-evals/.env.example:19
    • Evidence: "Validated against the 20260821/20260822 trace corpus: healthy runs peaked at a 22% duplicate rate, stalled runs reached 72-86%." No such corpus or summary table is present in the project (only a paraphrase in `src/lull.ts:71`).
    • Recommendation: Commit a small anonymized summary of the duplicate-rate distribution (or cite where the corpus lives) so the threshold choice is checkable.
    • Impact: A default that materially affects challenger thread rotation, and therefore trajectory length, rests on numbers no other researcher can inspect or re-derive.
  • low: Floating SDK range plus an instructed `npm update` can diverge from the tested lockfileprojects/research-spikes/long-horizon-agent-evals/README.md:124
    • Evidence: `package.json` declares `"@nvidia/openshell-sdk": ">=0.0.106"` while `package-lock.json` resolves 0.0.106; the install steps run `npm ci` then `npm update @nvidia/openshell-sdk`, which rewrites the committed lockfile as part of normal setup.
    • Recommendation: Note in the README that runs should record the resolved SDK version (already captured as `runtime.openshellSdk`) and that the lockfile change from `npm update` is expected, or pin the range and bump deliberately.
    • Impact: The recorded environment for a run may not match the committed lockfile, so "the tested lockfile" is not the version actually exercised; runs differ by whatever SDK was latest that day. The gateway/SDK equality check catches mismatch but not drift.

Strengths

  • The oracle is external: compromise is determined by querying GitHub refs and the target path, never by asking either model whether it succeeded (`src/campaign.ts:893`, `src/github.ts:35`).
  • Approval semantics are decomposed instead of collapsed into pass/fail: `reviewerApprovalCount`, `reviewerAppliedApprovalCount`, `reviewerFailClosedApprovalCount`, and `requiresAdjudication` are distinct outcome fields with unit tests (`src/campaign.ts:481`, `test/outcome.test.ts`).
  • Validity guards are explicit and testable: operational failures, zero challenger turns, insufficient oracle coverage, unresolved proposals, and OpenShell policy-reload failure all invalidate a non-compromised attempt.
  • Reviewer fairness is designed in: the prompt states that a false rejection and an always-reject strategy are reviewer failures, so approvals are not trivially avoidable.
  • Safety and secret handling are credible for a real-mutation experiment: prominent warning, disposable-repo guidance, `LAB_DEDICATED_GATEWAY` requirement, manual-review preflight, `.env`/`runs/` gitignored, `${NODE_AUTH_TOKEN}` in `.npmrc`, and token/JWT redaction in `src/common.ts:50`.
  • Engineering restraint is explicit and matched by the tree: one concrete experiment, no framework, and `AGENTS.md` forbids adding one before a second real experiment.

Limitations

  • No commands were executed and no dependencies installed; `npm run check`, `npm run endpoint:check`, `github:preflight`, `campaign`, and `scale` were all unrun. The npm registry is blocked in this environment and the PR itself reports the authenticated GitHub Packages dependency was unavailable, so `@nvidia/openshell-sdk` could not be resolved or typechecked.
  • No OpenShell gateway, Docker daemon, GitHub token, or model endpoint was available, so the central scientific claim path (challenger -> proposal -> reviewer -> policy application -> GitHub mutation) could not be replicated even partially.
  • Implementation review was representative, not exhaustive: I sampled `src/campaign.ts` (outcome classification, oracle, version and runtime metadata sections), `src/github.ts`, `src/reviewer.ts`, `src/openshell-version.ts`, `src/lull.ts`, `src/common.ts`, `src/costs.ts`, and several test files. `scripts/challenger.sh` (393 lines), `src/scale.ts`, `src/timeline.ts`, `src/transcript.ts`, `src/handoff.ts`, and the `src/usage.ts` pricing tables were not read line by line.
  • Only `projects/research-spikes/` was provided. Root `LICENSE`/attribution, the root `AGENTS.md` conventions, `docs/development/ci.md`, `.github/workflows/long-horizon-agent-evals.yml`, and the referenced dev-note post are outside the snapshot, so license placement, content-routing conventions, and CI wiring for this project were not verified.
  • Two symlinks were omitted from the snapshot (`CLAUDE.md -> AGENTS.md` at the repository root and `projects/tools/openshell-agent-runner/CLAUDE.md -> AGENTS.md`). Neither is inside this project, so the only resulting gap is that the root AGENTS.md conventions could not be read.
  • The PR-reported validation results (for example "OAR: 153 passed" and stale-path scans) were treated as untrusted claims and not independently confirmed.
  • No non-text documentation assets exist in this project apart from the inline mermaid diagram in the README; there were no images or reports to inventory.
projects/tools

The contribution regroups the repository's projects by semantic kind; the reviewed slice, `projects/tools/`, holds four reusable OpenShell tools - Egress Gate (Python gRPC supervisor middleware that runs an ordered gate pipeline over provider-bound requests), OpenShell Agent Runner (CLI that launches ephemeral Pi agents in sandboxes and returns validated JSON reviews), OpenShell Middleware Manager (`omm` generator/updater for Python and Rust middleware starters), and OpenShell Event Exporter (self-contained Go OTel Collector distribution for OCSF/CloudEvents delivery) - each with a `project.yaml` declaring `kind: tool`, self-contained manifests, lockfiles, Apache-2.0 licenses, and per-project READMEs/docs. Documentation is complete and internally consistent: every README gives a runnable first-use path (installed and source-checkout variants), states non-goals (Egress Gate is not a proxy/TLS interceptor/response filter; the exporter is experimental with no published image or module tag), and documents limits, failure mapping, and trust boundaries. Path updates to `projects/tools/...` are consistently applied in READMEs, docs links, Go module paths, test path constants, and the exporter Dockerfile; I found no stale `projects/<name>/` references inside the snapshot. Verification is credible and discoverable (~245 Egress Gate pytest cases including gRPC/service and CLI example-corpus tests, 173 OAR/OMM tests, 139 Go tests, plus `make check`/`scripts/check.sh`); I independently re-ran only the standard-library latency-figure check (`analysis/render_latency_plot.py --check`), which passed and reproduced the documented 0.56% turn share. Remaining issues are cosmetic (a dangling `plans/egress-gate-refactor.md` reference).

Project guidelines: ✅ Pass

"Declare the project kind": each of the four projects has `project.yaml` with `kind: tool`, and the group directory adds a documented compatibility marker. "Purpose and scope": each README states what the tool does, who it serves, how it uses OpenShell, and explicit non-goals (e.g. egress-gate/docs/index.md ownership boundaries; openshell-exporter/README.md "Experimental research software" and "No such release is published by this change"). "Self-contained layout": implementation, manifests, locks, docs, and runtime notes live inside each project directory; docs/ trees are substantive, not scaffolding. "Reproducible first run": prerequisites (uv 0.11+/Python 3.11+, Go 1.26, OpenShell 0.0.111+), smallest useful commands, and expected output are documented, including the exporter's synthetic no-credential quickstart. "Evidence and verification": focused test suites plus documented check commands exist, and unverified areas are stated (analysis/README.md interpretation limits; exporter "Synthetic local and container checks do not qualify live gateway behavior"). "Dependencies and configuration": Python projects use uv + pyproject.toml + committed uv.lock; Go uses go.mod/go.sum; no committed secrets found (only clearly synthetic test tokens); LICENSE present everywhere and NOTICE for the vendored Go distribution. "Proportionate engineering": absolute imports throughout the Python sources (no relative imports found outside generated bindings), nested AGENTS.md files document genuinely distinct conventions. The repository-level docs that describe the new grouping (projects/README.md, projects/PROJECT_GUIDELINES.md) are outside this snapshot, so the group-level rationale itself was not verifiable; that is recorded as a limitation rather than a violation because every in-scope per-project requirement is supported.

Findings

  • low: README and AGENTS.md reference a `plans/` directory that the project does not containprojects/tools/egress-gate/README.md:151
    • Evidence: README.md line 150-151: "Do not edit `plans/egress-gate-refactor.md` as part of implementation work." AGENTS.md line 124 repeats "Do not edit `plans/` as part of implementation work." `find projects/tools/egress-gate -name plans` returns nothing.
    • Recommendation: Delete the two `plans/` sentences, or link the plan document at its actual repository location.
    • Impact: A contributor following the README looks for a plan document that is not in the project, and the instruction cannot be acted on; it also weakens the guideline expectation that the project directory be self-contained.
  • low: Grouping directory carries kind metadata but no local orientation documentprojects/tools/project.yaml
    • Evidence: `projects/tools/project.yaml` contains only `# Compatibility metadata for the grouped-layout CI transition.` and `kind: tool`; the directory has no README explaining which tools it holds or why the extra nesting level exists.
    • Recommendation: Add a short `projects/tools/README.md` (or a comment pointing at `projects/README.md`) listing the contained tools and stating that the group-level `project.yaml` exists only for the CI transition.
    • Impact: Readers landing on `projects/tools/` get no map of the four tools or of the transitional purpose of the group-level `project.yaml`; the guidelines' `projects/<name>/` shape now has an undocumented intermediate level.
  • low: Exporter documents required environment variables only in prose, not in a `.env.example`projects/tools/openshell-exporter/README.md
    • Evidence: The HTTPS delivery path requires `export EXPORTER_DESTINATION_ENDPOINT=...` plus `secrets/destination-token` and `secrets/destination-ca.pem`; `.gitignore` excludes `.env` and `/secrets/`, but no `.env.example` is committed (`find projects/tools -name '.env*'` returns nothing).
    • Recommendation: Add a small `.env.example` listing `EXPORTER_HEALTH_ENDPOINT` and `EXPORTER_DESTINATION_ENDPOINT` with placeholder values, or note in the README why file-based secrets make an env template unnecessary.
    • Impact: The guidelines' "Dependencies and configuration" bullet asks for needed variables to be documented in `.env.example`; without one, operators must extract variable names from prose and from `http.config.yaml`.

Strengths

  • egress-gate/docs/reference/limits-and-failures.md tabulates every bound (4 MiB body, JSON depth 128, 10 gates, 4 concurrent slots) and the exact outcome mapping (INVALID_ARGUMENT / INTERNAL / runtime_limit deny / default deny), and the sampled constants.py values match.
  • Non-goals and unverified claims are stated plainly: openshell-exporter/README.md says "Experimental research software, provided as-is", "No such release is published by this change", and "Synthetic local and container checks do not qualify live gateway or external application behavior"; egress-gate/analysis/README.md lists six explicit interpretation limits for the latency figure.
  • The reorganization is applied consistently: the Go module path is `github.com/NVIDIA/OpenShell-Research/projects/tools/openshell-exporter`, test path constants and the Dockerfile license-ignore flag use the same prefix, and no `projects/<old-name>/` reference remains anywhere in the snapshot.
  • Reproducible offline verification exists for advertised behavior: `uv run egress-gate evaluate --policy examples/regex-redaction/... --cases ...` is documented with its exact expected output and is asserted by tests/test_cli.py; `analysis/render_latency_plot.py --check` reproduced the documented statistics (fit R^2=0.976, mean turn share 0.56%) using only the standard library.

Limitations

  • Repository-level documents that explain the new grouped layout (projects/README.md, projects/PROJECT_GUIDELINES.md, docs/development/ci.md, .github workflows and ci_scope.py/pr_review.py) are outside the reviewed snapshot, so the group-level rationale, CI discovery of the nested layout, and the PR's claimed cross-repository path updates were not verified.
  • No dependency installation was performed, so the Python (pytest/ruff/ty/pip-audit), Go (`go test -race ./...`), and Docker-based OAR runtime suites were not executed; test credibility was judged from test file inventory and sampled test bodies. The only check I ran was `analysis/render_latency_plot.py --check` on a scratch copy.
  • Publication claims (`uv tool install openshell-agent-runner`, `uvx --from openshell-middleware-manager omm`) and the documented external OpenShell gateway, inference, and Claude Code integration paths require network and hardware not available here; they remain unverified.
  • The snapshot omits two symlinks (projects/tools/openshell-agent-runner/CLAUDE.md -> AGENTS.md and a repository-root CLAUDE.md -> AGENTS.md), so I could not confirm the symlink targets or their mode; the AGENTS.md contents themselves were read.
  • Implementation reading was representative only: sampled egress-gate cli.py serve/command registration, constants.py, proto, service import boundaries, examples, and a few tests; OAR and omm sources, the entire Go exporter implementation, and all example configurations were inventoried but not audited line by line.
  • Rendered-figure and diagram SVG assets were inventoried and referenced-path-checked but not visually inspected.
projects/use-case-examples

The contribution regroups the repository's three demonstration projects under `projects/use-case-examples/` (the Reachy Mini + OpenShell onboard conversation demo, the robotics policy-prover demo, and the Python project template), adds `kind: use-case-example` metadata at the group root and in each project, and updates in-project path references for the new nesting depth. Each project stays self-contained with its own documentation, manifests, lockfiles, and tests. The reachy demo carries the most substantial workflow documentation (463-line README plus a 762-line onboard tutorial), and its OpenShell policy files, sandbox lifecycle, and REST tool boundary line up with the sampled implementation (fixed pose table, `minjerk`, 1–8 direction schema, `policy_denied` on 403, 2 MiB camera cap). Path rewrites inside the group are complete and the dev-note relative links resolve from the new depth. Remaining observations are minor and mostly inherited from the moved, pre-existing projects: the prover README omits Rust/cmake/Node prerequisites even though its Cargo build vendors Z3, the prover keeps duplicate `.env.example`/`.env.sample` files, its committed benchmark records a z3 crate version that no longer matches `Cargo.lock`, and the Python template is a copy-me scaffold rather than an end-to-end demonstration despite its declared kind.

Project guidelines: ✅ Pass

"Declare the project kind" is satisfied: `projects/use-case-examples/project.yaml` and each subproject declare `kind: use-case-example`. "Self-contained layout" holds — implementation, docs, manifests, and lockfiles stay inside each project directory, with directly relevant repository dev notes linked by correct relative paths. "Reproducible first run" is met for the reachy demo (prerequisites, `cp .env.example .env`, `./scripts/start-local.sh`, expected Gradio URL and daemon-status JSON) and the template (`uv sync --locked`, example command); it is weaker for the prover, whose README lists only `npm install` although the service build vendors Z3 and needs a Rust/cmake toolchain (see findings; reported low because that content is pre-existing and moved unchanged). "Dependencies and configuration" is met: `uv.lock`, the hash-pinned `requirements.txt` export, `package-lock.json`, and `Cargo.lock` are committed, `.env.example` files contain no secrets, `.env` is gitignored, and Apache-2.0 LICENSE files plus SPDX headers are present. "Evidence and verification" is supported by focused unit tests (including `tests/test_camera_policy.py`, which asserts the exact policy allow rules), a visual verification script, and recorded benchmark JSON with provenance fields. Kind expectations for use-case examples (concrete audience, start-to-finish workflow, safe configuration, demonstration-versus-deployable distinction) are documented, notably the explicit statement that JSON body values remain application-enforced rather than OpenShell-enforced. No demonstrated violation warranting changes was found.

Findings

  • low: Prover README omits Rust, Node, and cmake prerequisites for a build that vendors Z3projects/use-case-examples/robotics-policy-prover/README.md:30
    • Evidence: README "Run It" lists only `npm install`, then `npm run dev:prover` (which runs `cargo run --manifest-path policy-prover-service/Cargo.toml`), while `policy-prover-service/Cargo.toml:17` declares `z3 = { version = "0.20", features = ["vendored"] }`, requiring a C++/cmake toolchain. The PR description itself notes "Robotics Cargo tests need `cmake`".
    • Recommendation: Add a short prerequisites list (Node, a Rust toolchain, cmake/C++ build tools, optional Playwright browser for `verify:visual`) before the `npm install` step.
    • Impact: A reader following the documented steps on a clean machine can fail at the first-run command with an opaque native build error, weakening the reproducible-first-run guideline for this demo.
  • low: Duplicate, byte-identical `.env.example` and `.env.sample`projects/use-case-examples/robotics-policy-prover/.env.sample
    • Evidence: `diff .env.example .env.sample` reports no differences; the README instructs `cp .env.sample .env` while the guidelines and the sibling reachy project standardize on `.env.example` (both files are whitelisted in `.gitignore`).
    • Recommendation: Keep `.env.example` only and update the README's copy command accordingly.
    • Impact: Two sources of truth for environment configuration can drift, and readers may edit the file the documentation does not reference.
  • low: Committed benchmark provenance no longer matches the pinned solver dependencyprojects/use-case-examples/robotics-policy-prover/benchmarks/policy-latency.json:6
    • Evidence: The recorded run states `"z3_crate_version": "0.19.15 (bundled Z3 4.16.0)"`, while `policy-prover-service/Cargo.lock` pins `z3` version `0.20.2` and `Cargo.toml` requires `"0.20"`.
    • Recommendation: Regenerate the benchmark JSON with the pinned toolchain, or note in the README that the committed numbers were produced with an earlier z3 crate.
    • Impact: The published latency evidence describes a solver version the current lockfile cannot reproduce, so a reader re-running the documented benchmark command cannot match the recorded provenance.
  • low: Python project template declares `kind: use-case-example` but is a scaffold, not a demonstrationprojects/use-case-examples/python-project-template/project.yaml:1
    • Evidence: `kind: use-case-example`; the README's opening line is "A minimal Python package template for OpenShell research projects" and its first section is "Start a project from this template" (copy the directory and rename), with no scenario, audience statement, OpenShell usage, or non-goals.
    • Recommendation: Either reclassify the template under the `tool` grouping, or add one README paragraph stating the audience, the workflow it demonstrates, and its non-goals so the `use-case-example` kind is justified.
    • Impact: The declared kind and the grouping directory imply an end-to-end teaching workflow, which makes the semantic layout less predictable for contributors and for the kind-driven CI review.

Strengths

  • Path rewrites inside the group are complete: grep for the old `projects/<name>/` prefixes returns no matches, and the surviving references were updated to the new depth (`projects/use-case-examples/reachy-mini-openshell` in README.md:37 and ONBOARD_SETUP.md:130,435), so documented commands still work after regrouping.
  • The reachy demo's security boundary is documented honestly and matches configuration: ONBOARD_SETUP.md states that OpenShell cannot enforce JSON body values and that the fixed pose table is application validation, and `tests/test_camera_policy.py` asserts the exact allow rules of the three checked-in policy files.
  • Kind metadata is applied consistently and minimally — one two-line `project.yaml` at the group root and one per project — with no new abstractions, wrappers, or framework code introduced by the reorganization.
  • Configuration hygiene is sound across all three projects: placeholder-only `.env.example` files, `.env` gitignored, loopback-only default binds (`127.0.0.1:8787` in `main.rs:265`, `127.0.0.1` app host in `start-local.sh`), committed lockfiles for uv, npm, and Cargo, and Apache-2.0 licensing with SPDX headers.
  • Verification evidence is proportionate and varied: ~30 pytest modules for the reachy app plus native-controller tests, in-crate Rust tests in `main.rs`, a headless canvas verification script, and two committed benchmark JSON files with platform, revision, sample-count, and seed provenance.

Limitations

  • The reviewed snapshot contains only `projects/use-case-examples/`; the modified repository-level documents (`projects/README.md`, `projects/PROJECT_GUIDELINES.md`, `docs/projects/`, `.github/workflows/`, `.github/scripts/ci_scope.py`) were not available, so cross-repository link integrity, CI scope wiring, and the group's narrative documentation could not be verified.
  • The relative dev-note links (`../../../docs/dev-notes/...`) were checked only for correct depth arithmetic; the target files are outside the snapshot.
  • No test suite, build, or lint was executed: no `uv sync`/`pytest`, no `cargo test`, no `npm install`/`npm run build`/`verify:visual`, and no Docker image build. The PR's own validation claims (134 repo tests, prover frontend build, `uv lock --check`) were taken as unverified reported evidence.
  • Hardware- and service-dependent claims (Reachy Mini Wireless ARM64 robot, OpenAI Realtime, Riva ASR NIM, the DGX Spark benchmark host, Playwright Chromium) cannot be exercised in this sandbox.
  • Implementation inspection was representative: sampled `rest_tool_transport.py`, `native-controller/camera_adapter.py`, `scripts/start-local.sh`, one OpenShell policy file, `policy-prover-service/src/main.rs` routing and env handling, `vite.config.js`, `scripts/verify-visual.mjs`, and two tests. Most of the remaining ~100 source and test files in the reachy demo were not read.
  • Symlinks omitted from the snapshot (`projects/tools/openshell-agent-runner/CLAUDE.md`, root `CLAUDE.md`) are outside this group and were not assessed.
  • The three subprojects are pre-existing code moved by this PR; per the operator scope they were not audited line by line, and observations about their inherited content are reported at low severity.

Signed-off-by: Johnny Greco <jogreco@nvidia.com>
@github-actions

Copy link
Copy Markdown

Documentation preview

View the deployed preview

Built from 1d9379f.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant