Skip to content

test(bench): add node-vs-edge logical cost v0 - #503

Draft
aaltshuler wants to merge 1 commit into
mainfrom
codex/benchmark-v0
Draft

test(bench): add node-vs-edge logical cost v0#503
aaltshuler wants to merge 1 commit into
mainfrom
codex/benchmark-v0

Conversation

@aaltshuler

Copy link
Copy Markdown
Collaborator

Summary

  • add one selectively imported benchmark-v0 module for the node-versus-edge insert comparator, record schema, correctness oracle, ceilings, identity stamps, and S3 cleanup
  • extend the existing shared cost counters with manifest writes and aggregate logical-operation totals
  • run fresh indexed fixtures in AB/BA order on local storage and RustFS, requiring correctness, non-vacuous evidence, repeatability, and per-field upper ceilings
  • add a required local PR context and a post-merge/tag/manual RustFS qualifier, each retaining JSONL plus the full log for 14 days
  • document the current benchmark machinery, runnable commands, RFC-031 gaps, and CI/branch-protection policy

Calibrated v0 contract

Backend Node insert Edge insert
Local filesystem 14 ops (12 reads, 2 writes) 34 ops (32 reads, 2 writes)
RustFS 1.0.0-beta.12 37 ops (30 reads, 7 writes) 57 ops (50 reads, 7 writes)

Both backends reproduced exactly across AB/BA order. RustFS reproduced across independent runs and left no keys under either dedicated test prefix.

These are logical calls observed by Lance data-table and __manifest object-store wrappers. They do not claim physical HTTP attempts, SDK retries, non-Lance control-object I/O, bytes, billing, timing, or RSS.

Validation

  • cargo fmt --all --check
  • actionlint .github/workflows/ci.yml
  • shellcheck scripts/*.sh
  • cargo clippy --workspace --all-targets --locked -- -D warnings -W clippy::dbg_macro
  • cargo clippy --workspace --all-targets --locked --features omnigraph-engine/failpoints,omnigraph-cluster/failpoints -- -D warnings -W clippy::dbg_macro
  • RUST_MIN_STACK=16777216 cargo test --workspace --locked --features omnigraph-engine/failpoints,omnigraph-cluster/failpoints
  • full write_cost target: 13 passed
  • exact local comparator, including failpoints build and JSONL validation
  • exact configured RustFS comparator against the workflow-pinned image/digest
  • benchmark_scenario_contract plus warm-read, merge, and branch-control cost suites
  • scripts/check-agents-md.sh and git diff --check

Scope boundary

This is the minimal current-build v0 gate for one node insert versus one edge insert. It does not implement the RFC-031 previous-release-versus-candidate lifecycle matrix, physical counting proxy, timing/RSS/byte collection, reporter, baselines, waivers, release manifests, or evidence bundles.

The declarative branch-protection context is included in this PR. Applying it to live main protection remains a post-merge maintainer action.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant