Skip to content

feat(fast-inbox): world-state per-block message insertion (A-1380) - #24785

Closed
spalladino wants to merge 2 commits into
spl/a-1379-archiver-bucketsfrom
spl/a-1380-world-state-per-block
Closed

feat(fast-inbox): world-state per-block message insertion (A-1380)#24785
spalladino wants to merge 2 commits into
spl/a-1379-archiver-bucketsfrom
spl/a-1380-world-state-per-block

Conversation

@spalladino

@spalladino spalladino commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

Part of the Fast Inbox stack (AZIP-22, FI-10 / A-1380). Base: `spl/a-1379-archiver-buckets`.

What this changes

  • `NativeWorldStateService.handleL2BlockAndMessages` no longer throws when a non-first block carries an L1-to-L2 message bundle. Any block may now transition the L1-to-L2 message tree.
  • First-in-checkpoint bundles are still padded to `NUMBER_OF_L1_L2_MESSAGES_PER_ROLLUP` (so the legacy call shape produces bit-identical trees); non-first bundles are appended exactly as given (the post-flip compact append the circuits perform). Padding stays in this method as a transitional concern and moves to the caller at the flip.
  • The legacy "non-first blocks carry no messages" rule moves from a throw inside world-state to a transitional assertion owned by the synchronizer call site (`handleL2Block`). The synchronizer's per-checkpoint message fetch is unchanged pre-flip.
  • `MerkleTreeAdminDatabase.handleL2BlockAndMessages` JSDoc updated to document the per-block semantics.

C++ investigation outcome (no barretenberg change needed)

Investigated the native sync/unwind paths under `barretenberg/cpp/src/barretenberg/world_state/`. Per-block message insertion and per-block unwind already work with no C++ change:

  • The L1-to-L2 message tree is the same append-only `FrTree`/`FrStore` as the note-hash tree, which already varies its leaf count per block and unwinds per block.
  • `commit_block` increments the tree's block height and writes a per-block `BlockPayload` (`{size, root}`) on every `sync_block`, even when zero leaves are appended, so the message tree's per-block history stays in lockstep with the other trees.
  • `unwind_block` reverts exactly one block's appended leaves from that stored per-block metadata, generic over leaf count. C++ has no concept of an L2 checkpoint or "first block" — that is a TS-only invariant — so a mid-checkpoint block that appended message leaves unwinds correctly.

The TS-side pending-chain rollback (`handleChainPruned` -> `unwindBlocks`) just unwinds to a block number and carries no checkpoint assumption about the message tree either.

Tests

  • Every-block equivalence: a legacy-shaped multi-block checkpoint leaves the message tree byte-identical (root + size) on every non-first block, while the archive advances per block — proving the degenerate path is a no-op at every block, not just at checkpoint ends.
  • Per-block appends: a checkpoint whose blocks advance the message tree by exactly their (unpadded) bundle size on non-first blocks; forks opened mid-chain see exactly the bundles up to that block.
  • Per-block unwind: unwinding a message-carrying non-first block reverts exactly its appended leaves back to the pre-block state, and re-syncing afterwards reconverges.
  • Synchronizer guard: a non-first block carrying messages is rejected by the call-site assertion.

world-state type-checks clean and all world-state unit tests pass (76 across the two modified suites). The full `yarn build` still fails only in the pre-existing stale-artifact packages (noir-protocol-circuits-types, simulator, prover-client), unrelated to this change.

Review follow-up

Codex (routine review) confirmed the C++ no-change conclusion and the pre-flip synchronizer soundness, and flagged that the TXE world-state caller (txe/src/state_machine/synchronizer.ts) padded the bundle unconditionally. TXE mines one block per checkpoint (always the first block), so this was a no-op in practice, but the second commit makes its padding conditional on indexWithinCheckpoint === 0 to keep the caller aligned with the new per-block semantics.

@spalladino spalladino changed the title feat: world-state per-block message insertion (A-1380) feat(fast-inbox): world-state per-block message insertion (A-1380) Jul 19, 2026
@spalladino
spalladino force-pushed the spl/a-1379-archiver-buckets branch from 6300e86 to c1e6c72 Compare July 19, 2026 14:05
@spalladino
spalladino force-pushed the spl/a-1380-world-state-per-block branch from 5c3c18d to d491955 Compare July 19, 2026 14:05
@spalladino
spalladino force-pushed the spl/a-1379-archiver-buckets branch from c1e6c72 to cd6bd75 Compare July 19, 2026 17:57
@spalladino
spalladino force-pushed the spl/a-1380-world-state-per-block branch from d491955 to 8cf709e Compare July 19, 2026 17:57
@spalladino
spalladino force-pushed the spl/a-1379-archiver-buckets branch from cd6bd75 to 732487f Compare July 19, 2026 20:48
@spalladino
spalladino force-pushed the spl/a-1380-world-state-per-block branch 2 times, most recently from 5b1f4f2 to befe2cf Compare July 20, 2026 15:30
@spalladino
spalladino force-pushed the spl/a-1379-archiver-buckets branch from 0d89cc1 to a6944fa Compare July 20, 2026 17:28
@spalladino
spalladino force-pushed the spl/a-1380-world-state-per-block branch 2 times, most recently from 0b772e7 to fa3bf1f Compare July 20, 2026 21:21
@spalladino
spalladino force-pushed the spl/a-1379-archiver-buckets branch from 0cebbd3 to 62dcb67 Compare July 21, 2026 02:58
@spalladino
spalladino force-pushed the spl/a-1380-world-state-per-block branch 2 times, most recently from 5c1fb65 to ff85153 Compare July 21, 2026 03:39
@spalladino
spalladino force-pushed the spl/a-1379-archiver-buckets branch from 68bdb45 to b40507e Compare July 27, 2026 20:53
@spalladino
spalladino force-pushed the spl/a-1380-world-state-per-block branch from ff85153 to 6022263 Compare July 27, 2026 20:53
Lift the non-first-block guard out of NativeWorldStateService.handleL2BlockAndMessages
so any block may carry an L1-to-L2 message bundle and transition the message tree.
First-in-checkpoint bundles are still padded to NUMBER_OF_L1_L2_MESSAGES_PER_ROLLUP
(bit-identical trees for the legacy call shape); non-first bundles are appended
unpadded (post-flip compact-append semantics). The transitional non-first-blocks-
carry-no-messages invariant now lives at the synchronizer call site as an assertion.

Per-block unwind already works natively: the append-only message tree commits a
per-block snapshot on every sync_block (like the note-hash tree) and unwinds per
block regardless of leaf count, with no C++ change required.
…kpoint blocks (A-1380)

Align the TXE world-state caller with the per-block message-insertion semantics of
handleL2BlockAndMessages: pad the bundle only when the block is first in its checkpoint,
matching native world state. TXE mines one block per checkpoint (always the first block),
so this is a no-op for current behaviour but keeps the caller consistent with the new
per-block shape.
@spalladino

Copy link
Copy Markdown
Contributor Author

Superseded by #25037.

The Fast Inbox stack has been regrouped from 20 per-issue PRs (plus 2 umbrellas) down to 3 area PRs plus an umbrella, and rebased onto merge-train/spartan including #25007 (the noir-projects fnd//labs/ split). This PR's diff is preserved verbatim as a single squashed commit in #25037, and this description is preserved in that commit's message.

New structure: #25036 (circuits + L1) → #25037 (node + flip) → #25038 (cleanup), with #25039 as the full-stack umbrella. The original branch for this PR is left on the remote as a recovery point.

Closing here to cut the rebase and review overhead of maintaining 22 PRs; the work is not abandoned.

@spalladino spalladino closed this Jul 28, 2026
spalladino added a commit that referenced this pull request Jul 28, 2026
…387)

FI-17 of the Fast Inbox (AZIP-22) stack. Stacked on #24791 (A-1386); it removes the legacy `inHash` from the checkpoint header now that the consensus flip (#24789, A-1384) has switched validation to the streaming rolling hash. After this PR, `inboxRollingHash` is the checkpoint's only inbox commitment.

## Consensus-format change

- The checkpoint header loses its `inHash` field across all three preimage layers in lockstep: noir `CheckpointHeader` (`checkpoint_header.nr`), TS `CheckpointHeader` (`stdlib/src/rollup/checkpoint_header.ts`), and Solidity `ProposedHeaderLib` (struct + hash packing). The field ordering is otherwise unchanged; the header shrinks by one field (32 bytes: `CHECKPOINT_HEADER_SIZE_IN_BYTES` 380 to 348, `CHECKPOINT_HEADER_LENGTH` 14 to 13).
- The header-hash fixtures were regenerated from `checkpoint_header.test.ts` (the established workflow) and pasted into the noir tests; the p2p golden-byte fixture (`wire_compat_fixtures.ts`) was refreshed for the checkpoint-proposal serialization, which shrinks by exactly the one removed field. The block-proposal wire fixture is unchanged (block-level `inHash` is out of scope, see A-1388).

## Circuits and constants

- Strips the dead unconstrained `in_hash` pass-through from the sized `InboxParity<S>` circuit and `ParityPublicInputs` (noir + TS + the hand-written `noir-protocol-circuits-types` conversion wrappers). The circuit is kept (sized-parity, per FI-06's topology decision), so no VK-tree indices move.
- Retires `NUMBER_OF_L1_L2_MESSAGES_PER_ROLLUP` (replaced everywhere by `MAX_L1_TO_L2_MSGS_PER_CHECKPOINT`, same value 1024) and the now-dead `NUM_MSGS_PER_BASE_PARITY`, `NUM_BASE_PARITY_PER_ROOT_PARITY`, and `L1_TO_L2_MSG_SUBTREE_ROOT_SIBLING_PATH_LENGTH`. Generated constants (`constants.gen.ts`, `ConstantsGen.sol`) were regenerated via `remake-constants`.
- Deletes `FrontierLib.sol` and its tests (`Parity.t.sol`, the `Frontier` harness and merkle test) now that their last users are gone.

## Verification

- Noir: `types::checkpoint_header` (5/5), `rollup_lib::parity` and `checkpoint_root::parity_tests` (15/15), and the checkpoint-root header-assembly path pass locally. The full `checkpoint_root` suite (68 tests) is too slow to run in full locally and rides CI.
- Solidity: `forge test` green across `Rollup`, `RollupFieldRange`, `Inbox`, `InboxBuckets`, `RollupGetters` (76/76), covering header hashing and the regenerated checkpoint fixtures.
- Jest: `checkpoint_header`, the p2p wire-compat suites, and the parity/messaging suites pass (43 tests), confirming the fixture regeneration is byte-consistent.
- VK/artifact/Prover.toml regeneration and the generated Noir ABI types ride CI (local `yarn build` in `noir-protocol-circuits-types` / `simulator` / `prover-client` is expected to fail on stale circuit artifacts until then); e2e and the broken-zone typecheck are the CI-of-record for those.

## Stack

Part of the Fast Inbox stack #24784 through #24791 (umbrella #24774): A-1379 (#24784), A-1380 (#24785), A-1381 (#24786), A-1382 (#24787), A-1383 (#24788), A-1384 flip (#24789), A-1385 (#24790), A-1386 (#24791, base of this PR).

Resolves A-1387.


## l1 publisher inbox consumption via the streaming selector (CI fix, pass-8)

The A-1387 inHash-removal rewrite left `l1_publisher.integration.test.ts` hardcoding `previousInboxRollingHash = Fr.ZERO` and `bucketHint = 0n` while consuming real L1→L2 messages, so message-consuming checkpoints reverted on L1 with `Rollup__InvalidInboxRollingHash` / `UnconsumedInboxMessages` (masked every prior CI round behind earlier failures). `INBOX_LAG_SECONDS` is an L1-time censorship cutoff, not whole checkpoints, so no checkpoint-depth shift register reproduces the aged bucket set. The test now mirrors the real Inbox buckets into its `MockL1ToL2MessageSource` and reuses the production `selectInboxBucketForBlock` (which mirrors `ProposeLib.validateInboxConsumption`) to pick exactly the buckets each checkpoint must consume, deriving the consumed bundle, the propose bucket hint, and the header rolling hash from that one selection. Box: 13/13, verified at a-1387 and a-1388.


Replaces #24792.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant