Skip to content

Update protocol implementation to GP 0.8.0 - #402

Draft
qiweiii wants to merge 10 commits into
masterfrom
gp-0.8.0
Draft

qiweiii wants to merge 10 commits into
masterfrom
gp-0.8.0

Conversation

@qiweiii

@qiweiii qiweiii commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Update protocol implementation from GP 0.7.2 to 0.8.0: state/types, dynamic validator rules, host calls, PVM pipeline gas, and chain parameter encoding.
  • Pin davxy/jam-test-vectors to 39d0e35933305530fa1fe646bdedf221769398f1; follow the agreed deviations #533, #538, #559, #563 and non-manager bless behavior.
  • Derive erasure coding and reconstruction dimensions from the relevant validator set and retain coding-count metadata.
  • Include separate fixes for compact native JIT jump decoding, per-core privileged accumulation merging, and Merkle co-path traversal.

Validation

  • GP update commit 1570f45 passed all 11 packages across a full run and affected-package reruns after the Safrole ring-capacity correction.
  • Subsequently ran all 11 packages individually and sequentially with Swiftly's Swift 6.4.0 in debug mode; all passed.
  • That subsequent run included the pending availability changes now carried on the separate availability-networking branch. The published tip and its intermediate fix commits have not been independently retested without those pending changes.
  • Existing skips remain: historical GP 0.7.2 fuzz fixtures and the known-crashing Javajam STF fixture suite.
  • Native JIT execution was tested on ARM64; x86-64 decoder changes were not execution-tested here. STF execution-mode plumbing does not establish JIT coverage.

Update state encoding, dynamic validator rules, host calls and PVM pipeline gas accounting, including the agreed pinned-vector deviations. Derive erasure coding dimensions and network reconstruction counts from the relevant validator set, and align chain parameters with fetch encoding.

Keep pre-existing availability commitment, proof/wire-format and privilege-merge corrections for a separate follow-up. All 11 test packages passed across the full run and affected-package reruns after correcting Safrole ring-domain capacity.
@github-actions

Copy link
Copy Markdown

Found four actionable issues affecting x86-64 PVM execution and availability storage lifecycle/configuration.


Review Suggestions

PolkaVM/Sources/PolkaVM/InstructionTable.swift:L48

Mapping GP 0.8 opcode 2 to Unlikely leaves the x86-64 decoder's legacy opcode == 2 special case intact, where it still reports a two-byte JumpInd. Consequently the x86-64 compiler advances past the byte following every Unlikely, potentially skipping an instruction and diverging from the interpreter and ARM64 JIT. Remove the legacy opcode-2 sizing branch in x64_labeled_helper.cpp and derive this instruction's one-byte size from the supplied skip table/bitmask.


Blockchain/Sources/Blockchain/Validator/ErasureCodingDataStore/AuditBundleStore.swift:L103-L107

The coding-count metadata is loaded before checking the filesystem bundle. Availability entries created before this upgrade have no ec-validator-count record, so getBundle now throws metadataNotFound even when the complete bundle is locally available and needs no reconstruction. Check filesystemStore first, and only require the GP erasure dimensions when reconstruction is actually necessary; also provide a migration/fallback for existing shard-only entries.


Blockchain/Sources/Blockchain/Validator/ErasureCodingDataStore/AuditBundleStore.swift:L45-L47

The GP WB bundle limit is hard-coded to the mainnet-derived value even though this implementation decodes protocol constants dynamically and exposes ProtocolConfig.maxWorkBundleSize. A valid custom chain with a different WI, import, item, dependency, or extrinsic limit will incorrectly accept or reject bundles here. Pass the active config into this store and enforce config.value.maxWorkBundleSize instead of 13_904_950.


Blockchain/Sources/Blockchain/AuxDataStore/DataStoreProtocol.swift:L72-L80

Every erasure root now creates a separate validator-count metadata record, but the protocol provides no corresponding deletion operation and all audit/D3L cleanup paths only delete entries and shards. Long-running nodes therefore retain one RocksDB metadata row per expired availability object indefinitely. Add a deletion helper for this key and invoke it when audit or D3L data is removed, including iterative and aggressive cleanup paths.

@codecov

codecov Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 81.22935% with 284 lines in your changes missing coverage. Please review.
✅ Project coverage is 67.65%. Comparing base (0fb0b26) to head (282f74a).

Files with missing lines Patch % Lines
...chain/VMInvocations/HostCall/RefineHostCalls.swift 0.00% 71 Missing ⚠️
PolkaVM/Sources/CppHelper/a64_labeled_helper.cpp 0.00% 34 Missing ⚠️
...s/Blockchain/VMInvocations/HostCall/HostCall.swift 80.00% 16 Missing ⚠️
...aVM/Sources/PolkaVM/Executors/JIT/VMStateJIT.swift 0.00% 16 Missing ⚠️
PolkaVM/Sources/CppHelper/x64_labeled_helper.cpp 58.33% 11 Missing and 4 partials ⚠️
PolkaVM/Sources/PolkaVM/Engine.swift 52.38% 10 Missing ⚠️
Database/Sources/Database/RocksDBDataStore.swift 0.00% 9 Missing ⚠️
...ces/PolkaVM/Executors/JIT/ExecutorBackendJIT.swift 50.00% 9 Missing ⚠️
...AvailabilityService/ShardDistributionManager.swift 0.00% 7 Missing ⚠️
...dator/ErasureCodingDataStore/BatchOperations.swift 41.66% 7 Missing ⚠️
... and 30 more
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #402      +/-   ##
==========================================
- Coverage   67.70%   67.65%   -0.06%     
==========================================
  Files         476      484       +8     
  Lines       47161    48138     +977     
  Branches     1474     1495      +21     
==========================================
+ Hits        31931    32568     +637     
- Misses      14586    14927     +341     
+ Partials      644      643       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Continue directly past unreachable bytes in the x86-64 compilation loop. Expand gas-table regression coverage across entry PCs and add focused tests for report-derived reconstruction counts and atomic coding metadata.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant