Skip to content

fix: run the outcall cycles migration regardless of subnet activity - #11219

Open
mraszyk wants to merge 1 commit into
masterfrom
mraszyk/migrate-outcalls-cycles-every-round
Open

fix: run the outcall cycles migration regardless of subnet activity#11219
mraszyk wants to merge 1 commit into
masterfrom
mraszyk/migrate-outcalls-cycles-every-round

Conversation

@mraszyk

@mraszyk mraszyk commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

The migration added in #10844, which brings the HTTPOutcalls / ECDSAOutcalls entries of SubnetMetrics::consumed_cycles_by_use_case up to the legacy scalar fields, was invoked from observe_consumed_cycles_with_use_case. That only fires when the subnet observes a subnet-level use case, and those are only observed on HTTP outcalls, threshold signature outcalls, canister deletion and cycles lost to dropped messages. A subnet that does none of these never runs the migration and its use-case entries stay stale indefinitely.

This is not hypothetical: on mainnet subnet
2fq7c-slacv-26cgz-vzbx2-2jrcs-5edph-i5s2j-tck77-c3rlz-iobzx-mqe, consumed_cycles_ecdsa_outcalls is 244_900_000_000_000 while the ECDSAOutcalls use-case entry is 228_370_000_000_000, a gap of 16_530_000_000_000 cycles. Its consumed_cycles_by_use_case_as_counters map is empty, which (since that map is incremented on the very same code path) shows that no subnet-level use case has been observed there since the counters were introduced in #9922, i.e. well before the migration was deployed.

The migration now runs from the scheduler's finish_round, which is documented as running unconditionally after each round and is also reached by the heap-delta early return and by checkpoint_round_with_no_execution. Running it once per round instead of per observation is equivalent, because the call sites bump the scalar field and the matching use-case entry by the same amount: max(entry, scalar) + delta == max(entry + delta, scalar + delta). It is also idempotent, so the extra invocations are harmless.

The migration added in #10844, which brings the HTTPOutcalls / ECDSAOutcalls
entries of SubnetMetrics::consumed_cycles_by_use_case up to the legacy scalar
fields, was invoked from observe_consumed_cycles_with_use_case. That only fires
when the subnet observes a subnet-level use case, and those are only observed on
HTTP outcalls, threshold signature outcalls, canister deletion and cycles lost
to dropped messages. A subnet that does none of these never runs the migration
and its use-case entries stay stale indefinitely.

This is not hypothetical: on mainnet subnet
2fq7c-slacv-26cgz-vzbx2-2jrcs-5edph-i5s2j-tck77-c3rlz-iobzx-mqe,
consumed_cycles_ecdsa_outcalls is 244_900_000_000_000 while the ECDSAOutcalls
use-case entry is 228_370_000_000_000, a gap of 16_530_000_000_000 cycles. Its
consumed_cycles_by_use_case_as_counters map is empty, which (since that map is
incremented on the very same code path) shows that no subnet-level use case has
been observed there since the counters were introduced in #9922, i.e. well
before the migration was deployed.

The migration now runs from the scheduler's finish_round, which is documented as
running unconditionally after each round and is also reached by the heap-delta
early return and by checkpoint_round_with_no_execution. Running it once per
round instead of per observation is equivalent, because the call sites bump the
scalar field and the matching use-case entry by the same amount:
max(entry, scalar) + delta == max(entry + delta, scalar + delta). It is also
idempotent, so the extra invocations are harmless.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added the fix label Aug 19, 2026
@mraszyk
mraszyk marked this pull request as ready for review August 19, 2026 18:26
@mraszyk
mraszyk requested a review from a team as a code owner August 19, 2026 18:26
@zeropath-ai

zeropath-ai Bot commented Aug 19, 2026

Copy link
Copy Markdown

No security or compliance issues detected. Reviewed everything up to ab33d4a.

Security Overview
Detected Code Changes
Change Type Relevant files
Enhancement ► rs/execution_environment/src/scheduler.rs
    Backfill the HTTP/ECDSA outcall cycles from legacy scalar fields into use cases in finish_round()
Enhancement ► rs/execution_environment/src/scheduler/tests/metrics.rs
    Add test for migrating outcalls cycles into use cases on idle subnet
Enhancement ► rs/replicated_state/src/metadata_state.rs
    Make migrate_outcalls_cycles_to_use_cases public and remove inlined migration call from observe path; update comments
Enhancement ► rs/replicated_state/src/metadata_state/tests.rs
    Add tests for migrating outcalls scalar fields into use cases and related scenarios

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant