Skip to content

Single-pass expression analysis groundwork - answer type questions from ExpressionResults - #5857

Open
ondrejmirtes wants to merge 71 commits into
2.2.xfrom
resolve-type-rewrite-2
Open

Single-pass expression analysis groundwork - answer type questions from ExpressionResults#5857
ondrejmirtes wants to merge 71 commits into
2.2.xfrom
resolve-type-rewrite-2

Conversation

@ondrejmirtes

@ondrejmirtes ondrejmirtes commented Jun 12, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Groundwork for the "new world" where an expression is traversed once: after processExpr, its ExpressionResult knows the before/after scopes, the type (typeCallback) and the narrowing (specifyTypesCallback), composed from child results instead of re-walking subtrees. Handlers then stop implementing TypeResolvingExprHandler; the old entry points (MutatingScope::resolveType, the TypeSpecifier dispatcher) are guarded behind NewWorld::disableOldWorld() and get mass-deleted in PHPStan 3.0.

What's on the branch, bottom up:

  • Guards + ExpressionResultFactory: old-world type resolution entry points throw when NewWorld::disableOldWorld() is flipped (the migration meter); all ExpressionResult construction goes through a generated factory.
  • ExpressionResult carries beforeScope, expr, typeCallback, specifyTypesCallback and is stored per node in ExpressionResultStorage (layered O(1) duplicate()), replacing the stored before-Scope.
  • ExprHandler / TypeResolvingExprHandler split: resolveType/specifyTypes move to the sub-interface so handlers can shed them one by one.
  • ExpressionResultStorageStack: old-world consumers (TypeSpecifier dispatcher, extensions, rules below PHP 8.1, unconverted handlers' resolveType) keep working for converted handlers' nodes. Every scope shares the stack created by its internal scope factory; NodeScopeResolver pushes the storage of the analysis in progress through MutatingScope::pushExpressionResultStorage() (always popped in finally, throwing on imbalance), and MutatingScope answers from the stored result - or processes a synthetic node on demand. Scopes never reference a storage directly, so nothing pins the result graph with the cycle collector disabled in bin/phpstan. Also adds MutatingScope::applySpecifiedTypes - filterBySpecifiedTypes without Scope::getType().
  • First two migrations: ScalarHandler and ArrayHandler no longer implement TypeResolvingExprHandler. The array migration is a precision win the old world cannot reach: each item type is captured at its own evaluation point, so [$b = 1, $b + 1, $c = $b, $c + 2, $c++, $c] infers array{1, 2, 1, 3, 1, 2}.

Verified: full test suite green, make phpstan clean, and analysis memory back at baseline (no leak from the result graph despite gc_disable()).

Closes phpstan/phpstan#13944
Closes phpstan/phpstan#12207
Closes phpstan/phpstan#7155
Closes phpstan/phpstan#14396
Closes phpstan/phpstan#11953
Closes phpstan/phpstan#12780

🤖 Generated with Claude Code

Closes phpstan/phpstan#14999
Closes phpstan/phpstan#13334

Closes phpstan/phpstan#15004

@ondrejmirtes ondrejmirtes mentioned this pull request Jun 12, 2026
Closed
@ondrejmirtes
ondrejmirtes force-pushed the resolve-type-rewrite-2 branch 3 times, most recently from ebb19a0 to 457689b Compare June 12, 2026 17:15
staabm
staabm reviewed Jun 12, 2026
View reviewed changes
Comment thread src/Analyser/DirectInternalScopeFactory.php Outdated
return $this->withFlavor(false);
}

private function withFlavor(bool $fiber): self

@staabm staabm Jun 12, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this read withFiber?

@ondrejmirtes
ondrejmirtes force-pushed the resolve-type-rewrite-2 branch 2 times, most recently from eb31077 to 59cbf22 Compare June 19, 2026 11:44
@ondrejmirtes
ondrejmirtes force-pushed the resolve-type-rewrite-2 branch from 59cbf22 to 125cf22 Compare June 20, 2026 11:56
@ondrejmirtes ondrejmirtes mentioned this pull request Jun 25, 2026
Closed
@SanderMuller SanderMuller mentioned this pull request Jun 25, 2026
Merged
@ondrejmirtes
ondrejmirtes force-pushed the resolve-type-rewrite-2 branch 4 times, most recently from f98892f to 4455baa Compare July 6, 2026 22:20
@ondrejmirtes
ondrejmirtes force-pushed the resolve-type-rewrite-2 branch from 61fe06e to e38aadd Compare July 16, 2026 14:56
@marcella-galambos-framna marcella-galambos-framna mentioned this pull request Jul 16, 2026
Closed
ondrejmirtes referenced this pull request Jul 23, 2026
@ondrejmirtes @claude
Add bench for the expression chain walk speedup from #6084
cdc6ca2 
Every property fetch / method call resolves its type by walking down to
the chain root to detect a nullsafe operator (NullsafeShortCircuitingHelper),
costing O(N²) walk steps per chain of depth N — with or without an actual
nullsafe operator in the chain. Deep loop-wrapped plain chains make that
walk dominate: 3.71s -> 3.14s wall (-15%), -18% user CPU from the
recursion-to-loop rewrite. The real-world counterpart is Symfony
TreeBuilder fluent chains (300+ calls in one statement) in Sylius bundle
Configuration classes, which dropped up to 23% per file.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016szvNF5RXhACdfMQNc6DVL
@ondrejmirtes
ondrejmirtes force-pushed the resolve-type-rewrite-2 branch 12 times, most recently from fb22d34 to 84b1614 Compare July 28, 2026 17:31
ondrejmirtes and others added 29 commits August 24, 2026 23:12
@ondrejmirtes
Memoize node-callback type asks by node identity
4d3e8cd 
Rules and collectors re-ask the same nodes across a callback batch, and
the walk scope's resolvedTypes memo used to answer those repeats in O(1)
before the callback-facing scope existed. Every repeat paid the
stored-result guard - variable-state compares, node-key printing on
re-priced asks - which shipmonk's rule set (disallowed-calls formatting
every call, the dead-code collectors) multiplied into whole test-file
windows running twice as slow as 2.2.x. The entry pins the asked node:
a dropped synthetic's object id can be reused by the next synthetic, and
the identity check rejects the stale hit.
@ondrejmirtes
Remove unnecessary EXTENSION_TAG from ExprHandler
4fdbc50
@ondrejmirtes
Align the extracted stmt handlers with the single-pass engine seam
ba5f7fb
@ondrejmirtes
Read argument and gathered-expression types from stored ExpressionRes…
de50eea 
…ults

Engine code (NodeScopeResolver, StmtHandlers, ExprHandlers) no longer prices
expressions through Scope::getType()/getNativeType() - every walked node's
type is read from its stored ExpressionResult:

- ArgsResult::requireArgResult() reads a call argument's result
  non-optionally; closure arguments now land in the ArgsResult map too (the
  default closure-arg branch stored the refined result but never captured it).
- FuncCallHandler's array-function special cases (array_walk, array_pop/shift,
  shuffle, sort family, array_splice, extract, array_push/unshift appending)
  read argument results, priced on the post-args scope where the original read
  did. The dynamic-callee type comes from the callee's stored result.
- Closure::bind()'s bound-this/scope arguments read their stored results
  (processArgs orders closures last, so they are stored by the time the bind
  scope factory runs); degenerate code prices as mixed instead of walking.
- ClosureTypeResolver reads gathered return/yield expressions and array_map
  argument types from the walk's storage, threaded through
  getClosureType()/buildClosureTypeForClosure()/buildClosureTypeForArrowFunction();
  previously the assembly re-walked every return expression because the body
  walk's storage was discarded. The scope read remains only for rule-facing
  bridge asks (no storage) and immediately-invoked closures (whose invocation
  arguments are walked after the closure itself).
- OutputBufferHelper reads the tracked ob_get_level() holder through
  readScopeStateOrSyntheticType() instead of pricing the synthetic call.

The new NoScopeTypeReadInEngineRule build rule enforces the ban, with the
documented seams allowlisted.
@ondrejmirtes
Price NewHandler's synthetic nodes through the sanctioned on-demand walk
00cf1a9 
The never-returning-constructor check and the parent-constructor template
resolution both walk synthetic nodes the handler built; route them through
NodeScopeResolver::processSyntheticOnDemand() instead of Scope::getType(),
and drop NewHandler from the build rule's seam allowlist.
@ondrejmirtes
Extract function-call scope effects into FuncCallScopeEffectsHelper
b1c4b9e 
The scope-only side effects of a call - by-ref array function results
(array_pop(), sort(), array_splice(), ...), paired-read invalidation
(json_last_error(), file_get_contents()), possibly-impure value remembering,
output-buffer level tracking and volatile-expression invalidation - move out
of FuncCallHandler::processExpr() into a dedicated helper. Nothing in the
moved chain touches the call's own result state.

processExpr() runs for every function call and its frame is paid on every
recursion level; the extraction shrinks it from 1,536 to 960 bytes
(91 -> 55 compiled variables) and FuncCallHandler from 1,338 to ~900 lines.
@ondrejmirtes
Emit the closure and arrow-function argument callbacks after their re…
0209d3b 
…sults are stored

processArgs fired the Closure/ArrowFunction argument's own node callback
before the body walk, so callback-side consumers (DependencyResolver,
rules listening on closure nodes) asking getType() re-walked the
still-unstored node. Firing the callback after storeExpressionResult()
mirrors processExprNodeInternal(): the answer comes from the stored
ExpressionResult.
@ondrejmirtes
Emit intermediate assign-target links after their dimensions are proc…
38327c7 
…essed

The dimensions loop in doPrepareTarget() fired an intermediate chain
link's node callback before evaluating its dimension and before the
link's write-flavoured result existed, so callback-side consumers
(NonexistentOffsetInArrayDimFetchRule, DependencyResolver) re-walked the
yet-unstored sub-expressions. The callback now fires at the end of the
iteration, with the link's entry scope, after its result is stored.
@ondrejmirtes
Store the AssignOp result before applyWrite() emits assignment nodes
04ee461 
applyWrite() emits PropertyAssignNode, whose rules ask about the whole
`$lvalue OP= value` expression - without a stored result they re-walked
the node mid-processing. The composed result is stored before applyWrite()
and overwritten with the final result after the handler returns.
@ondrejmirtes
Emit array item callbacks after their key and value are processed
05a3dd3 
InvalidKeyInArrayItemRule asks about the item's key at the ArrayItem
callback - firing the callback after the key and value walks lets it
answer from the storage instead of re-walking them.
@ondrejmirtes
Defer Unset_, ClassConst, Const_ and While_ statement callbacks past …
dd972cc 
…their expressions

These statements' rules (UnsetRule, ValueAssignedToClassConstantRule,
ValueAssignedToGlobalConstantRule, WhileLoopAlwaysFalseConditionRule)
read the statement's child expressions, so the statement callback now
fires inside the handler after those expressions are processed - with
the entry scope, joining the existing deferred set (Return_, Expression,
Echo_, If_, Switch_, Foreach_). The per-constant Const node callbacks
move after their value walks for the same reason.
@ondrejmirtes
Read tracked state and constant expressions without pricing in resolv…
bf902e8 
…eScopeStateType

The tracked-expression branch mirrors resolveType()'s holder lookup
directly (with late-resolvable types resolved, deliberately skipping the
extension hook like the getVariableType() read), and constant expressions
(scalars, self::KEY, bare constants) price through
InitializerExprTypeResolver - the isset()/??/empty ensure reads no longer
walk unprocessed nodes through getType().
@ondrejmirtes
Price enum case values through InitializerExprTypeResolver in EnumSan…
98dcd1d 
…ityRule

The rule fires on InClassNode before the enum body is walked; case values
are initializer expressions, so the initializer resolver (already used by
the not-backed-enum message) is the right source - not Scope::getType()
on the yet-unprocessed node.
@ondrejmirtes
Give lazy property-type inference a sanctioned independent-pass walk
dfff298 
inferAndCachePropertyTypes() builds its own scope and prices constructor
assignment values - a reflection-level pass independent of the file's
main walk. The new NodeScopeResolver::processIndependentPassExpr()
processes such expressions on a fresh storage instead of routing through
Scope::getType().
@ondrejmirtes
Process class attributes before emitting InClassNode
bf47338 
ClassAttributesRule fires on InClassNode and reads the attribute
constructor arguments - walking the attribute groups first lets it answer
from the storage.
@ondrejmirtes
Build the offset-write chain after the root walk in doPrepareTarget()
1f28931 
The Set*OffsetValueTypeExpr nesting substituted a property base's holder
type while nothing of the target was walked yet, re-pricing the fetch on
demand. Building the chain after the root walk reads the property base
and its current type from the stored result.
@ondrejmirtes
Create method call impure points after the arguments are processed
a74e1c1 
SimpleImpurePoint::createFromVariant() reads an argument's type for
pure-unless-callable-is-impure parameters - moved past processArgs() in
MethodCallHandler and StaticCallHandler, mirroring FuncCallHandler.
@ondrejmirtes
Read closure use-variable native types from scope state
7c2cec7 
enterAnonymousFunction() priced the use variable's native flavour through
getNativeType() on the node; the by-name read on the natively-promoted
scope is the same state without walking the possibly-unprocessed node.
@ondrejmirtes
Read immediately-invoked closure arguments without walking ahead
109c79b 
The closure is the callee, so its invocation arguments are walked after
it - constant arguments price through InitializerExprTypeResolver and
plain variables read scope state by name; only the remaining shapes keep
the ask-ahead-of-walk scope read.
@ondrejmirtes
Create constructor impure points after the arguments are processed
e6af36e 
SimpleImpurePoint::resolvePureUnlessCallableIsImpureVerdict() reads an
argument's type for pure-unless-callable-is-impure parameters - the
impure-point computation moves out of processConstructorReflection()
past processArgs(), like the other call handlers.
@ondrejmirtes
Consult the complex-union guard for tracked expressions in applySpeci…
7cba6be 
…fiedTypes()

The old-world appliers (addTypeToExpression()/removeTypeFromExpression())
skip narrowing a union past COMPLEX_UNION_TYPE_MEMBER_LIMIT members with
HasOffsetValueType intersections - the circuit breaker for combinatorial
array|object offset-access growth. applySpecifiedTypes() only consulted
it on the untracked-expression path, so isset()-style checks on a tracked
variable kept doubling its union (tests/bench/data/wordpress-user.php:
64.5s -> 1.4s, output identical to 2.2.x again).
@ondrejmirtes
Short-circuit unchanged variables in askScopeVariableStateMatches()
42bbd61 
The rule-facing gate compared each read variable via isSuperTypeOf() -
O(keys^2) on large constant arrays, once per callback-side ask. Type
identity and equals() (O(keys)) answer the overwhelmingly common
unchanged-variable case first; the engine-facing equals() gets the same
identity short-circuit. tests/bench/data: bug-5081 4.3s -> 2.0s,
bug-11913 4.0s -> 1.3s, bug-8503/bug-10979/nullsafe-chain-walk back at
or below the 2.2.x baseline.
@ondrejmirtes
Bump expected turbo version
4b8dc9a
@ondrejmirtes
Sort collected data by file before running CollectedDataNode rules
0adedd1
@ondrejmirtes
Bump expected turbo version
d9f0a15
@ondrejmirtes @claude
Adopt the upstream convergence replay implementation
ea0c332 
The branch's own fixpoint-replay commits were dropped during the rebase
over the merged extraction chunks (#6249); this restores the replay in
upstream's final form - raw-recorded pairs wrapped at replay time, the
RecordingNodeCallback short-circuit in callNodeCallback - woven into the
branch's handler shapes (ambient storage push around pass walks, the
deferred While_ statement callback, the on-demand falsey cond
re-pricing). replayRecording() binds the storage through the scope's
push/pop like every other branch-side ambient binding. The consume
mechanism (#6251, closed unmerged) is gone entirely: convergence
behavior on this branch is now byte-for-byte upstream's algorithm.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GnwgpaeUXRkgSDyg95tfK8
@ondrejmirtes
Bump expected turbo version
e4cfba9
@ondrejmirtes @claude
Adopt the upstream gatherer frames
79dacde 
The rebase over the merged gathering cleanup (#6258) kept the branch's
wrapper-based gatherer sites while the base deleted
GatheringNodeCallback; this converts them to the upstream frame
mechanism - pushNodeGatherer()/popNodeGatherer() on NodeScopeResolver,
fed the raw walk scope by callNodeCallback() and per replayed pair by
replayRecording() - woven into the branch's shapes: the per-body
storages of method/function bodies, the ambient storage pushes, and the
scope-bound replayRecording() signature. Gatherer bodies are unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GnwgpaeUXRkgSDyg95tfK8
@ondrejmirtes
Bump expected turbo version
f6fa312
@ondrejmirtes
ondrejmirtes force-pushed the resolve-type-rewrite-2 branch from d9a567a to f6fa312 Compare August 24, 2026 21:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment