A second, flag-gated static-analysis stack ("v2") on JuliaSyntax 2.0 + JuliaLowering - #219
Open
davidanthoff wants to merge 163 commits into
Open
davidanthoff wants to merge 163 commits into
davidanthoff wants to merge 163 commits into
Conversation
# Conflicts: # Project.toml
Main's inventory now descends into @testitem/@testsnippet bodies as synthetic `#test…` scopes; v2 deliberately keeps those macrocalls opaque at the inventory level, so the differential excludes them. Also sort import tuples by repr, since `alias` can be `nothing` on both sides. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…c 12.1) Syntax diagnostics, test item detection and the syntax lint tier each ran their own JuliaSyntax parse of every file per pass (10.4% of a cold pass). `derived_julia_parse_products` now parses once and returns all three products as plain data; each consumer reads its slice through a selector, so the products backdate independently. The syntax checks all run unconditionally inside the fused query — enabling is a filter in `derived_syntax_lint_findings`, so a config edit no longer re-runs the parse. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…step 1) The v2 walker already visited every @testitem/@testmodule/@testsnippet macrocall and threw the detail away. It now scans the macrocall argument shape (a port of TestItemDetection.find_test_detail! onto the position-free BodyTree, same shapes accepted, same error strings) into V2TestItem / V2TestError records on the skeleton, plus a derived_v2_file_testitems projection. Records carry no ranges and no code string, so they backdate on position-only and body edits; _v2_test_macro_addresses supplies the preorder addresses the emission join will use to reattach ranges. Enumeration and descent are unchanged — the isolation rule is untouched. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… 2+3) `derived_v2_testitem_details` is the volatile emission join: it reattaches range, code_range, the code slice and a non-literal skip expression from `derived_v2_file_maps` onto the position-free V2TestItem records, then runs the same assembly (ids, outside-package check, duplicate labels) as the legacy path — factored out of `derived_testitems` into `_assemble_test_details` so the two engines share everything below detection. With `input_lowering_lint` on, `derived_testitems` reads the v2 path; flag-off keeps the fused-parse legacy path. Tested with parity units over every well-formed and malformed macro shape, backdating tests (position-only and body edits leave the records isequal), and a corpus-wide differential against TestItemDetection that currently declares zero divergences. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
`derived_v2_file_walk` parsed strictly, so any syntax error emptied the whole walk — under the flag, a broken file lost all its test items (and its entire v2 inventory), while legacy TestItemDetection kept finding intact items in the recovered tree. The walk now parses with `ignore_errors=true`: K"error" nodes flow into BodyTrees like any other kind, the lowering layer's per-item degrade contains them, and the error itself is still reported by derived_julia_syntax_diagnostics. Broken-file shapes (error above an item, garbage inside and between items, truncation at EOF inside a block) are parity-tested against the legacy engine — both parsers recover them identically. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
`juliadynamicanalysisprocess/expandMacros` takes a batch of macrocall source texts sharing one module context (ctxId + canonical import statements) and returns per-entry expansion text or error. The child builds and caches a context module per ctxId (the packages behind the imports are already loaded in its session from indexing), runs Revise.revise() before each batch so deved-package macro edits are picked up, and expands with Base.macroexpand recursive. First structured result type on this protocol. Part of DJP-side macro expansion M1a (see design plan). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The persistent env child (which already has the environment's packages loaded from indexing) now serves expandMacros batches: the reactor routes ExpansionBatchMsg to df.procs[env_key] through a per-key FIFO, the process FSM gains its one new edge (Done -> Indexing), and batch failures kill the child with every entry negative-cached — never touching pending_count, the launch cap, or the env failure budget. On the Salsa side (src/v2/layer_expansion.jl, behind the lazily-false input_macro_expansion flag): expansion sites are harvested position-free with _materialize's exact address accounting; cache keys are (env content hash, module-context hash, macrocall BodyTree hash), where the context hash folds in the own package's macro-definition body hashes so editing a deved macro re-keys exactly the affected sites (D2b); _reconcile_expansions! reattaches source text at the last mile (the volatile maps' third legitimate reader, host-side) and sends capped batches. Settled :ok text is re-parsed with the vendored JuliaSyntax into a position-free BodyTree and spliced into _materialize at address 0 while KEEPING the identifier-read fallback (the union guard), so a macro that drops an argument can never create a false positive; :failed and unparseable results keep today's behavior bit for bit. Tested process-free through input_macro_expansions (splice, union guard, negative cache, re-keying on macro-def edits) plus reactor settle/queue/ prune tests. End-to-end fixture test follows. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A warm environment completes its work item without ever launching a child (the prep task sees every symbol cache present), so expansion batches found no process to ride on and settled as failed — the common case, not the edge. The drain now waits while the env work item is still in flight, and for a done-but-process-less env revives a child through the refresh machinery (idle priority, at most once per key so a crashing child cannot loop); the refresh completion path emits the idempotent EnvironmentReadyResult for plain watched envs and kicks the drain. Verified live: a fixture package defining @double expands through a revived child in ~20s, and the expansion reaches lowered bindings (`*` read at addr 0). The same scenario is now a testitem gated behind JW_E2E_DYNAMIC=1. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
derived_file_expansion_ready mirrors derived_file_env_ready: true when the flag is off, the file is outside the v2 lint, it has no expansion sites, or every site's key has settled ok-or-failed. Nothing consults it yet — it exists for future use-before-definition-class rules, which must not run against the identifier-read fallback. Batches are now sent regardless of dynamic mode (the reactor settles non-persistent entries :failed immediately), so the gate cannot hang wherever a dynamic feature exists; and _reconcile_expansions! prunes settled entries and requested keys whose env_hash left the required set, bounding memory across env edits. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The walker computed the preorder address ranges for every statement but discarded them for module and using/import rows (which get no body). Module- and import-level findings (module_name, relative_import) need real ranges at the emission join, so the map — volatile anyway — is now stored for those rows too; bodies stay unstored. A test pins preorder address 3 as a module's name token. Part of the Harvest JuliaLowering milestone (commit 1 of 6). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every LoweringError from desugaring/validation/scope resolution was caught into ItemLowering(:error, ...) and silently discarded. The findings now route out of derived_item_semantic_findings, reported under a new lowering_errors rule (off by default, warning in strict) with a message->rule-id table ready for takeover routing. Three false-positive channels the corpus sweep exposed are guarded: items whose lowered form is synthetic (test-block let-wrapping), items enumerated from inside a macrocall's arguments (a `Salsa.@derived function` or `@kwdef struct` may be transformed by the macro — a new V2ItemRow.under_macrocall marker), and items containing stripped macrocalls (`function (@main)(args)` loses its name to materialization — detected via the expansion-sites query). Files with syntax errors also suppress the catch-all id, since lowering recovered trees duplicates what syntax_errors already reports. The sweep testitem pins the corpus at zero lowering_errors findings with an empty allowlist. Harvest JuliaLowering, commit 2 of 6. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
duplicate_function_argument, break_continue and global_const_decl now come from JuliaLowering under the flag: the exact vendored message strings map to the rule ids (plus a pinned prefix rule for the parameterized destructured-argument conflicts), and the three ids join LOWERING_TAKEOVER_RULES so StaticLint is suppressed when active. A refresh-gate testitem lowers one snippet per table row and asserts the routed id, so a vendored-JuliaLowering refresh that rewords a message fails loudly instead of silently demoting findings to lowering_errors. The corpus differential pins flag-off/flag-on agreement for the routed ids with an empty allowlist. Harvest JuliaLowering, commit 3 of 6. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A where-clause parameter mints two bindings at the declaration address: a :typevar whose reads are the signature uses and a :static_parameter whose reads are the body uses. The parameter is unused only when both are unread. Struct and type-alias parameters lower as :local and are deliberately not covered, matching v1's where-clause-only semantics (pinned by tests). v1 message wording verbatim; the corpus differential and sweep stay at empty allowlists. Harvest JuliaLowering, commit 4 of 6. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A new position-free per-file producer reads the skeleton plus the module-tree splice prefix (first root, the layer_expansion precedent): module_name fires when a module is named like its parent — including the cross-file case where the parent comes through the splice point, which v1's same-file check misses — and relative_import fires on more leading dots than the site's nesting allows (the exact unresolved-by- pops condition the module tree itself uses). Findings attach to module/import rows through the maps commit 1 started storing; both ids join the takeover set. Corpus differential extended to all six newly-taken ids, still with an empty allowlist. Harvest JuliaLowering, commit 5 of 6. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…d behaviours are v2-only Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…n default; v2 keeps an explicitly imported name's import binding over a bare redefinition main's default preset now has missing_reference / unresolved_import / incorrect_call_args off, so the v2 suites that assert on them give their workspaces a JuliaLint.toml turning them on. main's module tree also stopped recording a bare function/assignment definition of an explicitly imported name as a local declaration (the import stays the visible winner); the v2 module tree mirrors that rule so the visibility differential is zero again. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
# Conflicts: # test/test_includes.jl # test/test_scratch_env.jl
…is under main's borrowed manifests main gained #300's _failure_folder_uri, #302's fused parse (RawTest*Detail), #301's include walker (runtime targets, quoted skip, module scoping) and #303's _watch_target_for_project / derived_workspace_members. The v2 twins carried copies of all of these: the copies go (a same-name method would have replaced main's on the v1 path), v2's differently-scoped workspace query is renamed derived_workspace_members_v2, and the include twin shrinks to the diagnostics emission — the walk is v1's now, including main's decision to drop the @safetestset special case. With #291 a member borrows its parent's manifest in the shared folder table; derived_project_v2 treats a manifest that is not the folder's own as none, so the member is still synthesized from its [deps] closure and covered by the root's watch item. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
main's source-scan rule (#306): feature layers never read input_text_file directly, so cross-file results landing in an indirect file do not throw. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…must name StaticLint "v2 is everything under src/v2/" is literally true again. The analysis core stays CSTParser/StaticLint-free (the guard test enumerates src/v2/ itself); src/v2/bridge/ holds the four twins that honour a v1 contract and therefore name StaticLint or CSTParser — the diagnostics join, the include diagnostics, the feature layer and the environment seam. packagedef.jl still loads each twin right after the v1 layer whose gate dispatches to it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
bodytree moved under the aqua branch: it merged origin/main and split the code into v1/v2 twins (flag off == main exactly, v2 under src/v2 with src/v2/bridge for files that must name the v1 pipeline). The four package-quality rules follow that discipline — registration and option validation stay shared, emission joins only derived_diagnostics_v2: - missing_compat / unused_dependency: producers ride the layer_project_files -> v2/layer_project_files_v2 rename; their emission moves into the bridge diagnostics twin next to the project/manifest problem walk. - unbound_type_parameter: out of the shared SYNTAX_CHECKS tuple (src/lint_syntax_rules stays byte-identical to main); the check becomes its own producer query in src/v2/bridge/lint_unbound_type_parameter_v2.jl, joined in the twin. - undocumented_public_name: layer_undocumented_names moves to src/v2/bridge (it reads the v1 module tree); emission only in the twin. Tests move with the rules: the three rule suites live under test/v2 and turn the flag on, each pinning that strict emits nothing flag-off; the undefined-exports regression item enables missing_reference explicitly (main demoted it to off in default); the v1-parity flag-off code set gains the two project-file codes. check_v1_parity.sh: layer_diagnostics.jl moves from the gate-lines-only list to additions-only (the shared option validation). Docs: v2-only annotations on all four rules; the duplicated environment_errors testitem from the main merge gets a distinct (v2) name. Full suite: 1505 passed, 1 skipped (pre-existing), 0 failed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ew-plan Port Aqua.jl's static checks as four lint rules
Verbatim re-copy of the JuliaSyntax/ and JuliaLowering/ subdirectories of JuliaLang/julia at 7c019557415a97d0bb60d11c42db5405e5203416 (2026-09-12), replacing the previous pin b657e6a0 (2026-08-11). The two module entry files are unchanged upstream, so src/v2/vendor_lowering.jl keeps its structure and only its SHA comment moves; every API layer_lowering.jl consumes (rebase_layers, expand_forms_1/2, resolve_scopes, LoweringError, BindingInfo) and all six pinned diagnostic messages are unchanged. Gates per packages/VENDOR_JuliaLowering.md: test_lowering_layer.jl and test_lowering_differential.jl testitems, all of test/v2, and the full suite pass on Julia 1.12.7 and 1.13.0 with no JW-side adaptation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
src/lint_rules.jl reverts to byte-identical with main; everything the v2 stack emits is registered in its own complete registry, LINT_RULES_V2 in src/v2/bridge/lint_rules_v2.jl (bridge/, because the shared rules name StaticLint codes). Which registry drives config validation, presets and emission is decided by input_v2_enabled: the three lint-config queries in layer_diagnostics.jl gain gate lines to _v2 twins in src/v2/bridge/lint_config_v2.jl, which also owns materialize_v2 and the option validation for the v2-only rules, so the v1 validator matches main again. layer_diagnostics.jl moves from the additive to the gated class in check_v1_parity.sh, and lint_rules.jl to byte-identical. Intended behavior change: with v2 off, a JuliaLint.toml naming a v2-only rule (e.g. missing_compat) is reported as an unknown rule, exactly as on main; with v2 on, nothing changes. The registries agree on every shared rule today - pinned by a guard test - and the split is the seam for a new v2 rule-registration API that may diverge from the frozen v1 list. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…v1-v2 Split the lint-rule registry: v1 frozen at main, v2 fully independent
…ulialowering-vendor Refresh vendored JuliaSyntax v2 + JuliaLowering to julia master 7c019557
…ode!)
The DynamicMode was a constructor-only argument: changing it meant tearing
down the whole JuliaWorkspace. It is now a regular Salsa input
(input_dynamic_mode) with a public setter, and the set of running DJPs
adjusts immediately on every switch, so hosts can wire it to a live config
value (the LS's julia.enableDynamicIndexing) without a restart.
Following the set_v2_enabled!/set_max_alive_djps! pattern, the mode is
mirrored into the reactor as a RefValue (DynamicFeature.djp_mode) via a new
SetDynamicModeMsg, posted before the reconcile so the reconcile already runs
under the new rules. The transition handler enforces the new mode's child
lifecycle on existing state:
- down to DynamicOff: kill every child and settle all outstanding work the
way the Off branches would have (environments best-effort ready, work
needing a child skipped), keeping `done` so succeeded artifacts are not
overwritten;
- up from DynamicOff: forget the Off-parked completion and failure
bookkeeping wholesale (the retry_failed_dynamic_projects! license) and
un-settle host-side readiness, so the re-dispatched work re-opens
is_ready/wait_until_ready until it settles again;
- Persistent -> IndexingOnly: kill settled children and fail queued
expansion batches; in-flight children settle under the new mode;
- IndexingOnly -> Persistent: flip only — children settling from now on
stay alive, departed ones are revived on demand.
Off no longer means "no DynamicFeature": the feature and its reactor now
always exist (the symbolcache_download path already ran an Off-mode reactor),
which drops the Union{Nothing,DynamicFeature} guards throughout and gives
`is_ready` a single meaning; wait_until_ready sends the first reconcile
itself so a never-mutated workspace cannot block forever. The DynamicMode
enum moves to dynamic_messages.jl so the new message's field can reference
it at include time.
docs/design/2026-09-12-dynamic-config-audit.md classifies the remaining
constructor kwargs (max_concurrent_djps and resolve_workspace_environments
are the next candidates) and sketches the LS wiring that this unblocks.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…core
The TomlSyntax pattern applied to Markdown, minus the token lexer: block
structure is line-oriented, so a line classifier feeds the scanner, which
emits directly into JuliaSyntax's flat postorder green-node buffer and the
shared cursors and trees work unchanged. Deliberately block-level only --
fenced code blocks (backtick and tilde, CommonMark rules), indented code
blocks (so fence markers inside them are not fences), front matter and ATX
headings; prose is opaque trivia and container blocks are not modelled.
On top of the tree, the two products the workspace needs: julia_chunks
classifies the fences whose info string marks them as Julia (plain julia
fences, jmd/Quarto/Weave {julia ...} chunk headers, and Documenter's
plain-Julia @example/@setup/@repl/@eval blocks; doctests are REPL transcripts
and excluded), and julia_shadow_source renders the document as
byte-offset-preserving Julia source: chunk bytes verbatim, every other
non-EOL byte a space.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The pre-JuliaWorkspaces language server's parse_jmd design, rebuilt on
MarkdownSyntax instead of regexes: derived_julia_source_view (the new
layer_markdown.jl) serves a markdown document as byte-offset-preserving Julia
source, so every parser -- the fused JuliaSyntax parse, the legacy CSTParser
tree, and the v2 walk -- reads real Julia at real document offsets, and no
feature grows a position-mapping layer. Slice-and-scan sites that consume
parse-derived ranges (testitem code slices, v2 docstring/signature slices,
code-action insertion scans, macro expansion text) read the view too, so a
range that leaks across a fence boundary can only ever pick up blanked prose.
Gating: derived_julia_files admits markdown documents unconditionally (a
chunk-less file's view is all whitespace, which parses to an empty file
cheaply, and unconditional admission keeps the root set stable under edits).
The public.jl entry points gate file-level queries on the widened
_is_julia_analysis_uri and position-taking queries on the position sitting
inside a Julia chunk, so completions and hovers never fire in prose.
Formatting stays strictly Julia, and include("x.md") still never joins the
includer's tree. Two Salsa wins fall out: the chunk table and the view are
value-stable, so a prose keystroke in a README never reaches any Julia
analysis.
Test items inside markdown fences are detected with package-scoped ids and
run as-is by TestItemControllers, whose test process evals the code slice
against the real file path with a line/column offset -- extension-agnostic,
verified end to end (error stack frames point at the markdown line).
LanguageServer.jl needs no change: its document selector, file watchers and
per-language formatting registration already cover md/jmd, and its full test
suite passes against this JuliaWorkspaces.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ia-files Markdown support: analyze Julia code in md/jmd documents through a MarkdownSyntax-based Julia view
…e-api Make the dynamic mode a runtime-switchable Salsa input
Mechanical preparation for runtime setters: download_enabled, upstream_url, max_concurrent_djps, max_failure_attempts and djp_request_timeout_seconds become reactor-owned Base.RefValues like djp_mode/max_alive_djps/ v2_lifecycle, with every read site dereferencing. No behavior change; the Set*Msg messages the field comments reference land in the next commit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
set_max_concurrent_djps!, set_resolve_workspace_environments!, set_symbolcache! (download + upstream in one call/message), set_max_failure_attempts! and set_djp_request_timeout!, following the established reactor-message pattern; all new handlers carry the SetV2LifecycleMsg-style idempotence guard. Semantics worth noting: - Enabling env resolution or making downloads newly effective un-settles host readiness and forces a reconcile through (the set_dynamic_mode! Off->on triple), so is_ready/wait_until_ready genuinely track the re-dispatched work. - The symbolcache flip clears only `done` watch-environment keys — the one kind whose prep downloads — so a policy change re-preps open projects without respawning scratch/test children; failure bookkeeping is kept in all cases (retry_failed_dynamic_projects! stays the lever). - symbolcache values get eager Salsa mirrors (host-readable old values; the reactor Refs lag behind queued messages); the Int knobs need none. Also forwards resolve_workspace_environments through workspace_from_folders, the one constructor kwarg it was missing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Constructor docstring bullets gain their "changeable at runtime" cross references, functions.md lists the new setters, architecture.md states that every remaining runtime-relevant knob is now switchable, and the config audit records the decisions that differ from its original sketch (single symbolcache setter/message, selective watch-env re-prep instead of the wholesale reset, the readiness un-settle pairing, handler idempotence guards) plus the simplified LS wiring. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Reactor-level: cap raise drains the queue (incl. 0 = unlimited), cap lower gates admission only, symbolcache enable re-preps done watch-env keys while keeping scratch/test completions and failures, upstream changes re-prep only while downloads are on, failure-budget lowering exhausts and raising un-exhausts an identity with failed_projects keys staying barred, timeout Ref round-trip. Workspace-level: env-resolution toggle re-dispatches fabricated keys and un-settles readiness deterministically, the nothing-to-fabricate enable still settles wait_until_ready, set_symbolcache! re-preps a real on-disc project offline via an unreachable upstream, and workspace_from_folders forwards resolve_workspace_environments. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Runtime setters for the remaining changeable dynamic config options
Conflicts were the kwarg/constructor unions: JuliaWorkspace, workspace_from_folders and DynamicFeature keep bodytree's max_alive_djps / v2_lifecycle / launcher / always-on reactor and gain main's status_callback and err_handler, threaded through to the always-constructed DynamicFeature. The reactor loop reports a status snapshot after every message, so bodytree's expansion and v2-lifecycle messages feed the new seam with no further wiring. One auto-merged hunk was a semantic conflict: main's language gate on derived_testitems (#323) is _is_julia_uri, which on this branch would blank test items in markdown fences (markdown documents are Julia analysis sources here, layer_markdown.jl). The gate lands as _is_julia_analysis_uri — the same swap this branch made in derived_diagnostics — keeping main's fix (TOML/plain text stays out of the fused parse) and the markdown feature. The new language-gating tests are adapted the same way: the markdown file appears in the whole-workspace query with empty details rather than not at all. The v2 twins pick up main's fixes in follow-up commits: the language gate on derived_testitems_v2, the workspace-based test-env decision in derived_required_dynamic_projects_v2, and the 32-bit skip-range widening. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two fixes main made to the v1 query (#323, and the RawTestItemDetail widening) had the same latent bugs in the twin: derived_testitems_v2 had no language gate, so a TOML or plain-text URI handed to the per-file query went through the v2 parse (derived_v2_file_walk reads the source view of any uri) and could report items the whole-workspace query never does; and the join built option_skip from `incl`, whose UnitRange{Int} is Int32 on a 32-bit build, where the Union-typed field has no convert and construction throws. The gate is _is_julia_analysis_uri, matching the v1 query on this branch (markdown documents are Julia analysis sources), which also keeps the v1/v2 differential suite aligned. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Mirror of main's #320 in derived_required_dynamic_projects_v2: the isfile probe was untracked (creating test/runtests.jl later never invalidated the query), ignored a walk scope that deliberately excluded test/, and denied in-memory workspaces a test environment. The other half of #320, _covering_test_env_key, is shared v1 code this twin already calls, so it arrived with the merge. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Main's dynamic_status_snapshot reads df.max_concurrent_djps as the plain Int it is there; this branch made the knob a RefValue (set_max_concurrent_djps!), so the merged snapshot constructor threw a convert MethodError as soon as a status_callback was attached. The auto-merge could not see this: the field read lives in main-only code, the Ref conversion in bodytree-only commits. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
packages/JuliaSyntax and packages/JuliaLowering (the v2 stack's vendors) come from JuliaLang/julia subdirectories, so the release-based reconciliation in update_vendored_packages.jl cannot manage them; VENDOR_JuliaLowering.md does. The script ignores unlisted trees, so this is documentation, not behavior. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
davidanthoff
changed the title
davidanthoff
marked this pull request as ready for review
September 14, 2026 22:06
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR introduces a complete new static-analysis stack for JuliaWorkspaces, built on vendored copies of JuliaSyntax 2.0 and JuliaLowering instead of CSTParser + StaticLint. The entire stack is inert by default: it hangs off a single feature flag (
input_v2_enabled, toggled viaset_v2_enabled!), and with the flag off the package runs the legacy code paths unchanged. The goal is to let the two stacks coexist in one module while the v2 stack matures, and eventually to retire the v1 stack.The core idea: BodyTree
The heart of the new stack is
BodyTree, a position-free, trivia-free value type that every v2 analysis layer is written against. Because trees carry no positions, most edits produce values that areisequalto the old ones, which lets Salsa's early-cutoff ("backdating") stop invalidation from propagating — a whitespace edit no longer re-runs downstream analysis. Item identity is content-addressed, and positions are reconstructed only at the last mile when diagnostics are emitted. A consequence worth knowing: we no longer cache syntax trees at all — re-parsing turned out to be cheaper than the invalidation churn of cached, position-carrying trees (the design doc has measurements).What's in the stack
Everything lives under
src/v2/and is layered: an inventory walker (skeleton + body forest), a module tree across include edges, a tree-only visibility layer, a JuliaLowering-based binding analysis per item, and the lint rules that consume it. The core is CSTParser/StaticLint-free by construction (enforced by a guard test); the few pieces that must honour a v1 contract live insrc/v2/bridge/.On top of that substrate the branch delivers:
TomlSyntax(a TOML parser, validated against the vendored toml-test suite) andMarkdownSyntax(a block-level markdown parser that lets markdown documents feed Julia code into analysis).[workspace]/[sources]/extension support, environment selection, and a plain-data "environment seam" that gives the tree-based rules access to indexed environment information (exported names, method arities, member kinds).DynamicJuliaProcesschild that indexed the file's environment (it already has the packages loaded), and the expansions are spliced back into analysis. This comes with lifecycle work — an LRU cap on live children, revive-on-demand, and post-index teardown — gated so v1 behaviour is untouched.missing_reference,unresolved_import,incorrect_call_args' arity arm,type_piracy,kw_default_mismatch,construles,literal_use, and more), adds new rules that were previously infeasible (lowering_errors, a staticsoft_scope_ambiguity), and includes four Aqua.jl-derived rules that are v2-only. A large portion of the commit history is false-positive elimination from differential sweeps over real-world corpora.@testitemdiscovery no longer needs its own parse.How the two stacks coexist
One convention governs every fork point: a v1 query that has a v2 counterpart keeps its name and body byte-identical to
main, with a single inserted gate line (input_v2_enabled(rt) && return <name>_v2(...)), and the counterpart lives insrc/v2/. The lint-rule registry is split so v1 stays frozen atmainwhile v2 evolves independently.scripts/check_v1_parity.shandtest/test_v1_parity.jlenforce this property mechanically, so reviewing the v1 side of the diff reduces to checking a small allowlist of one-line gates.What v1 still does that v2 doesn't
The remaining gap is essentially everything that needs type-level information from the indexed environment — the seam deliberately ships only name- and arity-level data so far:
incorrect_call_args("at argument i: expected T"), and the typed arm ofincorrect_iter_spec. The arity-level halves of these rules are taken over; only the parts that need method signatures with types remain.using Basevia colon-list members, and names that modelled macros declare, bind as:unknown— deliberately silent (no false missing-reference findings), but hover/goto don't see through them.index_from_lengthis deferred on measurement — it produced exactly one finding on the corpus, not worth the machinery yet.So v2 today is a complete spine — identity, module structure, per-item binding semantics, a name-level environment edge — with the type-level half of the environment (method tables with types, store docs) as the main remaining frontier.
docs/design/v2-portability-survey.mdhas the rule-by-rule accounting.Where to start reading
docs/design/v2-architecture.mdis the full design document (layer by layer, with the reasoning and measurements);docs/design/v2-portability-survey.mdtracks which StaticLint rules have been ported and which remain. The vendored packages are underpackages/, with vendoring notes alongside.Status
The flag is off by default, so merging this changes nothing for users. With the flag on, the gaps in the section above are what still silently falls back to v1 behaviour or stays unavailable.