perf(gfql): index AUTO preserves polars frames (rework of #1767, stacked on #1743)

[lmeyerov]

DO NOT MERGE BEFORE #1743

Stacked on #1743 (fix/gfql-auto-engine-polars-native) and contains its commits until it lands (plus a sync merge of current master to pick up #1841's usable/reason columns). Draft against master so CI runs; once #1743 merges, this rebases/retargets to a clean index-only diff. Do not land this first — the whole point of the rework is the ordering.

What this is: the retracted #1767, made safe by sequencing

The retracted #1767 made gfql_index_all()/create_index() under engine='auto' preserve resident polars frames (index in place, no pandas coerce-and-replace). It was retracted because query-side AUTO still routed polars-frame graphs to pandas, so the AUTO-built polars index met an AUTO(pandas) query and every hop declined to the scan floor — a measured 125–534x DEFAULT-path regression (250K/1M/4M edges, dgx-spark, receipts in the #1767 thread).

#1743 removes the mismatch at its root: g.gfql(...) with no engine on an all-polars-frame graph now routes to the native polars engine. With that underneath, the index-side preservation inverts from the cliff into the win:

	index built (AUTO)	frames	AUTO gfql resolves	index serves?
master today	pandas	polars coerced to pandas copies	pandas	yes (but frames were swapped)
old #1767 alone	polars	preserved	pandas	no — 125–534x cliff
#1743 + this	polars	preserved	polars	yes

The inversion pin

test_inversion_auto_index_auto_gfql_serves_polars_index: gfql_index_all() + g.gfql(<index-served query>) with no engine argument anywhere — the exact two-default-spellings scenario that regressed — must show path=index, engine=polars in index_trace(), answer in polars frames, and value-match BOTH explicit-engine spellings (polars/polars and pandas/pandas oracles).

The #1841 flip

show_indexes() under AUTO now reports the routed truth: the "polars index + AUTO query" combination that #1841 was built to call out as usable=False flips to usable=True (test_polars_index_auto_query_usable_the_1841_flip, plus the all-default spelling in test_auto_built_index_reports_usable_under_auto). Explicit mismatched-engine previews (show_indexes(engine='pandas') against a polars index) keep #1838's decline wording, pinned per kind.

Change surface (index-only; no routing — that is #1743's job, underneath)

• graphistry/compute/gfql/index/api.py: new resolve_index_engine() — AUTO + both frames eager polars → Engine.POLARS; wired into create_index, _is_resident_index_valid, show_indexes. Explicit engines untouched.
• graphistry/compute/gfql/index/wire.py: the DDL CREATE INDEX NODE_PROP reuse check uses the same resolution (otherwise IF-valid reuse would mismatch the build it just did).

Ported from old #1767: the _resolve_index_engine mechanism (now also covering show_indexes/wire, which the old diff predated — #1841/#1838 didn't exist then), the frame-identity preservation contract, and the whole negative-pin test suite (LazyFrame, mixed-frame, nodes-only, edges-only, pandas-stays-pandas, cudf-stays-cudf, explicit-engine-still-coerces, index-service spy with pandas-oracle parity).
Dropped from old #1767: its 276x standalone-perf framing (the thread showed the explicit-engine fast path was already reachable on master; the honest claim here is default-path consistency), and its CHANGELOG entry (rewritten as behavior-only with the landing-order dependency stated).

Gate deliberately narrower than #1743's query gate, self-consistent because create_index coerces the frames it indexes (later AUTO queries route with the post-build frames): LazyFrame graphs keep the legacy pandas build (an index cannot gather rows from a lazy plan); edges-only graphs too — materialize_nodes() does not yet produce polars nodes from polars edges, a pre-existing gap also reachable via explicit engine='polars' on such a graph (probed empirically; guessing polars there would crash the build).

Known interaction left to #1743's review: on cudf-frame graphs, #1743's polars-gpu arm can route an AUTO query away from a resident cudf index (GPU lanes only); index resolution here deliberately leaves cuDF source-native and takes no position on that arm.

Tests (CPU lane, this box: cudf/gpu params env-excluded — no GPU libs)

• index suite graphistry/tests/compute/gfql/index/: 229 passed (83 deselected cudf/gpu)
• test_index.py alone: 142 passed; new/changed: inversion pin, fix(gfql): show_indexes reports engine usability, not just fingerprint validity #1841 flip x2, DDL SHOW GFQL INDEXES flip, per-kind reason under explicit mismatch, + the ported TestIndexAutoPreservesPolarsFrames (11 tests)
• polars cypher conformance + cache coverage lock: 177 passed, 3 skipped
• bin/lint.sh clean · scoped mypy (api.py, wire.py) clean · type-hygiene guard OK (no growth)
• Pre-existing env failures only: 2 cudf tests fail on this GPU-less box with libnvrtc OSError (master's own test_cudf_index_pandas_query_not_usable fails identically here); they are GPU-lane tests.

🤖 Generated with Claude Code

https://claude.ai/code/session_011AB4RZpph3uSFUpzKnZJcr

[lmeyerov]

Folded into #1873 and closing: the two halves are atomically coupled — this PR's index-gate agreement (and its flipped pins) need modern AUTO to exist, and #1873's resolver change leaves the old index gate red without this rework. Neither is green standalone at any merge point, so they land as one reviewable unit. All review items from this thread are addressed in the folded commits (715575b, c8c1325, e05d3cd, 3377041): typed DataFrameT, comment diet, resolve_index_engine now derives from frames instead of post-correcting pandas on a polars path, and cross-engine pos/neg agreement tests.