Skip to content

GFQL perf follow-ups from the #1905 trail-correction work (pandas ~2%, polars q8@20k) #1908

Description

@lmeyerov

Carried forward from the #1907 perf investigation (four passes, three board A/B rounds; full accounting in plans/gfql-benchmark-numbers/plan.md). Neither is release-blocking; both are honest debits.

1. pandas @100k carries ~+2% vs the pre-#1905 floor. Pooled per-iteration p50: floor 89.76 / prev-head 90.99 / current 91.75 on q8. This is the trail correction's absolute cost (~2.2 ms probe, _two_hop_trail_illegal_pairs) landing on a fast cell. Blocking it for pandas was measured and is worse (+1.84 ms, significant — per-block call overhead exceeds the pass saved), and five probe formulations cluster within 0.26 ms, so the pass itself is at its floor. The removable route is a soundness proof rather than a faster scan: consult a degree fact for self_loops == 0 on the matching partition ⇒ zero illegal pairs, skipping the scan entirely. Index-dependent, so it cannot fire on the board's --binding cold arm, but it would help indexed users.

2. polars q8 @20k nominally +6.7% over floor (2.39 vs 2.24 ms), p=0.11 — not resolvable at 18 slots/build. Either confirm with more slots or drop the claim; do not quote a 20k q8 delta in either direction as things stand.

3. Harness placement. The per-query harnesses written for this investigation (in-situ variant swap, lane instrumentation, formulation shoot-outs) live in a session scratchpad. Per the standing convention that benchmark CODE lives in pyg-bench and pygraphistry carries only results, they should move to pyg-bench if these measurements are meant to be durable.

Context for anyone picking this up: the residual cost IS the correctness fix — the count is one bandwidth-bound re-read of both endpoint columns (five formulations within 0.126 ms, ~19.4 MB at ~16 GB/s). The only two ways to skip it are DegreeFact.self_loops (O(1), needs an index) and the bounds-proof by-product now used on the polars dense lane.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions