Make browse/show/apply fast by indexing where each mutant lives - #543
Merged
Conversation
`browse`, `show` and `apply` parsed the whole mutated file to find one function. Mutated files are orders of magnitude bigger than the source they come from, so for a 61 KB source (2.9 MB mutated) that was 2.7s and 2.2 GB of RSS per diff, and `browse` did it on every keystroke. Mutant generation now records the lines each generated function occupies in `mutants/<path>.spans`, so reading a mutant only parses those lines. This is free at generation time: rendering the module statement by statement costs the same as rendering it in one go, and gives the line counts we need. Also make the browser load one diff at a time and drop loads for mutants that are no longer selected, instead of spawning a thread per keystroke and letting them pile up. Falls back to parsing the file when there is no index, so mutants directories from older versions keep working. Fixes #538
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #538.
browse,showandapplyparsed the entire mutated file just to pull out one function. Mutated files are orders of magnitude larger than the source they were generated from, andbrowsedid this on every keystroke, spawning a thread per selection with nothing to stop them piling up.What changed
Mutant generation records a line span index.
combine_mutations_to_sourcenow renders the mutated module statement by statement instead of callingmodule.codeonce. That costs the same (measured 0.62s vs 0.63s on a 2.9 MB output) and gives the line count of every statement, so the span of each generated function falls out for free — no second parse and noPositionProviderpass, which was the concern raised in the issue thread. The spans go tomutants/<path>.spans, kept separate from.metabecause.metais rewritten after every mutant result.Reading a mutant parses only its lines. A method's lines are wrapped in a
class _:stub so they parse at their original indentation, then the existing CST rendering is reused, so the output is identical to before by construction rather than by string surgery.The browser loads one diff at a time. The fire-and-forget
Threadis now an exclusive Textual worker plus a lock, so moving on drops loads that have not started yet.Numbers
On a 61 KB source file (2.9 MB mutated, 1322 mutants):
Driving the TUI headlessly over 13 rapid selections: before, 13 loads started with up to 4 running at once, each holding a full CST; after, 5 started and never more than 1 running.
Mutant generation end to end went from 3.89s to 3.96s (~2%). The only extra work is re-rendering class bodies that contain mutants, so files with no classes pay nothing.
Correctness
The fast path has to be byte-identical to parsing the whole file. Verified against the old path on:
WhitespaceInclusivePositionProviderover 45,646 functions in 3,504 files — 0 mismatchesmodule.codefor every input in the test suiteAn absent, incomplete, malformed or stale index falls back to parsing the file, so mutants directories from older versions keep working. New tests in
tests/mutation/test_line_spans.pyplus three intest_mutation.py; I checked they bite by breaking each mechanism in turn.Notes
mutate_file_contents/combine_mutations_to_sourcenow return aMutatedFiledataclass. This absorbs thehash_by_function_namethat feat: Mutation caching and transitive dependency tracking #509 added to the same return value, instead of a growing tuple.# <name>: <status>header print moved out ofget_diff_for_mutantintoshow; it was being printed into the TUI on every keystroke.showoutput is unchanged.create_mutants_for_filere-reads the written file andast.parses it to validate syntax, which on a 20 MB output is ~35s and 2 GB by the reporter's own measurement.🤖 Generated with Claude Code
https://claude.ai/code/session_01NxTfgecAqzBKWCWgeW2hxV