Skip to content

Make browse/show/apply fast by indexing where each mutant lives - #543

Merged
boxed merged 1 commit into
mainfrom
faster-browse-via-line-span-index
Jul 30, 2026
Merged

Make browse/show/apply fast by indexing where each mutant lives#543
boxed merged 1 commit into
mainfrom
faster-browse-via-line-span-index

Conversation

@boxed

@boxed boxed commented Jul 30, 2026

Copy link
Copy Markdown
Owner

Fixes #538.

browse, show and apply parsed the entire mutated file just to pull out one function. Mutated files are orders of magnitude larger than the source they were generated from, and browse did this on every keystroke, spawning a thread per selection with nothing to stop them piling up.

What changed

Mutant generation records a line span index. combine_mutations_to_source now renders the mutated module statement by statement instead of calling module.code once. That costs the same (measured 0.62s vs 0.63s on a 2.9 MB output) and gives the line count of every statement, so the span of each generated function falls out for free — no second parse and no PositionProvider pass, which was the concern raised in the issue thread. The spans go to mutants/<path>.spans, kept separate from .meta because .meta is rewritten after every mutant result.

Reading a mutant parses only its lines. A method's lines are wrapped in a class _: stub so they parse at their original indentation, then the existing CST rendering is reused, so the output is identical to before by construction rather than by string surgery.

The browser loads one diff at a time. The fire-and-forget Thread is now an exclusive Textual worker plus a lock, so moving on drops loads that have not started yet.

Numbers

On a 61 KB source file (2.9 MB mutated, 1322 mutants):

before after
per diff 2775 ms 4.6 ms
peak RSS 2.19 GB 0.13 GB

Driving the TUI headlessly over 13 rapid selections: before, 13 loads started with up to 4 running at once, each holding a full CST; after, 5 started and never more than 1 running.

Mutant generation end to end went from 3.89s to 3.96s (~2%). The only extra work is re-rendering class bodies that contain mutants, so files with no classes pay nothing.

Correctness

The fast path has to be byte-identical to parsing the whole file. Verified against the old path on:

  • 3189 mutants across all of mutmut's own source plus files covering CRLF, tabs, non-ASCII names, no trailing newline, multi-line signatures, a method containing an unindented multi-line string, async and decorated functions, and nested classes — 0 mismatches
  • 202 mutants across the e2e projects' real mutants directories — 0 mismatches
  • the span arithmetic against libcst's own WhitespaceInclusivePositionProvider over 45,646 functions in 3,504 files — 0 mismatches
  • with a temporary assert in place, the statement-by-statement output equalled module.code for every input in the test suite

An absent, incomplete, malformed or stale index falls back to parsing the file, so mutants directories from older versions keep working. New tests in tests/mutation/test_line_spans.py plus three in test_mutation.py; I checked they bite by breaking each mechanism in turn.

Notes

  • mutate_file_contents / combine_mutations_to_source now return a MutatedFile dataclass. This absorbs the hash_by_function_name that feat: Mutation caching and transitive dependency tracking #509 added to the same return value, instead of a growing tuple.
  • The # <name>: <status> header print moved out of get_diff_for_mutant into show; it was being printed into the TUI on every keystroke. show output is unchanged.
  • Not addressed here: the slow generation also reported in mutmut browse is very slow and uses tons of memory #538. Worth a separate look — create_mutants_for_file re-reads the written file and ast.parses it to validate syntax, which on a 20 MB output is ~35s and 2 GB by the reporter's own measurement.

🤖 Generated with Claude Code

https://claude.ai/code/session_01NxTfgecAqzBKWCWgeW2hxV

`browse`, `show` and `apply` parsed the whole mutated file to find one
function. Mutated files are orders of magnitude bigger than the source they
come from, so for a 61 KB source (2.9 MB mutated) that was 2.7s and 2.2 GB
of RSS per diff, and `browse` did it on every keystroke.

Mutant generation now records the lines each generated function occupies in
`mutants/<path>.spans`, so reading a mutant only parses those lines. This is
free at generation time: rendering the module statement by statement costs
the same as rendering it in one go, and gives the line counts we need.

Also make the browser load one diff at a time and drop loads for mutants
that are no longer selected, instead of spawning a thread per keystroke and
letting them pile up.

Falls back to parsing the file when there is no index, so mutants directories
from older versions keep working.

Fixes #538
@boxed
boxed merged commit 9e4af73 into main Jul 30, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

mutmut browse is very slow and uses tons of memory

1 participant