Skip to content

Call IsDirCoveredByWatch less often to improve performance#4582

Open
terite wants to merge 1 commit into
microsoft:mainfrom
terite:optimize-computeDesiredWatches
Open

Call IsDirCoveredByWatch less often to improve performance#4582
terite wants to merge 1 commit into
microsoft:mainfrom
terite:optimize-computeDesiredWatches

Conversation

@terite

@terite terite commented Jul 10, 2026

Copy link
Copy Markdown

I'm experiencing a significant performance penalty when using tsc --watch. I'm seeing tsc complete in ~2 seconds while tsc --watch will hang for ~15 seconds after reporting the first build status.

Profiling shows a large amount of time being spent doing string manipulation as a result of calls to watchmanager.IsDirCoveredByWatch. I noticed a couple of pieces of low hanging optimization fruit.

  1. watchmanager.IsDirCoveredByWatch was being called once for each path in seenFilePaths, even if two paths were in the same directory. For my project seenFilePaths had a length of around 40k files, but only 2k directories. This is where I was the most improvement for my project, reducing time spent in this part of the code from ~15s to ~200ms.
  2. The resolvedDirs map was (potentially) having new directories added to it each iteration of the loop. Since IsDirCoveredByWatch iterates over that map to check if a directory is covered, this led to some accidentally quadratic time complexity. Since all directories being added to it are not recursive, they don't affect behavior and can be added after doing all the checks. This improved time spent in this part of the code from ~200ms to ~20ms.

I can't easily share the project I'm testing with, so I also checked the performance difference when checking https://github.com/grafana/grafana

Before, using @typescript/native-preview@7.0.0-dev.20260707.2 (~30 seconds)

$ tsgo --version
Version 7.0.0-dev.20260707.2

$ tsgo --noemit --watch --preserveWatchOutput
[12:07:01 AM] Starting compilation in watch mode...

[12:07:31 AM] Found 0 errors. Watching for file changes.

After, with this PR (~1 second):

$ ~/code/typescript-go/built/local/tsgo --noemit --watch --preserveWatchOutput
[12:06:55 AM] Starting compilation in watch mode...

[12:06:56 AM] Found 0 errors. Watching for file changes.

AI Assistance: I used Github Coilot Completions, and asked Claude Code to give the commit + PR description a sanity check.

Copilot AI review requested due to automatic review settings July 10, 2026 08:07

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Optimizes watch-mode directory watch computation in internal/execute by reducing the number of expensive watchmanager.IsDirCoveredByWatch checks and avoiding repeatedly growing the directory map during iteration.

Changes:

  • Deduplicate seenFilePaths by directory so coverage checks run once per directory (not once per file).
  • Defer adding newly discovered non-recursive watch directories until after coverage checks to avoid accidental quadratic behavior from iterating an ever-growing map.

Comment on lines +188 to +192
toAddDirs := make(map[string]bool) // dir → should add
for _, filePath := range seenFilePaths {
dir := tspath.GetDirectoryPath(filePath)
if !watchmanager.IsDirCoveredByWatch(resolvedDirs, dir, opts) {
if watchmanager.CanWatchDirectory(dir) {
resolvedDirs[dir] = false
}
if _, has := toAddDirs[dir]; !has {
toAddDirs[dir] = !watchmanager.IsDirCoveredByWatch(resolvedDirs, dir, opts) && watchmanager.CanWatchDirectory(dir)
@AkisArou

Copy link
Copy Markdown

I tested this PR with tsgo --build --watch in a private monorepo containing
87 project configs, 36,528 physical directories, and 5,161 desired directory
watches. This PR still had not installed the watches after five minutes;
the original implementation eventually finished after approximately 13–14
minutes.

This PR optimizes internal/execute/watcher.go, but build mode computes
watches separately in internal/execute/build/orchestrator.go. That path
repeatedly calls IsDirCoveredByWatch for source files, build-info
dependencies, and package.json ancestors.

I prepared a proof-of-concept branch that replaces repeated full-map scans
with a canonical directory index and watch
initialization in the same repository becomes effectively immediate:

https://github.com/AkisArou/typescript-go/tree/fix/watch-directory-coverage

Commit: AkisArou@82eee7b4

The main changes are:

  • internal/execute/watchmanager/watchmanager.go: adds an indexed
    DirectorySet.

  • internal/execute/build/orchestrator.go: uses the index for solution-build
    watch discovery.

  • internal/execute/watcher.go: uses the same index for regular watch
    discovery.

  • internal/execute/watchmanager/watchmanager_test.go: adds case-sensitive/
    case-insensitive tests and benchmarks.

The core lookup walks only the candidate’s ancestors:

func (s *DirectorySet) IsCovered(dir string) bool {                          
    key := s.key(dir)                                                        
    if _, ok := s.canonical[key]; ok {                                       
        return true                                                          
    }                                                                        
    for parent := tspath.Path(tspath.GetDirectoryPath(string(key)));         
rent != "" && parent != key; parent =                                        
path.Path(tspath.GetDirectoryPath(string(parent))) {                         
        if existing, ok := s.canonical[parent]; ok && existing.recursive {   
            return true                                                      
        }                                                                    
        key = parent                                                         
    }                                                                        
    return false                                                             
}                                                                            

This makes exact and recursive-ancestor checks O(path depth) in both regular
and build watch modes, including case-insensitive filesystems, effectively immediate.

A 5,000-directory comparison measured approximately 2.75 ms for the indexed
implementation versus 16.65 seconds for the previous scan—about a 6,000×
improvement.

I’m sharing this branch as additional data for the build-mode case and for
comparison with this approach. I’m happy to coordinate on the overlapping
changes and follow whatever direction the maintainers prefer.

@terite

terite commented Jul 10, 2026

Copy link
Copy Markdown
Author

This PR only changes the code path used by tsgo --watch. I would not expect it to change the behavior of tsgo --build --watch at all for the reason you mentioned: it's a separate code path.

I chose not to modify the --build code path for two reasons:

  • The watch mode feature for has the status "prototype" which is defined to mean "proof-of-concept only; do not log bugs". I'm not sure how well an optimization to a prototype feature will be received by the team, so I made my change as small and straightforward as I could.
  • I do not use --build mode (or project references) in my project, so I would not be able to easily check if my changes were correct.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants