Skip to content

fix: prune excluded directories during Todo file discovery - #264

Merged
vycdev merged 1 commit into
developfrom
fix/prune-excluded-discovery
Oct 24, 2026
Merged

vycdev merged 1 commit into
developfrom
fix/prune-excluded-discovery

Conversation

@vycdev

@vycdev vycdev commented Oct 24, 2026

Copy link
Copy Markdown
Owner

Summary

Todo file discovery was much slower than in 5.2.0. On a workspace with about 35k files (default todo.file.include / todo.file.exclude and VS Code's default files.exclude), discoverFiles took 8.2–8.5 s. With this change it takes 0.40–0.44 s. The 5.2.0 approach, globby with the raw exclude list, took about 3.4 s on the same tree.

This affects dependency diagnostics, the Files view, and toggling tasks that have @depends.

What was slow

  1. globby 8 walks the tree once per include pattern. Its generateGlobTasks makes one fast-glob task per positive pattern, so the 18 default include globs read every directory 18 times (8,316 readdir calls for 462 directories). 5.2.0 had this too.
  2. Doubled exclude globs. expandDirectoryExcludeGlobs was added in Fix archive safety, asynchronous state, and codebase consistency #234 / 9bd73ef so that VS Code-style workspace excludes like **/.git also cover their contents (**/.git/**). It also turned every **/x/** default into **/x/**/**. That doesn't change what matches, but it doubles the patterns fast-glob tests every entry against, in each of the 18 walks. That's why 5.3.0 was about 2× slower than 5.2.0.

Changes

  • discoverFiles calls fast-glob directly with every pattern at once. fast-glob groups patterns by base directory, so the default config needs a single walk (457 readdir calls).
  • expandDirectoryExcludeGlobs no longer adds /** to globs that already end in /**. Globs that don't end in /**, such as **/.git, still get both forms, so their contents stay excluded.
  • Excludes are now matched against workspace-relative paths (absolute: false). With absolute: true, fast-glob also checked excludes against the absolute path, so a workspace located anywhere under a folder named out, build, dist, release, target, vendor, … found no Todo files at all. The new test covers this.
  • The literal-prefix symlink check now takes its base directory from fastGlob.generateTasks instead of globby.hasMagic.
  • The precise isFileIncluded / isAllowedFilePath post-filter is unchanged.

Dependencies

globby is replaced by fast-glob@^2.2.7, the exact version globby 8 already bundled. globby was used only in file-discovery.ts. The bundle doesn't gain any code: globby, dir-glob, array-union, array-uniq, arrify, ignore, path-type and slash drop out of the lockfile. globby 8 has no API that runs several patterns in one walk, so calling fast-glob directly is the only way to get one walk without writing our own walker.

Correctness check

On the same workspace I compared old and new discoverFiles with a mixed include list (**/*.md, **/*.json, src/**/*.ts, !**/*.test.ts, **/TODO*) and a workspace exclude with a dynamic basename (**/gen*). Both returned the same 727 files.

Tests

  • New: walks each directory once and never descends into excluded directories. It spies on fs.readdir and asserts that no subdirectory of node_modules, build or a files.exclude'd .git is read, and that no directory is read twice.
  • New: does not apply excludes to folders above the workspace root.
  • Both new tests fail on develop and pass here. npm run check passes (441 tests).

Scan each workspace root once for all include globs by calling fast-glob
directly. globby 8 started a separate walk per pattern, so the default 18
include globs read every directory 18 times.

Stop doubling exclude globs that already end in /** into /**/**, and match
excludes against workspace-relative paths so folders above the root (for
example a workspace inside an out or build folder) no longer hide every file.

Replace the globby dependency with fast-glob 2, which globby already bundled.
@vycdev
vycdev merged commit bcab828 into develop Oct 24, 2026
4 checks passed
@vycdev vycdev mentioned this pull request Oct 24, 2026
3 of 7 tasks
@vycdev
vycdev deleted the fix/prune-excluded-discovery branch October 24, 2026 17:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant