Repository navigation
fix: prune excluded directories during Todo file discovery - #264
Merged
Merged
Conversation
Scan each workspace root once for all include globs by calling fast-glob directly. globby 8 started a separate walk per pattern, so the default 18 include globs read every directory 18 times. Stop doubling exclude globs that already end in /** into /**/**, and match excludes against workspace-relative paths so folders above the root (for example a workspace inside an out or build folder) no longer hide every file. Replace the globby dependency with fast-glob 2, which globby already bundled.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Todo file discovery was much slower than in 5.2.0. On a workspace with about 35k files (default
todo.file.include/todo.file.excludeand VS Code's defaultfiles.exclude),discoverFilestook 8.2–8.5 s. With this change it takes 0.40–0.44 s. The 5.2.0 approach, globby with the raw exclude list, took about 3.4 s on the same tree.This affects dependency diagnostics, the Files view, and toggling tasks that have
@depends.What was slow
generateGlobTasksmakes one fast-glob task per positive pattern, so the 18 default include globs read every directory 18 times (8,316readdircalls for 462 directories). 5.2.0 had this too.expandDirectoryExcludeGlobswas added in Fix archive safety, asynchronous state, and codebase consistency #234 / 9bd73ef so that VS Code-style workspace excludes like**/.gitalso cover their contents (**/.git/**). It also turned every**/x/**default into**/x/**/**. That doesn't change what matches, but it doubles the patterns fast-glob tests every entry against, in each of the 18 walks. That's why 5.3.0 was about 2× slower than 5.2.0.Changes
discoverFilescalls fast-glob directly with every pattern at once. fast-glob groups patterns by base directory, so the default config needs a single walk (457readdircalls).expandDirectoryExcludeGlobsno longer adds/**to globs that already end in/**. Globs that don't end in/**, such as**/.git, still get both forms, so their contents stay excluded.absolute: false). Withabsolute: true, fast-glob also checked excludes against the absolute path, so a workspace located anywhere under a folder namedout,build,dist,release,target,vendor, … found no Todo files at all. The new test covers this.fastGlob.generateTasksinstead ofglobby.hasMagic.isFileIncluded/isAllowedFilePathpost-filter is unchanged.Dependencies
globbyis replaced byfast-glob@^2.2.7, the exact version globby 8 already bundled. globby was used only infile-discovery.ts. The bundle doesn't gain any code:globby,dir-glob,array-union,array-uniq,arrify,ignore,path-typeandslashdrop out of the lockfile. globby 8 has no API that runs several patterns in one walk, so calling fast-glob directly is the only way to get one walk without writing our own walker.Correctness check
On the same workspace I compared old and new
discoverFileswith a mixed include list (**/*.md,**/*.json,src/**/*.ts,!**/*.test.ts,**/TODO*) and a workspace exclude with a dynamic basename (**/gen*). Both returned the same 727 files.Tests
walks each directory once and never descends into excluded directories. It spies onfs.readdirand asserts that no subdirectory ofnode_modules,buildor afiles.exclude'd.gitis read, and that no directory is read twice.does not apply excludes to folders above the workspace root.developand pass here.npm run checkpasses (441 tests).