Repository navigation
Open
Labels
agent:triagedarch-auditFiled by a scheduled architecture audit routine (see the architecture review discussion)Filed by a scheduled architecture audit routine (see the architecture review discussion)priority:p1refactorStructural change: duplicated code or logic, missing abstraction, layering, dead codeStructural change: duplicated code or logic, missing abstraction, layering, dead code
Description
[agent] Filed by the scheduled architecture audit (audit-core). Register: register comment.
Kind: refactor. Register ID: A41.
Problem
crawlers/npm_crawler.rs decides which child of a node_modules is a package-manager store (pnpm-shaped .pnpm/.store, vlt .vlt, npm-linked .store, .modules.yaml, legacy pnpm store names) in three hand-written copies:
- the
find_by_purlsresolver,nested_node_modules_of: L3166-L3225 - the scan,
gather_node_modules: L3576-L3624 - the peer-variant finder,
find_store_peer_variant_copies_reusing: L4478-L4520
The copies have drifted:
- Store entries. Every arm of the scan is gated on
!store_entry, so a store entry's ownnode_modules/.pnpm(or.vlt,.store,.modules.yaml, a legacy name) is never treated as a store. The resolver has nostore_entryparameter and walks such a child as a store.applycan therefore resolve copies thatscannever reports. This is unexecuted and an edge case. - Missing kinds. The peer finder handles neither legacy pnpm stores nor the pnpm global virtual store.
- Empty
.modules.yamlarm. The resolver's arm pushes an empty entry list (L3207), andvisit_resolver_dirthen has to guard against it (L3037).
The same file also has:
- four shapes for one store entry:
StoreEntryDir,StoreEntry,(String, PathBuf)and(OsString, PathBuf). They are converted at L1751-L1788, L3673, L3692, L4087 and L4131, andlist_npm_store_entriesdecodes each name, discards the result, and decodes it again (L4243, then L4619); - about 7 copies of the "package-dir child" filter (not hidden, not
node_modules, a real dir): L3082, L3226-L3241, L3508, L3772, L4153, L4200, L600. They drift on whetheris_dirfollows symlinks (L4077 vs L4160/L4226) and on whether a scoped child namednode_modulesis excluded (L3238); - 5 copies of the
@scope/nameexpansion: L3088-L3102, L3234-L3245, L3645/L3762, L4213-L4222, L4719.
Proposed change
- Add one
StoreKindenum with aclassify(name, file_type, in_store_entry) -> Option<StoreKind>and use it in all three walks. The resolver passesin_store_entrythe way the scan does. - Keep one store-entry type (
StoreEntryDir), drop the tuple shapes, and decode each name once. - Add one
package_children(dir, listing)iterator that yields unscoped and@scope/namepackage dirs, and use it for the filter and expansion copies.
The peer-variant section (L4391-L4791) can then move to crawlers/npm_crawler/store.rs.
Size: 1 file (plus a new submodule), about −300/+150 lines.
Acceptance criteria
crawl_all,find_by_purlsandfind_store_peer_variant_copiesreturn the same results on every existing test innpm_crawler.rsandtests/crawler_npm_e2e.rs.- A new test pins the store-entry rule: a store entry holding a nested
node_modules/.pnpmis neither reported bycrawl_allnor resolved byfind_by_purls. Alternatively, if both are meant to walk it, the test asserts that both do.
Metadata
Metadata
Assignees
Labels
agent:triagedarch-auditFiled by a scheduled architecture audit routine (see the architecture review discussion)Filed by a scheduled architecture audit routine (see the architecture review discussion)priority:p1refactorStructural change: duplicated code or logic, missing abstraction, layering, dead codeStructural change: duplicated code or logic, missing abstraction, layering, dead code
Activity
audit-core: crawlers maven rest..npm, A40-A45, file #1899-#1902
mikolalysenko commentedon Nov 1, 2026
[agent] Triage: priority:p1 (npm crawler). No duplicate or open PR found. This is a large single-file refactor (about -300/+150), so it's a candidate to split by sub-cause (StoreKind classify, store-entry type, package_children iterator).
Generated by Claude Code
mikolalysenko commentedon Nov 2, 2026
[agent] Claiming this issue (shared root cause: npm_crawler classifies node_modules store children in hand-written copies that drifted). First slice: one StoreKind classifier used by the resolver and the scan. Branch: agent/fix-npm-store-kind-classify. Claim-ID: 2026-11-02T12:36:10Z-a7faf4
Generated by Claude Code
mikolalysenko commentedon Nov 2, 2026
[agent] Draft PR for the first slice (one StoreKind classifier shared by scan and resolver, plus the store-entry rule test): #1986. The peer-finder, store-entry type and package_children slices remain open here.
Generated by Claude Code
Classify npm store children through one helper
Skip nested stores in apply's peer fan-out too
Skip nested stores of every entry shape
Keep the peer fan-out's store when a store name sits above it
mikolalysenko commentedon Nov 2, 2026
[agent] First slice merged in #1986. Scan, the apply/vex resolver and the apply peer fan-out now share one
StoreKind::classifyand its store-entry rule: a store nested in a store entry is never walked. Regression tests cover every store name and every store-entry shape.Still open here (I'm releasing the claim so a later run can take them):
.registry.*stores or the pnpm global virtual store;StoreEntryDir) instead of four shapes, decoding each name once;package_childreniterator for the ~7 package-dir filters and the 5@scope/nameexpansions.Generated by Claude Code