Fix imports whose letter case does not match the file on disk - #188
Fix imports whose letter case does not match the file on disk#188shellygr wants to merge 3 commits into
Conversation
A project written on a case-insensitive filesystem can import a path that differs from the real filename, usually only in letter case. It compiles for its authors and fails on Linux. Compilation analysis now checks each component of an import path against the actual directory entries and rewrites the quoted literal to the spelling the filesystem holds. This runs before the existing import patcher. That patcher resolves relative imports against a map of real files and skips the ones that miss, so a misspelled import is precisely what it cannot canonicalize. As with the patcher, a rewrite survives if the retry compiles and is reverted if it does not. Nothing is resolved by guessing. Two candidates, a name that matches only by edit distance, or an import that goes through a package prefix all leave the file alone and log the reason. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The import scan accumulates lines until the next semicolon, so a commented-out import without one ran on to whatever semicolon came next and could land the rewrite on an ordinary string constant further down the file. Scanning now runs over a copy of the source with comment bytes replaced by spaces. Each byte is replaced one for one, so columns still address the real file and the rewrite is applied to it unchanged. Comment starts are found by scanning characters in context rather than by matching // or /* directly. A block comment spans lines, so recognising it at all needs state carried between them, and the same pass keeps string literals out of it for free. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
|
||
| - Fuzzy or edit-distance matching. A near-miss name is a different contract, and pointing an | ||
| import at it yields a project that compiles while verifying code nobody asked about. | ||
| - Guessing at name shape: CamelCase versus snake_case, singular versus plural. |
There was a problem hiding this comment.
Why do you even include these "out of scope" here? They are obviously out of scope. Or are you saying that mixing CamelCase and snake_case works just fine on Windows/macOS?
There was a problem hiding this comment.
Perhaps we can rename the class/file to something obviously saying os-specifing/agnostic spelling issues
There was a problem hiding this comment.
Claude answers: Fair. They are not case problems at all, and listing them only made the module look like a general spelling fixer. Dropped. What is left is the three bails you cannot see from the path alone: a remapped prefix, a dependency checkout, and two entries matching one component case-insensitively.
There was a problem hiding this comment.
Claude answers: Done. It is import_case_fix.py now, with ImportCaseRewrite and plan/apply/revert_import_case_fixes. I also dropped the right-basename-in-the-wrong-directory branch, which was the one part that was not an OS issue, so the name now covers everything the module does. No observed run ever needed it; the failure behind this PR is a plain basename case mismatch.
| spelling_fix_applied = self._run_import_spelling_fix( | ||
| output, compilation_config.get("packages") or [] | ||
| ) | ||
| import_patch_applied = self._run_import_patch() | ||
| import_patcher_applied = import_patch_applied |
There was a problem hiding this comment.
Don't run both together? what if spelling fix helps (applies), but import pach fails (and let's suppose we need only the spelling fix).
There was a problem hiding this comment.
Claude answers: Done. The case fix gets its own retry, and the patcher only runs if that retry still fails, on top of the case fix rather than instead of it. That costs a third compile only when the case fix applied and was not enough.
The name promised a general spelling repair, and the "out of scope" list in its docstring mostly ruled out things nobody would expect it to do anyway. The module is now import_case_fix, and it handles what a case-insensitive filesystem actually breaks: letter case in any path component, and Windows separators. The right-basename-in-the-wrong-directory branch is gone. An import that resolves nowhere is an absent file, and no observed run needed that branch. Compilation analysis runs the case fix on its own and recompiles before reaching for the import patcher. The patcher rewrites every relative import in the project, and a project the case fix already repaired should not have to survive that as well. If it is still needed it goes on top of the case fix rather than instead of it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
shellygr
changed the title
2 participants
What this fixes
A Solidity project written on macOS or Windows can import a file by a path that does not match
the real filename, usually differing only in letter case. Those filesystems are case-insensitive,
so the project compiles for its authors and cannot compile anywhere case-sensitive. Compilation
analysis used to stop there, with a source-not-found error and nothing to try next.
Approach
New module
certora_autosetup/setup/import_case_fix.py. It checks each component of an importpath against the real directory entries with
os.scandirinstead of asking whether the pathexists. An existence check on a case-insensitive filesystem resolves the mis-cased path happily,
which would make the defect invisible on the machine where the code is written and leave the fixer
firing only in production.
Handled: letter case in the basename, in any directory component, and in the extension, plus
Windows separators and doubled slashes. A path that names no file even case-insensitively is left
alone, because that is an absent file rather than a case problem.
Three bails are worth naming, since none of them is visible from the path alone:
business, not the project tree's, and repairing it is the packages workaround's job.
point into one, but never rewritten: a dependency install overwrites them.
case-sensitive filesystem, so an ambiguous match rewrites nothing.
Integration
In
run_compilation_analysisthe case fix gets a retry of its own. If it rewrote anything, thecompilation runs again, and on success the import patcher never runs at all. Only if that retry
still fails does the patcher go on top of the case fix rather than instead of it: the patcher
resolves each relative import against its map of real files and skips the ones that miss, so a
corrected spelling is one more import it can canonicalize. If the last attempt fails, both revert
in reverse order of application.
Rewrites are scoped to the quoted path literal, checked against the recorded bytes before anything
is written, kept single-line so the patcher's line-indexed revert stays correct, never applied
inside a dependency tree, and logged at WARNING with the old and new spelling.
Also adds a small
package_prefixes()helper toutils/import_diagnostics.pyso the fixer canleave remapped prefixes alone.
Testing
28 unit tests. They assert on planned and applied rewrites built from one real file plus a
wrong-cased import string, rather than creating two files that differ only in case. The latter is
impossible on a case-insensitive filesystem and would make the suite behave differently per
developer machine.
Full suite: 1125 passed, 10 skipped, 11 deselected, with 21 errors from the testcontainers-backed
test_rag_dbandtest_indexed_toolcases that need a database this machine does not start.pyright: 0 errors.
Known limitation
extract_imports_multilineaccumulates lines until the next;, so a commented-outimportwithno semicolon can make the scan land on an ordinary string constant further down the file and
rewrite it. Comment masking closes the common shape of this, but the accumulation is still there.
Closing it properly wants a real parser; the typed solc AST is not an option at this point because
it only exists after a successful build, and here the build has failed.
🤖 Generated with Claude Code