Repository navigation
Conversation
Python's default text-mode file writing translates "\n" to the
platform's os.linesep. On Windows that turns "\n" into "\r\n", which
silently corrupts fixture files that two testsuite tests hash in
binary mode and compare against a hardcoded MD5 computed from
Unix-style ("\n") content: test_checkers.py's test_text_file_binary
and test_assertions.py's test_assertFileMd5. Both are already marked
with @xfail_windows, consistent with this being the cause.
Write these fixtures with newline="" so the given content is stored
byte-for-byte, matching the precomputed MD5 sums on every platform.
Apply the same treatment to the other two fixtures written alongside
file_with_md5 in test_assertions.py, since test_assertFilesEqualMd5
hashes them against file_with_md5 in the same binary mode and would
otherwise start disagreeing with it once file_with_md5 alone stops
being translated.
This is a hypothesis for Windows CI to confirm, not a proven fix:
there is no Windows environment available to verify it here, and the
existing @xfail_windows markers are left in place pending that
confirmation.
This reworks two Windows-CI-testsuite commits ("handle OS newlines in
testsuite", authored by Stefan Blumentrath / @ninsbl) from the
unrebased OSGeo#2616, extracted and adapted against current
main with Claude's assistance rather than cherry-picked, since a
direct cherry-pick would also revert unrelated changes both files
have accumulated since 2022 (e.g. test_assertions.py switched from
open() to Path.write_text()).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
echoix
left a comment
There was a problem hiding this comment.
Reviewed the change. What I verified as correct:
Path.write_text(..., newline="")is valid on this project's floor (pyproject.tomlhasrequires-python = ">=3.11"; thenewlinekwarg landed in 3.10), andopen(..., "w", newline="")is correct — both suppress the\n→os.lineseptranslation.- Both hardcoded sums match the intended
\n-only content (reproduced locally:807bba4ffac4bb351bc3f27853009949and9dd6c4bb9d2cf6051b12f4b5f9d70523). - Extending the change to
file_with_same_content/file_with_different_contentis necessary rather than scope creep:test_assertFilesEqualMd5hashes them againstfile_with_md5in binary mode, so leaving them translated would break a currently-passing test on Windows. - No collateral damage to the text-mode tests:
text_file_md5()(python/grass/gunittest/checkers.py:633) reads with universal newlines, sotest_text_file_unixis unaffected either way, andcorrect_file_name_platform_nlis deliberately left on the default path.
One thing worth flagging:
Leftover @xfail_windows decorators. test_checkers.py:387 (test_text_file_binary) and test_assertions.py:389 (test_assertFileMd5) still carry @xfail_windows. If the fix works on Windows, those tests pass, get recorded as unexpectedSuccesses, and unittest.TestResult.wasSuccessful() returns False — so main.py:124 exits non-zero exactly when the fix succeeds. Windows gunittest goes red on success.
The PR description calls this a deliberate choice pending CI confirmation, which is defensible, but the decorators should come off in this same PR once CI confirms (or immediately, accepting a red run if the hypothesis turns out wrong).
Generated by Claude Code
unittest.TestResult.wasSuccessful() (which python/grass/gunittest/main.py relies on for its exit code) returns False whenever unexpectedSuccesses is non-empty, and @xfail_windows applies unittest.expectedFailure on Windows. Left in place, the two decorators removed here would flip a passing Windows gunittest run to a failing one exactly when the newline fix in the previous commit works, since the previously-failing tests would now succeed unexpectedly instead of failing as declared. Since the fix is only a hypothesis pending confirmation on real Windows CI, removing the decorators now is also the more honest choice either way: if the hypothesis holds, the run is green for the right reason; if it does not, the tests fail plainly instead of being silently tolerated. Thanks to the review on this PR for catching this. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
Good catch, thanks. Confirmed: Pushed 8c7b6d9, removing Generated by Claude Code |
|
Windows CI finished: Generated by Claude Code |
Description
Two
grass.gunittesttestsuite fixture files are written with Python'sdefault text-mode newline handling, which translates
\ntoos.linesepon write. On Windows that turns\ninto\r\n, silentlycorrupting fixtures that are later hashed in binary mode and
compared against a hardcoded MD5 computed from Unix-style (
\n)content:
python/grass/gunittest/testsuite/test_checkers.py:test_text_file_binaryhashescorrect_file_name_unix_nlwithfile_md5()(binary) and compares it tocorrect_md5sum, which wascomputed from a heredoc with
\nline endings.python/grass/gunittest/testsuite/test_assertions.py:test_assertFileMd5hashesfile_with_md5(binary, viaassertFileMd5) and compares it to a hardcoded MD5 computed the sameway.
Both tests are already marked
@xfail_windows, which is consistentwith this being the cause:
os.linesep == "\n"on Linux/macOS, so thebug is invisible there.
The fix opens/writes these two fixtures with
newline="", which keeps\nas-is instead of letting it get translated, per Python'sdocumented
open()newline semantics (verified locally: withnewline=""the written bytes are\n-only regardless of platform,whereas the default lets the platform newline through — on Linux this
is a no-op since
os.linesep == "\n"there, so it can't be observedwithout Windows).
I also extended the same treatment to the two other fixtures written
alongside
file_with_md5intest_assertions.py(
file_with_same_content,file_with_different_content), which theoriginal upstream commit did not touch.
test_assertFilesEqualMd5hashes
file_with_same_contentagainstfile_with_md5in the samebinary mode; leaving it on the default-translated path while
file_with_md5switches to raw bytes would make them disagree onWindows and turn a currently-passing test into a new failure.
Motivation and context
This extracts and reworks one small piece of the large, stale, never
rebased upstream PR
OSGeo/grass#2616 ("CI:
Address Windows specific issues in testsuite", by Stefan Blumentrath /
@ninsbl), so it can be reviewed and merged independently instead of as
part of a ~35-commit bundle. The underlying idea for these two specific
fixture writes comes from upstream's "handle OS newlines in testsuite"
commits (
fb075bfa3,45bb9d75c), authored by @ninsbl in 2022.That branch was never rebased, so its diff was not cherry-picked
directly: both files have moved on since 2022 (e.g.
test_assertions.pyswitched fromopen()toPath.write_text()), anda naive cherry-pick would have reverted those unrelated, independent
changes. Instead, I read the relevant upstream commits, compared the
pre-change baseline against current
mainfor these two files, andre-implemented just the newline handling against current
main(extending its scope slightly, per above).
This PR was extracted and reworked with the assistance of Claude
(Claude Code), per this repo's
AGENTS.mdguidance to disclosesubstantial AI use. I reviewed the underlying Python
open()/newlinebehavior directly and confirmed the two hardcoded MD5 sums against the
intended
\n-only content by hashing it myself; see caveats below.How has this been tested?
environment, so the fix has not been run against real Windows
CI. This is a hypothesis for CI to confirm, not a proven fix — please
treat it as such.
open(..., newline="")documented behavior, confirmedempirically on Linux (a no-op there since
os.linesep == "\n",which is exactly why this bug would not show up outside Windows).
correct_md5sumintest_checkers.py,file_md5intest_assertions.py) matchhashlib.md5()of theintended
\n-only content, confirming they were computed fromUnix-style line endings and that the fix targets the right content.
ruff check --previewandruff format --diff(ruff 0.15.17,matching
.pre-commit-config.yaml) pass on both changed files.@xfail_windowsdecorators ontest_text_file_binaryandtest_assertFileMd5in place rather thanremoving them, since I cannot confirm on real Windows CI that no
other issue remains in either test. If CI shows them passing, a
maintainer (or a follow-up PR) can remove the decorators per the
comment already in
xfail_windows()'s docstring.Screenshots (if appropriate)
N/A (test-only change).
Types of changes
functionality to not work as before)
Checklist
pre-defined prefixes
of this project.
existing test fixtures; no new behavior is introduced.)
🤖 Generated with Claude Code
https://claude.ai/code/session_013RUh1QAj3DCnVZEhe7HXuo
Generated by Claude Code