Skip to content

r.mfilter: Avoid reopening NamedTemporaryFile on Windows in testsuite - #677

Closed
echoix wants to merge 1 commit into
mainfrom
claude/grass-pr-2616-component-cgi1g2
Closed

echoix wants to merge 1 commit into
mainfrom
claude/grass-pr-2616-component-cgi1g2

Conversation

@echoix

@echoix echoix commented Sep 2, 2026

Copy link
Copy Markdown
Owner

Description

Extracts and reworks one self-contained change out of upstream PR
OSGeo/grass#2616 ("CI: Address
Windows specific issues in testsuite", open since 2022, 35 commits, by
Stefan Blumentrath / @ninsbl) so it can be evaluated on its own. Opened
against this fork, for testing, not upstream.

raster/r.mfilter/testsuite/test_r_mfilter.py creates its filter-definition
temp file with tempfile.NamedTemporaryFile() and keeps it open (via
.name) while r.mfilter opens the same path again. On Windows this second
open fails, because Windows does not allow reopening a file that another
handle (here, the still-open Python file object) has locked. That's why all
6 tests in this file carry @xfail_windows. OSGeo#2616 replaces this with
grass.script.tempfile(create=False), which only reserves a unique path (via
g.tempfile -d) without holding it open, then writes to it directly.

Motivation and context

An AI (Claude) review of OSGeo#2616 in July 2026, and a second one now, flagged
this as the easiest individual piece of that PR to land as its own, minimal
PR: it is confined to one test file, has a well-understood, verifiable root
cause (the double-open bug is exactly what @xfail_windows's own docstring
describes), and doesn't depend on any of the PR's other, more invasive
Windows changes.

That said, OSGeo#2616's own diff for this file is not usable as a straight
cherry-pick: it was branched in 2022 and never rebased, so its diff against
current main also reverts changes unrelated to the tempfile fix that
landed on main in the meantime — a docstring correction
(nc_spm_full_v2alphav2 -> nc_spm_full_v2beta1), tightened variance
reference values, and trailing-whitespace cleanup. This PR re-implements
just the tempfile fix against current main, so none of that unrelated
history gets clobbered. It also drops filter.close() (no longer needed,
since filter is now a path string, not a file object) and the
@xfail_windows markers themselves, per the instruction in
xfail_windows's own docstring: "Once the test is fixed and passing, remove
the decorator."

I have not verified this on an actual Windows machine. If the Windows CI job
here still fails, the @xfail_windows markers should be restored, and the
underlying reasoning re-examined — the AI analysis should not be taken as
proof the fix works, only as a plausible, checkable hypothesis.

How has this been tested?

  • python3 -m py_compile on the changed file.
  • ruff check --preview and ruff format --diff (both clean, ruff 0.15.17
    to match .pre-commit-config.yaml).
  • Not run against a live GRASS build or on Windows (no build available in
    this environment) — relying on this fork's CI to confirm the fix on
    Windows and non-regression on Linux/macOS.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing
    functionality to not work as before)

Checklist

  • PR title provides summary of the changes and starts with one of the
    pre-defined prefixes
  • My code follows the code style
    of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have added tests to cover my changes. (No new tests added; existing
    tests in this file are what the change fixes and un-xfails.)

Original fix idea from Stefan Blumentrath (@ninsbl) in
OSGeo/grass#2616, reworked here
against current main. AI-assisted (Claude) analysis and implementation;
please verify independently before relying on this, especially the removal
of the @xfail_windows markers.

🤖 Generated with Claude Code

https://claude.ai/code/session_01HjAoKyq8bFN7cnNaZkwtV6


Generated by Claude Code

@echoix echoix left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where and where in the code and time is the file deleted?

echoix commented Sep 2, 2026

Copy link
Copy Markdown
Owner Author

Re: "Where and where in the code and time is the file deleted?"

Short answer: it isn't, by this code. gs.tempfile(create=False) only reserves a unique path (g.tempfile -d, see lib/gis/tempfile.c); neither this patch nor the original #2616 commit it's derived from ever calls os.remove()/unlink() on the result. That's a real, if minor, behavior difference from NamedTemporaryFile(), which deletes itself deterministically on close/GC.

Tracing what actually removes it, through lib/gis/tempfile.c and lib/init/clean_temp.c:

  1. The file is named <pid>.<n> under <mapset>/.tmp/<hostname>/, where <pid> is the test process's own os.getpid() (python/grass/script/core.py:tempfile).
  2. GRASS's generic temp-file GC (etc/clean_temp) unlinks a pid.n file once its owning pid is no longer running (or unconditionally once it's >4 days old). It's invoked once when a new session opens that mapset (lib/init/grass.py ~line 2443) and once at normal session finish() (python/grass/script/setup.py:632) — neither can fire while the test process that created the file is still alive.
  3. For this test, run the documented way (grass --tmp-mapset ... --exec python test_r_mfilter.py), that's moot: the entire --tmp-mapset directory, .tmp included, is deleted when that ephemeral mapset tears down at the end of the run, taking the leftover filter file with it regardless of clean_temp.
  4. Run against a persistent mapset instead, the files would leak until a later session in that same mapset happens to run clean_temp after the pid has died, or until the 4-day sweep catches them.

This isn't a new gap introduced here: it matches the only other precedent for gs.tempfile() in a testsuite/ in this codebase, temporal/t.register/testsuite/test_t_register_raster_file.py, which also never unlinks its tempfiles. I think it's acceptable given that precedent and the tmp-mapset test runner, but flagging it explicitly since it is a real change in cleanup semantics versus what the code did before.


Generated by Claude Code

On Windows, a file opened via tempfile.NamedTemporaryFile() cannot be
reopened by another process while the Python handle is still open, so
r.mfilter failed to read the filter file created this way. Write the
filter contents to a path from grass.script.tempfile() instead, which
does not hold the file open, and drop the @xfail_windows markers this
fix should resolve.

Extracted and reworked as a standalone change from PR OSGeo#2616 (by Stefan
Blumentrath / @ninsbl), which combined this fix with many other
Windows-specific testsuite changes and has been open, unrebased, since
2022. Re-derived against current main rather than cherry-picked,
because @ninsbl's branch predates unrelated fixes already merged to
this same test file (a docstring/variance correction and other
@xfail_windows additions), which a direct cherry-pick would have
clobbered.

This has not been verified on an actual Windows run; CI on this PR
should confirm whether the @xfail_windows markers can indeed be
removed. Analysis of PR OSGeo#2616 assisted by AI (Claude), including
identifying this as an easily isolated, self-contained candidate for
its own PR; verify before merging.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@echoix
echoix force-pushed the claude/grass-pr-2616-component-cgi1g2 branch from f1d5501 to cce419a Compare September 2, 2026 21:13

echoix commented Sep 3, 2026

Copy link
Copy Markdown
Owner Author

This landed upstream as OSGeo#7878 and is now on this fork's main (5734e23), so this testing PR is redundant. Closing.


Generated by Claude Code

@echoix echoix closed this Sep 3, 2026
@echoix
echoix deleted the claude/grass-pr-2616-component-cgi1g2 branch September 12, 2026 19:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant