Skip to content

grass.pygrass: Pin fork start method for RPC servers on Python 3.14 - #7724

Merged
petrasovaa merged 3 commits into
OSGeo:mainfrom
petrasovaa:fix-python-3.14-temporal
Sep 24, 2026
Merged

petrasovaa merged 3 commits into
OSGeo:mainfrom
petrasovaa:fix-python-3.14-temporal

Conversation

@petrasovaa

Copy link
Copy Markdown
Contributor

Problem

Running temporal tools under Python 3.14 fails, e.g. t.list strds:

    Exception in thread Thread-1 (thread_checker):
    ...
    File ".../grass/temporal/c_libraries_interface.py", line 1480, in start_server
        self.server.start()
    ...
    File ".../multiprocessing/forkserver.py", line 94, in connect_to_new_process
        client.connect(self._forkserver_address)
    FileNotFoundError: [Errno 2] No such file or directory
    resource_tracker: There appear to be 1 leaked semaphore objects to clean up

Reported by @veroandreo (WSL, Python 3.14).

Cause

Python 3.14 changed the default multiprocessing start method from fork to forkserver on every platform except macOS
(python/cpython#84559).

GRASS's RPC servers were relying on the implicit fork default. Their worker processes run libgis through ctypes and depend on fork semantics: they inherit the already-initialized C library state and the live GRASS session environment (GISRC, etc.) from the parent. Under forkserver, the worker is forked from an early, clean forkserver process that never had the GRASS runtime set up, and the forkserver control socket gets torn down as the session exits — producing the FileNotFoundError in the checker thread and the leaked-semaphore warning.

Fix

This targeted fix pins fork to restore prior behavior. A longer-term move to spawn is worth evaluating separately...

Pin fork in one shared place and reuse it. The start_server pattern is duplicated across three implementations that all share RPCServerBase:

  • python/grass/pygrass/rpc/base.py — add a module-level MP_CONTEXT that selects fork where it was the previous default, and use it in start_server.
  • python/grass/pygrass/rpc/__init__.py (DataProvider) and python/grass/temporal/c_libraries_interface.py (CLibrariesInterface) — import MP_CONTEXT and use it in their overridden start_server.
if sys.platform != "darwin" and "fork" in mp.get_all_start_methods():
    MP_CONTEXT = mp.get_context("fork")
else:
    MP_CONTEXT = mp.get_context()

macOS was already on spawn (since 3.8) and Windows has no fork, so the change is a deliberate no-op on those platforms and on Python <= 3.13, where fork was already the default.

Testing

Tested with @veroandreo and locally on linux with Python 3.12, worked fine.

Python 3.14 changed the default multiprocessing start method from fork
to forkserver on all platforms except macOS (python/cpython#84559).
The RPC servers in grass.pygrass.rpc and grass.temporal rely on fork:
their worker processes call libgis via ctypes and must inherit the
initialized library state and the GRASS session environment from the
parent. Under forkserver the worker is forked from an early, clean
process without the GRASS runtime, and the forkserver socket teardown
raises FileNotFoundError in the server-checker thread.

Define a shared multiprocessing context in grass.pygrass.rpc.base that
pins fork where it was the previous default, and reuse it in the three
start_server implementations. macOS (already spawn) and Windows (no
fork) keep their default start method, so the change is a no-op there
and on Python <= 3.13.

Root-caused and drafted with the help of Claude (Opus 4.8).
@petrasovaa petrasovaa added the bug Something isn't working label Jul 15, 2026
@github-actions github-actions Bot added Python Related code is in Python libraries labels Jul 15, 2026

@wenzeslaus wenzeslaus left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a hot fix and given the testing, this is good, but the current comment is somewhat misleading.

Comment thread python/grass/pygrass/rpc/base.py Resolved
Co-authored-by: Vaclav Petras <wenzeslaus@gmail.com>
@ninsbl

ninsbl commented Sep 21, 2026

Copy link
Copy Markdown
Member

There is more usage of Process from multiprocessing in temporal that may need adjustment of the start method (extract.py, mapcalc.py in temporal). Also the pickling error in pygrass needs to be addressed to make everything work with Python 3.14 on Linux (see #7257).

@ninsbl

ninsbl commented Sep 21, 2026

Copy link
Copy Markdown
Member

Note also that tests here still do not run Python 3.14 yet...

@github-actions github-actions Bot added the tests Related to Test Suite label Sep 23, 2026

@echoix echoix left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I even prefer more the private helper here

@echoix

echoix commented Sep 23, 2026

Copy link
Copy Markdown
Member

Reran macOS because of timeout at FAILED ./temporal/t.rast.to.rast3/testsuite/test_strds_to_rast3.py - Timeout >300.0s

@echoix

echoix commented Sep 23, 2026

Copy link
Copy Markdown
Member

@wenzeslaus 's previous requested changes prevent auto-merge from happening

@petrasovaa
petrasovaa removed the request for review from wenzeslaus September 24, 2026 14:13
@petrasovaa
petrasovaa merged commit e39fb8c into OSGeo:main Sep 24, 2026
26 of 27 checks passed
@github-actions github-actions Bot added this to the 8.6.0 milestone Sep 24, 2026
@echoix

echoix commented Sep 25, 2026

Copy link
Copy Markdown
Member

Ubuntu pytest tests are now failing on main, it seems like it is the unclosed SQLite connection that has a message containing “traceback”, thus fails the new test added

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working libraries Python Related code is in Python tests Related to Test Suite

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants