Repository navigation
grass.pygrass: Pin fork start method for RPC servers on Python 3.14 - #7724
Conversation
Python 3.14 changed the default multiprocessing start method from fork to forkserver on all platforms except macOS (python/cpython#84559). The RPC servers in grass.pygrass.rpc and grass.temporal rely on fork: their worker processes call libgis via ctypes and must inherit the initialized library state and the GRASS session environment from the parent. Under forkserver the worker is forked from an early, clean process without the GRASS runtime, and the forkserver socket teardown raises FileNotFoundError in the server-checker thread. Define a shared multiprocessing context in grass.pygrass.rpc.base that pins fork where it was the previous default, and reuse it in the three start_server implementations. macOS (already spawn) and Windows (no fork) keep their default start method, so the change is a no-op there and on Python <= 3.13. Root-caused and drafted with the help of Claude (Opus 4.8).
wenzeslaus
left a comment
There was a problem hiding this comment.
As a hot fix and given the testing, this is good, but the current comment is somewhat misleading.
Co-authored-by: Vaclav Petras <wenzeslaus@gmail.com>
|
There is more usage of Process from multiprocessing in temporal that may need adjustment of the start method (extract.py, mapcalc.py in temporal). Also the pickling error in pygrass needs to be addressed to make everything work with Python 3.14 on Linux (see #7257). |
|
Note also that tests here still do not run Python 3.14 yet... |
echoix
left a comment
There was a problem hiding this comment.
I even prefer more the private helper here
|
Reran macOS because of timeout at FAILED ./temporal/t.rast.to.rast3/testsuite/test_strds_to_rast3.py - Timeout >300.0s |
|
@wenzeslaus 's previous requested changes prevent auto-merge from happening |
|
Ubuntu pytest tests are now failing on main, it seems like it is the unclosed SQLite connection that has a message containing “traceback”, thus fails the new test added |
Problem
Running temporal tools under Python 3.14 fails, e.g.
t.list strds:Reported by @veroandreo (WSL, Python 3.14).
Cause
Python 3.14 changed the default
multiprocessingstart method fromforktoforkserveron every platform except macOS(python/cpython#84559).
GRASS's RPC servers were relying on the implicit
forkdefault. Their worker processes runlibgisthrough ctypes and depend onforksemantics: they inherit the already-initialized C library state and the live GRASS session environment (GISRC, etc.) from the parent. Underforkserver, the worker is forked from an early, clean forkserver process that never had the GRASS runtime set up, and the forkserver control socket gets torn down as the session exits — producing theFileNotFoundErrorin the checker thread and the leaked-semaphore warning.Fix
This targeted fix pins fork to restore prior behavior. A longer-term move to spawn is worth evaluating separately...
Pin
forkin one shared place and reuse it. Thestart_serverpattern is duplicated across three implementations that all shareRPCServerBase:python/grass/pygrass/rpc/base.py— add a module-levelMP_CONTEXTthat selectsforkwhere it was the previous default, and use it instart_server.python/grass/pygrass/rpc/__init__.py(DataProvider) andpython/grass/temporal/c_libraries_interface.py(CLibrariesInterface) — importMP_CONTEXTand use it in their overriddenstart_server.macOS was already on spawn (since 3.8) and Windows has no fork, so the change is a deliberate no-op on those platforms and on Python <= 3.13, where fork was already the default.
Testing
Tested with @veroandreo and locally on linux with Python 3.12, worked fine.