Skip to content

build: Create output directory before generating resource_paths.py - #8009

Merged
echoix merged 1 commit into
OSGeo:mainfrom
echoix:fix-python-app-makefile-parallel-race
Oct 3, 2026
Merged

echoix merged 1 commit into
OSGeo:mainfrom
echoix:fix-python-app-makefile-parallel-race

Conversation

@echoix

@echoix echoix commented Oct 3, 2026

Copy link
Copy Markdown
Member

Problem

The Docker builds in CI intermittently fail, and often pass on rerun. The make -j 4 step fails in python/grass/app with:

sed ... resource_paths.py > .../etc/python/grass/app/resource_paths.py
/bin/sh: 1: cannot create .../etc/python/grass/app/resource_paths.py: Directory nonexistent
make[5]: *** [Makefile:23: .../resource_paths.py] Error 2

python/grass/app is then not installed, so g.extension fails with ModuleNotFoundError: No module named 'grass.app.runtime', and the build exits with code 2.

Failing runs with this error: 37145943141, 37127864658 (passed on rerun), 37127518048.

Cause

The $(DSTDIR)/resource_paths.py rule redirects sed output into $(DSTDIR) but does not depend on it. With parallel make, the redirect can run before the mkdir -p for that directory (started for another target) has finished. The pattern rule for the other files already has | $(DSTDIR).

Fix

Add $(DSTDIR) as an order-only prerequisite of the resource_paths.py rule.

Testing

The race window is tiny, so it was reproduced and checked on a copy of python/grass/app/Makefile with the real Python.make and a slowed mkdir (MKDIR='sleep 1; mkdir -p', make -j8):

  • Before the change: fails with the exact Directory nonexistent error above.
  • After the change: builds successfully and installs all files (__init__.py, __main__.py, cli.py, data.py, runtime.py, resource_paths.py).

Without the slowed mkdir, neither version failed in 300 plain -j8 runs, which matches the intermittent behavior in CI.

The Docker workflow does not run on PRs that only touch this Makefile, so it was dispatched manually on this branch's fork: run 37152838494 built all four images (alpine, debian, ubuntu, ubuntu_wxgui) successfully. Since the failure is intermittent, one green run is supporting evidence. The Docker workflow runs on main after merge will provide more.

Developed with AI assistance (Claude Code).

The resource_paths.py rule in python/grass/app/Makefile redirects sed
output into the destination directory without depending on it. With
parallel make, the redirect could run before the directory was created,
failing with "Directory nonexistent" and breaking the Docker builds
intermittently. Add the directory as an order-only prerequisite, as the
pattern rule for the other files already does.

Developed with AI assistance (Claude Code).
@echoix
echoix enabled auto-merge (squash) October 3, 2026 22:18
@echoix
echoix merged commit e7f07a3 into OSGeo:main Oct 3, 2026
26 checks passed
@github-actions github-actions Bot added this to the 8.6.0 milestone Oct 3, 2026
@echoix
echoix deleted the fix-python-app-makefile-parallel-race branch October 4, 2026 01:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants