[nilason]

GISBASE is used as a means to retrieve paths to various resources in source code. With FHS (#3661) this is no longer possible. This adds a number of new environment variables to address this.

Currently this is addressed with following workaround:

grass/CMakeLists.txt

Lines 176 to 194 in 5e998d3

	if(WITH_FHS)
	file(CREATE_LINK ${OUTDIR}/${GRASS_INSTALL_DEMODIR}
	${RUNTIME_GISBASE}/demolocation SYMBOLIC)
	file(CREATE_LINK ${OUTDIR}/${GRASS_INSTALL_FONTSDIR} ${RUNTIME_GISBASE}/fonts
	SYMBOLIC)
	file(CREATE_LINK ${OUTDIR}/${GRASS_INSTALL_ETCDIR}/colors
	${RUNTIME_GISBASE}/etc/colors SYMBOLIC)
	file(CREATE_LINK ${OUTDIR}/${GRASS_INSTALL_ETCDIR}/colors.desc
	${RUNTIME_GISBASE}/etc/colors.desc SYMBOLIC)
	file(CREATE_LINK ${OUTDIR}/${GRASS_INSTALL_ETCDIR}/element_list
	${RUNTIME_GISBASE}/etc/element_list SYMBOLIC)
	file(CREATE_LINK ${OUTDIR}/${GRASS_INSTALL_ETCDIR}/renamed_options
	${RUNTIME_GISBASE}/etc/renamed_options SYMBOLIC)
	file(CREATE_LINK ${OUTDIR}/${GRASS_INSTALL_ETCDIR}/VERSIONNUMBER
	${RUNTIME_GISBASE}/etc/VERSIONNUMBER SYMBOLIC)
	file(MAKE_DIRECTORY "${GISBASE}/gui/wxpython")
	file(CREATE_LINK ${OUTDIR}/${GRASS_INSTALL_GUIDIR}/wxpython/xml
	${RUNTIME_GISBASE}/gui/wxpython/xml SYMBOLIC)
	endif()

This is work in progress.

[nilason]

Hmm, the Windows tests fail with "ERROR: Incomplete GRASS session: Variable 'GRASS_ETCDIR' not set". However, running the tests with added set GRASS_ETCDIR=%GISBASE%\etc, like nilason@7452d64 they succeed. I suspect maybe something similar to 2ee5faa might be the reason. Difficult to debug this on CI though, maybe Claude can be of help (@echoix, @wenzeslaus, anyone...).

[echoix]

I only have a limited plan and no Fable. But ya, I'd rather have an agent loose its time rather than me so I could do more reviewing/deciding

[nilason]

The Windows CI runner's build script does not install with make install but by copying the dist.* directory:

grass/mswindows/osgeo4w/build_osgeo4w.sh

Line 144 in 1a88e4e

cp -a $dist $grass_path

This unusual solution has made things work so far with a lot of hacks. This is now failing because of the install-step configuration of

grass/lib/init/grass.py

Line 1788 in 1a88e4e

sys.stdout.write("%s\n" % r"@GRASS_PYDIR@")

and
https://github.com/OSGeo/grass/blob/main/python/grass/app/resource_paths.py
are skipped and the build-step configurations gets installed.

[echoix]

Hmm, the Windows tests fail with "ERROR: Incomplete GRASS session: Variable 'GRASS_ETCDIR' not set". However, running the tests with added set GRASS_ETCDIR=%GISBASE%\etc, like nilason@7452d64 they succeed. I suspect maybe something similar to 2ee5faa might be the reason. Difficult to debug this on CI though, maybe Claude can be of help (@echoix, @wenzeslaus, anyone...).

Ok, I made opus 5.5 take a look.
Here's its summary comment:

The Windows failure comes from runtime_env_is_active() in grass.script.setup. env.bat sets GISBASE and the pytest steps put %GISBASE%\bin on PATH, so ensure_runtime_env() (called by create_project()) thinks the runtime is already set up and never sets the new GRASS_*DIR variables. It's not the dist.* copy. Reproducible on Linux by setting only GISBASE, PATH and PYTHONPATH. Proposed fix (require the resource variables, and derive them from an existing GISBASE rather than the build-time prefix), with tests: echoix/grass@8af2d2a7. No workflow change needed.

It flagged a possible issue on the non-python side, in C. See the end of the third thread

I'm adding the thread here, in details

Details (first plan)

#5630: Windows CI fails with "Variable 'GRASS_ETCDIR' not set"

Context

The PR replaces hard-coded $GISBASE/etc and similar paths with per-resource
variables (GRASS_ETCDIR, GRASS_COLORSDIR, GRASS_LOCALEDIR, ...).
lib/gis/resource_dirs.c::get_g_env() calls G_fatal_error() when one of
them is unset. On Windows CI, the pytest steps fail with
Incomplete GRASS session: Variable 'GRASS_ETCDIR' not set. nilason's
workaround commit (nilason/grass@7452d64) adds set GRASS_ETCDIR=%GISBASE%\etc to
the three pytest steps in .github/workflows/osgeo4w.yml, and that makes the
tests pass.

Root cause

The dist.* copy and the unsubstituted @GRASS_PYDIR@ that nilason raised on
Oct 9 are not the cause. The $bin/grass.py copy behaves the same as on main,
and dist/etc/python/grass/app/resource_paths.py gets relative values such as
GRASS_ETCDIR = "etc" from python/grass/app/Makefile.

The real cause is a shortcut in python/grass/script/setup.py:

• Every Windows pytest step first runs call ...\opt\grass\etc\env.bat, which
  sets only GISBASE. It then runs path %GISBASE%\lib;%GISBASE%\bin;%PATH%.
• Tests create projects with gs.create_project(). Through local_env()
  (python/grass/script/core.py ~L1991), that calls
  ensure_runtime_env(env).
• ensure_runtime_env() calls setup_runtime_env() only if
  runtime_env_is_active() returns False. That function only checks that
  GISBASE is set and appears in PATH. On Windows CI both conditions hold,
  so setup_runtime_env() is skipped. That means
  RuntimePaths.set_env_variables() never runs and no GRASS_*DIR variable
  is set.
• g.proj then reaches G_etc_dir() (datum/ellipse tables,
  renamed_options, G_find_etc) and fails fatally. The PR's own change to
  create_project adds the message "g.proj failed ... but no error message
  was produced", which matches this symptom.
• Linux and macOS CI never set GISBASE beforehand, so they always run the
  full setup and pass.

This is not only a CI problem. Any user script that sets GISBASE and PATH
manually, which the Windows docs recommend, and then calls create_project
will break the same way. On main, GISBASE alone was enough.

Proposed fix (on claude/wonderful-shannon-g87w2g, based on the PR head pr5630)

1. python/grass/script/setup.py::runtime_env_is_active(): return False unless
   all resource variables managed by RuntimePaths are set, in addition to
   the existing GISBASE-in-PATH check. Expose the variable names publicly
   from RuntimePaths (for example a small class attribute or property in
   python/grass/app/runtime.py) rather than reading the private _env_vars.
2. setup_runtime_env(): when gisbase is not given, use the existing
   env["GISBASE"] as the prefix (RuntimePaths(env=env, prefix=...)
   already strips the GISBASE suffix for FHS). The derived dirs then point
   into the caller's install (opt\grass) instead of the build-time
   GRASS_PREFIX, which is the dist.* build tree on Windows CI. This also
   matches get_install_path(), which already prefers GISBASE.
3. Keep osgeo4w.yml unchanged, and drop the GRASS_ETCDIR workaround.

Verification

• Build the PR branch on Linux (cmake -B build -DCMAKE_INSTALL_PREFIX=..., or
  Autotools) and install it.
• Reproduce the Windows condition before applying the fix. Use a clean env
  with only GISBASE=<install> and PATH=<install>/bin:... set, plus
  PYTHONPATH, then run
  python -c "import grass.script as gs; gs.create_project('/tmp/p', crs='EPSG:4326')".
  Expect the GRASS_ETCDIR not set failure.
• Apply the fix and rerun. It should succeed with GRASS_ETCDIR under
  GISBASE.
• Run pytest python/grass/script/tests python/grass/app/tests and add a
  unit test for runtime_env_is_active() with only GISBASE set.
• Run pre-commit on the changed files.
• Commit and push to origin claude/wonderful-shannon-g87w2g. Do not open a
  PR. Give the user a short summary they can post on build: prepare for FHS, where GISBASE is not the absolute truth #5630.

Details (first step, results of first plan)

Can you investigate the problem mentionned in this comment #5630 (comment) and later comments?

I'm reading the PR thread and pulling the PR branch so I can trace the GRASS_ETCDIR failure on Windows.

I've read the build script. Now I'm checking how the Windows tests start GRASS and which step should set GRASS_ETCDIR.

I think I've found the root cause. I'm checking nilason's workaround commit to confirm it.

The tests don't fail because of the dist.* copy. They fail because of a shortcut in grass.script.setup. I traced this by reading the PR code. I couldn't open the Windows CI logs, and I haven't reproduced it locally yet.

Root cause

• Every pytest step in .github/workflows/osgeo4w.yml first runs call ...\opt\grass\etc\env.bat, which sets only GISBASE. It then puts %GISBASE%\bin on PATH.
• Most tests start with gs.create_project(), which calls ensure_runtime_env() (python/grass/script/core.py around line 1991).
• ensure_runtime_env() only runs the full setup when runtime_env_is_active() returns False (python/grass/script/setup.py). That check only asks whether GISBASE is set and appears in PATH. On Windows CI both are true, so the setup is skipped. RuntimePaths.set_env_variables() never runs, and none of the new GRASS_*DIR variables get set.
• g.proj then calls G_etc_dir() (datum and ellipse tables, renamed_options, G_find_etc). Its helper get_g_env() in lib/gis/resource_dirs.c stops with "Variable 'GRASS_ETCDIR' not set".
• Linux and macOS CI don't set GISBASE beforehand, so they always run the full setup and pass. This also explains why nilason's workaround (nilason/grass@7452d64, adding set GRASS_ETCDIR=%GISBASE%\etc to those three pytest steps) fixes the tests.

Why the Oct 9 theory doesn't fit: the copied grass.py resolves @GRASS_PYDIR@ to a build-tree path, but main does exactly the same today. The copied dist/.../resource_paths.py already has the new variables filled in with relative paths like GRASS_ETCDIR = "etc". The copy step doesn't cause this failure.

This affects more than CI. On main, setting GISBASE and PATH by hand (the documented way to use GRASS from Python on Windows) was enough. With the PR, such a script fails as soon as it calls create_project. So the fix belongs in the library, not in osgeo4w.yml.

Suggested fix (on top of the PR):

1. In runtime_env_is_active(), also return False unless every resource variable that RuntimePaths manages is set. RuntimePaths should list those names publicly rather than have callers read its private _env_vars.
2. In setup_runtime_env(), when no gisbase is passed, use the existing GISBASE as the prefix. The derived folders then point into the caller's install (opt\grass), not the build-time GRASS_PREFIX, which on Windows CI is the dist.* build tree. get_install_path() already prefers GISBASE this way.
3. Drop the GRASS_ETCDIR workaround from osgeo4w.yml.

To confirm on Linux, the plan is to build the PR, then run gs.create_project(...) with only GISBASE, PATH and PYTHONPATH set. It should fail with the same error before the fix and succeed after it.

The plan is in /root/.claude/plans/can-you-investigate-the-tranquil-pancake.md. Should I build, reproduce and push the fix to claude/wonderful-shannon-g87w2g, along with a short write-up you can post on #5630?

Details (my additionnal request)

yes, reproduce it and push the fix. Do you think removing setting the variables on windows CI for the pytest is a good way to go? Keep in mind the way grass is launched on windows with an OSGeo4W installation: there's a .bat script adding env vars among other actions.

Details (the end)

While it builds, I'll draft the fix.

The build is running in the background. I'll reproduce the failure on the unmodified tree once it finishes, then apply the fix.

Those commits are the PR author's and upstream history, so I won't rewrite them. My own commits will use the configured identity.

The build is still running (it's on raster/ now). I'll continue once it finishes.

The build succeeded. Now I'll reproduce the Windows CI setup: copy dist.* elsewhere, set only GISBASE, PATH and PYTHONPATH (as env.bat and the workflow do), then call create_project.

Reproduced. As a control, I'll take GISBASE out of PATH so the shortcut doesn't apply:

The control passes, which confirms the shortcut is the cause. Next I'll check which paths the full setup picks when GISBASE points to the copy:

Confirmed that relying only on the shortcut fix would leave setup_runtime_env() overwriting GISBASE with the build-time prefix, breaking relocated installs. I'll apply the fix to both parts now.

Now the edits.

Now I'll rebuild the Python packages, refresh the copy, and rerun the reproduction:

The reproduction passes now. Next I'll check that the paths point to the copied install:

The paths now resolve to the copy. Next I'll add regression tests:

Actually, the relocation test incorrectly assumes GISBASE equals the prefix, which breaks for FHS layouts. I'll rebuild it using RuntimePaths so it works for any layout and have the first test start from the real environment.

Running the tests against the fixed build:

All 131 pass. To make sure the new tests actually catch the bug, I'll run them against the original setup.py:

Both tests fail on the old code and pass with the fix. I'll check whether the PR head carries the CI workaround, then run pre-commit and the broader script tests:

While pre-commit sets up, I'll run the wider grass.script and grass.tools tests:

Two files can't be collected because numpy isn't installed in the pytest tool environment. I'll skip those two and run the rest:

All pre-commit hooks pass. Here's the final diff, which I've reviewed:

Committing and pushing:

I reproduced the failure on Linux, fixed it in grass.script, and pushed the fix as commit 8af2d2a7 to origin/claude/wonderful-shannon-g87w2g, on top of the head of #5630. I haven't run it on Windows: the OSGeo4W workflow only runs on pushes to main and release branches, so it won't run on this branch.

Reproduction. I built the PR with Autotools and copied dist.* elsewhere, as build_osgeo4w.sh does. I then set only GISBASE, PATH (with lib and bin) and PYTHONPATH, the way env.bat plus the workflow do. gs.create_project(..., crs="EPSG:4326") fails with exactly Incomplete GRASS session: Variable 'GRASS_ETCDIR' not set. With GISBASE taken out of PATH, it passes.

The fix has two parts:

1. The shortcut. runtime_env_is_active() now also requires every resource folder variable that RuntimePaths manages. Without them, ensure_runtime_env() runs the full setup.
2. Which paths it derives. Fixing only the shortcut would have exposed a second problem I confirmed: setup_runtime_env() replaced the caller's GISBASE with the build-time prefix, which on CI is the dist.* build tree, not opt\grass. For a relocated install that path is wrong. It now derives the paths from an existing GISBASE, as grass.py and get_install_path() already do.

I added two tests. Both fail on the PR's code and pass with the fix. The setup, app, location, grass.script and grass.tools tests pass (333 passed, 1 skipped, 1 expected failure), and pre-commit is clean. Two numpy-based test files weren't run because numpy isn't installed in the pytest environment here.

Should the variables be removed from the Windows CI pytest steps? Yes, and the PR's workflow doesn't even contain them: set GRASS_ETCDIR=%GISBASE%\etc only exists in nilason's debugging commit on their fork. Keeping the variables out is the better test, for these reasons:

• The steps model real use. They simulate Python code started outside the grass launcher, with only env.bat called first. Setting the variables in the workflow would hide this bug from CI.
• The launcher isn't affected. grass.bat calls o4w_env.bat and env.bat, then starts grass.py, which sets all the GRASS_*DIR variables itself. Interactive sessions and grass --exec worked even before the fix.
• env.bat is the wrong place. Setting the variables there would duplicate the layout that resource_paths.py already holds. The two copies could drift apart, and they wouldn't help anyone who sets GISBASE without using env.bat.

A remaining risk outside Python. Programs that set GISBASE and PATH and call tools directly, without Python or grass.py, still fail. QGIS Processing's GRASS provider is the likely case, though I haven't checked how it launches tools. The fix there would be in C: get_g_env() in lib/gis/resource_dirs.c could fall back to a build-time relative path under GISBASE (for example GISBASE/etc) instead of stopping with an error. I haven't made that change; it's worth raising on the PR.

Here's a short comment you can post on #5630:

The Windows failure comes from runtime_env_is_active() in grass.script.setup. env.bat sets GISBASE and the pytest steps put %GISBASE%\bin on PATH, so ensure_runtime_env() (called by create_project()) thinks the runtime is already set up and never sets the new GRASS_*DIR variables. It's not the dist.* copy. Reproducible on Linux by setting only GISBASE, PATH and PYTHONPATH. Proposed fix (require the resource variables, and derive them from an existing GISBASE rather than the build-time prefix), with tests: echoix/grass@8af2d2a7. No workflow change needed.

Details (second plan prompt, mine)

add the C fallback in get_g_env too, and verify it. Use a separate commit. Create a comment to post for the PR too that describes correctly what you did for an expert in C/C++ on the codebase.

Details (second plan, C problem)

C fallback for resource directories when only GISBASE is set

Context

Commit 8af2d2a on claude/wonderful-shannon-g87w2g fixes the Python side of
#5630: with only GISBASE set, grass.script now sets up the
GRASS_*DIR variables. Programs that run tools directly with only GISBASE
and GISRC set (no Python and no grass.py, for example the QGIS GRASS
provider) still hit G_fatal_error() in lib/gis/resource_dirs.c::get_g_env().
On main, these callers worked because the paths were G_gisbase() + /etc
and so on.

Goal: when the variable is unset, G_etc_dir(), G_etcbin_dir(),
G_colors_dir(), G_fonts_dir() and G_locale_dir() fall back to GISBASE
plus a path relative to it that is fixed at build time. They stop with the
existing fatal error only when GISBASE is unset too. This goes in a separate
commit.

Changes

1. New private header lib/gis/resource_dirs.h. It defines the default
   relative paths, each guarded with #ifndef, using the legacy layout. This
   is the only layout Autotools produces, so lib/gis/Makefile stays
   unchanged:
   GRASS_ETCDIR_REL "etc", GRASS_ETCBINDIR_REL "etc",
   GRASS_COLORSDIR_REL "etc/colors", GRASS_FONTSDIR_REL "fonts",
   GRASS_LOCALEDIR_REL "locale".
2. lib/gis/resource_dirs.c.
   • get_g_env(env_var, rel_path, ...) returns the variable when it is set
     and non-empty.
   • Otherwise, if GISBASE is set, it returns "<GISBASE>/<rel_path>". The
     path is built once per getter and cached in a static, guarded by the
     existing thread-safe G_is_initialized()/G_initialize_done() (as in
     lib/gis/datum.c). The getters keep returning const char * to static
     storage, so callers don't change.
   • Otherwise it calls the existing G_fatal_error.
   • It reads GISBASE with getenv(), not G_gisbase(). G_getenv() is
     fatal when unset and goes through gisrc, which would change the error.
3. lib/gis/locale.c G_init_locale(). The same fallback goes in a local
   buffer, built inline. It must not call G_locale_dir(), because the
   fatal-error recursion comment there still applies. This keeps translations
   working for GISBASE-only callers, as on main.
4. lib/gis/CMakeLists.txt. For ETCDIR ETCBINDIR COLORSDIR FONTSDIR LOCALEDIR, compute
   file(RELATIVE_PATH rel "/${GISBASE_DIR}" "/${GRASS_INSTALL_<DIR>}") and
   append -DGRASS_<DIR>_REL=\"${rel}\" to grass_gis_DEFS. With FHS this
   gives values like ../../share/grass/etc, and with legacy it gives etc.
   The build/output tree mirrors the install layout, so this works there too.
5. Test lib/gis/tests/lib_gis_resource_dirs_test.py. It uses the
   existing xy_session_for_module fixture, copies session.env without the
   GRASS_*DIR variables (keeping GISBASE), and runs tools that reach each
   main getter: r.colors -l (colors dir) and g.list type=raster (etc dir,
   element_list). It asserts that they succeed.

Verification

• Autotools, existing build. Run make -C lib/gis and relink the affected
  tools if needed.
  • With env -i and only GISBASE, GISRC, PATH and LD_LIBRARY_PATH
    set, run r.colors -l, g.list type=raster and
    g.proj -c epsg=4326 project=....
  • First run them with the old library (rebuilt from HEAD) to confirm the
    GRASS_ETCDIR not set failure, then with the new one to confirm success.
  • Also check that with GISBASE unset as well, the error is still the
    "Incomplete GRASS session" fatal error rather than a crash.
• CMake with FHS. Run cmake -B build-fhs -DWITH_FHS=ON with optional
  dependencies off, then cmake --build build-fhs --target grass_gis r.colors g.list. Run the same commands with GISBASE=build-fhs/output/<GISBASE_DIR>
  and no GRASS_*DIR, and check (with strings or a debug run) that the
  defines resolve to the share/.../etc tree.
• Tests. Run pytest lib/gis/tests python/grass/script/tests/grass_script_setup_test.py
  and pre-commit (clang-format, gersemi) on the changed files.
• Locale. No translations are built here and no non-C locale is
  installed, so translation loading may not be checkable. If it isn't, report
  that it wasn't verified.

Deliverables

• Separate commit libgis: Fall back to GISBASE for unset resource directories,
  pushed to origin claude/wonderful-shannon-g87w2g.
• A PR comment for C-savvy maintainers, written to
  <scratchpad>/pr5630_comment.md and also shown in the reply. It covers the
  root cause (the runtime_env_is_active() shortcut), both Python changes,
  the C fallback design (build-time relative paths from CMake
  file(RELATIVE_PATH), legacy defaults for Autotools, caching via
  G_is_initialized, getenv vs G_gisbase, the locale.c recursion
  constraint), what was verified (Autotools and CMake FHS on Linux), and what
  wasn't (Windows CI, translation loading).

Details (result of the second plan)

The plan for the C fallback is ready for your review.

First I'll reproduce the C-level failure with the current library: run tools directly with only GISBASE and GISRC set.

All three fail as expected. Now I'll implement the fallback, starting with the initialization helpers it will reuse:

The mutex being recursive means a getter can safely be called from inside another G_is_initialized() section, like from G_read_datum_table. Next I'll check the CMake minimum version to decide between cmake_path and file(RELATIVE_PATH).

Now the C changes:

G_getenv() special-cases GISBASE to read it with getenv(), so the only reason to avoid G_gisbase() is which variable the error message names. The comment says exactly that. Now locale.c:

Now the CMake defines:

Rebuilding lib/gis (Autotools) and rerunning the direct-tool reproduction:

All three tools now work with only GISBASE set. I'll check warnings under -Wall -Wextra for the two changed files, and the error path when GISBASE is also unset:

That check printed nothing because rc was taken from head. Rerunning it properly:

It exits with code 1 but prints no message. I'll check whether that's my change or existing behavior:

log_error() in error.c calls G_gisbase(), which is itself fatal without GISBASE, so the original message may be swallowed. I'll confirm with a debugger:

The fatal error from get_g_env() is pre-existing behavior in error.c unrelated to this change, since log_error() fails via G_gisbase() without GISBASE set. Moving on to the regression test now.

They pass. To confirm the new tests catch the bug, I'll run them against the old library:

The tests fail on the old library and pass on the new one. Now the CMake FHS build, to check the relative paths that cmake_path computes:

The defines are correct for FHS. Building the library and the test tools:

Meanwhile, I'll run pre-commit on the C, CMake and test files:

Formatting is applied and a second pre-commit run is clean. I'm waiting for the CMake FHS build.

While that builds, I'll draft the PR comment.

The CMake FHS build finished. I'll check the output tree and run the same tools:

With FHS, libexec/grass/etc doesn't contain element_list or colors; they're in share/grass/etc. So this run exercises the computed relative paths, not the legacy defaults:

CMake FHS works too, and the library contains the ../../share/... paths. Committing and pushing:

The comment says the grass.script and grass.tools suites pass, but I last ran those before the C change. Rerunning them on the final build:

I added the C fallback as a separate commit, 3c8cf2d1, pushed to origin/claude/wonderful-shannon-g87w2g after the Python fix (8af2d2a7). Tools run directly with only GISBASE and GISRC set now work, with both Autotools and a CMake FHS build. I haven't run the Windows CI.

What changed

• lib/gis/resource_dirs.c: when a GRASS_*DIR variable is unset or empty, get_g_env() now returns GISBASE plus a path relative to it. That path is built on the first call and then cached. It still calls G_fatal_error() when GISBASE is also unset.
• lib/gis/resource_dirs.h (new): default relative paths for the legacy layout, the only one Autotools builds, so the Makefiles are unchanged.
• lib/gis/CMakeLists.txt: computes the relative paths from the install layout. With FHS that gives, for example, ../../share/grass/etc.
• lib/gis/locale.c: G_init_locale() gets the same fallback, so translations keep working for these callers as they do on main.
• lib/gis/tests/lib_gis_resource_dirs_test.py (new): runs r.colors -l and g.list without the GRASS_*DIR variables.

Verification

• Autotools: with only GISBASE, GISRC, PATH and LD_LIBRARY_PATH set, r.colors -l, g.list type=raster and g.proj -c epsg=4326 all failed with "not set" before the change and pass after it.
• CMake with FHS: the same three commands pass from build/output. In that layout libexec/grass/etc has no element_list or colors, so this proves the computed share/... paths are used rather than the defaults.
• Tests: the new tests fail with the old library and pass with the new one. 569 tests pass across grass.script, grass.tools, grass.app and lib/gis, with 1 skipped and 1 expected failure. Two numpy-based files weren't run because numpy isn't installed in the pytest environment here. Pre-commit is clean (clang-format and gersemi reformatted the new code).
• Not verified: that translations actually load through the locale fallback, because there are no translation files or non-C locales in this build.

Existing issue I didn't change: if GISBASE is also unset, the tool still stops with a fatal error, but prints nothing and exits with status 1. A debugger trace shows why: the error-logging code in error.c calls G_gisbase(), which fails a second time and swallows the original message.

PR comment: the draft is in pr5630_comment.md in the scratchpad (sent above). It's written for C maintainers and covers:

• the root cause and both Python changes;
• the C design: build-time relative paths, caching under the existing recursive mutex, why it reads GISBASE with getenv(), and the locale recursion constraint;
• what was and wasn't verified.

It says "pytest suites pass" without mentioning the two skipped numpy files. Adjust that if you want it exact. It also ends with a line saying the work was done with the help of Claude Code.

Details (summary comment created for second plan, addressed for a C/C++ expert like Nicklas, but contains both sections)

I looked into the Windows failure (Incomplete GRASS session: Variable 'GRASS_ETCDIR' not set). It is not caused by copying dist.* instead of make install: the copied etc/python/grass/app/resource_paths.py already has the relative values (GRASS_ETCDIR = "etc", etc.), and grass.py gets @GRASS_PYDIR@ the same way as on main.

Root cause. Each pytest step calls env.bat, which sets only GISBASE, and then puts %GISBASE%\bin on PATH. gs.create_project() calls ensure_runtime_env(), which skips setup_runtime_env() because runtime_env_is_active() only checks that GISBASE is set and appears in PATH. So RuntimePaths.set_env_variables() never runs, and g.proj stops in get_g_env() (lib/gis/resource_dirs.c) on its first G_etc_dir() call. Linux and macOS CI never set GISBASE beforehand, so they always run the full setup. It reproduces on Linux with an Autotools build copied elsewhere and only GISBASE, PATH and PYTHONPATH set.

The same happens to anything that sets only GISBASE, not just CI: Python scripts that set the runtime up by hand, and programs that run tools directly with GISBASE and GISRC (e.g., the QGIS provider). On main, GISBASE alone was enough. Adding GRASS_ETCDIR to the workflow would only hide this, so I fixed it in the libraries instead. Two commits on top of this branch:

echoix/grass@8af2d2a7 (grass.script)

• runtime_env_is_active() also requires all the variables RuntimePaths manages (new RuntimePaths.env_variable_names()), so a GISBASE-only environment gets the full setup.
• setup_runtime_env() without gisbase now uses env["GISBASE"] as the prefix for RuntimePaths, the same way grass.py passes prefix=gisbase. Before, set_env_variables() took the build-time GRASS_PREFIX, which replaced the caller's GISBASE and pointed the GRASS_*DIR variables at the build tree. That is wrong for a relocated install.

echoix/grass@3c8cf2d1 (libgis)

• get_g_env(env_var, rel_path, &initialized, &fallback) returns the variable when it is set and non-empty. Otherwise it returns GISBASE "/" rel_path, and it still calls G_fatal_error() when GISBASE is unset too. The fallback path is built once per getter into a static char * (G_store), guarded by G_is_initialized()/G_initialize_done(). counter.c uses a recursive mutex, so callers that are already inside an init section (e.g., G_read_datum_table() calling G_etc_dir()) don't deadlock. The getters still return const char * to static storage, so no caller changes.
• It reads GISBASE with getenv(), not G_gisbase(), so the fatal message names the missing resource variable rather than GISBASE.
• The paths relative to GISBASE (GRASS_{COLORS,ETCBIN,ETC,FONTS,LOCALE}DIR_REL) are in a new private header lib/gis/resource_dirs.h. Its #ifndef defaults are the legacy layout, the only one Autotools produces, so the Makefiles are unchanged. CMake computes them in lib/gis/CMakeLists.txt with cmake_path(RELATIVE_PATH GRASS_INSTALL_<DIR> BASE_DIRECTORY ${GISBASE_DIR}), which is lexical. With WITH_FHS=ON this gives e.g. GRASS_ETCDIR_REL="../../share/grass/etc" and GRASS_ETCBINDIR_REL="etc". build/output has the same layout, so this works there too.
• G_init_locale() gets the same fallback inline in a local buffer. It still doesn't call G_locale_dir(), for the recursion reason noted there. This keeps translations working for GISBASE-only callers as on main.

Verified on Linux:

• Autotools: with env -i and only GISBASE, GISRC, PATH and LD_LIBRARY_PATH, r.colors -l, g.list type=raster and g.proj -c epsg=4326 failed before the change and pass after it.
• CMake with WITH_FHS=ON: the same commands with GISBASE=build/output/libexec/grass and no GRASS_*DIR pass. There, libexec/grass/etc has no element_list or colors, so this exercises the computed ../../share/grass/... paths, not the defaults.
• New tests: python/grass/script/tests/grass_script_setup_test.py (GISBASE-only env, GISBASE as prefix) and lib/gis/tests/lib_gis_resource_dirs_test.py (tools without the GRASS_*DIR variables). They fail without the changes and pass with them. The grass.script, grass.tools, grass.app and lib/gis pytest suites pass.

Not verified: the Windows CI itself (the fork doesn't run OSGeo4W on these branches), and translation loading through the locale fallback (no catalogs or non-C locales in my build).

A pre-existing issue, not changed here: if GISBASE is also unset, get_g_env() does reach G_fatal_error(), but log_error() in error.c calls G_gisbase(), which fails again, so the tool exits with status 1 and prints no message.

The investigation and patches were done with the help of Claude Code.

I made the branch with the fixes run in a PR on my fork, to see if it did the right thing when actually running on Windows, not only reproducing on Linux and fixing it. It runs at echoix#731

[nilason]

Thanks @echoix, a lot to read, and I will. But as I mentioned above, first and foremost the installation on CI must be fixed and the hacks removed. Only then can we make full use of the Win runner, and address issues properly and not add more hacks. I'll try to fix that first.

[echoix]

I know it's a lot, but I wanted you to be able to judge correctly without hiding anything that I saw.

Do you know if some parts of this kinda large could be merged independently and like now, in a more timely matter? It's been a long time since this work has been opened, and if I could be able to individually review something I'm certain of, it could get pieces in early

[wenzeslaus]

The Python changes make sense: The ensure env code was not made for FHS. The C code looks like it should be a different PR. Gathered from reading the text.

[nilason]

Now I took a closer look at the Claude analysis and suggestions.

It was as I feared, it just adds another layer of hacks. First of all it starts out with the premise that the current way Windows is setup and run is the correct way; that GISBASE is supposed to be set manually (via env.bat file). The whole point of this PR is to "re-define" or degrade the importance of GISBASE because it is not the same on legacy and FHS build. Everything that is needed is located in the python/grass/app/[resource_paths.py|runtime.py] combo. For very special cases (Conda?) we may need to allow manual setting of GRASS_PREFIX (which in legacy build is the same as GISBASE). In a normal GRASS session the resource paths are loaded with grass.py, now what we need is to load it with python scripts too.
Currently setup_runtime_env(gisbase=None, *, env=None) takes as argument gisbase, but it should only set up via runtime.py.

[nilason]

Seems the python suggestion at least make the Win runner pass. What do you say, shall I add it (with added TODOs to remove/rework) and merge this beast?
I’ll then open an issue regarding fixing Win runner properly and perhaps another to add warning or prevent setting GISBASE manually.

(This will have consequences for third parties like QGIS which depend on GISBASE.)

[echoix]

@wenzeslaus Can fable take a look

[nilason]

It’s my understanding that setup.py needs a rework, including delegate relevant parts to runtime.py. That might just as well be addressed in a follow-up PR.

[neteler]

(in case of too long AI outputs, maybe try https://github.com/ayghri/i-have-adhd, a "skill to stop your coding agent from burying the answer")

[nilason]

Tests are all green, I believe we should merge.

[wenzeslaus]

I can't look in detail right now, not sure if it is worth looking superficially. I'm fine either way, this is WIP even after merge.

[nilason]

I can't look in detail right now, not sure if it is worth looking superficially. I'm fine either way, this is WIP even after merge.

There's a lot to do still for FHS, this is only one big and crucial (non-breaking) step.

[nilason]

An approval is needed for bringing this in bay. I’m sure things will need to be worked out, I’m having a PR dropping hardcoded GRASS_PYDIR/GRASS_PREFIX up the sleeve. Good to get this moving forward.

[echoix]

Go ahead, and we’ll work from there

[nilason]

While working on the FHS successor PR, I hit some minor issues that belong to this. Please re-approve.

(We will need a FHS runner soon, I'm thinking of replacing the "GCC C/C++ standards check" could be an option. I noticed it often only runs way after one of the tests, not seldom the Mac runner, already finished. So it doesn't add much value anymore).

[echoix]

Here

[nilason]

build: decouple resource paths from GISBASE in preparation for FHS

Traditionally, GRASS resources have been located relative to GISBASE,
assuming a single installation directory containing executables,
libraries, configuration files, documentation, and other resources.

This assumption is no longer valid with installations following the
Filesystem Hierarchy Standard (FHS), where resources are distributed
across directories such as bin, lib, share, and etc.

Introduce dedicated environment variables for locating GRASS resources
independently of GISBASE, and update the C, Python, and GUI code to
use these paths instead of constructing them relative to GISBASE.

This allows the same resource lookup mechanisms to support both the
traditional installation layout and FHS-compliant installations,
without relying on symbolic links to recreate the legacy directory
structure.

These changes are part of the transition towards FHS-compliant
installations using CMake, as discussed in #3661.

Co-authored-by: Vaclav Petras wenzeslaus@gmail.com

[nilason]

Thank you!