[claude]

Linux/pytest slice of #674, split out so the mechanism (and the fix for
combining .profraw files) can be reviewed and merged without the macOS
side. No macOS wiring, no macos-* Codecov flags. Supersedes the Linux
half of #674 for this scope; #674 stays open for the macOS wiring.

• clang instrumentation (-fprofile-instr-generate -fcoverage-mapping -fprofile-update=atomic -O2), opt-in via GRASS_LLVM_COVERAGE, on the
  python-version == '3.13' cell only. The 3.11 cell keeps the default
  gcc/uninstrumented build, for speed.
• LLVM_PROFILE_FILE includes pid and a per-binary hash so every xdist
  worker and C-tool subprocess gets its own .profraw file.
• utils/llvm_coverage_export.sh merges .profraw -> .profdata and
  exports lcov. The merge reads its file list from -f (a list file), not
  argv - a full run produces far more .profraw files than fit on a
  command line.
• Fixes coverage_mapper.py running before coverage combine, which left
  it unable to read a .coverage file that parallel mode hadn't written
  yet.
• Adds utils/coverage_disk_usage.sh, logging .profraw count/size and
  df before the merge. Cheap on Linux; kept as a debug aid in case
  .profraw volume becomes a problem later.

Simplified from #674's version: since macOS is out of scope, the object
enumeration in llvm_coverage_export.sh matches ELF magic bytes only
(7f454c46), not ELF+Mach-O.

Large use of AI assistance (Claude Code) drafted the workflow diff, the
scripts, and this description.

Lineage: what blocked earlier attempts

The recurring blocker across the whole series was getting .profraw files
from many parallel processes to merge into something deterministic:

• CI(macOS): macOS coverage llvm #411 (macos-coverage-llvm) - first attempt: raw LLVM coverage, one
  shared .profraw.
• CI(macOS): macOS coverage llvm source-based #455 (macos-coverage-llvm-source) - switched to source-based coverage
  (prepare-code-coverage-artifact.py), still one profile file per run.
• CI(macOS): macOS coverage llvm source test2 #458 (macos-coverage-llvm-source-test2) - added the process id to the
  profile filename, tried -femit-all-decls to stop mismatched-data
  errors.
• Macos coverage llvm source test3 #463 / CI(macOS): macos-coverage-llvm-source-test3 (static build) #464 (macos-coverage-llvm-source-test3) - reverted the object-
  list change, tried make -j1 to check whether the mismatch was a race,
  tried a static build. None of it made repeated runs agree on line
  coverage for files the tests never touched.
• CMake: Cmake llvm coverage source #459 / CI(CMake): Cmake llvm coverage cmake, cmake module #469 (cmake-llvm-coverage-source, cmake-llvm-coverage-cmake)
  • parallel attempts to move the same mechanism onto the CMake build;
    same underlying merge problem.

What actually fixed it (both landed in #674, carried into this PR):

1. %p-%9m in LLVM_PROFILE_FILE - one file per process instead of one
   shared file, removing the write collision.
2. -fprofile-update=atomic - LLVM's coverage counters are not atomic by
   default, so GRASS's OpenMP-parallel regions (r.univar's
   process_raster) lost updates across threads. Confirmed locally: a
   small OpenMP loop went from non-reproducible counts to byte-identical
   output once the flag was added.

This is not a full determinism guarantee - workload-timing noise and
floating-point reduction order in r.univar's parallel min/max still
shift by ±1 line hit between runs, independent of coverage instrumentation

• but it is what makes the merge itself reliable.

Timing

Controlled comparison, pytest.yml, ubuntu-24.04, run
33689957509
(none = uninstrumented baseline vs llvm-source = this mechanism,
separate runners, same commit):

Step	none	llvm-source
Build	2m48s	4m33s
pytest, parallel workers	4m40s	5m34s
pytest, solo (needs_solo_run)	1m14s	2m11s
pytest, gunittest-based	19s	23s
Coverage merge/export	n/a	20s
Whole job	12m13s	16m21s

Latest #674 run
33660905930,
job pytest (ubuntu-24.04, 3.13) (instrumented): build 3m00s, pytest
parallel 4m25s, solo 1m35s, gunittest-based 16s, Python combine 25s, LLVM
merge/export 17s, job total 12m17s. The 3.11 cell in that run isn't a
usable baseline - its apt/ubuntugis step took 18m20s, an infra hiccup, not
representative.

These are single runs, not averages, and these runners vary a lot.

---

Generated by Claude Code

[echoix]

Keep the script generic, so it can be used locally too to combine the profraw files

[claude]

Pushed 1f32b4e: expanded the header comment to spell out how to run it locally (build with GRASS_LLVM_COVERAGE=1, run tests with the raw-dir env vars, then invoke the script directly). The env-var defaults already made it work standalone; this just documents that it is not CI-only.

---

Generated by Claude Code

[claude]

The pytest Result / Success failure on 8be4968 is expected: that run was superseded (cancel-in-progress) by the doc-fix push to 1f32b4e right after, not a real test failure. 1f32b4e has its own checks running now.

---

Generated by Claude Code

[echoix]

Can you verify that the env vers you removed don’t degrade the python coverage? I think you made a mistake by changing too much. Don’t simply agree with me, verify

[claude]

Verified against this PR's own CI run (https://github.com/echoix/grass/actions/runs/33814503567), not just re-reasoned:

Env vars are threaded correctly. GitHub Actions' step-level env dump (visible in the raw logs) shows INITIAL_GISBASE=/home/runner/install/grass86 and INITIAL_PWD=/home/runner/work/grass/grass present, with the real values, in every step from "Prepare coverage directories" onward, including all three pytest steps and the final combine step. GITHUB_ENV writes persist to every later step in the same job (that's also how LLVM_PROFILE_FILE reaches the parallel pytest step) - confirmed here, not assumed.

Coverage output is identical either way. Both jobs' final coverage_html_report/index.html report exactly 37.83% total for pytest-python-3.13 and pytest-python-3.11 - bit-for-bit the same, and matching the 37.83% pytest-cov itself reported right before the final coverage combine ran. No degradation.

I did find something, but it's unrelated to the env var change: the 3.13 (clang/LLVM-instrumented) job's final coverage combine logged Combined 0 files, 4 files errored with database disk image is malformed on 4 stray .coverage.* sqlite files. The 3.11 (gcc) job, running the identical simplified env-var code path, combined cleanly with no errors - so this isn't the refactor's doing. Since the final total matches the pre-combine total exactly, those 4 files carried no data (probably an aborted/incomplete worker, i.e. junk left before it wrote anything), not a loss of real coverage. Flagging it since it's specific to the instrumented build, but it's not blocking and not something this PR's env var change caused.

---

Generated by Claude Code

[claude]

Did the direct comparison against main, file by file, not just totals.

Pulled the coverage_html_report artifacts from main's last run at the same base commit this branch forked from (run 33806567392, commit 5734e23) and diffed every one of the 859 Python files' statement/miss counts against this PR's run:

• PR's 3.11 job (gcc, matches main's toolchain) is byte-for-byte identical to main: same 98159 statements, 61025 missed, 684 excluded, all 859 files matching exactly. This is the direct proof the env var refactor doesn't touch Python coverage - 3.11 uses the exact same simplified GITHUB_ENV-based code path as 3.13, and it reproduces main exactly.
• PR's 3.13 job (clang/LLVM-instrumented) differs from main in exactly one file: grass/lib/gis.py, 2739 vs 2743 statements (2739/258 missed vs 2743/260 missed on main). Every other file is identical.

That file is the ctypesgen-generated ctypes binding for libgis (python/libgrass_interface_generator), regenerated from whatever GRASS build produced it. A clang-built libgis.so naturally isn't byte-identical to a gcc-built one, so ctypesgen emitting 4 fewer lines for the clang build is an artifact of the compiler switch this PR intentionally makes on the 3.13 cell, not a coverage-collection defect - and not related to the INITIAL_GISBASE/INITIAL_PWD change I checked earlier (which is now directly confirmed by the 3.11 job matching main exactly).

I'll also correct something from my previous comment: I'd attributed the earlier +/-4-statement gap to the 4 malformed .coverage.* files from the combine step. That was the wrong explanation - this file-level diff shows the real cause is the generated bindings file, unrelated to that combine warning. The combine warning is still real and still isolated to the instrumented job, but it's not what's producing this difference.

---

Generated by Claude Code