[echoix]

About 20% faster pytests on Windows (considering the split), still no failures found after multiple
dozens of CI runs. (Note, there's other open PRs from me that would still need to be merged for the multiple retries validation to still hold strongly).

Description

Windows CI runs the pytest suite with a single worker and always in the same
order. Install pytest-xdist and pytest-randomly and mirror the parallel /
needs_solo_run split that pytest.yml already uses on Linux.

Motivation and context

xdist workers are long-lived processes which handle several test files each,
so state left behind by one test can affect a later one running in the same
worker. With a single worker and a fixed order, whichever test runs first
never changes, so a failure which depends on the order has no way to show up.
Randomizing the order is what makes that kind of failure surface.

Linux CI does not use pytest-randomly yet; it can be added there in a
separate step if this turns out to be useful.

How has this been tested?

Windows CI on this branch, and about 30 runs of the randomized job while
looking for an order-dependent failure on Windows. The results were identical
in all of them (the same pre-existing failures, none of them related to the
order), so the randomization does not add noise to the job.

Windows CI timing: xdist vs. single worker

Compared Windows CI (osgeo4w.yml) job data with pytest-xdist enabled (55
job records from 13 runs across this investigation's branches) against
single-worker pytest (118 job records spanning 11 months of main history,
2025-07-25 to 2026-06-21).

pytest step timing. xdist splits the old single step into two — a
parallel step plus a new needs_solo_run serial step, each paying its own
test-collection overhead — compared here against the combined total:

	n	mean	median	min	max
Single worker (main, before)	5*	15.1 min	13.3 min	10.9 min	19.5 min
xdist parallel step alone	53	10.7 min	10.9 min	6.7 min	13.7 min
xdist parallel + solo combined	8**	12.0 min	12.0 min	9.6 min	13.8 min

* GitHub's job-log/step retention (roughly 3-4 months) limits step-level
"before" data to the 5 most recent main runs; treat as indicative, not
definitive.
** Only the 8 full-scope, PR-triggered jobs ran both steps; 45
workflow_dispatch fishing-attempt jobs used earlier in this investigation
intentionally skipped the solo/gunittest/thorough steps to isolate the
parallel step, so they only contribute to the parallel-step-alone row.

Net: the pytest step is about 20% faster with xdist, even after paying for
a second collection pass.

Whole-job duration (compile + all test steps; not expected to move,
since pytest is a small slice of it):

	n	mean	median
Single worker (main, before)	114	84.5 min	84.4 min
xdist, full-scope PR jobs	5	84.0 min	90.3 min

No meaningful difference — the job is dominated by MSYS2 setup, compiling
GRASS, and the (unchanged) single-worker gunittest thorough-test step.

Timeouts / hangs. 0 GitHub Actions timed_out conclusions in either
group (0/53 xdist, 0/118 single-worker). A handful of cancelled jobs in
both groups (3/55 xdist, 4/118 single-worker) were checked individually and
are routine cancel-in-progress supersession from newer pushes to the same
branch, not hangs or timeouts.

---

Written with the help of Claude Code, under my review.

[echoix]

My original randomized order dozen reruns also included randomization of gunittest test files, but it was harder to review, so I took that out. But the good thing is, results were still stable even with a different file order.

[ninsbl]

Just wondering, if you set: OMP_NUM_THREADS: 1 would not that in a way deactivate OpenMP tests, or are tests with nprocs>1 stil testing the OPenMP path?

[echoix]

Just wondering, if you set: OMP_NUM_THREADS: 1 would not that in a way deactivate OpenMP tests, or are tests with nprocs>1 stil testing the OPenMP path?

We are doing this since San Michele this summer, in the other pytest-xdist test runs. The theory was that we had sometimes some tests that oversubscribed the CPU counts, or caused some kind of bottlenecks. We don’t seem to have hit as much of the sporadic failures like these. I could make an AI agent take a look and compare both. But maybe, only maybe, that’s what now makes pytest-xdist viable on Windows. The reason it wasn’t there before was more failures and longer test run times, but now it is faster