Repository navigation
Wire the unwired test suites into CI, and fix runANN #163
Description
TESTING.md states openly that only tests/runff is validated in CI. Current coverage:
| Suite | In CI? | Notes |
|---|---|---|
tests/runff |
✅ main.yml, macos.yml |
The only physics regression check |
tests/unit |
✅ (added in #156) | Model-level, 22 cases |
tests/python/test_wheel.py |
✅ wheels.yml |
Import and smoke test only |
tests/runANN |
❌ | Broken — see below |
tests/python/idealizedwind.py |
❌ | Also the best undocumented example |
tests/python/farsite_flat.py |
❌ | |
tests/python/percolation.py |
❌ | |
tests/mnh_ideal, tests/mnh_real_nested |
❌ | Need a Meso-NH build |
tests/run.bash |
❌ | Aggregate runner, never invoked by CI |
tests/runANN fails immediately, and has since it was committed
run.bash line 1 does:
../../bin/ANN_test Rothermel.ffann modelrun.csv print | grep result > result.txt
if ! diff result.txt result.txt.ref; thenresult.txt.ref is not in the repository. git ls-files tests/runANN/ returns five files and that is not one of them. Running it:
diff: result.txt.ref: No such file or directory
ANN results differs from reference.
It looks like result.txt was committed where result.txt.ref was meant — but a plain rename is not the fix. Running the test overwrites result.txt, and the regenerated output differs from the committed one in 13 of 1000+ lines, all last-digit float noise:
- result: 12412.1 observed 12412.1
+ result: 12412.2 observed 12412.1
So the check needs a numerical tolerance the way compare_nc.py already has, not an exact diff. As written it would be permanently red on any machine but the one that produced the file.
This matters more than it looks: runANN is the only coverage for ANNPropagationModel and BMapLoggerForANNTraining, which the unit suite in #156 cannot reach — both read a .ffann network in their constructor and abort when it is missing. Those two models are also the inverse-modelling path, one of the more distinctive things in the codebase.
On test_threading.py
It does not exist on dev or master — it arrives with the free-threading branch. So the free-threading work currently has no automated guard, and wiring it up should be part of landing that branch rather than a separate task.
Suggested approach
.github/workflows/invariants.yml (added in #158) is a good template — it builds pyforefire in a venv and runs a Python suite with no LFS fixtures.
In order of value: fix runANN and wire it; then the tests/python/ examples, which double as documentation and so are worth keeping executable; then decide whether mnh_* is testable at all without a Meso-NH build.
Drafted by Claude Opus 5 from a codebase audit. Reviewed by a maintainer before filing.
Activity
docs(tests): correct the test READMEs against the tree
docs(tests): correct the test READMEs against the tree
HugoFara commentedon Aug 14, 2026
Merged in #183, closing.