Skip to content

v0.2: consistent radiometry, metric labels, site-held-out evaluation, one inference path, tests and CI - #5

Merged
lbougan merged 23 commits into
mainfrom
feat/v0.2
Oct 29, 2026
Merged

lbougan merged 23 commits into
mainfrom
feat/v0.2

Conversation

@lbougan

@lbougan lbougan commented Oct 28, 2026

Copy link
Copy Markdown
Owner

Summary

v0.2 makes the pipeline consistent from pixels to map and makes its evaluation honest.

  • Radiometry (A1, A10): cloud mask from SCL at ingest, DNs to reflectance with the scene's scale and offset (offset -0.1 for processing baseline >= 04.00), NaN-median composites tagged with their band order (BANDSPEC).
  • One normalisation (A14): per-channel p2/p98 clip then mean/std from channel_stats.json, computed on training-AOI composites only; the same normalize() in training, evaluation, local inference and the ONNX export.
  • Metric labels and loss (A2, A3, A4): circles in metres (local AEQD), labels 0 / 0.5 ignore / 1 stored as GeoTIFFs, focal + Dice weighted per pixel (ignore ring weight 0), no label smoothing.
  • Site-held-out splits (A5, A6): test Satish Dhawan, Tanegashima, Guiana; val Xichang, Plesetsk, Sohae; 20 km exclusion; 17 AOIs so all 18 sites have imagery.
  • Evaluation (A7): evaluate.py reports recall@{5,10,20}, FP per 1000 km² and reliability on prob, and can write the README results block; checkpoints are selected on val/site_recall@10.
  • One code path each (A12, A13): one fit() training loop, one sliding window with a Predictor (local torch or Triton) and an optional memmap accumulator, a frozen Detection with separate prob and rank_score.
  • API and map (A8, A9): evidence thumbnails render from the stored composite (?mode=rgb|nir), MVT min_score, known-site seeding, prob exposed next to the rank score.
  • Serving (A11): ONNX export embeds the normalisation stats and band spec.
  • Removed: Sentinel-1, DualEncoderUNet, the async Triton client, dead dependencies.
  • Docs: results-first README with a pipeline diagram, docs/hardware.md (incl. the issue Local pipeline is too slow on 8-CPU CPU-only setups #1 decision: remote GPU via Triton instead of tuning the CPU path), updated model and dataset cards, scripts/gpu_day.sh.

The synthetic end-to-end test found and fixed a real bug: chip bboxes in UTM were read as lon/lat by the split assignment.

Tests

  • uv run pytest -q: 105 passed (unit + synthetic end-to-end on CPU, ~26 s); coverage 89.3 % against an 80 % gate.
  • uv run pytest -m api -q: 8 passed on PostGIS; API coverage 89 % (reported by the api job).
  • CI: lint-and-unit (ruff, pytest + coverage gate) and api (PostGIS service container).
  • Synthetic result (README): recall@5 = 1.00, FP 0.0 / 1000 km² on 26.2 km², 3 detections, 23 s on 16 CPU cores. This proves plumbing, not detection skill.

What the GPU day adds

scripts/gpu_day.sh ingests the 17 AOIs (--max-scenes 20), trains on a GPU (--precision 16-mixed, monitor on the validation composites), runs evaluate.py --split val --split test --update-readme README.md and exports ONNX for Triton. That fills the README results block with the first real-data numbers on the held-out sites.

Test plan

  • ruff check + format
  • pytest -q --cov (gate passes)
  • pytest -m api against local PostGIS
  • CI green on this PR
  • GPU day: run scripts/gpu_day.sh, commit README results and channel_stats.json
  • Tag v0.2.0 after merge

…moke test, predictor close, parity on NaN pixels
… needs val composites, skip site eval during sanity check
…he API, MVT min_score, known-site seeding, PostGIS API tests (A8, A9)
… % coverage gate

- Integration test: composite -> chips -> labels -> splits -> stats -> 2 CPU epochs ->
  LocalTorchPredictor -> evaluate on a composite 200 km east; recall@5 == 1.0.
- Fix found by it: chip bboxes are in the raster CRS (UTM for Sentinel-2) but
  assign_splits read them as lon/lat, so every chip went to train. chip_raster now also
  emits bbox_4326 and assign_splits uses it.
- tasks.py imports rasterio, sqlalchemy, the STAC client and torch-backed modules inside
  each task; importing the flows no longer loads torch (subprocess test).
- Unit tests for the pipeline task bodies; coverage omits libs/stac/client.py (network)
  and services/api/* (PostGIS, api job); CI drops --cov-fail-under=0.
…, api coverage report, clear error on old manifests

- fit: pl.seed_everything(cfg.seed, workers=True) before datasets and module are built
  (FitConfig.seed, default 0); replaces the late torch.manual_seed(0) that left model
  init unseeded.
- Integration test: logged duration, generous 300 s bound instead of the 120 s assert.
- assign_splits raises a clear ValueError on manifests without bbox_4326; dataset meta
  carries bbox_4326 when present.
- Lazy-import test also checks pytorch_lightning and sqlalchemy.
- api CI job reports services/api coverage (.coveragerc.api) and uploads coverage-api.xml.
- Unit test for the evaluate CLI (fake predictor, README block update).
… runbook

One default stats path (default_stats_path) for train_cli, evaluate and export_model; ingest --max-scenes passes through to ingest_aoi(limit=...); train_cli --precision. scripts/gpu_day.sh swaps in CUDA torch, passes the validation composites and uses the best checkpoint reported by training.
… and scanned area, fail-fast site validation, densified STAC bounds, lean deps
…ut AOIs fully held out, broadcast site metric, coverage gate on the api job
- /evidence/{id} no longer returns the server-side composite_path (shared
  public_evidence helper in schemas)
- export_model ships the checkpoint's own stats, checked against a given
  file via resolve_stats; --stats now defaults to None like evaluate
- detect flow defaults --stats to the local checkpoint's embedded stats and
  loads channel_stats.json only for Triton (detect_stats_path)
- a truncated crop from an interrupted ingest is re-downloaded instead of
  skipped on every run (reusable_crop)
- evaluate selects a split's composites by AOI (aoi_split), as the training
  monitor does, instead of by stray chip labels; --manifest -> --aoi-config
- site circles are reprojected once per (sites, CRS) across all chips of an
  AOI (lru_cache) instead of per chip
- memmap accumulator no longer writes 2 x H x W x 4 bytes of zeros eagerly
- web map draws the MVT polygon footprints at zoom >= 9
- synthetic planted-site generator moved to libs/synthetic so the benchmark
  imports a packaged module
- README: how to upgrade a database created before v0.2 (idempotent schema)
@lbougan
lbougan merged commit 19c830c into main Oct 29, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant