[Hendrik-code]

Why

A model trained on a narrow field of view has never been shown what lies outside it. On whole-body axial NAKO VIBE, the SpineGAN-trained models lose most of their Dice to false positives far from the spine. The contrast augmentations are not failing here: the network has never been asked to reject that context.

Diagnosis, measured on Ours (fp32) (paper config c69b9872 on 01bf0b3), NAKO VIBE test split:

vertebra instance precision, all four training sets, in-phase and water	0.01, median 76-125 false-positive components per subject (recall ~0.90)
FP voxels more than 50 mm from the reference spine (803 in-phase, n=10)	60%
FP voxels outside the body	1%
FP voxels by VibeSeg-100 class (803 in-phase / 804 water, n=15 each)	unlabelled 31/41%, muscle 14/21%, gluteals 9/6%, subcutaneous fat 8/4%, intestine 5/11%, femur 4/4%, sacrum 2/1%
Dice if only the largest connected component is kept (803 in-phase, n=15)	vertebra 0.729 → 0.804, IVD 0.616 → 0.696, canal 0.697 → 0.808

The cause is the training geometry. Preprocessed 803 has a median of 85 (LR) × 297 × 187 voxels at 1 mm, with a 128³ patch. The foreground leaves only 22 mm of LR free, so nnU-Net centres the slab and pads the rest:

• data is padded with 0;
• seg is padded with -1, which the tail then turns into 0;
• so every training window contains spine, and beyond ±42 mm laterally the network has only seen a flat slab labelled background.

What

RandomFOVExtensionGPU is a new GEO transform in smauglab/transforms/gpu/context.py. It runs directly after RandomAffineGPU and before every intensity transform. For each selected sample:

1. Shift. Image and mask move by the same whole-voxel offset, up to shift of the patch per axis. At shift: 1.0 the spine can leave the window entirely.
2. Fill region. Everything that is not scan: the padding (every channel exactly pad_value) plus whatever the shift vacated.
3. Synthesise it SynthSeg-style.
   • EM splits the patch's own background into a random 3-10 regions. Only voxels labelled 0, beyond margin of any label and off the padding seam are used.
   • A 3×3×3 majority filter and a random diffeomorphic field (SynthSeg's random_svf_field) post-process the region map.
   • layers randomly rotated, scaled, mirrored and translated copies of the map are composited into the fill.
   • Each region is rendered from its own mean and std, measured on the patch. The texture is spatially correlated (σ = 1 voxel), then blurred as SynthSeg blurs.
   • The fill is labelled 0.

Everything outside the fill region is the shifted input, bit for bit, and with shift: 0 only the padding changes.

Other details:

• Per-sample selection is pinned. Without a pipeline section the trainer forces same_on_batch=True onto every child, which would make p one coin per batch and could empty both samples of a step of spine.
• The EM is batched and weighted. It is a form of _em_gmm_1d with no boolean indexing and no host read.
• The EM moments use a broadcast product instead of resp.mT @ design: 0.04 ms against ~7.5 ms on this shape. Without it the transform cost 190 ms per batch.
  • synthseg.functional._em_gmm_1d has the same pattern and probably the same cost. It is untouched here and left for a separate PR.
• New config: transform_params_one-sequence-to-segment-them-all-fov.json (ae065824) is c69b9872 plus the transform at its defaults (shift 0.5, regions [3, 10], margin 4, layers 6, p 0.5) and nothing else.
• The README gets a "Field of view beyond the scan" section, and the matrix and all_augmentations.json are regenerated.

Checks

• pytest unit_tests -m "not slow" (CPU): 485 passed, 1574 subtests. main in the same venv: 468 passed, 1534 subtests.
  • The 17 new tests in test_fov_extension.py cover:
    • image and mask moving together, checked with a sentinel intensity under the label;
    • no foreground intensity reaching the fill;
    • bit-identity outside the fill;
    • the no-background case;
    • per-sample selection under same_on_batch=True;
    • seed determinism and bf16 autocast;
    • EM against two known populations.
  • The registry sweeps pick the transform up automatically.
• -m slow: 3 passed, 3 skipped (no GPU visible).
• Pre-commit hooks (ruff 0.16.1) pass. mypy smauglab/ reports no issues, run locally with mypy 2.1.0 (CI pins 2.4.0).
• Visual check on real 803 batches drawn through the trainer's own dataloader: the scan shifts with its labels, the padding is filled, and no label leaks into the fill.
• Cost, measured on a shared L40S:
  • the transform alone on both rows of a 2×128³ batch: ~40 ms;
  • the full fov config at p=0.5 against the paper config: 56 vs 26 ms per batch, including host-device copies.
  • Expect roughly +10% epoch time. The A40 number will come from the screen's training logs.

Known limits

• Background that the scanner itself zeroed normalises to a constant other than 0, so it counts as scan and stays. An example is the masked air of a Dixon reconstruction, which forms constant bands along some SpineGAN slab edges. nnU-Net's -1 marks only part of it, about 17% in the case checked, so -1 is not used.
• There is no femur or pelvis texture among the sources, so the ~5% of FPs on bone may persist.
• A label-to-image generator later in the pipeline (SynthSeg, PALETTE synthesis) overwrites the fill.
• The fill is always background, so a network could learn "synthetic texture = background" without transferring to real tissue. The screen below measures exactly that.

Evaluation (running; results will be added here before this leaves draft)

Dataset803, fold 0, A40, against ms0_base (the paper config on current main) and Ours (fp32):

arm	change from c69b9872
fov_fill	RandomFOVExtensionGPU with shift 0 (only the padding is filled)
fov_shift050	shift 0.5 (= the shipped config)
fov_shift100	shift 1.0 (spine-free windows are frequent)

• Datasets: NAKO VIBE (four parts, one cohort) and T2w, val for choosing, test for reporting, plus the paper cohorts.
• Pre-stated criteria:
  • the NAKO VIBE Dice gain exceeds the ms0_base vs Ours (fp32) replicate gap;
  • the gain survives against baseline + largest-connected-component post-processing;
  • paper-cohort Dice stays equivalent within ±0.01 (TOST).

🤖 Generated with Claude Code