This repository is the official implementation of "SAREO-FM: Decoupled Semantic Supervision for SAR-EO Foundation Models".
SAREO-FM is a unified MAE-based foundation model that learns from raw SAR and EO imagery with a shared encoder and supports SAR-only, EO-only, and joint SAR–EO transfer.
- 🧩 Decoupled Semantic Supervision: Alignment with a frozen EO vision foundation model (DINOv3) is assigned to learnable semantic queries, and masked pixel reconstruction to the SAR and EO modality tokens, while all three streams still interact within the shared encoder.
- 🛰️ Mixed Modality Masking: Independent masking, spatially shared masking, and complete dropping of either modality expose the same encoder to joint SAR–EO, SAR-only, and EO-only inputs during pretraining on the million-scale SAR-1M corpus.
- 📈 Label-Efficient Transfer: In controlled comparisons, SAREO-FM is best in five of the six SAR fine-tuning settings (+9.90 points over SARMAE* on out-of-corpus 40-shot SAR-ACD), surpasses its frozen DINOv3 teacher at every shot count on few-shot EuroSAT with EO input, and reaches 66.70 micro-AP on 5-shot BigEarthNet-MM with joint SAR–EO input, 2.28 points above its best unimodal input.
- Oct 2026: This repository is created.
Figure 1. Motivation of SAREO-FM. Prior methods directly align image tokens with EO-derived VFM features, coupling semantic alignment with pixel reconstruction. SAREO-FM instead supervises learnable semantic queries while reconstructing SAR and EO from modality tokens, separating the destinations of the two objectives within a unified encoder.
Figure 2. Overview of SAREO-FM. Visible SAR and EO tokens interact with learnable semantic queries through a unified encoder. The modality-token outputs are decoded independently for masked pixel reconstruction, whereas the semantic-query outputs are aligned patch-wise with features from a frozen pretrained VFM. Semantic queries are excluded from the reconstruction decoders, separating the destinations of pixel-level and semantic supervision. Modality masking exposes the encoder to SAR-only, EO-only, and joint SAR–EO inputs; during downstream transfer, task heads selectively consume modality-token outputs, semantic-query outputs, or both.
Downstream heads use modality-token features (mod), semantic-query features (sem), or their concatenation (both). SARMAE* is the official SARMAE ViT-B checkpoint re-evaluated under exactly the same downstream protocol as SAREO-FM.
Table 1. SAR target classification with end-to-end fine-tuning. The upper block reports published OA (%) under the source-specific protocol of each method and is included only as context. The lower block reports our controlled evaluation with identical data splits and downstream pipelines; mean and standard deviation over three seeds are shown. * denotes our re-evaluation of the official SARMAE checkpoint. Bold denotes the best controlled result.
| Method | Venue | Backbone | FUSAR-Ship 40-shot |
FUSAR-Ship 30% |
MSTAR 40-shot |
MSTAR 30% |
SAR-ACD 40-shot |
SAR-ACD 30% |
|---|---|---|---|---|---|---|---|---|
| Published results | ||||||||
| ResNet-50 | CVPR'16 | ResNet-50 | – | 58.41 | – | 89.94 | – | 59.70 |
| Swin Transformer | ICCV'21 | Swin-B | – | 60.79 | – | 82.97 | – | 67.50 |
| BEiT | ICLR'22 | ViT-B | 59.70 | 71.13 | 40.70 | 69.75 | – | 79.77 |
| CROMA | NeurIPS'23 | ViT-B | 83.71 | – | – | – | – | 88.99 |
| SAR-JEPA | ISPRS JPRS'24 | ViT-B | 85.80 | – | 91.60 | – | 75.50 | – |
| SARATR-X | TIP'25 | HiViT-B | 87.70 | – | 98.10 | – | 76.40 | – |
| LoMaR | WACV'25 | ViT-B | 82.70 | – | 77.00 | – | 67.40 | – |
| SUMMIT | IJAEOG'25 | ViT-B | 81.50 | 71.91 | 63.60 | 98.39 | 68.70 | 84.25 |
| Copernicus-FM | ICCV'25 | ViT-B | 87.61 | – | – | – | – | 92.63 |
| CoDe-MAE | arXiv'26 | HiViT-B | 89.40 | – | 98.70 | – | 78.30 | – |
| MaRS | AAAI'26 | SwinV2-B | 77.70 | – | 75.50 | – | 68.40 | – |
| SARMAE | CVPR'26 | ViT-B | 89.30 | 92.92 | 96.70 | 99.61 | – | 95.06 |
| SARMAE | CVPR'26 | ViT-L | 90.86 | 92.80 | 97.24 | 98.92 | – | 95.63 |
| Controlled evaluation | ||||||||
| Random initialization | – | ViT-B | 60.31±1.39 | 82.11±0.52 | 34.71±0.09 | 38.76±8.35 | 33.14±2.60 | 39.19±10.27 |
| SARMAE* | CVPR'26 | ViT-B | 90.09±0.36 | 92.82±0.21 | 94.54±1.72 | 97.18±0.32 | 68.50±2.60 | 95.01±0.85 |
| SAREO-FM (sem) | – | ViT-B | 91.05±0.16 | 93.13±0.14 | 97.57±0.42 | 99.13±0.54 | 78.40±1.24 | 93.34±0.55 |
| SAREO-FM (both) | – | ViT-B | 91.41±0.23 | 93.12±0.06 | 96.66±0.31 | 99.36±0.47 | 78.00±1.00 | 92.84±0.68 |
Figure 3. Frozen few-shot SAR transfer. We use fixed encoder features and train only the downstream classifier.
Exact values behind Figure 3 (Table 6 of the Appendix)
Table 6. Exact frozen few-shot SAR classification results corresponding to Fig. 3 of the main paper. We freeze the pretrained encoder and optimize only a linear downstream classifier. We report mean OA (%) and sample standard deviation over five matched support draws. sem uses semantic-query features, whereas both concatenates modality-token and semantic-query features. SARMAE* denotes the official checkpoint evaluated with the same downstream protocol. Bold and underline denote the best and second-best values within each dataset and shot count, respectively.
| Dataset | Method / Feature | 1-shot | 2-shot | 5-shot | 10-shot | 20-shot | 40-shot |
|---|---|---|---|---|---|---|---|
| FUSAR-Ship | SARMAE* | 54.19±8.69 | 64.95±6.93 | 70.03±3.48 | 78.18±3.28 | 83.44±2.03 | 88.35±0.83 |
| SAREO-FM (both) | 55.56±7.50 | 64.79±3.81 | 72.84±3.97 | 80.46±1.81 | 85.32±1.79 | 89.81±0.70 | |
| SAREO-FM (sem) | 55.79±9.07 | 68.54±4.35 | 75.74±2.46 | 81.65±2.91 | 85.50±1.71 | 89.56±1.22 | |
| SAR-ACD | SARMAE* | 35.09±7.77 | 39.56±6.52 | 47.12±3.16 | 51.68±4.08 | 60.30±2.86 | 70.11±1.82 |
| SAREO-FM (both) | 33.81±6.30 | 36.31±6.91 | 44.78±4.56 | 51.87±4.12 | 60.69±2.75 | 71.35±2.01 | |
| SAREO-FM (sem) | 33.64±6.95 | 36.87±5.83 | 48.10±3.02 | 53.65±3.19 | 61.65±3.35 | 71.72±1.74 |
Figure 4. Qualitative comparison on SAR-ACD. Representative test images misclassified by SARMAE* but correctly classified by SAREO-FM.
On the paired EuroSAT protocol, SAREO-FM is evaluated with frozen features from either SAR or EO input.
Table 2. Frozen few-shot EO transfer. We report accuracy (%) for SAR and EO inputs over five support draws. CoDe-MAE values use its separate 10-shot protocol and are included only as context.
| Method / Feature | SAR 1-shot |
SAR 5-shot |
SAR 10-shot |
SAR 40-shot |
EO 1-shot |
EO 5-shot |
EO 10-shot |
EO 40-shot |
|---|---|---|---|---|---|---|---|---|
| DINOv3 (teacher) | 38.17±4.61 | 50.72±2.68 | 56.01±1.81 | 64.35±1.56 | 59.51±3.53 | 75.96±3.15 | 83.34±1.42 | 90.88±0.65 |
| SARMAE* | 37.22±5.71 | 48.64±3.00 | 53.97±1.39 | 64.72±1.12 | 43.56±6.57 | 59.57±3.05 | 66.49±1.97 | 78.36±1.07 |
| CoDe-MAE | – | – | 59.88† | – | – | – | 81.18† | – |
| SAREO-FM (both) | 39.07±5.98 | 53.54±2.21 | 59.36±1.46 | 69.24±0.96 | 64.41±5.05 | 81.09±1.61 | 86.76±1.24 | 92.58±0.75 |
Table 3. Transfer on paired SAR–EO benchmarks. We report OA (%) on 5-shot So2Sat LCZ42 and micro-AP (%) on 5-shot BigEarthNet-MM. Bold and underlined values indicate the best and second-best results.
| Input | Method | Feature | So2Sat LCZ42 | BigEarthNet-MM |
|---|---|---|---|---|
| SAR+EO | Random initialization | mod | 40.56±2.25 | 54.29±2.30 |
| EO | DINOv3 (teacher) | mod | 54.25±3.87 | 64.73±1.43 |
| EO | SAREO-FM | mod | 58.14±2.90 | 64.42±1.44 |
| SAR | SARMAE* | mod | 28.96±1.68 | 57.44±0.69 |
| SAR | SAREO-FM | mod | 29.59±2.64 | 60.47±0.79 |
| SAR+EO | SAREO-FM | both | 58.79±4.19 | 66.70±0.58 |
Figure 5. Qualitative transfer on BRIGHT. Given pre-event EO and post-event SAR observations, SAREO-FM fusion recovers destroyed buildings that the corresponding single-modality models predict as intact or background.
Table 4. Cumulative component ablation. All variants use matched pretraining and evaluation protocols. We report 10-shot SAR linear-probing performance. Avg. denotes the arithmetic mean across the two benchmarks, while Δ measures the gain over the preceding configuration. Bold and underlined values indicate the best and second-best results.
| Pretraining configuration | MSTAR (OA↑) | FUSAR (OA↑) | Avg. (OA↑) | Δ |
|---|---|---|---|---|
| Random initialization | 26.26±1.86 | 47.86±4.89 | 37.06 | – |
| Plain MAE | 66.16±3.15 | 76.46±2.52 | 71.31 | +34.25 |
| + DINOv3 supervision | 73.49±2.31 | 79.90±2.49 | 76.70 | +5.39 |
| + Decoupled semantic supervision | 77.11±2.14 | 82.55±2.15 | 79.83 | +3.13 |
| + Raw EO modality (SAREO-FM) | 78.41±2.14 | 84.00±2.88 | 81.21 | +1.38 |
Please visit our project page for more results.
The code and pretrained models will be released soon.
- Pretraining code
- Pretrained models
- Downstream transfer code (fine-tuning and linear probing)
- Evaluation scripts
If you find SAREO-FM useful, please consider citing:
@article{do2026sareofm,
title={SAREO-FM: Decoupled Semantic Supervision for SAR-EO Foundation Models},
author={Do, Jeonghyeok and Kim, Munchurl},
journal={arXiv preprint arXiv:2610.09317},
year={2026}
}