---
license: mit
language:
- en
tags:
- remote-sensing
- earth-observation
- self-supervised-learning
- satellite
- multispectral
- feature-extraction
- vision
- satmae
- satmae-pp
- vit
- mae
- transformers
library_name: transformers
pipeline_tag: feature-extraction
datasets:
- fMoW
---

# SatMAE++ Transformers Models

Hugging Face–compatible checkpoints converted from the official [SatMAE++](https://arxiv.org/abs/2403.05419) pretrained weights. Each subfolder is a standalone model repo layout (`config.json`, `model.safetensors`, preprocessor, and remote code) for feature extraction on FMoW satellite imagery.

## Model Description

These models are ViT-Large encoders pretrained with multi-scale masked autoencoding (SatMAE++) on [fMoW-RGB](https://github.com/fMoW/dataset) and [fMoW-Sentinel](https://github.com/sustainlab-group/SatMAE) imagery.

This collection currently bundles **2 converted pretrain checkpoints**:

- **FMoW-RGB:** vanilla ViT-L/16, 3-channel BGR, 224×224
- **FMoW-Sentinel:** grouped-channel ViT-L/8, 10-band multispectral, 96×96

All folders ship self-contained remote code (`modeling_satmae_pp.py`, processor, pipeline) and load with `trust_remote_code=True`.

**Developed by:** [techmn / SatMAE++](https://github.com/techmn/satmae_pp)  
**Converted for Hugging Face by:** BiliSakura  
**License (weights):** MIT  
**Original paper:** [Rethinking Transformers Pre-training for Multi-Spectral Satellite Imagery](https://arxiv.org/abs/2403.05419) (CVPR 2024)

## Available checkpoints

| Folder | Dataset | Encoder | Channels | Image | Patch | Legacy file |
|--------|---------|---------|----------|-------|-------|-------------|
| `satmae-pp-vit-large-patch16-fmow-rgb-pretrain` | FMoW-RGB | vanilla ViT | 3 (BGR) | 224 | 16 | `checkpoint_ViT-L_pretrain_fmow_rgb.pth` |
| `satmae-pp-vit-large-patch8-fmow-sentinel-pretrain` | FMoW-Sentinel | group-channel ViT | 10 | 96 | 8 | `checkpoint_ViT-L_pretrain_fmow_sentinel.pth` |

Legacy `.pth` filename mapping is in [`conversion_manifest.json`](conversion_manifest.json).

## Usage

```python
from transformers import pipeline
import numpy as np

REPO = "BiliSakura/SATMAE-PP-transformers"
SUBFOLDER = "satmae-pp-vit-large-patch8-fmow-sentinel-pretrain"

pipe = pipeline(
    task="satmae-pp-feature-extraction",
    model=REPO,
    trust_remote_code=True,
    model_kwargs={"subfolder": SUBFOLDER},
)

# FMoW-Sentinel: 10 bands after dropping bands 0, 9, 10
image = np.random.randint(0, 255, (96, 96, 10), dtype=np.uint8)
features = pipe(image, pool=True, return_tensors=True)
print(features.shape)  # [1, 1024]
```

FMoW-RGB (BGR channel order):

```python
SUBFOLDER = "satmae-pp-vit-large-patch16-fmow-rgb-pretrain"
pipe = pipeline(
    task="satmae-pp-feature-extraction",
    model=REPO,
    trust_remote_code=True,
    model_kwargs={"subfolder": SUBFOLDER},
)
image = np.random.randint(0, 255, (224, 224, 3), dtype=np.uint8)
features = pipe(image, pool=True, return_tensors=True)
print(features.shape)  # [1, 1024]
```

Load components directly:

```python
from transformers import AutoModel, AutoImageProcessor

model = AutoModel.from_pretrained(REPO, subfolder=SUBFOLDER, trust_remote_code=True)
processor = AutoImageProcessor.from_pretrained(REPO, subfolder=SUBFOLDER, trust_remote_code=True)
```

## Normalization

The bundled image processor applies per-channel FMoW mean/std normalization by default (`do_normalize=True`). FMoW-RGB models expect **BGR** channel order; the processor swaps RGB→BGR when `channel_order="bgr"`.

For FMoW-Sentinel, inputs should be 10-band reflectance arrays (bands 0, 9, 10 already dropped) with FMoW-Sentinel statistics baked into the preprocessor config.

## Dependencies

- `transformers`, `torch`, `timm`, `safetensors`
- `opencv-python` (multispectral resize with more than 4 channels)

## Citation

```bibtex
@inproceedings{satmaepp2024rethinking,
  title={Rethinking Transformers Pre-training for Multi-Spectral Satellite Imagery},
  author={Mubashir Noman and Muzammal Naseer and Hisham Cholakkal and Rao Muhammad Anwar and Salman Khan and Fahad Shahbaz Khan},
  year={2024},
  booktitle={CVPR}
}
```