---
license: cc-by-nc-4.0
language: en
tags:
  - vision
  - image-feature-extraction
  - sar
  - remote-sensing
  - synthetic-aperture-radar
  - masked-autoencoder
  - model-hub
library_name: transformers
pipeline_tag: image-feature-extraction
datasets:
  - Wenquandan777/SAR-1M
arxiv: 2512.16635
---

# SARMAE Transformers Checkpoints

Official Hugging Face Transformers-format releases of [SARMAE](https://arxiv.org/abs/2512.16635) ViT encoders, converted for native `transformers` inference with `trust_remote_code=True`.

| Resource | Link |
|----------|------|
| Paper | [2512.16635](https://arxiv.org/abs/2512.16635) |
| Training dataset | [Wenquandan777/SAR-1M](https://huggingface.co/datasets/Wenquandan777/SAR-1M) |
| Legacy PyTorch weights | [Wenquandan777/SARMAE](https://huggingface.co/Wenquandan777/SARMAE) |
| Source code | [GitHub](https://github.com/BiliSakura/SARMAE-transformers) |

## Available checkpoints

| Variant | Backbone | Stage | Hidden | Layers | Heads | Input |
|---------|----------|-------|--------|--------|-------|-------|
| [sarmae-vit-base-patch16-pretrain](https://huggingface.co/BiliSakura/SARMAE-transformers/tree/main/vit-base-patch16-pretrain) | ViT-B | pretrain | 768 | 12 | 12 | 224 |
| [sarmae-vit-large-patch16-pretrain](https://huggingface.co/BiliSakura/SARMAE-transformers/tree/main/vit-large-patch16-pretrain) | ViT-L | pretrain | 1024 | 24 | 16 | 224 |

## Installation

```bash
pip install transformers timm torch torchvision safetensors huggingface_hub
```

## Usage

Point `transformers.pipeline` or `AutoModel.from_pretrained` at a variant subfolder:

```python
from transformers import pipeline

pipe = pipeline(
    task="image-feature-extraction",
    model="BiliSakura/SARMAE-transformers",
    revision="main",
    trust_remote_code=True,
    model_kwargs={"subfolder": "vit-base-patch16-pretrain"},
)
features = pipe(sar_image, pool=True, return_tensors=True)
```

Or load a variant directly:

```python
from transformers import AutoModel

model = AutoModel.from_pretrained(
    "BiliSakura/SARMAE-transformers",
    subfolder="vit-base-patch16-pretrain",
    trust_remote_code=True,
)
```

Each variant folder is a self-contained model repository with:

- `config.json` (`auto_map`, `custom_pipelines`)
- `model.safetensors`
- `preprocessor_config.json`
- `modeling_sarmae.py`, `image_processing_sarmae.py`, `pipeline_sarmae.py`

## Convert legacy checkpoints locally

```bash
python scripts/convert_checkpoint.py models/SARMAE_vitb_checkpoint-last
python scripts/convert_checkpoint.py models/SARMAE_vitl_checkpoint-last
```

## Upload to this Hub repo

```bash
python scripts/upload_to_hub.py models/sarmae-vit-base-patch16-pretrain \
  --path-in-repo vit-base-patch16-pretrain
python scripts/upload_to_hub.py models/sarmae-vit-large-patch16-pretrain \
  --path-in-repo vit-large-patch16-pretrain
python scripts/upload_to_hub.py --hub-readme-only
```

## Citation

```bibtex
@misc{liu2025sarmaemaskedautoencodersar,
  title={SARMAE: Masked Autoencoder for SAR Representation Learning},
  author={Danxu Liu and Di Wang and Hebaixu Wang and Haoyang Chen and Wentao Jiang and Yilin Cheng and Haonan Guo and Wei Cui and Jing Zhang},
  year={2025},
  eprint={2512.16635},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2512.16635},
}
```

## License

[CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/)