---
license: apache-2.0
language:
- en
tags:
- remote-sensing
- earth-observation
- satellite
- multispectral
- spatiotemporal
- foundation-model
- mae
- prithvi
- hls
- feature-extraction
- vision
- transformers
library_name: transformers
pipeline_tag: feature-extraction
---

# Prithvi-EO-2.0 Transformers Models

Hugging Face–compatible checkpoints converted from the official [Prithvi-EO-2.0](https://huggingface.co/ibm-nasa-geospatial/Prithvi-EO-2.0-300M-TL) foundation models (IBM, NASA, Jülich Supercomputing Centre). Each subfolder is a standalone model repo layout (`config.json`, `model.safetensors`, preprocessor, and remote code) for spatiotemporal feature extraction on HLS multispectral imagery.

## Model Description

Prithvi-EO-2.0 is a 3D ViT encoder pretrained with masked autoencoding on NASA HLS V2 (30 m, six bands: B02–B07). TL variants add learned temporal (year + Julian day) and location (lat/lon) embeddings.

This collection bundles **6 converted checkpoints**:

| Folder | Params | TL embeddings | Patch | Legacy file |
|--------|--------|---------------|-------|-------------|
| `prithvi-eo-v2-tiny-tl` | ~5M | yes | 16 | `Prithvi_EO_V2_tiny_TL.pt` |
| `prithvi-eo-v2-100m-tl` | ~100M | yes | 16 | `Prithvi_EO_V2_100M_TL.pt` |
| `prithvi-eo-v2-300m` | ~300M | no | 16 | `Prithvi_EO_V2_300M.pt` |
| `prithvi-eo-v2-300m-tl` | ~300M | yes | 16 | `Prithvi_EO_V2_300M_TL.pt` |
| `prithvi-eo-v2-600m` | ~600M | no | 14 | `Prithvi_EO_V2_600M.pt` |
| `prithvi-eo-v2-600m-tl` | ~600M | yes | 14 | `Prithvi_EO_V2_600M_TL.pt` |

All folders ship self-contained remote code (`modeling_prithvi.py`, processor, pipeline) and load with `trust_remote_code=True`.

**Developed by:** [IBM / NASA / JSC](https://huggingface.co/ibm-nasa-geospatial)  
**Packaged for Hugging Face by:** BiliSakura  
**License (weights):** Apache-2.0  
**Original paper:** [Prithvi-EO-2.0: A Versatile Multi-Temporal Foundation Model for Earth Observation Applications](https://arxiv.org/abs/2412.02732)

Legacy `.pt` filename mapping is in [`conversion_manifest.json`](conversion_manifest.json).

## Usage

Processors default to **`do_resize: false`**. Pass HLS reflectance values at native 224×224 (or another size divisible by the patch size); HLS mean/std normalization is applied by default.

### Custom pipeline (recommended)

```python
from transformers import pipeline
import numpy as np

REPO = "/home/czy/local/models/BiliSakura/Prithvi-EO-2.0-transformers"
SUBFOLDER = "prithvi-eo-v2-300m-tl"

pipe = pipeline(
    task="prithvi-eo-feature-extraction",
    model=REPO,
    trust_remote_code=True,
    model_kwargs={"subfolder": SUBFOLDER},
)

# Four HLS frames (T, H, W, C) in reflectance units — list of temporal frames
frames = [np.random.uniform(500, 3000, (224, 224, 6)).astype("float32") for _ in range(4)]
features = pipe(
    frames,
    pool=True,
    return_tensors=True,
    temporal_coords=[[2018, 26], [2018, 106], [2018, 201], [2018, 266]],
    location_coords=[19.5, -99.1],
)
print(features.shape)  # torch.Size([1, 1024])

# Dense token map (CLS + spatiotemporal patches)
tokens = pipe(frames, pool=False, return_tensors=True)
print(tokens.shape)    # torch.Size([1, 785, 1024]) for 224×224, patch 16, T=4
```

You can also pass a single array shaped `(C, T, H, W)` or `(T, H, W, C)`.

### Direct model loading

```python
from transformers import AutoModel, AutoImageProcessor
import torch

model_dir = f"{REPO}/{SUBFOLDER}"
model = AutoModel.from_pretrained(model_dir, trust_remote_code=True)
processor = AutoImageProcessor.from_pretrained(model_dir, trust_remote_code=True)

batch = processor(frames, return_tensors="pt")
with torch.no_grad():
    outputs = model(**batch, temporal_coords=batch["temporal_coords"], location_coords=batch["location_coords"])
print(outputs.pooler_output.shape)
```

### Standard `image-feature-extraction`

```python
pipe = pipeline(
    task="image-feature-extraction",
    model=f"{REPO}/{SUBFOLDER}",
    trust_remote_code=True,
)
```

## Normalization

The bundled image processor applies HLS pretraining normalization per band:

- **Mean:** `[1087, 1342, 1433, 2734, 1958, 1363]`
- **Std:** `[2248, 2179, 2178, 1850, 1242, 1049]`
- **Nodata:** `-9999` → `0.0001` before normalization

Band order: `B02, B03, B04, B05, B06, B07` (Blue, Green, Red, Narrow NIR, SWIR1, SWIR2).

## Re-converting checkpoints

```bash
conda activate rsgen
python /home/czy/local/models/BiliSakura/Prithvi-EO-2.0-transformers/convert_all_checkpoints.py
python /home/czy/local/models/BiliSakura/Prithvi-EO-2.0-transformers/test_prithvi.py --all
```

## Citation

```bibtex
@article{Prithvi-EO-V2-preprint,
    author  = {Szwarcman, Daniela and Roy, Sujit and others},
    title   = {{Prithvi-EO-2.0: A Versatile Multi-Temporal Foundation Model for Earth Observation Applications}},
    journal = {arXiv preprint arXiv:2412.02732},
    year    = {2024}
}
```