---
license: cc-by-nc-4.0
language:
- ko
- en
pipeline_tag: image-segmentation
library_name: transformers
tags:
- semantic-segmentation
- image-segmentation
- remote-sensing
- aerial-imagery
- land-cover
- korean
- dinov3
- tcam
- vision-transformer
---

<p align="center">
  <img src="logo.png" alt="AIOne-GeoSeg" width="160"/>
</p>

<h1 align="center">AIOne-GeoSeg-330M</h1>

<p align="center">
  <b>A 330M-parameter Vision Transformer with our TCAM decoder for Korean land-cover semantic segmentation on aerial and satellite imagery.</b>
</p>

<p align="center">
  📄 <a href="#">Paper (coming soon)</a>
</p>

---

## Model Description

**AIOne-GeoSeg-330M** is a semantic segmentation model that classifies every pixel of a Korean aerial / satellite ortho-image into one of 11 land-cover categories (background, building, parking lot, road, street tree, paddy, greenhouse, dry field, forest, bare land, farmland).

The model pairs a **DINOv3 ViT-L/16** backbone (sat-493M pretrain) with **TCAM (Terrain-Context Adaptive Mutual-attention) Head** — a segmentation decoder we designed in-house for Korean remote-sensing imagery. TCAM infers an implicit terrain context (mountain / plain / urban) from RGB alone, picks the right backbone scale per pixel, and runs a bidirectional cross-attention between learnable class prototypes and spatial features so that visually similar classes such as paddy / dry field / farmland or street tree / forest can be separated reliably.

- **Custom TCAM head.** 6 in-house submodules: FeatureBridge, TCE, SAA, CFMA ×3, BoundaryGuidedUpsampler, Classifier. **+8.4 % relative mIoU over DPT** on the same backbone and data.
- **DINOv3-ViT-L/16 backbone.** 24 layers, 1024 hidden, 4 register tokens, satellite-pretrained (sat-493M), unfrozen during training.
- **Single-shot, full-resolution mask.** 512×512 RGB input → 512×512 per-class logits, no sliding window required.
- **HF Transformers compatible.** Loads via `AutoModel` with `trust_remote_code=True`; ships an `AIOne_GeoSegImageProcessor`.

---

## Key Capabilities

- Pixel-wise classification of Korean ortho-imagery into 11 land-cover classes.
- Robust separation of similar classes (paddy / dry field / farmland, street tree / forest) via class-prototype mutual attention.
- Sharp object and parcel boundaries via boundary-guided upsampling — useful for buildings, roads, and field edges.
- Built-in Korean label set and per-class RGB palette for direct visualization.
- One-call loading through `transformers.AutoModel` / `AutoImageProcessor`.

### Classes

| ID | Korean | English | RGB |
| --- | --- | --- | --- |
| 0 | 배경 | Background | (0, 0, 0) |
| 1 | 건물 | Building | (184, 131, 237) |
| 2 | 주차장 | Parking lot | (16, 64, 178) |
| 3 | 도로 | Road | (42, 65, 247) |
| 4 | 가로수 | Street tree | (200, 229, 155) |
| 5 | 논 | Paddy | (191, 255, 255) |
| 6 | 비닐하우스 | Greenhouse | (220, 240, 255) |
| 7 | 밭 | Dry field | (102, 249, 247) |
| 8 | 산림 | Forest | (45, 75, 42) |
| 9 | 나지 | Bare land | (255, 242, 159) |
| 10 | 농경지 | Farmland | (210, 180, 140) |

---

## TCAM Head

**TCAM = Terrain-Context Adaptive Mutual-attention.** It takes multi-layer hidden states from DINOv3-ViT-L/16 at stages `{5, 11, 17, 23}` and decodes them through 6 submodules:

| # | Submodule | Role | Core idea |
|---|---|---|---|
| 1 | **FeatureBridge** ×4 | Reshape ViT hidden states `(B, N, C)` to 2D maps `(B, D, H, W)` | CLS readout is projected and concatenated to patch tokens, then 1×1 conv compresses channels |
| 2 | **TCE** (Terrain Context Estimator) | Infer an *implicit terrain context* (mountain / plain / urban) from the deepest feature; output FiLM parameters (γ, β) | GAP → MLP. No DEM required — γ is initialized with a +1.0 residual for identity start |
| 3 | **SAA** (Scale-Adaptive Aggregation) | Per-pixel softmax fusion of the 4 scales | FiLM-modulated by (γ, β) so that forest pixels lean on deep layers and building / road pixels lean on shallow layers automatically |
| 4 | **CFMA** ×3 (Class–Feature Mutual Attention) | Bidirectional cross-attention between learnable class prototypes and the fused spatial feature | (1) Class → Feature: each class embedding queries the spatial feature; (2) Feature → Class: spatial features re-query the refreshed prototypes. Major gain on confusable classes (paddy / dry field / farmland, street tree / forest) |
| 5 | **BoundaryGuidedUpsampler** | Extract a boundary attention map (sigmoid) from a shallow feature and run 4-stage ConvTranspose for 16× upsample | Recovers sharp parcel and mountain-ridge boundaries |
| 6 | **Classifier** | 3×3 conv → 1×1 conv | Pixel-wise logits |

**Pipeline:** `FeatureBridge ×4 → TCE → SAA → CFMA ×3 → BoundaryGuidedUpsampler → Classifier`

### Parameter Breakdown

| Module | Params |
|---|---:|
| FeatureBridge × 4 | 9.98 M |
| TCE | 0.26 M |
| SAA | 0.59 M |
| CFMA × 3 | 7.11 M |
| BoundaryGuidedUpsampler | 8.10 M |
| Classifier | 1.33 M |
| **TCAM Head total** | **27.37 M** |
| Backbone (DINOv3-ViT-L/16) | 303.13 M |
| **Total** | **330.50 M** |

vs DPT head: +3.24 M (+13.4 % head, +0.99 % total).

### TCAM Hyperparameters

| Field | Value |
|---|---|
| `tcam_hidden_size` (D) | 384 |
| `tcam_num_heads` (CFMA) | 12 |
| `tcam_cfma_layers` | 3 |
| `tcam_tce_hidden_size` | 192 |
| `tcam_boundary_channels` | 64 |
| `readout_type` | `project` |

---

## Training

| Item | Value |
|---|---|
| Dataset | Korean aerial photography + AI Hub land-cover |
| Classes | 11 (background + 10 land-cover) |
| Input | 512 × 512, 3-channel RGB |
| Backbone | DINOv3-ViT-L/16 (sat-493M pretrain), unfrozen |
| Selected layers | {5, 11, 17, 23} |
| Patch size | 16 (32 × 32 token grid) |
| Loss | Focal (γ = 2.5) + Dice |
| Optimizer | AdamW, lr 1.3e-4, wd 0.01 |
| Scheduler | CosineAnnealing, η_min = 1e-6 |
| Batch size | 32 |
| Mixed precision | bf16 |
| Sampler | Weighted (class-balanced) |
| Epochs | 40 |

---

## Results

### TCAM vs DPT (same backbone, same data, 40 epochs)

| Head | Params | Best Val mIoU | Best Epoch |
|---|---:|---:|---:|
| DPT | 24.13 M | 0.6505 | 39 |
| **TCAM (ours)** | **27.37 M** | **0.7054** | **40** |
| Δ | +3.24 M | **+0.0549 (+8.4 % rel.)** | — |

### Training Dynamics

- **Epochs 1–10 (warm-up).** mIoU oscillates between 0.53–0.66; train loss falls quickly from 0.486 to 0.361.
- **Epochs 11–35 (stabilization).** mIoU climbs monotonically 0.64 → 0.697; focal + dice combination drives minor-class learning.
- **Epochs 36–40 (fine convergence).** With LR in the 3e-5 → 1.6e-5 range, mIoU plateaus at 0.694 → 0.705.

---

## Quick Start

### Installation

```bash
pip install "transformers>=4.45" torch pillow
```

### Inference

```python
import torch
import numpy as np
from PIL import Image
from transformers import AutoModel, AutoImageProcessor

MODEL_ID = "JDONE-Research/AIOne-GeoSeg-330M"
device = "cuda" if torch.cuda.is_available() else "cpu"

model = AutoModel.from_pretrained(MODEL_ID, trust_remote_code=True).to(device).eval()
processor = AutoImageProcessor.from_pretrained(MODEL_ID, trust_remote_code=True)

image = Image.open("aerial.jpg").convert("RGB")
inputs = processor(images=image, return_tensors="pt").to(device)

with torch.no_grad():
    logits = model(pixel_values=inputs["pixel_values"]).logits  # (1, 11, 512, 512)

mask = logits.argmax(dim=1)[0].cpu().numpy()  # (512, 512) int class IDs

id2label = model.config.id2label
print("Detected classes:", sorted({id2label[str(i)] for i in np.unique(mask)}))
```

### Colorized mask

```python
palette = np.array(model.config.label_colors, dtype=np.uint8)  # (11, 3)
color_mask = palette[mask]                                     # (512, 512, 3)
Image.fromarray(color_mask).save("mask.png")

# Side-by-side overlay (50% blend)
resized = image.resize((512, 512))
overlay = (np.array(resized) * 0.5 + color_mask * 0.5).astype(np.uint8)
Image.fromarray(overlay).save("overlay.png")
```

### Batch inference

```python
images = [Image.open(p).convert("RGB") for p in paths]
inputs = processor(images=images, return_tensors="pt").to(device)
with torch.no_grad():
    masks = model(pixel_values=inputs["pixel_values"]).logits.argmax(dim=1).cpu().numpy()
# masks: (B, 512, 512)
```

---

## Model Specs

| Field | Value |
| --- | --- |
| Architecture | `AIOne_GeoSeg` |
| Backbone | DINOv3 ViT-L/16, sat-493M pretrain (24 layers, 1024 hidden, 16 heads, 4 register tokens) |
| Segmentation head | **TCAM (custom)** — FeatureBridge ×4, TCE, SAA, CFMA ×3, BoundaryGuidedUpsampler, Classifier |
| Total parameters | **330.50 M** (Backbone 303.13 M + TCAM Head 27.37 M) |
| Weights on disk | 1.3 GB (FP32) |
| Input | RGB image, 512×512 |
| Output | Per-pixel logits, shape `(B, 11, 512, 512)` |
| Backbone feature taps | stages `[5, 11, 17, 23]` |
| Number of classes | 11 (Korean land-cover) |
| Validation mIoU | 0.7054 |
| Training precision | bf16 |
| Released checkpoint precision | float32 |
| Domain | Korean aerial / satellite ortho-imagery |

---

## Intended Use

- Korean land-cover mapping from aerial or satellite ortho-imagery.
- Change-detection pipelines (run the model on two epochs and diff the masks).
- Urban-planning, agriculture, and forestry analytics that need per-pixel Korean class labels.
- Research baseline for comparing other segmentation heads against TCAM.

### Out-of-Scope Use

- **Non-commercial only.** This release is governed by CC-BY-NC-4.0; do not use it in revenue-generating products or services.
- Imagery from regions or sensors that differ substantially from the Korean ortho-imagery training distribution (expect degraded accuracy).
- Sole-source decision-making in legal, regulatory, or safety-critical contexts.
- Any analysis that infringes on personal privacy, property rights, or applicable geospatial-data regulations.

---

## License

This model is released under the **[Creative Commons Attribution-NonCommercial 4.0 International (CC-BY-NC-4.0)](https://creativecommons.org/licenses/by-nc/4.0/)** license.

- **Free** to use, share, and adapt for non-commercial purposes with attribution.
- **Not permitted** for commercial use. Contact the authors for a commercial license.
- Provided "as is" without warranties of any kind.

---

## Citation

If you use AIOne-GeoSeg in your research, please cite:

```bibtex
@misc{aione_geoseg_330m,
  title        = {AIOne-GeoSeg-330M: A DINOv3 Vision Transformer with TCAM Head for Korean Land-Cover Segmentation},
  author       = {JDONE Research},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/JDONE-Research/AIOne-GeoSeg-330M}}
}
```

A paper describing the TCAM head, training procedure, and full ablations will be released soon — citation details will be updated here when available.