---
license: gemma
library_name: transformers
pipeline_tag: visual-document-retrieval
base_model: vidore/colpali-v1.3-hf
tags:
- samer
- colpali
- visual-document-retrieval
- token-compression
- arxiv:2607.04605
---

# samer k64 colpali

This repository contains the adapter-only SaMer checkpoint for `vidore/colpali-v1.3-hf`.
It does not redistribute the base model weights. The repository stores only the
trained projection layer and custom `trust_remote_code` wrapper needed to apply
SaMer feature-spatial object-aware token merging at inference time.

## Usage

```python
import torch
from transformers import AutoModel, AutoProcessor

model = AutoModel.from_pretrained(
    "dmis-lab/samer-k64-colpali",
    trust_remote_code=True,
).to("cuda").eval()

processor = AutoProcessor.from_pretrained(
    model.config.base_model_name_or_path,
    trust_remote_code=True,
)

# Build processor inputs with the same convention as the base model.
# image_inputs = processor(images=[image], return_tensors="pt").to("cuda")
# query_inputs = processor(text=["a query"], return_tensors="pt", padding=True).to("cuda")

# image_tokens = model.encode_image(image_inputs)          # [B, 64, D]
# query_tokens, query_mask = model.encode_query(query_inputs, return_mask=True)
# scores = model.score(query_tokens, image_tokens, query_mask=query_mask)
```

## SaMer Configuration

- `num_regions`: 64
- `cluster_iters`: 3
- `spatial_weight`: 0.1
- `assignment_temperature`: 0.07
- training objective: retrieval loss only
- inference: bbox-free feature-spatial soft centroid merging

## Base Model

The base model is loaded from `vidore/colpali-v1.3-hf` at runtime. Please follow
the base model license and usage terms.

## Citation

```bibtex
@misc{park2026visualtokensmatterequally,
      title={Do All Visual Tokens Matter Equally? Object-Evidence Preserving Token Merging for Vision-Language Retrieval},
      author={Suhyeong Park and Junha Jung and Jungwoo Park and Jaewoo Kang},
      year={2026},
      eprint={2607.04605},
      archivePrefix={arXiv},
      primaryClass={cs.IR},
      url={https://arxiv.org/abs/2607.04605},
}
```