README.md

---
license: apache-2.0
language:
- en
- fr
base_model:
- Qwen/Qwen3.5-4B
pipeline_tag: image-text-to-text
library_name: transformers
tags:
- reranker
- cross-encoder
- multimodal
- text-ranking
- document-reranking
- vidore
- beir
- sentence-transformers
datasets:
- vidore/colpali_train_set
- lightonai/embeddings-fine-tuning
---

<div align="center">
<img src="rerank.png" alt="Reranking: the second stage that puts the best text and image candidates on top" width="560">

[![Website](https://img.shields.io/badge/LightOn-Website-blue?logo=google-chrome)](https://lighton.ai)
[![LinkedIn](https://img.shields.io/badge/LightOn-LinkedIn-0A66C2?logo=linkedin)](https://www.linkedin.com/company/lighton/)
[![X](https://img.shields.io/badge/@LightOnIO-X-black?logo=x)](https://x.com/LightOnIO)

📝 [Blog post](https://huggingface.co/blog/lightonai/lighton-rerank)

</div>

<h1 align="center">LightOn-rerank-PW-4B</h1>
<h3 align="center">Unified Text + Visual Document Reranker by LightOn</h3>

<p align="center">
<a href="https://huggingface.co/lightonai/LightOn-rerank-PW-0.8B">PW-0.8B</a> |
<a href="https://huggingface.co/lightonai/LightOn-rerank-LW-0.8B">LW-0.8B</a> |
<a href="https://huggingface.co/lightonai/LightOn-rerank-PW-2B">PW-2B</a> |
<a href="https://huggingface.co/lightonai/LightOn-rerank-LW-2B">LW-2B</a> |
<a href="https://huggingface.co/lightonai/LightOn-rerank-PW-4B">PW-4B</a> |
<a href="https://huggingface.co/lightonai/LightOn-rerank-LW-4B">LW-4B</a>
</p>

---


## About the LightOn-rerank family

Production retrieval pipelines usually need two rerankers: one for text passages and one for visual documents (PDF pages, slides, scans). **LightOn-rerank** models are *unified* cross-encoder rerankers: a single model scores both text passages and document page images against a query, on top of any first-stage retriever (BM25, dense embeddings, or ColPali-family late-interaction models).

The models are built on Qwen3.5 backbone (hybrid linear + full attention) and jointly fine-tuned on text and visual reranking data with mixed-modality batches (LoRA, merged into the released weights). Training data is English-only; French performance transfers zero-shot from the multilingual backbone.

The family comes in two scoring flavours × three sizes (0.8B / 2B / 4B):

- **PW (pointwise)**: each candidate is scored independently. The model judges whether the document answers the query, and the score is `logit("Yes") − logit("No")`. One forward pass per candidate and no generation.
- **LW (listwise)**: generative listwise ranking, where 4 candidates are placed in a single prompt and the model generates a permutation (`[2] > [4] > [1] > [3]`). Larger candidate pools are ranked with a sliding window (window 4, stride 2, bottom-to-top). Cross-document attention makes LW markedly stronger on hard visual reranking, and unlike pointwise scoring it keeps improving with backbone size.

**LightOn-rerank-PW-4B** is the 4B pointwise member of the family. On visual reranking it ties the 2B pointwise model (59.80 vs 59.87 nDCG@10 on ViDoRe V3): direct evidence that independent Yes/No scoring is a capacity bottleneck that extra parameters cannot fix. On text BEIR, however, it posts the strongest scores in the family (50.19 decontaminated mean). For 4B-budget vision-heavy deployments, [LightOn-rerank-LW-4B](https://huggingface.co/lightonai/LightOn-rerank-LW-4B) is +4.9 nDCG@10 at the same size.

## Results

**ViDoRe V3** (visual document reranking, 8 domains × EN/FR queries), overall nDCG@10, ColQwen2.5-v0.2 first stage, retrieve 100 / rerank 100. All models, including baselines, were re-evaluated under this same two-stage protocol, so numbers are mutually comparable but not comparable to vendor-reported end-to-end results.

| Model | Params | Scoring | ViDoRe V3 overall nDCG@10 |
|---|---|---|---|
| [LightOn-rerank-LW-4B](https://huggingface.co/lightonai/LightOn-rerank-LW-4B) | 4.5B | listwise | 64.69 |
| *Qwen3-VL-Reranker-8B* | *8B* | *pointwise (pooling)* | *64.23* |
| [LightOn-rerank-LW-2B](https://huggingface.co/lightonai/LightOn-rerank-LW-2B) | 2.2B | listwise | 62.66 |
| [LightOn-rerank-PW-2B](https://huggingface.co/lightonai/LightOn-rerank-PW-2B) | 2.2B | pointwise | 59.87 |
| **LightOn-rerank-PW-4B (this model)** | 4.5B | pointwise | **59.80** |
| *jina-reranker-m0* | *2.4B* | *pointwise* | *59.40* |
| *Qwen3-VL-Reranker-2B* | *2B* | *pointwise (pooling)* | *59.18* |
| [LightOn-rerank-LW-0.8B](https://huggingface.co/lightonai/LightOn-rerank-LW-0.8B) | 0.85B | listwise | 58.25 |
| *MonoQwen2-VL-v0.1* | *2B* | *pointwise* | *57.76* |
| *First-stage only (ColQwen2.5, no rerank)* | — | — | *55.60* |
| [LightOn-rerank-PW-0.8B](https://huggingface.co/lightonai/LightOn-rerank-PW-0.8B) | 0.85B | pointwise | 48.20 |

### ViDoRe V3 detail (nDCG@10, ColQwen2.5 first stage, rerank-100)

| Domain | EN | FR |
|---|---|---|
| finance_en | 70.98 | 68.54 |
| finance_fr | 45.96 | 37.92 |
| computer_science | 79.25 | 74.71 |
| hr | 58.67 | 53.54 |
| energy | 70.95 | 72.36 |
| industrial | 60.54 | 47.33 |
| pharmaceuticals | 68.11 | 67.70 |
| physics | 44.46 | 35.77 |
| **mean** | **62.37** | **57.23** |

Overall nDCG@10: **59.80** (EN 62.37 / FR 57.23).

### BEIR results (text reranking)

13 datasets, nDCG@10, BM25 first stage, retrieve 100 / rerank 100, same protocol as all other family members. ⚠️ marks datasets in the text training mix (NQ, MSMARCO); the decontaminated mean excludes them.

| Dataset | nDCG@10 |
|---|---|
| fever | 81.77 |
| scifact | 76.13 |
| trec-covid | 79.38 |
| hotpotqa | 70.79 |
| nq ⚠️ | 61.03 |
| dbpedia | 44.21 |
| arguana | 41.47 |
| fiqa | 40.91 |
| msmarco ⚠️ | 39.20 |
| nfcorpus | 37.41 |
| touche-2020 | 35.68 |
| climate-fever | 26.46 |
| scidocs | 17.87 |
| **Mean (13)** | **50.18** |
| **Decontaminated mean (11, excl. ⚠️)** | **50.19** |

This is the best text result in the LightOn-rerank family (2B pointwise: 49.13, 4B listwise: 48.33 decontaminated mean). Pointwise plateaus with scale on vision but not on text. If your workload is text-dominant and you have the 4B budget, this is the family's strongest text reranker.

## Model Details

- **Model type:** multimodal cross-encoder reranker (**pointwise**: each candidate is scored independently as `logit("Yes") − logit("No")`)
- **Base model:** [Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B) (Qwen3.5 hybrid linear + full attention VLM)
- **Parameters:** ≈4.5B (bfloat16, 9.1 GB)
- **Inputs:** query (text) + candidate document(s): text passage **or** page image
- **Fine-tuning:** joint text+vision LoRA (r=32, α=32, rsLoRA, merged into the released weights), mixed-modality batches (2 text + 2 vision groups per micro-batch), vision loss weight 1.3, lr 1e-4, 1 epoch (465 steps)
- **Data:** same 213k groups as the listwise models — 107k text groups (NQ, TriviaQA, MS MARCO; each a `[pos, neg_0, neg_1, neg_2]` 4-list with hard negatives mined via the NV-Retriever approach with GTE-ModernBERT) + 106k vision groups (ColPali train set with negatives mined by Nomic). For pointwise training each group is flattened into (query, document, Yes/No) triples.
- **Languages:** English (training), French (zero-shot transfer)
- **Requirements:** `transformers >= 5.4.0` (`qwen3_5` architecture); `sentence-transformers >= 5.4.0` for the CrossEncoder usage

## Usage: pointwise reranking

Each candidate is scored independently as `logit("Yes") − logit("No")` at the first generated position; sort candidates by descending score. The model was trained with a fixed system prompt and user template: use them verbatim for best results.

### Using Sentence Transformers

Install Sentence Transformers (`>= 5.4.0`):

```bash
pip install "sentence-transformers[image]"
```

The trained system prompt and user templates are baked into the bundled `reranker` chat template — including the non-thinking generation prompt, so no `enable_thinking` handling is needed — and query-document pairs are formatted correctly out of the box:

```python
from sentence_transformers import CrossEncoder

model = CrossEncoder("lightonai/LightOn-rerank-PW-4B")

query = "What is late interaction in neural information retrieval?"
documents = [
    "ColBERT computes token-level query-document interactions at search time...",
    "The Eiffel Tower is located on the Champ de Mars in Paris.",
]

pairs = [(query, doc) for doc in documents]
scores = model.predict(pairs)
print(scores)
# [-3.     -8.0625]

rankings = model.rank(query, documents)
print(rankings)
# [{'corpus_id': 0, 'score': -3.0}, {'corpus_id': 1, 'score': -8.0625}]
```

To rerank **page images**, pass a `PIL.Image` (or an image URL or file path string) as the document. Text and image candidates can be mixed in the same call:

```python
from PIL import Image

pairs = [
    (query, Image.open("page_1.png")),
    (query, "https://example.com/page_2.png"),
    (query, "A text passage candidate for the same query."),
]
scores = model.predict(pairs)
```

Scores are raw `logit("Yes") − logit("No")` differences. You can map them to 0...1 probabilities with `model.predict(pairs, activation_fn=torch.nn.Sigmoid())`.

### Using Transformers

```python
import torch
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "lightonai/LightOn-rerank-PW-4B"
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",  # optional, remove if flash-attn is not installed
    device_map="cuda",
).eval()
processor = AutoProcessor.from_pretrained(model_id)
processor.tokenizer.padding_side = "left"  # scores are read at the last position

YES_TOKEN_ID = 9175  # "Yes"
NO_TOKEN_ID = 2665   # "No"

SYSTEM_PROMPT = "Judge whether the document is relevant to the query. Answer Yes or No."
USER_TEMPLATE = (
    "Given a query, determine if the document is relevant. "
    "The query is: {query}\n\nDocument: {doc}"
)

query = "What is late interaction in neural information retrieval?"
documents = [
    "ColBERT computes token-level query-document interactions at search time...",
    "The Eiffel Tower is located on the Champ de Mars in Paris.",
]

texts = [
    processor.apply_chat_template(
        [
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": USER_TEMPLATE.format(query=query, doc=doc)},
        ],
        tokenize=False,
        add_generation_prompt=True,
        enable_thinking=False,  # REQUIRED on the 4B backbone (thinking on by default)
    )
    for doc in documents
]
inputs = processor(
    text=texts, return_tensors="pt", padding=True, truncation=True, max_length=2048
).to(model.device)
with torch.inference_mode():
    logits = model(**inputs).logits[:, -1]
scores = (logits[:, YES_TOKEN_ID] - logits[:, NO_TOKEN_ID]).tolist()
# [-3.0, -8.0625]

ranked = sorted(zip(scores, documents), reverse=True)
```

To score a **page image** instead of a text passage, replace the user message with:

```python
VISION_TEMPLATE = "Given a query, determine if the document image is relevant. The query is: {query}"

{"role": "user", "content": [
    {"type": "image", "image": page_image},  # PIL.Image
    {"type": "text", "text": VISION_TEMPLATE.format(query=query)},
]}
```

and pass `images=[page_image, ...]` to the processor call (keep the same system prompt).

### Serving with vLLM

```bash
vllm serve lightonai/LightOn-rerank-PW-4B --trust-remote-code --max-model-len 16384
```


```python
resp = client.chat.completions.create(
    model="lightonai/LightOn-rerank-PW-4B",
    messages=messages,          # same system + user messages as above
    max_tokens=1,
    logprobs=True,
    top_logprobs=20,
    temperature=0.0,
    extra_body={"chat_template_kwargs": {"enable_thinking": False}},  # REQUIRED
)
top = resp.choices[0].logprobs.content[0].top_logprobs
lp = {t.token: t.logprob for t in top}
score = lp.get("Yes", -100.0) - lp.get("No", -100.0)
```

Full-page document images can exceed 8k tokens, so keep `--max-model-len` at 16384 or higher when reranking page images.

## Notes & limitations

- **Thinking must be disabled for scoring.** Qwen3.5-4B's chat template enables `<think>` by default; with it on, the first generated token is a thinking token and the Yes/No logprobs are distorted. Pass `enable_thinking=False` to `apply_chat_template` (or `chat_template_kwargs={"enable_thinking": False}` via the vLLM OpenAI client) as shown above. The Sentence Transformers path handles this automatically: the bundled `reranker` chat template hardcodes the non-thinking generation prompt.
- Pointwise scoring does not benefit from the 2B→4B scale-up on vision (59.80 vs 59.87 for the 2B), while generative listwise gains +2.0 points over the same step. For vision workloads prefer [LightOn-rerank-LW-4B](https://huggingface.co/lightonai/LightOn-rerank-LW-4B) at this size, or [LightOn-rerank-PW-2B](https://huggingface.co/lightonai/LightOn-rerank-PW-2B) for the same visual quality at lower cost.
- Training data is English-only. French works zero-shot (the backbone is multilingual) but is slightly behind English on average.
- BEIR contamination flag: NQ and MSMARCO are part of the text training data; headline text figures use decontaminated means that exclude them.

## The LightOn-rerank family

| Model | Backbone | Scoring | ViDoRe V3 overall nDCG@10 |
|---|---|---|---|
| [LightOn-rerank-PW-0.8B](https://huggingface.co/lightonai/LightOn-rerank-PW-0.8B) | Qwen3.5-0.8B | pointwise | 48.20 |
| [LightOn-rerank-LW-0.8B](https://huggingface.co/lightonai/LightOn-rerank-LW-0.8B) | Qwen3.5-0.8B | listwise | 58.25 |
| [LightOn-rerank-PW-2B](https://huggingface.co/lightonai/LightOn-rerank-PW-2B) | Qwen3.5-2B | pointwise | 59.87 |
| [LightOn-rerank-LW-2B](https://huggingface.co/lightonai/LightOn-rerank-LW-2B) | Qwen3.5-2B | listwise | 62.66 |
| [LightOn-rerank-PW-4B](https://huggingface.co/lightonai/LightOn-rerank-PW-4B) | Qwen3.5-4B | pointwise | 59.80 |
| [LightOn-rerank-LW-4B](https://huggingface.co/lightonai/LightOn-rerank-LW-4B) | Qwen3.5-4B | listwise | **64.69** |

Rule of thumb: **LW** models are stronger at every size (and the gap grows with size); **PW** models are cheaper to serve and score candidates independently. For the best quality pick [LW-4B](https://huggingface.co/lightonai/LightOn-rerank-LW-4B); for the best quality/cost trade-off pick [LW-2B](https://huggingface.co/lightonai/LightOn-rerank-LW-2B); for maximum throughput on text-heavy workloads pick a PW model.

## Citation

```
@misc{ananya2026lightonrerank,
  title={One Adapter, Both Modalities: Field Notes from Building and Serving a Multimodal Reranker},
  author={Ananya, Ishrat Jahan and Chatelain, Amelie},
  year={2026},
  howpublished={\url{https://huggingface.co/blog/lightonai/lighton-rerank}},
}
```