--- license: apache-2.0 language: - en - fr base_model: - Qwen/Qwen3.5-2B pipeline_tag: image-text-to-text library_name: transformers tags: - reranker - cross-encoder - multimodal - text-ranking - document-reranking - vidore - beir ---
LightOn [![Website](https://img.shields.io/badge/LightOn-Website-blue?logo=google-chrome)](https://lighton.ai) [![LinkedIn](https://img.shields.io/badge/LightOn-LinkedIn-0A66C2?logo=linkedin)](https://www.linkedin.com/company/lighton/) [![X](https://img.shields.io/badge/@LightOnIO-X-black?logo=x)](https://x.com/LightOnIO) 📝 Blog post: *coming soon*

LightOn-rerank-PW-2B

Unified Text + Visual Document Reranker by LightOn

PW-0.8B | LW-0.8B | PW-2B | LW-2B | PW-4B | LW-4B

Reranking: the second stage that puts the best text and image candidates on top
--- ## About the LightOn-rerank family Production retrieval pipelines usually need two rerankers: one for text passages and one for visual documents (PDF pages, slides, scans). **LightOn-rerank** models are *unified* cross-encoder rerankers: a single model scores both text passages and document page images against a query, on top of any first-stage retriever (BM25, dense embeddings, or ColPali-family late-interaction models). The models are built on Qwen3.5 vision-language backbones (hybrid linear + full attention) and jointly fine-tuned on text and visual reranking data with mixed-modality batches (LoRA, merged into the released weights). Training data is English-only; French performance transfers zero-shot from the multilingual backbone. The family comes in two scoring flavours × three sizes (0.8B / 2B / 4B): - **PW (pointwise)**: each candidate is scored independently. The model judges whether the document answers the query, and the score is `logit("Yes") − logit("No")`. One forward pass per candidate and no generation, so it is simple to serve (vLLM-compatible) and embarrassingly parallel. - **LW (listwise)**: generative listwise ranking, where 4 candidates are placed in a single prompt and the model generates a permutation (`[2] > [4] > [1] > [3]`). Larger candidate pools are ranked with a sliding window (window 4, stride 2, bottom-to-top). Cross-document attention makes LW markedly stronger on hard visual reranking, and unlike pointwise scoring it keeps improving with backbone size. **LightOn-rerank-PW-2B** is the pointwise flagship of the family: a single 2B model that is simultaneously competitive with dedicated text rerankers on BEIR and with vision-specialised rerankers on ViDoRe V2/V3, with none of the usual trade-off between the two modalities, and the serving simplicity of independent per-candidate scoring. ## Results **ViDoRe V3** (visual document reranking, 8 domains × EN/FR queries), overall nDCG@10, ColQwen2.5-v0.2 first stage, retrieve 100 / rerank 100. All models, including baselines, were re-evaluated under this same two-stage protocol, so numbers are mutually comparable but not comparable to vendor-reported end-to-end results. | Model | Params | Scoring | ViDoRe V3 overall nDCG@10 | |---|---|---|---| | [LightOn-rerank-LW-4B](https://huggingface.co/lightonai/LightOn-rerank-LW-4B) | 4.5B | listwise | 0.6469 | | *Qwen3-VL-Reranker-8B* | *8B* | *pointwise (pooling)* | *0.6423* | | [LightOn-rerank-LW-2B](https://huggingface.co/lightonai/LightOn-rerank-LW-2B) | 2.2B | listwise | 0.6266 | | **LightOn-rerank-PW-2B (this model)** | 2.2B | pointwise | **0.5987** | | [LightOn-rerank-PW-4B](https://huggingface.co/lightonai/LightOn-rerank-PW-4B) | 4.5B | pointwise | 0.5980 | | *jina-reranker-m0* | *2.4B* | *pointwise* | *0.5939* | | *Qwen3-VL-Reranker-2B* | *2B* | *pointwise (pooling)* | *0.5918* | | [LightOn-rerank-LW-0.8B](https://huggingface.co/lightonai/LightOn-rerank-LW-0.8B) | 0.85B | listwise | 0.5825 | | [LightOn-rerank-PW-0.8B](https://huggingface.co/lightonai/LightOn-rerank-PW-0.8B) | 0.85B | pointwise | 0.4820 | ### ViDoRe V3 detail (nDCG, ColQwen2.5 first stage, rerank-100) | Domain | EN @5 | EN @10 | FR @5 | FR @10 | |---|---|---|---|---| | finance_en | 0.5893 | 0.6113 | 0.5585 | 0.5716 | | finance_fr | 0.4643 | 0.4934 | 0.4947 | 0.5260 | | computer_science | 0.7239 | 0.7398 | 0.6814 | 0.7047 | | hr | 0.6570 | 0.6665 | 0.5996 | 0.6139 | | energy | 0.6634 | 0.6921 | 0.6735 | 0.7013 | | industrial | 0.5420 | 0.5520 | 0.4808 | 0.4864 | | pharmaceuticals | 0.6298 | 0.6385 | 0.5958 | 0.6039 | | physics | 0.4551 | 0.4851 | 0.4568 | 0.4927 | | **mean** | **0.5906** | **0.6098** | **0.5676** | **0.5876** | Overall nDCG@10: **0.5987** (EN 0.6098 / FR 0.5876). On the easier ViDoRe V2 benchmark (same protocol), mean nDCG@10 is **0.8558**, in the same range as the strongest 2B-class baselines (jina-reranker-m0: 0.8596, MonoQwen2-VL-v0.1: 0.8454, Qwen3-VL-Reranker-2B: 0.8428). ### BEIR results (text reranking) 15 datasets, nDCG@10, retrieve 100 / rerank 100, all models re-evaluated under the same protocol. The decontaminated mean excludes NQ and MSMARCO (present in the training data). | First stage | LightOn-rerank-PW-2B | jina-reranker-m0 | Qwen3-VL-Reranker-2B | |---|---|---|---| | jina-embeddings-v3 (decontaminated mean) | 0.5227 | **0.5677** | 0.5025 | | BM25 (decontaminated mean) | 0.4968 | **0.5397** | 0.4815 | On text it sits between the two strongest 2B baselines. jina-reranker-m0 is the only baseline that is close on both modalities, and it trails on ViDoRe V3 (0.5939 vs 0.5987). ## Model Details - **Model type:** multimodal cross-encoder reranker (**pointwise**: each candidate is scored independently as `logit("Yes") − logit("No")`) - **Base model:** [Qwen/Qwen3.5-2B](https://huggingface.co/Qwen/Qwen3.5-2B) (Qwen3.5 hybrid linear + full attention VLM) - **Parameters:** ≈2.2B (bfloat16, 4.4 GB) - **Inputs:** query (text) + candidate document(s): text passage **or** page image - **Fine-tuning:** joint text+vision LoRA (r=32, α=32, rsLoRA, merged into the released weights), mixed-modality batches (2 text + 2 vision groups per micro-batch), vision loss weight 1.3, lr 1e-4, 1 epoch (465 steps) - **Data:** ~120k EN text queries (NQ, TriviaQA, MSMARCO) with mined hard negatives + ~118k visual query–page pairs with ColPali-family hard negatives (`dedup_120k_en_v3` + `ColPali_hard_neg_v2`) - **Languages:** English (training), French (zero-shot transfer) - **Requirements:** `transformers >= 5.4.0` (`qwen3_5` architecture) - **Internal experiment ID:** `exp30` ## Usage: pointwise reranking Each candidate is scored independently as `logit("Yes") − logit("No")` at the first generated position; sort candidates by descending score. The model was trained with a fixed system prompt and user template: use them verbatim for best results. ```python import torch from transformers import AutoModelForImageTextToText, AutoProcessor model_id = "lightonai/LightOn-rerank-PW-2B" model = AutoModelForImageTextToText.from_pretrained( model_id, dtype=torch.bfloat16, attn_implementation="flash_attention_2", # optional, remove if flash-attn is not installed device_map="cuda", ).eval() processor = AutoProcessor.from_pretrained(model_id) processor.tokenizer.padding_side = "left" # scores are read at the last position YES_TOKEN_ID = 9175 # "Yes" NO_TOKEN_ID = 2665 # "No" SYSTEM_PROMPT = "Judge whether the document is relevant to the query. Answer Yes or No." USER_TEMPLATE = ( "Given a query, determine if the document is relevant. " "The query is: {query} Document: {doc}" ) query = "What is late interaction in neural information retrieval?" documents = [ "ColBERT computes token-level query-document interactions at search time...", "The Eiffel Tower is located on the Champ de Mars in Paris.", ] texts = [ processor.apply_chat_template( [ {"role": "system", "content": SYSTEM_PROMPT}, {"role": "user", "content": USER_TEMPLATE.format(query=query, doc=doc)}, ], tokenize=False, add_generation_prompt=True, ) for doc in documents ] inputs = processor( text=texts, return_tensors="pt", padding=True, truncation=True, max_length=2048 ).to(model.device) with torch.inference_mode(): logits = model(**inputs).logits[:, -1] scores = (logits[:, YES_TOKEN_ID] - logits[:, NO_TOKEN_ID]).tolist() ranked = sorted(zip(scores, documents), reverse=True) ``` To score a **page image** instead of a text passage, replace the user message with: ```python VISION_TEMPLATE = "Given a query, determine if the document image is relevant. The query is: {query}" {"role": "user", "content": [ {"type": "image", "image": page_image}, # PIL.Image {"type": "text", "text": VISION_TEMPLATE.format(query=query)}, ]} ``` and pass `images=[page_image, ...]` to the processor call (keep the same system prompt). ### Serving with vLLM ```bash vllm serve lightonai/LightOn-rerank-PW-2B --trust-remote-code --max-model-len 16384 ``` ```python resp = client.chat.completions.create( model="lightonai/LightOn-rerank-PW-2B", messages=messages, # same system + user messages as above max_tokens=1, logprobs=True, top_logprobs=20, temperature=0.0, ) top = resp.choices[0].logprobs.content[0].top_logprobs lp = {t.token: t.logprob for t in top} score = lp.get("Yes", -100.0) - lp.get("No", -100.0) ``` Full-page document images can exceed 8k tokens, so keep `--max-model-len` at 16384 or higher when reranking page images. ## Notes & limitations - Pointwise scoring does not improve with backbone scale (the 4B pointwise model ties this one). If you have the compute budget, [LightOn-rerank-LW-2B](https://huggingface.co/lightonai/LightOn-rerank-LW-2B) is +0.03 nDCG@10 on ViDoRe V3 at the same size. - Training data is English-only. French works zero-shot (the backbone is multilingual) but is slightly behind English on average. - BEIR contamination flag: NQ and MSMARCO are part of the text training data; headline text figures use decontaminated means that exclude them. ## The LightOn-rerank family | Model | Backbone | Scoring | ViDoRe V3 overall nDCG@10 | |---|---|---|---| | [LightOn-rerank-PW-0.8B](https://huggingface.co/lightonai/LightOn-rerank-PW-0.8B) | Qwen3.5-0.8B | pointwise | 0.4820 | | [LightOn-rerank-LW-0.8B](https://huggingface.co/lightonai/LightOn-rerank-LW-0.8B) | Qwen3.5-0.8B | listwise | 0.5825 | | [LightOn-rerank-PW-2B](https://huggingface.co/lightonai/LightOn-rerank-PW-2B) | Qwen3.5-2B | pointwise | 0.5987 | | [LightOn-rerank-LW-2B](https://huggingface.co/lightonai/LightOn-rerank-LW-2B) | Qwen3.5-2B | listwise | 0.6266 | | [LightOn-rerank-PW-4B](https://huggingface.co/lightonai/LightOn-rerank-PW-4B) | Qwen3.5-4B | pointwise | 0.5980 | | [LightOn-rerank-LW-4B](https://huggingface.co/lightonai/LightOn-rerank-LW-4B) | Qwen3.5-4B | listwise | **0.6469** | Rule of thumb: **LW** models are stronger at every size (and the gap grows with size); **PW** models are cheaper to serve and score candidates independently. For the best quality pick [LW-4B](https://huggingface.co/lightonai/LightOn-rerank-LW-4B); for the best quality/cost trade-off pick [LW-2B](https://huggingface.co/lightonai/LightOn-rerank-LW-2B); for maximum throughput on text-heavy workloads pick a PW model.