File size: 2,646 Bytes
4bbace8 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 | ---
pipeline_tag: text-ranking
tags:
- gguf
- reranker
- qwen3
- llama-cpp
language:
- multilingual
base_model: jinaai/jina-reranker-v3
base_model_relation: quantized
inference: false
license: cc-by-nc-4.0
library_name: llama.cpp
---
# jina-reranker-v3-GGUF
GGUF quantizations of [jina-reranker-v3](https://huggingface.co/jinaai/jina-reranker-v3) using llama.cpp. A 0.6B parameter multilingual listwise reranker quantized for efficient inference.
## Requirements
- Python 3.8+
- llama.cpp binaries (`llama-embedding` and `llama-tokenize`)
- Hanxiao's llama.cpp fork recommended: https://github.com/hanxiao/llama.cpp
## Installation
```bash
pip install numpy safetensors
```
## Files
- `jina-reranker-v3-BF16.gguf` - Quantized model weights (BF16, 1.1GB)
- `projector.safetensors` - MLP projector weights (3MB)
- `rerank.py` - Reranker implementation
## Usage
```python
from rerank import GGUFReranker
# Initialize reranker
reranker = GGUFReranker(
model_path="jina-reranker-v3-BF16.gguf",
projector_path="projector.safetensors",
llama_embedding_path="/path/to/llama-embedding"
)
# Rerank documents
query = "What is the capital of France?"
documents = [
"Paris is the capital and largest city of France.",
"Berlin is the capital of Germany.",
"The Eiffel Tower is located in Paris."
]
results = reranker.rerank(query, documents)
for result in results:
print(f"Score: {result['relevance_score']:.4f}, Doc: {result['document'][:50]}...")
```
## API
### `GGUFReranker.rerank(query, documents, top_n=None, return_embeddings=False, instruction=None)`
**Arguments:**
- `query` (str): Search query
- `documents` (List[str]): Documents to rerank
- `top_n` (int, optional): Return only top N results
- `return_embeddings` (bool): Include embeddings in output
- `instruction` (str, optional): Custom ranking instruction
**Returns:**
List of dicts with keys: `index`, `relevance_score`, `document`, and optionally `embedding`
## Citation
If you find `jina-reranker-v3` useful in your research, please cite the [original paper](https://arxiv.org/abs/2509.25085):
```bibtex
@misc{wang2025jinarerankerv3lateinteractiondocument,
title={jina-reranker-v3: Last but Not Late Interaction for Document Reranking},
author={Feng Wang and Yuqing Li and Han Xiao},
year={2025},
eprint={2509.25085},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2509.25085},
}
```
## License
This MLX implementation follows the same CC BY-NC 4.0 license as the original model. For commercial usage inquiries, please [contact Jina AI](https://jina.ai/contact-sales/).
|