DUSUNEN Mercek 118M research v0

An experimental 117.6M-parameter Turkish query–document cross-encoder trained on balanced positive and model-mined negative pairs. This release deliberately publishes a negative fine-tuning result: reranking helps substantially, but the untouched multilingual base remains slightly better on the held-out benchmark.

The published weights are checkpoint 3000, selected only by the training validation metric before inspecting the held-out TurHistQuad result.

Held-out top-100 reranking

DUSUNEN Rota 270M v2 produced one frozen top-100 candidate list for 1,024 queries. Every cross-encoder scored the same 102,400 query–document pairs.

Stage	MRR@10	nDCG@10	Recall@10	Recall@100
Retriever only	0.574323	0.423477	0.475586	0.686035
Untouched multilingual base reranker	0.772581	0.530939	0.516602	0.686035
DUSUNEN Mercek selected checkpoint 3000	0.771222	0.527705	0.512695	0.686035
DUSUNEN Mercek final training step	0.771231	0.527192	0.511230	0.686035

The base reranker improves retriever-only MRR@10 by 0.198258 and nDCG@10 by 0.107462. The selected fine-tuned model does not improve the base; for this benchmark, use the base model. Publishing the trained weights, recipe and raw results makes the failed hypothesis reproducible instead of hiding it.

On the same RTX 5060 Laptop GPU, the selected model scored 102,400 pairs in 41.22 seconds (2,484 pairs/second). Throughput is hardware- and batch-dependent.

Use

from sentence_transformers import CrossEncoder

model = CrossEncoder("GoktugD/DUSUNEN-Mercek-118M-research-v0")
pairs = [
    ["Hard negative neden önemlidir?", "Zor negatifler karar sınırını güçlendirir."],
    ["Hard negative neden önemlidir?", "Ankara Türkiye'nin başkentidir."],
]
print(model.predict(pairs))

Scores are ranking signals, not calibrated probabilities. Apply the model only to a small candidate set from a first-stage retriever.

Training

Pinned base revision: 1427fd652930e4ba29e8149678df786c240d8825
50,000 mined triplets expanded to 100,000 balanced training pairs
2,000 validation triplets; BCE loss; maximum length 256
Effective batch size 32; learning rate 2e-5; one epoch; BF16; seed 3407
One RTX 5060 Laptop GPU; training time 415.3 seconds; peak allocation 1.081 GB
Training-validation MRR@2: base 0.9860; final 0.9845

Limitations

Fine-tuning did not beat the untouched base on the reported held-out task.
Mining with DUSUNEN Rota v1 can imprint retriever-specific biases.
A labeled negative may still be semantically relevant.
Reranking 100 candidates is slower than dense retrieval and does not recover relevant documents missing from the candidate set.
The benchmark does not establish quality for every Turkish domain or dialect.

The repository includes raw candidate-list metrics, training state, validation results, environment metadata and the exact evaluation implementation.