README.md

---
license: mit
base_model: ynie/roberta-large-snli_mnli_fever_anli_R1_R2_R3-nli
datasets:
  - lyutovad/TradeNewsEventDedup
language:
  - en
library_name: transformers
pipeline_tag: text-classification
tags:
  - text-classification
  - duplicate-detection
  - trade-news
  - information-retrieval
  - event-deduplication
  - nli
  - roberta
  - entailment
---

# trade-news-dedup-roberta-large-nli

**Event-level duplicate detection in trade news** — NLI-based classifier (RoBERTa-large), fine-tuned on the
weakly-supervised **SilverSet** of the *TradeNewsEventDedup* project.

Обнаружение дубликатов событий во внешнеторговых новостях: бинарная классификация пар
новостных саммари (дубликат / не-дубликат) на уровне идентичности торгового события.

> ⭐ **Highest PR-AUC and Recall** on the GoldSet; largest relative gain from fine-tuning.

## Task

Given two trade-news summaries, predict whether they describe **the same real-world trade
event** (same country, commodity, trade action, numerical values and time) — *not* mere text
similarity. The model was fine-tuned with **structured hard negatives** (semantically close but
materially different cases: updates `U` and related events `R`).

- **Base model:** [`ynie/roberta-large-snli_mnli_fever_anli_R1_R2_R3-nli`](https://huggingface.co/ynie/roberta-large-snli_mnli_fever_anli_R1_R2_R3-nli)
- **Training data:** [`lyutovad/TradeNewsEventDedup`](https://huggingface.co/datasets/lyutovad/TradeNewsEventDedup) — SilverSet (16,097 LLM-labeled pairs, fine-tuning only)
- **Evaluation:** GoldSet (1,469 manually validated pairs) — see results below
- **Output head:** Three-class NLI head. The duplicate signal is taken from the **entailment** logit (index `0`); use a symmetric average over both input orderings. See `finetune_meta.json` for `head_info`.

## Results (GoldSet, fine-tuned)

| Metric | Value |
|---|---|
| PR-AUC | 0.9396 |
| F1 | 0.8968 |
| Accuracy | 0.9095 |
| Recall | 0.9444 |

Thresholds are tuned on the GoldSet (optimistic estimate). Baselines (lexical/embedding
similarity) reach high recall but low precision; fine-tuning with hard negatives improves
precision and reduces false positives. Full protocol and ablations are in the paper.

## How to use

```python
import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

repo = "afafos/trade-news-dedup-roberta-large-nli"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForSequenceClassification.from_pretrained(repo).eval()

a = "Iraq and Lebanon signed an agreement and seven MoUs on trade and investment."
b = "Iraq and Lebanon announced a new partnership framework, including seven MoUs."

def dup_score(x, y):
    with torch.no_grad():
        logits = model(**tok(x, y, return_tensors="pt", truncation=True)).logits
    return torch.softmax(logits, dim=-1)[0, 0].item()   # index 0 == entailment ~ duplicate

# symmetric signal over both orderings (see finetune_meta.json -> head_info)
p_duplicate = 0.5 * (dup_score(a, b) + dup_score(b, a))
print(p_duplicate)
```

## Limitations

- Summaries are machine-translated to English; quality depends on the preprocessing pipeline.
- Training labels (SilverSet) are LLM-generated (weak supervision) — possible label bias.
- Decision threshold tuned on the evaluation set; validation by a single annotator.
- Evaluated on a single domain (trade / foreign-economic news).

## Links & citation

- 📦 Dataset: https://huggingface.co/datasets/lyutovad/TradeNewsEventDedup
- 💻 Code: https://github.com/SaidKamalov/trade-news-duplicates
- Paper: *Event-Level Duplicate Detection in Trade News under Hard-Negative Supervision* —
  D. Liutova, S. Kamalov, A. Afanasev, T. Mukhtarov.

```bibtex
@misc{tradenews_event_dedup,
  title  = {Event-Level Duplicate Detection in Trade News under Hard-Negative Supervision},
  author = {Liutova, Daria and Kamalov, Said and Afanasev, Andrew and Mukhtarov, Timerlan},
  year   = {2026},
  note   = {Dataset: lyutovad/TradeNewsEventDedup; Code: https://github.com/SaidKamalov/trade-news-duplicates}
}
```