Deehan1866/finetuned-deberta-wic-actual-version2

DeBERTa-v3-large Fine-tuned on WiC

Cross-encoder model for the Word-in-Context (WiC) binary sense disambiguation task. Both sentences are fed together so the model can attend across them simultaneously.

Target Word Marking

The target word is wrapped with <tgt>word</tgt> using the exact token position from the dataset (start1/start2 columns), so marking is always precise regardless of lemma or morphological variation.

Training Details

Hyperparameter	Value
Learning rate	8e-6
Effective batch	32
Epochs	10
Warmup ratio	0.15
Dropout	0.2
Label smoothing	0.05
LLRD decay	0.9

Performance

Split	Accuracy
Validation	0.7179
Test	0.7171

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

tokenizer = AutoTokenizer.from_pretrained("Deehan1866/finetuned-deberta-wic-actual-version2")
model = AutoModelForSequenceClassification.from_pretrained("Deehan1866/finetuned-deberta-wic-actual-version2")

s1 = "The <tgt>bank</tgt> raised its interest rates."
s2 = "She visited her local <tgt>bank</tgt> to deposit a cheque."

enc = tokenizer(s1, s2, return_tensors="pt", truncation=True, max_length=256)
with torch.no_grad():
    logits = model(**enc).logits
pred = torch.argmax(logits).item()
print("Same sense" if pred == 1 else "Different sense")