zodumair/sifta-fakenews-xlmr

SIFTA — Multilingual Fake News Detector (EN + UR)

Model: Fine-tuned xlm-roberta-base for binary fake news classification Languages: English · Urdu (اردو) Labels: TRUE (0) · FAKE (1) Student: Muhammad Umair Khan · 2023F-BCET-010 Institute: Sir Syed University of Engineering and Technology, Karachi Project: SIFTA — AI Misinformation Detection Platform (Final Year Project 2025–26)

---

Model Description

This model fine-tunes XLM-RoBERTa base on a combined English + Urdu fake news corpus. It is the core ML component of SIFTA — a multi-modal AI misinformation detection platform that also performs voice clone detection, document forgery analysis, and live source verification.

The raw probability output is passed through SIFTA's Decision Engine alongside live source verification signals to produce a five-class Trust Score verdict: Verified Real · Likely Real · Uncertain · Unable to Verify · Likely Fake.

---

Training Data

Dataset	Samples (train)	Language
ISOT	32482	English
UrduFakeNews-AxToGrind	8054	Urdu
Total	40536	EN + UR

Dataset details

Urdu — Ax-to-Grind UrduFakeNews Corpus

• 10,106 news items from Pakistani social media and news outlets
• Binary labels: TRUE / FAKE
• Near-balanced: ~5,039 FAKE / ~4,997 TRUE
• Average length: 351 characters per item

LIAR (Wang, 2017)

• 12,836 political statements from PolitiFact
• 6-way labels mapped to binary:
  • pants-fire / false / barely-true → FAKE
  • half-true / mostly-true / true → TRUE

ISOT Fake News Dataset (University of Victoria)

• ~44,000 real/fake news articles
• Real: Reuters. Fake: unreliable sources. Already binary labelled.

---

Training Configuration

Parameter	Value
Base model	xlm-roberta-base
Max token length	256
Epochs	5
Batch size	16 x 2 grad accum = 32 effective
Learning rate	2e-05
LR schedule	Cosine with 10% linear warmup
Weight decay	0.01
Dropout	0.1
Loss	CrossEntropyLoss with inverse-frequency class weights
Early stopping	Patience 3 on val macro-F1
Mixed precision	fp16 on T4 GPU
Random seed	42

---

Evaluation Results

Overall (held-out test set — 10% stratified)

Metric	Score
Accuracy	0.9696
Macro F1	0.9695
TRUE (class 0) F1	0.9676
FAKE (class 1) F1	0.9713

Per-language macro-F1

Language	Macro F1
EN	0.9854
UR	0.9053

Full classification report

              precision    recall  f1-score   support

        TRUE     0.9632    0.9721    0.9676      2369
        FAKE     0.9753    0.9674    0.9713      2698

    accuracy                         0.9696      5067
   macro avg     0.9693    0.9698    0.9695      5067
weighted avg     0.9697    0.9696    0.9696      5067

Confusion Matrix

	Predicted TRUE	Predicted FAKE
Actual TRUE	2,303 (TN)	66 (FP)
Actual FAKE	88 (FN)	2,610 (TP)

• TRUE recall: 97.2%
• FAKE recall: 96.7%

---

Usage

HuggingFace Inference API (used by SIFTA backend)

import httpx

API_URL = "https://api-inference.huggingface.co/models/zodumair/sifta-fakenews-xlmr"
headers = {"Authorization": "Bearer hf_your_token_here"}

def classify(text):
    return httpx.post(API_URL, headers=headers, json={'inputs': text}).json()

print(classify("Scientists confirm new vaccine is 77% effective."))
print(classify("حکومت نے خفیہ طور پر ملک کا سارا سونا بیرون ملک بھیج دیا"))

Local inference

from transformers import pipeline

classifier = pipeline("text-classification", model="zodumair/sifta-fakenews-xlmr")
print(classifier("SHOCKING: Scientists hiding cancer cure from the public."))
# [{'label': 'FAKE', 'score': 0.93}]

---

Limitations

• Optimised for headlines and short excerpts under 256 tokens
• Urdu performance is lower than English due to smaller Urdu training set
• Satire and opinion pieces may be misclassified
• Use alongside source verification — not as a standalone decision tool

---

Citation

@misc{sifta2026,
  author      = {Muhammad Umair Khan},
  title       = {SIFTA: AI Misinformation Detection Platform},
  year        = {2026},
  institution = {Sir Syed University of Engineering and Technology, Karachi},
  note        = {Final Year Project, Batch 2023F}
}

---

SIFTA FYP SSUET Karachi · Batch 2023F · 2023F-BCET-010