README.md

---
language:
- en
- ur
license: mit
tags:
- text-classification
- fake-news-detection
- misinformation
- xlm-roberta
- urdu
- english
- multilingual
datasets:
- liar
- GonzaloA/fake_news
- UrduFakeNews-AxToGrind
metrics:
- accuracy
- f1
model-index:
- name: sifta-fakenews-xlmr
  results:
  - task:
      type: text-classification
      name: Fake News Detection
    metrics:
    - type: accuracy
      value: 0.9696
    - type: f1
      value: 0.9695
      name: Macro F1
---

# SIFTA — Multilingual Fake News Detector (EN + UR)

**Model:** Fine-tuned `xlm-roberta-base` for binary fake news classification
**Languages:** English · Urdu (اردو)
**Labels:** `TRUE` (0) · `FAKE` (1)
**Student:** Muhammad Umair Khan · 2023F-BCET-010
**Institute:** Sir Syed University of Engineering and Technology, Karachi
**Project:** SIFTA — AI Misinformation Detection Platform (Final Year Project 2025–26)

---

## Model Description

This model fine-tunes [XLM-RoBERTa base](https://huggingface.co/xlm-roberta-base)
on a combined English + Urdu fake news corpus. It is the core ML component of
SIFTA — a multi-modal AI misinformation detection platform that also performs
voice clone detection, document forgery analysis, and live source verification.

The raw probability output is passed through SIFTA's Decision Engine alongside
live source verification signals to produce a five-class Trust Score verdict:
Verified Real · Likely Real · Uncertain · Unable to Verify · Likely Fake.

---

## Training Data

| Dataset | Samples (train) | Language |
|---|---|---|
| ISOT | 32482 | English |
| UrduFakeNews-AxToGrind | 8054 | Urdu |
| **Total** | **40536** | EN + UR |

### Dataset details

**Urdu — Ax-to-Grind UrduFakeNews Corpus**
- 10,106 news items from Pakistani social media and news outlets
- Binary labels: TRUE / FAKE
- Near-balanced: ~5,039 FAKE / ~4,997 TRUE
- Average length: 351 characters per item

**LIAR (Wang, 2017)**
- 12,836 political statements from PolitiFact
- 6-way labels mapped to binary:
  - pants-fire / false / barely-true → FAKE
  - half-true / mostly-true / true → TRUE

**ISOT Fake News Dataset (University of Victoria)**
- ~44,000 real/fake news articles
- Real: Reuters. Fake: unreliable sources. Already binary labelled.

---

## Training Configuration

| Parameter | Value |
|---|---|
| Base model | xlm-roberta-base |
| Max token length | 256 |
| Epochs | 5 |
| Batch size | 16 x 2 grad accum = 32 effective |
| Learning rate | 2e-05 |
| LR schedule | Cosine with 10% linear warmup |
| Weight decay | 0.01 |
| Dropout | 0.1 |
| Loss | CrossEntropyLoss with inverse-frequency class weights |
| Early stopping | Patience 3 on val macro-F1 |
| Mixed precision | fp16 on T4 GPU |
| Random seed | 42 |

---

## Evaluation Results

### Overall (held-out test set — 10% stratified)

| Metric | Score |
|---|---|
| Accuracy | 0.9696 |
| Macro F1 | 0.9695 |
| TRUE (class 0) F1 | 0.9676 |
| FAKE (class 1) F1 | 0.9713 |

### Per-language macro-F1

| Language | Macro F1 |
|---|---|
| EN | 0.9854 |
| UR | 0.9053 |

### Full classification report

```
              precision    recall  f1-score   support

        TRUE     0.9632    0.9721    0.9676      2369
        FAKE     0.9753    0.9674    0.9713      2698

    accuracy                         0.9696      5067
   macro avg     0.9693    0.9698    0.9695      5067
weighted avg     0.9697    0.9696    0.9696      5067

```
### Confusion Matrix

![Confusion Matrix](confusion_matrix.png)

| | Predicted TRUE | Predicted FAKE |
|---|---|---|
| **Actual TRUE** | 2,303 (TN) | 66 (FP) |
| **Actual FAKE** | 88 (FN) | 2,610 (TP) |

- **TRUE recall:** 97.2%
- **FAKE recall:** 96.7%

---

## Usage

### HuggingFace Inference API (used by SIFTA backend)

```python
import httpx

API_URL = "https://api-inference.huggingface.co/models/zodumair/sifta-fakenews-xlmr"
headers = {"Authorization": "Bearer hf_your_token_here"}

def classify(text):
    return httpx.post(API_URL, headers=headers, json={'inputs': text}).json()

print(classify("Scientists confirm new vaccine is 77% effective."))
print(classify("حکومت نے خفیہ طور پر ملک کا سارا سونا بیرون ملک بھیج دیا"))
```
### Local inference

```python
from transformers import pipeline

classifier = pipeline("text-classification", model="zodumair/sifta-fakenews-xlmr")
print(classifier("SHOCKING: Scientists hiding cancer cure from the public."))
# [{'label': 'FAKE', 'score': 0.93}]
```
---

## Limitations

- Optimised for headlines and short excerpts under 256 tokens
- Urdu performance is lower than English due to smaller Urdu training set
- Satire and opinion pieces may be misclassified
- Use alongside source verification — not as a standalone decision tool

---

## Citation

```
@misc{sifta2026,
  author      = {Muhammad Umair Khan},
  title       = {SIFTA: AI Misinformation Detection Platform},
  year        = {2026},
  institution = {Sir Syed University of Engineering and Technology, Karachi},
  note        = {Final Year Project, Batch 2023F}
}
```
---
*SIFTA FYP SSUET Karachi · Batch 2023F · 2023F-BCET-010*