Update model card: fix numbers, improve clarity, add role description

README.md

@@ -18,15 +18,17 @@ pipeline_tag: text-generation
 library_name: peft
 ---
 
-# Sawb — Qwen2.5-7B-Instruct (LoRA SFT)
+# Sawb — Qwen2.5-7B-Instruct (LoRA SFT — Research Baseline)
+
+Part of the **[Sawb Arabic Cultural Hallucination Detection Collection](https://huggingface.co/collections/HassanB4/sawb-arabic-cultural-hallucination-detection)** for ICAIRE 2026 Track 3.
 
 ## Overview
 
-**Sawb — Qwen2.5-7B-Instruct (LoRA SFT)** is an Arabic-language generative model for detecting and *explaining* cultural hallucinations in LLM outputs. A cultural hallucination occurs when an LLM produces a response that is factually or culturally incorrect within Arab/Islamic contexts.
+**Sawb — Qwen2.5-7B** is a LoRA adapter fine-tuned from [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) (7B parameters) for Arabic cultural hallucination detection and explanation. This model is released as a **research baseline** evaluated during the Sawb system development.
 
-This model is a LoRA adapter fine-tuned from [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) (7B parameters) using supervised fine-tuning (SFT) on the [Sawb Arabic Cultural Hallucination Dataset](https://huggingface.co/datasets/HassanB4/sawb-arabic-hallucination-dataset).
+It achieved a macro F1 of **0.5556** on the 457-example validation set — significantly below the Arabic BERT encoder models (F1 = 0.9246–0.9647) and below the DeepSeek-based pipeline. The underperformance is attributed to Qwen2.5-7B lacking the deep Arabic dialectal and Islamic cultural knowledge needed for the most challenging hallucination categories (dialectal confusion, religious misrepresentation).
 
-Qwen2.5-7B-Instruct fine-tuned with LoRA for Arabic cultural hallucination detection and explanation. Produces structured JSON output with hallucination label, category, and Arabic explanation.
+For production use, see the primary detection model: [HassanB4/sawb](https://huggingface.co/HassanB4/sawb) (AraBERT-Large + Glossary, F1=0.9246).
 
 ## Model Architecture
 
@@ -38,7 +40,7 @@ Qwen2.5-7B-Instruct fine-tuned with LoRA for Arabic cultural hallucination detec
 | LoRA alpha (α) | 8 |
 | LoRA dropout | 0.05 |
 | Target modules | `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj` |
-| Task type | `CAUSAL_LM` |
+| Task type | Causal Language Modeling |
 | Parameters (base) | 7B |
 
 ## Training
@@ -47,13 +49,19 @@ Qwen2.5-7B-Instruct fine-tuned with LoRA for Arabic cultural hallucination detec
 |---|---|
 | Training examples | 1,828 |
 | Method | Supervised Fine-Tuning (SFT) |
-| Framework | PEFT 0.19.1 + TRL 0.24.0 |
-| Transformers version | 5.5.0 |
-| PyTorch | 2.4.1+cu124 |
+| Framework | PEFT + TRL |
+
+## Evaluation Results
+
+| Metric | Value |
+|---|---|
+| Macro F1 (validation) | **0.5556** |
+| Task | Binary classification (hallucination / not) |
+| Evaluation set | 457 Arabic (question, LLM answer) pairs |
 
 ## Output Format
 
-The model is trained to produce structured JSON output:
+The model is trained to output structured JSON:
 
 ```json
 {
@@ -69,32 +77,28 @@ The model is trained to produce structured JSON output:
 ```python
 from peft import PeftModel
 from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
 
 base_model = "Qwen/Qwen2.5-7B-Instruct"
 adapter = "HassanB4/sawb-qwen25"
 
 tokenizer = AutoTokenizer.from_pretrained(base_model)
-model = AutoModelForCausalLM.from_pretrained(base_model, torch_dtype="auto", device_map="auto")
+model = AutoModelForCausalLM.from_pretrained(base_model, torch_dtype=torch.float16, device_map="auto")
 model = PeftModel.from_pretrained(model, adapter)
-
-system_prompt = (
-    "أنت نظام متخصص في اكتشاف الهلوسة الثقافية في مخرجات نماذج اللغة العربية. "
-    "مهمتك: تحليل زوج (سؤال، إجابة) وتحديد ما إذا كانت الإجابة تحتوي على هلوسة ثقافية. "
-    "أخرج إجابتك بتنسيق JSON فقط."
-)
+model.eval()
 
 question = "اشرح مفهوم النموذج اللغوي باللهجة النجدية"
 answer = "النموذج اللغوي هو نظام يستخدم الذكاء الاصطناعي لفهم اللغة..."
 
-user_msg = f"السؤال: {question}\n\nإجابة النموذج: {answer}"
-
 messages = [
-    {"role": "system", "content": system_prompt},
-    {"role": "user", "content": user_msg},
+    {"role": "system", "content": "أنت محكم متخصص في الكشف عن الهلوسة الثقافية. أخرج JSON فقط."},
+    {"role": "user", "content": f"السؤال: {question}\n\nإجابة النموذج: {answer}"},
 ]
 
 inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
-outputs = model.generate(inputs, max_new_tokens=256, temperature=0.1, do_sample=True)
+with torch.no_grad():
+    outputs = model.generate(inputs, max_new_tokens=256, temperature=0.1, do_sample=True)
+
 response = tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True)
 print(response)
 ```
@@ -103,14 +107,6 @@ print(response)
 
 Trained on [HassanB4/sawb-arabic-hallucination-dataset](https://huggingface.co/datasets/HassanB4/sawb-arabic-hallucination-dataset).
 
-The dataset covers 6 hallucination categories:
-- `ethical_framework_mismatch`
-- `religious_misrepresentation`
-- `historical_inaccuracy`
-- `social_norms_violation`
-- `dialectal_confusion`
-- `regional_context_errors`
-
 ## Collection
 
-Part of the [Sawb Arabic Cultural Hallucination Detection](https://huggingface.co/collections/HassanB4/sawb-arabic-cultural-hallucination-detection) collection.
+[Sawb Arabic Cultural Hallucination Detection](https://huggingface.co/collections/HassanB4/sawb-arabic-cultural-hallucination-detection)