---
license: apache-2.0
language:
- ar
base_model:
- oddadmix/50M-2048-Emhotob
datasets:
- oddadmix/4.1-mini-18-yearsold-10K
library_name: transformers
pipeline_tag: text-generation
tags:
- arabic
- egyptian
- persona
- small-language-model
- slm
- tiny
- llama
- from-scratch
- proof-of-concept
---

# Nawah-50M — Egyptian 18-Year-Old Persona (نواة)

**Nawah (نواة, "nucleus/seed")** is a **~51.8M-parameter** Arabic small language model
fine-tuned to speak with the voice of an **18-year-old Egyptian persona**. It is a
proof-of-concept for training capable Arabic models **from scratch at tiny scale**
— small enough to run on a CPU, yet coherent in Modern Standard Arabic with an
Egyptian cultural frame.

- **Base model:** [`oddadmix/50M-2048-Emhotob`](https://huggingface.co/oddadmix/50M-2048-Emhotob)
  — a Llama-architecture model pre-trained **from scratch on ~20B Arabic tokens** with a
  **2048-token** context window.
- **Fine-tuning data:** [`oddadmix/4.1-mini-18-yearsold-10K`](https://huggingface.co/datasets/oddadmix/4.1-mini-18-yearsold-10K)
  — 9.8K persona-grounded Arabic examples (see [Training data](#training-data)).
- **Architecture & recipe:** derived from [`SupraLabs/Supra-50M-Base`](https://huggingface.co/SupraLabs/Supra-50M-Base).
  This project is a **POC for training Arabic tiny models from scratch**.

> **الملخص بالعربية:** «نواة» نموذج لغوي عربي صغير (~51.8 مليون معامل) مبني على معمارية
> Llama ومدرَّب من الصفر، ثم ضُبط ليتحدث بشخصية شاب مصري في الثامنة عشرة من عمره.
> نموذج تجريبي (Proof of Concept) لإثبات إمكانية تدريب نماذج عربية مفيدة على نطاق صغير
> جدًا يمكن تشغيلها حتى على المعالج (CPU). التفاصيل الكاملة في الأقسام أدناه.

---

## Model details

| | |
|---|---|
| **Parameters** | ~51.8M |
| **Architecture** | Llama (`LlamaForCausalLM`) |
| **Hidden size** | 512 · **Layers** 12 · **Heads** 8 (GQA, 4 KV) · **head_dim** 64 |
| **Vocab size** | 32002 (32000 base + 2 ChatML control tokens) |
| **Context length** | 2048 |
| **Tied embeddings** | yes |
| **Chat format** | ChatML (`<|im_start|>` / `<|im_end|>`) |
| **Precision** | bfloat16 |
| **License** | Apache-2.0 (matches the base architecture) |

---

## Training data

Nawah is fine-tuned on **[`oddadmix/4.1-mini-18-yearsold-10K`](https://huggingface.co/datasets/oddadmix/4.1-mini-18-yearsold-10K)** — a
**synthetic, persona-grounded** Arabic dataset of **9,800 examples** covering the life
of an 18-year-old Egyptian (family dynamics, street culture, transport, school/university,
social expectations, and more). Each row carries rich conditioning metadata
(`life_stage`, `domain`, `persona_seed`, `socioeconomic_tilt`) and multiple output
formats (`factual_qa`, `open_ended_qa`, `multi_turn`), each with an LLM-as-a-judge
`quality_evaluation` score.

**Provenance of the data:**

- **Distilled from [`Qwen/Qwen3.5-9B`](https://huggingface.co/Qwen/Qwen3.5-9B)** — the
  teacher model that generated the persona conversations and answers.
- **Generated with [NVIDIA NeMo Data Designer](https://github.com/NVIDIA-NeMo/DataDesigner)**
  — NVIDIA's framework for producing high-quality synthetic data from scratch or from
  seeds, with dependency-aware fields, statistical samplers, and built-in
  LLM-as-a-judge quality scoring. The dataset's schema (persona seeds,
  socioeconomic tilt, per-row quality reasoning + score) reflects a Data Designer
  configuration.

> **البيانات:** ضُبط النموذج على مجموعة بيانات اصطناعية مبنية على شخصية مصرية عمرها 18 عامًا
> (9.8 ألف مثال). البيانات **مُقطَّرة من نموذج Qwen 3.5 9B** وتم **توليدها باستخدام
> [NVIDIA NeMo Data Designer](https://github.com/NVIDIA-NeMo/DataDesigner)**.

---

## Usage

The tokenizer uses the `TokenizersBackend` class, which requires **`transformers>=5.12`**.
There is no packaged chat template, so build the ChatML prompt manually:

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "oddadmix/Nawah-50M-Egyptian-18y-Persona"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16)

def chat(user_msg, system="أنت شاب مصري في الثامنة عشرة من عمرك."):
    prompt = (
        f"<|im_start|>system\n{system}<|im_end|>\n"
        f"<|im_start|>user\n{user_msg}<|im_end|>\n"
        f"<|im_start|>assistant\n"
    )
    ids = tok(prompt, return_tensors="pt").to(model.device)
    out = model.generate(
        **ids, max_new_tokens=256, do_sample=True,
        temperature=0.7, top_p=0.9, repetition_penalty=1.2,
        eos_token_id=tok.convert_tokens_to_ids("<|im_end|>"),
    )
    return tok.decode(out[0][ids.input_ids.shape[1]:], skip_special_tokens=True)

print(chat("احكيلي عن يومك في الجامعة."))
```

You can also try it in the demo Spaces:
[Nawah-50M-Demo](https://huggingface.co/spaces/oddadmix/Nawah-50M-Demo) ·
[Nawah-50M-Demo0ts](https://huggingface.co/spaces/oddadmix/Nawah-50M-Demo0ts).

---

## Intended use & limitations

**Intended use.** Research and demonstration of persona-conditioned Arabic generation at
tiny scale; lightweight, CPU-friendly Arabic text generation; a baseline for
from-scratch Arabic SLM experiments.

**Limitations.** At ~50M parameters this is a **proof of concept**, not a
production assistant. Expect limited world knowledge, weak long-form reasoning, and
occasional repetition or factual errors — use sampling with a `repetition_penalty`.
The persona and its cultural framing come from **synthetic** data, so outputs reflect
the teacher model's and the persona seed's assumptions rather than any real individual.
Do not rely on it for factual, medical, legal, or financial advice.

---

## Citation & credits

- **Base architecture & training scripts:** [`SupraLabs/Supra-50M-Base`](https://huggingface.co/SupraLabs/Supra-50M-Base) (Apache-2.0).
- **Base Arabic model:** [`oddadmix/50M-2048-Emhotob`](https://huggingface.co/oddadmix/50M-2048-Emhotob) — from-scratch, ~20B Arabic tokens, 2048 ctx.
- **Fine-tuning data:** [`oddadmix/4.1-mini-18-yearsold-10K`](https://huggingface.co/datasets/oddadmix/4.1-mini-18-yearsold-10K) — distilled from [`Qwen/Qwen3.5-9B`](https://huggingface.co/Qwen/Qwen3.5-9B), generated with [NVIDIA NeMo Data Designer](https://github.com/NVIDIA-NeMo/DataDesigner).