---
license: cc-by-nc-4.0
base_model: UBC-NLP/NileChat-3B-Base
base_model_relation: adapter
library_name: peft
pipeline_tag: text-generation
language:
- ar
- en
tags:
- translation
- arabic
- dialectal-arabic
- nilechat
- lora
- qlora
- peft
- back-translation
- data-augmentation
- alexandriax-2026
- arabicnlp2026
- shared-task
datasets:
- UBC-NLP/alexandria
- NAMAA-Space/alexandria-backtranslated-pairs
metrics:
- bleu
- chrf
model-index:
- name: alexandriax-nilechat-ctx-aux
  results:
  - task:
      type: translation
      name: Context-Aware English-to-Dialectal Arabic Dialogue Translation
    dataset:
      name: AlexandriaX-2026 Subtask 1 (development)
      type: UBC-NLP/alexandria
      split: dev
    metrics:
    - type: spbleu
      value: 22.77
      name: spBLEU (FLORES-200 tokenizer, country-macro)
    - type: chrf++
      value: 38.71
      name: chrF++ (word_order=2, country-macro)
---

# AlexandriaX-2026 · Subtask 1 — NileChat-3B QLoRA, context + back-translation (`nilechat_ctx_aux`)

English to **dialectal Arabic** dialogue translation over **13 Arabic varieties**, from the
**NAMAA Community** submission to **AlexandriaX-2026** (ArabicNLP 2026 / EMNLP). A QLoRA
adapter over [`UBC-NLP/NileChat-3B-Base`](https://huggingface.co/UBC-NLP/NileChat-3B-Base),
trained on the official turns **with conversation context** and augmented with
**348,787 synthetic back-translated pairs** built from a 358k-sentence dialectal corpus.

This is the **data-augmentation arm** of the team's NileChat ablation. Its result is negative
and worth publishing as such: augmenting with 5.5x more synthetic than real data **did not
beat** the plain context-free adapter on the official development set (22.77 vs 23.54 spBLEU).
No submitted system uses back-translated data because of this measurement.

| | |
|---|---|
| **Task** | AlexandriaX-2026 Subtask 1 (context-aware EN→DA dialogue translation) |
| **Base model** | `UBC-NLP/NileChat-3B-Base` (decoder-only, approx. 3B parameters, Arabic-dialect pretraining) |
| **Adapter** | LoRA r=16, α=32, dropout 0.05 — 119.8 MB, approx. 30M trainable parameters |
| **Dialect control** | prompt field (`Country`, dialect name) |
| **Context** | **yes** — the model's own previous turns are fed back as history |
| **Extra data** | [`NAMAA-Space/alexandria-backtranslated-pairs`](https://huggingface.co/datasets/NAMAA-Space/alexandria-backtranslated-pairs) (348,787 synthetic pairs) |
| **Dev** (12,250 turns, 11 countries) | **22.77** spBLEU · **38.71** chrF++ |
| **Internal hold-out** (3,350 turns, 5% of train) | 24.36 spBLEU · 40.16 chrF++ |
| **Blind test** | not run (this system was not part of a submitted bundle) |
| **Track** | constrained (provided data only, ≤5B parameters) |
| **License** | CC-BY-NC-4.0 — check `UBC-NLP/NileChat-3B-Base` for the base model's terms |

## Usage

The adapter was trained on one exact prompt format, with the conversation history filled in.
Reproduce it character-for-character, and feed back the model's **own** previous translations
turn by turn — that is how it was trained and how it was scored.

```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel

BASE = "UBC-NLP/NileChat-3B-Base"
ADAPTER = "NAMAA-Space/alexandriax-nilechat-ctx-aux"

bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
                         bnb_4bit_use_double_quant=True,
                         bnb_4bit_compute_dtype=torch.bfloat16)

tok = AutoTokenizer.from_pretrained(BASE, trust_remote_code=True)
tok.padding_side = "left"
if tok.pad_token is None:
    tok.pad_token = tok.eos_token

base = AutoModelForCausalLM.from_pretrained(BASE, quantization_config=bnb,
                                            device_map="auto", trust_remote_code=True)
model = PeftModel.from_pretrained(base, ADAPTER).eval()


def format_history(history):
    """history: list of {"speaker", "sentence", "translation"} for previous turns."""
    if not history:
        return "No previous turns (Start of conversation)."
    return "\n".join(f"{h.get('speaker') or 'Speaker'}: {h['sentence']}\n"
                     f"Translation: {h['translation']}" for h in history)


def build_prompt(dialect, country, domain, participants, speaker, direction, sentence,
                 history=None):
    return (f"You are an expert translator. Translate the English sentence into {dialect}.\n\n"
            f"### Metadata:\n- Country: {country}\n- Domain: {domain}\n"
            f"- Participants: {participants}\n- Speaker: {speaker}\n"
            f"- Speaker Direction: {direction}\n\n"
            f"### Conversation History:\n{format_history(history)}\n\n"
            f"### Sentence to Translate:\n{sentence.strip()}\n\n### Translation:\n")


def clean_generation(text):
    text = text.strip()
    for marker in ("\n###", "### Sentence to Translate:", "### Translation:"):
        if marker in text:
            text = text.split(marker, 1)[0].strip()
    return text


def translate_conversation(turns, **meta):
    """turns: list of {"speaker", "direction", "sentence"} in turn order."""
    history, out = [], []
    for t in turns:
        prompt = build_prompt(sentence=t["sentence"], speaker=t["speaker"],
                              direction=t["direction"], history=history, **meta)
        enc = tok([prompt], return_tensors="pt", padding=True, truncation=True,
                  max_length=1024).to(model.device)
        with torch.no_grad():
            gen = model.generate(**enc, max_new_tokens=128, do_sample=False)
        pred = clean_generation(
            tok.decode(gen[0][enc["input_ids"].shape[1]:], skip_special_tokens=True))
        out.append(pred)
        history.append({"speaker": t["speaker"], "sentence": t["sentence"],
                        "translation": pred})      # the model's own output, not gold
    return out


print(translate_conversation(
    [{"speaker": "Wholesale Buyer", "direction": "male -> female",
      "sentence": "Good morning. How much for the whole quantity?"},
     {"speaker": "Farmer", "direction": "female -> male",
      "sentence": "For you, a special price. Eight pounds a kilo."}],
    dialect="Egyptian Arabic (Cairene) Dialect", country="EG",
    domain="Agriculture and farming",
    participants="Wholesale Buyer, Wholesale Seller"))
```

Reference decoding for every number in this card: **greedy** (`do_sample=False`),
`max_new_tokens=128`, prompt truncated at 1024 tokens, left-padded batches.

> [!NOTE]
> Never place **gold** previous-turn Arabic in the history block when measuring. An earlier
> harness in this project did, and it inflated held-out scores by roughly **+2.4 spBLEU**.
> Every number in this card comes from the corrected, self-conditioned harness.

### Intended use

Research on data augmentation for low-resource dialectal MT — specifically as the evidence
that NLLB-generated back-translation into Arabic dialects does not transfer to this task's
reference style. Not for production translation without human review.

---

## The shared task

**AlexandriaX-2026** (ArabicNLP 2026 / EMNLP) — *Context-Aware Dialectal Arabic MT and MT Evaluation*.
This model was built for **Subtask 1: Context-Aware English-to-Dialectal Arabic Dialogue Translation**.

Given one **English dialogue turn** together with its **conversation history** and metadata —
target country/dialect, domain, participant roles, speaker, and speaker→addressee gender
direction — the system must produce the turn in the requested country's spoken Arabic,
preserving meaning while adapting lexical, morphological, pragmatic and sociolinguistic
choices to that variety.

Two tracks: **constrained** (provided data only, ≤5B parameters) and **unconstrained**
(any external data or model). Ranking is by **spBLEU** (primary) and **chrF++** (secondary),
each macro-averaged over countries.

### Official data (`UBC-NLP/alexandria`)

Split sizes in **turns**, as published by the organisers:

| Split | EG | JO | LB | LY | MA | MR | OM | PS | SA | SD | SY | TN | YE | **Total** |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| train | 3,108 | 5,501 | 8,906 | 0 | 2,573 | 5,515 | 6,280 | 14,933 | 8,470 | 0 | 6,071 | 2,034 | 3,089 | **66,480** |
| dev | 1,113 | 1,113 | 1,118 | 0 | 1,110 | 1,114 | 1,109 | 1,110 | 1,110 | 0 | 1,119 | 1,116 | 1,118 | **12,250** |
| public test | 1,118 | 1,107 | 1,106 | 1,109 | 1,115 | 1,112 | 1,118 | 1,109 | 1,113 | 1,106 | 1,114 | 1,109 | 1,106 | **14,442** |
| private (blind) test | 1,113 | 1,109 | 1,110 | 1,309 | 1,111 | 1,119 | 1,107 | 1,111 | 1,114 | 915 | 1,114 | 1,114 | 1,113 | **14,459** |

**Libyan (LY)** and **Sudanese (SD)** appear only at test time — they are **zero-shot** for
every system trained on this data.

Conversation-level counts: **21,146** train / **3,963** dev / **4,706** public-test
conversations; mean **3.13** turns per conversation (range 1–5). Mean length 102 characters
of English source, 74 characters of dialectal target.

**Dialects (13 countries).** Egyptian, Jordanian, Lebanese, Libyan, Moroccan, Mauritanian,
Omani, Palestinian, Saudi, Sudanese, Syrian, Tunisian, Yemeni. Labels are country + **sub-dialect**,
and several countries carry more than one: Palestinian 10 (Nabulsi and Albira urban, plus
Falahi varieties of Surif, Kobar, Noba, Ni'lin, Shuqba, Aboud, Silwad, Ramallah), Omani 5
(Suri, Rustaqi, Al-Wafi, Ibri, Seebi), Saudi 3 (Southern, Hijazi, Khaleeji), Yemeni 3
(Taiz, San'ani, Central), Syrian 2 (Levantine Standard, Homsi). The remaining countries carry
one label each (e.g. *Egyptian Arabic (Cairene)*, *Moroccan Standard Darija*,
*Mauritanian Hassaniya*, *Libyan Arabic (Misrati/Central)*).

**Domains (11, near-uniform).** Agriculture and farming, Commerce and transactions,
Construction and real estate, Education and academia, Energy and resources,
Everyday and social, Healthcare and medical, Legal and financial,
Logistics and transportation, Professional and workplace, Tourism and hospitality.

**Speaker direction** (turns, train+dev+public test): female→male 30,636 · male→female 30,203 ·
male→male 20,465 · female→female 11,868. The corpus carries 76 distinct translator IDs and
44 reviewer IDs.

**Code-switching in the gold** is strongly dialect-specific — the share of gold turns
containing Latin characters runs from **TN 39.1% / MA 33.8% / LB 18.1%** down to
**SY 1.2% / YE 0.8%**. Systems that normalise every borrowing into Arabic script are
penalised hardest on Maghrebi references (see *Known limitations*).

### Evaluation protocol

- **spBLEU** — `sacrebleu.BLEU(tokenize="flores200")`, corpus-level per country, then averaged over countries.
- **chrF++** — `sacrebleu.CHRF(word_order=2)`, same averaging.
- Decoding is **turn-by-turn**: at turn *n* the conversation history contains the system's
  **own** previous outputs, never the gold ones. (An early evaluation harness in this project
  leaked gold previous-turn Arabic into the prompt and inflated scores by ≈2.4 spBLEU; every
  number reported here comes from the corrected, self-conditioned harness.)

---

## Training data

Two sources, concatenated into one supervised mixture of **411,917 examples**:

| Source | Examples | Share | What it is |
|---|---|---|---|
| Official Subtask-1 train (context prompts) | 63,130 | 15.3% | human-translated dialogue turns, 11 countries, history filled with the previous turns |
| [`alexandria-backtranslated-pairs`](https://huggingface.co/datasets/NAMAA-Space/alexandria-backtranslated-pairs) | 348,787 | 84.7% | synthetic single-turn pairs, 14 varieties, history always empty |

**Real data, per country** (turns):

| PS | LB | SA | OM | SY | MR | JO | YE | EG | MA | TN |
|---|---|---|---|---|---|---|---|---|---|---|
| 14,183 | 8,464 | 8,035 | 5,965 | 5,760 | 5,234 | 5,224 | 2,946 | 2,943 | 2,443 | 1,933 |

**Synthetic data, per country** (pairs):

| SA | SY | MA | PS | DZ | EG | LB | JO | TN | OM | LY | SD | YE | MR |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 50,000 | 50,000 | 46,286 | 30,538 | 30,078 | 28,197 | 23,467 | 21,185 | 15,204 | 13,812 | 13,587 | 12,251 | 12,031 | 2,151 |

The synthetic side is the one place where **LY**, **SD** and Algerian (**DZ**, not a task
variety) get any training signal at all — 13,587 and 12,251 pairs for the two zero-shot test
varieties.

**How the synthetic pairs were made.** A monolingual corpus of **358,483** cleaned dialectal
sentences was assembled from public sources — IADD, QADI, NADI-2021, MADAR, and Egyptian,
Saudi, Moroccan, Levantine, Libyan, Sudanese and Yemeni web and transcript collections
(transcript columns only; no audio was ingested). URLs, mentions, hashtags, timestamps and
emoji were stripped, **all Latin letters deleted**, punctuation blanked, character runs
collapsed, length clipped to 3–1000 characters and near-duplicates removed. Each surviving
sentence was back-translated to English with **`facebook/nllb-200-distilled-600M`** (fp16,
greedy, 128 new tokens, batch 256, capped at 50,000 sentences per variety) and the pair
inverted into the training prompt, yielding 348,787 English→dialect pairs. Full parameters and
the build script are in the
[dataset card](https://huggingface.co/datasets/NAMAA-Space/alexandria-backtranslated-pairs).

> Two filter choices deserve flagging in hindsight. **Latin script is deleted**, so no
> synthetic target contains a single Latin token — while 33.8% of Moroccan and 39.1% of Tunisian
> *gold* turns do. And **punctuation is stripped** from the targets but not from the English
> prompts. Both push the model away from the reference style of exactly the varieties the
> augmentation was meant to help.

## Training procedure — every hyperparameter

Extracted verbatim from `AlexandriaX_NB2_Finetune.ipynb`. Runnable single-file version, which
reproduces all three variants behind one `--variant` flag:
**[`train_nilechat_qlora.py`](./train_nilechat_qlora.py)**.

```bash
python train_nilechat_qlora.py --variant ctx_aux --out ./runs/nilechat_ctx_aux
```

Every setting below is identical to the context-free sibling except the **training mixture**
and therefore the **step count** — that is what makes this an ablation rather than a
different system.

### Model, adapter and data

| Setting | Value | Note |
|---|---|---|
| Base model | `UBC-NLP/NileChat-3B-Base` | decoder-only; the organisers' baseline model |
| Regime | **QLoRA** | 4-bit frozen base + LoRA adapters, SFT via TRL's `SFTTrainer` |
| LoRA rank `r` | **16** | |
| LoRA `alpha` | **32** | scaling 32/16 = 2 |
| LoRA `dropout` | **0.05** | |
| LoRA `bias` | `"none"` | |
| `task_type` | `CAUSAL_LM` | |
| `target_modules` | **`"all-linear"`** | resolved to `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj` |
| Trainable parameters | **≈30M (~1% of the base)** | 119.8 MB adapter |
| Loss | **completion-only** (`completion_only_loss=True`) | the prompt is masked out of the loss |
| Data schema | `{"prompt": ..., "completion": response + eos_token}` | TRL prompt/completion format |
| `max_length` | **1024** tokens | matters more here: context prompts are longer |
| `packing` | **False** | |
| **Training examples** | **411,917** | 63,130 real (with history) **+** 348,787 back-translated |
| Mixture handling | plain concatenation, then shuffle | **no** curriculum, upsampling or loss reweighting |
| `padding_side` | `right` for training, `left` for generation | |

### Quantisation

| Setting | Value |
|---|---|
| `load_in_4bit` | True |
| `bnb_4bit_quant_type` | **`nf4`** |
| `bnb_4bit_use_double_quant` | **True** |
| `bnb_4bit_compute_dtype` | `bfloat16` (fp16 fallback) |
| `device_map` | `"auto"` |
| `prepare_model_for_kbit_training` | called before attaching LoRA |

### Optimisation

| Setting | Value | Note |
|---|---|---|
| Optimiser | **`paged_adamw_8bit`** | |
| Learning rate | **2e-4** | |
| LR scheduler | **`cosine`** | over 6.4k steps here, 987 for the context-free sibling |
| Warmup | **`warmup_ratio=0.03`** | ratio, so ≈193 steps here against ≈30 there |
| Epochs | **1** | over the full mixture |
| `per_device_train_batch_size` | **32** | |
| `gradient_accumulation_steps` | **2** | |
| **Effective batch** | **64** | |
| Optimiser steps | **≈6,436** | 411,917 / 64 — 6.5× the sibling's |
| Weight decay / clipping | 0.0 / 1.0 | framework defaults |

> The step count is a confound worth naming: this variant takes 6.5× as many optimiser steps
> as the context-free sibling it is compared against, because "one epoch" is defined over a
> 6.5× larger mixture. The comparison is therefore *equal-epoch*, not *equal-step*.

### Precision, memory, hardware

| Setting | Value |
|---|---|
| Hardware | **1 × A100-80GB** |
| `bf16` / `fp16` | bf16 True (fp16 as fallback) |
| Attention | FlashAttention-2 when the load succeeds, else the default implementation |
| `gradient_checkpointing` | **True** |
| `use_cache` | `False` during training, `True` for generation |

### Bookkeeping

| Setting | Value |
|---|---|
| `logging_steps` / `save_steps` / `save_total_limit` | 20 / 200 / 2 |
| Resume | automatic from the highest `checkpoint-*` |
| Libraries | `transformers`, `trl`, `peft` 0.19.1, `bitsandbytes` |

### Inference (used for every score in this card)

| Setting | Value |
|---|---|
| Decoding | **greedy** — `do_sample=False`, no beams |
| `max_new_tokens` | 128 |
| Prompt truncation | 1024 tokens |
| Generation batch | up to 512, left-padded |
| Post-processing | split off anything after `\n###`, `### Sentence to Translate:` or `### Translation:` |
| Context | the model's **own** previous outputs, appended turn by turn |

> **Training uses gold history, evaluation uses generated history.** At training time the
> history block holds the gold previous turns (ordinary teacher forcing); at evaluation time it
> must hold the model's own previous outputs. Conflating the two inflated an early measurement
> in this project by roughly **+2.4 spBLEU**.

## Results

### The ablation this checkpoint belongs to

Three variants, identical hyperparameters, differing only in prompt history and training
mixture. Both evaluation sets are decoded turn-by-turn with the model's **own** previous
outputs as history.

| Variant | History in prompt | Extra data | dev spBLEU | dev chrF++ | hold-out spBLEU | hold-out chrF++ |
|---|---|---|---|---|---|---|
| [`nilechat_noctx`](https://huggingface.co/NAMAA-Space/alexandriax-nilechat-lora) | none | — | **23.54** | **39.68** | **24.84** | **40.36** |
| `nilechat_ctx` | own previous turns | — | 22.87 | 39.11 | 23.37 | 39.35 |
| **`nilechat_ctx_aux` (this model)** | own previous turns | 348,787 back-translated pairs | 22.77 | 38.71 | 24.36 | 40.16 |

*dev* = official 12,250-turn development set, 11 countries. *hold-out* = the team's internal
3,350-turn split of the training conversations.

**The two sets disagree, and this is the crux of the card.** On the internal hold-out,
back-translation looks like a clear win over the context variant (+0.99 spBLEU, 24.36 vs
23.37) and nearly catches the context-free model. On the official development set the same
comparison reverses — −0.10 spBLEU and −0.40 chrF++ against the context variant — and the gap
to the context-free model widens to −0.77 spBLEU.
The internal split shares conversations, speakers and phrasing with the training data;
the official dev set does not. **Trust the official dev column** — and, more generally, do not
size an augmentation decision on a hold-out carved from the training conversations.

### Per-country, official dev set (12,250 turns)

| | EG | JO | LB | MA | MR | OM | PS | SA | SY | TN | YE | **macro** |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| **spBLEU** | 26.90 | 29.03 | 25.21 | 15.90 | **9.31** | 22.29 | 25.32 | 25.99 | **29.54** | 22.67 | 18.32 | **22.77** |
| **chrF++** | 41.51 | 44.26 | 40.75 | 31.67 | 26.47 | 39.06 | 41.12 | 42.05 | 45.88 | 37.90 | 35.12 | **38.71** |

### Per-country, internal hold-out (3,350 turns)

| | EG | JO | LB | MA | MR | OM | PS | SA | SY | TN | YE | **macro** |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| **spBLEU** | 24.75 | 29.99 | 25.58 | 19.15 | 14.08 | 22.68 | 25.84 | 30.56 | 31.17 | 26.27 | 17.89 | **24.36** |
| **chrF++** | 39.75 | 45.52 | 41.03 | 35.30 | 30.72 | 39.82 | 41.78 | 46.05 | 47.28 | 39.70 | 34.76 | **40.16** |

### Where the augmentation actually moved things

Against the context-only variant on official dev, adding 348k synthetic pairs helped Egyptian
(+0.34), Tunisian (+1.42) and Omani (+0.87) and hurt Moroccan (−0.82), Syrian (−0.93),
Palestinian (−0.83) and Saudi (−0.84). The two countries that gained most are among the
scarcest in real data; the ones that lost most are among the best resourced, where synthetic
noise displaces good supervision. The net is roughly flat, which is the honest summary:
**NLLB back-translation into Arabic dialects buys little here and costs where real data is
already plentiful.**

## Known limitations

- **Negative result.** Do not pick this checkpoint expecting the augmentation to help; pick
  the context-free sibling for quality, and this one to study or extend the augmentation.
- **Synthetic-heavy mixture.** 84.7% of the training examples are machine-generated, all
  single-turn with empty history, which dilutes the context signal the variant was meant to
  exploit.
- **Latin-script deletion fights the references** on exactly the code-switching varieties
  (MA, TN, LB) that most needed help, and punctuation stripping costs character-level credit
  everywhere — see the note above.
- **Back-translation direction is untested.** The English side comes from NLLB and is not
  human-verified; some of it is visibly disfluent, and the corpus includes informal web text
  whose register differs from the task's role-play dialogues.
- **DZ (Algerian) is in the synthetic data but not in the task**, so approximately 8.6% of the
  mixture targets a variety that is never evaluated.
- **Prompt-format sensitivity**: trained on exactly one template with completion-only loss.
- **LY and SD** get synthetic-only supervision; their blind-test behaviour was never measured
  for this checkpoint.
- **Not evaluated on the blind test**; **metric-only evaluation** (no human, no COMET).
- Base-model licence and usage terms apply; check
  [`UBC-NLP/NileChat-3B-Base`](https://huggingface.co/UBC-NLP/NileChat-3B-Base).

---

## Where this model sits in the NAMAA system

All Subtask-1 systems built by the team, scored on the **official 12,250-turn dev set**
(11 countries) and, where they were run, on the **14,459-turn private blind test**
(13 countries). Country-macro spBLEU / chrF++.

| System | Params / arch. | dev spBLEU | dev chrF++ | blind spBLEU | blind chrF++ | Released |
|---|---|---|---|---|---|---|
| Gemma, beam search *(submitted, constrained)* | ~3.1B, dec-only | — | — | **27.413** | 42.58 | no |
| Routed ensemble *(submitted, unconstrained)* | — | — | — | **27.412** | **43.05** | n/a |
| Gemini 2.5 Flash, 5-shot | API | — | — | 26.68 | 42.49 | n/a |
| Claude Sonnet 4.5, 5-shot | API | — | — | 26.36 | 42.26 | n/a |
| **AraT5v2 full fine-tune** | 368M, enc–dec | **25.12** | **40.66** | **23.26** | **39.03** | [`alexandriax-arat5v2-base`](https://huggingface.co/NAMAA-Space/alexandriax-arat5v2-base) |
| Qwen2.5-1.5B LoRA | 1.5B, dec-only | 23.71 | 40.51 | 21.24 | 38.06 | no |
| **NileChat-3B QLoRA, context-free** | 3B, dec-only | **23.54** | **39.68** | — | — | [`alexandriax-nilechat-lora`](https://huggingface.co/NAMAA-Space/alexandriax-nilechat-lora) |
| NileChat-3B QLoRA, +context | 3B, dec-only | 22.87 | 39.11 | — | — | no |
| **NileChat-3B QLoRA, +context +back-translation** | 3B, dec-only | **22.77** | **38.71** | — | — | [`alexandriax-nilechat-ctx-aux`](https://huggingface.co/NAMAA-Space/alexandriax-nilechat-ctx-aux) |
| Gemma-3-1B LoRA | 1B, dec-only | 22.71 | 38.86 | 20.09 | 36.20 | no |
| **NLLB-200-1.3B QLoRA** | 1.3B, enc–dec | **21.83** | **38.13** | — | — | [`alexandriax-nllb-1.3b-lora`](https://huggingface.co/NAMAA-Space/alexandriax-nllb-1.3b-lora) |
| **AraT5v2, dialect-rebalanced** | 368M, enc–dec | void run¹ | | — | — | [`alexandriax-arat5v2-balanced`](https://huggingface.co/NAMAA-Space/alexandriax-arat5v2-balanced) |
| **mT5-large, dialect-rebalanced** | 1.23B, enc–dec | not evaluated² | | — | — | [`alexandriax-mt5-large-balanced`](https://huggingface.co/NAMAA-Space/alexandriax-mt5-large-balanced) |
| MBR over 3 NileChat variants | — | 23.57 | 39.86 | — | — | n/a |
| MBR over 5 samples, one model | — | 20.09 | 37.50 | — | — | n/a |
| Linear adapter merge | — | 19.90 | 35.33 | — | — | n/a |

¹ That run was trained against destroyed targets — a tokenizer fallback substituted `t5-base`
(32,100 English tokens) for AraT5v2's 110,208-token vocabulary, so every Arabic character
became `<unk>`. It scored 0.00 spBLEU and cannot be recovered without retraining; the
post-mortem and a fixed training script are in its card.

² That run stopped at step 2,500 of a planned 31,568 (epoch 0.63 of 8) and was never decoded on
the development set, so no score exists for it. Its card carries the full recovered configuration.

**Two findings from this bank of models are worth carrying elsewhere.**

1. **Parameter count does not predict rank below the cap.** The 368M encoder–decoder AraT5v2
   beats every larger decoder-only fine-tune on identical data, and among the decoder-only
   models spBLEU falls Qwen2.5-1.5B > NileChat-3B > Gemma-3-1B — the reverse of their size
   order. A reading consistent with this: the metric rewards fidelity to the annotators'
   conventions over generative fluency. A translator fine-tuned on the provided targets
   acquires those conventions; a decoder-only model several times its size contributes fluency
   *n*-gram overlap does not credit.
2. **Combination is not free.** Fitted and evaluated on disjoint halves of the dev
   conversations: routing per country **+0.07**, per country + sub-dialect **+0.28**,
   per country + domain **−0.32**, MBR consensus over 5 systems **−0.57**, MBR over the top-2
   per dialect **−0.81** — against a best single system of 24.98. The per-turn oracle reaches
   **32.25 (+7.27)**, so the right output is usually in the pool and the failure is in
   *selection*: three NileChat variants agree with one another and outvote the single strongest
   system, so consensus weights model-family size rather than quality. The submitted system
   therefore routes per dialect under a ±0.40 spBLEU margin guard instead of voting.

---

## The collection

All released artefacts live in
[**NAMAA at AlexandriaX-2026**](https://huggingface.co/collections/FatimahEmadEldin/namaa-at-alexandriax-2026):

| Repo | What it is |
|---|---|
| [`alexandriax-arat5v2-base`](https://huggingface.co/NAMAA-Space/alexandriax-arat5v2-base) | AraT5v2-base full fine-tune — **best small fine-tune**, 25.12 dev / 23.26 blind spBLEU |
| [`alexandriax-arat5v2-balanced`](https://huggingface.co/NAMAA-Space/alexandriax-arat5v2-balanced) | the same recipe on a temperature-rebalanced dialect mixture — **void run**, released for the post-mortem and the fixed script |
| [`alexandriax-nilechat-lora`](https://huggingface.co/NAMAA-Space/alexandriax-nilechat-lora) | NileChat-3B QLoRA, context-free — best of the three NileChat variants, 23.54 dev spBLEU |
| [`alexandriax-nilechat-ctx-aux`](https://huggingface.co/NAMAA-Space/alexandriax-nilechat-ctx-aux) | NileChat-3B QLoRA, context + back-translation — the augmentation ablation, 22.77 dev spBLEU |
| [`alexandriax-nllb-1.3b-lora`](https://huggingface.co/NAMAA-Space/alexandriax-nllb-1.3b-lora) | NLLB-200-1.3B QLoRA with per-dialect language codes, 21.83 dev spBLEU |
| [`alexandriax-mt5-large-balanced`](https://huggingface.co/NAMAA-Space/alexandriax-mt5-large-balanced) | mT5-large on the rebalanced mixture — **partial run** (2,500/31,568 steps), never evaluated |
| [`alexandria-backtranslated-pairs`](https://huggingface.co/datasets/NAMAA-Space/alexandria-backtranslated-pairs) | 348,787 synthetic EN→dialect pairs over 14 varieties |

Every model repo above carries a single-file `train_*.py` reproduction script with the exact
hyperparameters that produced its checkpoint; the dataset repo carries
`build_backtranslated_pairs.py`.

Official task data: [`UBC-NLP/alexandria`](https://huggingface.co/datasets/UBC-NLP/alexandria).
Base models: [`UBC-NLP/AraT5v2-base-1024`](https://huggingface.co/UBC-NLP/AraT5v2-base-1024),
[`UBC-NLP/NileChat-3B-Base`](https://huggingface.co/UBC-NLP/NileChat-3B-Base),
[`facebook/nllb-200-1.3B`](https://huggingface.co/facebook/nllb-200-1.3B),
[`google/mt5-large`](https://huggingface.co/google/mt5-large).

---

## Team

**NAMAA Community** — Fatimah Emad Eldin (Cairo University) · Omer Nacar (Tuwaiq Academy) ·
Khloud Al Jallad (Arab International University) · Mona Abdelazim (Ain Shams University).

## Citation

**Coming soon.** The NAMAA system-description paper for AlexandriaX-2026 is under review for
the ArabicNLP 2026 (EMNLP) proceedings; this card will be updated with the final ACL Anthology
reference and DOI when the proceedings are published. Until then, please cite as:

```bibtex
@inproceedings{namaa-alexandriax-2026,
  title     = {{NAMAA} Community at {AlexandriaX-2026}: Prompting, Fine-Tuning and Agreement
               Voting for Dialectal Arabic Translation and Evaluation},
  author    = {Emad Eldin, Fatimah and Nacar, Omer and Al Jallad, Khloud and Abdelazim, Mona},
  booktitle = {Proceedings of the Fourth Arabic Natural Language Processing Conference
               (ArabicNLP 2026)},
  year      = {2026},
  note      = {To appear. Citation coming soon.}
}
```

Please also cite the shared task and the base model:

```bibtex
@inproceedings{alexandriax2026,
  title     = {{AlexandriaX-2026} Shared Task: Context-Aware Dialectal Arabic Machine
               Translation and MT Evaluation},
  author    = {El Mekki, Abdellah and Elmadany, AbdelRahim A. and Magdy, Samar M. and
               Ezzini, Saad and El-Haj, Mo and Jarrar, Mustafa and El-Beltagy, Samhaa and
               Abbas, Mourad and Zaraket, Fadi and Al Mandhari, Salim and Alyafeai, Zaid and
               Ghanem, Bernard and Abdul-Mageed, Muhammad},
  booktitle = {Proceedings of the Fourth Arabic Natural Language Processing Conference
               (ArabicNLP 2026)},
  year      = {2026},
  note      = {Overview paper. Citation coming soon.}
}
```

## Acknowledgements

Thanks to the AlexandriaX-2026 organisers for the data, the evaluation infrastructure and
their responsiveness during the evaluation phases.