Instructions to use NAMAA-Space/alexandriax-nilechat-ctx-aux with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use NAMAA-Space/alexandriax-nilechat-ctx-aux with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("UBC-NLP/NileChat-3B-Base") model = PeftModel.from_pretrained(base_model, "NAMAA-Space/alexandriax-nilechat-ctx-aux") - Notebooks
- Google Colab
- Kaggle
license: cc-by-nc-4.0
base_model: UBC-NLP/NileChat-3B-Base
base_model_relation: adapter
library_name: peft
pipeline_tag: text-generation
language:
- ar
- en
tags:
- translation
- arabic
- dialectal-arabic
- nilechat
- lora
- qlora
- peft
- back-translation
- data-augmentation
- alexandriax-2026
- arabicnlp2026
- shared-task
datasets:
- UBC-NLP/alexandria
- NAMAA-Space/alexandria-backtranslated-pairs
metrics:
- bleu
- chrf
model-index:
- name: alexandriax-nilechat-ctx-aux
results:
- task:
type: translation
name: Context-Aware English-to-Dialectal Arabic Dialogue Translation
dataset:
name: AlexandriaX-2026 Subtask 1 (development)
type: UBC-NLP/alexandria
split: dev
metrics:
- type: spbleu
value: 22.77
name: spBLEU (FLORES-200 tokenizer, country-macro)
- type: chrf++
value: 38.71
name: chrF++ (word_order=2, country-macro)
AlexandriaX-2026 · Subtask 1 — NileChat-3B QLoRA, context + back-translation (nilechat_ctx_aux)
English to dialectal Arabic dialogue translation over 13 Arabic varieties, from the
NAMAA Community submission to AlexandriaX-2026 (ArabicNLP 2026 / EMNLP). A QLoRA
adapter over UBC-NLP/NileChat-3B-Base,
trained on the official turns with conversation context and augmented with
348,787 synthetic back-translated pairs built from a 358k-sentence dialectal corpus.
This is the data-augmentation arm of the team's NileChat ablation. Its result is negative and worth publishing as such: augmenting with 5.5x more synthetic than real data did not beat the plain context-free adapter on the official development set (22.77 vs 23.54 spBLEU). No submitted system uses back-translated data because of this measurement.
| Task | AlexandriaX-2026 Subtask 1 (context-aware EN→DA dialogue translation) |
| Base model | UBC-NLP/NileChat-3B-Base (decoder-only, approx. 3B parameters, Arabic-dialect pretraining) |
| Adapter | LoRA r=16, α=32, dropout 0.05 — 119.8 MB, approx. 30M trainable parameters |
| Dialect control | prompt field (Country, dialect name) |
| Context | yes — the model's own previous turns are fed back as history |
| Extra data | NAMAA-Space/alexandria-backtranslated-pairs (348,787 synthetic pairs) |
| Dev (12,250 turns, 11 countries) | 22.77 spBLEU · 38.71 chrF++ |
| Internal hold-out (3,350 turns, 5% of train) | 24.36 spBLEU · 40.16 chrF++ |
| Blind test | not run (this system was not part of a submitted bundle) |
| Track | constrained (provided data only, ≤5B parameters) |
| License | CC-BY-NC-4.0 — check UBC-NLP/NileChat-3B-Base for the base model's terms |
Usage
The adapter was trained on one exact prompt format, with the conversation history filled in. Reproduce it character-for-character, and feed back the model's own previous translations turn by turn — that is how it was trained and how it was scored.
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel
BASE = "UBC-NLP/NileChat-3B-Base"
ADAPTER = "NAMAA-Space/alexandriax-nilechat-ctx-aux"
bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
bnb_4bit_use_double_quant=True,
bnb_4bit_compute_dtype=torch.bfloat16)
tok = AutoTokenizer.from_pretrained(BASE, trust_remote_code=True)
tok.padding_side = "left"
if tok.pad_token is None:
tok.pad_token = tok.eos_token
base = AutoModelForCausalLM.from_pretrained(BASE, quantization_config=bnb,
device_map="auto", trust_remote_code=True)
model = PeftModel.from_pretrained(base, ADAPTER).eval()
def format_history(history):
"""history: list of {"speaker", "sentence", "translation"} for previous turns."""
if not history:
return "No previous turns (Start of conversation)."
return "\n".join(f"{h.get('speaker') or 'Speaker'}: {h['sentence']}\n"
f"Translation: {h['translation']}" for h in history)
def build_prompt(dialect, country, domain, participants, speaker, direction, sentence,
history=None):
return (f"You are an expert translator. Translate the English sentence into {dialect}.\n\n"
f"### Metadata:\n- Country: {country}\n- Domain: {domain}\n"
f"- Participants: {participants}\n- Speaker: {speaker}\n"
f"- Speaker Direction: {direction}\n\n"
f"### Conversation History:\n{format_history(history)}\n\n"
f"### Sentence to Translate:\n{sentence.strip()}\n\n### Translation:\n")
def clean_generation(text):
text = text.strip()
for marker in ("\n###", "### Sentence to Translate:", "### Translation:"):
if marker in text:
text = text.split(marker, 1)[0].strip()
return text
def translate_conversation(turns, **meta):
"""turns: list of {"speaker", "direction", "sentence"} in turn order."""
history, out = [], []
for t in turns:
prompt = build_prompt(sentence=t["sentence"], speaker=t["speaker"],
direction=t["direction"], history=history, **meta)
enc = tok([prompt], return_tensors="pt", padding=True, truncation=True,
max_length=1024).to(model.device)
with torch.no_grad():
gen = model.generate(**enc, max_new_tokens=128, do_sample=False)
pred = clean_generation(
tok.decode(gen[0][enc["input_ids"].shape[1]:], skip_special_tokens=True))
out.append(pred)
history.append({"speaker": t["speaker"], "sentence": t["sentence"],
"translation": pred}) # the model's own output, not gold
return out
print(translate_conversation(
[{"speaker": "Wholesale Buyer", "direction": "male -> female",
"sentence": "Good morning. How much for the whole quantity?"},
{"speaker": "Farmer", "direction": "female -> male",
"sentence": "For you, a special price. Eight pounds a kilo."}],
dialect="Egyptian Arabic (Cairene) Dialect", country="EG",
domain="Agriculture and farming",
participants="Wholesale Buyer, Wholesale Seller"))
Reference decoding for every number in this card: greedy (do_sample=False),
max_new_tokens=128, prompt truncated at 1024 tokens, left-padded batches.
Never place gold previous-turn Arabic in the history block when measuring. An earlier harness in this project did, and it inflated held-out scores by roughly +2.4 spBLEU. Every number in this card comes from the corrected, self-conditioned harness.
Intended use
Research on data augmentation for low-resource dialectal MT — specifically as the evidence that NLLB-generated back-translation into Arabic dialects does not transfer to this task's reference style. Not for production translation without human review.
The shared task
AlexandriaX-2026 (ArabicNLP 2026 / EMNLP) — Context-Aware Dialectal Arabic MT and MT Evaluation. This model was built for Subtask 1: Context-Aware English-to-Dialectal Arabic Dialogue Translation.
Given one English dialogue turn together with its conversation history and metadata — target country/dialect, domain, participant roles, speaker, and speaker→addressee gender direction — the system must produce the turn in the requested country's spoken Arabic, preserving meaning while adapting lexical, morphological, pragmatic and sociolinguistic choices to that variety.
Two tracks: constrained (provided data only, ≤5B parameters) and unconstrained (any external data or model). Ranking is by spBLEU (primary) and chrF++ (secondary), each macro-averaged over countries.
Official data (UBC-NLP/alexandria)
Split sizes in turns, as published by the organisers:
| Split | EG | JO | LB | LY | MA | MR | OM | PS | SA | SD | SY | TN | YE | Total |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| train | 3,108 | 5,501 | 8,906 | 0 | 2,573 | 5,515 | 6,280 | 14,933 | 8,470 | 0 | 6,071 | 2,034 | 3,089 | 66,480 |
| dev | 1,113 | 1,113 | 1,118 | 0 | 1,110 | 1,114 | 1,109 | 1,110 | 1,110 | 0 | 1,119 | 1,116 | 1,118 | 12,250 |
| public test | 1,118 | 1,107 | 1,106 | 1,109 | 1,115 | 1,112 | 1,118 | 1,109 | 1,113 | 1,106 | 1,114 | 1,109 | 1,106 | 14,442 |
| private (blind) test | 1,113 | 1,109 | 1,110 | 1,309 | 1,111 | 1,119 | 1,107 | 1,111 | 1,114 | 915 | 1,114 | 1,114 | 1,113 | 14,459 |
Libyan (LY) and Sudanese (SD) appear only at test time — they are zero-shot for every system trained on this data.
Conversation-level counts: 21,146 train / 3,963 dev / 4,706 public-test conversations; mean 3.13 turns per conversation (range 1–5). Mean length 102 characters of English source, 74 characters of dialectal target.
Dialects (13 countries). Egyptian, Jordanian, Lebanese, Libyan, Moroccan, Mauritanian, Omani, Palestinian, Saudi, Sudanese, Syrian, Tunisian, Yemeni. Labels are country + sub-dialect, and several countries carry more than one: Palestinian 10 (Nabulsi and Albira urban, plus Falahi varieties of Surif, Kobar, Noba, Ni'lin, Shuqba, Aboud, Silwad, Ramallah), Omani 5 (Suri, Rustaqi, Al-Wafi, Ibri, Seebi), Saudi 3 (Southern, Hijazi, Khaleeji), Yemeni 3 (Taiz, San'ani, Central), Syrian 2 (Levantine Standard, Homsi). The remaining countries carry one label each (e.g. Egyptian Arabic (Cairene), Moroccan Standard Darija, Mauritanian Hassaniya, Libyan Arabic (Misrati/Central)).
Domains (11, near-uniform). Agriculture and farming, Commerce and transactions, Construction and real estate, Education and academia, Energy and resources, Everyday and social, Healthcare and medical, Legal and financial, Logistics and transportation, Professional and workplace, Tourism and hospitality.
Speaker direction (turns, train+dev+public test): female→male 30,636 · male→female 30,203 · male→male 20,465 · female→female 11,868. The corpus carries 76 distinct translator IDs and 44 reviewer IDs.
Code-switching in the gold is strongly dialect-specific — the share of gold turns containing Latin characters runs from TN 39.1% / MA 33.8% / LB 18.1% down to SY 1.2% / YE 0.8%. Systems that normalise every borrowing into Arabic script are penalised hardest on Maghrebi references (see Known limitations).
Evaluation protocol
- spBLEU —
sacrebleu.BLEU(tokenize="flores200"), corpus-level per country, then averaged over countries. - chrF++ —
sacrebleu.CHRF(word_order=2), same averaging. - Decoding is turn-by-turn: at turn n the conversation history contains the system's own previous outputs, never the gold ones. (An early evaluation harness in this project leaked gold previous-turn Arabic into the prompt and inflated scores by ≈2.4 spBLEU; every number reported here comes from the corrected, self-conditioned harness.)
Training data
Two sources, concatenated into one supervised mixture of 411,917 examples:
| Source | Examples | Share | What it is |
|---|---|---|---|
| Official Subtask-1 train (context prompts) | 63,130 | 15.3% | human-translated dialogue turns, 11 countries, history filled with the previous turns |
alexandria-backtranslated-pairs |
348,787 | 84.7% | synthetic single-turn pairs, 14 varieties, history always empty |
Real data, per country (turns):
| PS | LB | SA | OM | SY | MR | JO | YE | EG | MA | TN |
|---|---|---|---|---|---|---|---|---|---|---|
| 14,183 | 8,464 | 8,035 | 5,965 | 5,760 | 5,234 | 5,224 | 2,946 | 2,943 | 2,443 | 1,933 |
Synthetic data, per country (pairs):
| SA | SY | MA | PS | DZ | EG | LB | JO | TN | OM | LY | SD | YE | MR |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 50,000 | 50,000 | 46,286 | 30,538 | 30,078 | 28,197 | 23,467 | 21,185 | 15,204 | 13,812 | 13,587 | 12,251 | 12,031 | 2,151 |
The synthetic side is the one place where LY, SD and Algerian (DZ, not a task variety) get any training signal at all — 13,587 and 12,251 pairs for the two zero-shot test varieties.
How the synthetic pairs were made. A monolingual corpus of 358,483 cleaned dialectal
sentences was assembled from public sources — IADD, QADI, NADI-2021, MADAR, and Egyptian,
Saudi, Moroccan, Levantine, Libyan, Sudanese and Yemeni web and transcript collections
(transcript columns only; no audio was ingested). URLs, mentions, hashtags, timestamps and
emoji were stripped, all Latin letters deleted, punctuation blanked, character runs
collapsed, length clipped to 3–1000 characters and near-duplicates removed. Each surviving
sentence was back-translated to English with facebook/nllb-200-distilled-600M (fp16,
greedy, 128 new tokens, batch 256, capped at 50,000 sentences per variety) and the pair
inverted into the training prompt, yielding 348,787 English→dialect pairs. Full parameters and
the build script are in the
dataset card.
Two filter choices deserve flagging in hindsight. Latin script is deleted, so no synthetic target contains a single Latin token — while 33.8% of Moroccan and 39.1% of Tunisian gold turns do. And punctuation is stripped from the targets but not from the English prompts. Both push the model away from the reference style of exactly the varieties the augmentation was meant to help.
Training procedure — every hyperparameter
Extracted verbatim from AlexandriaX_NB2_Finetune.ipynb. Runnable single-file version, which
reproduces all three variants behind one --variant flag:
train_nilechat_qlora.py.
python train_nilechat_qlora.py --variant ctx_aux --out ./runs/nilechat_ctx_aux
Every setting below is identical to the context-free sibling except the training mixture and therefore the step count — that is what makes this an ablation rather than a different system.
Model, adapter and data
| Setting | Value | Note |
|---|---|---|
| Base model | UBC-NLP/NileChat-3B-Base |
decoder-only; the organisers' baseline model |
| Regime | QLoRA | 4-bit frozen base + LoRA adapters, SFT via TRL's SFTTrainer |
LoRA rank r |
16 | |
LoRA alpha |
32 | scaling 32/16 = 2 |
LoRA dropout |
0.05 | |
LoRA bias |
"none" |
|
task_type |
CAUSAL_LM |
|
target_modules |
"all-linear" |
resolved to q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Trainable parameters | ≈30M (~1% of the base) | 119.8 MB adapter |
| Loss | completion-only (completion_only_loss=True) |
the prompt is masked out of the loss |
| Data schema | {"prompt": ..., "completion": response + eos_token} |
TRL prompt/completion format |
max_length |
1024 tokens | matters more here: context prompts are longer |
packing |
False | |
| Training examples | 411,917 | 63,130 real (with history) + 348,787 back-translated |
| Mixture handling | plain concatenation, then shuffle | no curriculum, upsampling or loss reweighting |
padding_side |
right for training, left for generation |
Quantisation
| Setting | Value |
|---|---|
load_in_4bit |
True |
bnb_4bit_quant_type |
nf4 |
bnb_4bit_use_double_quant |
True |
bnb_4bit_compute_dtype |
bfloat16 (fp16 fallback) |
device_map |
"auto" |
prepare_model_for_kbit_training |
called before attaching LoRA |
Optimisation
| Setting | Value | Note |
|---|---|---|
| Optimiser | paged_adamw_8bit |
|
| Learning rate | 2e-4 | |
| LR scheduler | cosine |
over 6.4k steps here, 987 for the context-free sibling |
| Warmup | warmup_ratio=0.03 |
ratio, so ≈193 steps here against ≈30 there |
| Epochs | 1 | over the full mixture |
per_device_train_batch_size |
32 | |
gradient_accumulation_steps |
2 | |
| Effective batch | 64 | |
| Optimiser steps | ≈6,436 | 411,917 / 64 — 6.5× the sibling's |
| Weight decay / clipping | 0.0 / 1.0 | framework defaults |
The step count is a confound worth naming: this variant takes 6.5× as many optimiser steps as the context-free sibling it is compared against, because "one epoch" is defined over a 6.5× larger mixture. The comparison is therefore equal-epoch, not equal-step.
Precision, memory, hardware
| Setting | Value |
|---|---|
| Hardware | 1 × A100-80GB |
bf16 / fp16 |
bf16 True (fp16 as fallback) |
| Attention | FlashAttention-2 when the load succeeds, else the default implementation |
gradient_checkpointing |
True |
use_cache |
False during training, True for generation |
Bookkeeping
| Setting | Value |
|---|---|
logging_steps / save_steps / save_total_limit |
20 / 200 / 2 |
| Resume | automatic from the highest checkpoint-* |
| Libraries | transformers, trl, peft 0.19.1, bitsandbytes |
Inference (used for every score in this card)
| Setting | Value |
|---|---|
| Decoding | greedy — do_sample=False, no beams |
max_new_tokens |
128 |
| Prompt truncation | 1024 tokens |
| Generation batch | up to 512, left-padded |
| Post-processing | split off anything after \n###, ### Sentence to Translate: or ### Translation: |
| Context | the model's own previous outputs, appended turn by turn |
Training uses gold history, evaluation uses generated history. At training time the history block holds the gold previous turns (ordinary teacher forcing); at evaluation time it must hold the model's own previous outputs. Conflating the two inflated an early measurement in this project by roughly +2.4 spBLEU.
Results
The ablation this checkpoint belongs to
Three variants, identical hyperparameters, differing only in prompt history and training mixture. Both evaluation sets are decoded turn-by-turn with the model's own previous outputs as history.
| Variant | History in prompt | Extra data | dev spBLEU | dev chrF++ | hold-out spBLEU | hold-out chrF++ |
|---|---|---|---|---|---|---|
nilechat_noctx |
none | — | 23.54 | 39.68 | 24.84 | 40.36 |
nilechat_ctx |
own previous turns | — | 22.87 | 39.11 | 23.37 | 39.35 |
nilechat_ctx_aux (this model) |
own previous turns | 348,787 back-translated pairs | 22.77 | 38.71 | 24.36 | 40.16 |
dev = official 12,250-turn development set, 11 countries. hold-out = the team's internal 3,350-turn split of the training conversations.
The two sets disagree, and this is the crux of the card. On the internal hold-out, back-translation looks like a clear win over the context variant (+0.99 spBLEU, 24.36 vs 23.37) and nearly catches the context-free model. On the official development set the same comparison reverses — −0.10 spBLEU and −0.40 chrF++ against the context variant — and the gap to the context-free model widens to −0.77 spBLEU. The internal split shares conversations, speakers and phrasing with the training data; the official dev set does not. Trust the official dev column — and, more generally, do not size an augmentation decision on a hold-out carved from the training conversations.
Per-country, official dev set (12,250 turns)
| EG | JO | LB | MA | MR | OM | PS | SA | SY | TN | YE | macro | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| spBLEU | 26.90 | 29.03 | 25.21 | 15.90 | 9.31 | 22.29 | 25.32 | 25.99 | 29.54 | 22.67 | 18.32 | 22.77 |
| chrF++ | 41.51 | 44.26 | 40.75 | 31.67 | 26.47 | 39.06 | 41.12 | 42.05 | 45.88 | 37.90 | 35.12 | 38.71 |
Per-country, internal hold-out (3,350 turns)
| EG | JO | LB | MA | MR | OM | PS | SA | SY | TN | YE | macro | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| spBLEU | 24.75 | 29.99 | 25.58 | 19.15 | 14.08 | 22.68 | 25.84 | 30.56 | 31.17 | 26.27 | 17.89 | 24.36 |
| chrF++ | 39.75 | 45.52 | 41.03 | 35.30 | 30.72 | 39.82 | 41.78 | 46.05 | 47.28 | 39.70 | 34.76 | 40.16 |
Where the augmentation actually moved things
Against the context-only variant on official dev, adding 348k synthetic pairs helped Egyptian (+0.34), Tunisian (+1.42) and Omani (+0.87) and hurt Moroccan (−0.82), Syrian (−0.93), Palestinian (−0.83) and Saudi (−0.84). The two countries that gained most are among the scarcest in real data; the ones that lost most are among the best resourced, where synthetic noise displaces good supervision. The net is roughly flat, which is the honest summary: NLLB back-translation into Arabic dialects buys little here and costs where real data is already plentiful.
Known limitations
- Negative result. Do not pick this checkpoint expecting the augmentation to help; pick the context-free sibling for quality, and this one to study or extend the augmentation.
- Synthetic-heavy mixture. 84.7% of the training examples are machine-generated, all single-turn with empty history, which dilutes the context signal the variant was meant to exploit.
- Latin-script deletion fights the references on exactly the code-switching varieties (MA, TN, LB) that most needed help, and punctuation stripping costs character-level credit everywhere — see the note above.
- Back-translation direction is untested. The English side comes from NLLB and is not human-verified; some of it is visibly disfluent, and the corpus includes informal web text whose register differs from the task's role-play dialogues.
- DZ (Algerian) is in the synthetic data but not in the task, so approximately 8.6% of the mixture targets a variety that is never evaluated.
- Prompt-format sensitivity: trained on exactly one template with completion-only loss.
- LY and SD get synthetic-only supervision; their blind-test behaviour was never measured for this checkpoint.
- Not evaluated on the blind test; metric-only evaluation (no human, no COMET).
- Base-model licence and usage terms apply; check
UBC-NLP/NileChat-3B-Base.
Where this model sits in the NAMAA system
All Subtask-1 systems built by the team, scored on the official 12,250-turn dev set (11 countries) and, where they were run, on the 14,459-turn private blind test (13 countries). Country-macro spBLEU / chrF++.
| System | Params / arch. | dev spBLEU | dev chrF++ | blind spBLEU | blind chrF++ | Released |
|---|---|---|---|---|---|---|
| Gemma, beam search (submitted, constrained) | ~3.1B, dec-only | — | — | 27.413 | 42.58 | no |
| Routed ensemble (submitted, unconstrained) | — | — | — | 27.412 | 43.05 | n/a |
| Gemini 2.5 Flash, 5-shot | API | — | — | 26.68 | 42.49 | n/a |
| Claude Sonnet 4.5, 5-shot | API | — | — | 26.36 | 42.26 | n/a |
| AraT5v2 full fine-tune | 368M, enc–dec | 25.12 | 40.66 | 23.26 | 39.03 | alexandriax-arat5v2-base |
| Qwen2.5-1.5B LoRA | 1.5B, dec-only | 23.71 | 40.51 | 21.24 | 38.06 | no |
| NileChat-3B QLoRA, context-free | 3B, dec-only | 23.54 | 39.68 | — | — | alexandriax-nilechat-lora |
| NileChat-3B QLoRA, +context | 3B, dec-only | 22.87 | 39.11 | — | — | no |
| NileChat-3B QLoRA, +context +back-translation | 3B, dec-only | 22.77 | 38.71 | — | — | alexandriax-nilechat-ctx-aux |
| Gemma-3-1B LoRA | 1B, dec-only | 22.71 | 38.86 | 20.09 | 36.20 | no |
| NLLB-200-1.3B QLoRA | 1.3B, enc–dec | 21.83 | 38.13 | — | — | alexandriax-nllb-1.3b-lora |
| AraT5v2, dialect-rebalanced | 368M, enc–dec | void run¹ | — | — | alexandriax-arat5v2-balanced |
|
| mT5-large, dialect-rebalanced | 1.23B, enc–dec | not evaluated² | — | — | alexandriax-mt5-large-balanced |
|
| MBR over 3 NileChat variants | — | 23.57 | 39.86 | — | — | n/a |
| MBR over 5 samples, one model | — | 20.09 | 37.50 | — | — | n/a |
| Linear adapter merge | — | 19.90 | 35.33 | — | — | n/a |
¹ That run was trained against destroyed targets — a tokenizer fallback substituted t5-base
(32,100 English tokens) for AraT5v2's 110,208-token vocabulary, so every Arabic character
became <unk>. It scored 0.00 spBLEU and cannot be recovered without retraining; the
post-mortem and a fixed training script are in its card.
² That run stopped at step 2,500 of a planned 31,568 (epoch 0.63 of 8) and was never decoded on the development set, so no score exists for it. Its card carries the full recovered configuration.
Two findings from this bank of models are worth carrying elsewhere.
- Parameter count does not predict rank below the cap. The 368M encoder–decoder AraT5v2 beats every larger decoder-only fine-tune on identical data, and among the decoder-only models spBLEU falls Qwen2.5-1.5B > NileChat-3B > Gemma-3-1B — the reverse of their size order. A reading consistent with this: the metric rewards fidelity to the annotators' conventions over generative fluency. A translator fine-tuned on the provided targets acquires those conventions; a decoder-only model several times its size contributes fluency n-gram overlap does not credit.
- Combination is not free. Fitted and evaluated on disjoint halves of the dev conversations: routing per country +0.07, per country + sub-dialect +0.28, per country + domain −0.32, MBR consensus over 5 systems −0.57, MBR over the top-2 per dialect −0.81 — against a best single system of 24.98. The per-turn oracle reaches 32.25 (+7.27), so the right output is usually in the pool and the failure is in selection: three NileChat variants agree with one another and outvote the single strongest system, so consensus weights model-family size rather than quality. The submitted system therefore routes per dialect under a ±0.40 spBLEU margin guard instead of voting.
The collection
All released artefacts live in NAMAA at AlexandriaX-2026:
| Repo | What it is |
|---|---|
alexandriax-arat5v2-base |
AraT5v2-base full fine-tune — best small fine-tune, 25.12 dev / 23.26 blind spBLEU |
alexandriax-arat5v2-balanced |
the same recipe on a temperature-rebalanced dialect mixture — void run, released for the post-mortem and the fixed script |
alexandriax-nilechat-lora |
NileChat-3B QLoRA, context-free — best of the three NileChat variants, 23.54 dev spBLEU |
alexandriax-nilechat-ctx-aux |
NileChat-3B QLoRA, context + back-translation — the augmentation ablation, 22.77 dev spBLEU |
alexandriax-nllb-1.3b-lora |
NLLB-200-1.3B QLoRA with per-dialect language codes, 21.83 dev spBLEU |
alexandriax-mt5-large-balanced |
mT5-large on the rebalanced mixture — partial run (2,500/31,568 steps), never evaluated |
alexandria-backtranslated-pairs |
348,787 synthetic EN→dialect pairs over 14 varieties |
Every model repo above carries a single-file train_*.py reproduction script with the exact
hyperparameters that produced its checkpoint; the dataset repo carries
build_backtranslated_pairs.py.
Official task data: UBC-NLP/alexandria.
Base models: UBC-NLP/AraT5v2-base-1024,
UBC-NLP/NileChat-3B-Base,
facebook/nllb-200-1.3B,
google/mt5-large.
Team
NAMAA Community — Fatimah Emad Eldin (Cairo University) · Omer Nacar (Tuwaiq Academy) · Khloud Al Jallad (Arab International University) · Mona Abdelazim (Ain Shams University).
Citation
Coming soon. The NAMAA system-description paper for AlexandriaX-2026 is under review for the ArabicNLP 2026 (EMNLP) proceedings; this card will be updated with the final ACL Anthology reference and DOI when the proceedings are published. Until then, please cite as:
@inproceedings{namaa-alexandriax-2026,
title = {{NAMAA} Community at {AlexandriaX-2026}: Prompting, Fine-Tuning and Agreement
Voting for Dialectal Arabic Translation and Evaluation},
author = {Emad Eldin, Fatimah and Nacar, Omer and Al Jallad, Khloud and Abdelazim, Mona},
booktitle = {Proceedings of the Fourth Arabic Natural Language Processing Conference
(ArabicNLP 2026)},
year = {2026},
note = {To appear. Citation coming soon.}
}
Please also cite the shared task and the base model:
@inproceedings{alexandriax2026,
title = {{AlexandriaX-2026} Shared Task: Context-Aware Dialectal Arabic Machine
Translation and MT Evaluation},
author = {El Mekki, Abdellah and Elmadany, AbdelRahim A. and Magdy, Samar M. and
Ezzini, Saad and El-Haj, Mo and Jarrar, Mustafa and El-Beltagy, Samhaa and
Abbas, Mourad and Zaraket, Fadi and Al Mandhari, Salim and Alyafeai, Zaid and
Ghanem, Bernard and Abdul-Mageed, Muhammad},
booktitle = {Proceedings of the Fourth Arabic Natural Language Processing Conference
(ArabicNLP 2026)},
year = {2026},
note = {Overview paper. Citation coming soon.}
}
Acknowledgements
Thanks to the AlexandriaX-2026 organisers for the data, the evaluation infrastructure and their responsiveness during the evaluation phases.