README.md

---
license: cc-by-4.0
language:
- ar
tags:
- automatic-speech-recognition
- arabic
- diacritization
- tashkeel
- speech
pipeline_tag: automatic-speech-recognition
---

# Nemo-Arabic-STT-Diacritized

An Arabic speech-to-text pipeline that produces fully diacritized (tashkeel) transcripts. It
combines two independently developed models in sequence; it is a packaged inference pipeline,
not a single jointly trained architecture.

## Pipeline

1. **Speech recognition**: [NVIDIA NeMo FastConformer Hybrid](https://huggingface.co/nvidia/stt_ar_fastconformer_hybrid_large_pcd_v1.0)
   (`stt_ar_fastconformer_hybrid_large_pcd_v1.0`) transcribes Arabic speech to plain,
   undiacritized text.
2. **Diacritization**: [CATT](https://github.com/abjadai/catt) (Character-based Arabic Tashkeel
   Transformer) adds tashkeel to the transcript.

Audio in, diacritized Arabic text out. Neither checkpoint was retrained or fine-tuned for this
repository.

## Files

| File | Description | Source |
|---|---|---|
| `stt_ar_fastconformer_hybrid_large_pcd_v1.0.nemo` | ASR checkpoint, unmodified | NVIDIA, CC-BY-4.0 |
| `best_ed_mlm_ns_epoch_178.pt` | Diacritizer checkpoint, unmodified | abjadai/CATT, Apache-2.0 |
| `diacritize.py`, `catt/` | Diacritizer inference code (vendored from CATT) | abjadai/CATT, Apache-2.0 |
| `pipeline.py` | `DiacritizedASR` — loads both models once, audio in / diacritized text out in a single call | This repository |
| `server.py` | Reference FastAPI server implementing the full pipeline | This repository |

## Usage

```python
from pipeline import DiacritizedASR

model = DiacritizedASR(
    nemo_path="stt_ar_fastconformer_hybrid_large_pcd_v1.0.nemo",
    catt_ckpt="best_ed_mlm_ns_epoch_178.pt",
)
diacritized = model.transcribe("audio.wav")
```

Both models load once at construction; each `.transcribe()` call runs ASR followed immediately
by diacritization in the same process. For the two steps individually:

```python
import nemo.collections.asr as nemo_asr
from diacritize import Diacritizer

asr_model = nemo_asr.models.EncDecHybridRNNTCTCBPEModel.restore_from(
    "stt_ar_fastconformer_hybrid_large_pcd_v1.0.nemo"
)
diacritizer = Diacritizer(ckpt="best_ed_mlm_ns_epoch_178.pt")

text = asr_model.transcribe(["audio.wav"])[0].text
diacritized = diacritizer.diacritize_texts([text])[0]
```

Or run `server.py` directly for an HTTP API (`POST /transcribe`, `GET /health`).

## Example

Input audio (Arabic speech) transcribed and diacritized:

```
plain:       السلام عليكم ورحمة الله وبركاته كيف يمكنني مساعدتك اليوم؟
diacritized: السَّلَامُ عَلَيْكُمْ وَرَحْمَةُ اللَّهِ وَبَرَكَاتُهُ كَيْفَ يُمْكِنُنِي مُسَاعَدَتُكَ الْيَوْمَ؟
```

## Limitations

- Diacritization quality depends on ASR transcript quality; transcription errors propagate to
  diacritization.
- CATT diacritizes using full-sentence context; very short or ambiguous transcripts may
  diacritize imperfectly.
- Developed and tested on general Modern Standard Arabic conversational speech, not evaluated on
  Quranic recitation.

## Attribution and License

- ASR model: NVIDIA, [stt_ar_fastconformer_hybrid_large_pcd_v1.0](https://huggingface.co/nvidia/stt_ar_fastconformer_hybrid_large_pcd_v1.0), CC-BY-4.0.
- Diacritizer: [abjadai/CATT](https://github.com/abjadai/catt), Apache-2.0.
- This repository (packaging and inference code) is released under CC-BY-4.0, consistent with
  the ASR model's license and compatible with CATT's Apache-2.0 terms.

Built for the [Muslim](https://huggingface.co/NightPrince) Arabic voice AI companion project.