---
language:
- ar
license: apache-2.0
base_model: SparkAudio/Spark-TTS-0.5B
tags:
- text-to-speech
- arabic
- spark-tts
- tts
- voice-cloning
- classical-arabic
datasets:
- MBZUAI/ClArTTS
library_name: transformers
pipeline_tag: text-to-speech
---

# Spark-TTS Arabic (Fine-tuned on ClArTTS)

Fine-tuned version of [SparkAudio/Spark-TTS-0.5B](https://huggingface.co/SparkAudio/Spark-TTS-0.5B) specialized for Arabic text-to-speech synthesis. The LLM component has been fine-tuned on the ClArTTS dataset (Classical Arabic Text-to-Speech corpus) containing 12 hours of high-quality single-speaker recordings.

## 📋 Model Description

Spark-TTS is a neural text-to-speech system that combines a language model (Qwen2) with a neural audio codec (BiCodec) for high-quality speech synthesis. This version has been specifically optimized for Arabic through fine-tuning on Classical Arabic speech data.

**Architecture Components:**
- **LLM (Qwen2):** Fine-tuned for Arabic text-to-semantic token generation
- **BiCodec:** Neural audio codec for semantic-to-audio token conversion (unchanged)
- **wav2vec2-large-xlsr-53:** Speech encoder for voice cloning (unchanged)

**What Changed:**
Only the LLM component was fine-tuned. The audio tokenizer and speech encoder remain identical to the base model, ensuring compatibility with the original Spark-TTS architecture.

**Key Features:**
- Voice cloning with 5-30 seconds of reference audio
- Natural prosody and intonation for Classical Arabic
- Single-speaker consistency
- Controllable generation parameters

## 🎯 Intended Use

### Direct Use
- Arabic audiobook narration (Classical/MSA)
- Voice-over for Arabic educational content
- Accessibility tools for Arabic text
- Voice cloning for Arabic speakers
- Arabic language learning applications

### Downstream Use
Can be further fine-tuned for:
- Dialectal Arabic variants (Egyptian, Levantine, Gulf)
- Domain-specific terminology (religious texts, literature)
- Multi-speaker scenarios
- Emotional or expressive speech

### Out-of-Scope Use
**Not recommended for:**
- Real-time speech synthesis (model is relatively slow)
- Non-diacritized Arabic text (requires tashkeel)
- Languages other than Arabic
- Singing or non-speech audio generation

**Very Important Note:** This model requires the official Spark-TTS repository for inference. The model files alone are not sufficient - you must clone the Spark-TTS repo and use their inference pipeline.

## 🚀 How to Use

### Installation

First, clone the official Spark-TTS repository (required for inference):
```bash
# Clone Spark-TTS
git clone https://github.com/SparkAudio/Spark-TTS
cd Spark-TTS

# Install dependencies
pip install transformers soundfile huggingface_hub omegaconf torch
```

### Download Model
```python
from huggingface_hub import snapshot_download

# Download the fine-tuned model
model_dir = snapshot_download(
    repo_id="azeddinShr/Spark-TTS-Arabic-Complete",
    local_dir="./arabic_model"
)
```

### Setup Inference Environment
```python
import sys
import torch
import soundfile as sf

# Add Spark-TTS to path
sys.path.insert(0, './cli')

# Import SparkTTS class
from SparkTTS import SparkTTS

# Initialize device
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print(f"Using device: {device}")
```

### Load Model
```python
# Load the fine-tuned Arabic model
tts = SparkTTS("./arabic_model", device)
print("✅ Model loaded successfully!")
```

### Basic Text-to-Speech
```python
# Prepare input text (must include diacritics)
text = "مَرْحَبًا بِكُمْ فِي نَمُوذَجِ تَحْوِيلِ النَّصِّ إِلَى كَلَامٍ بِاللُّغَةِ الْعَرَبِيَّةِ."

# Reference audio and its transcript
reference_audio = "path/to/reference.wav"  # 5-30 seconds of clear Arabic speech
reference_text = "النَّصُّ الْمُطَابِقُ لِلصَّوْتِ الْمَرْجِعِيِّ"

# Generate speech
wav = tts.inference(
    text,
    prompt_speech_path=reference_audio,
    prompt_text=reference_text
)

# Save output
sf.write("output.wav", wav, samplerate=16000)
print("✅ Audio generated!")
```

### Advanced Generation with Parameters
```python
# Generate with custom parameters
wav = tts.inference(
    text,
    prompt_speech_path=reference_audio,
    prompt_text=reference_text,
    temperature=0.8,      # Controls randomness (0.1-1.5, default: 0.8)
    top_k=50,            # Top-k sampling (default: 50)
    top_p=0.95           # Nucleus sampling (default: 0.95)
)

sf.write("output_custom.wav", wav, samplerate=16000)
```

## ⚠️ Important Requirements

### Input Text Requirements
- **Diacritization (Tashkeel) is REQUIRED**
- Text must include full Arabic diacritics (فَتْحَة، كَسْرَة، ضَمَّة، سُكُون، etc.)
- Use AI tools (ChatGPT, Claude) or [online diacritizers](https://tahadz.com/mishkal) to add tashkeel

**Example:**
- ❌ Bad: "مرحبا بكم في النموذج"
- ✅ Good: "مَرْحَبًا بِكُمْ فِي النَّمُوذَجِ"

### Reference Audio Requirements
- **Duration:** 5-30 seconds of clear speech
- **Quality:** Clean recording, minimal background noise
- **Speaker:** Single speaker only
- **Language:** Arabic (preferably MSA or Classical)
- **Format:** WAV file recommended

### Reference Transcript Requirements
- Must match reference audio exactly
- Must include full diacritics
- Text alignment is critical for quality

## 📊 Training Details

### Training Data

**Dataset:** [MBZUAI/ClArTTS](https://huggingface.co/datasets/MBZUAI/ClArTTS)
- **Full dataset size:** 12 hours, 9,500 utterances
- **Training subset:** 30% (~2,850 utterances)
- **Speaker:** Single male speaker
- **Language:** Classical Arabic (MSA)
- **Sample rate:** 40.1 kHz (resampled to 24 kHz for training)
- **Text quality:** Fully diacritized

### Training Procedure

**Fine-tuning Framework:** Axolotl + LoRA

**Training Configuration:**
```yaml
Base Model: SparkAudio/Spark-TTS-0.5B (LLM component only)
Fine-tuning Method: Full fine-tuning (not LoRA)
Epochs: 20
Batch Size: 8 (1 per device × 8 gradient accumulation)
Learning Rate: 2e-4
Optimizer: AdamW (torch fused)
LR Scheduler: Cosine
Warmup Steps: 10
Sequence Length: 1024
Precision: bfloat16
Gradient Checkpointing: Enabled
```

**Data Processing:**
1. Audio resampled to 24 kHz
2. Semantic tokens extracted using BiCodec
3. Training pairs: `[text, semantic_tokens]` created for LLM training
4. Text normalized to lowercase during processing

**Training Infrastructure:**
- Hardware: Single NVIDIA GPU (Colab)
- Training Time: ~3-4 hours
- Framework: PyTorch + Transformers + Axolotl

**Data Preparation Steps:**
```python
# 1. Load ClArTTS from HuggingFace
# 2. Resample audio from 40.1 kHz → 24 kHz
# 3. Extract semantic tokens using BiCodec
# 4. Create metadata: [audio_path, text]
# 5. Generate training pairs: [text → semantic_tokens]
```

**Base Model:**
```bibtex
@misc{sparktts2024,
  title={Spark-TTS: Zero-Shot Multi-Style Text-to-Speech via Large Language Models},
  author={SparkAudio Team},
  year={2024},
  url={https://github.com/SparkAudio/Spark-TTS}
}
```

**Training Dataset:**
```bibtex
@inproceedings{kulkarni2023clartts,
  author={Ajinkya Kulkarni and Atharva Kulkarni and Sara Shatnawi and Hanan Aldarmaki},
  title={ClArTTS: An Open-Source Classical Arabic Text-to-Speech Corpus},
  year={2023},
  booktitle={INTERSPEECH 2023},
  pages={5511--5515},
  doi={10.21437/Interspeech.2023-2224}
}
```

## 👍 Acknowledgments

- **Base Model:** [SparkAudio Team](https://github.com/SparkAudio) for Spark-TTS-0.5B
- **Dataset:** MBZUAI for ClArTTS corpus
- **Frameworks:** Hugging Face Transformers, Axolotl, PyTorch

## 📄 License

Apache 2.0 (same as base model)

## 📧 Contact

For questions, collaboration, or support:
- **Email:** azdinsahir11@gmail.com
- **Hugging Face:** [@azeddinShr](https://huggingface.co/azeddinShr)
- **Model Discussions:** Use the Community tab above

---

**Note:** This model requires the official Spark-TTS repository for inference. The model files alone are not sufficient - you must clone the Spark-TTS repo and use their inference pipeline.