---
license: other
license_name: coqui-public-model-license
license_link: https://coqui.ai/cpml
language:
- ne
library_name: coqui
pipeline_tag: text-to-speech
tags:
- text-to-speech
- tts
- xtts
- xtts-v2
- voice-cloning
- nepali
- coqui
base_model: coqui/XTTS-v2
---

# XTTS v2 — Nepali (नेपाली) fine-tune

A fine-tune of [Coqui **XTTS v2**](https://huggingface.co/coqui/XTTS-v2) that adds
**Nepali (`ne`)** — a language the base model does not ship. XTTS v2 is a zero-shot
voice-cloning TTS model: given a few seconds of any voice and a line of text, it
speaks that line in that voice. This checkpoint teaches it to do so fluently in Nepali.

- **Base model:** coqui/XTTS-v2
- **Method:** full GPT fine-tune (not frozen), restored from the base checkpoint
- **Data:** 4,140 single-speaker Nepali clips (3,933 train / 207 eval, 95/5)
- **Language route:** a new `ne` code registered in `config.json`, over the existing Devanagari tokenizer
- **Output:** 24 kHz

## Checkpoints

Two checkpoints are provided as self-contained folders. **Epoch 10 is recommended** —
it generalises best; later epochs overfit the single training speaker (eval mel-CE rises
while text-CE stays flat).

| Folder | When to use |
|---|---|
| [`epoch-10/`](./epoch-10) | **Recommended.** Best generalisation. |
| [`epoch-20/`](./epoch-20) | More adapted to the training speaker, less general. |

Each folder is drop-in XTTS format: `model.pth`, `config.json`, `vocab.json`, `speakers_xtts.pth`.

## Usage

```python
from huggingface_hub import snapshot_download
from TTS.tts.configs.xtts_config import XttsConfig
from TTS.tts.models.xtts import Xtts
import torchaudio, torch

d = snapshot_download("Oshara/xtts-v2-nepali", allow_patterns=["epoch-10/*"]) + "/epoch-10"

config = XttsConfig(); config.load_json(f"{d}/config.json")
model = Xtts.init_from_config(config)
model.load_checkpoint(config, checkpoint_path=f"{d}/model.pth",
                      vocab_path=f"{d}/vocab.json",
                      speaker_file_path=f"{d}/speakers_xtts.pth", eval=True)
model.cuda()

out = model.synthesize(
    "नेपालका हिमालहरू संसारभर प्रसिद्ध छन्।",
    config, speaker_wav="reference_voice.wav", language="ne",
    temperature=0.65, repetition_penalty=5.0,
)
torchaudio.save("out.wav", torch.tensor(out["wav"]).unsqueeze(0), 24000)
```

**Tip:** for long or numeral-heavy text, normalise first (strip zero-width chars, split on
the danda `।`, expand Nepali digits to words, insert short inter-sentence silence). This
prevents the autoregressive GPT from drifting into noise between sentences.

## Evaluation

Evaluated on **NepTTS-Bench** (205 sentences) plus voice-cloning and prosody metrics.
The intelligibility and quality numbers place it among the strongest Nepali systems.

| Metric | Score | Note |
|---|---|---|
| Whisper round-trip CER | **0.380** | 2nd best on the benchmark |
| MMS-1b round-trip CER | **0.199** | 2nd best |
| XLS-R Nepali CER | 0.183 | mid-pack |
| SCOREQ auto-MOS | **4.21** | 3rd overall |
| SQUIM MOS (est.) | 4.69 | reference-free naturalness |
| SQUIM STOI / PESQ / SI-SDR | 0.99 / 3.78 / 27.6 dB | clean signal |
| Speaker similarity (SECS, WavLM-SV) | **0.923** | high cloning fidelity (same-speaker threshold ≈ 0.86) |
| Pitch expressiveness (F0 std) | 3.58 st | somewhat flatter than natural (~4.95 st) — the main soft spot |

## Limitations

- Single-speaker training corpus; timbre diversity comes from XTTS's zero-shot cloning, not the fine-tune.
- Slightly **reduced pitch expressiveness** vs. natural Nepali speech (flatter intonation).
- Nepali is routed over the existing Devanagari tokenizer; no vocabulary was added.

## License

Inherits the [Coqui Public Model License (CPML)](https://coqui.ai/cpml) from the XTTS v2 base model.