---
license: mit
language:
- tg
- uz
- kk
- ky
library_name: chatterbox
pipeline_tag: text-to-speech
tags:
- text-to-speech
- tts
- voice-cloning
- chatterbox
- central-asian
- tajik
- uzbek
- kazakh
- kyrgyz
extra_gated_prompt: "Access to this model is manually gated. Please describe your intended use."
---

# Chatterbox-MTL Central-Asian (Tajik · Uzbek · Kazakh · Kyrgyz)

A **single** text-to-speech model that speaks **four Central-Asian languages** — Tajik,
Uzbek, Kazakh and Kyrgyz — with the target language selected at inference time via a
`language_id`. It is [Chatterbox Multilingual v3](https://github.com/resemble-ai/chatterbox)
fine-tuned **jointly** on all four languages. Voice cloning is supported: every generation is
conditioned on a reference `.wav`.

## Model summary

| | |
|---|---|
| Base | Chatterbox Multilingual v3 (ResembleAI) |
| Fine-tuning | Joint multilingual, **t3-only** (s3gen and ve frozen) |
| Precision | fp32 |
| Learning rate | 1e-5 |
| Vocabulary | **extended to 2458 text tokens** (stock is 2454) with per-utterance language tokens `[tg]` `[uz]` `[kk]` `[ky]` at ids 2454–2457 |
| Steps trained | 20,000 |
| **Winner checkpoint** | **step 10,000** (best joint quality; later checkpoints regress) |
| License | MIT |
| Watermark | Perth (Resemble) watermarked output |

Only the fine-tuned extended **t3** (`t3_mtl23ls_v2.safetensors`) is trained; `s3gen.pt`,
`ve.pt` and `conds.pt` are the frozen Chatterbox components, and
`grapheme_mtl_merged_expanded_v1.json` is the **extended 2458-token** tokenizer.

## Training data

**344,031 utterances total:**

| Language | Utts | Source |
|---|---|---|
| Tajik (tg) | 100,000 | audiobook + diarized |
| Uzbek (uz) | 100,000 | uzbekvoice |
| Kazakh (kk) | 100,000 | KSC2 |
| Kyrgyz (ky) | 7,339 clips, **oversampled 6×** | CommonVoice 17 (ky) + FLEURS (ky) |

FLEURS test sets were held out for tg / uz / kk.

## Results (round-trip TTS eval, n = 60 per language)

Generate from held-out text → transcribe with an ASR scorer → compare to reference.

| Language | Metric | This combined model | Specialist (single-lang FT) |
|---|---|---|---|
| Tajik | WER | **11.8%** median | 10.7% median |
| Uzbek | WER | **18.2%** median | 14.8% median |
| Kazakh | CER | **6.9%** median | 7.4% median (combined is **better**) |
| Kyrgyz | — | not scored (no trusted Kyrgyz ASR scorer yet) | — |

Notes:
- **Kazakh is reported as CER** (character error rate): Kazakh is agglutinative and a
  base-whisper WER scorer inflates the number, so CER is the trustworthy metric.
- **Uzbek is the weak language** in joint training — its Latin-minority script destabilizes
  relative to the Cyrillic-script Tajik/Kazakh.

Raw evidence is in [`eval/`](./eval): `per_utt_{tg,uz,kk}_n60.jsonl` (raw `ref`/`hyp`/`wer`
per utterance at step 10000) and `scores.txt`.

## Inference

> **CRITICAL:** set `CB_TEXT_VOCAB=2458` before loading. This model has **2458** text tokens;
> without this env the loader assumes 2454 and fails.

```python
import os
os.environ["CB_TEXT_VOCAB"] = "2458"  # REQUIRED — extended vocab

from chatterbox.mtl_tts import ChatterboxMultilingualTTS

model = ChatterboxMultilingualTTS.from_local("path/to/model_dir", device="cuda")

# eager-attention workaround
model.t3.tfmr.config._attn_implementation = "eager"

wav = model.generate(
    "Матни шумо дар ин ҷо",       # your text
    language_id="tg",            # one of: tg | uz | kk | ky
    audio_prompt_path="ref.wav", # reference voice (cloning) — required
)
```

- `language_id` selects the spoken language: `"tg"`, `"uz"`, `"kk"`, or `"ky"`.
- A reference `.wav` (`audio_prompt_path`) is required — this is a voice-cloning TTS.
- Output audio is **Perth-watermarked**.

## Files

- `t3_mtl23ls_v2.safetensors` — fine-tuned extended t3 (trained)
- `s3gen.pt` — frozen S3 generator
- `ve.pt` — frozen voice encoder
- `conds.pt` — default conditioning
- `grapheme_mtl_merged_expanded_v1.json` — extended 2458-token tokenizer (`[tg][uz][kk][ky]`)

## License & attributions

- Model: **MIT** (following Chatterbox).
- Training data attributions: **uzbekvoice** (CC0), **KSC2** (Kazakh Speech Corpus 2),
  **CommonVoice** (CC0), **FLEURS** (CC-BY).