You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Access to this model is manually gated. Please describe your intended use.

Chatterbox-MTL Central-Asian (Tajik · Uzbek · Kazakh · Kyrgyz)

A single text-to-speech model that speaks four Central-Asian languages — Tajik, Uzbek, Kazakh and Kyrgyz — with the target language selected at inference time via a language_id. It is Chatterbox Multilingual v3 fine-tuned jointly on all four languages. Voice cloning is supported: every generation is conditioned on a reference .wav.

Model summary


Base	Chatterbox Multilingual v3 (ResembleAI)
Fine-tuning	Joint multilingual, t3-only (s3gen and ve frozen)
Precision	fp32
Learning rate	1e-5
Vocabulary	extended to 2458 text tokens (stock is 2454) with per-utterance language tokens `[tg]` `[uz]` `[kk]` `[ky]` at ids 2454–2457
Steps trained	20,000
Winner checkpoint	step 10,000 (best joint quality; later checkpoints regress)
License	MIT
Watermark	Perth (Resemble) watermarked output

Only the fine-tuned extended t3 (t3_mtl23ls_v2.safetensors) is trained; s3gen.pt, ve.pt and conds.pt are the frozen Chatterbox components, and grapheme_mtl_merged_expanded_v1.json is the extended 2458-token tokenizer.

Training data

344,031 utterances total:

Language	Utts	Source
Tajik (tg)	100,000	audiobook + diarized
Uzbek (uz)	100,000	uzbekvoice
Kazakh (kk)	100,000	KSC2
Kyrgyz (ky)	7,339 clips, oversampled 6×	CommonVoice 17 (ky) + FLEURS (ky)

FLEURS test sets were held out for tg / uz / kk.

Results (round-trip TTS eval, n = 60 per language)

Generate from held-out text → transcribe with an ASR scorer → compare to reference.

Language	Metric	This combined model	Specialist (single-lang FT)
Tajik	WER	11.8% median	10.7% median
Uzbek	WER	18.2% median	14.8% median
Kazakh	CER	6.9% median	7.4% median (combined is better)
Kyrgyz	—	not scored (no trusted Kyrgyz ASR scorer yet)	—

Notes:

Kazakh is reported as CER (character error rate): Kazakh is agglutinative and a base-whisper WER scorer inflates the number, so CER is the trustworthy metric.
Uzbek is the weak language in joint training — its Latin-minority script destabilizes relative to the Cyrillic-script Tajik/Kazakh.

Raw evidence is in eval/: per_utt_{tg,uz,kk}_n60.jsonl (raw ref/hyp/wer per utterance at step 10000) and scores.txt.

Inference

CRITICAL: set CB_TEXT_VOCAB=2458 before loading. This model has 2458 text tokens; without this env the loader assumes 2454 and fails.

import os
os.environ["CB_TEXT_VOCAB"] = "2458"  # REQUIRED — extended vocab

from chatterbox.mtl_tts import ChatterboxMultilingualTTS

model = ChatterboxMultilingualTTS.from_local("path/to/model_dir", device="cuda")

# eager-attention workaround
model.t3.tfmr.config._attn_implementation = "eager"

wav = model.generate(
    "Матни шумо дар ин ҷо",       # your text
    language_id="tg",            # one of: tg | uz | kk | ky
    audio_prompt_path="ref.wav", # reference voice (cloning) — required
)

language_id selects the spoken language: "tg", "uz", "kk", or "ky".
A reference .wav (audio_prompt_path) is required — this is a voice-cloning TTS.
Output audio is Perth-watermarked.

Files

t3_mtl23ls_v2.safetensors — fine-tuned extended t3 (trained)
s3gen.pt — frozen S3 generator
ve.pt — frozen voice encoder
conds.pt — default conditioning
grapheme_mtl_merged_expanded_v1.json — extended 2458-token tokenizer ([tg][uz][kk][ky])

License & attributions

Model: MIT (following Chatterbox).
Training data attributions: uzbekvoice (CC0), KSC2 (Kazakh Speech Corpus 2), CommonVoice (CC0), FLEURS (CC-BY).

Downloads last month: -