muli commited on
Commit
59904ba
·
verified ·
1 Parent(s): 32e1387

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +17 -17
README.md CHANGED
@@ -114,16 +114,16 @@ pipeline_tag: text-to-speech
114
  library_name: transformers
115
  ---
116
 
117
- # Higgs Audio v3 TTS
118
 
119
  <div align="left" style="display: flex; justify-content: flex-start; margin-top: 10px;"> <a href="https://www.boson.ai/blog/higgs-audio-v3-tts"> <img src="https://img.shields.io/badge/💚-Boson%20Blog-228B22" style="margin-right: 5px;"> </a> </div>
120
 
121
- Higgs Audio v3 TTS is built for voice chat: it **speaks, not just reads**. It turns model responses into expressive conversational speech across **100+ languages**, with **zero-shot voice cloning** and **inline control** over emotion, style, prosody, pauses, and sound effects.
122
 
123
  > [!TIP]
124
- > Released for research and non-commercial use under the **Boson Higgs Audio v3 Research and Non-Commercial License**. Production, hosted APIs, or revenue-generating use requires a separate commercial license. Prohibited: voice cloning without consent, impersonation, fraud, election deception, biometric surveillance, or any unlawful use.
125
 
126
- ![Higgs Audio v3 TTS Architecture](./assets/model_architecture.png)
127
 
128
  Higgs autoregressive decoder consumes interleaved text and audio tokens. Audio is encoded by the **Higgs Tokenizer** into 8 codebooks at 25 fps, staggered via a **delay pattern**, then mapped to backbone hidden states through a **multi-codebook fused embedding**. Output codes pass through a **multi-codebook fused head**, are de-delayed, and decoded back to waveform.
129
 
@@ -187,7 +187,7 @@ All tags follow `<|category:value|>` syntax and can be inserted mid-utterance.
187
 
188
  ### Multilingual Voice Clone
189
 
190
- We evaluate Higgs Audio v3 TTS on public multilingual TTS suites and our internal 111-language Higgs-Multilingual set, covering both common and lower-resource languages.
191
 
192
  WER / CER (↓, ×100) macro-averaged across each benchmark's language set. Lower is better; **bold** marks the best per row. All numbers are reproducible end-to-end with original metrics and normalization.
193
 
@@ -195,8 +195,8 @@ WER / CER (↓, ×100) macro-averaged across each benchmark's language set. Lowe
195
  <table style="width:100%;border-collapse:collapse;font-size:clamp(11px,1.4vw,13px);text-align:center">
196
  <thead><tr>
197
  <th style="padding:18px 24px;text-align:center;font-weight:600;border-bottom:2px solid #16a34a;color:#16a34a">Benchmark</th>
198
- <th style="padding:18px 24px;text-align:center;font-weight:500;white-space:nowrap;border-bottom:2px solid #16a34a;color:#16a34a;font-size:14px">Higgs Audio v2</th>
199
- <th style="padding:18px 24px;text-align:center;font-weight:500;white-space:nowrap;border-bottom:2px solid #16a34a;color:#16a34a;font-size:14px">Higgs Audio v3</th>
200
  <th style="padding:18px 24px;text-align:center;font-weight:500;white-space:nowrap;border-bottom:2px solid #16a34a;color:#16a34a;font-size:14px">Fish Audio S2 Pro</th>
201
  <th style="padding:18px 24px;text-align:center;font-weight:500;white-space:nowrap;border-bottom:2px solid #16a34a;color:#16a34a;font-size:14px">Qwen3-TTS-1.7B</th>
202
  <th style="padding:18px 24px;text-align:center;font-weight:500;white-space:nowrap;border-bottom:2px solid #16a34a;color:#16a34a;font-size:14px">VibeVoice-7B</th>
@@ -286,7 +286,7 @@ Win-rate (↑) per category — judge preference vs the BASELINE row; **bold** m
286
  </tr></thead>
287
  <tbody>
288
  <tr>
289
- <td style="padding:16px 24px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">Higgs Audio v3</td>
290
  <td style="padding:16px 24px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);font-weight:600">53.65%</td>
291
  <td style="padding:16px 24px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">53.75%</td>
292
  <td style="padding:16px 24px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);font-weight:600">48.75%</td>
@@ -372,10 +372,10 @@ uv pip install -v -e .
372
 
373
  ```bash
374
  export HF_TOKEN=hf_xxxxxxxxxxxxxxxx
375
- hf download bosonai/higgs-audio-v3-tts-4b
376
 
377
  sgl-omni serve \
378
- --model-path bosonai/higgs-audio-v3-tts-4b \
379
  --port 8000
380
  ```
381
 
@@ -515,9 +515,9 @@ To reproduce the results, follow the instructions in [this script](https://githu
515
  You can also serve these weights with [**vLLM-Omni**](https://github.com/vllm-project/vllm-omni), which exposes the same OpenAI-compatible `/v1/audio/speech` API with zero-shot voice cloning.
516
 
517
  ```bash
518
- hf download bosonai/higgs-audio-v3-tts-4b
519
 
520
- vllm-omni serve bosonai/higgs-audio-v3-tts-4b \
521
  --host 0.0.0.0 --port 8095 \
522
  --trust-remote-code --omni
523
  ```
@@ -525,11 +525,11 @@ vllm-omni serve bosonai/higgs-audio-v3-tts-4b \
525
  ```bash
526
  curl -X POST http://localhost:8095/v1/audio/speech \
527
  -H "Content-Type: application/json" \
528
- -d '{"model": "bosonai/higgs-audio-v3-tts-4b", "input": "Hello, how are you?"}' \
529
  --output output.wav
530
  ```
531
 
532
- Plain-text TTS, voice-clone, and benchmark recipes are in the [vLLM-Omni Higgs Audio v3 recipe](https://github.com/vllm-project/vllm-omni/blob/main/recipes/BosonAI/Higgs-Audio-V3-TTS.md).
533
 
534
  ### API Usage
535
 
@@ -539,13 +539,13 @@ For zero-ops deployment, use the [**Boson AI API**](https://docs.boson.ai/models
539
 
540
  ```bibtex
541
  @misc{bosonai_higgs_audio_tts_v3_2026,
542
- title = {Higgs Audio v3 TTS: Conversational Speech for Voice AI from Boson AI},
543
  author = {Boson AI},
544
  year = {2026},
545
- howpublished = {https://huggingface.co/bosonai/higgs-audio-v3-tts-4b},
546
  }
547
  ```
548
 
549
  ## License
550
 
551
- Boson Higgs Audio v3 Research and Non-Commercial License — see [LICENSE](./LICENSE).
 
114
  library_name: transformers
115
  ---
116
 
117
+ # Higgs TTS v3
118
 
119
  <div align="left" style="display: flex; justify-content: flex-start; margin-top: 10px;"> <a href="https://www.boson.ai/blog/higgs-audio-v3-tts"> <img src="https://img.shields.io/badge/💚-Boson%20Blog-228B22" style="margin-right: 5px;"> </a> </div>
120
 
121
+ Higgs TTS v3 is built for voice chat: it **speaks, not just reads**. It turns model responses into expressive conversational speech across **100+ languages**, with **zero-shot voice cloning** and **inline control** over emotion, style, prosody, pauses, and sound effects.
122
 
123
  > [!TIP]
124
+ > Released for research and non-commercial use under the **Boson Higgs TTS v3 Research and Non-Commercial License**. Production, hosted APIs, or revenue-generating use requires a separate commercial license. Prohibited: voice cloning without consent, impersonation, fraud, election deception, biometric surveillance, or any unlawful use.
125
 
126
+ ![Higgs TTS v3 Architecture](./assets/model_architecture.png)
127
 
128
  Higgs autoregressive decoder consumes interleaved text and audio tokens. Audio is encoded by the **Higgs Tokenizer** into 8 codebooks at 25 fps, staggered via a **delay pattern**, then mapped to backbone hidden states through a **multi-codebook fused embedding**. Output codes pass through a **multi-codebook fused head**, are de-delayed, and decoded back to waveform.
129
 
 
187
 
188
  ### Multilingual Voice Clone
189
 
190
+ We evaluate Higgs TTS v3 on public multilingual TTS suites and our internal 111-language Higgs-Multilingual set, covering both common and lower-resource languages.
191
 
192
  WER / CER (↓, ×100) macro-averaged across each benchmark's language set. Lower is better; **bold** marks the best per row. All numbers are reproducible end-to-end with original metrics and normalization.
193
 
 
195
  <table style="width:100%;border-collapse:collapse;font-size:clamp(11px,1.4vw,13px);text-align:center">
196
  <thead><tr>
197
  <th style="padding:18px 24px;text-align:center;font-weight:600;border-bottom:2px solid #16a34a;color:#16a34a">Benchmark</th>
198
+ <th style="padding:18px 24px;text-align:center;font-weight:500;white-space:nowrap;border-bottom:2px solid #16a34a;color:#16a34a;font-size:14px">Higgs TTS v2</th>
199
+ <th style="padding:18px 24px;text-align:center;font-weight:500;white-space:nowrap;border-bottom:2px solid #16a34a;color:#16a34a;font-size:14px">Higgs TTS v3</th>
200
  <th style="padding:18px 24px;text-align:center;font-weight:500;white-space:nowrap;border-bottom:2px solid #16a34a;color:#16a34a;font-size:14px">Fish Audio S2 Pro</th>
201
  <th style="padding:18px 24px;text-align:center;font-weight:500;white-space:nowrap;border-bottom:2px solid #16a34a;color:#16a34a;font-size:14px">Qwen3-TTS-1.7B</th>
202
  <th style="padding:18px 24px;text-align:center;font-weight:500;white-space:nowrap;border-bottom:2px solid #16a34a;color:#16a34a;font-size:14px">VibeVoice-7B</th>
 
286
  </tr></thead>
287
  <tbody>
288
  <tr>
289
+ <td style="padding:16px 24px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">Higgs TTS v3</td>
290
  <td style="padding:16px 24px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);font-weight:600">53.65%</td>
291
  <td style="padding:16px 24px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">53.75%</td>
292
  <td style="padding:16px 24px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);font-weight:600">48.75%</td>
 
372
 
373
  ```bash
374
  export HF_TOKEN=hf_xxxxxxxxxxxxxxxx
375
+ hf download bosonai/higgs-tts-v3-4b
376
 
377
  sgl-omni serve \
378
+ --model-path bosonai/higgs-tts-v3-4b \
379
  --port 8000
380
  ```
381
 
 
515
  You can also serve these weights with [**vLLM-Omni**](https://github.com/vllm-project/vllm-omni), which exposes the same OpenAI-compatible `/v1/audio/speech` API with zero-shot voice cloning.
516
 
517
  ```bash
518
+ hf download bosonai/higgs-tts-v3-4b
519
 
520
+ vllm-omni serve bosonai/higgs-tts-v3-4b \
521
  --host 0.0.0.0 --port 8095 \
522
  --trust-remote-code --omni
523
  ```
 
525
  ```bash
526
  curl -X POST http://localhost:8095/v1/audio/speech \
527
  -H "Content-Type: application/json" \
528
+ -d '{"model": "bosonai/higgs-tts-v3-4b", "input": "Hello, how are you?"}' \
529
  --output output.wav
530
  ```
531
 
532
+ Plain-text TTS, voice-clone, and benchmark recipes are in the [vLLM-Omni Higgs TTS v3 recipe](https://github.com/vllm-project/vllm-omni/blob/main/recipes/BosonAI/Higgs-Audio-V3-TTS.md).
533
 
534
  ### API Usage
535
 
 
539
 
540
  ```bibtex
541
  @misc{bosonai_higgs_audio_tts_v3_2026,
542
+ title = {Higgs TTS v3: Conversational Speech for Voice AI from Boson AI},
543
  author = {Boson AI},
544
  year = {2026},
545
+ howpublished = {https://huggingface.co/bosonai/higgs-tts-v3-4b},
546
  }
547
  ```
548
 
549
  ## License
550
 
551
+ Boson Higgs TTS v3 Research and Non-Commercial License — see [LICENSE](./LICENSE).