bosonai
/

higgs-tts-2-3b-base

@@ -10,11 +10,12 @@ pipeline_tag: text-to-speech
 # Higgs Audio V2: Redefining Expressiveness in Audio Generation
 We are open-sourcing Higgs Audio v2, a powerful audio foundation model pretrained on over 10 million hours of audio data and a diverse set of text data.
 Despite having no post-training or fine-tuning, Higgs Audio v2 excels in expressive audio generation, thanks to its deep language and acoustic understanding.
 On [EmergentTTS-Eval](https://github.com/boson-ai/emergenttts-eval-public), the model achieves win rates of **75.7%** and **55.7%** over "gpt-4o-mini-tts" on the "Emotions" and "Questions" categories, respectively. It also obtains state-of-the-art performance on traditional TTS benchmarks like Seed-TTS Eval and Emotional Speech Dataset (ESD). Moreover, the model demonstrates capabilities rarely seen in previous systems, including automatic prosody adaptation during narration, zero-shot generation of natural multi-speaker dialogues in multiple languages, melodic humming with the cloned voice, and simultaneous generation of speech and background music.
-Check our open-source repository https://github.com/boson-ai/higgs-audio for more details.
 <p>
@@ -44,7 +45,7 @@ Higgs Audio v2 adopts the "generation variant" depicted in the architecture figu
 We introduce a new discretized audio tokenizer that runs at just 25 frames per second while keeping—or even improving—audio quality compared to tokenizers with twice the bitrate.
 Our model is the first to train on 24 kHz data covering speech, music, and sound events in one unified system.
 It also uses a simple non-diffusion encoder/decoder for fast, batch inference. It achieves state-of-the-art performance in semantic and acoustic evaluations.
-Check https://huggingface.co/bosonai/higgs-audio-v2-tokenizer-staging for more information about the tokenizer.
 ### Model Architecture -- Dual FFN
@@ -54,7 +55,7 @@ DualFFN acts as an audio-specific expert, boosting the LLM's performance with mi
 Our implementation preserves 91% of the original LLM’s training speed with the inclusion of DualFFN, which has 2.2B parameters.
 Thus, the total number of parameter for Higgs Audio v2 is 3.6B (LLM) + 2.2B (Audio Dual FFN), and it has the same training / inference FLOPs as Llama-3.2-3B.
 Ablation study shows that the model equipped with DualFFN consistently outperforms its counterpart in terms of word error rate (WER) and speaker similarity.
-See [Higgs-Audio Architecture Blog](https://github.com/boson-ai/higgs-audio/tech_blogs/ARCHITECTURE_BLOG.md) for more information.
 ## Evaluation
@@ -77,7 +78,7 @@ We prompt Higgs Audio v2 with `<ref_text, ref_audio, text>` for zero-shot TTS. W
 #### EmergentTTS-Eval ("Emotions" and "Questions")
-Following the [EmergentTTS-Eval Paper](https://arxiv.org/abs/2505.23009), we report the win-rate over "gpt-4o-mini-tts" with the "alloy" voice.
 | Model                              | Emotions (%) ↑ | Questions (%) ↑ |
 |------------------------------------|--------------|----------------|
@@ -116,7 +117,7 @@ We evaluate the word-error-rate (WER) and the geometric mean between intra-speak
 ## Get Started
-You need to first install the [higgs-audio codebase](https://github.com/boson-ai/higgs-audio):
 ```bash
 git clone https://github.com/boson-ai/higgs-audio.git
@@ -139,8 +140,8 @@ import torchaudio
 import time
 import click
-MODEL_PATH = "bosonai/higgs-audio-v2-generation-3B-staging"
-AUDIO_TOKENIZER_PATH = "bosonai/higgs-audio-v2-tokenizer-staging"
 system_prompt = (
     "Generate audio following instruction.\n\n<|scene_desc_start|>\nSPEAKER0: british accent\n<|scene_desc_end|>"

 # Higgs Audio V2: Redefining Expressiveness in Audio Generation
+Check our open-source repository https://github.com/boson-ai/higgs-audio for more details!
 We are open-sourcing Higgs Audio v2, a powerful audio foundation model pretrained on over 10 million hours of audio data and a diverse set of text data.
 Despite having no post-training or fine-tuning, Higgs Audio v2 excels in expressive audio generation, thanks to its deep language and acoustic understanding.
 On [EmergentTTS-Eval](https://github.com/boson-ai/emergenttts-eval-public), the model achieves win rates of **75.7%** and **55.7%** over "gpt-4o-mini-tts" on the "Emotions" and "Questions" categories, respectively. It also obtains state-of-the-art performance on traditional TTS benchmarks like Seed-TTS Eval and Emotional Speech Dataset (ESD). Moreover, the model demonstrates capabilities rarely seen in previous systems, including automatic prosody adaptation during narration, zero-shot generation of natural multi-speaker dialogues in multiple languages, melodic humming with the cloned voice, and simultaneous generation of speech and background music.
 <p>
 We introduce a new discretized audio tokenizer that runs at just 25 frames per second while keeping—or even improving—audio quality compared to tokenizers with twice the bitrate.
 Our model is the first to train on 24 kHz data covering speech, music, and sound events in one unified system.
 It also uses a simple non-diffusion encoder/decoder for fast, batch inference. It achieves state-of-the-art performance in semantic and acoustic evaluations.
+Check https://huggingface.co/bosonai/higgs-audio-v2-tokenizer for more information about the tokenizer.
 ### Model Architecture -- Dual FFN
 Our implementation preserves 91% of the original LLM’s training speed with the inclusion of DualFFN, which has 2.2B parameters.
 Thus, the total number of parameter for Higgs Audio v2 is 3.6B (LLM) + 2.2B (Audio Dual FFN), and it has the same training / inference FLOPs as Llama-3.2-3B.
 Ablation study shows that the model equipped with DualFFN consistently outperforms its counterpart in terms of word error rate (WER) and speaker similarity.
+See [our architecture blog](https://github.com/boson-ai/higgs-audio/tech_blogs/ARCHITECTURE_BLOG.md) for more information.
 ## Evaluation
 #### EmergentTTS-Eval ("Emotions" and "Questions")
+Following the [EmergentTTS-Eval Paper](https://arxiv.org/abs/2505.23009), we report the win-rate over "gpt-4o-mini-tts" with the "alloy" voice. Results of Higgs Audio v2 is obtained with the voice of "belinda".
 | Model                              | Emotions (%) ↑ | Questions (%) ↑ |
 |------------------------------------|--------------|----------------|
 ## Get Started
+You need to first install the [higgs-audio](https://github.com/boson-ai/higgs-audio):
 ```bash
 git clone https://github.com/boson-ai/higgs-audio.git
 import time
 import click
+MODEL_PATH = "bosonai/higgs-audio-v2-generation-3B-base"
+AUDIO_TOKENIZER_PATH = "bosonai/higgs-audio-v2-tokenizer"
 system_prompt = (
     "Generate audio following instruction.\n\n<|scene_desc_start|>\nSPEAKER0: british accent\n<|scene_desc_end|>"