linyueqian commited on
Commit
82561b9
·
verified ·
1 Parent(s): 58f6e41

Add vLLM-Omni as a self-hosting option for Higgs Audio v3

Browse files

Adds a vLLM-Omni Usage subsection alongside SGLang Usage. vLLM-Omni has first-class Higgs Audio v3 TTS support merged upstream (vllm-project/vllm-omni#4169, perf #4204), serving the same OpenAI-compatible /v1/audio/speech API with zero-shot voice cloning. Serve command mirrors the official vLLM-Omni recipe.

Files changed (1) hide show
  1. README.md +21 -0
README.md CHANGED
@@ -510,6 +510,27 @@ Throughput on Seed-TTS EN (full set, **N=1088** per run). Client `--max-concurre
510
 
511
  To reproduce the results, follow the instructions in [this script](https://github.com/sgl-project/sglang-omni/blob/main/benchmarks/eval/benchmark_tts_seedtts.py).
512
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
513
  ### API Usage
514
 
515
  For zero-ops deployment, use the [**Boson AI API**](https://docs.boson.ai/models/higgs-audio-tts/overview).
 
510
 
511
  To reproduce the results, follow the instructions in [this script](https://github.com/sgl-project/sglang-omni/blob/main/benchmarks/eval/benchmark_tts_seedtts.py).
512
 
513
+ ### vLLM-Omni Usage
514
+
515
+ You can also serve these weights with [**vLLM-Omni**](https://github.com/vllm-project/vllm-omni), which exposes the same OpenAI-compatible `/v1/audio/speech` API with zero-shot voice cloning.
516
+
517
+ ```bash
518
+ hf download bosonai/higgs-audio-v3-tts-4b
519
+
520
+ vllm-omni serve bosonai/higgs-audio-v3-tts-4b \
521
+ --host 0.0.0.0 --port 8095 \
522
+ --trust-remote-code --omni
523
+ ```
524
+
525
+ ```bash
526
+ curl -X POST http://localhost:8095/v1/audio/speech \
527
+ -H "Content-Type: application/json" \
528
+ -d '{"model": "bosonai/higgs-audio-v3-tts-4b", "input": "Hello, how are you?"}' \
529
+ --output output.wav
530
+ ```
531
+
532
+ Plain-text TTS, voice-clone, and benchmark recipes are in the [vLLM-Omni Higgs Audio v3 recipe](https://github.com/vllm-project/vllm-omni/blob/main/recipes/BosonAI/Higgs-Audio-V3-TTS.md).
533
+
534
  ### API Usage
535
 
536
  For zero-ops deployment, use the [**Boson AI API**](https://docs.boson.ai/models/higgs-audio-tts/overview).