ct03/omnivoice-onnx-int8hq

OmniVoice ONNX (INT8-HQ) — CPU inference

INT8-HQ ONNX export of k2-fsa/OmniVoice, optimized to run on CPU with onnxruntime (no GPU, no PyTorch needed at inference time).

• Multilingual TTS (646 languages incl. Vietnamese) + zero-shot voice cloning + voice design.
• 24 kHz output, 8 codebooks, frame rate 75.
• INT8-HQ: cosine > 0.9999 vs FP32, perceptually almost indistinguishable.

Components

Folder	Role	Size
omnivoice_lm_int8_hq/	Diffusion LM (INT8-HQ, audio head kept FP32)	~611 MB
audio_tokenizer_decoder_int8/	tokens → 24 kHz waveform	~82 MB
audio_tokenizer_encoder_int8/	reference audio → codec tokens (voice cloning)	~378 MB
tokenizer.json, config.json	text tokenizer + config	small

Each model is model.onnx + model.onnx_data (ONNX external data) — download both files.

ONNX I/O

Model	Input	Output
LM	input_ids[batch,8,seq] int64; audio_mask[batch,seq] bool; attention_mask[batch,1,seq,seq] bool; position_ids[batch,seq] int64	logits[batch,8,seq,1025] f32
Encoder	audio[batch,1,num_samples] f32 @24kHz	audio_codes[batch,8,num_frames] int64
Decoder	audio_codes[batch,8,num_frames] int64	audio[batch,1,num_samples] f32

Inference

The sampling loop (diffusion + CFG) runs in Python; the LM / encoder / decoder run via ONNX. Reference inference code: AFun9/Omnivoice-onnx (infer_onnx.py). Example:

python infer_onnx.py --variant int8hq --num-step 16 --only demo_voice_clone

Performance (CPU)

RTF ~3–7 depending on the CPU (i9-14900KF ~2.8–6.7; slower CPUs are slower). Suitable for batch / offline use, not real-time. Increase --num-step for higher quality (slower).

License

This export bundles two components under different licenses — you must comply with both:

• Diffusion LM (omnivoice_lm_int8_hq/): derived from Qwen3-0.6B / OmniVoice — Apache-2.0.
• Audio tokenizer (audio_tokenizer_*_int8/): derived from Higgs Audio 2 (Boson AI) — Boson Higgs Audio 2 Community License (based on the Meta Llama 3 Community License); see LICENSE. It includes an acceptable-use policy and Llama-style terms.

Because it contains the Higgs Audio 2 component, the repo license is declared as other. This is an ONNX export + quantization only — weights are unchanged; all rights belong to the original authors (k2-fsa/OmniVoice, Boson AI, Qwen).