cstr/higgs-audio-v3-stt-GGUF

higgs-audio-v3-stt — GGUF

GGUF conversions of bosonai/higgs-audio-v3-stt for CrispASR.

higgs-audio-v3-stt is a speech-to-text model: a Whisper-large-v3 audio encoder → depthwise-temporal-conv projector → Qwen3-1.7B decoder. A <|AUDIO|> placeholder in a ChatML prompt is replaced by the projected audio embeddings and the LLM autoregressively decodes the transcript.

Files

File	Quant	Size	Notes
higgs-stt-f16.gguf	F16	5.0 GB	full precision
higgs-stt-q8_0.gguf	Q8_0	3.2 GB	near-lossless; recommended
higgs-stt-q4_k.gguf	Q4_K	2.3 GB	smallest; transcribes verbatim on test clips

All three transcribe the canonical JFK clip and a 45 s multi-chunk clip verbatim, matching the bf16 reference. The tied input/output embeddings (token_embd.weight / output.weight, the lm_head) are kept at F16 in every quant — quantizing them perturbs every logit.

Usage (CrispASR)

# auto-download:
crispasr -m auto --backend higgs-stt audio.wav
# or an explicit file:
crispasr -m higgs-stt-q8_0.gguf --backend higgs-stt audio.wav

Architecture note (chunked encoder)

higgs-audio does not encode audio as a single padded 30 s Whisper window. It splits the waveform into chunk_size_seconds (4 s) chunks, encodes each chunk independently (chunk-local positional embeddings, within-chunk attention) through the Whisper tower + projector, then concatenates the per-chunk audio embeddings. The decoder then attends over the whole concatenated sequence in a single pass, so long audio is handled without cross-chunk context loss. Encoding one global window instead corrupts the conditioning (every valid frame attends to the silence padding) and derails the decoder. This is the key detail for any reimplementation.

License

Apache-2.0, inherited from the base model bosonai/higgs-audio-v3-stt.