Voxtral-Mini-3B-2507-GGUF

GGUF builds of mistralai/Voxtral-Mini-3B-2507 for voxtral.cpp, a ggml-based C++ runtime for Metal / CUDA / Vulkan / CPU.

This is the offline Voxtral model similar to a Whisper-style audio encoder + Ministral-3B decoder that transcribes by generating text tokens and very fast for batch / long-form transcription.

Quants

File	Notes
Q8_0	near-lossless, largest
Q6_K, Q5_K, Q5_1, Q5_0	high quality
Q4_K_M	recommended best size/quality trade-off
Q4_K, Q4_1, Q4_0	smaller
Q3_K	smallest usable (some quality loss)

Q2_K is intentionally not provided as it degrades this model and produces unusable output.

Usage

# build voxtral.cpp, then:
./tools/download_model.sh Q4_K_M --model mini
./build/voxtral --model models/voxtral-3b/Q4_K_M.gguf --audio clip.wav

Audio must be 16 kHz mono WAV. Long audio is transcribed in 30 s windows. GPU is auto-detected; force with --gpu metal|cuda|vulkan|none.

License

Apache 2.0, inherited from the base model.