---
license: apache-2.0
base_model: mistralai/Voxtral-Mini-3B-2507
pipeline_tag: automatic-speech-recognition
library_name: gguf
tags:
  - voxtral
  - speech-to-text
  - transcription
  - ggml
  - gguf
  - voxtral.cpp
language:
  - en
  - es
  - fr
  - pt
  - hi
  - de
  - nl
  - it
---

# Voxtral-Mini-3B-2507-GGUF

GGUF builds of [mistralai/Voxtral-Mini-3B-2507](https://huggingface.co/mistralai/Voxtral-Mini-3B-2507)
for [voxtral.cpp](https://github.com/andrijdavid/voxtral.cpp), a ggml-based C++
runtime for Metal / CUDA / Vulkan / CPU.

This is the **offline** Voxtral model similar to a Whisper-style audio encoder + Ministral-3B
decoder that transcribes by generating text tokens and very fast for batch / long-form transcription.

## Quants

| File | Notes |
| --- | --- |
| `Q8_0` | near-lossless, largest |
| `Q6_K`, `Q5_K`, `Q5_1`, `Q5_0` | high quality |
| **`Q4_K_M`** | **recommended** best size/quality trade-off |
| `Q4_K`, `Q4_1`, `Q4_0` | smaller |
| `Q3_K` | smallest usable (some quality loss) |

> `Q2_K` is intentionally not provided as it degrades this model and produces unusable output.

## Usage

```bash
# build voxtral.cpp, then:
./tools/download_model.sh Q4_K_M --model mini
./build/voxtral --model models/voxtral-3b/Q4_K_M.gguf --audio clip.wav
```

Audio must be 16 kHz mono WAV. Long audio is transcribed in 30 s windows. GPU is
auto-detected; force with `--gpu metal|cuda|vulkan|none`.

## License

Apache 2.0, inherited from the base model.