metadata
license: gpl-3.0
base_model: CohereLabs/cohere-transcribe-03-2026
tags:
- automatic-speech-recognition
- mlx
- speech-recognition
- transcription
- audio
- apple
- macos
- on-device
- quantized
- mixed-precision
- 3bit
- 4bit
language:
- en
- fr
- de
- es
- it
- pt
- nl
- ja
- ko
- zh
- ar
- hi
- ru
- pl
pipeline_tag: automatic-speech-recognition
library_name: mlx
Cohere Transcribe 03-2026 — MLX Mixed 3-bit/4-bit
The most aggressive MLX quantization of CohereLabs/cohere-transcribe-03-2026 that still produces correct transcripts. Encoder at 3-bit, decoder at 4-bit. Runs entirely on-device via Apple MLX on Apple Silicon.
Key Metrics
| Metric | Value |
|---|---|
| Size | 891 MB (vs 3.9 GB FP16 — 4.4x smaller) |
| WER (LibriSpeech test-clean) | 1.07% |
| WER (LibriSpeech test-other) | 2.17% |
| Composite WER | 1.62% |
| RTFx (M4 Air) | 23.9x real-time |
| Effective bits/param | ~3.25 |
Compression Details
| Component | Quantization |
|---|---|
| Encoder | 3-bit linear (per-group scale, group size 64) |
| Decoder | 4-bit affine (per-group scale, group size 64) |
| Format | MLX safetensors (model.safetensors) |
1x1 Conv1d layers are converted to Linear equivalents to enable quantization of convolutional layers.
Architecture
- Base model: Cohere Transcribe 03-2026 (~2B params)
- Encoder: FastConformer (48 layers, d=1280)
- Decoder: Transformer (8 layers, d=1024)
- Tokenizer: SentencePiece (16,384 tokens)
Usage
Requires mlx-audio installed from git main:
pip install "mlx-audio[stt] @ git+https://github.com/Blaizzy/mlx-audio.git"
from mlx_audio.stt import load
model, processor = load("MarkChen1214/cohere-transcribe-03-2026-MLX-Mixed-3bit4bit")
result = model.generate(audio="audio.wav")
print(result["text"])
Note: Requires the quantization patch (--apply-patch with mlx_audio_cohere_quant_patch.py) when using the mlx-audio CLI.
Eval Results (Full LibriSpeech)
| Dataset | Samples | Audio Hours | WER | RTFx |
|---|---|---|---|---|
| LibriSpeech test-clean | 2,620 | 5.4h | 1.07% | 25.2x |
| LibriSpeech test-other | 2,939 | 5.34h | 2.17% | 22.8x |
License
GPL-3.0 — see LICENSE.
The base model (CohereLabs/cohere-transcribe-03-2026) is Apache 2.0.