---
license: gpl-3.0
base_model: CohereLabs/cohere-transcribe-03-2026
tags:
  - automatic-speech-recognition
  - mlx
  - speech-recognition
  - transcription
  - audio
  - apple
  - macos
  - on-device
  - quantized
  - mixed-precision
  - 3bit
  - 4bit
language:
  - en
  - fr
  - de
  - es
  - it
  - pt
  - nl
  - ja
  - ko
  - zh
  - ar
  - hi
  - ru
  - pl
pipeline_tag: automatic-speech-recognition
library_name: mlx
---

# Cohere Transcribe 03-2026 — MLX Mixed 3-bit/4-bit

The most aggressive MLX quantization of [CohereLabs/cohere-transcribe-03-2026](https://huggingface.co/CohereLabs/cohere-transcribe-03-2026) that still produces correct transcripts. Encoder at 3-bit, decoder at 4-bit. Runs entirely on-device via Apple MLX on Apple Silicon.

## Key Metrics

| Metric | Value |
|--------|-------|
| **Size** | 891 MB (vs 3.9 GB FP16 — 4.4x smaller) |
| **WER (LibriSpeech test-clean)** | 1.07% |
| **WER (LibriSpeech test-other)** | 2.17% |
| **Composite WER** | 1.62% |
| **RTFx (M4 Air)** | 23.9x real-time |
| **Effective bits/param** | ~3.25 |

## Compression Details

| Component | Quantization |
|-----------|-------------|
| Encoder | 3-bit linear (per-group scale, group size 64) |
| Decoder | 4-bit affine (per-group scale, group size 64) |
| Format | MLX safetensors (`model.safetensors`) |

1x1 Conv1d layers are converted to Linear equivalents to enable quantization of convolutional layers.

## Architecture

- **Base model**: Cohere Transcribe 03-2026 (~2B params)
- **Encoder**: FastConformer (48 layers, d=1280)
- **Decoder**: Transformer (8 layers, d=1024)
- **Tokenizer**: SentencePiece (16,384 tokens)

## Usage

Requires [mlx-audio](https://github.com/Blaizzy/mlx-audio) installed from git main:

```bash
pip install "mlx-audio[stt] @ git+https://github.com/Blaizzy/mlx-audio.git"
```

```python
from mlx_audio.stt import load

model, processor = load("MarkChen1214/cohere-transcribe-03-2026-MLX-Mixed-3bit4bit")
result = model.generate(audio="audio.wav")
print(result["text"])
```

**Note:** Requires the quantization patch (`--apply-patch` with `mlx_audio_cohere_quant_patch.py`) when using the mlx-audio CLI.

## Eval Results (Full LibriSpeech)

| Dataset | Samples | Audio Hours | WER | RTFx |
|---------|---------|-------------|-----|------|
| LibriSpeech test-clean | 2,620 | 5.4h | 1.07% | 25.2x |
| LibriSpeech test-other | 2,939 | 5.34h | 2.17% | 22.8x |

## License

GPL-3.0 — see [LICENSE](LICENSE).

The base model ([CohereLabs/cohere-transcribe-03-2026](https://huggingface.co/CohereLabs/cohere-transcribe-03-2026)) is Apache 2.0.