---
license: cc-by-4.0
language:
- en
library_name: mlx
tags:
- automatic-speech-recognition
- parakeet
- fastconformer
- rnnt
- mlx
- apple-silicon
- quantized
- int4
base_model: nvidia/parakeet-unified-en-0.6b
pipeline_tag: automatic-speech-recognition
---

# parakeet-unified-en-0.6b-mlx-int4

4-bit affine-quantized MLX weights for NVIDIA's
[`parakeet-unified-en-0.6b`](https://huggingface.co/nvidia/parakeet-unified-en-0.6b)
Cache-Aware FastConformer-RNNT, for the witness MLX C++ engine on Apple Silicon.

Same quantization scope as the int8 variant (linear/projection matmuls only,
group size 64; conv2d / depthwise / norms / biases / position bias stay dense).

## When to use int4 vs int8

int4 is the **smallest** variant (0.39 GB) and is **WER-neutral for the offline
(full-context) path**, so it is a good fit for memory-constrained offline
transcription. For the **streaming** (cache-aware chunked) path it shows a small
WER regression vs int8/dense — prefer
[`-int8`](https://huggingface.co/littlebearlabs/parakeet-unified-en-0.6b-mlx-int8)
for streaming. The int4 RTF is within noise of int8, so the trade is memory, not
speed.

## Measured (M4, 45 LibriSpeech samples / 300s, witness rtf_bench)

| Variant | Size | Offline WER | Offline RTF | Streaming WER | Streaming RTF |
|---|---|---|---|---|---|
| dense fp32 | 2.47 GB | 1.78% | 0.0084 | 11.35% | 0.0319 |
| int8 | 0.70 GB | 1.78% | 0.0075 | 11.35% | 0.0197 |
| **int4** | **0.39 GB** | **1.78%** | **~0.0080** | **12.68%** | **0.0206** |

## Use

```
WITNESS_PARAKEET_UNIFIED_MODEL_DIR=/path/to/this/dir
```

Produced by `crates/mlx-parakeet/scripts/quantize_parakeet_unified.py --bits 4`.