parakeet-unified-en-0.6b-mlx-int4

4-bit affine-quantized MLX weights for NVIDIA's parakeet-unified-en-0.6b Cache-Aware FastConformer-RNNT, for the witness MLX C++ engine on Apple Silicon.

Same quantization scope as the int8 variant (linear/projection matmuls only, group size 64; conv2d / depthwise / norms / biases / position bias stay dense).

When to use int4 vs int8

int4 is the smallest variant (0.39 GB) and is WER-neutral for the offline (full-context) path, so it is a good fit for memory-constrained offline transcription. For the streaming (cache-aware chunked) path it shows a small WER regression vs int8/dense — prefer -int8 for streaming. The int4 RTF is within noise of int8, so the trade is memory, not speed.

Measured (M4, 45 LibriSpeech samples / 300s, witness rtf_bench)

Variant	Size	Offline WER	Offline RTF	Streaming WER	Streaming RTF
dense fp32	2.47 GB	1.78%	0.0084	11.35%	0.0319
int8	0.70 GB	1.78%	0.0075	11.35%	0.0197
int4	0.39 GB	1.78%	~0.0080	12.68%	0.0206

Use

WITNESS_PARAKEET_UNIFIED_MODEL_DIR=/path/to/this/dir

Produced by crates/mlx-parakeet/scripts/quantize_parakeet_unified.py --bits 4.