BarathwajAnandan/cohere-transcribe-03-2026-CoreML-4bit

Cohere Transcribe 03-2026 — CoreML 4-bit (Palettize4)

CoreML conversion of CohereLabs/cohere-transcribe-03-2026 with 4-bit palettization (16 clusters per weight block) and cross-attention K/V pre-computation optimization.

Model Details

Property	Value
Base model	CohereLabs/cohere-transcribe-03-2026
Compute precision	FP16 (activations), 4-bit palettized (weights)
Frontend precision	FP32 (STFT/log-mel)
Total size	~1.0 GB
WER (LibriSpeech test-clean, 500 samples)	2.50%
Speed (GPU only)	~48x realtime
Speed (ANE+GPU async)	~72x realtime
Platform	macOS 13+ / Apple Silicon

Pipeline Components

Package	Size	Role
cohere_frontend.mlpackage	1.5 MB	Audio → log-mel features (FP32)
cohere_encoder.mlpackage	893 MB	Conformer encoder (palettize4)
cohere_cross_kv_projector.mlpackage	8.1 MB	Pre-compute cross-attn K/V once per chunk
cohere_decoder_cached.mlpackage	65 MB	Autoregressive decoder with KV cache
cohere_decoder_fullseq_masked.mlpackage	73 MB	Full-sequence decoder (for validation)

Usage

# Build Swift CLI
cd swift_runner && swift build -c release

# Run (GPU only)
.build/release/pure_coreml_asr_cli \
  --audio input.mp3 \
  --artifacts-dir ./artifacts \
  --compute gpu

# Run (ANE+GPU heterogeneous, ~72x realtime)
.build/release/pure_coreml_asr_cli \
  --audio input.mp3 \
  --artifacts-dir ./artifacts \
  --compute gpu \
  --compute-split ane_small

Variants

Variant	Repo	Size	WER
4-bit (this)	CoreML-4bit	1.0 GB	2.50%
6-bit	CoreML-6bit	2.9 GB	2.43%
FP16	CoreML-fp16	~3.5 GB	baseline

License

Apache 2.0 (following base model license)