akashicmarga/Nemotron-Labs-Audex-2B-MLX-4bit

Nemotron-Labs-Audex-2B — MLX (4-bit)

A 4-bit MLX conversion of NVIDIA's Nemotron-Labs-Audex-2B for Apple Silicon. The language model is quantized to 4-bit (group size 64); the speech decoder and audio encoder are kept in bf16 (small and quality-sensitive).

The LM shrinks from 4.5 GB to **1.3 GB** and speech synthesis is roughly 2× faster, with no measurable quality loss on my probes (identical text answers; 0% WER on a Whisper round-trip; full speech-to-speech intact). See the bf16 build for the reference weights.

quant.json in this repo records the quantization, so load_model applies it automatically — no flags needed.

Capabilities (one model)

• Text generation · Text-to-speech · Speech-to-text (ASR) · Speech-to-speech

Usage

Loader/inference code: akashicMarga/mlx-audio-train (models/audex/).

git clone https://github.com/akashicMarga/mlx-audio-train
pip install mlx mlx-lm mlx-whisper soundfile transformers
huggingface-cli download akashicmarga/Nemotron-Labs-Audex-2B-MLX-4bit --local-dir audex_4bit

from models.audex import load_model, s2s_generate
import soundfile as sf

m = load_model("audex_4bit")             # 4-bit applied automatically via quant.json
reply, wav = s2s_generate(m, "question.wav")
sf.write("reply.wav", wav, 16000)
print(reply)

Benchmark (single-prompt probes)

build	LM size	TTS latency	TTS WER (round-trip)	text answer
bf16	4.50 GB	2.5 s	0%	correct
8-bit	2.39 GB	1.5 s	0%	correct
4-bit	1.27 GB	1.2 s	0%	correct

License & attribution

Derived from nvidia/Nemotron-Labs-Audex-2B; NVIDIA OneWay Noncommercial License (non-commercial use only). Model and training by NVIDIA (arXiv:2607.05196); this repo provides only an MLX 4-bit conversion.