mxfp8 weights produce all-NaN logits — re-quant from BF16 may be needed
Heads-up that this checkpoint generates <unk> for every token on any input. Sibling quants on the same model (4-bit / 5-bit / 6-bit affine) work fine, so the mxfp8 build looks like it needs re-uploading.
In layer 1, shared_experts.up_proj returns inf even on tiny inputs — the dequantized weights are overflowing. Scanning all 162 E8M0 scale tensors, 30 contain values ≥ 200 (max 255). That decodes to 2^128 ≈ 3.4e38, past fp16's 65,504 limit; anything ≥ 144 overflows.
Best guess: the conversion path (mlx-vlm 0.4.5 → nn.quantize) doesn't clamp outlier scales when assigning E8M0, so a single large-magnitude weight in a 32-element group pushes the scale to 255. A re-quant from nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16 with current mlx-vlm may fix it.
Workaround in the meantime: the -4bit sibling runs end-to-end.
Environment: mlx-vlm 0.5.0, mlx-lm 0.31.3, mlx 0.31.2, transformers 5.5.4, Python 3.12, Apple Silicon.