---
library_name: mlx
license: other
license_name: nvidia-nemotron-open-model-license
license_link: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-nemotron-open-model-license/
pipeline_tag: text-generation
language:
- en
- fr
- es
- it
- de
- ja
- zh
tags:
- nvidia
- nemotron-3
- mlx
- quantized
- 2bit
- mixed-precision
- optiq
- static
- moe
- ssd-streaming
- apple-silicon
- text-generation
base_model: nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16
---

# mlx-community/NVIDIA-Nemotron-3-Super-120B-A12B-OptiQ-2bit

> **Built with [mlx-optiq](https://mlx-optiq.com)**, the MLX-native toolkit to quantize, fine-tune, and serve LLMs locally on Apple Silicon, no PyTorch and no cloud. [Read the write-up](https://mlx-optiq.com/blog/stream-122b-on-a-mac) · [All OptiQ quants](https://mlx-optiq.com/models) · [Docs](https://mlx-optiq.com/docs/)

**A 120-billion-parameter model that runs on a 36 GB Mac.** This is a 2-bit mixed-precision MLX quant of NVIDIA's Nemotron-3-Super-120B-A12B (247 GB at bf16), produced by [mlx-optiq](https://mlx-optiq.com/). It is 47.5 GB on disk. While it generates, only ~14 GB sits in RAM: the Mamba blocks, attention, router and shared experts stay resident, and the 34 GB of routed mixture-of-experts weights stream off the SSD as the router selects them, through `optiq serve --stream-experts`.

Nemotron-3-Super is a hybrid: Mamba2 state-space blocks interleaved with attention and a 512-expert sparse MoE (22 active per token). Asked to write Flappy Bird in a single HTML file, the 2-bit model produced a complete, working game. Here it is playing it:

![Nemotron-3-Super-120B-A12B 2-bit playing the Flappy Bird it wrote, on a 36 GB Mac](flappy.gif)

## What it is

| Property | Value |
|---|---|
| Base | NVIDIA-Nemotron-3-Super-120B-A12B (hybrid Mamba2 + attention + 512-expert MoE, 22 active) |
| Method | OptiQ `static` — structural per-layer bit allocation, no calibration |
| Bit-widths | 4-bit on Mamba / attention / router / shared experts / edges, 2-bit on the routed experts |
| Achieved bits-per-weight | 2.52 |
| On disk | 47.5 GB |
| Resident while running | ~14 GB (routed experts streamed) |
| Decode speed | ~3 tok/s on an M3 Max (36 GB) |

For a model this large, exact calibration-driven sensitivity is impractical (it would run for days and needs the full model resident as a reference), so OptiQ's `static` method assigns bits from architecture alone. See the [methods comparison](https://mlx-optiq.com/docs/sensitivity).

## Run it

This is a Nemotron hybrid (`model_type: nemotron_h`), so it needs **mlx-lm from main** and **`import optiq`** (install from git, not a version pin):

```bash
pip install -U mlx-optiq "mlx-lm @ git+https://github.com/ml-explore/mlx-lm.git"
```

Serve it with SSD expert streaming (auto-enabled for a MoE too big to fit resident; `--stream-experts` forces it):

```bash
optiq serve --model mlx-community/NVIDIA-Nemotron-3-Super-120B-A12B-OptiQ-2bit --stream-experts
```

Then open the Lab, ask for a game, and watch it render in the Canvas pane. Only the routed experts stream per token; the Mamba state, attention and shared experts stay resident, so the footprint stays ~14 GB no matter how large the model on disk is.

## Notes

This is an **extreme quant**. 2-bit on the routed experts is lossy, and the point of this artifact is that a 120 B hybrid MoE runs at all on consumer Apple Silicon, with coherent output. For reference quality on this base, use the bf16 weights or a higher-bit quant. The full story is in the [blog post](https://mlx-optiq.com/blog/stream-122b-on-a-mac).