---
license: apache-2.0
base_model: WeiboAI/VibeThinker-1.5B
tags:
- mlx
- nvfp4
- logic
- math
- code
- quantized
- reasoning
- apple-silicon
pipeline_tag: text-generation
---

# VibeThinker-1.5B-MLX-nvfp4

This repository contains the **4-bit NVFP4 quantized weights** for [WeiboAI/VibeThinker-1.5B](https://huggingface.co/WeiboAI/VibeThinker-1.5B), optimized for low-latency, edge-based deployment on Apple Silicon hardware using the `oMLX` framework.

`VibeThinker-1.5B` is a dense Transformer reasoning model created by Sina Weibo Inc. It is engineered to challenge traditional scaling laws using the *Spectrum-to-Signal Principle (SSP)*—combining Two-Stage Diversity-Exploring Distillation with MaxEnt-Guided Policy Optimization (MGPO) to extract deep math and coding capabilities from a tiny 1.5B parameter core.

---

## ⚡ Inference Generation Breakthroughs (vs. VibeThinker-3B-nvfp4)

When benchmarked on Apple Silicon via the [oMLX inference engine](https://github.com/jundot/omlx), this ultra-compact 1.5B parameter NVFP4 quantization delivers staggering speedups and resource savings compared directly to its 3B NVFP4 sibling.

### Core Efficiency Multipliers:
* **🏎️ Speed Jump (Token Generation):** Output velocity increases by **+84.6%** in standard generation, skyrocketing to **454.2 tok/s** (compared to the 3B variant's 246.0 tok/s). 
* **📉 Massive VRAM Savings:** Reduces peak VRAM footprint by **-38.6%**, requiring a mere **1.51 GB** of memory (vs. 2.46 GB for the 3B model), making it trivial to run on base-tier Mac hardware.
* **⚡ Prefill Processing Acceleration:** The prompt prefill rate surges by **+30.3%** under standard context lengths (`pp TPS` climbs from 3,659 tok/s to **4,768.5 tok/s**). Under massive 4k context limits, prefill speeds leap by **+60.3%** to hit **10,039.5 tok/s**.
* **🚀 Concurrent Scaling (4x Batching):** Under continuous multi-request batching, token throughput pushes forward to an incredible **773.5 tok/s**—outperforming the 3B batched configuration by **+68.5%**.
* **⏱️ Near-Instant Turnaround:** Total end-to-end processing latency drops by **-37.9%**, fulfilling a full reasoning response cycle in just **0.497 seconds**.

---

## 🛠️ Deployment & Execution Quickstart

To run this model, use an inference engine configured to process the optimized `nvfp4` memory layout natively on Mac (such as `oMLX`).

### Example running with `oMLX`

```bash
# Execute local evaluation benches natively using the optimized Auto engine pipeline:
omlx bench --model your-hf-username/VibeThinker-1.5B-MLX-nvfp4 --prompt "Integrate x^2 ln(x) dx step by step."
```
<img src="/bkideas/Qwen2.5-Coder-3B-MLX-nvfp4/resolve/main/benchmark.svg" alt="Benchmark table" width="100%" />