| --- |
| license: apache-2.0 |
| base_model: WeiboAI/VibeThinker-1.5B |
| tags: |
| - mlx |
| - nvfp4 |
| - logic |
| - math |
| - code |
| - quantized |
| - reasoning |
| - apple-silicon |
| pipeline_tag: text-generation |
| --- |
| |
| # VibeThinker-1.5B-MLX-nvfp4 |
|
|
| This repository contains the **4-bit NVFP4 quantized weights** for [WeiboAI/VibeThinker-1.5B](https://huggingface.co/WeiboAI/VibeThinker-1.5B), optimized for low-latency, edge-based deployment on Apple Silicon hardware using the `oMLX` framework. |
|
|
| `VibeThinker-1.5B` is a dense Transformer reasoning model created by Sina Weibo Inc. It is engineered to challenge traditional scaling laws using the *Spectrum-to-Signal Principle (SSP)*—combining Two-Stage Diversity-Exploring Distillation with MaxEnt-Guided Policy Optimization (MGPO) to extract deep math and coding capabilities from a tiny 1.5B parameter core. |
|
|
| --- |
|
|
| ## ⚡ Inference Generation Breakthroughs (vs. VibeThinker-3B-nvfp4) |
|
|
| When benchmarked on Apple Silicon via the [oMLX inference engine](https://github.com/jundot/omlx), this ultra-compact 1.5B parameter NVFP4 quantization delivers staggering speedups and resource savings compared directly to its 3B NVFP4 sibling. |
|
|
| ### Core Efficiency Multipliers: |
| * **🏎️ Speed Jump (Token Generation):** Output velocity increases by **+84.6%** in standard generation, skyrocketing to **454.2 tok/s** (compared to the 3B variant's 246.0 tok/s). |
| * **📉 Massive VRAM Savings:** Reduces peak VRAM footprint by **-38.6%**, requiring a mere **1.51 GB** of memory (vs. 2.46 GB for the 3B model), making it trivial to run on base-tier Mac hardware. |
| * **⚡ Prefill Processing Acceleration:** The prompt prefill rate surges by **+30.3%** under standard context lengths (`pp TPS` climbs from 3,659 tok/s to **4,768.5 tok/s**). Under massive 4k context limits, prefill speeds leap by **+60.3%** to hit **10,039.5 tok/s**. |
| * **🚀 Concurrent Scaling (4x Batching):** Under continuous multi-request batching, token throughput pushes forward to an incredible **773.5 tok/s**—outperforming the 3B batched configuration by **+68.5%**. |
| * **⏱️ Near-Instant Turnaround:** Total end-to-end processing latency drops by **-37.9%**, fulfilling a full reasoning response cycle in just **0.497 seconds**. |
|
|
| --- |
|
|
| ## 🛠️ Deployment & Execution Quickstart |
|
|
| To run this model, use an inference engine configured to process the optimized `nvfp4` memory layout natively on Mac (such as `oMLX`). |
|
|
| ### Example running with `oMLX` |
|
|
| ```bash |
| # Execute local evaluation benches natively using the optimized Auto engine pipeline: |
| omlx bench --model your-hf-username/VibeThinker-1.5B-MLX-nvfp4 --prompt "Integrate x^2 ln(x) dx step by step." |
| ``` |
| <img src="/bkideas/Qwen2.5-Coder-3B-MLX-nvfp4/resolve/main/benchmark.svg" alt="Benchmark table" width="100%" /> |
