---
license: mit
tags:
- coreml
- ANE
- Apple Neural Engine
- Qwen2.5
- VibeThinker
- math
- reasoning
---
# anemll-WeiboAI-VibeThinker-1.5B-ctx2048

CoreML conversion of [WeiboAI/VibeThinker-1.5B](https://huggingface.co/WeiboAI/VibeThinker-1.5B) for Apple Neural Engine inference, converted using [ANEMLL](https://github.com/anemll/anemll) v0.3.5.

## Original Model

**VibeThinker-1.5B** is a 1.5B parameter dense language model by WeiboAI, fine-tuned from Qwen2.5-Math-1.5B for competitive math and algorithm coding. Despite its small size, it achieves remarkable results:

| Benchmark | VibeThinker-1.5B | DeepSeek R1 (671B) |
|-----------|----------------:|-------------------:|
| AIME24    | 80.3            | 79.8               |
| AIME25    | 74.4            | 70.0               |
| HMMT25    | 50.4            | 41.7               |

Best results with English prompts on competitive-style math and coding problems.

## Conversion Details

| Parameter | Value |
|-----------|-------|
| Architecture | Qwen 2.5 |
| Context Length | 2048 |
| Batch Size | 64 |
| Chunks | 4 (hybrid) |
| FFN Quantization | LUT6 (Apple Neural Engine), per-channel group size 4 |
| LM Head Quantization | LUT6 (Apple Neural Engine), per-channel group size 4 |
| Embeddings | Unquantized |
| Argmax in Model | No |

### Hybrid FP32 First Chunk

This model uses a **hybrid precision** layout. Chunk 1 of 4 (`qwen25_FFN_PF_lut6_chunk_01of04.mlmodelc`) runs the first transformer layer's attention in **FP32** (unquantized), while chunks 2-4 use standard LUT6 quantization.

**Why:** VibeThinker has unusually high-magnitude Q/K projection biases in layer 0, causing FP16 attention logit overflow (`+inf`) before masking. This leads to softmax saturation and catastrophic output divergence. Running first-layer attention at full precision eliminates the overflow and recovers near-exact parity with the original Hugging Face model.

Chunk 1 is significantly smaller (~15 MB vs ~310-340 MB for the other chunks) since it only contains the first-layer attention residual path.

### Recommended Sampling

```yaml
temperature: 0.6
top_p: 0.95
top_k: 0
do_sample: true
```

## Quick Start

```bash
pip install coremltools transformers

# Basic chat
python chat.py --meta ./meta.yaml

# Full conversation mode with history
python chat_full.py --meta ./meta.yaml
```

> Note: First load takes time as macOS places the model on the Neural Engine. Subsequent loads are fast.

### iOS/macOS App

Try the ANEMLL Chat app on TestFlight:
1. Install [TestFlight](https://apps.apple.com/app/testflight/id899247664)
2. Join beta: [TestFlight Link](https://testflight.apple.com/join/jrQq1D1C)
3. Add this model via its HuggingFace URL

## Links

- [ANEMLL GitHub](https://github.com/anemll/anemll)
- [ANEMLL Models on HuggingFace](https://huggingface.co/anemll)
- [Original Model: WeiboAI/VibeThinker-1.5B](https://huggingface.co/WeiboAI/VibeThinker-1.5B)

## License

MIT (ANEMLL conversion). The original model is also MIT licensed.