---
license: apache-2.0
base_model: XiaomiMiMo/MiMo-V2.5
tags:
- peng
- colibri
- moe
- int4
- cpu-inference
- expert-streaming
---

# MiMo-V2.5 — colibri/peng int4 container

> ✅ **Re-validated** (2026-07-20): `The capital of France is → Paris.` (greedy),
> per-rank guard `verify_mimo_qkv.py` **PASS 5/5 layers, all ranks** against the
> official FP8 checkpoint, and teacher-forcing perplexity **prose 9.97 / code
> 2.07** (code axis = exact match with the peng repo reference, §40).
>
> ⚠️ **Integrity notice (2026-07-16 incident, repaired 2026-07-20):** the
> original upload (2026-07-11) shipped the `qkv_proj` of all 48 layers with
> **TP ranks 1..3 corrupted** (the per-rank fp8 scale grid was applied flat;
> rank 0 intact) — the model produced plausible-looking but broken output and
> the original one-prompt validation did not catch it. Shard
> **`out-00001.safetensors` was re-converted and re-uploaded** on 2026-07-20
> (commit `cb41912`). If you downloaded the container **before that date**, you
> only need to re-download **that single shard** (12.6 GB) and verify with
> `python3 c/tools/verify_mimo_qkv.py --int4 <container> --fp8 <fp8_src>`.
> The zstd variant (`-int4-zstd`) is pending a repack from this repaired
> container — do not use it for quality work until further notice.

Quantized container of **MiMo-V2.5 (311B MoE, 15B active)** for
[**peng**](https://github.com/FiveTechSoft/peng-mimo), the port of colibri's
expert-streaming technology ([colibri](https://github.com/JustVugg/colibri)) to
the `mimo_v2` architecture. It runs the model on a consumer machine (~32 GB RAM)
by reading experts on demand from NVMe.

## Format

- Routed experts (47 layers × 256): packed **int4**, per-row f32 scales (`.qs`)
- Resident dense (fused attention qkv, o_proj, layer-0 MLP): **int8**
- Embeddings and lm_head: **f32**
- Norms, router and sink bias: **f32**
- Source: official FP8 checkpoint (128×128 blocks, **per-rank** scale grid),
  o_proj bf16, exact dequant
- Total size ≈ 152 GB (16 `out-*.safetensors` shards)

## Usage

```bash
git clone https://github.com/FiveTechSoft/peng-mimo && cd peng-mimo/c
make mimo
SNAP=/path/to/this/repo ./mimo 64 4 8        # validation
# interactive chat: see the peng repo README
```

Engine: pure C, zero runtime dependencies, AVX2 (+ optional CUDA backend).
Validated token-exact against `transformers` on a tiny oracle (TF 32/32) and a
396M fixture (TF 20/20); the container is verified identical to runtime
quantization.

## Measured performance (Xeon W-2140B 8c/16t, 32 GB RAM, NVMe 2.75 GB/s, WSL2, RTX 3060)

- Load: ~35 s · resident dense 4.8–9.2 GB (depends on `CUDA_DENSE`) · peak RSS ~17.5 GB
- Cold decode: ~0.3–0.5 tok/s (cold expert cache)
- Warm decode: **0.60–0.89 tok/s** (`TAO=1 SPEED=1`, `COLI_CUDA=1 CUDA_DENSE=1
  CUDA_ATTN=1`, expert hit-rate ~75–80%; session median ~0.75, WSL2 host drift)
- After replacing shards: delete `.coli_usage/.coli_traj/.coli_pathpack` — an
  expert cache learned on other weights routes worse until it re-warms

## Credits and licenses

- peng/colibri engine: Apache 2.0 — colibri is by [JustVugg](https://github.com/JustVugg/colibri); peng is a community derivative
- MiMo-V2.5 weights: © Xiaomi, under the license of the original repo [XiaomiMiMo/MiMo-V2.5](https://huggingface.co/XiaomiMiMo/MiMo-V2.5)
- FP8→int4 conversion: `tools/convert_fp8_to_int4.py --arch mimo` from the peng repo