---
base_model:
- XiaomiMiMo/MiMo-V2.5-Pro
---
## Notes
- 05/07/26: Unless some *really* asks for the Q8_0, I'm not planning on uploading it due to HF storage space constraints. Bartowski / Unsloth should have the Q8_0 quant when they get to this model.

## Model
This repo contains specialized MoE-quants for MiMo-V2.5-Pro. The idea being that given the huge size of the FFN tensors compared to the rest of the tensors in the model, it should be possible to achieve a better quality while keeping the overall size of the entire model smaller compared to a similar naive quantization. To that end, the quantization type default is kept in high quality and the FFN UP + FFN GATE tensors are quanted down along with the FFN DOWN tensors.

| Quant  | Size                   | Mixture                        | PPL                 | 1-(Mean PPL(Q)/PPL(base)) | KLD                 |
| :----- | :--------------------- | :----------------------------- | :------------------ | :------------------------ | :------------------ |
| Q8_0   | 1012.92 GiB (8.50 BPW) | Q8_0                           | 3.194827 ± 0.016728 | +0%                       | 0                   |
| Q5_K_M | 704.84 GiB (5.92 BPW)  | Q8_0 / Q5_K / Q5_K / Q6_K      | 3.195406 ± 0.016711 | +0.0698%                  | 0.036850 ± 0.000815 |
| Q4_K_M | 585.98 GiB (4.92 BPW)  | Q8_0 / Q4_K / Q4_K / Q5_K      | 3.220761 ± 0.016918 | +0.8638%                  | 0.055113 ± 0.000954 |
| IQ4_XS | 454.99 GiB (3.82 BPW)  | Q8_0 / IQ3_S / IQ3_S / IQ4_XS  | 3.317572 ± 0.017483 | +3.8956%                  | 0.098682 ± 0.001013 |
| IQ3_S  | 350.82 GiB (2.95 BPW)  | Q6_K / IQ2_S / IQ2_S / IQ3_S   | 3.644244 ± 0.019594 | +14.1259%                 | 0.207702 ± 0.001453 |
| IQ2_S  | 297.45 GiB (2.50 BPW)  | Q6_K / IQ2_XS / IQ2_XS / IQ2_S | 4.068630 ± 0.022989 | +27.4163%                 | 0.318892 ± 0.001943 |

![kld_graph](kld_data/01_kld_vs_filesize.png "Chart showing Pareto KLD analysis of quants")
![ppl_graph](kld_data/02_ppl_vs_filesize.png "Chart showing Pareto PPL analysis of quants")