---
license: other
base_model: poolside/Laguna-M.1
tags:
- gguf
- llama.cpp
- ik_llama.cpp
- laguna
- imatrix
- poolside
---

# Laguna M.1 GGUF imatrix quants

This repository contains GGUF quantizations of [poolside/Laguna-M.1](https://huggingface.co/poolside/Laguna-M.1).

These GGUFs were produced for the Laguna support work in [ikawrakow/ik_llama.cpp#2003](https://github.com/ikawrakow/ik_llama.cpp/pull/2003), "Add Laguna M.1 GGUF support". At the time of quantization, the support branch commit was `be7d53ceec0c823c07d540e21828d7d187b02d3b`.

Until Laguna support lands in the runners you use, these files should be treated as requiring the linked ik_llama.cpp PR or an equivalent Laguna GGUF implementation. The vanilla-compatible quant names below are chosen so that they should be useful if equivalent Laguna support lands in upstream llama.cpp later.

## Quantization summary

| Quant | Lane | Status | Size | PPL | KL divergence | Notes |
| --- | --- | ---: | ---: | ---: | ---: | --- |
| `IQ2_K_R4` | `ik-spark-core` | finished | 64.35 GiB | 2.6686 +/- 0.04561 | pending | Best tiny ik-specific CUDA fast-path candidate. |
| `IQ3_K_R4` | `ik-spark-core` | finished | 91.81 GiB | 2.4770 +/- 0.04102 | pending | Better quality while still on the ik CUDA fast path. |
| `IQ4_KS_R4` | `ik-spark-core` | finished | 113.20 GiB | 2.4140 +/- 0.03950 | pending | Strong quality while still optimized for ik CUDA. |
| `IQ3_S` | `vanilla-core` | finished | 91.81 GiB | 2.4683 +/- 0.04051 | pending | Vanilla CUDA MMQ-supported imatrix low-bit baseline. |
| `Q4_K_M` | `vanilla-core` | finished | 127.33 GiB | 2.4084 +/- 0.03947 | pending | Broadly compatible CUDA-supported 4-bit-ish baseline. |
| `IQ4_XS` | `vanilla-core` | pending | pending | pending | pending | Vanilla CUDA-supported nonlinear 4-bit-ish candidate. |
| `Q5_K_M` | `vanilla-core` | pending | pending | pending | pending | Higher-quality broadly compatible reference. |
| `Q6_K` | `vanilla-optional` | pending | pending | pending | pending | Optional near-reference quant if pool/upload budget allows. |
| `IQ4_K_R4` | `ik-optional` | pending | pending | pending | pending | Larger/higher-quality ik-specific CUDA fast-path candidate. |
| `IQ5_KS_R4` | `ik-optional` | pending | pending | pending | pending | Optional high-quality ik-specific reference lane. |
| `IQ5_K_R4` | `ik-optional` | pending | pending | pending | pending | Optional larger high-quality ik-specific reference lane. |

## Imatrix and calibration corpus

The imatrix used for these quants is the 256-chunk public run:

- `laguna-m1-q8-public-v1-256-gatefix.imatrix`
- SHA256: `449427a1a4c5d2480d98e3fcccc3884bd3aafa325d937741a162adf84e7e7eed`
- Source model for imatrix collection: corrected Q8_0 GGUF of Laguna M.1
- Chunks: 256
- Tokens processed: 1,048,576
- Final imatrix-run PPL: `2.6084 +/- 0.00786`

This is a mechanically augmented copy of the original 256-chunk imatrix
(`c8a98474542e2ae7636d7ad0578552398eec3d769b6f9a2981f111a4ba06af12`).
The augmentation clones the matching `ffn_up` imatrix entries to `ffn_gate`
entries where Laguna's fused up/gate graph records the shared activation stream
under the up-projection tensor name. This avoids silently quantizing
`ffn_gate_shexp` without imatrix weights.

The calibration corpus is `laguna-m1-imatrix-calibration-v1.txt`:

- Size: 6,526,884 bytes
- Lines: 130,856
- SHA256: `4d478cc0d10da862b0f70040b0c1d948fb88ddac83a5307059e61f02c206622f`

The corpus combines a public imatrix calibration baseline with a deterministic Laguna-specific supplement covering Laguna M.1/XS.2 metadata, GGUF conversion details, CUDA inference, KV-cache quantization, and imatrix collection. The Poolside technical report itself was not copied into the calibration corpus.

## Compatibility notes

- `IQ3_S`, `Q4_K_M`, `IQ4_XS`, `Q5_K_M`, and `Q6_K` are the broad compatibility lane.
- `IQ2_K_R4`, `IQ3_K_R4`, `IQ4_KS_R4`, `IQ4_K_R4`, `IQ5_KS_R4`, and `IQ5_K_R4` are ik_llama.cpp/Spark-oriented quants selected for direct CUDA MMQ fast paths in ik_llama.cpp.
- The `*_R4` quants are not intended as universal llama.cpp artifacts today; use them with ik_llama.cpp or another runner that explicitly supports those tensor types.

## Metrics

The PPL/KL numbers in the table are measured by `llama-perplexity` on a fixed slice of the public calibration corpus. They are intended as reproducibility and regression checks for these artifacts, not as a general benchmark of Laguna M.1.

KL divergence is reported only when a compatible BF16 or Q8_0 reference log-probability file is available for the same token slice. If KL is marked pending, the quant was produced but the high-precision reference pass still needs to be run on a machine large enough to load the reference model comfortably.

Validation tries full GPU offload first and falls back to lower `-ngl` values if
the quant is too large for full offload on the validation machine. Each quant's
`eval-offload.tsv` records which offload setting succeeded for smoke, PPL, and
KL checks.

## Reproducibility

Quantization command shape:

```bash
llama-quantize \
  --keep-split \
  --partial-requant \
  --imatrix laguna-m1-q8-public-v1-256-gatefix.imatrix \
  --output-tensor-type q6_K \
  --token-embedding-type q6_K \
  Laguna-M.1-BF16-00001-of-00010.gguf \
  Laguna-M.1-<QUANT>-imatrix-public-v1.gguf \
  <QUANT> \
  $(nproc)
```

Evaluation command shape:

```bash
llama-perplexity \
  -m Laguna-M.1-<QUANT>-imatrix-public-v1-00001-of-00010.gguf \
  -f laguna-m1-imatrix-calibration-v1.txt \
  -c 4096 \
  --chunks 8 \
  -ngl 99 \
  -b 512 \
  -ub 128 \
  --defer-experts \
  --seed 42
```