| --- |
| license: openmdw-1.1 |
| library_name: vllm |
| pipeline_tag: text-generation |
| base_model: poolside/Laguna-S-2.1 |
| base_model_relation: quantized |
| tags: |
| - laguna |
| - laguna-s-2.1 |
| - quantized |
| - mixture-of-experts |
| - nvfp4 |
| - exl3 |
| - trellis3 |
| - hybrid-quantization |
| - custom_code |
| --- |
| |
| # Laguna S 2.1 Hybrid 3.25bpw |
|
|
| > **Verified package.** This package was materialized only after the weight |
| > transform, compact-package load, and hybrid-serving validation gates passed. |
| > The included release manifest seals the source revision, tier map, runtime |
| > overlay, validation evidence, and every packaged artifact hash. |
|
|
| Laguna S 2.1 Hybrid 3.25bpw is a two-tier, expert-level quantization of |
| [`poolside/Laguna-S-2.1-NVFP4`](https://huggingface.co/poolside/Laguna-S-2.1-NVFP4). |
| It preserves every expert and the source router topology. REAP saliency is used |
| only to choose the NVFP4 hot tier; it is not expert pruning. |
|
|
| ## Quantization layout |
|
|
| Each of the 47 sparse MoE layers has 256 routed experts: |
|
|
| | Tier | Experts per sparse layer | Storage | |
| |---|---:|---| |
| | Hot | 64 | source NVFP4, retained byte-exact | |
| | Tail | 192 | calibrated EXL3 Trellis-3 | |
|
|
| This is a nominal 3.25-bit mixture across the expert weights. Shared experts, |
| the router, attention, embeddings, LM head, and other non-expert tensors remain |
| in their source representation. All 256 experts remain routable. |
|
|
| ## How the hot tier was selected |
|
|
| The hot experts are ranked independently in each sparse layer using router- |
| weighted expert-output activation saliency over a held-out calibration mixture |
| covering coding, tool and agentic calls, reasoning, mathematics, science, |
| terminal work, CUDA, cybersecurity, and long-context prompts. The score is the |
| mean, over tokens routed to an expert, of the applied router weight multiplied |
| by that expert's output L2 norm. This release includes the exact tier map, |
| source revision, calibration provenance, and artifact hashes. |
|
|
| ## Runtime status and compatibility |
|
|
| The hybrid requires the accompanying EXL3-tail runtime. It is not a drop-in |
| stock-vLLM checkpoint. The runtime supports two deployment layouts: |
|
|
| - single-Spark TP1, which reconstructs each calibrated tail expert from both |
| stored artifact slices on one device; |
| - two-Spark TP2, which loads one stored artifact slice per tensor-parallel |
| rank. |
|
|
| Both layouts retain normal compiled execution and CUDA graphs. The package |
| includes the pinned runtime overlay and exact launch recipes. The standalone |
| 53 GB package passed a fresh compact-package load on one DGX Spark with 200k |
| context, Poolside reasoning and tool parsers, structured output support, and |
| full/piecewise CUDA graph capture. Its sealed API suite covers reasoning, |
| parsed tool calls, tool history, JSON schema, exact ASCII output, and a |
| 194,978-token long-context marker-recovery request. The measured stable |
| single-Spark throughput sweep selected concurrency 4 at 58.20 aggregate |
| completion tokens/s. The source checkpoint contains no native MTP tensors; |
| the release therefore does not claim native-MTP throughput. |
|
|
| Do not infer benchmark quality from this card. At the time this package was |
| published, no Terminal-Bench 2.1 result was claimed. |
|
|
| ## Provenance |
|
|
| - Base model: `poolside/Laguna-S-2.1-NVFP4` |
| - Base-model license: [OpenMDW-1.1](https://huggingface.co/poolside/Laguna-S-2.1-NVFP4/blob/main/LICENSE.md) |
| - Hybrid method: source NVFP4 hot experts plus calibrated EXL3 Trellis-3 tails |
| - REAP reference: [REAP the Experts](https://arxiv.org/abs/2510.13999) |
|
|
| ## Responsible use |
|
|
| This derivative remains subject to the upstream OpenMDW-1.1 license and should |
| be used consistently with the upstream model's acceptable-use guidance. |
|
|
