Add pre-release model card
Browse files
README.md
ADDED
|
@@ -0,0 +1,73 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: openmdw-1.1
|
| 3 |
+
library_name: vllm
|
| 4 |
+
pipeline_tag: text-generation
|
| 5 |
+
base_model:
|
| 6 |
+
- poolside/Laguna-S-2.1-NVFP4
|
| 7 |
+
tags:
|
| 8 |
+
- laguna
|
| 9 |
+
- laguna-s-2.1
|
| 10 |
+
- mixture-of-experts
|
| 11 |
+
- nvfp4
|
| 12 |
+
- exl3
|
| 13 |
+
- trellis3
|
| 14 |
+
- hybrid-quantization
|
| 15 |
+
- custom_code
|
| 16 |
+
---
|
| 17 |
+
|
| 18 |
+
# Laguna S 2.1 Hybrid 3.25bpw
|
| 19 |
+
|
| 20 |
+
> **Pre-release build record.** This repository is intentionally private while
|
| 21 |
+
> the weight transform and serving validation are in progress. Weight files are
|
| 22 |
+
> not claimed to be available until the release manifest and validation evidence
|
| 23 |
+
> are uploaded with them.
|
| 24 |
+
|
| 25 |
+
Laguna S 2.1 Hybrid 3.25bpw is a two-tier, expert-level quantization of
|
| 26 |
+
[`poolside/Laguna-S-2.1-NVFP4`](https://huggingface.co/poolside/Laguna-S-2.1-NVFP4).
|
| 27 |
+
It preserves every expert and the source router topology. REAP saliency is used
|
| 28 |
+
only to choose the NVFP4 hot tier; it is not expert pruning.
|
| 29 |
+
|
| 30 |
+
## Quantization layout
|
| 31 |
+
|
| 32 |
+
Each of the 47 sparse MoE layers has 256 routed experts:
|
| 33 |
+
|
| 34 |
+
| Tier | Experts per sparse layer | Storage |
|
| 35 |
+
|---|---:|---|
|
| 36 |
+
| Hot | 64 | source NVFP4, retained byte-exact |
|
| 37 |
+
| Tail | 192 | calibrated EXL3 Trellis-3 |
|
| 38 |
+
|
| 39 |
+
This is a nominal 3.25-bit mixture across the expert weights. Shared experts,
|
| 40 |
+
the router, attention, embeddings, LM head, and other non-expert tensors remain
|
| 41 |
+
in their source representation. All 256 experts remain routable.
|
| 42 |
+
|
| 43 |
+
## How the hot tier was selected
|
| 44 |
+
|
| 45 |
+
The hot experts are ranked independently in each sparse layer using router-
|
| 46 |
+
weighted expert-output activation saliency over a held-out calibration mixture
|
| 47 |
+
covering coding, tool and agentic calls, reasoning, mathematics, science,
|
| 48 |
+
terminal work, CUDA, cybersecurity, and long-context prompts. The score is the
|
| 49 |
+
mean, over tokens routed to an expert, of the applied router weight multiplied
|
| 50 |
+
by that expert's output L2 norm. The published release will include the exact
|
| 51 |
+
tier map, source revision, calibration provenance, and artifact hashes.
|
| 52 |
+
|
| 53 |
+
## Runtime status and compatibility
|
| 54 |
+
|
| 55 |
+
The hybrid requires the accompanying TP2 EXL3-tail runtime. It is not yet a
|
| 56 |
+
drop-in stock-vLLM checkpoint. The release will include a pinned runtime overlay
|
| 57 |
+
and an exact launch recipe only after the package passes load, reasoning,
|
| 58 |
+
structured-output, tool-call, long-context, and throughput validation.
|
| 59 |
+
|
| 60 |
+
Do not infer benchmark quality from this card. At the time this draft was
|
| 61 |
+
created, no release benchmark result is claimed.
|
| 62 |
+
|
| 63 |
+
## Provenance
|
| 64 |
+
|
| 65 |
+
- Base model: `poolside/Laguna-S-2.1-NVFP4`
|
| 66 |
+
- Base-model license: [OpenMDW-1.1](https://huggingface.co/poolside/Laguna-S-2.1-NVFP4/blob/main/LICENSE.md)
|
| 67 |
+
- Hybrid method: source NVFP4 hot experts plus calibrated EXL3 Trellis-3 tails
|
| 68 |
+
- REAP reference: [REAP the Experts](https://arxiv.org/abs/2510.13999)
|
| 69 |
+
|
| 70 |
+
## Responsible use
|
| 71 |
+
|
| 72 |
+
This derivative remains subject to the upstream OpenMDW-1.1 license and should
|
| 73 |
+
be used consistently with the upstream model's acceptable-use guidance.
|