0xSero commited on
Commit
32e3e16
·
verified ·
1 Parent(s): 036fb41

Add pre-release model card

Browse files
Files changed (1) hide show
  1. README.md +73 -0
README.md ADDED
@@ -0,0 +1,73 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: openmdw-1.1
3
+ library_name: vllm
4
+ pipeline_tag: text-generation
5
+ base_model:
6
+ - poolside/Laguna-S-2.1-NVFP4
7
+ tags:
8
+ - laguna
9
+ - laguna-s-2.1
10
+ - mixture-of-experts
11
+ - nvfp4
12
+ - exl3
13
+ - trellis3
14
+ - hybrid-quantization
15
+ - custom_code
16
+ ---
17
+
18
+ # Laguna S 2.1 Hybrid 3.25bpw
19
+
20
+ > **Pre-release build record.** This repository is intentionally private while
21
+ > the weight transform and serving validation are in progress. Weight files are
22
+ > not claimed to be available until the release manifest and validation evidence
23
+ > are uploaded with them.
24
+
25
+ Laguna S 2.1 Hybrid 3.25bpw is a two-tier, expert-level quantization of
26
+ [`poolside/Laguna-S-2.1-NVFP4`](https://huggingface.co/poolside/Laguna-S-2.1-NVFP4).
27
+ It preserves every expert and the source router topology. REAP saliency is used
28
+ only to choose the NVFP4 hot tier; it is not expert pruning.
29
+
30
+ ## Quantization layout
31
+
32
+ Each of the 47 sparse MoE layers has 256 routed experts:
33
+
34
+ | Tier | Experts per sparse layer | Storage |
35
+ |---|---:|---|
36
+ | Hot | 64 | source NVFP4, retained byte-exact |
37
+ | Tail | 192 | calibrated EXL3 Trellis-3 |
38
+
39
+ This is a nominal 3.25-bit mixture across the expert weights. Shared experts,
40
+ the router, attention, embeddings, LM head, and other non-expert tensors remain
41
+ in their source representation. All 256 experts remain routable.
42
+
43
+ ## How the hot tier was selected
44
+
45
+ The hot experts are ranked independently in each sparse layer using router-
46
+ weighted expert-output activation saliency over a held-out calibration mixture
47
+ covering coding, tool and agentic calls, reasoning, mathematics, science,
48
+ terminal work, CUDA, cybersecurity, and long-context prompts. The score is the
49
+ mean, over tokens routed to an expert, of the applied router weight multiplied
50
+ by that expert's output L2 norm. The published release will include the exact
51
+ tier map, source revision, calibration provenance, and artifact hashes.
52
+
53
+ ## Runtime status and compatibility
54
+
55
+ The hybrid requires the accompanying TP2 EXL3-tail runtime. It is not yet a
56
+ drop-in stock-vLLM checkpoint. The release will include a pinned runtime overlay
57
+ and an exact launch recipe only after the package passes load, reasoning,
58
+ structured-output, tool-call, long-context, and throughput validation.
59
+
60
+ Do not infer benchmark quality from this card. At the time this draft was
61
+ created, no release benchmark result is claimed.
62
+
63
+ ## Provenance
64
+
65
+ - Base model: `poolside/Laguna-S-2.1-NVFP4`
66
+ - Base-model license: [OpenMDW-1.1](https://huggingface.co/poolside/Laguna-S-2.1-NVFP4/blob/main/LICENSE.md)
67
+ - Hybrid method: source NVFP4 hot experts plus calibrated EXL3 Trellis-3 tails
68
+ - REAP reference: [REAP the Experts](https://arxiv.org/abs/2510.13999)
69
+
70
+ ## Responsible use
71
+
72
+ This derivative remains subject to the upstream OpenMDW-1.1 license and should
73
+ be used consistently with the upstream model's acceptable-use guidance.