Add files using upload-large-folder tool

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+model.safetensors.index.json filter=lfs diff=lfs merge=lfs -text

LICENSE.md

@@ -0,0 +1,49 @@
+OpenMDW License Agreement, version 1.1 (OpenMDW-1.1)
+
+By exercising rights granted to you under this agreement, you accept and agree
+to its terms.
+
+As used in this agreement, "Model Materials" means the materials provided to
+you under this agreement, consisting of: (1) one or more machine learning
+models (including architecture and parameters); and (2) all related artifacts
+(including associated data, documentation and software) that are provided to
+you hereunder.
+
+Subject to your compliance with this agreement, permission is hereby granted,
+free of charge, to deal in the Model Materials without restriction, including
+under all copyright, patent, database, and trade secret rights included or
+embodied therein.
+
+If you distribute any portion of the Model Materials, you shall retain in your
+distribution (1) a copy of this agreement, and (2) all copyright notices and
+other notices of origin included in the Model Materials that are applicable to
+your distribution.
+
+If you file, maintain, or voluntarily participate in a lawsuit against any
+person or entity asserting that the Model Materials directly or indirectly
+infringe any patent or copyright, then all rights and grants made to you
+hereunder are terminated, unless that lawsuit was in response to a
+corresponding lawsuit first brought against you.
+
+This agreement does not impose any restrictions or obligations with respect to
+any use, modification, or sharing of any outputs generated by using the Model
+Materials.
+
+THE MODEL MATERIALS ARE PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
+OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE, TITLE, NONINFRINGEMENT, ACCURACY, OR THE
+ABSENCE OF LATENT OR OTHER DEFECTS OR ERRORS, WHETHER OR NOT DISCOVERABLE, ALL
+TO THE GREATEST EXTENT PERMISSIBLE UNDER APPLICABLE LAW.
+
+YOU ARE SOLELY RESPONSIBLE FOR (1) CLEARING RIGHTS OF OTHER PERSONS THAT MAY
+APPLY TO THE MODEL MATERIALS OR ANY USE THEREOF, INCLUDING WITHOUT LIMITATION
+ANY PERSON'S COPYRIGHTS OR OTHER RIGHTS INCLUDED OR EMBODIED IN THE MODEL
+MATERIALS; (2) OBTAINING ANY NECESSARY CONSENTS, PERMISSIONS OR OTHER RIGHTS
+REQUIRED FOR ANY USE OF THE MODEL MATERIALS; OR (3) PERFORMING ANY DUE
+DILIGENCE OR UNDERTAKING ANY OTHER INVESTIGATIONS INTO THE MODEL MATERIALS OR
+ANYTHING INCORPORATED OR EMBODIED THEREIN.
+
+IN NO EVENT SHALL THE PROVIDERS OF THE MODEL MATERIALS BE LIABLE FOR ANY CLAIM,
+DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
+OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE MODEL MATERIALS, THE
+USE THEREOF OR OTHER DEALINGS THEREIN.

README.md

@@ -17,10 +17,10 @@ tags:
 
 # Laguna S 2.1 Hybrid 3.25bpw
 
-> **Pre-release build record.** This repository is intentionally private while
-> the weight transform and serving validation are in progress. Weight files are
-> not claimed to be available until the release manifest and validation evidence
-> are uploaded with them.
+> **Verified package.** This package was materialized only after the weight
+> transform, compact-package load, and hybrid-serving validation gates passed.
+> The included release manifest seals the source revision, tier map, runtime
+> overlay, validation evidence, and every packaged artifact hash.
 
 Laguna S 2.1 Hybrid 3.25bpw is a two-tier, expert-level quantization of
 [`poolside/Laguna-S-2.1-NVFP4`](https://huggingface.co/poolside/Laguna-S-2.1-NVFP4).
@@ -47,18 +47,32 @@ weighted expert-output activation saliency over a held-out calibration mixture
 covering coding, tool and agentic calls, reasoning, mathematics, science,
 terminal work, CUDA, cybersecurity, and long-context prompts. The score is the
 mean, over tokens routed to an expert, of the applied router weight multiplied
-by that expert's output L2 norm. The published release will include the exact
-tier map, source revision, calibration provenance, and artifact hashes.
+by that expert's output L2 norm. This release includes the exact tier map,
+source revision, calibration provenance, and artifact hashes.
 
 ## Runtime status and compatibility
 
-The hybrid requires the accompanying TP2 EXL3-tail runtime. It is not yet a
-drop-in stock-vLLM checkpoint. The release will include a pinned runtime overlay
-and an exact launch recipe only after the package passes load, reasoning,
-structured-output, tool-call, long-context, and throughput validation.
-
-Do not infer benchmark quality from this card. At the time this draft was
-created, no release benchmark result is claimed.
+The hybrid requires the accompanying EXL3-tail runtime. It is not a drop-in
+stock-vLLM checkpoint. The runtime supports two deployment layouts:
+
+- single-Spark TP1, which reconstructs each calibrated tail expert from both
+  stored artifact slices on one device;
+- two-Spark TP2, which loads one stored artifact slice per tensor-parallel
+  rank.
+
+Both layouts retain normal compiled execution and CUDA graphs. The package
+includes the pinned runtime overlay and exact launch recipes. The standalone
+53 GB package passed a fresh compact-package load on one DGX Spark with 200k
+context, Poolside reasoning and tool parsers, structured output support, and
+full/piecewise CUDA graph capture. Its sealed API suite covers reasoning,
+parsed tool calls, tool history, JSON schema, exact ASCII output, and a
+194,978-token long-context marker-recovery request. The measured stable
+single-Spark throughput sweep selected concurrency 4 at 58.20 aggregate
+completion tokens/s. The source checkpoint contains no native MTP tensors;
+the release therefore does not claim native-MTP throughput.
+
+Do not infer benchmark quality from this card. At the time this package was
+published, no Terminal-Bench 2.1 result was claimed.
 
 ## Provenance
 

chat_template.jinja

@@ -0,0 +1,94 @@
+{#- Iteration on laguna_glm_thinking_v8/chat_template.jinja -#}
+{#- No formatting instructions -#}
+{{- "〈|EOS|〉" -}}
+{%- set enable_thinking = enable_thinking | default(true) -%}
+{%- set add_generation_prompt = add_generation_prompt | default(false) -%}
+{%- set preserve_thinking = preserve_thinking | default(false) -%}
+
+{#- ───── header (system message) ───── -#}
+{#- A caller-supplied system message with empty content opts out of the default below, producing no <system> block — used to train without a system message. -#}
+{%- set system_message = "You are a helpful, conversationally-fluent assistant made by Poolside. You are here to be helpful to users through natural language conversations." -%}
+{%- if messages and messages[0].role == "system" -%}
+  {%- set system_message = messages[0].content -%}
+  {%- set messages = messages[1:] -%}
+{%- endif -%}
+
+{%- set has_sys = system_message and system_message.strip() -%}
+{%- if has_sys or tools or enable_thinking -%}
+  {{- "<system>" -}}
+
+  {%- if has_sys -%}
+    {{- system_message.rstrip() -}}
+    {%- if tools -%}{{- "\n\n" -}}{%- endif -%}
+  {%- endif -%}
+
+  {%- if tools -%}
+    {{- "### Tools\n\n" -}}
+    {{- "You may call functions to assist with the user query.\n" -}}
+    {{- "All available function signatures are listed below:\n" -}}
+    {{- "<available_tools>\n" -}}
+    {%- for tool in tools -%}
+      {{- (tool | tojson) ~ "\n" -}}
+    {%- endfor -%}
+    {{- "</available_tools>" -}}
+  {%- endif -%}
+
+  {{- "</system>\n" -}}
+{%- endif -%}
+
+{#- ───── main loop ───── -#}
+{%- for message in messages -%}
+  {%- set content = message.content if message.content is string else "" -%}
+  {%- if message.role == "user" -%}
+    {{- "<user>" + content + "</user>\n" -}}
+  {%- elif message.role == "assistant" -%}
+    {%- generation -%}
+      {{- "<assistant>" -}}
+      {#- Extract reasoning content from message.reasoning (vLLM field name) or message.reasoning_content -#}
+      {%- set reasoning_content = '' -%}
+      {%- if message.reasoning is string -%}
+        {%- set reasoning_content = message.reasoning -%}
+      {%- elif message.reasoning_content is string -%}
+        {%- set reasoning_content = message.reasoning_content -%}
+      {%- endif -%}
+      {#- Display reasoning content for all messages if enable_thinking -#}
+      {%- if enable_thinking or preserve_thinking -%}
+        {{- '<think>' + reasoning_content + '</think>' -}}
+      {%- else -%}
+        {{- '</think>' -}}
+      {%- endif -%}
+      {#- Display main content (trailing newline only when no tool_calls follow) -#}
+      {%- if content -%}
+        {{- content -}}
+      {%- endif -%}
+      {%- if message.tool_calls -%}
+        {%- for tool_call in message.tool_calls -%}
+          {%- set function_data = tool_call.function -%}
+          {{- '<tool_call>' + function_data.name -}}
+          {%- set _args = function_data.arguments -%}
+          {%- for k, v in _args.items() -%}
+            {{- "<arg_key>" ~ k ~ "</arg_key>" -}}
+            {{- "<arg_value>" -}}{{- v | tojson(ensure_ascii=False) if v is not string else v -}}{{- "</arg_value>" -}}
+          {%- endfor -%}
+          {{- "</tool_call>" -}}
+        {%- endfor -%}
+      {%- endif -%}
+      {{- "</assistant>\n" -}}
+    {%- endgeneration -%}
+  {%- elif message.role == "tool" -%}
+    {{- "<tool_response>" + content + "</tool_response>\n" -}}
+  {%- elif message.role == "system" -%}
+    {#- Render additional system messages (the first one, if any, is handled separately in the header and was sliced off above) -#}
+    {{- "<system>" + content + "</system>\n" -}}
+  {%- endif -%}
+{%- endfor -%}
+{#- ───── generation prompt ───── -#}
+{%- if add_generation_prompt -%}
+  {{- "<assistant>" -}}
+  {#- ───── Include reasoning mode directive ───── -#}
+  {%- if enable_thinking -%}
+    {{- '<think>' -}}
+  {%- else -%}
+    {{- '</think>' -}}
+  {%- endif -%}
+{%- endif -%}

config.json

@@ -0,0 +1,337 @@
+{
+  "architectures": [
+    "LagunaForCausalLM"
+  ],
+  "auto_map": {
+    "AutoConfig": "configuration_laguna.LagunaConfig",
+    "AutoModelForCausalLM": "modeling_laguna.LagunaForCausalLM"
+  },
+  "model_type": "laguna",
+  "vocab_size": 100352,
+  "hidden_size": 3072,
+  "intermediate_size": 12288,
+  "num_hidden_layers": 48,
+  "num_attention_heads": 48,
+  "num_key_value_heads": 8,
+  "head_dim": 128,
+  "max_position_embeddings": 262144,
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "rms_norm_eps": 1e-06,
+  "num_experts": 256,
+  "num_experts_per_tok": 10,
+  "moe_intermediate_size": 1024,
+  "shared_expert_intermediate_size": 1024,
+  "norm_topk_prob": true,
+  "router_aux_loss_coef": 0.0,
+  "moe_router_logit_softcapping": 0.0,
+  "decoder_sparse_step": 1,
+  "mlp_only_layers": [
+    0
+  ],
+  "bos_token_id": 2,
+  "eos_token_id": [
+    2,
+    24
+  ],
+  "pad_token_id": 9,
+  "tie_word_embeddings": false,
+  "use_cache": true,
+  "torch_dtype": "bfloat16",
+  "gating": "per-head",
+  "sliding_window": 512,
+  "rope_parameters": {
+    "full_attention": {
+      "rope_theta": 500000.0,
+      "rope_type": "yarn",
+      "factor": 32.0,
+      "original_max_position_embeddings": 8192,
+      "beta_slow": 1.0,
+      "beta_fast": 32.0,
+      "attention_factor": 1.3465735902799727,
+      "partial_rotary_factor": 0.5
+    },
+    "sliding_attention": {
+      "rope_type": "default",
+      "rope_theta": 10000.0,
+      "partial_rotary_factor": 1.0
+    }
+  },
+  "layer_types": [
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention"
+  ],
+  "moe_apply_router_weight_on_input": false,
+  "mlp_layer_types": [
+    "dense",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse"
+  ],
+  "gating_types": [
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head"
+  ],
+  "moe_routed_scaling_factor": 2.5,
+  "num_attention_heads_per_layer": [
+    48,
+    72,
+    72,
+    72,
+    48,
+    72,
+    72,
+    72,
+    48,
+    72,
+    72,
+    72,
+    48,
+    72,
+    72,
+    72,
+    48,
+    72,
+    72,
+    72,
+    48,
+    72,
+    72,
+    72,
+    48,
+    72,
+    72,
+    72,
+    48,
+    72,
+    72,
+    72,
+    48,
+    72,
+    72,
+    72,
+    48,
+    72,
+    72,
+    72,
+    48,
+    72,
+    72,
+    72,
+    48,
+    72,
+    72,
+    72
+  ],
+  "quantization_config": {
+    "config_groups": {
+      "group_0": {
+        "format": "nvfp4-pack-quantized",
+        "input_activations": {
+          "actorder": null,
+          "block_structure": null,
+          "dynamic": "local",
+          "group_size": 16,
+          "num_bits": 4,
+          "observer": "static_minmax",
+          "observer_kwargs": {},
+          "scale_dtype": "torch.float8_e4m3fn",
+          "strategy": "tensor_group",
+          "symmetric": true,
+          "type": "float",
+          "zp_dtype": null
+        },
+        "output_activations": null,
+        "targets": [
+          "re:.*experts\\.[0-9]+\\.(gate_proj|up_proj|down_proj)$"
+        ],
+        "weights": {
+          "actorder": null,
+          "block_structure": null,
+          "dynamic": false,
+          "group_size": 16,
+          "num_bits": 4,
+          "observer": "memoryless_minmax",
+          "observer_kwargs": {},
+          "scale_dtype": "torch.float8_e4m3fn",
+          "strategy": "tensor_group",
+          "symmetric": true,
+          "type": "float",
+          "zp_dtype": null
+        }
+      }
+    },
+    "format": "nvfp4-pack-quantized",
+    "global_compression_ratio": null,
+    "ignore": [
+      "lm_head",
+      "re:.*\\.self_attn\\.q_proj$",
+      "re:.*\\.self_attn\\.k_proj$",
+      "re:.*\\.self_attn\\.v_proj$",
+      "re:.*\\.self_attn\\.o_proj$",
+      "re:.*\\.self_attn\\.g_proj$",
+      "re:.*\\.mlp\\.gate$",
+      "model.layers.0.mlp.gate_proj",
+      "model.layers.0.mlp.up_proj",
+      "model.layers.0.mlp.down_proj",
+      "re:.*\\.mlp\\.shared_expert\\.gate_proj$",
+      "re:.*\\.mlp\\.shared_expert\\.up_proj$",
+      "re:.*\\.mlp\\.shared_expert\\.down_proj$"
+    ],
+    "kv_cache_scheme": {
+      "actorder": null,
+      "block_structure": null,
+      "dynamic": false,
+      "group_size": null,
+      "num_bits": 8,
+      "observer": "minmax",
+      "observer_kwargs": {},
+      "scale_dtype": null,
+      "strategy": "tensor",
+      "symmetric": true,
+      "type": "float",
+      "zp_dtype": null
+    },
+    "quant_method": "compressed-tensors",
+    "quantization_status": "compressed",
+    "sparsity_config": {},
+    "transform_config": {},
+    "version": "0.14.1.dev11+gf2ee47b"
+  }
+}

configuration_laguna.py

@@ -0,0 +1,256 @@
+# ruff: noqa
+# Copyright 2025 Poolside and the HuggingFace Inc. team. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from transformers.configuration_utils import PretrainedConfig as PreTrainedConfig
+try:
+    from transformers.modeling_rope_utils import RopeParameters
+except ImportError:
+    RopeParameters = dict
+try:
+    from transformers.utils.import_utils import is_causal_conv1d_available, is_flash_linear_attention_available
+except ImportError:
+    def is_causal_conv1d_available(): return False
+    def is_flash_linear_attention_available(): return False
+
+
+class LagunaConfig(PreTrainedConfig):
+    r"""
+    Configuration class for Laguna model.
+
+    Laguna is Poolside's MoE architecture with:
+    - Attention output gating (softplus gate)
+    - Sigmoid routing instead of softmax
+    - No QKV bias
+    - Explicit head_dim parameter
+
+    Args:
+        head_dim (`int`, *optional*, defaults to 128):
+            Dimension of attention heads. Laguna uses explicit head_dim rather than
+            computing it from hidden_size // num_attention_heads.
+        qkv_bias (`bool`, *optional*, defaults to `False`):
+            Whether to add bias to QKV projections. Laguna uses no QKV bias.
+        attention_bias (`bool`, *optional*, defaults to `False`):
+            Whether to add bias to attention output projection. Laguna uses no attention bias.
+        gating (`bool` or `str`, *optional*, defaults to `True`):
+            Attention output gating mode. When ``True`` or ``"per-element"`` a g_proj
+            linear layer with output size ``num_attention_heads * head_dim`` is added
+            and ``attn_output = attn_output * softplus(g_proj(x))``. When ``"per-head"``
+            g_proj has output size ``num_attention_heads`` and the gate broadcasts across
+            ``head_dim``. When ``False`` no gating is applied.
+        partial_rotary_factor (`float`, *optional*):
+            Fraction of head_dim to apply rotary embeddings to. When set, this value is
+            injected into ``rope_parameters`` (and ``swa_rope_parameters``) if not already
+            specified there. When ``None`` the default behaviour of the rope implementation
+            is used (typically full rotary).
+        num_attention_heads_per_layer (`list[int]`, *optional*):
+            Optional per-layer override for ``num_attention_heads``. When provided the list
+            length must equal ``num_hidden_layers`` and each entry is the head count used by
+            that layer. When ``None`` every layer uses ``num_attention_heads``.
+        vocab_size (`int`, *optional*, defaults to 100352):
+            Vocabulary size of the Laguna model.
+        hidden_size (`int`, *optional*, defaults to 2048):
+            Dimension of the hidden representations.
+        intermediate_size (`int`, *optional*, defaults to 8192):
+            Dimension of the MLP representations for dense layers.
+        num_hidden_layers (`int`, *optional*, defaults to 48):
+            Number of hidden layers in the Transformer.
+        num_attention_heads (`int`, *optional*, defaults to 32):
+            Number of attention heads.
+        num_key_value_heads (`int`, *optional*, defaults to 8):
+            Number of key-value heads for GQA.
+        max_position_embeddings (`int`, *optional*, defaults to 4096):
+            Maximum sequence length.
+        rms_norm_eps (`float`, *optional*, defaults to 1e-6):
+            Epsilon for RMSNorm layers.
+        sliding_window (`int`, *optional*):
+            Sliding window attention size. Used by layers whose type in ``layer_types``
+            is ``"sliding_attention"``. When ``None``, all layers use full attention.
+        layer_types (`list[str]`, *optional*):
+            Per-layer attention type. Each element should be ``"sliding_attention"`` or
+            ``"full_attention"``. Length must equal ``num_hidden_layers``. When ``None``,
+            all layers default to global attention.
+        swa_attention_sink_enabled (`bool`, *optional*, defaults to `False`):
+            Whether to enable learnable attention sinks on sliding-window attention layers.
+            When enabled, a per-head bias parameter is added that allows the model to attend
+            to position 0 even when it falls outside the sliding window.
+        swa_rope_parameters (`RopeParameters`, *optional*):
+            Separate RoPE configuration for sliding-window attention layers. When ``None``,
+            SWA layers use the same RoPE as global attention layers.
+        num_experts (`int`, *optional*, defaults to 256):
+            Number of routed experts.
+        num_experts_per_tok (`int`, *optional*, defaults to 16):
+            Number of experts selected per token (top-k).
+        moe_intermediate_size (`int`, *optional*, defaults to 1024):
+            Intermediate size of routed experts.
+        shared_expert_intermediate_size (`int`, *optional*, defaults to 1024):
+            Intermediate size of the shared expert.
+        norm_topk_prob (`bool`, *optional*, defaults to `True`):
+            Whether to normalize top-k routing probabilities.
+        decoder_sparse_step (`int`, *optional*, defaults to 1):
+            Frequency of MoE layers (1 = every layer is MoE after mlp_only_layers).
+        mlp_only_layers (`list[int]`, *optional*, defaults to `[0]`):
+            Layer indices that use dense MLP instead of MoE.
+        router_aux_loss_coef (`float`, *optional*, defaults to 0.001):
+            Auxiliary loss coefficient for load balancing.
+        moe_routed_scaling_factor (`float`, *optional*, defaults to 1.0):
+            Scalar multiplier applied to the routed-expert output before combining with the
+            shared-expert output.
+        moe_apply_router_weight_on_input (`bool`, *optional*, defaults to `False`):
+            When ``True`` the top-k routing weights are multiplied into each expert's input
+            rather than its output. Matches the numerical form used by the trained checkpoint.
+        moe_router_logit_softcapping (`float`, *optional*, defaults to 0.0):
+            Optional soft-capping value ``c`` applied to router logits as
+            ``x = tanh(x / c) * c`` before sigmoid + top-k. Disabled when ``0``.
+        rope_parameters (`RopeParameters`, *optional*):
+            RoPE configuration. Defaults to rope_theta=500000.0.
+    """
+
+    model_type = "laguna"
+    keys_to_ignore_at_inference = ["past_key_values"]
+    # PreTrainedConfig in transformers v5 no longer auto-declares these; subclasses
+    # opt in by providing class-level annotations with defaults.
+    pad_token_id: int | None = None
+    bos_token_id: int | None = None
+    eos_token_id: int | list[int] | None = None
+    base_model_tp_plan = {
+        "layers.*.self_attn.q_proj": "colwise",
+        "layers.*.self_attn.k_proj": "colwise",
+        "layers.*.self_attn.v_proj": "colwise",
+        "layers.*.self_attn.g_proj": "colwise",  # Laguna-specific gating projection
+        "layers.*.self_attn.o_proj": "rowwise",
+        "layers.*.mlp.gate_proj": "colwise",
+        "layers.*.mlp.up_proj": "colwise",
+        "layers.*.mlp.down_proj": "rowwise",
+    }
+    base_model_pp_plan = {
+        "embed_tokens": (["input_ids"], ["inputs_embeds"]),
+        "layers": (["hidden_states", "attention_mask"], ["hidden_states"]),
+        "norm": (["hidden_states"], ["hidden_states"]),
+    }
+
+    def __init__(
+        self,
+        vocab_size: int = 100352,
+        hidden_size: int = 2048,
+        intermediate_size: int = 8192,
+        num_hidden_layers: int = 48,
+        num_attention_heads: int = 32,
+        num_key_value_heads: int = 8,
+        head_dim: int = 128,
+        qkv_bias: bool = False,
+        attention_bias: bool = False,
+        gating: bool | str = True,
+        hidden_act: str = "silu",
+        max_position_embeddings: int = 4096,
+        initializer_range: float = 0.02,
+        rms_norm_eps: float = 1e-6,
+        use_cache: bool = True,
+        tie_word_embeddings: bool = False,
+        rope_parameters: RopeParameters | dict[str, RopeParameters] | None = None,
+        partial_rotary_factor: float | None = None,
+        attention_dropout: float = 0.0,
+        sliding_window: int | None = None,
+        layer_types: list[str] | None = None,
+        num_attention_heads_per_layer: list[int] | None = None,
+        swa_attention_sink_enabled: bool = False,
+        swa_rope_parameters: RopeParameters | None = None,
+        num_experts: int = 256,
+        num_experts_per_tok: int = 16,
+        moe_intermediate_size: int = 1024,
+        shared_expert_intermediate_size: int = 1024,
+        norm_topk_prob: bool = True,
+        decoder_sparse_step: int = 1,
+        mlp_only_layers: list[int] | None = None,
+        router_aux_loss_coef: float = 0.001,
+        moe_routed_scaling_factor: float = 1.0,
+        moe_apply_router_weight_on_input: bool = False,
+        moe_router_logit_softcapping: float = 0.0,
+        output_router_logits: bool = False,
+        **kwargs,
+    ):
+        # Default mlp_only_layers: first layer is dense (moe_first_k_dense_replace=1)
+        if mlp_only_layers is None:
+            mlp_only_layers = [0]
+
+        # Default layer_types: all layers use full attention (Laguna-M). Laguna-XS
+        # ships an explicit list with a mix of "full_attention" and "sliding_attention".
+        # Downstream mask builders (``create_masks_for_generate``) iterate
+        # ``layer_types``, so it must be a list — not left as ``None``.
+        if layer_types is None:
+            layer_types = ["full_attention"] * num_hidden_layers
+
+        # Default rope_parameters with Laguna's theta
+        if rope_parameters is None:
+            rope_parameters = {"rope_type": "default", "rope_theta": 500000.0}
+
+        # config.json stores SWA rope nested in rope_parameters["sliding_attention"]
+        # and carries no top-level swa_rope_parameters. Derive it here, else the
+        # sliding-window layers silently reuse the full-attention rope.
+        if swa_rope_parameters is None and isinstance(rope_parameters, dict):
+            swa_rope_parameters = rope_parameters.get("sliding_attention")
+
+        # If ``partial_rotary_factor`` is set at the top level, inject it into any
+        # rope dict that does not already carry one so the rotary embedding picks
+        # it up consistently for both full-attention and SWA layers.
+        if partial_rotary_factor is not None:
+            if isinstance(rope_parameters, dict) and "partial_rotary_factor" not in rope_parameters:
+                rope_parameters = {**rope_parameters, "partial_rotary_factor": partial_rotary_factor}
+            if isinstance(swa_rope_parameters, dict) and "partial_rotary_factor" not in swa_rope_parameters:
+                swa_rope_parameters = {
+                    **swa_rope_parameters,
+                    "partial_rotary_factor": partial_rotary_factor,
+                }
+
+        self.vocab_size = vocab_size
+        self.hidden_size = hidden_size
+        self.intermediate_size = intermediate_size
+        self.num_hidden_layers = num_hidden_layers
+        self.num_attention_heads = num_attention_heads
+        self.num_key_value_heads = num_key_value_heads
+        self.head_dim = head_dim
+        self.qkv_bias = qkv_bias
+        self.attention_bias = attention_bias
+        self.gating = gating
+        self.hidden_act = hidden_act
+        self.max_position_embeddings = max_position_embeddings
+        self.initializer_range = initializer_range
+        self.rms_norm_eps = rms_norm_eps
+        self.use_cache = use_cache
+        self.rope_parameters = rope_parameters
+        self.partial_rotary_factor = partial_rotary_factor
+        self.attention_dropout = attention_dropout
+        # Sliding window attention arguments
+        self.sliding_window = sliding_window
+        self.layer_types = layer_types
+        self.num_attention_heads_per_layer = num_attention_heads_per_layer
+        self.swa_attention_sink_enabled = swa_attention_sink_enabled
+        self.swa_rope_parameters = swa_rope_parameters
+        # MoE arguments
+        self.num_experts = num_experts
+        self.num_experts_per_tok = num_experts_per_tok
+        self.moe_intermediate_size = moe_intermediate_size
+        self.shared_expert_intermediate_size = shared_expert_intermediate_size
+        self.norm_topk_prob = norm_topk_prob
+        self.decoder_sparse_step = decoder_sparse_step
+        self.mlp_only_layers = mlp_only_layers
+        self.router_aux_loss_coef = router_aux_loss_coef
+        self.moe_routed_scaling_factor = moe_routed_scaling_factor
+        self.moe_apply_router_weight_on_input = moe_apply_router_weight_on_input
+        self.moe_router_logit_softcapping = moe_router_logit_softcapping
+        self.output_router_logits = output_router_logits
+
+        super().__init__(tie_word_embeddings=tie_word_embeddings, **kwargs)
+
+
+__all__ = ["LagunaConfig"]

generation_config.json

@@ -0,0 +1,24 @@
+{
+  "bos_token_id": 2,
+  "do_sample": true,
+  "eos_token_id": [
+    2,
+    24
+  ],
+  "pad_token_id": 9,
+  "temperature": 1.0,
+  "top_p": 1.0,
+  "speculative_config": {
+    "method": "dflash",
+    "source": "huggingface",
+    "model": "poolside/Laguna-S-2.1-DFlash-NVFP4",
+    "num_speculative_tokens": 15
+  },
+  "tool_call_parser": "poolside_v1",
+  "reasoning_parser": "poolside_v1",
+  "default_chat_template_kwargs": {
+    "enable_thinking": true
+  },
+  "top_k": 20,
+  "min_p": 0.0
+}

hybrid/provenance/compact-runtime-validation.json

@@ -0,0 +1,27 @@
+{
+  "api": {
+    "ascii_exact.json": "d612d240beba543a0195b14bda552dbd11d99ce12bc8922a41cdbcd3e05edc5f",
+    "json_schema.json": "df181e6bfeaca7467ef1da7157424d9a9888c4490ad2ba70930f16bd016e793f",
+    "manifest.json": "66bde0d76afa10216756cb5414e28d67de11af12ace319a732a23b734cf6cd54",
+    "reasoning.json": "cfc1aedd8b12893dfd28fd511a8c146fdd54d2ade964e8da5cf3045838f3b2e3",
+    "tool_call.json": "375b6e470e6f17cc6f02a14ac46e33bb584ecf4887e91ded008e736493a23998",
+    "tool_history.json": "2b62deca1ef70c8f7b4176d0318bcf69c908b65e9b70658c2132400ce7af94c9"
+  },
+  "compacted_index_sha256": "7f6e2d40d6a6f4571d99a0f2a5c7f987af729beebdb66af98301f77c521e3de5",
+  "compaction_evidence_sha256": "425f7ec4ee5fc0f857f183e73ad4efe738d44c4c272e68cd788f5f842f5b7031",
+  "context_configured": 200000,
+  "cuda_graphs": true,
+  "generation_limit_fields_present": false,
+  "package": "/home/sero/laguna-reap-saliency-v1/hf-release-staging-20260725-a3/Laguna-S-2.1-Hybrid-3.25bpw",
+  "schema": "laguna-compact-runtime-validation/v1",
+  "service": {
+    "artifact_tensor_parallel_size": 2,
+    "head_inspect_sha256": "dbf827997fb08f73bc26190b2aa43c166c2825a9190221b537786baef7fe954f",
+    "head_log_sha256": "eb7841603c44358b9d7352e44a4c5f98e290422b09769c9e716def32df62fff1",
+    "runtime_tensor_parallel_size": 1,
+    "worker_inspect_sha256": null,
+    "worker_log_sha256": null
+  },
+  "speculative_decode_method": "none",
+  "status": "complete"
+}

hybrid/provenance/encoder-validation.json

@@ -0,0 +1,59 @@
+{
+  "capture_manifest_sha256": "976ba9935c2ae028117276f32bd33195862733a2b76cacb5514d3f2c2fb12a69",
+  "codebook": "mcg",
+  "encoded_bytes_total": 50432669432,
+  "encoder_script_sha256": "30ccbf529dcab421ec904d8069a0ef33714d25408abf615ff76fccbcffae6b44",
+  "experts_per_sparse_layer": 256,
+  "fallback_slices": 0,
+  "generated_at": "2026-07-25T16:19:07.556438+00:00",
+  "hessian": "routed-real-activations",
+  "mcg_multiplier": "0xcbac1fed",
+  "partition_manifests": {
+    "0": {
+      "path": "/work/exl3-tail-current-encode-20260723-a4/partition-0.manifest.json",
+      "sha256": "b2e2377dea353f883e72c563063f36866d00f9acb04cd6fad030e437f4c6fd3e"
+    },
+    "1": {
+      "path": "/work/exl3-tail-current-encode-20260723-a4/partition-1.manifest.json",
+      "sha256": "1ed41207a55fc462952340945de6843f0a8f4aa507affb7cc037dffc345549b7"
+    }
+  },
+  "schema": "laguna-exl3-tail-validation/v1",
+  "source_index_sha256": "c872bd9b5ac116cebfd78a5f27565787dbceae76985cef041179110d0b03cd2b",
+  "sparse_layers": 47,
+  "status": "complete",
+  "tiers": {
+    "TR2": {
+      "bits": 2,
+      "encoded_bytes": 18085239032,
+      "expert_files": 7520,
+      "fallback_slices": 0,
+      "hot_nvfp4_experts_per_layer": 96,
+      "hot_saliency_coverage_mean": 0.5584435128448573,
+      "hot_saliency_coverage_min": 0.4937639739328926,
+      "maximum_calibration_rows_per_expert": 2048,
+      "minimum_calibration_rows_per_expert": 4,
+      "projection_rank_slices": 45120,
+      "tail_trellis_experts_per_layer": 160,
+      "tier_map": "/work/final-current-observations-20260723-a1/tier-maps/laguna-tr2-tier-map.json",
+      "tier_map_sha256": "bef6b3cda9d47938cfef8c0a50e2f088e7f2471c6a88a93571f3904e5bd02406"
+    },
+    "TR3": {
+      "bits": 3,
+      "encoded_bytes": 32347430400,
+      "expert_files": 9024,
+      "fallback_slices": 0,
+      "hot_nvfp4_experts_per_layer": 64,
+      "hot_saliency_coverage_mean": 0.4211937699439102,
+      "hot_saliency_coverage_min": 0.35016990543040455,
+      "maximum_calibration_rows_per_expert": 2048,
+      "minimum_calibration_rows_per_expert": 4,
+      "projection_rank_slices": 54144,
+      "tail_trellis_experts_per_layer": 192,
+      "tier_map": "/work/final-current-observations-20260723-a1/tier-maps/laguna-tr3-tier-map.json",
+      "tier_map_sha256": "98bb2a6cfab92807201180c1cde4447d50fc7bd991bde8f2f6a869a84e2ff45a"
+    }
+  },
+  "tp": 2,
+  "validator_script_sha256": "ac83b4b1db7f325c8f33cf7f1da58e01963a45090e3a563341371644ea1502b3"
+}

hybrid/provenance/hot-nvfp4-compaction.json

@@ -0,0 +1,101 @@
+{
+  "compacted_index_sha256": "7f6e2d40d6a6f4571d99a0f2a5c7f987af729beebdb66af98301f77c521e3de5",
+  "hot_nvfp4_experts_per_sparse_layer": 64,
+  "kept_expert_tensor_names": 36096,
+  "kept_tensor_names": 36865,
+  "package": "/package",
+  "removed_cold_expert_tensor_names": 108288,
+  "schema": "laguna-hot-nvfp4-compaction/v1",
+  "shards": [
+    {
+      "bytes": 5086273312,
+      "path": "model-00001-of-00014.safetensors",
+      "sha256": "80518caa724f3ec5bd0470767196354a7f64abdb35d48272d4e99ad69033f5bc",
+      "tensors": 568
+    },
+    {
+      "bytes": 3711336800,
+      "path": "model-00002-of-00014.safetensors",
+      "sha256": "18428e5d5c5b4b613abd6cbd87c1df1a83317956408a1922bab071a27c465f27",
+      "tensors": 1941
+    },
+    {
+      "bytes": 1261986736,
+      "path": "model-00003-of-00014.safetensors",
+      "sha256": "1f04555a5d0b62586899d8a03230810996ab56b68a002a14395454ce4304b0cd",
+      "tensors": 2852
+    },
+    {
+      "bytes": 1281456824,
+      "path": "model-00004-of-00014.safetensors",
+      "sha256": "54a9f15b8b7847eda2dae096ebab2809e0581e619e0c165977eb60998da2a393",
+      "tensors": 2896
+    },
+    {
+      "bytes": 1290308968,
+      "path": "model-00005-of-00014.safetensors",
+      "sha256": "6f880b76d49467e6607115d319a4c9951485e21edb970d36b139e21feac671a5",
+      "tensors": 2916
+    },
+    {
+      "bytes": 1274379368,
+      "path": "model-00006-of-00014.safetensors",
+      "sha256": "a53e29d037d4dc8fc75c66b27d01c0d93379ea724bf18da2a4c0538f13a21ad1",
+      "tensors": 2880
+    },
+    {
+      "bytes": 1269069432,
+      "path": "model-00007-of-00014.safetensors",
+      "sha256": "dde2007d751a617b88be63e09182569ae9ccf677152fae952fe43eb5cc337279",
+      "tensors": 2868
+    },
+    {
+      "bytes": 1295618832,
+      "path": "model-00008-of-00014.safetensors",
+      "sha256": "209a3440bb4a2fc4bab013c996da84409099e2937aa33e21c0b0dad99170c04c",
+      "tensors": 2928
+    },
+    {
+      "bytes": 1261989344,
+      "path": "model-00009-of-00014.safetensors",
+      "sha256": "844d74c0e0dcb4c15a90b1fe14b470f1f89c45f81c0d638524c2a52439db6c1a",
+      "tensors": 2852
+    },
+    {
+      "bytes": 1281459120,
+      "path": "model-00010-of-00014.safetensors",
+      "sha256": "cdeee15029516ff926f56aeaccbf74f5d5ba28caec0c30b7c4a1609360c7e6d4",
+      "tensors": 2896
+    },
+    {
+      "bytes": 1269069432,
+      "path": "model-00011-of-00014.safetensors",
+      "sha256": "2432c5b7f5fcb066f6f72bbc7fd5464e843bd017ebef40372c305f9daaa21d8c",
+      "tensors": 2868
+    },
+    {
+      "bytes": 1300928888,
+      "path": "model-00012-of-00014.safetensors",
+      "sha256": "a011d968f6201e75622e15c3efa724b4632971314b6399dd748460d9bb411079",
+      "tensors": 2940
+    },
+    {
+      "bytes": 1269069264,
+      "path": "model-00013-of-00014.safetensors",
+      "sha256": "55bc235a0f77c1ba75d15736d4ef4f3d121b89e64da8cd071e67508f9a45a7da",
+      "tensors": 2868
+    },
+    {
+      "bytes": 1146941096,
+      "path": "model-00014-of-00014.safetensors",
+      "sha256": "d6971de0d660d9d3a4ff71c904df692aa43e866ad78bd9c1dc064ee62c171fda",
+      "tensors": 2592
+    }
+  ],
+  "source_index_sha256": "c872bd9b5ac116cebfd78a5f27565787dbceae76985cef041179110d0b03cd2b",
+  "status": "complete",
+  "tail_experts_per_sparse_layer": 192,
+  "tensor_bytes": 23995371200,
+  "tier": "TR3",
+  "tier_map_sha256": "98bb2a6cfab92807201180c1cde4447d50fc7bd991bde8f2f6a869a84e2ff45a"
+}

hybrid/provenance/hybrid-validation.json

@@ -0,0 +1,51 @@
+{
+  "api_validator_script_sha256": "805b211515cb38261ddf5e008392963dd9ba2a22d44b22e24e0862fc3e4f5ce5",
+  "artifact_tensor_parallel_size": 2,
+  "bundle_validator_script_sha256": "65fe66b618817598511cb0719548886acebd191c9053a69d293938613259b967",
+  "dflash_audit_sha256": "59a5df3241775bf55434dda37715e06f319233eae98bc4afee9d40cbb940af9b",
+  "full_195k": {
+    "ascii_exact.json": "45b0b5aa3a1eafa51386acd29ec944cd4c4391c820a3a22a33a60aa44a6a18ce",
+    "json_schema.json": "b8977c3d565fb6c2a5efcac2e3c92133b7e7827f506db2256df00941f27a3d90",
+    "long_context_195k.json": "4b7cb9e1b942a726f0afe1b7bb539b559f4ae6f154a74aa5358c655037ee58f7",
+    "manifest.json": "1364cc601d549c288ccd7f13e812bd1b4ea2b084feae55c4dd9c2ffe214c0d72",
+    "reasoning.json": "149a13f2b5791fd3d2a94ad1f3493cafca11b19f8c50fd2afd424e4348ced751",
+    "tool_call.json": "0a6b51f5a806ff347718abbda6d46e4de024086ac5839a3242e06a366ea4d75e",
+    "tool_history.json": "c35494a39d89818eea3c013b69b97873c2d03bc6e8334b20d01d9d5272bfd3d7"
+  },
+  "model": "laguna-s-2.1-hybrid-3.25bpw",
+  "mtp_audit_sha256": "9fcd8af34988fd8a6e24fcf5fc2eac2fb996f7567d1f9dc9a74ea3bc8a1b148b",
+  "runtime_tensor_parallel_size": 1,
+  "schema": "laguna-hybrid-validation-bundle/v1",
+  "service": {
+    "head.inspect.json": "fc7dbca5c5d353dca3d1f24e7787e8aaa6cb044996aa549d621a22a76efc0198",
+    "head.log": "629ebe4763c2f9324a1fea37add4afb6fdddbe41a980197fafa8dbcc5e017eae"
+  },
+  "small": {
+    "ascii_exact.json": "45b0b5aa3a1eafa51386acd29ec944cd4c4391c820a3a22a33a60aa44a6a18ce",
+    "json_schema.json": "b8977c3d565fb6c2a5efcac2e3c92133b7e7827f506db2256df00941f27a3d90",
+    "manifest.json": "4f10e592b0e74a8ab88d77f1893e0920b047ce6a50532b2cc7e50595a89c255f",
+    "reasoning.json": "149a13f2b5791fd3d2a94ad1f3493cafca11b19f8c50fd2afd424e4348ced751",
+    "tool_call.json": "0a6b51f5a806ff347718abbda6d46e4de024086ac5839a3242e06a366ea4d75e",
+    "tool_history.json": "c35494a39d89818eea3c013b69b97873c2d03bc6e8334b20d01d9d5272bfd3d7"
+  },
+  "status": "complete",
+  "throughput": {
+    "completion_tokens": 1220,
+    "dflash_draft_acceptance_rate": null,
+    "dflash_mean_acceptance_length": null,
+    "native_mtp_available": false,
+    "native_mtp_measured": false,
+    "selected_aggregate_completion_tokens_per_second": 58.19561887597636,
+    "selected_concurrency": 4,
+    "sha256": "907a6e5da8f6e70843818dcc741ef8a05aa179ad6ae8fb87f3d1be1962e1587a",
+    "speculative_decode_method": "none",
+    "speculative_decode_native_mtp": false,
+    "sweeps": {
+      "1": 16.157817458300887,
+      "2": 29.467371630315867,
+      "4": 58.19561887597636
+    }
+  },
+  "throughput_validator_script_sha256": "cfa026eff1b7af6a52717a0de3a9235d28c1b5a78ab0e20fbece48fdd5aa72f8",
+  "tier": "TR3"
+}

hybrid/tails/tr3/layer-26/expert-013.json

@@ -0,0 +1,82 @@
+{
+  "bits": 3,
+  "bytes": 3584600,
+  "calibration_rows": 2048,
+  "calibration_sampling": "deterministic evenly spaced routed-row indices",
+  "capture_manifest_sha256": "976ba9935c2ae028117276f32bd33195862733a2b76cacb5514d3f2c2fb12a69",
+  "encoder_script_sha256": "30ccbf529dcab421ec904d8069a0ef33714d25408abf615ff76fccbcffae6b44",
+  "expert": 13,
+  "fallback_slices": 0,
+  "layer": 26,
+  "output": "/out/tr3/layer-26/expert-013.safetensors",
+  "routed_observation_rows_total": "3353",
+  "schema": "laguna-exl3-expert-encode/v1",
+  "sha256": "e559e50c55d624f0938c66dc8710faaac617fee7570d12cd71f7368f2ec91f03",
+  "slices": {
+    "down_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9397321041054102,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.01190834367455955,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "down_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9216421086391007,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.011454986657873702,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0037454840254873543,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.00018310546875,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.003748725730490386,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9250969903450001,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0047551796588070185,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9543672178658202,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.004773790994314254,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    }
+  },
+  "source_index_sha256": "c872bd9b5ac116cebfd78a5f27565787dbceae76985cef041179110d0b03cd2b",
+  "status": "complete",
+  "tier": "TR3",
+  "tier_map_sha256": "98bb2a6cfab92807201180c1cde4447d50fc7bd991bde8f2f6a869a84e2ff45a",
+  "tp": 2
+}

hybrid/tails/tr3/layer-26/expert-021.json

@@ -0,0 +1,82 @@
+{
+  "bits": 3,
+  "bytes": 3584600,
+  "calibration_rows": 2048,
+  "calibration_sampling": "deterministic evenly spaced routed-row indices",
+  "capture_manifest_sha256": "976ba9935c2ae028117276f32bd33195862733a2b76cacb5514d3f2c2fb12a69",
+  "encoder_script_sha256": "30ccbf529dcab421ec904d8069a0ef33714d25408abf615ff76fccbcffae6b44",
+  "expert": 21,
+  "fallback_slices": 0,
+  "layer": 26,
+  "output": "/out/tr3/layer-26/expert-021.safetensors",
+  "routed_observation_rows_total": "5304",
+  "schema": "laguna-exl3-expert-encode/v1",
+  "sha256": "6ee0248a1ce23dbeddfa2f3550ea1ac897be848485b3480d9affc09829ec539f",
+  "slices": {
+    "down_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9397321041054102,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.01222266101919929,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "down_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.01249527955800585,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.00018310546875,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9599573338930756,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0030492760547702444,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9306871063722555,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0031049715569851986,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9362772223995108,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0037786707120726687,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9397321041054102,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.003703849894337033,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    }
+  },
+  "source_index_sha256": "c872bd9b5ac116cebfd78a5f27565787dbceae76985cef041179110d0b03cd2b",
+  "status": "complete",
+  "tier": "TR3",
+  "tier_map_sha256": "98bb2a6cfab92807201180c1cde4447d50fc7bd991bde8f2f6a869a84e2ff45a",
+  "tp": 2
+}

hybrid/tails/tr3/layer-26/expert-041.json

@@ -0,0 +1,82 @@
+{
+  "bits": 3,
+  "bytes": 3584600,
+  "calibration_rows": 2048,
+  "calibration_sampling": "deterministic evenly spaced routed-row indices",
+  "capture_manifest_sha256": "976ba9935c2ae028117276f32bd33195862733a2b76cacb5514d3f2c2fb12a69",
+  "encoder_script_sha256": "30ccbf529dcab421ec904d8069a0ef33714d25408abf615ff76fccbcffae6b44",
+  "expert": 41,
+  "fallback_slices": 0,
+  "layer": 26,
+  "output": "/out/tr3/layer-26/expert-041.safetensors",
+  "routed_observation_rows_total": "4804",
+  "schema": "laguna-exl3-expert-encode/v1",
+  "sha256": "3dc04471555461b5b13a77723d06d5640e8d9912572c56f9534fbf2cca4af92f",
+  "slices": {
+    "down_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9397321041054102,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.011156002251789211,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "down_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9397321041054102,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.011212271551339418,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.003350300688952894,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9397321041054102,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.003351279587542907,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9362772223995108,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0038472564203765134,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9362772223995108,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0038839594189082325,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    }
+  },
+  "source_index_sha256": "c872bd9b5ac116cebfd78a5f27565787dbceae76985cef041179110d0b03cd2b",
+  "status": "complete",
+  "tier": "TR3",
+  "tier_map_sha256": "98bb2a6cfab92807201180c1cde4447d50fc7bd991bde8f2f6a869a84e2ff45a",
+  "tp": 2
+}

hybrid/tails/tr3/layer-26/expert-045.json

@@ -0,0 +1,82 @@
+{
+  "bits": 3,
+  "bytes": 3584600,
+  "calibration_rows": 2048,
+  "calibration_sampling": "deterministic evenly spaced routed-row indices",
+  "capture_manifest_sha256": "976ba9935c2ae028117276f32bd33195862733a2b76cacb5514d3f2c2fb12a69",
+  "encoder_script_sha256": "30ccbf529dcab421ec904d8069a0ef33714d25408abf615ff76fccbcffae6b44",
+  "expert": 45,
+  "fallback_slices": 0,
+  "layer": 26,
+  "output": "/out/tr3/layer-26/expert-045.safetensors",
+  "routed_observation_rows_total": "6557",
+  "schema": "laguna-exl3-expert-encode/v1",
+  "sha256": "a7a3f0f5763ae6240de79566833d92943104c92af6c6810a41246e6acef9db50",
+  "slices": {
+    "down_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.013418506434469695,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "down_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.013560148070835793,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9599573338930756,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.003650578700766417,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.003670895211106669,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.005038167534271601,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9362772223995108,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.005084570639361082,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    }
+  },
+  "source_index_sha256": "c872bd9b5ac116cebfd78a5f27565787dbceae76985cef041179110d0b03cd2b",
+  "status": "complete",
+  "tier": "TR3",
+  "tier_map_sha256": "98bb2a6cfab92807201180c1cde4447d50fc7bd991bde8f2f6a869a84e2ff45a",
+  "tp": 2
+}

hybrid/tails/tr3/layer-26/expert-047.json

@@ -0,0 +1,82 @@
+{
+  "bits": 3,
+  "bytes": 3584600,
+  "calibration_rows": 263,
+  "calibration_sampling": "deterministic evenly spaced routed-row indices",
+  "capture_manifest_sha256": "976ba9935c2ae028117276f32bd33195862733a2b76cacb5514d3f2c2fb12a69",
+  "encoder_script_sha256": "30ccbf529dcab421ec904d8069a0ef33714d25408abf615ff76fccbcffae6b44",
+  "expert": 47,
+  "fallback_slices": 0,
+  "layer": 26,
+  "output": "/out/tr3/layer-26/expert-047.safetensors",
+  "routed_observation_rows_total": "263",
+  "schema": "laguna-exl3-expert-encode/v1",
+  "sha256": "dc3f1998f04872b903dfb7982f6fb8f744f09366587fd384246787b7398ae9fb",
+  "slices": {
+    "down_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 263,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.004581292799514752,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "down_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 263,
+      "g_scale": 0.9250969903450001,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.004442808941635246,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 263,
+      "g_scale": 0.9543672178658202,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0007419767118913538,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 263,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0007530752266272285,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0009765625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 263,
+      "g_scale": 0.9543672178658202,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0009514474878519153,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 263,
+      "g_scale": 0.9397321041054102,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0009681822044176422,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    }
+  },
+  "source_index_sha256": "c872bd9b5ac116cebfd78a5f27565787dbceae76985cef041179110d0b03cd2b",
+  "status": "complete",
+  "tier": "TR3",
+  "tier_map_sha256": "98bb2a6cfab92807201180c1cde4447d50fc7bd991bde8f2f6a869a84e2ff45a",
+  "tp": 2
+}

hybrid/tails/tr3/layer-26/expert-050.json

@@ -0,0 +1,82 @@
+{
+  "bits": 3,
+  "bytes": 3584600,
+  "calibration_rows": 2048,
+  "calibration_sampling": "deterministic evenly spaced routed-row indices",
+  "capture_manifest_sha256": "976ba9935c2ae028117276f32bd33195862733a2b76cacb5514d3f2c2fb12a69",
+  "encoder_script_sha256": "30ccbf529dcab421ec904d8069a0ef33714d25408abf615ff76fccbcffae6b44",
+  "expert": 50,
+  "fallback_slices": 0,
+  "layer": 26,
+  "output": "/out/tr3/layer-26/expert-050.safetensors",
+  "routed_observation_rows_total": "9191",
+  "schema": "laguna-exl3-expert-encode/v1",
+  "sha256": "0ae928ca358285ec8812d36c4edae6fe867dc542763ea1d8f5fd1cbcb4092e0b",
+  "slices": {
+    "down_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.011779910660187153,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "down_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.012099880125766788,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.002730701414743581,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0026910380859019012,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9543672178658202,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0031192887907079783,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.003098162624897391,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    }
+  },
+  "source_index_sha256": "c872bd9b5ac116cebfd78a5f27565787dbceae76985cef041179110d0b03cd2b",
+  "status": "complete",
+  "tier": "TR3",
+  "tier_map_sha256": "98bb2a6cfab92807201180c1cde4447d50fc7bd991bde8f2f6a869a84e2ff45a",
+  "tp": 2
+}

hybrid/tails/tr3/layer-26/expert-051.json

@@ -0,0 +1,82 @@
+{
+  "bits": 3,
+  "bytes": 3584600,
+  "calibration_rows": 2048,
+  "calibration_sampling": "deterministic evenly spaced routed-row indices",
+  "capture_manifest_sha256": "976ba9935c2ae028117276f32bd33195862733a2b76cacb5514d3f2c2fb12a69",
+  "encoder_script_sha256": "30ccbf529dcab421ec904d8069a0ef33714d25408abf615ff76fccbcffae6b44",
+  "expert": 51,
+  "fallback_slices": 0,
+  "layer": 26,
+  "output": "/out/tr3/layer-26/expert-051.safetensors",
+  "routed_observation_rows_total": "5191",
+  "schema": "laguna-exl3-expert-encode/v1",
+  "sha256": "704ba011c9e392a3401012a1b6c95b6bf4c7bd9baf454af311626f4d0aaebfe6",
+  "slices": {
+    "down_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9306871063722555,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.011228021244543845,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "down_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9397321041054102,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.011355284666927118,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0032538705944831748,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9397321041054102,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0033172352150124774,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9634122155989749,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.004028058318016975,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.003993520838633128,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    }
+  },
+  "source_index_sha256": "c872bd9b5ac116cebfd78a5f27565787dbceae76985cef041179110d0b03cd2b",
+  "status": "complete",
+  "tier": "TR3",
+  "tier_map_sha256": "98bb2a6cfab92807201180c1cde4447d50fc7bd991bde8f2f6a869a84e2ff45a",
+  "tp": 2
+}

hybrid/tails/tr3/layer-26/expert-062.json

@@ -0,0 +1,82 @@
+{
+  "bits": 3,
+  "bytes": 3584600,
+  "calibration_rows": 2048,
+  "calibration_sampling": "deterministic evenly spaced routed-row indices",
+  "capture_manifest_sha256": "976ba9935c2ae028117276f32bd33195862733a2b76cacb5514d3f2c2fb12a69",
+  "encoder_script_sha256": "30ccbf529dcab421ec904d8069a0ef33714d25408abf615ff76fccbcffae6b44",
+  "expert": 62,
+  "fallback_slices": 0,
+  "layer": 26,
+  "output": "/out/tr3/layer-26/expert-062.safetensors",
+  "routed_observation_rows_total": "8132",
+  "schema": "laguna-exl3-expert-encode/v1",
+  "sha256": "c14253bda0bc5ed7c58639fc31d46819017fbbc951c56fbdf00d5b47c1150db9",
+  "slices": {
+    "down_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.013346146630912905,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "down_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9397321041054102,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.013242899463589843,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.003617838391116708,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0036358786379500517,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.004334873889221136,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.004331254367431256,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    }
+  },
+  "source_index_sha256": "c872bd9b5ac116cebfd78a5f27565787dbceae76985cef041179110d0b03cd2b",
+  "status": "complete",
+  "tier": "TR3",
+  "tier_map_sha256": "98bb2a6cfab92807201180c1cde4447d50fc7bd991bde8f2f6a869a84e2ff45a",
+  "tp": 2
+}

hybrid/tails/tr3/layer-26/expert-073.json

@@ -0,0 +1,82 @@
+{
+  "bits": 3,
+  "bytes": 3584600,
+  "calibration_rows": 2048,
+  "calibration_sampling": "deterministic evenly spaced routed-row indices",
+  "capture_manifest_sha256": "976ba9935c2ae028117276f32bd33195862733a2b76cacb5514d3f2c2fb12a69",
+  "encoder_script_sha256": "30ccbf529dcab421ec904d8069a0ef33714d25408abf615ff76fccbcffae6b44",
+  "expert": 73,
+  "fallback_slices": 0,
+  "layer": 26,
+  "output": "/out/tr3/layer-26/expert-073.safetensors",
+  "routed_observation_rows_total": "3432",
+  "schema": "laguna-exl3-expert-encode/v1",
+  "sha256": "1d743261a4fd7da8a67ebe06c3b9b92958bf3948ee5361cd029bc7201b95009e",
+  "slices": {
+    "down_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9362772223995108,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.011544699497591645,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "down_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.011705302223544295,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0031108557967209093,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9362772223995108,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.00307365995702531,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9397321041054103,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0036444665215095715,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0036778327632856485,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    }
+  },
+  "source_index_sha256": "c872bd9b5ac116cebfd78a5f27565787dbceae76985cef041179110d0b03cd2b",
+  "status": "complete",
+  "tier": "TR3",
+  "tier_map_sha256": "98bb2a6cfab92807201180c1cde4447d50fc7bd991bde8f2f6a869a84e2ff45a",
+  "tp": 2
+}

hybrid/tails/tr3/layer-26/expert-074.json

@@ -0,0 +1,82 @@
+{
+  "bits": 3,
+  "bytes": 3584600,
+  "calibration_rows": 2048,
+  "calibration_sampling": "deterministic evenly spaced routed-row indices",
+  "capture_manifest_sha256": "976ba9935c2ae028117276f32bd33195862733a2b76cacb5514d3f2c2fb12a69",
+  "encoder_script_sha256": "30ccbf529dcab421ec904d8069a0ef33714d25408abf615ff76fccbcffae6b44",
+  "expert": 74,
+  "fallback_slices": 0,
+  "layer": 26,
+  "output": "/out/tr3/layer-26/expert-074.safetensors",
+  "routed_observation_rows_total": "5368",
+  "schema": "laguna-exl3-expert-encode/v1",
+  "sha256": "f11894746748e6c9b9fa9294bef82eef1271259e98afe1c9df4754261070c1ce",
+  "slices": {
+    "down_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.011217107712484553,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "down_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9543672178658202,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.010506694236282192,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9397321041054102,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0033860802794123523,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9362772223995108,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.003419268075599958,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.004271929378551926,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.00427387342955624,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    }
+  },
+  "source_index_sha256": "c872bd9b5ac116cebfd78a5f27565787dbceae76985cef041179110d0b03cd2b",
+  "status": "complete",
+  "tier": "TR3",
+  "tier_map_sha256": "98bb2a6cfab92807201180c1cde4447d50fc7bd991bde8f2f6a869a84e2ff45a",
+  "tp": 2
+}

hybrid/tails/tr3/layer-26/expert-102.json

@@ -0,0 +1,82 @@
+{
+  "bits": 3,
+  "bytes": 3584600,
+  "calibration_rows": 2048,
+  "calibration_sampling": "deterministic evenly spaced routed-row indices",
+  "capture_manifest_sha256": "976ba9935c2ae028117276f32bd33195862733a2b76cacb5514d3f2c2fb12a69",
+  "encoder_script_sha256": "30ccbf529dcab421ec904d8069a0ef33714d25408abf615ff76fccbcffae6b44",
+  "expert": 102,
+  "fallback_slices": 0,
+  "layer": 26,
+  "output": "/out/tr3/layer-26/expert-102.safetensors",
+  "routed_observation_rows_total": "8287",
+  "schema": "laguna-exl3-expert-encode/v1",
+  "sha256": "6993d14b66cc930c0c7796e0e8ed7c9ce0fbcb7d99e4bcb37c66210b3392a431",
+  "slices": {
+    "down_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.011878555360705884,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "down_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9362772223995108,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.01177035119606449,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9397321041054103,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0031464580340177586,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9543672178658202,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0031737514982912362,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.00018310546875,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9250969903450001,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.004056537001417225,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.003995592575612048,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    }
+  },
+  "source_index_sha256": "c872bd9b5ac116cebfd78a5f27565787dbceae76985cef041179110d0b03cd2b",
+  "status": "complete",
+  "tier": "TR3",
+  "tier_map_sha256": "98bb2a6cfab92807201180c1cde4447d50fc7bd991bde8f2f6a869a84e2ff45a",
+  "tp": 2
+}

hybrid/tails/tr3/layer-26/expert-107.json

@@ -0,0 +1,82 @@
+{
+  "bits": 3,
+  "bytes": 3584600,
+  "calibration_rows": 2048,
+  "calibration_sampling": "deterministic evenly spaced routed-row indices",
+  "capture_manifest_sha256": "976ba9935c2ae028117276f32bd33195862733a2b76cacb5514d3f2c2fb12a69",
+  "encoder_script_sha256": "30ccbf529dcab421ec904d8069a0ef33714d25408abf615ff76fccbcffae6b44",
+  "expert": 107,
+  "fallback_slices": 0,
+  "layer": 26,
+  "output": "/out/tr3/layer-26/expert-107.safetensors",
+  "routed_observation_rows_total": "15992",
+  "schema": "laguna-exl3-expert-encode/v1",
+  "sha256": "efd2b628067d6097dccd9e0c13f982d17d410517b5378a77450f7326333c3a29",
+  "slices": {
+    "down_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.011894459825396603,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "down_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9599573338930756,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.011983845460879472,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9397321041054102,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.003291044465538201,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9362772223995108,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.003412380821643795,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9397321041054103,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.003734575489724514,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9543672178658202,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.003833148055553648,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    }
+  },
+  "source_index_sha256": "c872bd9b5ac116cebfd78a5f27565787dbceae76985cef041179110d0b03cd2b",
+  "status": "complete",
+  "tier": "TR3",
+  "tier_map_sha256": "98bb2a6cfab92807201180c1cde4447d50fc7bd991bde8f2f6a869a84e2ff45a",
+  "tp": 2
+}

hybrid/tails/tr3/layer-26/expert-112.json

@@ -0,0 +1,82 @@
+{
+  "bits": 3,
+  "bytes": 3584600,
+  "calibration_rows": 2048,
+  "calibration_sampling": "deterministic evenly spaced routed-row indices",
+  "capture_manifest_sha256": "976ba9935c2ae028117276f32bd33195862733a2b76cacb5514d3f2c2fb12a69",
+  "encoder_script_sha256": "30ccbf529dcab421ec904d8069a0ef33714d25408abf615ff76fccbcffae6b44",
+  "expert": 112,
+  "fallback_slices": 0,
+  "layer": 26,
+  "output": "/out/tr3/layer-26/expert-112.safetensors",
+  "routed_observation_rows_total": "5055",
+  "schema": "laguna-exl3-expert-encode/v1",
+  "sha256": "557fe6f29a1b5a8c0dac2b61bf2d8b7db767984824eabd7fa2984c43b0b6f16a",
+  "slices": {
+    "down_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.009245232755793815,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "down_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.010273713108750577,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.003566257270597336,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.003615594251938286,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9306871063722555,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.004498458901675987,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9397321041054102,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.004479393538456486,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    }
+  },
+  "source_index_sha256": "c872bd9b5ac116cebfd78a5f27565787dbceae76985cef041179110d0b03cd2b",
+  "status": "complete",
+  "tier": "TR3",
+  "tier_map_sha256": "98bb2a6cfab92807201180c1cde4447d50fc7bd991bde8f2f6a869a84e2ff45a",
+  "tp": 2
+}

hybrid/tails/tr3/layer-26/expert-129.json

@@ -0,0 +1,82 @@
+{
+  "bits": 3,
+  "bytes": 3584600,
+  "calibration_rows": 2048,
+  "calibration_sampling": "deterministic evenly spaced routed-row indices",
+  "capture_manifest_sha256": "976ba9935c2ae028117276f32bd33195862733a2b76cacb5514d3f2c2fb12a69",
+  "encoder_script_sha256": "30ccbf529dcab421ec904d8069a0ef33714d25408abf615ff76fccbcffae6b44",
+  "expert": 129,
+  "fallback_slices": 0,
+  "layer": 26,
+  "output": "/out/tr3/layer-26/expert-129.safetensors",
+  "routed_observation_rows_total": "8855",
+  "schema": "laguna-exl3-expert-encode/v1",
+  "sha256": "e5be4d2ad1e51a29850706541209c3829d55874336d56e18e0066160f04f3095",
+  "slices": {
+    "down_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.012331515789121667,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "down_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.012238527082040129,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0032811251152776834,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9599573338930756,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0033327011440708355,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0047660500765981115,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9397321041054102,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.004690801732999302,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    }
+  },
+  "source_index_sha256": "c872bd9b5ac116cebfd78a5f27565787dbceae76985cef041179110d0b03cd2b",
+  "status": "complete",
+  "tier": "TR3",
+  "tier_map_sha256": "98bb2a6cfab92807201180c1cde4447d50fc7bd991bde8f2f6a869a84e2ff45a",
+  "tp": 2
+}

hybrid/tails/tr3/layer-26/expert-132.json

@@ -0,0 +1,82 @@
+{
+  "bits": 3,
+  "bytes": 3584600,
+  "calibration_rows": 2048,
+  "calibration_sampling": "deterministic evenly spaced routed-row indices",
+  "capture_manifest_sha256": "976ba9935c2ae028117276f32bd33195862733a2b76cacb5514d3f2c2fb12a69",
+  "encoder_script_sha256": "30ccbf529dcab421ec904d8069a0ef33714d25408abf615ff76fccbcffae6b44",
+  "expert": 132,
+  "fallback_slices": 0,
+  "layer": 26,
+  "output": "/out/tr3/layer-26/expert-132.safetensors",
+  "routed_observation_rows_total": "4585",
+  "schema": "laguna-exl3-expert-encode/v1",
+  "sha256": "7fa90ebc79f7c3c4eaf65c33c63d60c4f5925ab0307d1dd586a6d20e69036064",
+  "slices": {
+    "down_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9397321041054102,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.011349068188812764,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "down_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.011698707323318172,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.002344417538619204,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9397321041054102,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0023378519653806896,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9543672178658202,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.003004228855430139,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9599573338930756,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0029677362662067315,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    }
+  },
+  "source_index_sha256": "c872bd9b5ac116cebfd78a5f27565787dbceae76985cef041179110d0b03cd2b",
+  "status": "complete",
+  "tier": "TR3",
+  "tier_map_sha256": "98bb2a6cfab92807201180c1cde4447d50fc7bd991bde8f2f6a869a84e2ff45a",
+  "tp": 2
+}

hybrid/tails/tr3/layer-26/expert-135.json

@@ -0,0 +1,82 @@
+{
+  "bits": 3,
+  "bytes": 3584600,
+  "calibration_rows": 2048,
+  "calibration_sampling": "deterministic evenly spaced routed-row indices",
+  "capture_manifest_sha256": "976ba9935c2ae028117276f32bd33195862733a2b76cacb5514d3f2c2fb12a69",
+  "encoder_script_sha256": "30ccbf529dcab421ec904d8069a0ef33714d25408abf615ff76fccbcffae6b44",
+  "expert": 135,
+  "fallback_slices": 0,
+  "layer": 26,
+  "output": "/out/tr3/layer-26/expert-135.safetensors",
+  "routed_observation_rows_total": "7041",
+  "schema": "laguna-exl3-expert-encode/v1",
+  "sha256": "04569dd410d6cf3b5c7f1fa7de40e4dff2c51fc58eed373dd42bb5f59dd3ea7e",
+  "slices": {
+    "down_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.01314117071913363,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "down_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9543672178658202,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.013159933190621004,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.003780736590079384,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.003786506991632136,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.00432152794586183,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9306871063722555,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.00435990242892599,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    }
+  },
+  "source_index_sha256": "c872bd9b5ac116cebfd78a5f27565787dbceae76985cef041179110d0b03cd2b",
+  "status": "complete",
+  "tier": "TR3",
+  "tier_map_sha256": "98bb2a6cfab92807201180c1cde4447d50fc7bd991bde8f2f6a869a84e2ff45a",
+  "tp": 2
+}

hybrid/tails/tr3/layer-26/expert-138.json

@@ -0,0 +1,82 @@
+{
+  "bits": 3,
+  "bytes": 3584600,
+  "calibration_rows": 2048,
+  "calibration_sampling": "deterministic evenly spaced routed-row indices",
+  "capture_manifest_sha256": "976ba9935c2ae028117276f32bd33195862733a2b76cacb5514d3f2c2fb12a69",
+  "encoder_script_sha256": "30ccbf529dcab421ec904d8069a0ef33714d25408abf615ff76fccbcffae6b44",
+  "expert": 138,
+  "fallback_slices": 0,
+  "layer": 26,
+  "output": "/out/tr3/layer-26/expert-138.safetensors",
+  "routed_observation_rows_total": "4291",
+  "schema": "laguna-exl3-expert-encode/v1",
+  "sha256": "76e4d1913f7db9981e8f0efa555405443c754abaa58fa212bb88e53d21410100",
+  "slices": {
+    "down_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9397321041054102,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.011041938259091574,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "down_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.011210197693413073,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9397321041054103,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.001253200723420547,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.001261341186704722,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.004417570643665325,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.004521752059385283,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.00018310546875,
+      "reconstruction_mismatch_ratio": 0.0
+    }
+  },
+  "source_index_sha256": "c872bd9b5ac116cebfd78a5f27565787dbceae76985cef041179110d0b03cd2b",
+  "status": "complete",
+  "tier": "TR3",
+  "tier_map_sha256": "98bb2a6cfab92807201180c1cde4447d50fc7bd991bde8f2f6a869a84e2ff45a",
+  "tp": 2
+}

hybrid/tails/tr3/layer-26/expert-140.json

@@ -0,0 +1,82 @@
+{
+  "bits": 3,
+  "bytes": 3584600,
+  "calibration_rows": 2048,
+  "calibration_sampling": "deterministic evenly spaced routed-row indices",
+  "capture_manifest_sha256": "976ba9935c2ae028117276f32bd33195862733a2b76cacb5514d3f2c2fb12a69",
+  "encoder_script_sha256": "30ccbf529dcab421ec904d8069a0ef33714d25408abf615ff76fccbcffae6b44",
+  "expert": 140,
+  "fallback_slices": 0,
+  "layer": 26,
+  "output": "/out/tr3/layer-26/expert-140.safetensors",
+  "routed_observation_rows_total": "8099",
+  "schema": "laguna-exl3-expert-encode/v1",
+  "sha256": "27c61aa4b5eced20b5c06fe7c7aa8f5aa79115b419c59d8837cf664eda4953c8",
+  "slices": {
+    "down_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.010973506476197844,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "down_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9216421086391007,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.011077125283919242,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9362772223995108,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.002200342451839082,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9397321041054103,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0023201265665101617,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.00018310546875,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0028949514416910576,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.002856161038369223,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    }
+  },
+  "source_index_sha256": "c872bd9b5ac116cebfd78a5f27565787dbceae76985cef041179110d0b03cd2b",
+  "status": "complete",
+  "tier": "TR3",
+  "tier_map_sha256": "98bb2a6cfab92807201180c1cde4447d50fc7bd991bde8f2f6a869a84e2ff45a",
+  "tp": 2
+}

hybrid/tails/tr3/layer-26/expert-151.json

@@ -0,0 +1,82 @@
+{
+  "bits": 3,
+  "bytes": 3584600,
+  "calibration_rows": 2048,
+  "calibration_sampling": "deterministic evenly spaced routed-row indices",
+  "capture_manifest_sha256": "976ba9935c2ae028117276f32bd33195862733a2b76cacb5514d3f2c2fb12a69",
+  "encoder_script_sha256": "30ccbf529dcab421ec904d8069a0ef33714d25408abf615ff76fccbcffae6b44",
+  "expert": 151,
+  "fallback_slices": 0,
+  "layer": 26,
+  "output": "/out/tr3/layer-26/expert-151.safetensors",
+  "routed_observation_rows_total": "8691",
+  "schema": "laguna-exl3-expert-encode/v1",
+  "sha256": "ce94186b6e32ea5dd055533e786241f9af36a3175044d85d4e0a04dfa6d0f466",
+  "slices": {
+    "down_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9543672178658202,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.012372725387243612,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "down_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.012176130903093435,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.003371398938060717,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0033323399808304678,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.003809018373833184,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.003777047238847949,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    }
+  },
+  "source_index_sha256": "c872bd9b5ac116cebfd78a5f27565787dbceae76985cef041179110d0b03cd2b",
+  "status": "complete",
+  "tier": "TR3",
+  "tier_map_sha256": "98bb2a6cfab92807201180c1cde4447d50fc7bd991bde8f2f6a869a84e2ff45a",
+  "tp": 2
+}

hybrid/tails/tr3/layer-26/expert-164.json

@@ -0,0 +1,82 @@
+{
+  "bits": 3,
+  "bytes": 3584600,
+  "calibration_rows": 2048,
+  "calibration_sampling": "deterministic evenly spaced routed-row indices",
+  "capture_manifest_sha256": "976ba9935c2ae028117276f32bd33195862733a2b76cacb5514d3f2c2fb12a69",
+  "encoder_script_sha256": "30ccbf529dcab421ec904d8069a0ef33714d25408abf615ff76fccbcffae6b44",
+  "expert": 164,
+  "fallback_slices": 0,
+  "layer": 26,
+  "output": "/out/tr3/layer-26/expert-164.safetensors",
+  "routed_observation_rows_total": "24752",
+  "schema": "laguna-exl3-expert-encode/v1",
+  "sha256": "2f2dc7d83238a8a8016e7fbf43fa19b18de8f34fb393d33e09eb51294928efec",
+  "slices": {
+    "down_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9362772223995108,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.012347316276682915,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "down_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9397321041054102,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.01275300844718002,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0037603398536440627,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9397321041054102,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0038696171192667463,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9543672178658202,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.00430643569536159,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9543672178658202,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.004361414311432928,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    }
+  },
+  "source_index_sha256": "c872bd9b5ac116cebfd78a5f27565787dbceae76985cef041179110d0b03cd2b",
+  "status": "complete",
+  "tier": "TR3",
+  "tier_map_sha256": "98bb2a6cfab92807201180c1cde4447d50fc7bd991bde8f2f6a869a84e2ff45a",
+  "tp": 2
+}

hybrid/tails/tr3/layer-26/expert-165.json

@@ -0,0 +1,82 @@
+{
+  "bits": 3,
+  "bytes": 3584600,
+  "calibration_rows": 2048,
+  "calibration_sampling": "deterministic evenly spaced routed-row indices",
+  "capture_manifest_sha256": "976ba9935c2ae028117276f32bd33195862733a2b76cacb5514d3f2c2fb12a69",
+  "encoder_script_sha256": "30ccbf529dcab421ec904d8069a0ef33714d25408abf615ff76fccbcffae6b44",
+  "expert": 165,
+  "fallback_slices": 0,
+  "layer": 26,
+  "output": "/out/tr3/layer-26/expert-165.safetensors",
+  "routed_observation_rows_total": "6391",
+  "schema": "laguna-exl3-expert-encode/v1",
+  "sha256": "cc910a6a81a90690771d7da40fa43c30d709cf9a5802e65e1af5651adbb97348",
+  "slices": {
+    "down_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9362772223995108,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.011168691747213624,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "down_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9306871063722555,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.011067914371880346,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9216421086391007,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.002239688514422917,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9599573338930756,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0022020749643125886,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9543672178658202,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.003005852833459597,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9634122155989749,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0029746279697012586,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    }
+  },
+  "source_index_sha256": "c872bd9b5ac116cebfd78a5f27565787dbceae76985cef041179110d0b03cd2b",
+  "status": "complete",
+  "tier": "TR3",
+  "tier_map_sha256": "98bb2a6cfab92807201180c1cde4447d50fc7bd991bde8f2f6a869a84e2ff45a",
+  "tp": 2
+}

hybrid/tails/tr3/layer-26/expert-184.json

@@ -0,0 +1,82 @@
+{
+  "bits": 3,
+  "bytes": 3584600,
+  "calibration_rows": 2048,
+  "calibration_sampling": "deterministic evenly spaced routed-row indices",
+  "capture_manifest_sha256": "976ba9935c2ae028117276f32bd33195862733a2b76cacb5514d3f2c2fb12a69",
+  "encoder_script_sha256": "30ccbf529dcab421ec904d8069a0ef33714d25408abf615ff76fccbcffae6b44",
+  "expert": 184,
+  "fallback_slices": 0,
+  "layer": 26,
+  "output": "/out/tr3/layer-26/expert-184.safetensors",
+  "routed_observation_rows_total": "3877",
+  "schema": "laguna-exl3-expert-encode/v1",
+  "sha256": "0097b58998b515ea0bbd8f0e1e1a2c6a9952c6e5b0e6c8f99690dffcef543e98",
+  "slices": {
+    "down_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.012838662769301299,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "down_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9362772223995108,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.012657011176535674,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9599573338930756,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0034886447367760056,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9543672178658202,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0034523103501488198,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.00472809139818344,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.004726522159240503,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    }
+  },
+  "source_index_sha256": "c872bd9b5ac116cebfd78a5f27565787dbceae76985cef041179110d0b03cd2b",
+  "status": "complete",
+  "tier": "TR3",
+  "tier_map_sha256": "98bb2a6cfab92807201180c1cde4447d50fc7bd991bde8f2f6a869a84e2ff45a",
+  "tp": 2
+}

hybrid/tails/tr3/layer-26/expert-189.json

@@ -0,0 +1,82 @@
+{
+  "bits": 3,
+  "bytes": 3584600,
+  "calibration_rows": 2048,
+  "calibration_sampling": "deterministic evenly spaced routed-row indices",
+  "capture_manifest_sha256": "976ba9935c2ae028117276f32bd33195862733a2b76cacb5514d3f2c2fb12a69",
+  "encoder_script_sha256": "30ccbf529dcab421ec904d8069a0ef33714d25408abf615ff76fccbcffae6b44",
+  "expert": 189,
+  "fallback_slices": 0,
+  "layer": 26,
+  "output": "/out/tr3/layer-26/expert-189.safetensors",
+  "routed_observation_rows_total": "4906",
+  "schema": "laguna-exl3-expert-encode/v1",
+  "sha256": "56da71ac7d6e37678d5d47e71c61ccbdbd7c9ec7eacc5a554a33bea54059325f",
+  "slices": {
+    "down_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9362772223995108,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.01209903249192329,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "down_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9543672178658202,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.011807675405275792,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9362772223995108,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0031502343327367882,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.003166299844559142,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9362772223995108,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.004337512502993425,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9543672178658202,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.004420601067241631,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    }
+  },
+  "source_index_sha256": "c872bd9b5ac116cebfd78a5f27565787dbceae76985cef041179110d0b03cd2b",
+  "status": "complete",
+  "tier": "TR3",
+  "tier_map_sha256": "98bb2a6cfab92807201180c1cde4447d50fc7bd991bde8f2f6a869a84e2ff45a",
+  "tp": 2
+}

hybrid/tails/tr3/layer-26/expert-194.json

@@ -0,0 +1,82 @@
+{
+  "bits": 3,
+  "bytes": 3584600,
+  "calibration_rows": 2048,
+  "calibration_sampling": "deterministic evenly spaced routed-row indices",
+  "capture_manifest_sha256": "976ba9935c2ae028117276f32bd33195862733a2b76cacb5514d3f2c2fb12a69",
+  "encoder_script_sha256": "30ccbf529dcab421ec904d8069a0ef33714d25408abf615ff76fccbcffae6b44",
+  "expert": 194,
+  "fallback_slices": 0,
+  "layer": 26,
+  "output": "/out/tr3/layer-26/expert-194.safetensors",
+  "routed_observation_rows_total": "5775",
+  "schema": "laguna-exl3-expert-encode/v1",
+  "sha256": "45adaaeb52aea1af74438bc89d43bddc79bd6f265367dd46a1cb2c88526dc8dd",
+  "slices": {
+    "down_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.012321118066526844,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "down_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.012036087436401529,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0033936758210734034,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9397321041054103,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.003432042814141155,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0043342974892542395,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9362772223995108,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.004325905997546744,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    }
+  },
+  "source_index_sha256": "c872bd9b5ac116cebfd78a5f27565787dbceae76985cef041179110d0b03cd2b",
+  "status": "complete",
+  "tier": "TR3",
+  "tier_map_sha256": "98bb2a6cfab92807201180c1cde4447d50fc7bd991bde8f2f6a869a84e2ff45a",
+  "tp": 2
+}

hybrid/tails/tr3/layer-26/expert-199.json

@@ -0,0 +1,82 @@
+{
+  "bits": 3,
+  "bytes": 3584600,
+  "calibration_rows": 2048,
+  "calibration_sampling": "deterministic evenly spaced routed-row indices",
+  "capture_manifest_sha256": "976ba9935c2ae028117276f32bd33195862733a2b76cacb5514d3f2c2fb12a69",
+  "encoder_script_sha256": "30ccbf529dcab421ec904d8069a0ef33714d25408abf615ff76fccbcffae6b44",
+  "expert": 199,
+  "fallback_slices": 0,
+  "layer": 26,
+  "output": "/out/tr3/layer-26/expert-199.safetensors",
+  "routed_observation_rows_total": "2450",
+  "schema": "laguna-exl3-expert-encode/v1",
+  "sha256": "6bad28e33150695c328c9c1c3d024e24fd4114bcc0199d0c4795bcb739909e0b",
+  "slices": {
+    "down_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.012850600932133565,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "down_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9362772223995108,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.012772774469988304,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0035834865980615555,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9250969903450001,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0035417390848722875,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.00398742502383989,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9543672178658202,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.003993167261212206,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    }
+  },
+  "source_index_sha256": "c872bd9b5ac116cebfd78a5f27565787dbceae76985cef041179110d0b03cd2b",
+  "status": "complete",
+  "tier": "TR3",
+  "tier_map_sha256": "98bb2a6cfab92807201180c1cde4447d50fc7bd991bde8f2f6a869a84e2ff45a",
+  "tp": 2
+}

hybrid/tails/tr3/layer-26/expert-204.json

@@ -0,0 +1,82 @@
+{
+  "bits": 3,
+  "bytes": 3584600,
+  "calibration_rows": 2048,
+  "calibration_sampling": "deterministic evenly spaced routed-row indices",
+  "capture_manifest_sha256": "976ba9935c2ae028117276f32bd33195862733a2b76cacb5514d3f2c2fb12a69",
+  "encoder_script_sha256": "30ccbf529dcab421ec904d8069a0ef33714d25408abf615ff76fccbcffae6b44",
+  "expert": 204,
+  "fallback_slices": 0,
+  "layer": 26,
+  "output": "/out/tr3/layer-26/expert-204.safetensors",
+  "routed_observation_rows_total": "8520",
+  "schema": "laguna-exl3-expert-encode/v1",
+  "sha256": "0088b2f2c45b6db2617225395f62ce527133cd0ade9bcb1f2d923189929a6874",
+  "slices": {
+    "down_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9397321041054102,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.011788392009966795,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "down_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.01147542452937945,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.002699792380161709,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.002669911977152665,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9397321041054102,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.004358584819215757,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.004334854962648009,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.000244140625,
+      "reconstruction_mismatch_ratio": 0.0
+    }
+  },
+  "source_index_sha256": "c872bd9b5ac116cebfd78a5f27565787dbceae76985cef041179110d0b03cd2b",
+  "status": "complete",
+  "tier": "TR3",
+  "tier_map_sha256": "98bb2a6cfab92807201180c1cde4447d50fc7bd991bde8f2f6a869a84e2ff45a",
+  "tp": 2
+}

hybrid/tails/tr3/layer-26/expert-206.json

@@ -0,0 +1,82 @@
+{
+  "bits": 3,
+  "bytes": 3584600,
+  "calibration_rows": 2048,
+  "calibration_sampling": "deterministic evenly spaced routed-row indices",
+  "capture_manifest_sha256": "976ba9935c2ae028117276f32bd33195862733a2b76cacb5514d3f2c2fb12a69",
+  "encoder_script_sha256": "30ccbf529dcab421ec904d8069a0ef33714d25408abf615ff76fccbcffae6b44",
+  "expert": 206,
+  "fallback_slices": 0,
+  "layer": 26,
+  "output": "/out/tr3/layer-26/expert-206.safetensors",
+  "routed_observation_rows_total": "7293",
+  "schema": "laguna-exl3-expert-encode/v1",
+  "sha256": "78a2af944a45683ad8fef5de60949309edbff588a64c3b4059aa64260dd16a99",
+  "slices": {
+    "down_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.009819756574190895,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "down_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.009307429208444613,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.00334784260019728,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "gate_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9453222201326655,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.0033013031938355646,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank0": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9509123361599208,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.004271086510602213,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    },
+    "up_proj.rank1": {
+      "apply_out_scales": true,
+      "calibration_rows": 2048,
+      "g_scale": 0.9397321041054102,
+      "pack_unpack_pack_equal": true,
+      "proxy_error": 0.004335320221539458,
+      "q_fallback": false,
+      "reconstruction_max_abs": 0.0001220703125,
+      "reconstruction_mismatch_ratio": 0.0
+    }
+  },
+  "source_index_sha256": "c872bd9b5ac116cebfd78a5f27565787dbceae76985cef041179110d0b03cd2b",
+  "status": "complete",
+  "tier": "TR3",
+  "tier_map_sha256": "98bb2a6cfab92807201180c1cde4447d50fc7bd991bde8f2f6a869a84e2ff45a",
+  "tp": 2
+}

modeling_laguna.py

@@ -0,0 +1,886 @@
+# ruff: noqa
+# Copyright 2025 Poolside and the HuggingFace Inc. team. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from collections.abc import Callable
+
+import torch
+import torch.nn.functional as F
+from torch import nn
+
+from transformers.activations import ACT2FN
+from transformers.cache_utils import Cache
+from transformers.integrations import use_experts_implementation, use_kernelized_func
+from transformers.modeling_flash_attention_utils import FlashAttentionKwargs
+from transformers.modeling_layers import GradientCheckpointingLayer
+from transformers.modeling_outputs import MoeModelOutputWithPast
+from transformers.modeling_utils import ALL_ATTENTION_FUNCTIONS
+from transformers.processing_utils import Unpack
+from transformers.utils import auto_docstring, can_return_tuple, is_grouped_mm_available
+from transformers.utils.generic import TransformersKwargs, merge_with_config_defaults
+from transformers.utils.output_capturing import OutputRecorder, capture_outputs
+from transformers.cache_utils import DynamicCache
+from transformers.generation import GenerationMixin
+from transformers.integrations import use_kernel_forward_from_hub
+from transformers.masking_utils import create_causal_mask
+from transformers.modeling_rope_utils import ROPE_INIT_FUNCTIONS, dynamic_rope_update
+from transformers.modeling_utils import PreTrainedModel
+from transformers.utils.generic import maybe_autocast
+from .configuration_laguna import LagunaConfig
+
+from transformers import initialization as init
+from transformers.masking_utils import create_sliding_window_causal_mask
+from transformers.modeling_outputs import MoeCausalLMOutputWithPast
+from transformers.utils.import_utils import is_causal_conv1d_available, is_flash_linear_attention_available
+
+
+@use_kernel_forward_from_hub("RMSNorm")
+class LagunaRMSNorm(nn.Module):
+    def __init__(self, hidden_size, eps: float = 1e-6) -> None:
+        """
+        LagunaRMSNorm is equivalent to T5LayerNorm
+        """
+        super().__init__()
+        self.weight = nn.Parameter(torch.ones(hidden_size))
+        self.variance_epsilon = eps
+
+    def forward(self, hidden_states: torch.Tensor) -> torch.Tensor:
+        input_dtype = hidden_states.dtype
+        hidden_states = hidden_states.to(torch.float32)
+        variance = hidden_states.pow(2).mean(-1, keepdim=True)
+        hidden_states = hidden_states * torch.rsqrt(variance + self.variance_epsilon)
+        return self.weight * hidden_states.to(input_dtype)
+
+    def extra_repr(self):
+        return f"{tuple(self.weight.shape)}, eps={self.variance_epsilon}"
+
+
+class LagunaRotaryEmbedding(nn.Module):
+    inv_freq: torch.Tensor  # fix linting for `register_buffer`
+
+    def __init__(self, config: LagunaConfig, device=None):
+        super().__init__()
+        self.max_seq_len_cached = config.max_position_embeddings
+        self.original_max_seq_len = config.max_position_embeddings
+
+        self.config = config
+
+        self.rope_type = self.config.rope_parameters["rope_type"]
+        rope_init_fn: Callable = self.compute_default_rope_parameters
+        if self.rope_type != "default":
+            rope_init_fn = ROPE_INIT_FUNCTIONS[self.rope_type]
+        inv_freq, self.attention_scaling = rope_init_fn(self.config, device)
+
+        self.register_buffer("inv_freq", inv_freq, persistent=False)
+        self.register_buffer("original_inv_freq", inv_freq.clone(), persistent=False)
+
+    @staticmethod
+    def compute_default_rope_parameters(config, device=None, seq_len=None) -> tuple["torch.Tensor", float]:
+        """
+        Computes the inverse frequencies according to the original RoPE implementation
+        Args:
+            config ([`~transformers.PreTrainedConfig`]):
+                The model configuration.
+            device (`torch.device`):
+                The device to use for initialization of the inverse frequencies.
+            seq_len (`int`, *optional*):
+                The current sequence length. Unused for this type of RoPE.
+        Returns:
+            Tuple of (`torch.Tensor`, `float`), containing the inverse frequencies for the RoPE embeddings and the
+            post-processing scaling factor applied to the computed cos/sin (unused in this type of RoPE).
+        """
+        base = config.rope_parameters["rope_theta"]
+        head_dim = getattr(config, "head_dim", None) or config.hidden_size // config.num_attention_heads
+        partial = config.rope_parameters.get("partial_rotary_factor", 1.0)
+        dim = int(head_dim * partial)
+        inv_freq = 1.0 / (
+            base ** (torch.arange(0, dim, 2, dtype=torch.int64).to(device=device, dtype=torch.float) / dim)
+        )
+        return inv_freq, 1.0
+
+    @torch.no_grad()
+    @dynamic_rope_update  # power user: used with advanced RoPE types (e.g. dynamic rope)
+    def forward(self, x, position_ids):
+        inv_freq_expanded = self.inv_freq[None, :, None].float().expand(position_ids.shape[0], -1, 1).to(x.device)
+        position_ids_expanded = position_ids[:, None, :].float()
+
+        device_type = x.device.type if isinstance(x.device.type, str) and x.device.type != "mps" else "cpu"
+        with maybe_autocast(device_type=device_type, enabled=False):  # Force float32
+            freqs = (inv_freq_expanded.float() @ position_ids_expanded.float()).transpose(1, 2)
+            emb = torch.cat((freqs, freqs), dim=-1)
+            cos = emb.cos() * self.attention_scaling
+            sin = emb.sin() * self.attention_scaling
+
+        return cos.to(dtype=x.dtype), sin.to(dtype=x.dtype)
+
+
+class LagunaMLP(nn.Module):
+    def __init__(self, config, intermediate_size=None):
+        super().__init__()
+        self.config = config
+        self.hidden_size = config.hidden_size
+        self.intermediate_size = config.intermediate_size if intermediate_size is None else intermediate_size
+        self.gate_proj = nn.Linear(self.hidden_size, self.intermediate_size, bias=False)
+        self.up_proj = nn.Linear(self.hidden_size, self.intermediate_size, bias=False)
+        self.down_proj = nn.Linear(self.intermediate_size, self.hidden_size, bias=False)
+        self.act_fn = ACT2FN[config.hidden_act]
+
+    def forward(self, x):
+        down_proj = self.down_proj(self.act_fn(self.gate_proj(x)) * self.up_proj(x))
+        return down_proj
+
+
+class LagunaTopKRouter(nn.Module):
+    """Laguna MoE router using sigmoid scoring (not softmax).
+
+    Supports optional router-logit soft-capping and auxiliary-loss-free load
+    balancing (arXiv:2408.15664): the per-expert bias ``e_score_correction_bias``
+    is added to selection scores but the returned routing weights remain unbiased.
+    The bias lives on the router so accelerate's per-module hooks can co-locate it
+    with the gate — moving it to the experts module would cross a hook boundary
+    and leave the bias on meta under ``device_map="auto"`` / CPU-offload.
+    """
+
+    def __init__(self, config):
+        super().__init__()
+        self.top_k = config.num_experts_per_tok
+        self.num_experts = config.num_experts
+        self.norm_topk_prob = config.norm_topk_prob
+        self.hidden_dim = config.hidden_size
+        self.weight = nn.Parameter(torch.zeros(self.num_experts, self.hidden_dim))
+        # Zero-initialised so inference on checkpoints that don't ship the bias
+        # is a no-op. ``_checkpoint_conversion_mapping`` below remaps the
+        # ``mlp.experts.e_score_correction_bias`` key from vLLM-trained
+        # checkpoints onto this attribute.
+        self.e_score_correction_bias = nn.Parameter(torch.zeros(config.num_experts), requires_grad=False)
+        self.router_logit_softcapping = float(getattr(config, "moe_router_logit_softcapping", 0.0) or 0.0)
+
+    def forward(
+        self,
+        hidden_states: torch.Tensor,
+    ) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor]:
+        hidden_states = hidden_states.reshape(-1, self.hidden_dim)
+        router_logits = F.linear(hidden_states, self.weight).float()
+        if self.router_logit_softcapping > 0.0:
+            router_logits = torch.tanh(router_logits / self.router_logit_softcapping) * self.router_logit_softcapping
+        routing_scores = torch.sigmoid(router_logits)
+        scores_for_selection = routing_scores + self.e_score_correction_bias.to(routing_scores.dtype)
+        _, selected_experts = torch.topk(scores_for_selection, self.top_k, dim=-1)
+        routing_weights = routing_scores.gather(-1, selected_experts)
+        if self.norm_topk_prob:
+            routing_weights = routing_weights / routing_weights.sum(dim=-1, keepdim=True)
+        routing_weights = routing_weights.to(hidden_states.dtype)
+        return router_logits, routing_weights, selected_experts
+
+
+@use_experts_implementation
+class LagunaExperts(nn.Module):
+    """Fused expert weights as 3D tensors for batched execution."""
+
+    def __init__(self, config):
+        super().__init__()
+        self.num_experts = config.num_experts
+        self.hidden_dim = config.hidden_size
+        self.intermediate_dim = config.moe_intermediate_size
+        self.gate_up_proj = nn.Parameter(torch.empty(self.num_experts, 2 * self.intermediate_dim, self.hidden_dim))
+        self.down_proj = nn.Parameter(torch.empty(self.num_experts, self.hidden_dim, self.intermediate_dim))
+        self.act_fn = ACT2FN[config.hidden_act]
+
+    def forward(
+        self,
+        hidden_states: torch.Tensor,
+        top_k_index: torch.Tensor,
+        top_k_weights: torch.Tensor,
+    ) -> torch.Tensor:
+        final_hidden_states = torch.zeros_like(hidden_states)
+        with torch.no_grad():
+            expert_mask = F.one_hot(top_k_index, num_classes=self.num_experts)
+            expert_mask = expert_mask.permute(2, 1, 0)
+            expert_hit = torch.greater(expert_mask.sum(dim=(-1, -2)), 0).nonzero()
+
+        for expert_idx in expert_hit:
+            expert_idx = expert_idx[0]
+            if expert_idx == self.num_experts:
+                continue
+            top_k_pos, token_idx = torch.where(expert_mask[expert_idx])
+            current_state = hidden_states[token_idx]
+            gate, up = F.linear(current_state, self.gate_up_proj[expert_idx]).chunk(2, dim=-1)
+            current_hidden_states = self.act_fn(gate) * up
+            current_hidden_states = F.linear(current_hidden_states, self.down_proj[expert_idx])
+            current_hidden_states = current_hidden_states * top_k_weights[token_idx, top_k_pos, None]
+            final_hidden_states.index_add_(0, token_idx, current_hidden_states.to(final_hidden_states.dtype))
+
+        return final_hidden_states
+
+
+class LagunaSparseMoeBlock(nn.Module):
+    """Laguna MoE block using sigmoid router, fused expert tensors, and a shared expert."""
+
+    def __init__(self, config):
+        super().__init__()
+        self.num_experts = config.num_experts
+        self.routed_scaling_factor = float(getattr(config, "moe_routed_scaling_factor", 1.0))
+        # ``moe_apply_router_weight_on_input=True`` would require scaling each expert's
+        # input (rather than its output) by the routing weight. Supporting it cleanly
+        # alongside the fused experts kernels (``grouped_mm`` / ``batched_mm``) is future
+        # work; for now we fail loudly so a checkpoint that needs it can't silently
+        # diverge from its numerical form.
+        if getattr(config, "moe_apply_router_weight_on_input", False):
+            raise NotImplementedError(
+                "moe_apply_router_weight_on_input=True is not yet supported in the "
+                "transformers implementation of Laguna."
+            )
+        self.gate = LagunaTopKRouter(config)
+        self.experts = LagunaExperts(config)
+        self.shared_expert = LagunaMLP(config, intermediate_size=config.shared_expert_intermediate_size)
+
+    def forward(self, hidden_states: torch.Tensor) -> torch.Tensor:
+        batch_size, sequence_length, hidden_dim = hidden_states.shape
+        hidden_states = hidden_states.view(-1, hidden_dim)
+
+        shared_expert_output = self.shared_expert(hidden_states)
+        _, routing_weights, selected_experts = self.gate(hidden_states)
+        expert_output = self.experts(hidden_states, selected_experts, routing_weights)
+        if self.routed_scaling_factor != 1.0:
+            expert_output = expert_output * self.routed_scaling_factor
+
+        expert_output = expert_output + shared_expert_output
+        expert_output = expert_output.reshape(batch_size, sequence_length, hidden_dim)
+        return expert_output
+
+
+def rotate_half(x):
+    """Rotates half the hidden dims of the input."""
+    x1 = x[..., : x.shape[-1] // 2]
+    x2 = x[..., x.shape[-1] // 2 :]
+    return torch.cat((-x2, x1), dim=-1)
+
+
+# Adapted from transformers.models.glm.modular_glm.apply_rotary_pos_emb
+def apply_rotary_pos_emb(q, k, cos, sin, unsqueeze_dim=1):
+    """Applies Rotary Position Embedding to the query and key tensors.
+
+    Removes the interleaving of cos and sin from GLM
+
+    Args:
+        q (`torch.Tensor`): The query tensor.
+        k (`torch.Tensor`): The key tensor.
+        cos (`torch.Tensor`): The cosine part of the rotary embedding.
+        sin (`torch.Tensor`): The sine part of the rotary embedding.
+        unsqueeze_dim (`int`, *optional*, defaults to 1):
+            The 'unsqueeze_dim' argument specifies the dimension along which to unsqueeze cos[position_ids] and
+            sin[position_ids] so that they can be properly broadcasted to the dimensions of q and k. For example, note
+            that cos[position_ids] and sin[position_ids] have the shape [batch_size, seq_len, head_dim]. Then, if q and
+            k have the shape [batch_size, heads, seq_len, head_dim], then setting unsqueeze_dim=1 makes
+            cos[position_ids] and sin[position_ids] broadcastable to the shapes of q and k. Similarly, if q and k have
+            the shape [batch_size, seq_len, heads, head_dim], then set unsqueeze_dim=2.
+    Returns:
+        `tuple(torch.Tensor)` comprising of the query and key tensors rotated using the Rotary Position Embedding.
+    """
+    cos = cos.unsqueeze(unsqueeze_dim)
+    sin = sin.unsqueeze(unsqueeze_dim)
+
+    # Keep half or full tensor for later concatenation
+    rotary_dim = cos.shape[-1]
+    q_rot, q_pass = q[..., :rotary_dim], q[..., rotary_dim:]
+    k_rot, k_pass = k[..., :rotary_dim], k[..., rotary_dim:]
+
+    # Apply rotary embeddings on the first half or full tensor
+    q_embed = (q_rot * cos) + (rotate_half(q_rot) * sin)
+    k_embed = (k_rot * cos) + (rotate_half(k_rot) * sin)
+
+    # Concatenate back to full shape
+    q_embed = torch.cat([q_embed, q_pass], dim=-1)
+    k_embed = torch.cat([k_embed, k_pass], dim=-1)
+    return q_embed, k_embed
+
+
+def repeat_kv(hidden_states: torch.Tensor, n_rep: int) -> torch.Tensor:
+    """
+    This is the equivalent of torch.repeat_interleave(x, dim=1, repeats=n_rep). The hidden states go from (batch,
+    num_key_value_heads, seqlen, head_dim) to (batch, num_attention_heads, seqlen, head_dim)
+    """
+    batch, num_key_value_heads, slen, head_dim = hidden_states.shape
+    if n_rep == 1:
+        return hidden_states
+    hidden_states = hidden_states[:, :, None, :, :].expand(batch, num_key_value_heads, n_rep, slen, head_dim)
+    return hidden_states.reshape(batch, num_key_value_heads * n_rep, slen, head_dim)
+
+
+def eager_attention_forward(
+    module: nn.Module,
+    query: torch.Tensor,
+    key: torch.Tensor,
+    value: torch.Tensor,
+    attention_mask: torch.Tensor | None,
+    scaling: float,
+    dropout: float = 0.0,
+    **kwargs: Unpack[TransformersKwargs],
+):
+    key_states = repeat_kv(key, module.num_key_value_groups)
+    value_states = repeat_kv(value, module.num_key_value_groups)
+
+    attn_weights = torch.matmul(query, key_states.transpose(2, 3)) * scaling
+    if attention_mask is not None:
+        attn_weights = attn_weights + attention_mask
+
+    attn_weights = nn.functional.softmax(attn_weights, dim=-1, dtype=torch.float32).to(query.dtype)
+    attn_weights = nn.functional.dropout(attn_weights, p=dropout, training=module.training)
+    attn_output = torch.matmul(attn_weights, value_states)
+    attn_output = attn_output.transpose(1, 2).contiguous()
+
+    return attn_output, attn_weights
+
+
+# Laguna attention is identical to Qwen2MoE attention except:
+# - No QKV bias
+# - Explicit head_dim from config
+# - Output gating: attn_output = attn_output * softplus(g_proj(hidden_states)) (optional)
+# - Per-layer sliding window attention with optional attention sinks
+@use_kernelized_func(apply_rotary_pos_emb)
+class LagunaAttention(nn.Module):
+    def __init__(self, config: LagunaConfig, layer_idx: int, num_heads: int | None = None):
+        super().__init__()
+        self.config = config
+        self.layer_idx = layer_idx
+        self.head_dim = config.head_dim
+        # Allow the caller (decoder layer) to supply a per-layer head count; fall back
+        # to config.num_attention_heads when not provided.
+        self.num_heads = num_heads if num_heads is not None else config.num_attention_heads
+        self.num_key_value_groups = self.num_heads // config.num_key_value_heads
+        self.scaling = self.head_dim**-0.5
+        self.attention_dropout = config.attention_dropout
+        self.is_causal = True
+
+        # Per-layer sliding window (follows Gemma2/Cohere2 convention)
+        layer_types = getattr(config, "layer_types", None)
+        if layer_types is not None:
+            self.is_sliding = layer_types[layer_idx] == "sliding_attention"
+            self.sliding_window = config.sliding_window if self.is_sliding else None
+        else:
+            self.is_sliding = False
+            self.sliding_window = None
+
+        # Laguna: no QKV bias, explicit head_dim
+        self.q_proj = nn.Linear(config.hidden_size, self.num_heads * config.head_dim, bias=False)
+        self.k_proj = nn.Linear(config.hidden_size, config.num_key_value_heads * config.head_dim, bias=False)
+        self.v_proj = nn.Linear(config.hidden_size, config.num_key_value_heads * config.head_dim, bias=False)
+        self.o_proj = nn.Linear(self.num_heads * config.head_dim, config.hidden_size, bias=False)
+
+        # Laguna-specific: optional gating projection.
+        # ``gating`` may be:
+        #   - True / "per-element": one gate per (head, head_dim) channel
+        #   - "per-head":           one gate per head, broadcast across head_dim
+        #   - False:                no gating
+        gating = getattr(config, "gating", True)
+        self.gating = bool(gating)
+        self.gate_per_head = gating == "per-head"
+        if self.gating:
+            g_out = self.num_heads if self.gate_per_head else self.num_heads * config.head_dim
+            self.g_proj = nn.Linear(config.hidden_size, g_out, bias=False)
+
+        # Attention sinks (learnable per-head bias for SWA layers)
+        if self.is_sliding and getattr(config, "swa_attention_sink_enabled", False):
+            self.sink = nn.Parameter(torch.zeros(self.num_heads))
+
+        # QK normalization (RMSNorm applied per-head after reshape, before RoPE)
+        self.q_norm = LagunaRMSNorm(config.head_dim, eps=config.rms_norm_eps)
+        self.k_norm = LagunaRMSNorm(config.head_dim, eps=config.rms_norm_eps)
+
+    def forward(
+        self,
+        hidden_states: torch.Tensor,
+        position_embeddings: tuple[torch.Tensor, torch.Tensor],
+        attention_mask: torch.Tensor | None,
+        past_key_values: Cache | None = None,
+        **kwargs: Unpack[FlashAttentionKwargs],
+    ) -> tuple[torch.Tensor, torch.Tensor | None]:
+        input_shape = hidden_states.shape[:-1]
+        hidden_shape = (*input_shape, -1, self.head_dim)
+
+        query_states = self.q_proj(hidden_states)
+        key_states = self.k_proj(hidden_states)
+        value_states = self.v_proj(hidden_states)
+
+        query_states = query_states.view(hidden_shape).transpose(1, 2)
+        key_states = key_states.view(hidden_shape).transpose(1, 2)
+        value_states = value_states.view(hidden_shape).transpose(1, 2)
+
+        # QK normalization (applied per-head before RoPE)
+        query_states = self.q_norm(query_states)
+        key_states = self.k_norm(key_states)
+
+        cos, sin = position_embeddings
+        query_states, key_states = apply_rotary_pos_emb(query_states, key_states, cos, sin)
+
+        if past_key_values is not None:
+            key_states, value_states = past_key_values.update(key_states, value_states, self.layer_idx)
+
+        # ``attention_mask`` here is already the correct mask for this layer type —
+        # ``LagunaModel.forward`` builds separate full-attention and sliding-attention
+        # masks (using ``create_causal_mask`` / ``create_sliding_window_causal_mask``)
+        # and the decoder layer passes the right one in.
+        attention_interface: Callable = eager_attention_forward
+        if self.config._attn_implementation != "eager":
+            attention_interface = ALL_ATTENTION_FUNCTIONS[self.config._attn_implementation]
+
+        attn_output, attn_weights = attention_interface(
+            self,
+            query_states,
+            key_states,
+            value_states,
+            attention_mask,
+            dropout=0.0 if not self.training else self.attention_dropout,
+            scaling=self.scaling,
+            **kwargs,
+        )
+
+        attn_output = attn_output.reshape(*input_shape, -1).contiguous()
+
+        # Laguna-specific: apply gating BEFORE o_proj (optional)
+        if self.gating:
+            gate = F.softplus(self.g_proj(hidden_states).float()).to(attn_output.dtype)
+            if self.gate_per_head:
+                # gate: [..., num_heads]; broadcast across head_dim
+                attn_shape = attn_output.shape
+                attn_output = (
+                    attn_output.view(*attn_shape[:-1], self.num_heads, self.head_dim) * gate.unsqueeze(-1)
+                ).view(attn_shape)
+            else:
+                attn_output = attn_output * gate
+
+        attn_output = self.o_proj(attn_output)
+
+        return attn_output, attn_weights
+
+
+class LagunaDecoderLayer(GradientCheckpointingLayer):
+    """Laguna decoder layer with gated attention and sigmoid-routed MoE."""
+
+    def __init__(self, config: LagunaConfig, layer_idx: int):
+        super().__init__()
+        per_layer_heads = getattr(config, "num_attention_heads_per_layer", None)
+        layer_num_heads = per_layer_heads[layer_idx] if per_layer_heads is not None else config.num_attention_heads
+        # Layer type drives mask and position-embedding dispatch in ``LagunaModel.forward``.
+        layer_types = getattr(config, "layer_types", None)
+        self.attention_type = layer_types[layer_idx] if layer_types is not None else "full_attention"
+        self.self_attn = LagunaAttention(config, layer_idx, num_heads=layer_num_heads)
+        # Use MoE or dense MLP based on layer configuration
+        if (layer_idx not in config.mlp_only_layers) and (
+            config.num_experts > 0 and (layer_idx + 1) % config.decoder_sparse_step == 0
+        ):
+            self.mlp = LagunaSparseMoeBlock(config)
+        else:
+            self.mlp = LagunaMLP(config, intermediate_size=config.intermediate_size)
+        self.input_layernorm = LagunaRMSNorm(config.hidden_size, eps=config.rms_norm_eps)
+        self.post_attention_layernorm = LagunaRMSNorm(config.hidden_size, eps=config.rms_norm_eps)
+        self.hidden_size = config.hidden_size
+
+    def forward(
+        self,
+        hidden_states: torch.Tensor,
+        attention_mask: torch.Tensor | None = None,
+        position_ids: torch.LongTensor | None = None,
+        past_key_values: Cache | None = None,
+        use_cache: bool | None = False,
+        position_embeddings: tuple[torch.Tensor, torch.Tensor] | None = None,
+        **kwargs: Unpack[TransformersKwargs],
+    ) -> torch.Tensor:
+        residual = hidden_states
+        hidden_states = self.input_layernorm(hidden_states)
+        # Self Attention
+        hidden_states, _ = self.self_attn(
+            hidden_states=hidden_states,
+            attention_mask=attention_mask,
+            position_ids=position_ids,
+            past_key_values=past_key_values,
+            use_cache=use_cache,
+            position_embeddings=position_embeddings,
+            **kwargs,
+        )
+        hidden_states = residual + hidden_states
+
+        # Fully Connected
+        residual = hidden_states
+        hidden_states = self.post_attention_layernorm(hidden_states)
+        hidden_states = self.mlp(hidden_states)
+        hidden_states = residual + hidden_states
+        return hidden_states
+
+
+@auto_docstring
+class LagunaPreTrainedModel(PreTrainedModel):
+    config: LagunaConfig
+    base_model_prefix = "model"
+    supports_gradient_checkpointing = True
+    _no_split_modules = ["LagunaDecoderLayer"]
+    _skip_keys_device_placement = ["past_key_values"]
+    _supports_flash_attn = True
+    _supports_sdpa = True
+    _supports_flex_attn = True
+    _can_compile_fullgraph = (
+        is_grouped_mm_available()
+    )  # https://huggingface.co/docs/transformers/experts_interface#torchcompile
+    _supports_attention_backend = True
+    _can_record_outputs = {
+        "router_logits": OutputRecorder(LagunaTopKRouter, index=0),
+        "hidden_states": LagunaDecoderLayer,
+        "attentions": LagunaAttention,
+    }
+    # vLLM-trained Laguna checkpoints store the aux-loss-free routing bias on the
+    # experts module (``mlp.experts.e_score_correction_bias``). In this impl the
+    # bias lives on the router to stay co-located with its consumer across
+    # accelerate's per-module hooks, so remap the legacy key on load.
+    _checkpoint_conversion_mapping = {
+        r"^(.*)\.mlp\.experts\.e_score_correction_bias$": r"\1.mlp.gate.e_score_correction_bias",
+    }
+
+    @torch.no_grad()
+    def _init_weights(self, module):
+        super()._init_weights(module)
+        std = self.config.initializer_range
+        if isinstance(module, LagunaExperts):
+            init.normal_(module.gate_up_proj, mean=0.0, std=std)
+            init.normal_(module.down_proj, mean=0.0, std=std)
+        elif isinstance(module, LagunaTopKRouter):
+            init.normal_(module.weight, mean=0.0, std=std)
+        # Bare ``nn.Parameter``s that are not covered by the parent's generic
+        # Linear/Embedding/norm handling need their own rules so that the
+        # __init__ and from_pretrained(state_dict={}) paths produce identical
+        # weights under a fixed seed.
+        if isinstance(module, LagunaTopKRouter):
+            torch.nn.init.zeros_(module.e_score_correction_bias)
+        if isinstance(module, LagunaAttention) and hasattr(module, "sink"):
+            torch.nn.init.zeros_(module.sink)
+
+
+class LagunaModel(LagunaPreTrainedModel):
+    def __init__(self, config: LagunaConfig):
+        super().__init__(config)
+        self.padding_idx = config.pad_token_id
+        self.vocab_size = config.vocab_size
+
+        self.embed_tokens = nn.Embedding(config.vocab_size, config.hidden_size, self.padding_idx)
+        self.layers = nn.ModuleList(
+            [LagunaDecoderLayer(config, layer_idx) for layer_idx in range(config.num_hidden_layers)]
+        )
+        self.norm = LagunaRMSNorm(config.hidden_size, eps=config.rms_norm_eps)
+
+        # ``LagunaRotaryEmbedding`` inherits ``Qwen2MoeRotaryEmbedding``'s flat-shape
+        # contract — it reads ``config.rope_parameters["rope_type"]`` at the outer
+        # level. Laguna stores rope nested by layer type (``{"full_attention": {...},
+        # ...}``), so pass a config clone with the full-attention sub-dict flattened.
+        rp = getattr(config, "rope_parameters", None)
+        if isinstance(rp, dict) and isinstance(rp.get("full_attention"), dict):
+            import copy
+
+            full_config = copy.deepcopy(config)
+            full_config.rope_parameters = dict(rp["full_attention"])
+            self.rotary_emb = LagunaRotaryEmbedding(config=full_config)
+        else:
+            self.rotary_emb = LagunaRotaryEmbedding(config=config)
+
+        # Separate RoPE for sliding-window attention layers (when configured).
+        # Be careful with ``partial_rotary_factor`` — ``PreTrainedConfig.standardize_rope_params``
+        # unconditionally overwrites ``rope_parameters["partial_rotary_factor"]`` with
+        # ``self.partial_rotary_factor``, so we must align the top-level field on the
+        # cloned config to the SWA value, otherwise the global partial factor silently
+        # clobbers the SWA one.
+        if getattr(config, "swa_rope_parameters", None) is not None:
+            import copy
+
+            swa_config = copy.deepcopy(config)
+            swa_config.rope_parameters = dict(config.swa_rope_parameters)
+            swa_partial = swa_config.rope_parameters.get("partial_rotary_factor")
+            swa_config.partial_rotary_factor = swa_partial
+            self.swa_rotary_emb = LagunaRotaryEmbedding(config=swa_config)
+        else:
+            self.swa_rotary_emb = None
+
+        self.gradient_checkpointing = False
+
+        # Initialize weights and apply final processing
+        self.post_init()
+
+    @merge_with_config_defaults
+    @capture_outputs
+    @auto_docstring
+    def forward(
+        self,
+        input_ids: torch.LongTensor | None = None,
+        attention_mask: torch.Tensor | None = None,
+        position_ids: torch.LongTensor | None = None,
+        past_key_values: Cache | None = None,
+        inputs_embeds: torch.FloatTensor | None = None,
+        use_cache: bool | None = None,
+        **kwargs: Unpack[TransformersKwargs],
+    ) -> MoeModelOutputWithPast:
+        from transformers.cache_utils import DynamicCache
+        from transformers.masking_utils import create_causal_mask, create_sliding_window_causal_mask
+
+        if (input_ids is None) ^ (inputs_embeds is not None):
+            raise ValueError("You must specify exactly one of input_ids or inputs_embeds")
+
+        if inputs_embeds is None:
+            inputs_embeds = self.embed_tokens(input_ids)
+
+        if use_cache and past_key_values is None:
+            past_key_values = DynamicCache(config=self.config)
+
+        if position_ids is None:
+            past_seen_tokens = past_key_values.get_seq_length() if past_key_values is not None else 0
+            position_ids = (
+                torch.arange(inputs_embeds.shape[1], device=inputs_embeds.device) + past_seen_tokens
+            ).unsqueeze(0)
+
+        # Build one mask per layer-type so each layer can be dispatched with the right
+        # attention pattern (follows the afmoe / cohere2 v5 convention).
+        layer_types = getattr(self.config, "layer_types", None)
+        has_swa = layer_types is not None and "sliding_attention" in layer_types
+        if not isinstance(causal_mask_mapping := attention_mask, dict):
+            mask_kwargs = {
+                "config": self.config,
+                "inputs_embeds": inputs_embeds,
+                "attention_mask": attention_mask,
+                "past_key_values": past_key_values,
+                "position_ids": position_ids,
+            }
+            causal_mask_mapping = {"full_attention": create_causal_mask(**mask_kwargs)}
+            if has_swa:
+                causal_mask_mapping["sliding_attention"] = create_sliding_window_causal_mask(**mask_kwargs)
+
+        hidden_states = inputs_embeds
+        global_pe = self.rotary_emb(hidden_states, position_ids)
+        # Per-layer-type position embeddings: Laguna optionally uses a different rope for
+        # sliding layers (``swa_rope_parameters``). When absent, SWA layers share the
+        # global rope.
+        if has_swa:
+            swa_pe = self.swa_rotary_emb(hidden_states, position_ids) if self.swa_rotary_emb is not None else global_pe
+            position_embeddings_mapping = {"full_attention": global_pe, "sliding_attention": swa_pe}
+        else:
+            position_embeddings_mapping = None
+
+        for decoder_layer in self.layers[: self.config.num_hidden_layers]:
+            layer_attn_mask = causal_mask_mapping[decoder_layer.attention_type]
+            layer_pos_emb = (
+                position_embeddings_mapping[decoder_layer.attention_type]
+                if position_embeddings_mapping is not None
+                else global_pe
+            )
+            hidden_states = decoder_layer(
+                hidden_states,
+                attention_mask=layer_attn_mask,
+                position_ids=position_ids,
+                past_key_values=past_key_values,
+                use_cache=use_cache,
+                position_embeddings=layer_pos_emb,
+                **kwargs,
+            )
+
+        hidden_states = self.norm(hidden_states)
+
+        return MoeModelOutputWithPast(
+            last_hidden_state=hidden_states,
+            past_key_values=past_key_values,
+        )
+
+
+def load_balancing_loss_func(
+    gate_logits: torch.Tensor | tuple[torch.Tensor] | None,
+    num_experts: int | None = None,
+    top_k=2,
+    attention_mask: torch.Tensor | None = None,
+) -> torch.Tensor | int:
+    r"""
+    Computes auxiliary load balancing loss as in Switch Transformer - implemented in Pytorch.
+
+    See Switch Transformer (https://huggingface.co/papers/2101.03961) for more details. This function implements the loss
+    function presented in equations (4) - (6) of the paper. It aims at penalizing cases where the routing between
+    experts is too unbalanced.
+
+    Args:
+        gate_logits:
+            Logits from the `gate`, should be a tuple of model.config.num_hidden_layers tensors of
+            shape [batch_size X sequence_length, num_experts].
+        num_experts:
+            Number of experts
+        top_k:
+            The number of experts to route per-token, can be also interpreted as the `top-k` routing
+            parameter.
+        attention_mask (`torch.Tensor`, *optional*):
+            The attention_mask used in forward function
+            shape [batch_size X sequence_length] if not None.
+
+    Returns:
+        The auxiliary loss.
+    """
+    if gate_logits is None or not isinstance(gate_logits, tuple):
+        return 0
+
+    if isinstance(gate_logits, tuple):
+        compute_device = gate_logits[0].device
+        concatenated_gate_logits = torch.cat([layer_gate.to(compute_device) for layer_gate in gate_logits], dim=0)
+
+    routing_weights = torch.nn.functional.softmax(concatenated_gate_logits, dim=-1)
+
+    _, selected_experts = torch.topk(routing_weights, top_k, dim=-1)
+
+    expert_mask = torch.nn.functional.one_hot(selected_experts, num_experts)
+
+    if attention_mask is None:
+        # Compute the percentage of tokens routed to each experts
+        tokens_per_expert = torch.mean(expert_mask.float(), dim=0)
+
+        # Compute the average probability of routing to these experts
+        router_prob_per_expert = torch.mean(routing_weights, dim=0)
+    else:
+        batch_size, sequence_length = attention_mask.shape
+        num_hidden_layers = concatenated_gate_logits.shape[0] // (batch_size * sequence_length)
+
+        # Compute the mask that masks all padding tokens as 0 with the same shape of expert_mask
+        expert_attention_mask = (
+            attention_mask[None, :, :, None, None]
+            .expand((num_hidden_layers, batch_size, sequence_length, top_k, num_experts))
+            .reshape(-1, top_k, num_experts)
+            .to(compute_device)
+        )
+
+        # Compute the percentage of tokens routed to each experts
+        tokens_per_expert = torch.sum(expert_mask.float() * expert_attention_mask, dim=0) / torch.sum(
+            expert_attention_mask, dim=0
+        )
+
+        # Compute the mask that masks all padding tokens as 0 with the same shape of tokens_per_expert
+        router_per_expert_attention_mask = (
+            attention_mask[None, :, :, None]
+            .expand((num_hidden_layers, batch_size, sequence_length, num_experts))
+            .reshape(-1, num_experts)
+            .to(compute_device)
+        )
+
+        # Compute the average probability of routing to these experts
+        router_prob_per_expert = torch.sum(routing_weights * router_per_expert_attention_mask, dim=0) / torch.sum(
+            router_per_expert_attention_mask, dim=0
+        )
+
+    overall_loss = torch.sum(tokens_per_expert * router_prob_per_expert.unsqueeze(0))
+    return overall_loss * num_experts
+
+
+@auto_docstring
+class LagunaForCausalLM(LagunaPreTrainedModel, GenerationMixin):
+    _tied_weights_keys = {"lm_head.weight": "model.embed_tokens.weight"}
+    _tp_plan = {"lm_head": "colwise_gather_output"}
+    _pp_plan = {"lm_head": (["hidden_states"], ["logits"])}
+
+    def __init__(self, config):
+        super().__init__(config)
+        self.model = LagunaModel(config)
+        self.vocab_size = config.vocab_size
+        self.lm_head = nn.Linear(config.hidden_size, config.vocab_size, bias=False)
+        self.router_aux_loss_coef = config.router_aux_loss_coef
+        self.num_experts = config.num_experts
+        self.num_experts_per_tok = config.num_experts_per_tok
+
+        # Initialize weights and apply final processing
+        self.post_init()
+
+    @can_return_tuple
+    @auto_docstring
+    def forward(
+        self,
+        input_ids: torch.LongTensor | None = None,
+        attention_mask: torch.Tensor | None = None,
+        position_ids: torch.LongTensor | None = None,
+        past_key_values: Cache | None = None,
+        inputs_embeds: torch.FloatTensor | None = None,
+        labels: torch.LongTensor | None = None,
+        use_cache: bool | None = None,
+        output_router_logits: bool | None = None,
+        logits_to_keep: int | torch.Tensor = 0,
+        **kwargs: Unpack[TransformersKwargs],
+    ) -> MoeCausalLMOutputWithPast:
+        r"""
+        labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*):
+            Labels for computing the masked language modeling loss. Indices should either be in `[0, ...,
+            config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored
+            (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`.
+        """
+
+        output_router_logits = (
+            output_router_logits if output_router_logits is not None else self.config.output_router_logits
+        )
+
+        # decoder outputs consists of (dec_features, layer_state, dec_hidden, dec_attn)
+        outputs: MoeModelOutputWithPast = self.model(
+            input_ids=input_ids,
+            attention_mask=attention_mask,
+            position_ids=position_ids,
+            past_key_values=past_key_values,
+            inputs_embeds=inputs_embeds,
+            use_cache=use_cache,
+            output_router_logits=output_router_logits,
+            **kwargs,
+        )
+
+        hidden_states = outputs.last_hidden_state
+        # Only compute necessary logits, and do not upcast them to float if we are not computing the loss
+        slice_indices = slice(-logits_to_keep, None) if isinstance(logits_to_keep, int) else logits_to_keep
+        logits = self.lm_head(hidden_states[:, slice_indices, :])
+
+        loss = None
+        if labels is not None:
+            loss = self.loss_function(logits, labels, self.vocab_size, **kwargs)
+
+        aux_loss = None
+        if output_router_logits:
+            aux_loss = load_balancing_loss_func(
+                outputs.router_logits,
+                self.num_experts,
+                self.num_experts_per_tok,
+                attention_mask,
+            )
+            if labels is not None:
+                loss += self.router_aux_loss_coef * aux_loss.to(loss.device)  # make sure to reside in the same device
+
+        return MoeCausalLMOutputWithPast(
+            loss=loss,
+            aux_loss=aux_loss,
+            logits=logits,
+            past_key_values=outputs.past_key_values,
+            hidden_states=outputs.hidden_states,
+            attentions=outputs.attentions,
+            router_logits=outputs.router_logits,
+        )
+
+
+__all__ = ["LagunaForCausalLM", "LagunaModel", "LagunaPreTrainedModel"]
+
+
+# --- Added: register the native Laguna checkpoint-conversion for trust_remote_code loads.
+# transformers >=5.12 skips checkpoint-conversion mappings for custom (remote) code
+# unless explicitly registered, which broke loading the shipped per-expert MoE weights.
+try:
+    from transformers.conversion_mapping import (
+        get_checkpoint_conversion_mapping as _lg_get,
+        register_checkpoint_conversion_mapping as _lg_reg,
+        USER_REGISTERED_MAPPINGS as _lg_user,
+    )
+
+    if "laguna" not in _lg_user:
+        _lg_m = _lg_get("laguna")
+        if _lg_m is not None:
+            _lg_reg("laguna", _lg_m, overwrite=True)
+except Exception:
+    pass

runtime/Dockerfile

@@ -0,0 +1,53 @@
+FROM ghcr.io/anemll/dspark-vllm-gx10:0.1.1
+
+ARG EXLLAMAV3_COMMIT=c5d9c657966ffeeaa9353f0cc899f18629da4a13
+ARG EXLLAMAV3_ARCHIVE_SHA256=87cf1ed1abad0fd2eb84744418e6a2ea3ec09a874552c1a3e0f226eac03cb8f9
+ARG FLASHINFER_VERSION=0.6.13
+
+ENV DEBIAN_FRONTEND=noninteractive \
+    TORCH_CUDA_ARCH_LIST=12.1 \
+    MAX_JOBS=4 \
+    LIBRARY_PATH=/usr/local/lib/python3.12/dist-packages/nvidia/cu13/lib
+
+COPY patch_exllamav3_arm64.py /opt/patch_exllamav3_arm64.py
+COPY vendor/exllamav3-c5d9c657966ffeeaa9353f0cc899f18629da4a13.tar.gz /opt/exllamav3.tar.gz
+
+RUN echo "${EXLLAMAV3_ARCHIVE_SHA256}  /opt/exllamav3.tar.gz" | sha256sum -c - \
+    && for header in \
+      /usr/local/lib/python3.12/dist-packages/nvidia/cu13/include/cusparse*.h \
+      /usr/local/lib/python3.12/dist-packages/nvidia/cu13/include/cusolver*.h; \
+    do \
+      ln -sf "$header" "/usr/local/cuda/include/$(basename "$header")"; \
+    done \
+    && EXLLAMA_NOCOMPILE=1 python3 -m pip install \
+      --no-cache-dir --no-deps --no-build-isolation \
+      /opt/exllamav3.tar.gz \
+    && python3 /opt/patch_exllamav3_arm64.py \
+      /usr/local/lib/python3.12/dist-packages/exllamav3/exllamav3_ext \
+    && python3 -m pip install --no-cache-dir --force-reinstall --no-deps \
+      "flashinfer-python==${FLASHINFER_VERSION}" \
+    && python3 - <<'PY'
+import importlib.metadata as im
+import flashinfer
+
+expected = "0.6.13"
+actual = im.version("flashinfer-python")
+cubin = im.version("flashinfer-cubin")
+if actual != expected or cubin != expected or flashinfer.__version__ != expected:
+    raise SystemExit(
+        f"FlashInfer ABI mismatch: python={actual} cubin={cubin} "
+        f"module={flashinfer.__version__} expected={expected}"
+    )
+print(f"FlashInfer ABI coherent: python={actual} cubin={cubin}")
+PY
+
+COPY sitecustomize.py /usr/local/lib/python3.12/dist-packages/sitecustomize.py
+
+LABEL org.opencontainers.image.source="https://github.com/turboderp-org/exllamav3" \
+      org.opencontainers.image.revision="${EXLLAMAV3_COMMIT}" \
+      ai.laguna.exllamav3_archive_sha256="${EXLLAMAV3_ARCHIVE_SHA256}" \
+      ai.laguna.flashinfer_python="${FLASHINFER_VERSION}" \
+      ai.laguna.flashinfer_cubin="${FLASHINFER_VERSION}" \
+      ai.laguna.runtime_fix="graph-safe hot expert remap and additive TP1 reconstruction of TP2 EXL3 slices" \
+      ai.laguna.cuda_graphs="required" \
+      ai.laguna.purpose="Laguna TP1 or TP2 saliency hybrid NVFP4 plus EXL3 tails"

runtime/launch_single_spark.sh

@@ -0,0 +1,88 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+if [[ $# -ne 5 ]]; then
+  echo "usage: $0 TR2_OR_TR3 ATTEMPT_NAME API_PORT TAIL_DIR MAP_DIR" >&2
+  exit 2
+fi
+
+TIER="${1^^}"
+ATTEMPT_NAME="$2"
+API_PORT="$3"
+TAIL_DIR="$4"
+MAP_DIR="$5"
+
+if [[ "$TIER" != "TR2" && "$TIER" != "TR3" ]]; then
+  echo "tier must be TR2 or TR3" >&2
+  exit 2
+fi
+case "$API_PORT" in
+  ''|*[!0-9]*) echo "API port must be an integer" >&2; exit 2 ;;
+esac
+
+IMAGE="${LAGUNA_IMAGE:-laguna-vllm-hybrid:20260725-a10-tp1}"
+MODEL_DIR="${LAGUNA_MODEL_DIR:-/home/sero/models/Laguna-S-2.1-NVFP4}"
+MAP_FILE="${LAGUNA_MAP_FILE:-laguna-${TIER,,}-tier-map.json}"
+SERVED_MODEL="${LAGUNA_SERVED_MODEL:-laguna-s-2.1-${TIER,,}-hybrid}"
+MAX_NUM_SEQS="${LAGUNA_MAX_NUM_SEQS:-4}"
+MAX_BATCHED_TOKENS="${LAGUNA_MAX_BATCHED_TOKENS:-4096}"
+GPU_MEMORY_UTILIZATION="${LAGUNA_GPU_MEMORY_UTILIZATION:-0.86}"
+
+for value in "$MAX_NUM_SEQS" "$MAX_BATCHED_TOKENS"; do
+  case "$value" in
+    ''|*[!0-9]*) echo "sequence and batching settings must be integers" >&2; exit 2 ;;
+    0) echo "sequence and batching settings must be positive" >&2; exit 2 ;;
+  esac
+done
+
+docker inspect "$ATTEMPT_NAME" >/dev/null 2>&1 \
+  && { echo "container already exists: $ATTEMPT_NAME" >&2; exit 3; }
+docker image inspect "$IMAGE" >/dev/null
+[[ -d "$MODEL_DIR" ]] || { echo "missing source model: $MODEL_DIR" >&2; exit 3; }
+[[ -d "$TAIL_DIR/${TIER,,}" ]] \
+  || { echo "missing tail tier: $TAIL_DIR/${TIER,,}" >&2; exit 3; }
+[[ -s "$MAP_DIR/$MAP_FILE" ]] \
+  || { echo "missing tier map: $MAP_DIR/$MAP_FILE" >&2; exit 3; }
+
+docker run -d \
+  --name "$ATTEMPT_NAME" \
+  --gpus all \
+  --network host \
+  --ipc host \
+  --shm-size 64g \
+  --ulimit memlock=-1 \
+  --ulimit stack=67108864 \
+  -v "$MODEL_DIR:/model:ro" \
+  -v "$TAIL_DIR:/tail:ro" \
+  -v "$MAP_DIR:/maps:ro" \
+  -e "PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True" \
+  -e "TORCH_CUDA_ARCH_LIST=12.1a" \
+  -e "CUTE_DSL_ARCH=sm_121a" \
+  -e "MAX_JOBS=4" \
+  -e "VLLM_NO_USAGE_STATS=1" \
+  -e "LAGUNA_HYBRID_TIER=$TIER" \
+  -e "LAGUNA_HYBRID_MODEL_DIR=/model" \
+  -e "LAGUNA_HYBRID_TAIL_DIR=/tail" \
+  -e "LAGUNA_HYBRID_TIER_MAP=/maps/$MAP_FILE" \
+  -e "LAGUNA_HYBRID_TRELLIS_CHUNK=128" \
+  -e "PYTHONPATH=/usr/local/lib/python3.12/dist-packages" \
+  "$IMAGE" \
+  /model \
+  --served-model-name "$SERVED_MODEL" \
+  --enable-auto-tool-choice \
+  --tool-call-parser poolside_v1 \
+  --reasoning-parser poolside_v1 \
+  --structured-outputs-config '{"enable_in_reasoning":true}' \
+  --compilation-config '{"inductor_compile_config":{"benchmark_combo_kernel":false}}' \
+  --override-generation-config '{"temperature":0.7,"top_p":0.95}' \
+  --max-num-seqs "$MAX_NUM_SEQS" \
+  --max-num-batched-tokens "$MAX_BATCHED_TOKENS" \
+  --max-model-len 200000 \
+  --gpu-memory-utilization "$GPU_MEMORY_UTILIZATION" \
+  --tensor-parallel-size 1 \
+  --moe-backend flashinfer_cutlass \
+  --attention-backend TRITON_ATTN \
+  --host 0.0.0.0 \
+  --port "$API_PORT"
+
+printf '%s\n' "$ATTEMPT_NAME"

runtime/launch_tp2.sh

@@ -0,0 +1,158 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+if [[ $# -ne 5 ]]; then
+  echo "usage: $0 TR2_OR_TR3 ATTEMPT_NAME MASTER_PORT API_PORT TAIL_DIR" >&2
+  exit 2
+fi
+
+TIER="${1^^}"
+ATTEMPT_NAME="$2"
+MASTER_PORT="$3"
+API_PORT="$4"
+TAIL_DIR="$5"
+
+if [[ "$TIER" != "TR2" && "$TIER" != "TR3" ]]; then
+  echo "tier must be TR2 or TR3" >&2
+  exit 2
+fi
+SPECULATIVE_MODE="${LAGUNA_SPECULATIVE_MODE:-none}"
+if [[ "$SPECULATIVE_MODE" != "dflash" && "$SPECULATIVE_MODE" != "none" ]]; then
+  echo "LAGUNA_SPECULATIVE_MODE must be dflash or none" >&2
+  exit 2
+fi
+for value in "$MASTER_PORT" "$API_PORT"; do
+  case "$value" in
+    ''|*[!0-9]*) echo "ports must be integers" >&2; exit 2 ;;
+  esac
+done
+
+IMAGE="laguna-vllm-hybrid:20260724-a9-hot-remap"
+MODEL_DIR="${LAGUNA_MODEL_DIR:-/home/sero/models/Laguna-S-2.1-NVFP4}"
+DFLASH_DIR="${LAGUNA_DFLASH_DIR:-/home/sero/models/Laguna-S-2.1-DFlash-NVFP4}"
+MAP_DIR="${LAGUNA_MAP_DIR:-/home/sero/laguna-reap-saliency-v1/final-20260723-a2/tier-maps}"
+WORKER_HOST="10.0.0.2"
+HEAD_IP="10.0.0.1"
+SERVED_MODEL="laguna-s-2.1-${TIER,,}-hybrid"
+MAP_FILE="${LAGUNA_MAP_FILE:-laguna-${TIER,,}-tier-map.json}"
+
+for host in head worker; do
+  if [[ "$host" == "head" ]]; then
+    docker inspect "$ATTEMPT_NAME" >/dev/null 2>&1 \
+      && { echo "head container already exists: $ATTEMPT_NAME" >&2; exit 3; }
+  else
+    ssh -o BatchMode=yes -o ConnectTimeout=10 "$WORKER_HOST" \
+      "docker inspect '$ATTEMPT_NAME' >/dev/null 2>&1" \
+      && { echo "worker container already exists: $ATTEMPT_NAME" >&2; exit 3; }
+  fi
+done
+[[ -d "$TAIL_DIR/${TIER,,}" ]] || { echo "missing tail tier: $TAIL_DIR/${TIER,,}" >&2; exit 3; }
+[[ -s "$MAP_DIR/$MAP_FILE" ]] || { echo "missing tier map: $MAP_DIR/$MAP_FILE" >&2; exit 3; }
+if [[ "$SPECULATIVE_MODE" == "dflash" ]]; then
+  [[ -s "$DFLASH_DIR/model.safetensors" ]] \
+    || { echo "missing pinned Laguna DFlash checkpoint" >&2; exit 3; }
+  ssh -o BatchMode=yes -o ConnectTimeout=10 "$WORKER_HOST" \
+    "test -s '$DFLASH_DIR/model.safetensors'" \
+    || { echo "worker is missing pinned Laguna DFlash checkpoint" >&2; exit 3; }
+fi
+DFLASH_MOUNT=()
+if [[ "$SPECULATIVE_MODE" == "dflash" ]]; then
+  DFLASH_MOUNT=(-v "$DFLASH_DIR:/dflash:ro")
+fi
+
+cleanup_failed_launch() {
+  docker stop "$ATTEMPT_NAME" >/dev/null 2>&1 || true
+  ssh -o BatchMode=yes -o ConnectTimeout=10 "$WORKER_HOST" \
+    "docker stop '$ATTEMPT_NAME' >/dev/null 2>&1 || true" || true
+}
+trap cleanup_failed_launch ERR
+
+COMMON_ENV=(
+  -e "NCCL_IB_DISABLE=0"
+  -e "NCCL_IB_HCA=rocep1s0f1,roceP2p1s0f1"
+  -e "NCCL_IB_GID_INDEX=0"
+  -e "NCCL_CUMEM_ENABLE=0"
+  -e "NCCL_DEBUG=WARN"
+  -e "TORCH_CUDA_ARCH_LIST=12.1a"
+  -e MASTER_ADDR="$HEAD_IP"
+  -e MASTER_PORT="$MASTER_PORT"
+  -e "GLOO_SOCKET_IFNAME=enp1s0f1np1"
+  -e "NCCL_IB_ROCE_VERSION_NUM=2"
+  -e "MN_IF_NAME=enp1s0f1np1"
+  -e "PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True"
+  -e "MAX_JOBS=4"
+  -e "NCCL_SOCKET_IFNAME=enp1s0f1np1"
+  -e "NCCL_NET=IB"
+  -e "NCCL_CROSS_NIC=1"
+  -e "NCCL_IGNORE_CPU_AFFINITY=1"
+  -e "OMPI_MCA_btl_tcp_if_include=enp1s0f1np1"
+  -e "CUTE_DSL_ARCH=sm_121a"
+  -e "NCCL_IB_ADDR_FAMILY=AF_INET"
+  -e "NCCL_NVLS_ENABLE=0"
+  -e "TP_SOCKET_IFNAME=enp1s0f1np1"
+  -e "LAGUNA_HYBRID_TIER=$TIER"
+  -e "LAGUNA_HYBRID_MODEL_DIR=/model"
+  -e "LAGUNA_HYBRID_TAIL_DIR=/tail"
+  -e "LAGUNA_HYBRID_TIER_MAP=/maps/$MAP_FILE"
+  -e "LAGUNA_HYBRID_TRELLIS_CHUNK=128"
+  -e "PYTHONPATH=/usr/local/lib/python3.12/dist-packages"
+)
+
+COMMON_ARGS=(
+  /model
+  --served-model-name "$SERVED_MODEL"
+  --enable-auto-tool-choice
+  --tool-call-parser poolside_v1
+  --reasoning-parser poolside_v1
+  --structured-outputs-config '{"enable_in_reasoning":true}'
+  --override-generation-config '{"temperature":0.7,"top_p":0.95}'
+  --max-num-seqs 4
+  --max-num-batched-tokens 4096
+  --max-model-len 200000
+  --gpu-memory-utilization 0.86
+  --tensor-parallel-size 2
+  --distributed-executor-backend mp
+  --moe-backend flashinfer_cutlass
+  --attention-backend TRITON_ATTN
+  --host 0.0.0.0
+  --port "$API_PORT"
+  --nnodes 2
+  --master-addr "$HEAD_IP"
+  --master-port "$MASTER_PORT"
+)
+if [[ "$SPECULATIVE_MODE" == "dflash" ]]; then
+  COMMON_ARGS+=(
+    --speculative-config '{"model":"/dflash","num_speculative_tokens":15,"method":"dflash"}'
+  )
+fi
+
+printf -v WORKER_ENV ' %q' "${COMMON_ENV[@]}"
+printf -v WORKER_ARGS ' %q' "${COMMON_ARGS[@]}"
+WORKER_DFLASH_MOUNT=""
+if ((${#DFLASH_MOUNT[@]})); then
+  printf -v WORKER_DFLASH_MOUNT ' %q' "${DFLASH_MOUNT[@]}"
+fi
+ssh -o BatchMode=yes -o ConnectTimeout=10 "$WORKER_HOST" \
+  "docker run -d --name '$ATTEMPT_NAME' --gpus all --network host --ipc host --shm-size 64g --device /dev/infiniband --ulimit memlock=-1 --ulimit stack=67108864 -v '$MODEL_DIR:/model:ro'$WORKER_DFLASH_MOUNT -v '$TAIL_DIR:/tail:ro' -v '$MAP_DIR:/maps:ro'$WORKER_ENV -e VLLM_HOST_IP=10.0.0.2 '$IMAGE'$WORKER_ARGS --node-rank 1 --headless"
+
+docker run -d \
+  --name "$ATTEMPT_NAME" \
+  --gpus all \
+  --network host \
+  --ipc host \
+  --shm-size 64g \
+  --device /dev/infiniband \
+  --ulimit memlock=-1 \
+  --ulimit stack=67108864 \
+  -v "$MODEL_DIR:/model:ro" \
+  "${DFLASH_MOUNT[@]}" \
+  -v "$TAIL_DIR:/tail:ro" \
+  -v "$MAP_DIR:/maps:ro" \
+  "${COMMON_ENV[@]}" \
+  -e VLLM_HOST_IP=10.0.0.1 \
+  "$IMAGE" \
+  "${COMMON_ARGS[@]}" \
+  --node-rank 0
+
+trap - ERR
+printf '%s\n' "$ATTEMPT_NAME"

runtime/patch_exllamav3_arm64.py

@@ -0,0 +1,120 @@
+#!/usr/bin/env python3
+"""Disable ExLlamaV3's x86-only CPU all-reduce sources on pinned ARM64 builds.
+
+Laguna hybrid inference uses vLLM TP2 collectives, not ExLlamaV3's optional
+native CPU all-reduce backend. The CUDA EXL3 quantizer, trellis pack/unpack,
+reconstruction, and MoE kernels remain compiled. Every upstream x86 source is
+hash-gated so a source revision cannot be patched accidentally.
+"""
+
+from __future__ import annotations
+
+import hashlib
+import platform
+import sys
+from pathlib import Path
+
+
+EXPECTED = {
+    "avx2_target.cpp": "40e7445f6d25d0ec26185c9c258dc5409291218141e11b22352e96e0f0e9d7d0",
+    "avx512_target.cpp": "4f9b51075f769c34884e2b1510328b814b71fa0bb71fb0fdd92b72ebc9ba447a",
+    "parallel/all_reduce_cpu_avx2.cpp": "903f14178e4e1451b259fe06d5c3987f86e3c0f5de2de4715998faafd6040b81",
+    "parallel/all_reduce_cpu_avx512.cpp": "ccfd0eedfba4ace032bbcc5d2864b8118f30e6447080e4b5cb925f6d082b865b",
+}
+
+PACKAGE_INITIALIZERS = {
+    "__init__.py": "59430846a04ceba4e33b0ab0233f551b68ec580fb034534c11a965d3cfb42856",
+    "modules/__init__.py": "a820ac4a5ab202b8862f7e42a6c21afae78a9653f4e5aaed3b12b271aa69fa21",
+    "modules/quant/__init__.py": "cb10d8f118205ce78a35e8c27e322d8cfb2a3bc32be2ca63807437ae055f3df5",
+}
+
+MINIMAL_INITIALIZERS = {
+    "__init__.py": (
+        'from importlib.metadata import version\n\n'
+        '__version__ = version("exllamav3")\n'
+    ),
+    "modules/__init__.py": (
+        '"""Minimal module namespace for the Laguna EXL3 encoder image."""\n'
+    ),
+    "modules/quant/__init__.py": (
+        '"""EXL3 quantizer submodules are imported explicitly by the encoder."""\n'
+    ),
+}
+
+STUB = r"""#include <cstddef>
+#include <cstdint>
+#include <stdexcept>
+#include "avx2_target.h"
+#include "avx512_target.h"
+#include "parallel/all_reduce_cpu_avx2.h"
+#include "parallel/all_reduce_cpu_avx512.h"
+
+bool is_avx2_supported() { return false; }
+bool is_avx512_supported() { return false; }
+void enable_fast_fp() {}
+void enable_fast_fp_avx2() {}
+void enable_fast_fp_avx512() {}
+
+[[noreturn]] static void unavailable()
+{
+    throw std::runtime_error(
+        "ExLlamaV3 native x86 CPU all-reduce is unavailable on ARM64; "
+        "use the serving runtime's device collective backend");
+}
+
+void perform_cpu_reduce(
+    PGContext*, size_t, uint32_t, uint8_t*, size_t) { unavailable(); }
+void perform_cpu_reduce_avx2(
+    PGContext*, size_t, uint32_t, uint8_t*, size_t) { unavailable(); }
+void perform_cpu_reduce_avx512(
+    PGContext*, size_t, uint32_t, uint8_t*, size_t) { unavailable(); }
+void bf16_add_inplace_avx512(
+    uint16_t*, const uint16_t*, size_t) { unavailable(); }
+"""
+
+
+def digest(path: Path) -> str:
+    return hashlib.sha256(path.read_bytes()).hexdigest()
+
+
+def main() -> int:
+    if len(sys.argv) != 2:
+        raise SystemExit("usage: patch_exllamav3_arm64.py EXLLAMAV3_EXT_DIR")
+    machine = platform.machine().lower()
+    if machine not in {"aarch64", "arm64"}:
+        raise RuntimeError(f"refusing ARM64 patch on architecture {machine!r}")
+    root = Path(sys.argv[1]).resolve()
+    if not root.is_dir():
+        raise FileNotFoundError(root)
+    for relative, expected in EXPECTED.items():
+        source = root / relative
+        if digest(source) != expected:
+            raise RuntimeError(f"upstream source hash mismatch: {relative}")
+        disabled = source.with_suffix(source.suffix + ".x86-disabled")
+        if disabled.exists():
+            raise FileExistsError(disabled)
+        source.rename(disabled)
+    package = root.parent
+    for relative, expected in PACKAGE_INITIALIZERS.items():
+        initializer = package / relative
+        if digest(initializer) != expected:
+            raise RuntimeError(
+                f"upstream package initializer hash mismatch: {relative}"
+            )
+        initializer.write_text(
+            MINIMAL_INITIALIZERS[relative], encoding="utf-8"
+        )
+    stub = root / "arm64_no_x86.cpp"
+    if stub.exists():
+        raise FileExistsError(stub)
+    stub.write_text(STUB, encoding="utf-8")
+    print(
+        "ARM64 compatibility applied: disabled four x86-only CPU "
+        "all-reduce sources and narrowed optional package imports; "
+        "CUDA EXL3 sources retained"
+    )
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

runtime/sitecustomize.py

@@ -0,0 +1,777 @@
+"""Laguna hybrid NVFP4 + calibrated EXL3 tail runtime for TP1 or TP2.
+
+Enabled only when LAGUNA_HYBRID_TIER is TR2 or TR3.  The stock compressed-
+tensors NVFP4 MoE method is replaced with a fail-closed two-tier method:
+
+* saliency-selected hot experts are loaded byte-for-byte from the source
+  checkpoint into a compact native vLLM CUTLASS NVFP4 kernel;
+* the remaining experts are loaded from the calibrated TP2 rank-sliced EXL3
+  tail; TP1 reconstructs the full expert by executing both stored slices into
+  the kernel's additive fp32 output buffer;
+* both tiers consume the same router weights and their local outputs are added
+  before vLLM performs its normal TP reduction.
+
+The trellis path uses preallocated scratch and fixed chunking and is therefore
+compatible with normal CUDA graph capture.  This module does not change vLLM's
+execution mode or graph configuration.
+"""
+
+from __future__ import annotations
+
+import dataclasses
+import json
+import os
+import re
+import threading
+from pathlib import Path
+from typing import Any
+
+
+TIER = os.environ.get("LAGUNA_HYBRID_TIER", "").upper()
+
+
+if TIER in ("TR2", "TR3"):
+    import torch
+    import torch.nn as nn
+    from safetensors import safe_open
+    from vllm.distributed import (
+        get_tensor_model_parallel_rank,
+        get_tensor_model_parallel_world_size,
+    )
+    from vllm.model_executor.layers.fused_moe import SharedExperts
+    from vllm.model_executor.layers.fused_moe.activation import MoEActivation
+    from vllm.model_executor.layers.fused_moe.config import FusedMoEParallelConfig
+    from vllm.model_executor.layers.fused_moe.oracle.nvfp4 import (
+        convert_to_nvfp4_moe_kernel_format,
+        make_nvfp4_moe_kernel,
+        make_nvfp4_moe_quant_config,
+        select_nvfp4_moe_backend,
+    )
+    from vllm.model_executor.layers.quantization.compressed_tensors.compressed_tensors_moe.compressed_tensors_moe_w4a4_nvfp4 import (
+        CompressedTensorsW4A4Nvfp4MoEMethod as _StockNvfp4MoE,
+    )
+    from vllm.model_executor.layers.quantization.utils.quant_utils import (
+        kNvfp4Dynamic,
+        kNvfp4Static,
+    )
+    from vllm.model_executor.utils import set_weight_attrs
+
+    from exllamav3.ext import exllamav3_ext as _exl3
+
+    BITS = 2 if TIER == "TR2" else 3
+    ARTIFACT_TP = 2
+    EXPERTS = 256
+    HIDDEN = 3072
+    INTERMEDIATE = 1024
+    TOP_K = 10
+    MCG = 0xCBAC1FED
+    CHUNK = int(os.environ.get("LAGUNA_HYBRID_TRELLIS_CHUNK", "128"))
+    if CHUNK <= 0:
+        raise RuntimeError("LAGUNA_HYBRID_TRELLIS_CHUNK must be positive")
+
+    MODEL_DIR = Path(os.environ.get("LAGUNA_HYBRID_MODEL_DIR", "/model"))
+    TAIL_DIR = Path(os.environ.get("LAGUNA_HYBRID_TAIL_DIR", "/tail"))
+    MAP_PATH = Path(
+        os.environ.get(
+            "LAGUNA_HYBRID_TIER_MAP",
+            str(TAIL_DIR / f"laguna-{TIER.lower()}-tier-map.json"),
+        )
+    )
+    if not MAP_PATH.is_file():
+        raise FileNotFoundError(f"Laguna hybrid tier map is missing: {MAP_PATH}")
+    _MAP_PAYLOAD = json.loads(MAP_PATH.read_text(encoding="utf-8"))
+    if _MAP_PAYLOAD.get("tier") != TIER:
+        raise RuntimeError(
+            f"tier map says {_MAP_PAYLOAD.get('tier')!r}, runtime requested {TIER}"
+        )
+    if not _MAP_PAYLOAD.get("coverage_gate", {}).get("passed"):
+        raise RuntimeError("Laguna hybrid tier map failed its coverage gate")
+    _EXPECTED_BUDGET = {
+        "TR2": {
+            "tail_encoding": "trellis2",
+            "tail_weight_bits": 2,
+            "hot_experts_per_sparse_layer": 96,
+            "tail_experts_per_sparse_layer": 160,
+            "average_weight_bits": 2.75,
+        },
+        "TR3": {
+            "tail_encoding": "trellis3",
+            "tail_weight_bits": 3,
+            "hot_experts_per_sparse_layer": 64,
+            "tail_experts_per_sparse_layer": 192,
+            "average_weight_bits": 3.25,
+        },
+    }[TIER]
+    _BUDGET = _MAP_PAYLOAD.get("bit_budget")
+    if not isinstance(_BUDGET, dict):
+        raise RuntimeError("Laguna hybrid tier map lacks a bit-budget contract")
+    _REQUIRED_BUDGET = {
+        "hot_encoding": "nvfp4",
+        "hot_weight_bits": 4,
+        "tail_encoding": _EXPECTED_BUDGET["tail_encoding"],
+        "tail_weight_bits": _EXPECTED_BUDGET["tail_weight_bits"],
+        "hot_experts_per_sparse_layer": _EXPECTED_BUDGET[
+            "hot_experts_per_sparse_layer"
+        ],
+        "tail_experts_per_sparse_layer": _EXPECTED_BUDGET[
+            "tail_experts_per_sparse_layer"
+        ],
+    }
+    for _field, _value in _REQUIRED_BUDGET.items():
+        if _BUDGET.get(_field) != _value:
+            raise RuntimeError(
+                f"Laguna {TIER} bit budget {_field}="
+                f"{_BUDGET.get(_field)!r} != {_value!r}"
+            )
+    _average = _EXPECTED_BUDGET["average_weight_bits"]
+    if (
+        float(_BUDGET.get("target_average_weight_bits", -1.0)) != _average
+        or float(_BUDGET.get("nominal_average_weight_bits", -1.0)) != _average
+        or _BUDGET.get("budget_passed") is not True
+    ):
+        raise RuntimeError(
+            f"Laguna {TIER} tier map failed its exact bit-budget contract"
+        )
+
+    _LAYER_RE = re.compile(r"(?:^|\.)layers\.(\d+)(?:\.|$)")
+    _RUNTIME: dict[tuple[int, int], dict[str, torch.Tensor | int]] = {}
+    _RUNTIME_LOCK = threading.Lock()
+
+
+    def _layer_index(value: str | None) -> int:
+        match = _LAYER_RE.search(str(value or ""))
+        if match is None:
+            raise RuntimeError(f"cannot resolve Laguna layer index from {value!r}")
+        layer = int(match.group(1))
+        if not 1 <= layer <= 47:
+            raise RuntimeError(f"unexpected Laguna sparse layer index: {layer}")
+        return layer
+
+
+    def _tier_layer(layer: int) -> tuple[list[int], list[int]]:
+        entry = _MAP_PAYLOAD["layers"].get(str(layer))
+        if entry is None:
+            raise RuntimeError(f"tier map has no sparse layer {layer}")
+        hot = [int(value) for value in entry["hot_experts"]]
+        tail = [int(value) for value in entry["tail_experts"]]
+        expected_hot = 96 if TIER == "TR2" else 64
+        expected_tail = EXPERTS - expected_hot
+        if len(hot) != expected_hot or len(tail) != expected_tail:
+            raise RuntimeError(
+                f"layer {layer}: hot/tail counts {len(hot)}/{len(tail)} "
+                f"!= {expected_hot}/{expected_tail}"
+            )
+        if set(hot) | set(tail) != set(range(EXPERTS)) or set(hot) & set(tail):
+            raise RuntimeError(f"layer {layer}: tier map is not a 256-expert partition")
+        expected_tail_encoding = "trellis2" if TIER == "TR2" else "trellis3"
+        if (
+            entry.get("hot_encoding") != "nvfp4"
+            or entry.get("tail_encoding") != expected_tail_encoding
+        ):
+            raise RuntimeError(
+                f"layer {layer}: tier encodings do not match {TIER}"
+            )
+        return hot, tail
+
+
+    def _runtime(device: torch.device, max_rows: int) -> dict[str, Any]:
+        key = (device.index if device.index is not None else torch.cuda.current_device(), max_rows)
+        with _RUNTIME_LOCK:
+            existing = _RUNTIME.get(key)
+            if existing is not None:
+                return existing
+            concurrency = int(_exl3.exl3_moe_max_concurrency(key[0]))
+            value: dict[str, Any] = {
+                "max_rows": max_rows,
+                "cap": CHUNK,
+                "xh": torch.empty((max_rows, HIDDEN), dtype=torch.float16, device=device),
+                "out32": torch.empty((max_rows, HIDDEN), dtype=torch.float32, device=device),
+                "tg": torch.empty(
+                    (concurrency, CHUNK, HIDDEN), dtype=torch.float16, device=device
+                ),
+                "tu": torch.empty(
+                    (concurrency, CHUNK, HIDDEN), dtype=torch.float16, device=device
+                ),
+                "ig": torch.empty(
+                    (concurrency, CHUNK, INTERMEDIATE // ARTIFACT_TP),
+                    dtype=torch.float16,
+                    device=device,
+                ),
+                "iu": torch.empty(
+                    (concurrency, CHUNK, INTERMEDIATE // ARTIFACT_TP),
+                    dtype=torch.float16,
+                    device=device,
+                ),
+                "flat_token": torch.arange(
+                    CHUNK, dtype=torch.int64, device=device
+                ).repeat_interleave(TOP_K),
+                "ones": torch.ones(CHUNK * TOP_K, dtype=torch.int64, device=device),
+            }
+            _RUNTIME[key] = value
+            print(
+                f"[laguna-hybrid] shared EXL3 runtime allocated: max_rows={max_rows} "
+                f"chunk={CHUNK} concurrency={concurrency}",
+                flush=True,
+            )
+            return value
+
+
+    class LagunaHybridNvfp4Exl3MoE(_StockNvfp4MoE):
+        def __init__(self, moe, layer_name: str | None = None, use_a16: bool = False):
+            super().__init__(moe, layer_name, use_a16)
+            self.layer_name = layer_name
+            self.layer_index = _layer_index(layer_name)
+            self.hot, self.tail = _tier_layer(self.layer_index)
+            self.hot_pos = {expert: index for index, expert in enumerate(self.hot)}
+            self.tail_pos = {expert: index for index, expert in enumerate(self.tail)}
+            self._seen: set[tuple[int, str, str]] = set()
+            self.hot_kernel = None
+            self.hot_layer = None
+            self.hot_quant_config = None
+            self.tail_slabs: dict[int, dict[str, dict[str, torch.Tensor]]] = {}
+            self.tail_ptrs: dict[int, list[torch.Tensor]] = {}
+            self.runtime_tp = 0
+            self.hot_expert_map = None
+            self.tail_lut = None
+            self.runtime = None
+
+        def create_weights(
+            self,
+            layer: torch.nn.Module,
+            num_experts: int,
+            hidden_size: int,
+            intermediate_size_per_partition: int,
+            params_dtype: torch.dtype,
+            **extra_weight_attrs,
+        ) -> None:
+            if num_experts != EXPERTS or hidden_size != HIDDEN:
+                raise RuntimeError(
+                    f"unexpected Laguna MoE geometry E={num_experts} H={hidden_size}"
+                )
+            runtime_tp = get_tensor_model_parallel_world_size()
+            if runtime_tp not in (1, ARTIFACT_TP):
+                raise RuntimeError(
+                    "Laguna hybrid runtime supports tensor parallel world size 1 or 2"
+                )
+            expected_local_i = INTERMEDIATE // runtime_tp
+            if intermediate_size_per_partition != expected_local_i:
+                raise RuntimeError(
+                    "Laguna hybrid intermediate partition mismatch: "
+                    f"{intermediate_size_per_partition} != {expected_local_i} "
+                    f"for TP{runtime_tp}"
+                )
+            self.runtime_tp = runtime_tp
+            layer.num_experts = num_experts
+            layer.params_dtype = params_dtype
+            hot_count = len(self.hot)
+            rank = get_tensor_model_parallel_rank()
+
+            def weight_loader(
+                param,
+                loaded,
+                name_mapped=None,
+                *,
+                shard_id=None,
+                expert_id=None,
+                return_success=False,
+                **_kwargs,
+            ):
+                expert = int(expert_id)
+                if expert in self.tail_pos:
+                    return True if return_success else None
+                local = self.hot_pos.get(expert)
+                if local is None:
+                    return False if return_success else None
+                name = str(name_mapped or "")
+                shard = str(shard_id)
+                if shard not in ("w1", "w2", "w3"):
+                    raise RuntimeError(f"unexpected expert shard {shard!r}")
+                family = "w13" if ".w13_" in name else "w2"
+                if "input_global_scale" in name:
+                    field = "input_global_scale"
+                elif "weight_global_scale" in name:
+                    field = "weight_global_scale"
+                elif "weight_scale" in name:
+                    field = "weight_scale"
+                elif "weight_packed" in name:
+                    field = "weight_packed"
+                else:
+                    raise RuntimeError(f"unrecognized compact NVFP4 parameter: {name}")
+
+                if loaded.ndim >= 2 and runtime_tp == ARTIFACT_TP:
+                    if shard in ("w1", "w3"):
+                        loaded = loaded.chunk(ARTIFACT_TP, 0)[rank]
+                    else:
+                        loaded = loaded.chunk(ARTIFACT_TP, 1)[rank]
+                destination = param.data[local]
+                if family == "w13":
+                    if field in ("weight_packed", "weight_scale"):
+                        half = destination.shape[0] // 2
+                        destination = (
+                            destination[:half] if shard == "w1" else destination[half:]
+                        )
+                    elif field in ("weight_global_scale", "input_global_scale"):
+                        destination = destination[0 if shard == "w1" else 1]
+                destination.copy_(loaded.reshape(destination.shape).to(destination.dtype))
+                self._seen.add((expert, shard, field))
+                return True if return_success else None
+
+            def parameter(name: str, shape: tuple[int, ...], dtype: torch.dtype) -> None:
+                value = nn.Parameter(
+                    torch.empty(
+                        shape,
+                        dtype=dtype,
+                        device=torch.cuda.current_device(),
+                    ),
+                    requires_grad=False,
+                )
+                set_weight_attrs(value, {**extra_weight_attrs, "weight_loader": weight_loader})
+                layer.register_parameter(name, value)
+
+            local_i = expected_local_i
+            parameter(
+                "w13_weight_packed",
+                (hot_count, 2 * local_i, HIDDEN // 2),
+                torch.uint8,
+            )
+            parameter(
+                "w2_weight_packed",
+                (hot_count, HIDDEN, local_i // 2),
+                torch.uint8,
+            )
+            parameter(
+                "w13_weight_scale",
+                (hot_count, 2 * local_i, HIDDEN // 16),
+                torch.float8_e4m3fn,
+            )
+            parameter(
+                "w2_weight_scale",
+                (hot_count, HIDDEN, local_i // 16),
+                torch.float8_e4m3fn,
+            )
+            parameter("w13_weight_global_scale", (hot_count, 2), torch.float32)
+            parameter("w2_weight_global_scale", (hot_count,), torch.float32)
+            parameter("w13_input_global_scale", (hot_count, 2), torch.float32)
+            parameter("w2_input_global_scale", (hot_count,), torch.float32)
+            print(
+                f"[laguna-hybrid] layer {self.layer_index}: allocated "
+                f"{hot_count} NVFP4 + {len(self.tail)} trellis{BITS} experts "
+                f"for TP{runtime_tp} rank {rank}",
+                flush=True,
+            )
+
+        def _validate_hot_load(self) -> None:
+            required = {
+                (expert, shard, field)
+                for expert in self.hot
+                for shard in ("w1", "w2", "w3")
+                for field in (
+                    "weight_packed",
+                    "weight_scale",
+                    "weight_global_scale",
+                    "input_global_scale",
+                )
+            }
+            missing = required - self._seen
+            if missing:
+                raise RuntimeError(
+                    f"layer {self.layer_index}: missing {len(missing)} hot NVFP4 tensors; "
+                    f"first={sorted(missing)[:5]}"
+                )
+
+        def _build_hot_kernel(self, layer) -> None:
+            backend, experts_cls = select_nvfp4_moe_backend(
+                config=dataclasses.replace(
+                    self.moe,
+                    num_experts=len(self.hot),
+                    num_local_experts=len(self.hot),
+                    num_logical_experts=len(self.hot),
+                    intermediate_size=self.moe.intermediate_size_per_partition,
+                    moe_parallel_config=FusedMoEParallelConfig.make_no_parallel(),
+                ),
+                weight_key=kNvfp4Static,
+                activation_key=kNvfp4Dynamic,
+            )
+            kept_moe = dataclasses.replace(
+                self.moe,
+                num_experts=len(self.hot),
+                num_local_experts=len(self.hot),
+                num_logical_experts=len(self.hot),
+                intermediate_size=self.moe.intermediate_size_per_partition,
+                moe_parallel_config=FusedMoEParallelConfig.make_no_parallel(),
+            )
+            compact = nn.Module()
+            compact.activation = getattr(layer, "activation", MoEActivation.SILU)
+            compact.moe_config = kept_moe
+            compact.local_num_experts = len(self.hot)
+            compact.swiglu_limit = getattr(layer, "swiglu_limit", None)
+            converted = convert_to_nvfp4_moe_kernel_format(
+                nvfp4_backend=backend,
+                layer=compact,
+                w13=layer.w13_weight_packed,
+                w13_scale=layer.w13_weight_scale,
+                w13_scale_2=(1.0 / layer.w13_weight_global_scale[:, 0].contiguous()),
+                a13_scale=(1.0 / layer.w13_input_global_scale),
+                w2=layer.w2_weight_packed,
+                w2_scale=layer.w2_weight_scale,
+                w2_scale_2=(1.0 / layer.w2_weight_global_scale),
+                a2_scale=(1.0 / layer.w2_input_global_scale),
+                is_act_and_mul=True,
+            )
+            (
+                compact.w13_weight,
+                compact.w13_weight_scale,
+                compact.w13_weight_scale_2,
+                compact.w13_input_scale,
+                compact.w2_weight,
+                compact.w2_weight_scale,
+                compact.w2_weight_scale_2,
+                compact.w2_input_scale,
+            ) = converted
+            quant_config = make_nvfp4_moe_quant_config(
+                backend=backend,
+                w13_scale=compact.w13_weight_scale,
+                w2_scale=compact.w2_weight_scale,
+                w13_scale_2=compact.w13_weight_scale_2,
+                w2_scale_2=compact.w2_weight_scale_2,
+                a13_scale=compact.w13_input_scale,
+                a2_scale=compact.w2_input_scale,
+                swiglu_limit=compact.swiglu_limit,
+                layer=compact,
+            )
+            kernel = make_nvfp4_moe_kernel(
+                moe_quant_config=quant_config,
+                moe_config=kept_moe,
+                experts_cls=experts_cls,
+                backend=backend,
+                routing_tables=None,
+                layer=compact,
+            )
+            kernel.fused_experts.process_weights_after_loading(compact)
+            self.hot_kernel = kernel
+            self.hot_layer = compact
+            self.hot_quant_config = quant_config
+            self.moe_kernel = kernel
+            self._hot_keepalive = converted
+
+            device = compact.w13_weight.device
+            # CutlassExpertsFp4 explicitly does not support expert_map.  Its
+            # apply implementation accepts that argument for the modular
+            # kernel interface but does not forward it to run_cutlass_moe_fp4.
+            # Keep a lookup table here and remap router IDs before invoking the
+            # compact kernel instead.
+            hot_expert_map = torch.full(
+                (EXPERTS,), -1, dtype=torch.int32, device=device
+            )
+            for expert, local in self.hot_pos.items():
+                hot_expert_map[expert] = local
+            self.hot_expert_map = hot_expert_map
+
+            for name in (
+                "w13_weight_packed",
+                "w2_weight_packed",
+                "w13_weight_scale",
+                "w2_weight_scale",
+                "w13_weight_global_scale",
+                "w2_weight_global_scale",
+                "w13_input_global_scale",
+                "w2_input_global_scale",
+            ):
+                if hasattr(layer, name):
+                    delattr(layer, name)
+            print(
+                f"[laguna-hybrid] layer {self.layer_index}: native hot kernel built "
+                f"with backend={backend} experts={len(self.hot)}",
+                flush=True,
+            )
+
+        def _load_tail(self, layer) -> None:
+            runtime_tp = get_tensor_model_parallel_world_size()
+            rank = get_tensor_model_parallel_rank()
+            if runtime_tp != self.runtime_tp:
+                raise RuntimeError(
+                    f"Laguna hybrid TP changed during load: {self.runtime_tp} -> {runtime_tp}"
+                )
+            artifact_ranks = (
+                [rank] if runtime_tp == ARTIFACT_TP else list(range(ARTIFACT_TP))
+            )
+            device = self.hot_layer.w13_weight.device
+            count = len(self.tail)
+            shapes = {
+                "gate_proj": {
+                    "trellis": (
+                        count,
+                        HIDDEN // 16,
+                        (INTERMEDIATE // ARTIFACT_TP) // 16,
+                        16 * BITS,
+                    ),
+                    "suh": (count, HIDDEN),
+                    "svh": (count, INTERMEDIATE // ARTIFACT_TP),
+                },
+                "up_proj": {
+                    "trellis": (
+                        count,
+                        HIDDEN // 16,
+                        (INTERMEDIATE // ARTIFACT_TP) // 16,
+                        16 * BITS,
+                    ),
+                    "suh": (count, HIDDEN),
+                    "svh": (count, INTERMEDIATE // ARTIFACT_TP),
+                },
+                "down_proj": {
+                    "trellis": (
+                        count,
+                        (INTERMEDIATE // ARTIFACT_TP) // 16,
+                        HIDDEN // 16,
+                        16 * BITS,
+                    ),
+                    "suh": (count, INTERMEDIATE // ARTIFACT_TP),
+                    "svh": (count, HIDDEN),
+                },
+            }
+            slabs_by_rank: dict[int, dict[str, dict[str, torch.Tensor]]] = {}
+            for artifact_rank in artifact_ranks:
+                slabs: dict[str, dict[str, torch.Tensor]] = {}
+                for projection, fields in shapes.items():
+                    slabs[projection] = {
+                        field: torch.empty(
+                            shape,
+                            dtype=(
+                                torch.int16
+                                if field == "trellis"
+                                else torch.float16
+                            ),
+                            device=device,
+                        )
+                        for field, shape in fields.items()
+                    }
+                slabs_by_rank[artifact_rank] = slabs
+            for local, expert in enumerate(self.tail):
+                path = (
+                    TAIL_DIR
+                    / TIER.lower()
+                    / f"layer-{self.layer_index:02d}"
+                    / f"expert-{expert:03d}.safetensors"
+                )
+                if not path.is_file():
+                    raise FileNotFoundError(f"missing Laguna tail expert artifact: {path}")
+                with safe_open(str(path), framework="pt", device="cpu") as handle:
+                    metadata = handle.metadata() or {}
+                    expected_meta = {
+                        "format": "exl3-trellis",
+                        "bits": str(BITS),
+                        "tp": str(ARTIFACT_TP),
+                        "layer": str(self.layer_index),
+                        "expert": str(expert),
+                        "mcg_multiplier": hex(MCG),
+                        "hessian": "routed-real-activations",
+                    }
+                    for key, expected in expected_meta.items():
+                        if metadata.get(key) != expected:
+                            raise RuntimeError(
+                                f"{path}: metadata {key}={metadata.get(key)!r} != {expected!r}"
+                            )
+                    for artifact_rank in artifact_ranks:
+                        for projection in ("gate_proj", "up_proj", "down_proj"):
+                            marker = handle.get_tensor(
+                                f"{projection}.rank{artifact_rank}.mcg"
+                            )
+                            if (int(marker.item()) & 0xFFFFFFFF) != MCG:
+                                raise RuntimeError(f"{path}: wrong MCG marker")
+                            for field in ("trellis", "suh", "svh"):
+                                source = handle.get_tensor(
+                                    f"{projection}.rank{artifact_rank}.{field}"
+                                )
+                                destination = slabs_by_rank[artifact_rank][projection][
+                                    field
+                                ][local]
+                                if tuple(source.shape) != tuple(destination.shape):
+                                    raise RuntimeError(
+                                        f"{path}:{projection}.rank{artifact_rank}.{field} "
+                                        f"shape {tuple(source.shape)} != "
+                                        f"{tuple(destination.shape)}"
+                                    )
+                                destination.copy_(source.to(destination.dtype))
+            self.tail_slabs = slabs_by_rank
+            pointers_by_rank: dict[int, list[torch.Tensor]] = {}
+            for artifact_rank, slabs in slabs_by_rank.items():
+                pointers: list[torch.Tensor] = []
+                for projection in ("gate_proj", "up_proj", "down_proj"):
+                    for field in ("trellis", "suh", "svh"):
+                        slab = slabs[projection][field]
+                        step = slab.stride(0) * slab.element_size()
+                        pointers.append(
+                            torch.tensor(
+                                [
+                                    slab.data_ptr() + index * step
+                                    for index in range(count)
+                                ],
+                                dtype=torch.int64,
+                                device=device,
+                            )
+                        )
+                pointers_by_rank[artifact_rank] = pointers
+            self.tail_ptrs = pointers_by_rank
+            tail_lut = torch.full((EXPERTS,), count, dtype=torch.int64, device=device)
+            for expert, local in self.tail_pos.items():
+                tail_lut[expert] = local
+            self.tail_lut = tail_lut
+
+            from vllm.config import get_current_vllm_config
+
+            max_rows = int(
+                get_current_vllm_config().scheduler_config.max_num_batched_tokens
+            )
+            self.runtime = _runtime(device, max_rows)
+            print(
+                f"[laguna-hybrid] layer {self.layer_index}: loaded {count} calibrated "
+                f"trellis{BITS} experts from artifact ranks {artifact_ranks} "
+                f"for TP{runtime_tp} rank {rank}",
+                flush=True,
+            )
+
+        def _validate_execution_contract(self, layer) -> None:
+            if not bool(getattr(self.moe, "is_act_and_mul", False)):
+                raise RuntimeError(
+                    "Laguna hybrid requires SwiGLU act-and-multiply experts"
+                )
+            activation = getattr(layer, "activation", MoEActivation.SILU)
+            if activation != MoEActivation.SILU:
+                raise RuntimeError(
+                    f"Laguna hybrid requires SiLU experts, got {activation!r}"
+                )
+            swiglu_limit = getattr(layer, "swiglu_limit", None)
+            if swiglu_limit not in (None, 0, 0.0):
+                raise RuntimeError(
+                    "Laguna hybrid EXL3 tail does not support a nonzero "
+                    f"SwiGLU limit, got {swiglu_limit!r}"
+                )
+            if bool(getattr(layer, "apply_router_weight_on_input", False)):
+                raise RuntimeError(
+                    "Laguna hybrid requires router weights to be applied to "
+                    "expert outputs"
+                )
+
+        def process_weights_after_loading(self, layer) -> None:
+            self._validate_execution_contract(layer)
+            self._validate_hot_load()
+            self._build_hot_kernel(layer)
+            self._load_tail(layer)
+
+        def get_fused_moe_quant_config(self, layer):
+            if self.hot_quant_config is None:
+                raise RuntimeError("Laguna hot quant config requested before initialization")
+            return self.hot_quant_config
+
+        def _apply_tail(self, x, topk_weights, topk_ids):
+            runtime = self.runtime
+            if runtime is None or self.tail_lut is None:
+                raise RuntimeError("Laguna trellis runtime is not initialized")
+            rows = int(x.shape[0])
+            if rows > int(runtime["max_rows"]):
+                raise RuntimeError(
+                    f"Laguna trellis rows {rows} exceed planned capacity "
+                    f"{runtime['max_rows']}"
+                )
+            xh = runtime["xh"][:rows]
+            xh.copy_(x)
+            out = runtime["out32"][:rows]
+            out.zero_()
+            local_ids = self.tail_lut[topk_ids.long()]
+            weights = topk_weights.to(torch.float16)
+            count = len(self.tail)
+            cap = int(runtime["cap"])
+            for start in range(0, rows, cap):
+                chunk_rows = min(cap, rows - start)
+                flat = local_ids[start : start + chunk_rows].reshape(-1)
+                order = torch.argsort(flat)
+                token = runtime["flat_token"][: chunk_rows * TOP_K].index_select(
+                    0, order
+                ).contiguous()
+                sorted_weights = weights[start : start + chunk_rows].reshape(-1).index_select(
+                    0, order
+                ).contiguous()
+                counts = torch.zeros(count + 1, dtype=torch.int64, device=x.device)
+                counts.scatter_add_(0, flat, runtime["ones"][: chunk_rows * TOP_K])
+                # The EXL3 fused down projection scatter-adds with atomicAdd.
+                # In TP1, executing both physically stored TP2 artifact slices
+                # into the same zeroed fp32 buffer reconstructs the exact
+                # tensor-parallel sum without allocating inside graph replay.
+                for artifact_rank in sorted(self.tail_ptrs):
+                    _exl3.exl3_moe(
+                        xh[start : start + chunk_rows],
+                        out[start : start + chunk_rows],
+                        counts,
+                        token,
+                        sorted_weights,
+                        runtime["tg"],
+                        runtime["tu"],
+                        runtime["ig"],
+                        runtime["iu"],
+                        0,
+                        BITS,
+                        BITS,
+                        BITS,
+                        *self.tail_ptrs[artifact_rank],
+                        True,
+                        False,
+                        True,
+                        False,
+                        True,
+                        False,
+                        0.0,
+                    )
+            return out
+
+        def apply(
+            self,
+            layer,
+            x,
+            topk_weights,
+            topk_ids,
+            shared_experts: SharedExperts | None,
+            shared_experts_input,
+        ):
+            if (
+                self.hot_kernel is None
+                or self.hot_expert_map is None
+                or self.hot_layer is None
+            ):
+                raise RuntimeError("Laguna hot kernel is not initialized")
+            # The router produces original IDs in [0, 256).  Remap hot routes
+            # into the compact [0, hot_count) space.  Tail routes are sent to
+            # the valid placeholder expert 0 with an exact zero router weight;
+            # this avoids unsupported/out-of-range IDs without renormalizing
+            # the original top-k weights.
+            local_hot_ids = self.hot_expert_map[topk_ids.long()]
+            hot_route = local_hot_ids >= 0
+            local_hot_ids = torch.where(
+                hot_route, local_hot_ids, torch.zeros_like(local_hot_ids)
+            )
+            local_hot_weights = topk_weights * hot_route.to(topk_weights.dtype)
+            hot = self.hot_kernel.apply(
+                x,
+                self.hot_layer.w13_weight,
+                self.hot_layer.w2_weight,
+                local_hot_weights,
+                local_hot_ids,
+                activation=self.hot_layer.activation,
+                global_num_experts=len(self.hot),
+                expert_map=None,
+                apply_router_weight_on_input=layer.apply_router_weight_on_input,
+                shared_experts=shared_experts,
+                shared_experts_input=shared_experts_input,
+            )
+            tail = self._apply_tail(x, topk_weights, topk_ids)
+            return torch.add(hot.float(), tail).to(x.dtype)
+
+
+    import vllm.model_executor.layers.quantization.compressed_tensors.compressed_tensors_moe.compressed_tensors_moe_w4a4_nvfp4 as _target
+
+    _target.CompressedTensorsW4A4Nvfp4MoEMethod = LagunaHybridNvfp4Exl3MoE
+    print(
+        f"[laguna-hybrid] installed {TIER} runtime: bits={BITS} "
+        f"artifact_tp={ARTIFACT_TP} runtime_tp=1-or-2 "
+        f"map={MAP_PATH} tail={TAIL_DIR} CUDA graphs unchanged",
+        flush=True,
+    )

special_tokens_map.json

@@ -0,0 +1,9 @@
+{
+  "bos_token": "〈|EOS|〉",
+  "cls_token": "〈|CLS|〉",
+  "eos_token": "〈|EOS|〉",
+  "mask_token": "〈|MASK|〉",
+  "pad_token": "〈|PAD|〉",
+  "sep_token": "〈|SEP|〉",
+  "unk_token": "〈|UNK|〉"
+}

tokenizer_config.json

@@ -0,0 +1,576 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "〈|UNK|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "〈|CODE_START|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "〈|EOS|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "〈|CODE_END|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "4": {
+      "content": "〈|META_START|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "5": {
+      "content": "〈|META_END|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "6": {
+      "content": "〈|FIM_MIDDLE|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "7": {
+      "content": "〈|FIM_SUFFIX|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "8": {
+      "content": "〈|SEP|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "9": {
+      "content": "〈|PAD|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "10": {
+      "content": "〈|CLS|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "11": {
+      "content": "〈|FIM_START|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "12": {
+      "content": "〈|MASK|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "13": {
+      "content": "|◊|",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "14": {
+      "content": "〈|",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "15": {
+      "content": "|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "16": {
+      "content": "〈|/",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "17": {
+      "content": "/|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "20": {
+      "content": "〈|SPECIAL_1|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "21": {
+      "content": "〈|SPECIAL_2|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "22": {
+      "content": "〈|SPECIAL_3|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "27": {
+      "content": "〈|SPECIAL_8|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "28": {
+      "content": "〈|SPECIAL_9|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "29": {
+      "content": "〈|SPECIAL_10|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "30": {
+      "content": "〈|SPECIAL_11|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "31": {
+      "content": "〈|SPECIAL_12|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "32": {
+      "content": "〈|SPECIAL_13|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "33": {
+      "content": "〈|SPECIAL_14|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "34": {
+      "content": "〈|SPECIAL_15|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "35": {
+      "content": "〈|SPECIAL_16|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "36": {
+      "content": "〈|SPECIAL_17|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "37": {
+      "content": "〈|SPECIAL_18|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "38": {
+      "content": "〈|SPECIAL_19|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "39": {
+      "content": "〈|SPECIAL_20|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "40": {
+      "content": "〈|SPECIAL_21|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "41": {
+      "content": "〈|SPECIAL_22|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "42": {
+      "content": "〈|SPECIAL_23|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "43": {
+      "content": "〈|SPECIAL_24|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "44": {
+      "content": "〈|SPECIAL_25|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "45": {
+      "content": "〈|SPECIAL_26|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "46": {
+      "content": "〈|SPECIAL_27|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "47": {
+      "content": "〈|SPECIAL_28|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "48": {
+      "content": "〈|SPECIAL_29|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "49": {
+      "content": "〈|SPECIAL_30|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50": {
+      "content": "〈|SPECIAL_31|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "51": {
+      "content": "〈|SPECIAL_32|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "52": {
+      "content": "〈|SPECIAL_33|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "53": {
+      "content": "〈|SPECIAL_34|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "54": {
+      "content": "〈|SPECIAL_35|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "55": {
+      "content": "〈|SPECIAL_36|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "56": {
+      "content": "〈|SPECIAL_37|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "57": {
+      "content": "〈|SPECIAL_38|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "58": {
+      "content": "〈|SPECIAL_39|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "59": {
+      "content": "〈|SPECIAL_40|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "60": {
+      "content": "〈|SPECIAL_41|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "61": {
+      "content": "〈|SPECIAL_42|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "62": {
+      "content": "〈|SPECIAL_43|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "63": {
+      "content": "〈|SPECIAL_44|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "64": {
+      "content": "〈|SPECIAL_45|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "65": {
+      "content": "〈|SPECIAL_46|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "66": {
+      "content": "〈|SPECIAL_47|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "67": {
+      "content": "〈|SPECIAL_48|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "68": {
+      "content": "〈|SPECIAL_49|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "69": {
+      "content": "〈|SPECIAL_50|〉",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "18": {
+      "content": "<think>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "19": {
+      "content": "</think>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "23": {
+      "content": "<assistant>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "24": {
+      "content": "</assistant>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "25": {
+      "content": "<tool_call>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "26": {
+      "content": "</tool_call>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    }
+  },
+  "bos_token": "〈|EOS|〉",
+  "clean_up_tokenization_spaces": false,
+  "cls_token": "〈|CLS|〉",
+  "eos_token": "〈|EOS|〉",
+  "extra_special_tokens": {},
+  "mask_token": "〈|MASK|〉",
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "〈|PAD|〉",
+  "sep_token": "〈|SEP|〉",
+  "tokenizer_class": "PreTrainedTokenizerFast",
+  "unk_token": "〈|UNK|〉",
+  "chat_template": "{% include 'chat_template.jinja' %}"
+}