---
license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
tags:
  - merge
  - model-merging
  - crane
  - qwen3-next
  - moe
  - code
  - reasoning
base_model:
  - Qwen/Qwen3-Next-80B-A3B-Instruct
  - Qwen/Qwen3-Next-80B-A3B-Thinking
---

# crane-next-80b

**crane-next-80b** is a CRANE merge: it is produced by *merging* two **Qwen3-Next-80B-A3B** checkpoints (an Instruct base and a Thinking donor) with the **CRANE** method — it is **not** trained or fine-tuned from scratch. CRANE (*Constrained Reasoning Injection for Code Agents via Nullspace Editing*) injects reasoning ability from the Thinking donor into the tool-disciplined Instruct / code base while preserving the base model's output format and tool-calling behavior.

**Project page:** **[https://rpi-nsl.github.io/CRANE/](https://rpi-nsl.github.io/CRANE/)** · Code: [github.com/rpi-nsl/CRANE](https://github.com/rpi-nsl/CRANE)

> Note: this is the CRANE *weight-merging* method for code agents. It is unrelated to the similarly-named "CRANE: Reasoning with constrained LLM generation" (arXiv 2502.09061), despite the shared acronym.

## How it was made (CRANE)

CRANE is a **training-free, parameter-editing weight merge** that injects reasoning ability from a "Thinking" donor into a tool-disciplined Instruct / code base, while constraining the edit so the base model's output format and tool-calling behavior are preserved. It treats the **Thinking − Instruct delta** \\(\delta = \theta_{\text{think}} - \theta_{\text{inst}}\\) as a pool of candidate reasoning edits, and applies three composable stages per layer \\(l\\) and parameter component \\(c\\):

$$
\theta_{\text{merged}}^{(l,c)} = \theta_{\text{inst}}^{(l,c)} + \underbrace{\Pi_{\tau,\,q(l,c)}^{\text{GSP}}}_{\text{Stage 3}}\!\Big( \alpha \cdot \underbrace{S_{\text{CTG}}(c,l)}_{\text{Stage 2}} \cdot \underbrace{T\big(\delta^{(l,c)}\big)}_{\text{Stage 1}} \Big)
$$

![CRANE / GSP merge pipeline](gsp_pipeline_clean.png)

Three small calibration sets drive the stages — \\(\mathcal{D}_R\\) (reasoning transfer), \\(\mathcal{D}_A\\) (agent-behavior / tool-use preservation), and \\(\mathcal{D}_F\\) (format preservation):

- **Stage 1 — Magnitude thresholding \\(T(\delta)\\).** A deterministic median-magnitude threshold keeps only the larger (top-half) delta coordinates and rescales them by 2, discarding low-confidence noise.
- **Stage 2 — Conservative Taylor Gate \\(S_{\text{CTG}}\\).** From a signed, direction-aware score \\(s_K(j) = -g_{K,j}\,\delta_j\\) per calibration loss, CTG keeps the positive part of the per-coordinate minimum over the reasoning and agent-behavior objectives, \\(p_j = [\min\{s_R(j), s_A(j)\}]_+\\) — rewarding a coordinate only when the edit helps *both*. These aggregate into the per-component, per-layer coefficient \\(S_{\text{CTG}}(c,l)\\), scaled by the single global merge strength \\(\alpha\\).
- **Stage 3 — Graduated Sigmoidal Projection (GSP).** From the SVD of format-critical Instruct activations \\(H_q = U_q\Sigma_q V_q^{\top}\\), a smooth sigmoidal weight \\(\mathbf{w}_q\\) (set by singular amplitude and threshold \\(\tau\\)) gives the projector \\(\Pi_{\tau,q}^{\text{GSP}}(\Delta_q) = \Delta_q - \Delta_q V_q \operatorname{diag}(\mathbf{w}_q) V_q^{\top}\\), attenuating high-amplitude format directions so reasoning is injected without perturbing chat-template tokens, tool-call delimiters, or JSON/schema structure.

The result is a merge that gains planning / reflection / recovery reasoning while keeping the base agent's compact, tool-call-disciplined behavior — the entire merge is a closed-form edit of the Instruct weights, with no fine-tuning.

### This checkpoint's recipe

This checkpoint merges **`Qwen/Qwen3-Next-80B-A3B-Instruct`** (base) and **`Qwen/Qwen3-Next-80B-A3B-Thinking`** (donor) with:

- **Global injection strength** — \\(\alpha = 0.15\\), multiplied by the per-component CTG coefficients, so the Thinking delta is added at low strength.
- **Per-layer / per-component gating** — attention, expert (FFN), norm, and router components each get their own \\(S_{\text{CTG}}(c,l)\\) coefficient, varying by layer index rather than a single flat scalar.
- **Architecture-aware norm handling** — Qwen3-Next's zero-centered \\((1 + w)\\) RMSNorm keeps the effective norm multiplier bounded, so norm edits are merged through the standard gated path without special clamping.
- **GSP projector** — a freshly rebuilt Qwen3-Next-80B graduated-sigmoidal projector protects the format / tool-call subspace before injection.

## Architecture

The merge preserves the standard **Qwen3-Next-80B-A3B** (hybrid MoE) topology unchanged:

| Property | Value |
| --- | --- |
| model_type | qwen3_next |
| Architecture class | Qwen3NextForCausalLM |
| Total params | ~80B |
| Active params | ~3B |
| hidden_size | 2048 |
| num_hidden_layers | 48 |
| num_experts | 512 (+ 1 shared expert) |
| num_experts_per_tok | 10 |
| Attention | hybrid: full attention every 4th layer (16 query / 2 KV heads, head_dim 256, 25% partial RoPE) + gated linear (Gated DeltaNet) attention elsewhere |
| max_position_embeddings | 262144 |
| vocab_size | 151936 |
| dtype | bfloat16 |
| rope_theta | 10000000 |

## Usage

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "zmzfpc/crane-next-80b"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {"role": "user", "content": "Write a Python function that returns the nth Fibonacci number."},
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))
```

> Requires a recent `transformers` with Qwen3-Next support (the model was exported with `transformers >= 4.57.0.dev0`).

## Citation / attribution

If you use this model or the CRANE method, please cite:

```bibtex
@misc{zhu2026crane,
  title        = {CRANE: Constrained Reasoning Injection for Code Agents via Nullspace Editing},
  author       = {Zhu, Mingzhi and Merler, Michele and Pavuluri, Raju and Patterson, Stacy},
  year         = {2026},
  eprint       = {2605.14084},
  archivePrefix= {arXiv},
  primaryClass = {cs.SE},
  url          = {https://arxiv.org/abs/2605.14084}
}
```

**Project page:** [https://rpi-nsl.github.io/CRANE/](https://rpi-nsl.github.io/CRANE/) · **Code:** [github.com/rpi-nsl/CRANE](https://github.com/rpi-nsl/CRANE)

**Base models** — built from two Apache-2.0 checkpoints:

- [Qwen/Qwen3-Next-80B-A3B-Instruct](https://huggingface.co/Qwen/Qwen3-Next-80B-A3B-Instruct) (base / backbone)
- [Qwen/Qwen3-Next-80B-A3B-Thinking](https://huggingface.co/Qwen/Qwen3-Next-80B-A3B-Thinking) (reasoning donor)

**License:** Apache-2.0 (consistent with both base models and the CRANE code).