---
base_model: sailing-lab/SR2AM-v1.0-30B
base_model_relation: quantized
license: apache-2.0
library_name: gguf
pipeline_tag: text-generation
tags:
  - gguf
  - quantized
  - qwen3
  - qwen3_moe
  - sr2am
  - moe
  - iq2_xxs
  - imatrix
  - asymmetric
---

# SR2AM-v1.0-30B — Asymmetric IQ2_XXS/Q2_K GGUF

Asymmetric quantization of [`sailing-lab/SR2AM-v1.0-30B`](https://huggingface.co/sailing-lab/SR2AM-v1.0-30B) (Qwen3-MoE, 48 layers, 128 experts, top-8) using the strategy popularized by [antirez/ds4](https://github.com/antirez/ds4): aggressively quantize the bulky routed-expert FFN tensors and keep the rest at higher precision so the whole model fits on a 24 GB consumer GPU at full speed.

## Quantization recipe

| Tensor family | Type |
|---|---|
| `ffn_up_exps.weight` (routed) | **IQ2_XXS** |
| `ffn_gate_exps.weight` (routed) | **IQ2_XXS** |
| `ffn_down_exps.weight` (routed) | **Q2_K** |
| attention proj (`attn_q/k/v/output`) | Q6_K |
| shared/non-expert FFN, norms, router | Q6_K |
| `token_embd.weight` | Q8_0 |
| `output.weight` | Q6_K |

Imatrix calibration: 100 chunks × 512 tokens from [bartowski calibration_datav3](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8).

Tool: [`llama.cpp`](https://github.com/ggml-org/llama.cpp) `llama-quantize` with `--imatrix` + `--tensor-type` overrides.

## File

| File | Size | BPW |
|---|---|---|
| `SR2AM-v1.0-30B-IQ2XXS-asym.gguf` | **8.9 GB** | 2.50 |

vs. source BF16 GGUF (58.3 GB, 16.01 BPW) → **6.5× compression**.

## Runtime (RTX 3090 24 GB, full GPU offload)

| Setting | Value |
|---|---|
| VRAM (with 40k ctx) | 13.9 GB / 24 GB |
| Prefill | 420.6 t/s |
| Decode | 168.3 t/s |

## Quality

| Benchmark | Score | Notes |
|---|---|---|
| MATH500 (subset N=30) | **24/30 = 80.0 %** | Qwen-spec sampling (see below). 0 truncations at the run config. |

Comparable to published Qwen3-30B-A3B-Thinking-2507 BF16 numbers on MATH500 (~85–90 %); ~5–10 point delta at **2.50 BPW** is in line with what asymmetric MoE quants achieve on reasoning benchmarks.

## Usage

This is a **thinking-mode** model. Use generous output budgets and Qwen3 official sampling, otherwise reasoning chains get truncated and the model emits empty content.

```bash
llama-server \
  -m SR2AM-v1.0-30B-IQ2XXS-asym.gguf \
  -ngl 99 \
  -c 40960 \
  --jinja
```

Request body for evals / hard reasoning prompts:
```json
{
  "messages": [...],
  "max_tokens": 32768,
  "temperature": 0.6,
  "top_p": 0.95,
  "top_k": 20,
  "min_p": 0.0,
  "presence_penalty": 1.0
}
```

Use a recent `llama.cpp` build with IQ2_XXS CUDA kernels (tested on b9124).

> ⚠️ Do **not** set `temperature=0` — off-spec for Qwen3 thinking models and triggers degenerate long chains. Do **not** rely on `chat_template_kwargs.enable_thinking=false` — broken in current llama.cpp ([#20182](https://github.com/ggml-org/llama.cpp/issues/20182), [#13189](https://github.com/ggml-org/llama.cpp/issues/13189)).

## Credits

- Base model: [sailing-lab/SR2AM-v1.0-30B](https://huggingface.co/sailing-lab/SR2AM-v1.0-30B)
- Asymmetric-quant strategy: [antirez/ds4](https://github.com/antirez/ds4)
- Calibration data: [bartowski1182](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8)
- Tooling: [llama.cpp](https://github.com/ggml-org/llama.cpp)