---
language:
- en
library_name: mlx
license: other
license_name: openmdw-1.1
license_link: https://openmdw.ai/
pipeline_tag: text-generation
base_model: poolside/Laguna-S-2.1
tags:
- mlx
- jang
- jang-4m
- awq
- quantized
- apple-silicon
- laguna
- moe
- agentic-coding
---

<p align="center"><a href="https://osaurus.ai"><img src="./osaurus-x-banner.png" alt="Osaurus AI"></a></p>

# JANGQ-AI/Laguna-S-2.1-JANG_4M

JANG_4M JANG affine bundle of [poolside/Laguna-S-2.1](https://huggingface.co/poolside/Laguna-S-2.1) — 118B-parameter / ~8B-active MoE for agentic coding and long-horizon work, quantized for Apple Silicon with AWQ-protected routed experts.

Everything quality-critical stays high precision: attention (including the softplus output gate), the shared expert path, router, and norms. The 256-routed-expert bulk (~97% of parameters) carries the low bits, protected by activation-aware (AWQ) scales selected by **measured** 
quantization error on real expert weights and real calibration activations — the search always includes a no-AWQ baseline, so scales ship only because they beat it (2.9% lower reconstruction error at this bit width).

## Bundle

| Field | Value |
|---|---|
| Source | `poolside/Laguna-S-2.1` @ `a50e85e` (BF16, 235 GB) |
| Architecture | `laguna` / `LagunaForCausalLM` — 48 layers (12 global + 36 SWA w512), GQA 8 KV heads, 256 routed experts top-10 + 1 shared, dense layer 0, 1M context |
| On-disk size | 68.1 GB (15 shards) |
| Routed experts | 4-bit gate/up/down affine, group 64, AWQ input scales folded |
| Attention q/k/v/o + g_proj | 8-bit affine |
| Shared expert / dense FFN | 8-bit affine |
| Embeddings / lm_head | 6-bit / 8-bit affine |
| Router, e_score bias, norms | fp16 passthrough (routing bit-identical to unfolded math) |
| Modality | text-only (verified from tensor index — no vision/audio/video weights) |

## Measured performance (M5 Max, 128 GB)

| Metric | Value |
|---|---|
| Decode | 33-44 tok/s greedy (cold-warm) |
| Load time | 5.3 s |
| Long-context cache parity | teacher-forced top-1 agreement 0.996 / 1.000 across a 2,913-token pass (pre / post the 512 sliding window) |
| BF16 vs quantized | greedy smoke + chat-with-thinking verified coherent |

Memory note: budget bundle size + ~10 GB. The reference Python runtime auto-sets the Metal wired limit (`min(bundle x 1.2 + 8 GB, 118 GB)`) — without a wired limit, decode throughput roughly halves on 64 GB+ working sets.

## Chat protocol

- GLM-style think tags. **Thinking is ON by default** (vendor `default_chat_template_kwargs.enable_thinking=true`); pass `enable_thinking=False` to disable. The generation prompt ends `<assistant><think>` (thinking) or `<assistant></think>` (off).
- Stop tokens: `eos_token_id = [2, 24]` — id 24 is end-of-turn and MUST be in the stop set.
- The template emits its own leading `〈|EOS|〉` (= bos id 2): do not prepend another BOS.
- Tool calls: `<tool_call>name<arg_key>k</arg_key><arg_value>v</arg_value></tool_call>` (GLM-4.7-compatible parsing).
- Sampling defaults (vendor): temperature 1.0, top_p 1.0, top_k 20.

## Run it

[Osaurus](https://osaurus.ai) loads this bundle natively. Python reference runtime (mixed-precision affine aware):

```bash
pip install mlx mlx-lm transformers
python -m jang_tools.laguna.runtime --src ./Laguna-S-2.1-JANG_4M \
    --prompt 'def fibonacci(n):' --max-new 64
```

`config.json[quantization]` carries per-module `{bits, group_size, mode}` overrides — loaders must honor per-module bits (a single top-level width mis-dequantizes the low-bit experts).

## Source model quality (poolside, BF16 base)

Per the [upstream model card](https://huggingface.co/poolside/Laguna-S-2.1): Terminal-Bench 2.1 **70.2%**, SWE-bench Multilingual **78.5%**, SWE-Bench Pro **59.4%** — competitive with much larger frontier MoEs. These figures are for the BF16 source model.

---

Quantized and verified by **Jinho Jang** (eric@jangq.ai) with the JANG toolchain. License: [OpenMDW-1.1](https://openmdw.ai/) (inherited from the source model).