avlp12/Laguna-S-2.1-Alis-MLX-Dynamic-3.31bpw

Laguna S 2.1 Compact 3.31 bpw

This is the compact MLX ALIS/DWQ build of poolside/Laguna-S-2.1, pinned at 88796b991a17fc691abf1c1ad0d9f459dae73834. The finished checkpoint is 3.31 bpw when rounded to two decimal places. The exact measured value remains in artifact_manifest.json. Indexed tensor bytes: 48,594,443,776 bytes (45.257 GiB).

Quantization policy: Quality-first mixed recipe: routed experts Q3/g128, attention/dense/shared Q4/g64, embeddings and LM head Q6/g64, control paths floating. No routed-expert precision promotions were required. Routers, normalization parameters, correction bias, and other control paths remain floating. Anchor-guarded clipping ran before DWQ with maximum error slack 1.1. FFN permutation and router KD were disabled.

Exact runtime requirement

This checkpoint requires the Laguna implementation at the exact mlx-lm commit e4338e29c8521d03d7e0c42ad139ab2bf2f07f6e; review thread: https://github.com/ml-explore/mlx-lm/pull/1601. Install that immutable revision:

python -m pip install "git+https://github.com/avlp12/mlx-lm.git@e4338e29c8521d03d7e0c42ad139ab2bf2f07f6e"

The ALIS/DWQ build evidence is bound to exact commit 486aa94cd37b27beac3816e3e684a4633fafc4b7; review thread: https://github.com/avlp12/alis-dwq/pull/12. A pull request link, when shown, is review context; the full commit above is the reproducibility pin. The receipt is governed by execution laguna-33d87555bb0c7ab6dc48, arm compact-base, and the frozen base phase; its frozen planner manifest SHA-256 is aff845df24f41a7561b60f60807c39ae13095848a8de8a6cc737d63a8c8eb82a.

from mlx_lm import generate, load

model, tokenizer = load("avlp12/Laguna-S-2.1-Alis-MLX-Dynamic-3.31bpw")
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Implement a bounded worker queue."}],
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=True,
)
print(generate(model, tokenizer, prompt=prompt, max_tokens=512))

Set enable_thinking=False for direct answers. For agentic use, preserve prior reasoning_content between tool calls as described by the source model card.

Measured evaluation

Calibration, validation, and held-out raw bytes are disjoint. The measured data contract contains 80 training, 40 validation, and 100 held-out rows. Held-out coverage is code 45%, English reasoning/tool 35%, Korean 10%, and Chinese 10%. The data manifest binds the source tokenizer file hashes and the exact fix_mistral_regex=true option. The runtime-effective tokenizer file hashes differ from the source-data tokenizer file hashes; byte identity is not assumed. The release is accepted only because the equivalence contract re-verifies equal token IDs and token counts for all 220 declared rows (train 80, validation 40, held-out 100). All comparison arms use the same held-out bytes, BF16 reference manifest, runtime-effective tokenizer inputs/options, tokenization policy, and seed.

Perplexity

Checkpoint	All	Code	English	Korean	Chinese
BF16 teacher	11.045	10.650	8.842	11.890	22.385
Stock Q4/g64	11.824	11.541	9.423	12.315	24.419
Pre-DWQ	10.856	10.056	8.713	11.826	23.754
Final ALIS/DWQ	9.746	9.493	7.462	11.045	21.029

Divergence from BF16

Checkpoint	Mean KL	Top-1 flip	Eval tok/s	Peak memory
Stock Q4/g64	0.225565	13.631%	652.53	62.58 GiB
Pre-DWQ	0.406616	18.836%	645.15	46.20 GiB
Final ALIS/DWQ	0.269725	14.247%	645.54	46.20 GiB

The recomputed quality gate passed: code PPL is within 3% of stock Q4; English, English reasoning, and Korean are each within 5%; and final-vs-pre-DWQ paired degradation is not wholly above zero at 95% confidence on any required major slice. The raw reports and recomputed intervals are under evaluation/.

Fixed greedy generation covered both thinking-on and thinking-off prompts. The same six-row structured-message suite and seed passed on the full BF16 reference, stock Q4, and final candidate. BF16 measured aggregate prefill 178.31 tok/s and aggregate decode 26.10 tok/s. The final arm measured aggregate prefill 305.07 tok/s and aggregate decode 43.40 tok/s on Apple M3 Ultra. No new tail cycle, empty generation, tool-format break, or distinct-4gram regression appeared relative to stock Q4.

Measured environment: Python 3.13.12 on macOS-26.5.2-arm64-arm-64bit-Mach-O; mlx 0.32.0, mlx-lm 0.31.3, safetensors 0.8.0. Throughput is specific to this environment and is not a cross-device estimate.

DWQ and memory evidence

The BF16 teacher targets were dumped and checksummed before training. DWQ ran deepest-first, one layer per round, with validation rollback. Accepted rounds: 24; reverted rounds: 24. Validation KL moved from 0.220100 to 0.167400.

The full run's peak working set was 55.27 GiB (12.28% of MLX's recommended wired limit); maximum additional swap was 0.000 GiB. The enforced stop thresholds were 90% of the recommended wired set and 16 GiB of added swap.

Context validation

The full BF16 reference and final candidate separately passed cached-vs-one-shot logit checks at offsets 511, 512, 513, and 520 with 12 full KV caches and 36 rotating window-512 caches. Both passed needle retrieval at 8,192, 32,768 actual prompt tokens, so the release claim is capped at 32,768 measured tokens. The source architecture advertises 1,048,576 tokens, but this conversion did not execute a one-million-token test; that number is calculated architectural support, not a measured claim.

Evidence files

• conversion_plan.json: completed laguna.dwq-receipt/v3 with native alis-dwq.run/v2 completion and alis-dwq.targets/v1 teacher-target binding; also binds artifact, data, build code, all 48 rounds, and memory gates
• quantization_config.json: emitted tensor/module quantization mapping
• evaluation/model-card-evidence.json: this card's path-free laguna.model-card-evidence/v3 claim contract
• evaluation/: BF16, Q4, pre-DWQ and final logit reports; BF16, Q4 and final generation reports; recomputed gates; and separate BF16/final context reports
• artifact_manifest.json: byte size and SHA-256 for every release payload
• NOTICE.md and SOURCE_MODEL_CARD.md: source origin and pinned source-card snapshot

Publication policy

This card is generated before remote reproduction. Release tooling will create a fresh public repository, apply a manual access gate before any upload, upload the accepted directory once without changing it to private, resolve its immutable Hub SHA, re-download that exact SHA into a clean project-local cache, and rerun manifest verification plus a strict Metal generation. Both builds must independently produce passing remote-reproduction receipts, and both live repositories must still be public, manual-gated, and pinned to those SHAs in one read-only preflight before either gate is removed. No file upload or repository settings mutation is permitted after a staging SHA is captured; the only later settings mutation is the separate two-build ungating phase.

License, origin, and acceptable use

This derivative retains the source OpenMDW-1.1 license in LICENSE.md. Source identity and applicable origin notice are preserved separately; see the pinned source model card and Poolside acceptable-use policy before deployment. The evaluated languages are English, Korean, and Chinese; no untested language tags are claimed.