Add private release-card draft

README.md

@@ -0,0 +1,74 @@
+---
+license: apache-2.0
+base_model:
+  - Qwen/Qwen3.5-2B
+base_model_relation: quantized
+library_name: llama.cpp
+pipeline_tag: text-generation
+tags:
+  - gguf
+  - llama.cpp
+  - quantization
+  - mixed-precision
+---
+
+# Qwen3.5 2B — TheStageAI GGUF
+
+Four text-only GGUF checkpoints from **0.738 GB** to **2.013 GB**, evaluated on IFEval and MMLU-Pro where a complete release score is available.
+
+[Qwen3.5 0.8B](https://huggingface.co/TheStageAI/Qwen3.5-0.8B-GGUF) · **[Qwen3.5 2B](https://huggingface.co/TheStageAI/Qwen3.5-2B-GGUF)** · [Qwen3.5 4B](https://huggingface.co/TheStageAI/Qwen3.5-4B-GGUF) · [Qwen3.5 9B](https://huggingface.co/TheStageAI/Qwen3.5-9B-GGUF) · [Gemma 4 E2B IT](https://huggingface.co/TheStageAI/gemma-4-E2B-it-GGUF) · [Gemma 4 E4B IT](https://huggingface.co/TheStageAI/gemma-4-E4B-it-GGUF) · [Gemma 4 12B IT](https://huggingface.co/TheStageAI/gemma-4-12B-it-GGUF)
+
+## Choose a file
+
+| Variant | Hub class | File | Size | Whole-file BPW | IFEval P / I (%) | MMLU-Pro (%) | Use |
+| --- | --- | --- | ---: | ---: | ---: | ---: | --- |
+| XS | `Q3_K_S` | `Qwen3.5-2B-XS-TS-Q3_K_S.gguf` | 0.738 GB | 3.140 | 52.68 / 64.15 | — | Minimum size |
+| S | `Q4_K_S` | `Qwen3.5-2B-S-TS-Q4_K_S.gguf` | 0.967 GB | 4.109 | 63.22 / 73.38 | — | Compact |
+| **M** | `Q4_K_M` | `Qwen3.5-2B-M-TS-Q4_K_M.gguf` | 1.073 GB | 4.562 | 66.54 / 75.54 | — | **Recommended · Uniform Q4** |
+| L | `Q8_0` | `Qwen3.5-2B-L-TS-Q8_0.gguf` | 2.013 GB | 8.558 | 65.80 / 74.94 | — | Uniform Q8 |
+
+> **Hub class versus file type:** XS and S are TheStage mixed-precision schedules. Their Hub labels describe the whole-file size class for discoverability; they are not stock `Q3_K_S`, `Q3_K_L`, or `Q4_K_S` conversions. M and L use the actual `MOSTLY_Q4_K_M` and `MOSTLY_Q8_0` GGUF file types. Exact tensor-type counts and SHA-256 digests are in [`release-manifest.json`](./release-manifest.json).
+
+IFEval is shown as prompt-strict / instruction-strict. The BF16 reference scored **65.43 / 74.70** on IFEval. MMLU-Pro is not reported for the BF16 reference in this release.
+
+## Quickstart
+
+Use a current [llama.cpp](https://github.com/ggml-org/llama.cpp) build. The command below selects the recommended **M** file explicitly through its Hub quant label:
+
+```bash
+llama-cli -hf TheStageAI/Qwen3.5-2B-GGUF:Q4_K_M
+```
+
+## Evaluation
+
+- **IFEval:** 541 prompts, native chat template, `enable_thinking=false`, temperature 0.
+- **MMLU-Pro:** 12,032 questions for complete rows, native chat template, `enable_thinking=true`, temperature 1, top-p 0.95.
+- A dash means **not reported**, not zero and not a score reconstructed from partial subjects.
+
+These protocols intentionally exercise different operating modes. IFEval measures non-thinking instruction and format adherence; MMLU-Pro includes long sampled reasoning.
+
+## What this release is
+
+This release starts from [`Qwen/Qwen3.5-2B`](https://huggingface.co/Qwen/Qwen3.5-2B) at revision [`15852e8c16360a2fea060d615a32b45270f8a8fc`](https://huggingface.co/Qwen/Qwen3.5-2B/tree/15852e8c16360a2fea060d615a32b45270f8a8fc) and applies TheStageAI's scale-tuned deployment compression pipeline.
+
+- **XS:** minimum-size adaptive mixed precision.
+- **S:** compact adaptive mixed precision.
+- **M:** uniform Q4 operating point.
+- **L:** uniform Q8 operating point.
+
+The four products are operating points, not a promise that benchmark scores increase monotonically with file size. The recommended row is selected separately for each base model from the release evaluations.
+
+## Limitations
+
+- The files in this repository contain the language-model GGUF. Multimodal projector files are not included.
+- Small score reversals between BF16 and quantized rows should be read as evaluation variation, not as a claim that quantization improves the base model.
+- For XS, use `enable_thinking=false`. A headline thinking-mode MMLU-Pro score is not reported when long generations do not produce a stable product metric.
+- MMLU-Pro is not reported for this model in this release; diagnostic partial subjects are deliberately not aggregated.
+
+## Provenance
+
+The exact base revision, file sizes, GGUF file types, whole-file BPW, SHA-256 digests, held-out KL values, and evaluation IDs are recorded in [`release-manifest.json`](./release-manifest.json). Release filenames differ from the internal artifact paths; the payload bytes must match those digests exactly.
+
+## License
+
+The model weights are released under the upstream model's **apache-2.0** license. llama.cpp and any surrounding runtime code retain their own licenses.

release-manifest.json

@@ -0,0 +1,217 @@
+{
+  "artifact_scope": "language_model_gguf",
+  "base_model": "Qwen/Qwen3.5-2B",
+  "base_model_revision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
+  "benchmark_protocols": {
+    "ifeval": {
+      "decoding": "temperature=0",
+      "mode": "native chat, enable_thinking=false",
+      "sample_count": 541
+    },
+    "mmlu_pro": {
+      "decoding": "temperature=1, top_p=0.95",
+      "mode": "native chat, enable_thinking=true",
+      "protocol": "mmlu_pro_cardlike_thinking_v2",
+      "sample_count": 12032
+    }
+  },
+  "bf16_reference": {
+    "ifeval": {
+      "instruction_loose": 0.7853717026378897,
+      "instruction_strict": 0.7470023980815348,
+      "prompt_loose": 0.7024029574861368,
+      "prompt_strict": 0.6543438077634011,
+      "run_id": "release_20260715_bf16_ifeval_public1280_v1_qwen3p5_2b_bf16_lm_eval",
+      "sample_count": 541,
+      "status": "complete"
+    },
+    "mmlu_pro": {
+      "accuracy": null,
+      "correct": null,
+      "run_id": null,
+      "sample_count": null,
+      "status": "not_reported"
+    }
+  },
+  "display_name": "Qwen3.5 2B",
+  "family": "Qwen 3.5",
+  "generated_at": "2026-07-20T21:32:16.184050+00:00",
+  "license": "apache-2.0",
+  "model_key": "qwen3p5_2b",
+  "recommended_product": "M",
+  "repo_id": "TheStageAI/Qwen3.5-2B-GGUF",
+  "schema": "thestageai.gguf_release_manifest_v1",
+  "variants": [
+    {
+      "benchmarks": {
+        "ifeval": {
+          "instruction_loose": 0.6858513189448441,
+          "instruction_strict": 0.6414868105515588,
+          "prompt_loose": 0.5748613678373382,
+          "prompt_strict": 0.5268022181146026,
+          "run_id": "release_20260715_quant_ifeval_public1280_v1_qwen3p5_2b_xs_lm_eval",
+          "sample_count": 541,
+          "status": "complete"
+        },
+        "mmlu_pro": {
+          "accuracy": null,
+          "correct": null,
+          "run_id": null,
+          "sample_count": null,
+          "status": "not_reported"
+        }
+      },
+      "evaluation_id": "d8c4c0602b65f13ecbf487c38876c8d0286614de6076619c3f7819fccd5b19b5",
+      "file_bytes": 738499904,
+      "filename": "Qwen3.5-2B-XS-TS-Q3_K_S.gguf",
+      "gguf_file_type": "MOSTLY_Q2_K",
+      "heldout_kl_mean": 0.16548452705393263,
+      "hub_quant_label": "Q3_K_S",
+      "hub_size_class": "3-bit",
+      "lm_head_policy": "skip_tied",
+      "output_embedding_mode": "tied_alias",
+      "parameter_count": 1881825088,
+      "positioning": "Minimum size",
+      "product": "XS",
+      "recommended": false,
+      "sha256": "c0d4816bdf93a21f307893675d0c47c083c36fad3bfee49663b91456e8940d30",
+      "tensor_count": 320,
+      "tensor_type_counts": {
+        "BF16": 36,
+        "F32": 133,
+        "Q2_K": 87,
+        "Q3_K": 42,
+        "Q4_K": 20,
+        "Q5_K": 2
+      },
+      "whole_file_bpw": 3.139504978265015
+    },
+    {
+      "benchmarks": {
+        "ifeval": {
+          "instruction_loose": 0.7685851318944844,
+          "instruction_strict": 0.7338129496402878,
+          "prompt_loose": 0.6728280961182994,
+          "prompt_strict": 0.6321626617375231,
+          "run_id": "release_20260715_quant_ifeval_public1280_v1_qwen3p5_2b_s_lm_eval",
+          "sample_count": 541,
+          "status": "complete"
+        },
+        "mmlu_pro": {
+          "accuracy": null,
+          "correct": null,
+          "run_id": null,
+          "sample_count": null,
+          "status": "not_reported"
+        }
+      },
+      "evaluation_id": "61cc4cfe035a572833ab91e039129118569a8d0596e415b58159069bc53a7081",
+      "file_bytes": 966516096,
+      "filename": "Qwen3.5-2B-S-TS-Q4_K_S.gguf",
+      "gguf_file_type": "MOSTLY_Q2_K",
+      "heldout_kl_mean": 0.04493591902074028,
+      "hub_quant_label": "Q4_K_S",
+      "hub_size_class": "4-bit",
+      "lm_head_policy": "skip_tied",
+      "output_embedding_mode": "tied_alias",
+      "parameter_count": 1881825088,
+      "positioning": "Compact",
+      "product": "S",
+      "recommended": false,
+      "sha256": "3a7171517f56f500ef8e6fc1d20b56a155cb2e6bd770d0728213298a2d562ea3",
+      "tensor_count": 320,
+      "tensor_type_counts": {
+        "BF16": 36,
+        "F32": 133,
+        "Q2_K": 1,
+        "Q3_K": 35,
+        "Q4_K": 91,
+        "Q5_K": 20,
+        "Q6_K": 4
+      },
+      "whole_file_bpw": 4.108845618706089
+    },
+    {
+      "benchmarks": {
+        "ifeval": {
+          "instruction_loose": 0.7865707434052758,
+          "instruction_strict": 0.7553956834532374,
+          "prompt_loose": 0.7042513863216266,
+          "prompt_strict": 0.6654343807763401,
+          "run_id": "release_20260715_quant_ifeval_public1280_v1_qwen3p5_2b_m_lm_eval",
+          "sample_count": 541,
+          "status": "complete"
+        },
+        "mmlu_pro": {
+          "accuracy": null,
+          "correct": null,
+          "run_id": null,
+          "sample_count": null,
+          "status": "not_reported"
+        }
+      },
+      "evaluation_id": "0b8c9cb609568247d30150522bacc4886b4aaac899dd52161f693324966ad2f7",
+      "file_bytes": 1073069440,
+      "filename": "Qwen3.5-2B-M-TS-Q4_K_M.gguf",
+      "gguf_file_type": "MOSTLY_Q4_K_M",
+      "heldout_kl_mean": 0.020367745511607253,
+      "hub_quant_label": "Q4_K_M",
+      "hub_size_class": "4-bit",
+      "lm_head_policy": "skip_tied",
+      "output_embedding_mode": "tied_alias",
+      "parameter_count": 1881825088,
+      "positioning": "Uniform Q4",
+      "product": "M",
+      "recommended": true,
+      "sha256": "8d497863b95e392baf022258f864c34f4a28613df340c500bb647486c52657ae",
+      "tensor_count": 320,
+      "tensor_type_counts": {
+        "BF16": 36,
+        "F32": 133,
+        "Q4_K": 151
+      },
+      "whole_file_bpw": 4.561824355909533
+    },
+    {
+      "benchmarks": {
+        "ifeval": {
+          "instruction_loose": 0.7865707434052758,
+          "instruction_strict": 0.749400479616307,
+          "prompt_loose": 0.7060998151571165,
+          "prompt_strict": 0.6580406654343808,
+          "run_id": "release_20260715_quant_ifeval_public1280_v1_qwen3p5_2b_l_lm_eval",
+          "sample_count": 541,
+          "status": "complete"
+        },
+        "mmlu_pro": {
+          "accuracy": null,
+          "correct": null,
+          "run_id": null,
+          "sample_count": null,
+          "status": "not_reported"
+        }
+      },
+      "evaluation_id": "09ef25ae7982f25e20a1812bbc8de081a026e7966ae401dec2eae3ffbb3bb6c3",
+      "file_bytes": 2013117824,
+      "filename": "Qwen3.5-2B-L-TS-Q8_0.gguf",
+      "gguf_file_type": "MOSTLY_Q8_0",
+      "heldout_kl_mean": 0.000715782698124712,
+      "hub_quant_label": "Q8_0",
+      "hub_size_class": "8-bit",
+      "lm_head_policy": "skip_tied",
+      "output_embedding_mode": "tied_alias",
+      "parameter_count": 1881825088,
+      "positioning": "Uniform Q8",
+      "product": "L",
+      "recommended": false,
+      "sha256": "fe5c7daff8dffebfee0f8d74d108c5ba0d03093eeb41b385916772e55cb27baf",
+      "tensor_count": 320,
+      "tensor_type_counts": {
+        "BF16": 36,
+        "F32": 133,
+        "Q8_0": 151
+      },
+      "whole_file_bpw": 8.558150645720374
+    }
+  ]
+}