Add private release-card draft

README.md

@@ -0,0 +1,73 @@
+---
+license: apache-2.0
+base_model:
+  - Qwen/Qwen3.5-4B
+base_model_relation: quantized
+library_name: llama.cpp
+pipeline_tag: text-generation
+tags:
+  - gguf
+  - llama.cpp
+  - quantization
+  - mixed-precision
+---
+
+# Qwen3.5 4B — TheStageAI GGUF
+
+Four text-only GGUF checkpoints from **1.519 GB** to **4.486 GB**, evaluated on IFEval and MMLU-Pro where a complete release score is available.
+
+[Qwen3.5 0.8B](https://huggingface.co/TheStageAI/Qwen3.5-0.8B-GGUF) · [Qwen3.5 2B](https://huggingface.co/TheStageAI/Qwen3.5-2B-GGUF) · **[Qwen3.5 4B](https://huggingface.co/TheStageAI/Qwen3.5-4B-GGUF)** · [Qwen3.5 9B](https://huggingface.co/TheStageAI/Qwen3.5-9B-GGUF) · [Gemma 4 E2B IT](https://huggingface.co/TheStageAI/gemma-4-E2B-it-GGUF) · [Gemma 4 E4B IT](https://huggingface.co/TheStageAI/gemma-4-E4B-it-GGUF) · [Gemma 4 12B IT](https://huggingface.co/TheStageAI/gemma-4-12B-it-GGUF)
+
+## Choose a file
+
+| Variant | Hub class | File | Size | Whole-file BPW | IFEval P / I (%) | MMLU-Pro (%) | Use |
+| --- | --- | --- | ---: | ---: | ---: | ---: | --- |
+| XS | `Q3_K_S` | `Qwen3.5-4B-XS-TS-Q3_K_S.gguf` | 1.519 GB | 2.889 | 70.43 / 78.30 | — | Minimum size |
+| S | `Q4_K_S` | `Qwen3.5-4B-S-TS-Q4_K_S.gguf` | 1.904 GB | 3.622 | 77.82 / 83.93 | 74.39 | Compact |
+| **M** | `Q4_K_M` | `Qwen3.5-4B-M-TS-Q4_K_M.gguf` | 2.386 GB | 4.538 | 80.22 / 86.09 | 78.86 | **Recommended · Uniform Q4** |
+| L | `Q8_0` | `Qwen3.5-4B-L-TS-Q8_0.gguf` | 4.486 GB | 8.533 | 81.70 / 87.05 | 79.59 | Uniform Q8 |
+
+> **Hub class versus file type:** XS and S are TheStage mixed-precision schedules. Their Hub labels describe the whole-file size class for discoverability; they are not stock `Q3_K_S`, `Q3_K_L`, or `Q4_K_S` conversions. M and L use the actual `MOSTLY_Q4_K_M` and `MOSTLY_Q8_0` GGUF file types. Exact tensor-type counts and SHA-256 digests are in [`release-manifest.json`](./release-manifest.json).
+
+IFEval is shown as prompt-strict / instruction-strict. The BF16 reference scored **82.44 / 87.53** on IFEval and **79.55** on MMLU-Pro.
+
+## Quickstart
+
+Use a current [llama.cpp](https://github.com/ggml-org/llama.cpp) build. The command below selects the recommended **M** file explicitly through its Hub quant label:
+
+```bash
+llama-cli -hf TheStageAI/Qwen3.5-4B-GGUF:Q4_K_M
+```
+
+## Evaluation
+
+- **IFEval:** 541 prompts, native chat template, `enable_thinking=false`, temperature 0.
+- **MMLU-Pro:** 12,032 questions for complete rows, native chat template, `enable_thinking=true`, temperature 1, top-p 0.95.
+- A dash means **not reported**, not zero and not a score reconstructed from partial subjects.
+
+These protocols intentionally exercise different operating modes. IFEval measures non-thinking instruction and format adherence; MMLU-Pro includes long sampled reasoning.
+
+## What this release is
+
+This release starts from [`Qwen/Qwen3.5-4B`](https://huggingface.co/Qwen/Qwen3.5-4B) at revision [`851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a`](https://huggingface.co/Qwen/Qwen3.5-4B/tree/851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a) and applies TheStageAI's scale-tuned deployment compression pipeline.
+
+- **XS:** minimum-size adaptive mixed precision.
+- **S:** compact adaptive mixed precision.
+- **M:** uniform Q4 operating point.
+- **L:** uniform Q8 operating point.
+
+The four products are operating points, not a promise that benchmark scores increase monotonically with file size. The recommended row is selected separately for each base model from the release evaluations.
+
+## Limitations
+
+- The files in this repository contain the language-model GGUF. Multimodal projector files are not included.
+- Small score reversals between BF16 and quantized rows should be read as evaluation variation, not as a claim that quantization improves the base model.
+- For XS, use `enable_thinking=false`. A headline thinking-mode MMLU-Pro score is not reported when long generations do not produce a stable product metric.
+
+## Provenance
+
+The exact base revision, file sizes, GGUF file types, whole-file BPW, SHA-256 digests, held-out KL values, and evaluation IDs are recorded in [`release-manifest.json`](./release-manifest.json). Release filenames differ from the internal artifact paths; the payload bytes must match those digests exactly.
+
+## License
+
+The model weights are released under the upstream model's **apache-2.0** license. llama.cpp and any surrounding runtime code retain their own licenses.

release-manifest.json

@@ -0,0 +1,216 @@
+{
+  "artifact_scope": "language_model_gguf",
+  "base_model": "Qwen/Qwen3.5-4B",
+  "base_model_revision": "851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a",
+  "benchmark_protocols": {
+    "ifeval": {
+      "decoding": "temperature=0",
+      "mode": "native chat, enable_thinking=false",
+      "sample_count": 541
+    },
+    "mmlu_pro": {
+      "decoding": "temperature=1, top_p=0.95",
+      "mode": "native chat, enable_thinking=true",
+      "protocol": "mmlu_pro_cardlike_thinking_v2",
+      "sample_count": 12032
+    }
+  },
+  "bf16_reference": {
+    "ifeval": {
+      "instruction_loose": 0.9052757793764988,
+      "instruction_strict": 0.8752997601918465,
+      "prompt_loose": 0.8650646950092421,
+      "prompt_strict": 0.8243992606284658,
+      "run_id": "release_20260715_bf16_ifeval_public1280_v1_qwen3p5_4b_bf16_lm_eval",
+      "sample_count": 541,
+      "status": "complete"
+    },
+    "mmlu_pro": {
+      "accuracy": 0.7954621010638298,
+      "correct": 9571,
+      "run_id": "release_20260715_bf16_qwen35_mmlu_pro_cardlike_v2_qwen3p5_4b_bf16_mmlu_pro_official",
+      "sample_count": 12032,
+      "status": "complete"
+    }
+  },
+  "display_name": "Qwen3.5 4B",
+  "family": "Qwen 3.5",
+  "generated_at": "2026-07-20T21:32:16.184050+00:00",
+  "license": "apache-2.0",
+  "model_key": "qwen3p5_4b",
+  "recommended_product": "M",
+  "repo_id": "TheStageAI/Qwen3.5-4B-GGUF",
+  "schema": "thestageai.gguf_release_manifest_v1",
+  "variants": [
+    {
+      "benchmarks": {
+        "ifeval": {
+          "instruction_loose": 0.8141486810551559,
+          "instruction_strict": 0.7829736211031175,
+          "prompt_loose": 0.7375231053604436,
+          "prompt_strict": 0.7042513863216266,
+          "run_id": "release_20260715_quant_ifeval_public1280_v1_qwen3p5_4b_xs_lm_eval",
+          "sample_count": 541,
+          "status": "complete"
+        },
+        "mmlu_pro": {
+          "accuracy": null,
+          "correct": null,
+          "run_id": null,
+          "sample_count": null,
+          "status": "not_reported"
+        }
+      },
+      "evaluation_id": "7795c3073ccadeb4c6bda64f984e005bf38abf7cc4ab871b1922aa7c9edea0ad",
+      "file_bytes": 1518977312,
+      "filename": "Qwen3.5-4B-XS-TS-Q3_K_S.gguf",
+      "gguf_file_type": "MOSTLY_Q2_K",
+      "heldout_kl_mean": 0.17456917879548242,
+      "hub_quant_label": "Q3_K_S",
+      "hub_size_class": "3-bit",
+      "lm_head_policy": "skip_tied",
+      "output_embedding_mode": "tied_alias",
+      "parameter_count": 4205751296,
+      "positioning": "Minimum size",
+      "product": "XS",
+      "recommended": false,
+      "sha256": "a1d49f1d782073748589d0df0893658040231866ae6d69667994bef5cc6ce61f",
+      "tensor_count": 426,
+      "tensor_type_counts": {
+        "BF16": 48,
+        "F32": 177,
+        "Q2_K": 159,
+        "Q3_K": 34,
+        "Q4_K": 8
+      },
+      "whole_file_bpw": 2.8893335912557014
+    },
+    {
+      "benchmarks": {
+        "ifeval": {
+          "instruction_loose": 0.8693045563549161,
+          "instruction_strict": 0.8393285371702638,
+          "prompt_loose": 0.8096118299445472,
+          "prompt_strict": 0.7781885397412199,
+          "run_id": "release_20260715_quant_ifeval_public1280_v1_qwen3p5_4b_s_lm_eval",
+          "sample_count": 541,
+          "status": "complete"
+        },
+        "mmlu_pro": {
+          "accuracy": 0.7439328457446809,
+          "correct": 8951,
+          "run_id": "release_20260715_quant_qwen35_mmlu_pro_cardlike_v2_qwen3p5_4b_s_mmlu_pro_official",
+          "sample_count": 12032,
+          "status": "complete"
+        }
+      },
+      "evaluation_id": "2a822bbb0cecfc257f43d7230b9f6cbb2dbd4ba346eb330b3fb93b5e436953b9",
+      "file_bytes": 1904339232,
+      "filename": "Qwen3.5-4B-S-TS-Q4_K_S.gguf",
+      "gguf_file_type": "MOSTLY_Q2_K",
+      "heldout_kl_mean": 0.08894240464308578,
+      "hub_quant_label": "Q4_K_S",
+      "hub_size_class": "4-bit",
+      "lm_head_policy": "skip_tied",
+      "output_embedding_mode": "tied_alias",
+      "parameter_count": 4205751296,
+      "positioning": "Compact",
+      "product": "S",
+      "recommended": false,
+      "sha256": "9f020bfc77a9503a641e3e0511de04b4bfba16b9facfa62c3772b4f8e318ba55",
+      "tensor_count": 426,
+      "tensor_type_counts": {
+        "BF16": 48,
+        "F32": 177,
+        "Q2_K": 51,
+        "Q3_K": 107,
+        "Q4_K": 35,
+        "Q5_K": 6,
+        "Q6_K": 2
+      },
+      "whole_file_bpw": 3.6223525319933705
+    },
+    {
+      "benchmarks": {
+        "ifeval": {
+          "instruction_loose": 0.8884892086330936,
+          "instruction_strict": 0.8609112709832134,
+          "prompt_loose": 0.8391866913123844,
+          "prompt_strict": 0.8022181146025879,
+          "run_id": "release_20260715_quant_ifeval_public1280_v1_qwen3p5_4b_m_lm_eval",
+          "sample_count": 541,
+          "status": "complete"
+        },
+        "mmlu_pro": {
+          "accuracy": 0.7886469414893617,
+          "correct": 9489,
+          "run_id": "release_20260715_quant_qwen35_mmlu_pro_cardlike_v2_qwen3p5_4b_m_mmlu_pro_official",
+          "sample_count": 12032,
+          "status": "complete"
+        }
+      },
+      "evaluation_id": "62b3e9269b087d285daded1afba6d01426e81c4d83e5e92149297656f74fa06f",
+      "file_bytes": 2385660192,
+      "filename": "Qwen3.5-4B-M-TS-Q4_K_M.gguf",
+      "gguf_file_type": "MOSTLY_Q4_K_M",
+      "heldout_kl_mean": 0.015916582520935554,
+      "hub_quant_label": "Q4_K_M",
+      "hub_size_class": "4-bit",
+      "lm_head_policy": "skip_tied",
+      "output_embedding_mode": "tied_alias",
+      "parameter_count": 4205751296,
+      "positioning": "Uniform Q4",
+      "product": "M",
+      "recommended": true,
+      "sha256": "f8e45572b9cc35161d4772b09bccfd383fe0bb03fc6d69b40a9138731302290b",
+      "tensor_count": 426,
+      "tensor_type_counts": {
+        "BF16": 48,
+        "F32": 177,
+        "Q4_K": 201
+      },
+      "whole_file_bpw": 4.537900649082983
+    },
+    {
+      "benchmarks": {
+        "ifeval": {
+          "instruction_loose": 0.9016786570743405,
+          "instruction_strict": 0.8705035971223022,
+          "prompt_loose": 0.8595194085027726,
+          "prompt_strict": 0.8170055452865065,
+          "run_id": "release_20260715_quant_ifeval_public1280_v1_qwen3p5_4b_l_lm_eval",
+          "sample_count": 541,
+          "status": "complete"
+        },
+        "mmlu_pro": {
+          "accuracy": 0.7958776595744681,
+          "correct": 9576,
+          "run_id": "release_20260715_quant_qwen35_mmlu_pro_cardlike_v2_qwen3p5_4b_l_mmlu_pro_official",
+          "sample_count": 12032,
+          "status": "complete"
+        }
+      },
+      "evaluation_id": "9021d506f8010ebb7ff1fd3b2446f98d009119b522a2cd2e84cf77f8b0745bcd",
+      "file_bytes": 4486088992,
+      "filename": "Qwen3.5-4B-L-TS-Q8_0.gguf",
+      "gguf_file_type": "MOSTLY_Q8_0",
+      "heldout_kl_mean": 0.0006302701462033587,
+      "hub_quant_label": "Q8_0",
+      "hub_size_class": "8-bit",
+      "lm_head_policy": "skip_tied",
+      "output_embedding_mode": "tied_alias",
+      "parameter_count": 4205751296,
+      "positioning": "Uniform Q8",
+      "product": "L",
+      "recommended": false,
+      "sha256": "b81d86c3ccbd1af44474597c0ef97af8e4d71ba40f3b5e7a20b4d6c2f709203c",
+      "tensor_count": 426,
+      "tensor_type_counts": {
+        "BF16": 48,
+        "F32": 177,
+        "Q8_0": 201
+      },
+      "whole_file_bpw": 8.533246359605949
+    }
+  ]
+}