michellemoorre commited on
Commit
57f1859
·
verified ·
1 Parent(s): 7127d3a

Update private release-card draft v3

Browse files
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ assets/thestage-edge-models-header.png filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -5,35 +5,52 @@ base_model:
5
  base_model_relation: quantized
6
  library_name: llama.cpp
7
  pipeline_tag: text-generation
 
8
  tags:
9
  - gguf
10
  - llama.cpp
11
- - quantization
12
  - mixed-precision
 
 
13
  ---
14
 
15
- # Gemma 4 E4B IT — TheStageAI GGUF
 
 
16
 
17
- Four optimized language-model GGUF files for text-only deployment with llama.cpp.
18
 
19
- > **Recommended: S · 3.76 GB** — **69.48 MMLU-Pro**, 99.4% of the BF16 reference.
 
 
 
 
 
 
 
 
 
 
 
 
20
 
21
  **Gemma 4 family:** [E2B](https://huggingface.co/TheStageAI/gemma-4-E2B-it-GGUF) · **[E4B](https://huggingface.co/TheStageAI/gemma-4-E4B-it-GGUF)** · [12B](https://huggingface.co/TheStageAI/gemma-4-12B-it-GGUF)
22
 
23
- ## Choose a checkpoint
24
 
25
- | Variant | Size | Best for | File |
26
  | --- | ---: | --- | --- |
27
- | XS | 2.91 GB | Minimum footprint · non-thinking only | [GGUF](./gemma-4-E4B-it-XS-TS-Q3_K_S.gguf) |
28
- | **S** | **3.76 GB** | **Recommended · Compact** | [GGUF](./gemma-4-E4B-it-S-TS-Q4_K_S.gguf) |
29
- | M | 4.33 GB | Q4 reference | [GGUF](./gemma-4-E4B-it-M-TS-Q4_K_M.gguf) |
30
- | L | 8.02 GB | Maximum fidelity | [GGUF](./gemma-4-E4B-it-L-TS-Q8_0.gguf) |
31
 
32
  Sizes use decimal MB/GB. Exact byte counts are recorded in [`release-manifest.json`](./release-manifest.json).
33
 
34
- ## Quickstart
35
 
36
- The command below selects the recommended **S** checkpoint by exact filename:
37
 
38
  ```bash
39
  llama-cli \
@@ -51,29 +68,35 @@ llama-cli \
51
  | M | 85.40 / 89.81 | 69.55 |
52
  | L | 84.84 / 89.09 | 70.01 |
53
 
54
- - **IFEval:** 541 prompts, native chat template, `enable_thinking=false`, temperature 0.
55
- - **MMLU-Pro:** complete 12,032-question runs only, native chat template, `enable_thinking=true`, temperature 1, top-p 0.95, 32,768-token output limit.
56
- - `—` means that no complete headline MMLU-Pro run is reported. Partial-subject accuracies are not used as release scores.
57
 
58
  The benchmarks intentionally exercise different modes: IFEval measures non-thinking instruction and format adherence, while MMLU-Pro includes long sampled reasoning.
59
 
60
- ## Reasoning mode
 
 
 
 
 
 
 
 
 
61
 
62
- **XS is not a supported reasoning operating point.** It is released for minimum-footprint, non-thinking chat and instruction following. Use S, M, or L when reasoning quality matters. The complete Gemma XS MMLU-Pro result is still shown for transparency; it does not change the intended deployment mode.
63
 
64
- ## About this release
65
 
66
- This release starts from [`google/gemma-4-E4B-it`](https://huggingface.co/google/gemma-4-E4B-it) at revision [`fee6332c1abaafb77f6f9624236c63aa2f1d0187`](https://huggingface.co/google/gemma-4-E4B-it/tree/fee6332c1abaafb77f6f9624236c63aa2f1d0187) and applies TheStageAI's adaptive per-tensor precision assignment with tuned quantization scales.
67
 
68
- - **XS:** minimum-footprint adaptive mixed precision for non-thinking use.
69
- - **S:** compact adaptive mixed precision.
70
- - **M:** uniform Q4 tensor assignment and reference operating point.
71
- - **L:** uniform Q8_0 tensor assignment for maximum fidelity.
72
 
73
- The recommendation is selected separately for each base model from the release evaluations.
74
 
75
  <details>
76
- <summary><b>Quantization labels and technical details</b></summary>
77
 
78
  | Variant | Hub selector | GGUF file type | Whole-file BPW |
79
  | --- | --- | --- | ---: |
@@ -82,19 +105,21 @@ The recommendation is selected separately for each base model from the release e
82
  | M | `Q4_K_M` | `MOSTLY_Q4_K_M` | 4.646 |
83
  | L | `Q8_0` | `MOSTLY_Q8_0` | 8.602 |
84
 
85
- XS and S are TheStage mixed-precision schedules. Their public Q-type suffixes provide an approximate size class for Hub discoverability; exact tensor-type inventories are recorded in [`release-manifest.json`](./release-manifest.json). M and L use uniform Q4_K and Q8_0 assignments for quantized decoder tensors.
 
 
86
 
87
  </details>
88
 
89
- ## Limitations
 
 
90
 
91
- - These are language-model GGUF files for text-only deployment. Multimodal projector files are not included.
92
- - File size does not include KV cache or runtime buffers; total memory use grows with context length.
93
- - Small score reversals between BF16 and quantized rows should be read as evaluation variation, not as a claim that quantization improves the base model.
94
 
95
- ## Provenance
96
 
97
- The exact base revision, byte sizes, GGUF file types, whole-file BPW, tensor inventories, SHA-256 digests, held-out KL values, and evaluation IDs are recorded in [`release-manifest.json`](./release-manifest.json). Use its SHA-256 digests to verify downloaded files.
98
 
99
  ## License
100
 
 
5
  base_model_relation: quantized
6
  library_name: llama.cpp
7
  pipeline_tag: text-generation
8
+ thumbnail: https://huggingface.co/TheStageAI/gemma-4-E4B-it-GGUF/resolve/main/assets/thestage-edge-models-header.png
9
  tags:
10
  - gguf
11
  - llama.cpp
12
+ - quantized
13
  - mixed-precision
14
+ - local-inference
15
+ - gemma4
16
  ---
17
 
18
+ <p align="center">
19
+ <img src="./assets/thestage-edge-models-header.png" width="100%" alt="TheStageAI Edge Models — the right model at every memory budget">
20
+ </p>
21
 
22
+ <h1 align="center">Gemma 4 E4B IT TheStageAI GGUF</h1>
23
 
24
+ <p align="center">
25
+ Four GGUF checkpoints for local inference with llama.cpp, from 2.91 GB to 8.02 GB.
26
+ <br>
27
+ <strong>Start with S — 3.76 GB and 100% of the BF16 instruction-strict IFEval score in our evaluation.</strong>
28
+ </p>
29
+
30
+ <p align="center">
31
+ <a href="https://github.com/TheStageAI/edge-lm"><img src="./assets/cta-edge-lm.svg" height="42" alt="Explore edge-lm on GitHub"></a>
32
+ &nbsp;
33
+ <a href="https://docs.thestage.ai/"><img src="./assets/cta-docs.svg" height="42" alt="Read TheStageAI documentation"></a>
34
+ &nbsp;
35
+ <a href="https://app.thestage.ai/"><img src="./assets/cta-platform.svg" height="42" alt="Open TheStageAI Platform"></a>
36
+ </p>
37
 
38
  **Gemma 4 family:** [E2B](https://huggingface.co/TheStageAI/gemma-4-E2B-it-GGUF) · **[E4B](https://huggingface.co/TheStageAI/gemma-4-E4B-it-GGUF)** · [12B](https://huggingface.co/TheStageAI/gemma-4-12B-it-GGUF)
39
 
40
+ ## Start here
41
 
42
+ | Tier | Size | Use | Download |
43
  | --- | ---: | --- | --- |
44
+ | XS | 2.91 GB | Smallest | [GGUF](./gemma-4-E4B-it-XS-TS-Q3_K_S.gguf) |
45
+ | **S** | **3.76 GB** | **Recommended · Adaptive compact** | [GGUF](./gemma-4-E4B-it-S-TS-Q4_K_S.gguf) |
46
+ | M | 4.33 GB | Fixed Q4 reference | [GGUF](./gemma-4-E4B-it-M-TS-Q4_K_M.gguf) |
47
+ | L | 8.02 GB | Q8 fidelity reference | [GGUF](./gemma-4-E4B-it-L-TS-Q8_0.gguf) |
48
 
49
  Sizes use decimal MB/GB. Exact byte counts are recorded in [`release-manifest.json`](./release-manifest.json).
50
 
51
+ ## Run with llama.cpp
52
 
53
+ This command selects the recommended **S** checkpoint by exact filename:
54
 
55
  ```bash
56
  llama-cli \
 
68
  | M | 85.40 / 89.81 | 69.55 |
69
  | L | 84.84 / 89.09 | 70.01 |
70
 
71
+ Only complete 12,032-question MMLU-Pro runs are reported. `` means that no complete release score is available; partial-subject results are not promoted to headline metrics.
 
 
72
 
73
  The benchmarks intentionally exercise different modes: IFEval measures non-thinking instruction and format adherence, while MMLU-Pro includes long sampled reasoning.
74
 
75
+ > **Reasoning:** use S, M, or L for long-form reasoning. XS remains available when minimum footprint is the priority.
76
+
77
+ <details>
78
+ <summary><b>Evaluation protocol and XS reasoning note</b></summary>
79
+
80
+ - **IFEval:** 541 prompts, native chat template, `enable_thinking=false`, temperature 0.
81
+ - **MMLU-Pro:** 12,032 questions, native chat template, `enable_thinking=true`, temperature 1, top-p 0.95, 32,768-token output limit.
82
+ - The headline BF16-retention percentage uses instruction-strict IFEval and is capped at 100%; the uncapped raw scores are shown in the table.
83
+
84
+ XS prioritizes minimum footprint and ordinary chat or instruction following. For long-form reasoning, start with S, M, or L. Complete Gemma XS MMLU-Pro results remain in the table so the full measured trade-off stays visible.
85
 
86
+ </details>
87
 
88
+ ## How these checkpoints were built
89
 
90
+ This release starts from [`google/gemma-4-E4B-it`](https://huggingface.co/google/gemma-4-E4B-it) at revision [`fee6332c1abaafb77f6f9624236c63aa2f1d0187`](https://huggingface.co/google/gemma-4-E4B-it/tree/fee6332c1abaafb77f6f9624236c63aa2f1d0187).
91
 
92
+ - **XS and S** use model-specific mixed-precision schedules selected for explicit size targets.
93
+ - **M** keeps fixed Q4_K precision across quantized decoder tensors.
94
+ - **L** keeps fixed Q8_0 precision as the high-fidelity reference.
 
95
 
96
+ All four checkpoints receive scale tuning without changing their stored quantized codes or tensor layout. The recommended tier is selected independently for each base model from end-to-end evaluation.
97
 
98
  <details>
99
+ <summary><b>File types, Hub selectors, and effective BPW</b></summary>
100
 
101
  | Variant | Hub selector | GGUF file type | Whole-file BPW |
102
  | --- | --- | --- | ---: |
 
105
  | M | `Q4_K_M` | `MOSTLY_Q4_K_M` | 4.646 |
106
  | L | `Q8_0` | `MOSTLY_Q8_0` | 8.602 |
107
 
108
+ XS and S are TheStageAI mixed-precision schedules. Their public Q-type suffixes provide approximate Hub size classes for discoverability; they do not describe every tensor in the payload. M and L use fixed Q4_K and Q8_0 assignments for quantized decoder tensors.
109
+
110
+ The exact tensor-type inventory and SHA-256 digest for every file are recorded in [`release-manifest.json`](./release-manifest.json). File size does not include KV cache or runtime buffers, which grow with context length. GGUF runtimes treat multimodal projectors as separate artifacts; this release contains the language-model files listed above.
111
 
112
  </details>
113
 
114
+ ## TheStageAI edge stack
115
+
116
+ These checkpoints provide a portable llama.cpp deployment path. For native compressed models on Apple Silicon, explore [edge-lm](https://github.com/TheStageAI/edge-lm). For custom compression, compilation, and deployment workflows, use the [TheStageAI Platform](https://app.thestage.ai/) and [documentation](https://docs.thestage.ai/).
117
 
118
+ Optimizing for a specific device, latency target, or memory budget? [Talk to the TheStageAI team →](https://app.thestage.ai/contact)
 
 
119
 
120
+ ## Reproducibility
121
 
122
+ The exact base revision, byte sizes, GGUF file types, whole-file BPW, tensor inventories, SHA-256 digests, held-out KL values, and evaluation IDs are recorded in [`release-manifest.json`](./release-manifest.json). Use its SHA-256 digests to verify downloaded files. Export and load gates used [llama.cpp revision `bec4772f6a2527d371557b5d2032641e5ff7619c`](https://github.com/ggml-org/llama.cpp/commit/bec4772f6a2527d371557b5d2032641e5ff7619c).
123
 
124
  ## License
125
 
assets/cta-docs.svg ADDED
assets/cta-edge-lm.svg ADDED
assets/cta-platform.svg ADDED
assets/thestage-edge-models-header.png ADDED

Git LFS Details

  • SHA256: 7a92eef43d92f92bcb3156e74b11d007a86081c5818d78cb9a6b258cb2a1d8ed
  • Pointer size: 132 Bytes
  • Size of remote file: 1.04 MB
release-manifest.json CHANGED
@@ -36,7 +36,7 @@
36
  },
37
  "display_name": "Gemma 4 E4B IT",
38
  "family": "Gemma 4",
39
- "generated_at": "2026-07-20T22:39:04.914041+00:00",
40
  "license": "apache-2.0",
41
  "model_key": "gemma4_e4b",
42
  "reasoning_policy": {
@@ -48,8 +48,14 @@
48
  "xs": "non_thinking_only"
49
  },
50
  "recommended_product": "S",
 
 
 
 
 
51
  "repo_id": "TheStageAI/gemma-4-E4B-it-GGUF",
52
- "schema": "thestageai.gguf_release_manifest_v2",
 
53
  "variants": [
54
  {
55
  "benchmarks": {
@@ -81,7 +87,7 @@
81
  "lm_head_policy": "skip_tied",
82
  "output_embedding_mode": "tied_alias",
83
  "parameter_count": 7463013674,
84
- "positioning": "Minimum footprint \u00b7 non-thinking only",
85
  "product": "XS",
86
  "reasoning_support": "non_thinking_only",
87
  "recommended": false,
@@ -128,7 +134,7 @@
128
  "lm_head_policy": "skip_tied",
129
  "output_embedding_mode": "tied_alias",
130
  "parameter_count": 7463013674,
131
- "positioning": "Compact",
132
  "product": "S",
133
  "reasoning_support": "supported",
134
  "recommended": true,
@@ -176,7 +182,7 @@
176
  "lm_head_policy": "skip_tied",
177
  "output_embedding_mode": "tied_alias",
178
  "parameter_count": 7463013674,
179
- "positioning": "Q4 reference",
180
  "product": "M",
181
  "reasoning_support": "supported",
182
  "recommended": false,
@@ -219,7 +225,7 @@
219
  "lm_head_policy": "skip_tied",
220
  "output_embedding_mode": "tied_alias",
221
  "parameter_count": 7463013674,
222
- "positioning": "Maximum fidelity",
223
  "product": "L",
224
  "reasoning_support": "supported",
225
  "recommended": false,
 
36
  },
37
  "display_name": "Gemma 4 E4B IT",
38
  "family": "Gemma 4",
39
+ "generated_at": "2026-07-20T23:43:48.671876+00:00",
40
  "license": "apache-2.0",
41
  "model_key": "gemma4_e4b",
42
  "reasoning_policy": {
 
48
  "xs": "non_thinking_only"
49
  },
50
  "recommended_product": "S",
51
+ "recommended_quality": {
52
+ "metric": "ifeval_instruction_strict",
53
+ "ratio_to_bf16": 1.0053691275167784,
54
+ "retention_percent_display": 100
55
+ },
56
  "repo_id": "TheStageAI/gemma-4-E4B-it-GGUF",
57
+ "schema": "thestageai.gguf_release_manifest_v3",
58
+ "tested_llama_cpp_revision": "bec4772f6a2527d371557b5d2032641e5ff7619c",
59
  "variants": [
60
  {
61
  "benchmarks": {
 
87
  "lm_head_policy": "skip_tied",
88
  "output_embedding_mode": "tied_alias",
89
  "parameter_count": 7463013674,
90
+ "positioning": "Smallest",
91
  "product": "XS",
92
  "reasoning_support": "non_thinking_only",
93
  "recommended": false,
 
134
  "lm_head_policy": "skip_tied",
135
  "output_embedding_mode": "tied_alias",
136
  "parameter_count": 7463013674,
137
+ "positioning": "Adaptive compact",
138
  "product": "S",
139
  "reasoning_support": "supported",
140
  "recommended": true,
 
182
  "lm_head_policy": "skip_tied",
183
  "output_embedding_mode": "tied_alias",
184
  "parameter_count": 7463013674,
185
+ "positioning": "Fixed Q4 reference",
186
  "product": "M",
187
  "reasoning_support": "supported",
188
  "recommended": false,
 
225
  "lm_head_policy": "skip_tied",
226
  "output_embedding_mode": "tied_alias",
227
  "parameter_count": 7463013674,
228
+ "positioning": "Q8 fidelity reference",
229
  "product": "L",
230
  "reasoning_support": "supported",
231
  "recommended": false,