Add Step 3.7 ROCmFPX Q3 QualityPlus card

.gitattributes

@@ -1,35 +1,2 @@
-*.7z filter=lfs diff=lfs merge=lfs -text
-*.arrow filter=lfs diff=lfs merge=lfs -text
-*.bin filter=lfs diff=lfs merge=lfs -text
-*.bz2 filter=lfs diff=lfs merge=lfs -text
-*.ckpt filter=lfs diff=lfs merge=lfs -text
-*.ftz filter=lfs diff=lfs merge=lfs -text
-*.gz filter=lfs diff=lfs merge=lfs -text
-*.h5 filter=lfs diff=lfs merge=lfs -text
-*.joblib filter=lfs diff=lfs merge=lfs -text
-*.lfs.* filter=lfs diff=lfs merge=lfs -text
-*.mlmodel filter=lfs diff=lfs merge=lfs -text
-*.model filter=lfs diff=lfs merge=lfs -text
-*.msgpack filter=lfs diff=lfs merge=lfs -text
-*.npy filter=lfs diff=lfs merge=lfs -text
-*.npz filter=lfs diff=lfs merge=lfs -text
-*.onnx filter=lfs diff=lfs merge=lfs -text
-*.ot filter=lfs diff=lfs merge=lfs -text
-*.parquet filter=lfs diff=lfs merge=lfs -text
-*.pb filter=lfs diff=lfs merge=lfs -text
-*.pickle filter=lfs diff=lfs merge=lfs -text
-*.pkl filter=lfs diff=lfs merge=lfs -text
-*.pt filter=lfs diff=lfs merge=lfs -text
-*.pth filter=lfs diff=lfs merge=lfs -text
-*.rar filter=lfs diff=lfs merge=lfs -text
-*.safetensors filter=lfs diff=lfs merge=lfs -text
-saved_model/**/* filter=lfs diff=lfs merge=lfs -text
-*.tar.* filter=lfs diff=lfs merge=lfs -text
-*.tar filter=lfs diff=lfs merge=lfs -text
-*.tflite filter=lfs diff=lfs merge=lfs -text
-*.tgz filter=lfs diff=lfs merge=lfs -text
-*.wasm filter=lfs diff=lfs merge=lfs -text
-*.xz filter=lfs diff=lfs merge=lfs -text
-*.zip filter=lfs diff=lfs merge=lfs -text
-*.zst filter=lfs diff=lfs merge=lfs -text
-*tfevents* filter=lfs diff=lfs merge=lfs -text
+*.gguf filter=lfs diff=lfs merge=lfs -text
+*.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,231 @@
+---
+license: apache-2.0
+base_model:
+- stepfun-ai/Step-3.7-Flash
+- notSnix/Step-3.7-Flash-MTP-Draft-GGUF
+base_model_relation: quantized
+language:
+- en
+library_name: llama.cpp
+pipeline_tag: text-generation
+tags:
+- stepfun
+- step3p7
+- step-3.7
+- step-3.7-flash
+- gguf
+- mtp
+- speculative-decoding
+- rocm
+- vulkan
+- rocmfpx
+- fpx3
+- q3
+- q3_0_rocmfpx
+- qualityplus
+- amd
+- ryzen-ai-max-395
+- strix-halo
+- agentic
+- tool-calling
+- long-context
+---
+
+![StepFun Step 3.7 ROCmFPX Q3 QualityPlus](assets/stepfun.png)
+
+# Step-3.7-Flash-ROCmFPX-Q3-QualityPlus
+
+This is an extremely high quality FPX3 / ROCmFPX Q3 GGUF build of `stepfun-ai/Step-3.7-Flash`, tuned for AMD Strix Halo local serving with Step MTP.
+
+The goal is simple: keep Step 3.7 Flash useful at 64K+ context on a 128 GB unified-memory machine without falling back to a nearly-FP4-sized "Q3" file. This release is a true tight Q3-weight build: `3.57 BPW`, `81.77 GiB` of language-model shards, and strong agent/tool behavior in local evals.
+
+Use this if you want the Step 3.7 behavior profile, MTP support, and a much smaller local footprint than the stock GGUF Q3_K_L or ROCmFP4 STRIX_LEAN builds.
+
+> Runtime note: these GGUFs use ROCmFPX tensor types. They are intended for the Chadrock / ROCmFPX llama.cpp runner family, not stock upstream llama.cpp.
+
+## Why This One
+
+Step 3.7 is huge. The practical local problem is not only speed; it is fitting enough context, KV, and agent workload into memory.
+
+This FPX3/Q3 QualityPlus recipe was built for that constraint:
+
+- `3.57 BPW` effective language-model size
+- `81.77 GiB` total language GGUF shards
+- `16.31%` smaller than the local ROCmFP4 STRIX_LEAN build
+- `14.35%` smaller than StepFun's original `Q3_K_L` GGUF split
+- 64K one-slot serving profile with q8_0 target KV and q8_0 draft KV
+- Step MTP Q8 draft support through `draft-mtp`
+- fixed Step tool/chat template using native `tool_response` observations and protocol-boundary escaping
+
+In practice, the original StepFun `Q3_K_L` local split was not a compact 3-bit-feeling model: it measured about `95.46 GiB`, or roughly `4.17 BPW` by effective size. This QualityPlus build is the one I would publish/use as the FPX3 lane.
+
+## Size Comparison
+
+Measured from local GGUF shards:
+
+| Build | Effective BPW | Shard total | Difference vs this release |
+| --- | ---: | ---: | ---: |
+| **ROCmFPX Q3 QualityPlus** | `3.57 BPW` | `81.77 GiB` | baseline |
+| StepFun original `Q3_K_L` | `~4.17 BPW` | `95.46 GiB` | `+13.70 GiB` larger |
+| ROCmFP4 STRIX_LEAN | `~4.27 BPW` | `97.70 GiB` | `+15.93 GiB` larger |
+
+That size gap matters because Step 3.7 needs memory for long context, q8 KV, and MTP draft state. On the tested Strix Halo host, the Q3 QualityPlus 64K MTP profile used about `96.3 GiB` peak pooled GPU memory during long tool/Hermes runs, leaving enough RAM headroom to run the evals cleanly.
+
+## Quality Highlights
+
+This is not a throwaway low-bit build. The recipe protects the tensors that were most important for behavior while pushing the giant expert FFN tensors into `q3_0_rocmfpx`.
+
+Local quality results on AMD Ryzen AI Max+ 395 / Strix Halo:
+
+| Benchmark | Result | Notes |
+| --- | ---: | --- |
+| Tool-Eval full, 69 scenarios | [`88/100`, `122/138` raw points](evals/tool-eval-q3-qualityplus.json) | Same headline score as the recorded Step ROCmFP4 tool-eval row |
+| HermesAgent-20, best Q3 template run | `85/100` | `13.40 min`, `35.31 tok/s` decode, `96.37 GiB` peak pooled GPU |
+| HermesAgent-20, native tool-response template run | `82/100` | `12.82 min`, `35.76 tok/s` decode, `96.30 GiB` peak pooled GPU |
+
+The best recorded Q3 HermesAgent-20 run was very close to the local BF16 Qwen3.6 27B MTP reference row:
+
+| Model / row | HermesAgent-20 score | Wall time |
+| --- | ---: | ---: |
+| BF16 Qwen3.6 27B MTP GGUF | `87/100` | `42.4 min` |
+| **Step 3.7 ROCmFPX Q3 QualityPlus** | `85/100` | `13.4 min` |
+
+That is within two points of the BF16 Qwen3.6 27B row on the local HermesAgent-20 suite, while running in a much more compact Step 3.7 Q3 package.
+
+Exact Q3 QualityPlus tool-eval score summary: [`evals/tool-eval-q3-qualityplus.json`](evals/tool-eval-q3-qualityplus.json). Public reference page for the Step 3.7 tool-calling work: [StepFun Step 3.7 Tool Eval on llm.ciru.ai](https://llm.ciru.ai/tooleval/stepfun-step37-rocmfp4-mtp-vulkan-64k/). The Q3 QualityPlus full run used the same 69-scenario tool-eval harness and scored `88/100` locally.
+
+## Speed
+
+Q3 QualityPlus speed was effectively tied with the local ROCmFP4 Step build while using much less disk space.
+
+Short-context MTP speed, Vulkan0, q8_0/q8_0 target KV, q8_0/q8_0 draft KV, one slot, `n_max=2`, `p_min=0.75`, `b8192/u2048`, 128 generated tokens:
+
+| Prompt | PP tok/s | TG tok/s |
+| ---: | ---: | ---: |
+| `2k` | `309.44` | `29.97` |
+| `4k` | `325.18` | `29.39` |
+| `8k` | `311.15` | `28.58` |
+| `16k` | `306.37` | `26.26` |
+
+Compared with the local ROCmFP4 Step build:
+
+| Prompt | Q3 QualityPlus TG | ROCmFP4 TG | Takeaway |
+| ---: | ---: | ---: | --- |
+| `2k` | `29.97` | `26.52` | Q3 faster |
+| `4k` | `29.39` | `29.37` | tied |
+| `8k` | `28.58` | `28.02` | tied/slightly Q3 |
+| `16k` | `26.26` | `26.42` | tied |
+
+128K stress row:
+
+| Context | PP tok/s | TG tok/s | Peak pooled GPU |
+| ---: | ---: | ---: | ---: |
+| `~130k prompt` | `146.67` | `14.52` | `~95.36 GiB` |
+
+At 128K, MTP initialized but produced no accepted drafts in that particular row, so treat the 128K decode number as an effective no-draft long-context decode reference.
+
+## Files
+
+Published shard names intentionally match the model name:
+
+```text
+Step-3.7-Flash-ROCmFPX-Q3-QualityPlus-00001-of-00009.gguf
+Step-3.7-Flash-ROCmFPX-Q3-QualityPlus-00002-of-00009.gguf
+Step-3.7-Flash-ROCmFPX-Q3-QualityPlus-00003-of-00009.gguf
+Step-3.7-Flash-ROCmFPX-Q3-QualityPlus-00004-of-00009.gguf
+Step-3.7-Flash-ROCmFPX-Q3-QualityPlus-00005-of-00009.gguf
+Step-3.7-Flash-ROCmFPX-Q3-QualityPlus-00006-of-00009.gguf
+Step-3.7-Flash-ROCmFPX-Q3-QualityPlus-00007-of-00009.gguf
+Step-3.7-Flash-ROCmFPX-Q3-QualityPlus-00008-of-00009.gguf
+Step-3.7-Flash-ROCmFPX-Q3-QualityPlus-00009-of-00009.gguf
+```
+
+The Step MTP draft model is not duplicated here. Use the Q8 draft from [`notSnix/Step-3.7-Flash-MTP-Draft-GGUF`](https://huggingface.co/notSnix/Step-3.7-Flash-MTP-Draft-GGUF), for example `Step-3.7-Flash-MTP-Q8_0.gguf`.
+
+## Recommended Serving Profile
+
+The locally tested 64K profile:
+
+```text
+context: 65536
+slots: 1
+backend: Vulkan0 target + Vulkan0 draft
+MTP: --spec-type draft-mtp
+speculative.n_max: 2
+speculative.n_min: 0
+speculative.p_min: 0.75
+speculative.p_split: 0.10
+batch / ubatch: 8192 / 2048
+target KV: q8_0 / q8_0
+draft KV: q8_0 / q8_0
+sampler: temperature 1.0, top_p 0.95, min_p 0.0, repeat_penalty 1.0
+reasoning: on, DeepSeek format
+chat template: Step native tool_response template with protocol-boundary escaping
+```
+
+Example shape:
+
+```bash
+/path/to/ROCmFPX/build/bin/llama-server \
+  -m Step-3.7-Flash-ROCmFPX-Q3-QualityPlus-00001-of-00009.gguf \
+  --alias step-3.7-flash-rocmfpx-q3-qualityplus \
+  --host 127.0.0.1 \
+  --port 8080 \
+  --jinja \
+  -c 65536 \
+  --reasoning on \
+  --reasoning-format deepseek \
+  --reasoning-budget -1 \
+  -dev Vulkan0 \
+  -ngl 999 \
+  -fa on \
+  -b 8192 \
+  -ub 2048 \
+  --parallel 1 \
+  --no-mmap \
+  --ctk q8_0 \
+  --ctv q8_0 \
+  --spec-draft-model Step-3.7-Flash-MTP-Q8_0.gguf \
+  --spec-draft-device Vulkan0 \
+  --spec-type draft-mtp \
+  --spec-draft-ngl all \
+  --spec-draft-type-k q8_0 \
+  --spec-draft-type-v q8_0 \
+  --spec-draft-n-max 2 \
+  --spec-draft-n-min 0 \
+  --spec-draft-p-min 0.75 \
+  --spec-draft-p-split 0.10 \
+  --chat-template-file step37-native-tool-response-template.jinja \
+  --metrics
+```
+
+## Template Note
+
+The best local Step setup uses a Step-native tool/chat template that renders tool outputs as `tool_response` turns and escapes protocol-boundary tokens inside tool output. This is a general protocol-adapter fix: tool/file/search results stay observations instead of being flattened into user text.
+
+That matters for real agents because Step 3.7 can otherwise confuse tool output with conversation authority, especially in file/search-result injection cases.
+
+## Build Notes
+
+The QualityPlus policy used here:
+
+- huge `ffn_*_exps` tensors: `q3_0_rocmfpx`
+- attention q/output protected at `q5_K`
+- attention k/v protected at `q4_K`
+- shared/dense FFN protected at `q5_K`
+- output/token embeddings at `q4_0_rocmfp4_fast`
+
+Converter-reported size: `83726.08 MiB / 3.57 BPW`, 9 shards.
+
+## Credits
+
+- Base model: [`stepfun-ai/Step-3.7-Flash`](https://huggingface.co/stepfun-ai/Step-3.7-Flash)
+- MTP draft GGUF source: [`notSnix/Step-3.7-Flash-MTP-Draft-GGUF`](https://huggingface.co/notSnix/Step-3.7-Flash-MTP-Draft-GGUF)
+- Runtime family: [`ciru-ai/ROCmFPX`](https://github.com/ciru-ai/ROCmFPX)
+- Quantization, Strix Halo profile, and local benchmark work: Crown / Ciru
+
+## Caveats
+
+- This is a custom ROCmFPX GGUF release. Use a compatible ROCmFPX/Chadrock llama.cpp runner.
+- Quality numbers are local Strix Halo measurements and depend on runtime, chat template, KV type, and MTP settings.
+- The model is strong but not perfect at autonomous email/message side effects; it can be cautious and ask for subject/body/recipient details instead of sending with inferred defaults.

evals/tool-eval-q3-qualityplus.json

@@ -0,0 +1,42 @@
+{
+  "model": "Step-3.7-Flash-ROCmFPX-Q3-QualityPlus",
+  "benchmark": "Tool-Eval full",
+  "harness_version": "2.0.7",
+  "run_id": "2026-07-05T06-07-28.330900Z_405f5a32",
+  "scenario_count": 69,
+  "final_score": 88,
+  "deployability": 70,
+  "responsiveness": 27,
+  "total_points": 122,
+  "max_points": 138,
+  "status_counts": {
+    "pass": 57,
+    "partial": 8,
+    "fail": 4
+  },
+  "runtime": {
+    "backend": "llama.cpp ROCmFPX / Vulkan",
+    "context": 65536,
+    "mtp": true,
+    "speculative_n_max": 2,
+    "speculative_p_min": 0.75,
+    "target_kv": "q8_0/q8_0",
+    "draft_kv": "q8_0/q8_0",
+    "batch": 8192,
+    "ubatch": 2048
+  },
+  "speed_and_memory": {
+    "pp_tps": 145.65656394243823,
+    "tg_tps": 32.338072639262656,
+    "prompt_tokens": 34333,
+    "predicted_tokens": 33430,
+    "peak_pooled_gpu_gib": 96.34914779663086,
+    "peak_ram_used_gib": 107.00336074829102,
+    "wall_time": "21:21.23"
+  },
+  "notes": [
+    "Local AMD Ryzen AI Max+ 395 / Strix Halo measurement.",
+    "Uses the Step native tool_response chat template with protocol-boundary escaping.",
+    "The public llm.ciru.ai StepFun tool-eval page currently documents the Step tool-calling methodology and matching FP4 score row; this JSON records the exact Q3 QualityPlus run summary."
+  ]
+}