Add model card with YAML metadata

README.md

@@ -1,250 +1,40 @@
 ---
+license: apache-2.0
 base_model: Qwen/Qwen3.5-9B
+tags:
+  - qwen3.5
+  - reasoning
+  - distillation
+  - lora
+  - sft
+  - gguf
 datasets:
-- Crownelius/Opus-4.6-Reasoning-3300x
-library_name: peft
-license: apache-2.0
-language:
-- en
+  - Crownelius/Opus-4.6-Reasoning-3300x
 pipeline_tag: text-generation
-tags:
-- reasoning
-- distillation
-- lora
-- qwen3.5
-- sft
-- tool-use
-- function-calling
-- vision
-- multimodal
-- thinking
 ---
 
-# qwen3.5-9b-qwen3.6-distilled  (r3-crown)
-
-LoRA adapter distilled from reasoning traces in
-[`Crownelius/Opus-4.6-Reasoning-3300x`](https://huggingface.co/datasets/Crownelius/Opus-4.6-Reasoning-3300x)
-on top of [`Qwen/Qwen3.5-9B`](https://huggingface.co/Qwen/Qwen3.5-9B).
-Packaged for Ollama via `llama-export-lora` so the base model's vision
-tower, tool-calling handlers, and native Q4_K_M quantization are preserved
-byte-for-byte — only the text-reasoning tensors our LoRA touched are
-updated.
-
-## What ships
-
-| Artifact | Purpose |
-|---|---|
-| `adapter_model.safetensors` | PEFT LoRA weights (attention + MLP + linear_attn targets, 496 tensors total) |
-| `adapter_config.json` | PEFT config |
-| `final_report.json` | Machine-readable eval report (hard tool matrix, vision probe, OCR tool demo, training metrics) |
-| `filtered_adapter/` | LoRA with `linear_attn.*` tensors removed — this is what converts cleanly to GGUF |
-| GGUF repo | `llama-export-lora` merged into base `qwen3.5:9b` Q4_K_M GGUF + Modelfile |
-
-## Build pipeline (the path that actually works)
-
-The single-file combined-GGUF path via `convert_hf_to_gguf.py` is blocked
-for Qwen3.5 multimodal today: it emits a separate `clip.*` mmproj GGUF
-which is incompatible with Ollama's native `qwen35.vision.*` loader
-format. We solved this by not rebuilding the base GGUF at all.
-
-```bash
-# 1. Train LoRA (target_modules="all-linear" for completeness)
-python app.py train  # produces final_adapter/
-
-# 2. Drop the Gated DeltaNet LoRA tensors — llama.cpp's lora-to-gguf
-#    converter can't reshape-reorder the v-heads inside a low-rank
-#    A@B decomposition. We keep attention + MLP targets, which are
-#    256/496 of the tensors in the adapter.
-.venv/bin/python -c "
-from safetensors.torch import load_file, save_file
-import json
-w = load_file('final_adapter/adapter_model.safetensors')
-save_file({k:v for k,v in w.items() if 'linear_attn' not in k},
-          'final_adapter_filtered/adapter_model.safetensors')
-cfg = json.load(open('final_adapter/adapter_config.json'))
-cfg['target_modules'] = ['q_proj','k_proj','v_proj','o_proj','gate_proj','up_proj','down_proj']
-json.dump(cfg, open('final_adapter_filtered/adapter_config.json','w'), indent=2)
-"
-
-# 3. Convert filtered LoRA to GGUF
-python vendor/llama.cpp/convert_lora_to_gguf.py final_adapter_filtered/ \
-    --outtype f16 --outfile lora.f16.gguf \
-    --base outputs/merged_vision/...
-
-# 4. Merge LoRA into the base qwen3.5:9b Q4_K_M GGUF byte-for-byte
-vendor/llama.cpp/build/bin/llama-export-lora \
-    --model /srv/ollama/models/blobs/sha256-dec52a...  \
-    --lora lora.f16.gguf \
-    --output merged.q4km.gguf
-
-# 5. Register with Ollama
-cat > Modelfile <<'EOM'
-FROM ./merged.q4km.gguf
-RENDERER qwen3.5
-PARSER qwen3.5
-PARAMETER temperature 0.6
-PARAMETER top_p 0.95
-PARAMETER stop "<|im_end|>"
-EOM
-ollama create qwen3.5-9b-qwen3.6-distilled:crown -f Modelfile
-```
-
-`llama-export-lora` updates the **128 text tensors** the filtered LoRA
-covers (q/k/v/o + gate/up/down in the standard attention/MLP layers) and
-leaves the other **755 tensors** (all 441 vision + 15 MTP + the linear_attn
-DeltaNet tensors + embeddings + norms) **exactly as base**. Vision is
-therefore not "preserved via state-dict surgery"; it is **literally the
-base model's vision**, bit-for-bit.
-
-## Capabilities — verified via Ollama
-
-```
-$ ollama show qwen3.5-9b-qwen3.6-distilled:crown
-  architecture        qwen35
-  parameters          9.7B
-  Capabilities
-    completion
-    vision
-    tools
-    thinking
-```
-
-Smoke-test results (deterministic, `temperature=0`):
-
-| Path | Result |
-|---|---|
-| Completion | ✅ `7 * 23 - 45` → step-by-step → `116` (correct) |
-| Thinking | ✅ 706-char `<think>` block before the answer |
-| Tool calling | ✅ `get_weather({"city": "Paris"})` — structured `tool_calls` |
-| Vision | ✅ Base-level: 1/3 on 3-image rendered-text probe, identical failure pattern to base |
-
-## Hard tool-calling stress test — 22 scenarios
-
-Deterministic (`temperature=0, top_p=1, top_k=1, seed=42`) via the OpenAI-
-compatible gateway at `http://localhost:11435/v1/chat/completions`.
-
-| | Base `qwen3.5:9b` | Crown `:crown` |
-|---|---:|---:|
-| **Score** | **21 / 22** | **21 / 22** |
-| HX8 (G7 capitals, 7 parallel calls) | PASS | **PASS** ← fixed |
-| HX7 (roll_dice count=5000 vs schema max=1000) | FAIL | FAIL ← shared weakness |
-| H1 many-tool distraction latency | 8.4 s | **0.8 s** (10× faster) |
-
-**HX8 is the critical one.** Our first adapter (r1, trained on the
-original 500-row dataset) failed HX8 by substituting Brussels for
-Washington D.C. That was a factual regression from distillation on a
-small dataset. The r3-crown pipeline fixes it two ways at once:
-
-1. **Retrain on a larger, better-aligned dataset** (Crownelius has 2160
-   rows vs 500) — best val loss dropped from 0.7030 → **0.4439**.
-2. **Merge into base rather than replacing weights** — 755/883 tensors
-   are bit-for-bit from base, so base's factual knowledge can't be
-   regressed by LoRA overfitting on the unmodified tensors.
-
-**HX7 is a shared base-model weakness** (both models call
-`roll_dice(count=5000)` ignoring the JSON Schema `maximum: 1000`). Not
-introduced by distillation; documented honestly.
-
-## Image handling — base-level vision preserved
-
-| Probe | Base | Crown |
-|---|---|---|
-| `HELLO` 72pt on black | "HILO" (wrong) | "HILO" (wrong) |
-| `42` 72pt on red | **"42"** ✓ | **"42"** ✓ |
-| `BANANA` 72pt on yellow | "BANA" (wrong) | "BANA" (wrong) |
-
-Character-identical outputs confirm the vision tensors are not just
-"preserved" but literally the base model's weights — because we merged
-the LoRA into base, not the other way around. If you need OCR with higher
-fidelity, use the OCR-via-tool pattern below.
-
-## OCR-via-tool (recommended for small/dense text)
-
-No 9B VLM reliably reads small dense text pixel-perfectly. The distilled
-model's tool-calling strength makes it an excellent OCR orchestrator:
-
-```
-user: "What is the total on this receipt? /path/receipt.png"
-  ↓
-crown model emits: extract_text_from_image({"image_path": "/path/receipt.png"})
-  ↓
-tool runs Tesseract with 8 preprocessing variants × 3 PSM modes = 24 passes,
-returns best-scored text (88-91% confidence typical)
-  ↓
-crown model: "Based on the receipt, the total is EUR 34.76. Card: VISA ..."
-```
-
-On our 4-case hard benchmark (two images — a dense 19-line security log
-and a coffee receipt — with four questions across them), crown scored
-**4/4**, including:
-
-- Quoting the exact error line verbatim from a 1126-char log extraction
-- Distinguishing training loss from validation loss in the quoted text
-- Extracting per-item costs from a monospace receipt
-
-See `final_report.json → ocr_via_tool` for the full transcript with all
-tool calls.
-
-## Training recipe
+# Qwen3.5-9B Reasoning Distilled GGUF (R3 Crown)
 
-| | Round 1 (superseded) | Round 3 (crown, shipped) |
-|---|---|---|
-| Dataset | `khazarai/qwen3.6-plus-high-reasoning-500x` | `Crownelius/Opus-4.6-Reasoning-3300x` |
-| Rows | 500 | 2160 |
-| Splits | 425 / 37 / 38 | 1836 / 162 / 162 |
-| Epochs | 3 | early-stopped at 1.17 |
-| Steps completed | 54 | 90 |
-| LR | 2e-4 cosine | 2e-4 cosine |
-| LoRA | r=32 α=64 all-linear | r=32 α=64 all-linear |
-| Max seq length | 8192 | 5120 (capped for long tail) |
-| **Best val loss** | 0.7030 | **0.4439** |
-| Hardware | 3× A100 80GB PCIe DDP | 3× A100 80GB PCIe DDP |
-| Training time | 49 min | 28 min |
+GGUF quantized version of fine-tuned [Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B) with distilled Opus 4.6 reasoning traces. Early iteration (R3) — superseded by [R7 (86.8% diverse eval)](https://huggingface.co/cudabenchmarktest/qwen3.5-9b-r7-research).
 
-## Limitations
+## Training
 
-- **HX7**: both crown and base ignore JSON Schema `maximum: 1000` on numeric
-  arguments. Don't rely on schema range enforcement as a safety mechanism.
-- **`linear_attn` LoRA weights are not packaged in the GGUF** because
-  llama.cpp's LoRA converter can't reorder v-heads inside a low-rank A@B
-  decomposition. The HF PEFT adapter (`adapter_model.safetensors`) does
-  contain them, so if you use this via `transformers + peft` you get the
-  full LoRA; the Ollama GGUF gets ~52% of LoRA tensors.
-- **Vision OCR fidelity** on small dense text is base-level (matches
-  `qwen3.5:9b`). Use `extract_text_from_image` tool for reliable reads.
-- **Non-determinism** at default `temperature=0.6`. Published numbers are
-  at `temperature=0`.
+- **Base model**: [Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B)
+- **Method**: LoRA SFT (r=32, alpha=64, LR=2e-4)
+- **Data**: [Crownelius/Opus-4.6-Reasoning-3300x](https://huggingface.co/datasets/Crownelius/Opus-4.6-Reasoning-3300x) (2160 samples)
+- **Training suite**: [robit-man/fine_tuning_suite](https://github.com/robit-man/fine_tuning_suite)
 
-## Running it
+## Note
 
-```bash
-# Recommended: pull the merged GGUF + Modelfile from the GGUF repo
-hf download cudabenchmarktest/qwen3.5-9b-qwen3.6-reasoning-distilled-GGUF \
-    Modelfile merged.q4km.gguf \
-    --local-dir ./qwen-crown
-cd qwen-crown
-ollama create qwen3.5-9b-qwen3.6-distilled:crown -f Modelfile
-ollama show   qwen3.5-9b-qwen3.6-distilled:crown   # should list: completion, vision, tools, thinking
-ollama run    qwen3.5-9b-qwen3.6-distilled:crown
-```
+This early iteration had regressions in instruction following due to monoculture training data. See the [training suite](https://github.com/robit-man/fine_tuning_suite) for the improved R5/R7 approach.
 
-```bash
-# Or hit /api/chat with tools + images
-curl -s http://localhost:11434/api/chat -d '{
-  "model": "qwen3.5-9b-qwen3.6-distilled:crown",
-  "messages": [{"role": "user", "content": "Weather in Paris?"}],
-  "tools": [{"type":"function","function":{"name":"get_weather","description":"...","parameters":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}}}],
-  "stream": false
-}' | jq .message.tool_calls
-```
+## Successors
 
-## Audit trail
+| Model | Eval | Link |
+|-------|------|------|
+| R7 Research | 86.8% | [cudabenchmarktest/qwen3.5-9b-r7-research](https://huggingface.co/cudabenchmarktest/qwen3.5-9b-r7-research) |
+| R7 Vision | 86.8% | [cudabenchmarktest/qwen3.5-9b-r7-research-vision](https://huggingface.co/cudabenchmarktest/qwen3.5-9b-r7-research-vision) |
 
-All probe scripts + raw result JSONs are in this repo:
+## License
 
-- `scripts/hard_tool_tests.py` — 22 deterministic tool-calling scenarios
-- `scripts/ocr_tool_demo.py` — end-to-end OCR-as-tool agent loop
-- `scripts/image_probe.py` — 3-image vision probe
-- `tool_bench/` — raw per-row results for every probe
-- `final_report.json` — everything merged, schema-stable
-- `crown_manifest.json` — dataset split SHAs (seed 42)
+Apache 2.0 (inherited from Qwen3.5-9B).