Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,32 @@
+---
+license: apache-2.0
+base_model: Qwen/Qwen3-4B
+tags: [code-review, distillation, taid, knowledge-distillation, mlx, qwen3]
+language: [en]
+pipeline_tag: text-generation
+---
+
+# Qwen3-4B Code-Review (TAID, from Qwen3-8B)
+
+A **Qwen3-4B** student distilled (forward-KL logit-level KD / TAID) from the **Qwen3-8B**
+teacher to emit structured JSON code-review findings.
+
+- **Held-out F1:** **0.562** on `eval_set_100` (precision 0.517 / recall 0.616).
+- **Best model in the project.** Beats the prior best single student (0.509) by +0.053 and the best weight-space merge (0.535) by +0.027, with the highest precision AND recall of any student.
+- **License:** Apache-2.0 (pure Qwen3 lineage -- no third-party teacher-license strings).
+
+**Method.** Cached-logit knowledge distillation on Apple MLX. The **Qwen3-8B** teacher (8-bit) reviews
+each chunk once (thinking disabled -> clean JSON targets); we cache its top-k=50 logits over the
+*response* positions and train a LoRA student (rank 16 / 16 layers, lr 1e-4, seq 768, 30 epochs) toward
+that distribution (forward-KL / TAID, native temperature), then fuse. All Qwen3 sizes share a tokenizer,
+so logit-level distillation from the 8B works directly.
+
+**Prompt contract.** Given a numbered code chunk, emit one JSON object
+`{"findings":[{category, subtype, severity, confidence, title, body, evidence, line}]}`; `evidence` is a
+verbatim source substring, `line` 1-based. Held-out eval: `eval_set_100` (100 labeled chunks, py/js/c/go).
+
+**Why this teacher.** The project's dominant finding was that *teacher quality* is the single biggest
+lever. Swapping the Qwen2.5-Coder-7B / Gemma-9B teachers (best prior student F1 0.509, best merge 0.535)
+for Qwen3-8B produced a new best on the first attempt, at both sizes -- while a dozen objective/capacity
+tricks (reverse-KL, temperature, hybrid loss, LoRA depth, on-policy GKD) all failed to move the ceiling.
+