base_model: mlx-community/gemma-4-12B-it-qat-4bit
license: gemma
library_name: mlx-vlm
tags:
- gemma4
- agentic
- coding
- tool-use
- mlx
- lora-merged
datasets:
- greghavens/fable-5-coding-and-debugging-traces
- greghavens/gpt-5.6-sol-coding-and-debugging-traces
- greghavens/kimi-k3-coding-and-debugging-traces
gemma-4-12b-it-qat-4bit-frontierdistill
mlx-community/gemma-4-12B-it-qat-4bit with a rank-8 LoRA adapter fused in, fine-tuned for agentic coding and tool use. Standalone checkpoint — no separate adapter needed. Trained and merged on Apple Silicon (MLX) via mlx-vlm.
Training data
16,730 examples combined from three frontier-model distillation trace datasets (all CC-BY-4.0):
- greghavens/fable-5-coding-and-debugging-traces
- greghavens/gpt-5.6-sol-coding-and-debugging-traces
- greghavens/kimi-k3-coding-and-debugging-traces
Split by trajectory (not row) so no trajectory leaks across train/valid/test.
Training config
| Method | LoRA, all language_model linear layers |
| Rank / alpha | 8 / 16 |
| Iterations | 1000 (~1.1 epochs) |
| Effective batch | 16 (grad accumulation) |
| Learning rate | 2e-5, constant |
| Max sequence length | 8192 |
| Loss | response-only (prompt masked) |
Evaluation
Held-out test split (never seen in training), base vs. this adapter:
| Metric | Base | +LoRA | Δ |
|---|---|---|---|
| Perplexity (held-out NLL) | 4.23 | 2.15 | −49% |
| HumanEval pass@1 (n=164) | 92.1% | 89.0% | −3.1pp |
| Tool-name accuracy (n=150) | 68.0% | 74.0% | +6.0pp |
| Tool-argument overlap (n=150) | 23.7% | 41.3% | +17.7pp |
The adapter meaningfully improves tool selection and argument correctness — the actual training objective — at a small cost to isolated, out-of-distribution coding-puzzle accuracy (HumanEval). Perplexity alone overstates the improvement; treat it alongside the functional numbers, not in place of them.
Usage
pip install mlx-vlm
python -m mlx_vlm.generate \
--model True2456/gemma-4-12b-it-qat-4bit-frontierdistill \
--prompt "Your prompt here"