gemma-4-12b-it-qat-4bit-frontierdistill

mlx-community/gemma-4-12B-it-qat-4bit with a rank-8 LoRA adapter fused in, fine-tuned for agentic coding and tool use. Standalone checkpoint — no separate adapter needed. Trained and merged on Apple Silicon (MLX) via mlx-vlm.

Training data

16,730 examples combined from three frontier-model distillation trace datasets (all CC-BY-4.0):

• greghavens/fable-5-coding-and-debugging-traces
• greghavens/gpt-5.6-sol-coding-and-debugging-traces
• greghavens/kimi-k3-coding-and-debugging-traces

Split by trajectory (not row) so no trajectory leaks across train/valid/test.

Training config

Method	LoRA, all language_model linear layers
Rank / alpha	8 / 16
Iterations	1000 (~1.1 epochs)
Effective batch	16 (grad accumulation)
Learning rate	2e-5, constant
Max sequence length	8192
Loss	response-only (prompt masked)

Evaluation

Held-out test split (never seen in training), base vs. this adapter:

Metric	Base	+LoRA	Δ
Perplexity (held-out NLL)	4.23	2.15	−49%
HumanEval pass@1 (n=164)	92.1%	89.0%	−3.1pp
Tool-name accuracy (n=150)	68.0%	74.0%	+6.0pp
Tool-argument overlap (n=150)	23.7%	41.3%	+17.7pp

The adapter meaningfully improves tool selection and argument correctness — the actual training objective — at a small cost to isolated, out-of-distribution coding-puzzle accuracy (HumanEval). Perplexity alone overstates the improvement; treat it alongside the functional numbers, not in place of them.

Usage

pip install mlx-vlm
python -m mlx_vlm.generate \
    --model True2456/gemma-4-12b-it-qat-4bit-frontierdistill \
    --prompt "Your prompt here"