harness: protoLab evals @ claw-eval 49d2b6a

judge: protolabs/reasoning

kind: gate

grading: deterministic


grading: solver


grading: execution


trials: 1 (+3x outliers both sides)

grading: judge

note: T28 excluded (bimodal judge); T12 reproducible miss 0/3 (duplicate detection)


kind: coherence

grading: detectors+adversarial-judge


kind: speed

hardware: RTX PRO 6000 Blackwell (sm120)

method: vllm bench serve, random dataset seed 42, client-side


engine: llama.cpp b9829

method: llama-cli -n 200 greedy; blackwell mtp = mean n=6 prompts


note: acceptance 0.52 (NVFP4) vs 0.49 (Q4_K_M) same-box