jackasda211233 commited on
Commit
0ec7d9e
·
verified ·
1 Parent(s): c81c3dc

Upload PATCHCODE_TESTING_PROCESS.html with huggingface_hub

Browse files
Files changed (1) hide show
  1. PATCHCODE_TESTING_PROCESS.html +9 -4
PATCHCODE_TESTING_PROCESS.html CHANGED
@@ -179,6 +179,7 @@ li{margin:.25em 0}
179
  </tbody>
180
  </table>
181
  <p>At λ=1.0 the adapter was net-neutral-to-harmful (one checkpoint fell <em>below</em> the un-adapted base). The mechanism: an over-loud LoRA delta pushes activations into regimes that hurt calibrated behaviour (preamble returns, over-claiming). λ=0.5 (effective alpha/r = 1.0) keeps the style direction but respects base calibration. So the merge was done at <strong>λ=0.5 onto SignalLatch (ckpt386-s010)</strong>, then exported to BF16 GGUF. (A future v2 could bake the good strength in by training at alpha=r=32, removing the inference-time knob.)</p>
 
182
  <h2>Why the final testing moved to merged IQ4_NL</h2>
183
  <p>The key question was not "best adapter in BF16" — it was "what we would actually deploy". The deploy target was a merged GGUF, <code>IQ4_NL</code>, imatrix-quantized, on the custom ik-llama runtime (Jinja + DeepSeek reasoning format + flash attention + graph split, temp <code>0.7</code>).</p>
184
  <p>Live LoRA loading is not the production path for this release (the tested serving profile uses flash attention, which conflicts with live LoRA on this runtime). So the long-term path became: <strong>merge the adapter first, then export + quantize a full GGUF.</strong> That is why the upload is a merged GGUF, not an adapter.</p>
@@ -271,10 +272,14 @@ li{margin:.25em 0}
271
  <p><strong>Rule:</strong> 3-seed differences &lt;0.13 on this suite are meaningless. Use <strong>5+ seeds, same-condition head-to-head</strong> before any ship call. Only non-noise axes (size, recipe methodology/safety, long-context at ceiling) reliably tiebreak. HumanEval was rejected — it saturates on Qwen and is the wrong mode for an agent.</p>
272
  <p>This is exactly how a 3-seed pass almost shipped the <em>weaker</em> model.</p>
273
  <h2>The ship decision</h2>
274
- <p>With build, discipline, long-context, and autonomy all <strong>tied within noise</strong>, the decision fell to non-noise axes, where plain <code>IQ4_NL</code> wins all three:
275
- - <strong>smaller</strong> (16.6 G vs 20–29 G)
276
- - <strong>marginal long-context</strong> edge (0.975 vs 0.935–0.969)
277
- - <strong>plain-quant recipe</strong> — the fleet's proven pattern; promotion/mixed recipes carry evidence-harmful risk (discipline collapse) for zero measured benefit</p>
 
 
 
 
278
  <p>Ship: <strong>plain <code>IQ4_NL</code> (reasoning-imatrix)</strong>. The mixed-recipe <code>c76</code> is retained on disk as the build-heavy fallback if a future, harder build-gate ever discriminates beyond the noise floor (use 5+ seeds).</p>
279
  <h2>What the testing says and does not say</h2>
280
  <p><strong>Does say:</strong>
 
179
  </tbody>
180
  </table>
181
  <p>At λ=1.0 the adapter was net-neutral-to-harmful (one checkpoint fell <em>below</em> the un-adapted base). The mechanism: an over-loud LoRA delta pushes activations into regimes that hurt calibrated behaviour (preamble returns, over-claiming). λ=0.5 (effective alpha/r = 1.0) keeps the style direction but respects base calibration. So the merge was done at <strong>λ=0.5 onto SignalLatch (ckpt386-s010)</strong>, then exported to BF16 GGUF. (A future v2 could bake the good strength in by training at alpha=r=32, removing the inference-time knob.)</p>
182
+ <p><img alt="Merge strength sweep — λ=0.5 wins on all three checkpoints; trained default λ=1.0 is over-applied." src="assets/lambda_sweep.png" /></p>
183
  <h2>Why the final testing moved to merged IQ4_NL</h2>
184
  <p>The key question was not "best adapter in BF16" — it was "what we would actually deploy". The deploy target was a merged GGUF, <code>IQ4_NL</code>, imatrix-quantized, on the custom ik-llama runtime (Jinja + DeepSeek reasoning format + flash attention + graph split, temp <code>0.7</code>).</p>
185
  <p>Live LoRA loading is not the production path for this release (the tested serving profile uses flash attention, which conflicts with live LoRA on this runtime). So the long-term path became: <strong>merge the adapter first, then export + quantize a full GGUF.</strong> That is why the upload is a merged GGUF, not an adapter.</p>
 
272
  <p><strong>Rule:</strong> 3-seed differences &lt;0.13 on this suite are meaningless. Use <strong>5+ seeds, same-condition head-to-head</strong> before any ship call. Only non-noise axes (size, recipe methodology/safety, long-context at ceiling) reliably tiebreak. HumanEval was rejected — it saturates on Qwen and is the wrong mode for an agent.</p>
273
  <p>This is exactly how a 3-seed pass almost shipped the <em>weaker</em> model.</p>
274
  <h2>The ship decision</h2>
275
+ <p>With build, discipline, long-context, and autonomy all <strong>tied within noise</strong>, the decision fell to non-noise axes, where plain <code>IQ4_NL</code> wins all three:</p>
276
+ <p><img alt="No candidate clears BOTH build and discipline (≥0.90) promotion destroys discipline; precision does not fix build." src="assets/bothquest.png" /></p>
277
+ <p><img alt="Ship scoreboard (5-seed): IQ4_NL ties the field within noise on build/long-context/discipline, and wins on size." src="assets/ship_scoreboard.png" /></p>
278
+ <ul>
279
+ <li><strong>smaller</strong> (16.6 G vs 20–29 G)</li>
280
+ <li><strong>marginal long-context</strong> edge (0.975 vs 0.935–0.969)</li>
281
+ <li><strong>plain-quant recipe</strong> — the fleet's proven pattern; promotion/mixed recipes carry evidence-harmful risk (discipline collapse) for zero measured benefit</li>
282
+ </ul>
283
  <p>Ship: <strong>plain <code>IQ4_NL</code> (reasoning-imatrix)</strong>. The mixed-recipe <code>c76</code> is retained on disk as the build-heavy fallback if a future, harder build-gate ever discriminates beyond the noise floor (use 5+ seeds).</p>
284
  <h2>What the testing says and does not say</h2>
285
  <p><strong>Does say:</strong>