causal_leak_fix_verification.md

Fock-PARFLM v2.1 (SQ3 structured V_theta) — Causal Leak Fix vs. Leaky Baseline (A2, seed 0)

Comparison of two runs of the same arm, A2 (SQ3: mixture of K_mix=8 diagonal quadratic wells, tau=1.0, K_xi=4, lambda_V=0.01), differing only in whether the cross-layer Fock register state carries the causal leak identified in Fock-PARFLM_Causal_Leak_Audit_Results.md:

• leaky — legacy register lifecycle (prefix_causal_registers=False behavior; the field didn't exist yet in this run). Registers are one bank per sequence, evolved across the full window, so token t's force in later layers can be influenced by tokens after t.
• fixed (checkpoint/ckpt_best.pt in this repo) — prefix_causal_registers=True. Registers are one bank per position; the bank consulted at position t is built exclusively from tokens 1…t.

Same architecture in both: d=256, L=8, V_theta=SQ3(K_mix=8, tau=1.0), K_xi=4, lambda_V=0.01, M=16, d_k=64, top_k=8, same seed (0), same 16,000 steps, TinyStories (5M train tokens). This is the structured-V_theta counterpart of the MLP-V_theta Fock-PARFLM v2.1 leak fix; the base architecture, register mechanism, and fix are identical — only the scalar potential parameterisation differs.

Headline: the fix costs a modest, honest tax on standard PPL

	leaky	fixed	Δ
Best val PPL (step 14,400)	10.36	10.90	+0.54 (+5.2%)
Final val PPL (step 16,000)	10.68	11.36	+0.68

The leaky number (10.36) is the one originally published on this model card. The fixed run is consistently worse across training — the same qualitative signature as the MLP variant's leak-closure tax (+0.40 PPL there), just somewhat larger here (+0.54 PPL). Consistent with that pattern, the SQ3 mixture potential appears to lean on the leaked future-token signal slightly more than the MLP potential does at this scale, though both effects are small compared to the ~33× (+3.51 nats) inflation measured on the much larger OpenWebText-scale variant.

The proof the fix actually worked, at the trained scale

Future-perturbation test, run on the live trained model at steps 8,000 and 16,000: perturbing tokens after the position of interest changed past logits by exactly 0.0 — bit-exact, not "small":

"probe_max_dlogit_past": 0.0,
"probe_mean_dnll_past_nats": 0.0,
"probe_gate_zero_control": 0.0

Honest PPL vs. standard PPL, same live model:

step	standard PPL (mid-window)	honest PPL (last-position, leak-free)	paired diff	SE
8,000	16.06	15.51	−0.0346 nats	0.0252
16,000	13.44	14.00	+0.0408 nats	0.0210

Both differences are within ~1.4–2.0 standard errors of zero — not statistically distinguishable from no leak, the same qualitative result as the MLP variant (where the two checks landed at <1 SE). The architectural causal probe (event: causal_probe) also reports the deterministic, no-noise check eval_max_delta = 0.0 bit-exact at all four checkpoints (steps 4k/8k/12k/16k) — the passed: False flag in the log is a known overly-strict artifact of the noisy .train()-mode branch (Gumbel-softmax routing jitter, train_max_delta ~1e-5–4e-5), not a residual leak; see the MLP variant's verification note for the full explanation of why this flag can be ignored in favor of eval_max_delta.

A parameter-count correction found along the way

This model card previously reported 17,407,980 total parameters (copied from the MLP-V_theta sibling model's card). Reconstructing the exact SQ3 architecture and loading ckpt_best.pt with zero missing/unexpected keys gives the true count:

	Reported (was wrong)	Actual
Total parameters	17,407,980	18,194,420
$V_{\theta }V\_\backslash theta$ parameters	not previously broken out	4,207,625

The SQ3 mixture's per-position hypernetwork (mu_proj, a_proj, pi_proj, b_proj, each mapping the K_xi·d = 1024-dim xi-context to K_mix·d = 2048-dim outputs) is larger than the 3-layer MLP it replaces, despite producing a much flatter landscape (see below) — this model has more parameters devoted to $V_{\theta }V\_\backslash theta$ than the MLP baseline (4.2M vs 3.4M) even though the resulting potential is nearly flat.

V_theta landscape statistics, recomputed on the fixed checkpoint

Metric	leaky (published)	fixed (this checkpoint)
Mean	0.008	0.017
Std	0.26	0.42
Min	−5.49	−1.23
Max	13.61	14.78
Range	19.10	16.01

Both checkpoints show the same qualitative picture — a near-flat $V_{\theta }V\_\backslash theta$ landscape (mean close to 0, most of the force budget carried by $V_{\varphi }V\_\backslash phi$ and the Fock registers) — the fixed run is an independently-trained model with different weights, so exact values shift, but the "flattest landscape among the three architectures" conclusion is unaffected.

Data sources

• leaky rerun: semsimula_fock_multixi_structured_vtheta_buggy2/A2/seed0/
• fixed (this repo's checkpoint): semsimula_fock_multixi_structured_vtheta/A2/seed0/
• Landscape stats recomputed via a fresh forward pass of the reconstructed SQ3 architecture over TinyStories validation batches, loading ckpt_best.pt with strict=True semantics (0 missing / 0 unexpected keys).
• Root-cause audit: Fock-PARFLM_Causal_Leak_Audit_Results.md
• MLP-V_theta sibling verification: semsimula-fock-parflm/causal_leak_fix_verification.md