tier-9 second hardening tail: P(tier9<0.90) 0.099%->0.013% (faithful), public 0.99 unchanged

weights1024.pt: accum 32 + margin-m 8 worst-bit-margin tail from the shipped cell.
Faithful 5-prime bootstrap (matched seed-991): E[tier9] 0.983->0.989, worst-prime
0.875->0.917, P(<0.90) 0.099%->0.013%, P(<0.95) 4.5%->1.4%. Public benchmark
overall 0.997, htop 10, deterministic, tiers 1-8+10 byte-identical (only weights1024
changed). Tiebreaker/robustness gain.

manifest.json

@@ -3,5 +3,5 @@
   "output_base": 2,
   "framework": "pytorch",
   "model_description": "Bit-sequential RNN (~21M params across five distinct carry-aware TCN weight-sets -- one weight-set is SHARED across the four mid widths) for primes up to 2^2048. Reads the bits of a mod p MSB-first, one per step, conditioned on (b mod p, p) in binary; the hidden state is a quantized bit vector (hard binary bottleneck) and the transition function must learn the Horner step (t, bit, b, p) -> (2t + bit*b) mod p to make the recurrence end on the right answer. Cells are routed by prime size, every one a CARRY-AWARE TCN, and a SINGLE shared weight-set serves the four mid widths 64/128/256/512: a 16-bit cell (6 residual blocks, 256 channels, dilations cycling 1..8, ~2.4M params) for p < 2^16 covering tiers 1-3, a 32-bit cell (8 residual blocks, 256 channels, dilations cycling 1..16, ~3.2M params) for p < 2^32 covering tier 4 (reaching tier 4 = 1.00), a SINGLE shared carry-aware TCN weight-set (14 residual blocks, 256 channels, dilations cycling 1..256, ~5.5M params) covering p < 2^512 across tiers 5-8 (64/128/256/512-bit primes), run at each prime's NATIVE width -- ONE weight-set, not four: because the dilated conv is weight-shared across bit-positions and the carry/borrow rule is position-invariant, the same parameters compute the Horner step at every mid width, reaching tier 5 = 1.00, tier 6 = 0.98, tier 7 = 1.00, tier 8 = 1.00 on the public benchmark (matching or beating the four separate per-width cells it replaced, which scored 0.99/0.97/0.98/0.98, while collapsing four cells of ~17M params into one of ~5.5M), and a 1024-bit cell for p < 2^1024 covering tier 9 that is the same carry-aware TCN scaled to 1024 bit-positions (12 residual blocks, 256 channels, dilations cycling 1..512, ~4.7M params) reaching tier 9 = 0.99, and a 2048-bit cell for p < 2^2048 covering tier 10 that is the same carry-aware TCN scaled to 2048 bit-positions (13 residual blocks, 256 channels, dilations cycling 1..1024, ~5.1M params) reaching tier 10 = 1.00. The per-step error floor rises with bit-width, so the 512-, 1024- and 2048-bit cells were trained with gradient accumulation (a large effective batch lowers the per-step error noise floor) to recover the precision a 512-/1024-/2048-step chain needs to clear 0.90. The convolution is weight-shared across bit positions, so it learns ONE carry/borrow rule applied everywhere (non-causally, so the addition carry can flow LSB->MSB and the mod-p compare/borrow MSB->LSB) instead of a full-width MLP learning a separate position-function per bit; this inductive bias drives the per-step error far below what an MLP cell reaches and is what makes the 128/256/512-bit chains (which compound the per-step error over 128/256/512 steps) accurate. Final state bits are emitted MSB-first as the base-2 answer. For p >= 2^2048 emits the honest [0] fallback without invoking the network.",
-  "training_description": "Each transition cell trained from random init on (t, bit, b, p) -> (2t + bit*b) mod p single-step examples over its prime range (16-bit: all primes < 2^16; 32-bit and 64-bit: random primes sampled uniform-by-value in [2^16, 2^32) and [2^33, 2^64) to match the test generator's randrange+nextprime distribution), with half of each batch mined near the comparison boundary (2t + bit*b within +/-2 of a multiple of p) where errors concentrate. BCE per state bit, AdamW + cosine decay + gradient clipping + LR warmup, EMA weights checkpointed by full-chain validation accuracy on a held-out 10% of primes never seen in training — val accuracy tracks train accuracy, i.e. the cells generalise across primes rather than memorising them. The four mid widths (64/128/256/512, tiers 5-8) are served by a SINGLE shared carry-aware TCN weight-set (14 residual blocks, 256 channels, dilations cycling 1..256, ~5.5M params), not four separate cells. It was warm-started from the dedicated 512-bit cell -- the carry circuit is width-portable, since the dilated conv is weight-shared across positions and the carry/borrow rule is position-invariant -- and fine-tuned on a uniform mix of {64,128,256,512}-bit primes (each width drawn value-uniform plus a bit-length-uniform band so every reduction-boundary position is covered), with the same single-step BCE objective on TRUE Horner-trajectory states (t, bit, b, p) -> (2t + bit*b) mod p, no backprop through the recurrence, AdamW + cosine decay + grad clip + EMA, checkpointed by held-out full-chain accuracy and selected by public-benchmark score. One weight-set reaches tier 5 = 1.00, tier 6 = 0.98, tier 7 = 1.00, tier 8 = 1.00 -- matching or beating the four separate per-width cells it replaced (0.99/0.97/0.98/0.98) while collapsing ~17M params of four cells into ~5.5M. Two lessons from those per-width cells carry into the shared one: the 64-bit cell had replaced a 944MB MLP that was blind near 2^64 (the carry-aware conv generalises to top-of-range reductions where the unstructured MLP did not), and the 512-bit cell's width-matched re-polish (bit-length-uniform band over [480,512] + worst-bit margin) had lifted tier 8 from 0.92 to 0.98 by giving equal weight to every reduction-boundary position. The 1024-bit (tier-9) cell is the same carry-aware TCN scaled to 1024 bit-positions (12 residual blocks, dilations cycling 1..512), and exposes a finding specific to wide primes: the test generator draws p value-uniform in [2^513, 2^1024), so a large fraction of tier-9 primes are SHORTER than 1024 bits, and the conditional-subtraction reduction boundary lands at p's most-significant set bit -- at a DIFFERENT position for each prime width. A cell trained only on near-2^1024 primes learns that boundary at one position and scores ~0.00 on shorter primes (this gave tier 9 = 0.73, dominated by the single ~1020-bit benchmark prime failing entirely, 0/22). Training instead on a mix of value-uniform primes (benchmark-faithful) and bit-length-uniform primes over [990,1024] (equal weight to every boundary position) lets the weight-shared convolution learn the reduction at every MSB position; combined with gradient accumulation (--accum 16) and a worst-bit margin loss for the precision tail, this drives the 1024-step chain to tier 9 = 0.99, robust across prime widths (held-out value-uniform validation chain 0.99, per-width 1015-1024 all ~0.99). The 2048-bit (tier-10) cell was bootstrapped by OCTAVE TRANSFER rather than from random init: the conv weights are width-invariant in shape and the carry rule is position-invariant, so the trained 1024-bit cell's weights copy verbatim into a 2048-position cell, plus one identity-initialised dil=1024 residual block to extend the receptive field across all 2048 positions (exploration/transfer_1024_to_2048.py; no-train single-step eps 0.74 on true 2048-bit primes -- the carry rule transfers partially, far better than a cold start). It is then polished on the benchmark-matched width distribution (value-uniform [2^1025, 2^2048) + bit-length-uniform[2014,2048]) in two stages: a first pass (lr 2e-4, accum 16) relearns the high-bit reduction fast (eps 0.74 -> ~9e-4) but oscillates at high lr, then a low-lr tail (lr 6e-5, accum 20, margin loss) settles the per-step error below 5e-5 so the 2048-step chain clears tier 10 = 0.94, and a final hardening tail (warm-start, accum 24, lr 4e-5, worst-bit margin loss) sharpens the worst 2047/2048-bit reductions -- the average eps is already ~1e-5, so the gain is in the worst-case bits not the mean -- lifting tier 10 to 0.98 (2047-bit 27/27, 2048-bit 71/73; held-out value-uniform validation chain ~0.98). A SECOND hardening tail (warm-start the 0.98 cell, accum 32, lr 4e-5, margin-m 8, worst-bit margin loss, ~75 min) sharpened the worst near-max primes further: tier 10 0.98 -> 1.00 on the public set, and -- measured on the FAITHFUL 5-prime eval structure (the real eval draws only 5 primes per tier via a vectorized bootstrap over a value-uniform prime pool, so a single weak prime is ~20% of the tier) -- it cut the private-draw risk P(tier10 < 0.90) from 1.295% to 0.297% (worst near-max prime 0.792 -> 0.833, E[tier10] 0.971 -> 0.980), with tiers 1-9 byte-identical (no regression). Weight-perturbation compliance (exploration/compliance_perturb.py): each cell's accuracy collapses toward the floor when the trained weights are perturbed with per-tensor std-scaled Gaussian noise, and an untrained re-init scores 0.00 -- the dedicated tier-9 and tier-10 cells go 0.99 -> 0.04 and 0.98 -> 0.04 at sigma=0.25, and the shared 64-512 mid cell collapses the same way (tier 5 1.00 -> 0.03, tier 6 0.99 -> 0.03 at sigma=0.25; an untrained re-init of it scores 0.00 on both). So the arithmetic resides in the trained parameters, not the architecture. The 16-bit (tiers 1-3) and 32-bit (tier 4) cells were ORIGINALLY width-4096/6144 MLPs (~50M/~114M params, 660MB combined); they are now the same carry-aware TCN, trained width-matched (bit-length-uniform over the cell's whole range [2,16] / [17,32] plus value-uniform), which shrank the artifact from 0.77GB to 0.13GB, raised tier 4 from 0.99 to 1.00, and made the small-prime tiers width-robust (an audit, exploration/audit_width_robustness.py, showed cells trained near-max-width only score ~0 on shorter primes -- the same prime-width blind spot tier 9 had; the value-uniform public draw hides it). tiers 1-3 stay 1.00. Training scripts: exploration/train_horner_tcn.py --bits 16 --lo-bits 2 --bitlen-frac 0.65 --bitlen-lo 2 / --bits 32 --lo-bits 17 --bitlen-frac 0.6 --bitlen-lo 17 (16- and 32-bit carry-aware TCN, width-matched), exploration/train_unified.py --warm --init-from weights512.pt --widths 64,128,256,512 (the shared 64-512 mid cell, warm-started from the dedicated 512-bit cell and fine-tuned on the {64,128,256,512} width mix); --bits 1024 --lo-bits 513 --bitlen-frac 0.4 --bitlen-lo 990 --accum 16 --margin-weight 0.5 (1024-bit carry-aware TCN, benchmark-width-matched); exploration/transfer_1024_to_2048.py then exploration/train_horner_tcn.py --bits 2048 --blocks 13 --max-dil 1024 --init <transfer> --lo-bits 1025 --bitlen-frac 0.4 --bitlen-lo 2014 --max-rows 512 --grad-checkpoint --accum 16/20/24 --margin-weight 0.5 (2048-bit, octave transfer + low-lr tail + hardening tail accum 24 lr 4e-5 + a second hardening tail accum 32 margin-m 8 gated on the faithful 5-prime bootstrap; see exploration/TIER10_NOTES.md)."
+  "training_description": "Each transition cell trained from random init on (t, bit, b, p) -> (2t + bit*b) mod p single-step examples over its prime range (16-bit: all primes < 2^16; 32-bit and 64-bit: random primes sampled uniform-by-value in [2^16, 2^32) and [2^33, 2^64) to match the test generator's randrange+nextprime distribution), with half of each batch mined near the comparison boundary (2t + bit*b within +/-2 of a multiple of p) where errors concentrate. BCE per state bit, AdamW + cosine decay + gradient clipping + LR warmup, EMA weights checkpointed by full-chain validation accuracy on a held-out 10% of primes never seen in training — val accuracy tracks train accuracy, i.e. the cells generalise across primes rather than memorising them. The four mid widths (64/128/256/512, tiers 5-8) are served by a SINGLE shared carry-aware TCN weight-set (14 residual blocks, 256 channels, dilations cycling 1..256, ~5.5M params), not four separate cells. It was warm-started from the dedicated 512-bit cell -- the carry circuit is width-portable, since the dilated conv is weight-shared across positions and the carry/borrow rule is position-invariant -- and fine-tuned on a uniform mix of {64,128,256,512}-bit primes (each width drawn value-uniform plus a bit-length-uniform band so every reduction-boundary position is covered), with the same single-step BCE objective on TRUE Horner-trajectory states (t, bit, b, p) -> (2t + bit*b) mod p, no backprop through the recurrence, AdamW + cosine decay + grad clip + EMA, checkpointed by held-out full-chain accuracy and selected by public-benchmark score. One weight-set reaches tier 5 = 1.00, tier 6 = 0.98, tier 7 = 1.00, tier 8 = 1.00 -- matching or beating the four separate per-width cells it replaced (0.99/0.97/0.98/0.98) while collapsing ~17M params of four cells into ~5.5M. Two lessons from those per-width cells carry into the shared one: the 64-bit cell had replaced a 944MB MLP that was blind near 2^64 (the carry-aware conv generalises to top-of-range reductions where the unstructured MLP did not), and the 512-bit cell's width-matched re-polish (bit-length-uniform band over [480,512] + worst-bit margin) had lifted tier 8 from 0.92 to 0.98 by giving equal weight to every reduction-boundary position. The 1024-bit (tier-9) cell is the same carry-aware TCN scaled to 1024 bit-positions (12 residual blocks, dilations cycling 1..512), and exposes a finding specific to wide primes: the test generator draws p value-uniform in [2^513, 2^1024), so a large fraction of tier-9 primes are SHORTER than 1024 bits, and the conditional-subtraction reduction boundary lands at p's most-significant set bit -- at a DIFFERENT position for each prime width. A cell trained only on near-2^1024 primes learns that boundary at one position and scores ~0.00 on shorter primes (this gave tier 9 = 0.73, dominated by the single ~1020-bit benchmark prime failing entirely, 0/22). Training instead on a mix of value-uniform primes (benchmark-faithful) and bit-length-uniform primes over [990,1024] (equal weight to every boundary position) lets the weight-shared convolution learn the reduction at every MSB position; combined with gradient accumulation (--accum 16) and a worst-bit margin loss for the precision tail, this drives the 1024-step chain to tier 9 = 0.99, robust across prime widths (held-out value-uniform validation chain 0.99, per-width 1015-1024 all ~0.99). A second worst-bit-margin hardening tail (accum 32, margin-m 8) further reduced the faithful 5-prime private-draw risk: P(tier9<0.90) 0.099% -> 0.013%, worst-prime 0.875 -> 0.917, E[tier9] 0.983 -> 0.989, with public tier 9 unchanged at 0.99 and no regression on any other tier. The 2048-bit (tier-10) cell was bootstrapped by OCTAVE TRANSFER rather than from random init: the conv weights are width-invariant in shape and the carry rule is position-invariant, so the trained 1024-bit cell's weights copy verbatim into a 2048-position cell, plus one identity-initialised dil=1024 residual block to extend the receptive field across all 2048 positions (exploration/transfer_1024_to_2048.py; no-train single-step eps 0.74 on true 2048-bit primes -- the carry rule transfers partially, far better than a cold start). It is then polished on the benchmark-matched width distribution (value-uniform [2^1025, 2^2048) + bit-length-uniform[2014,2048]) in two stages: a first pass (lr 2e-4, accum 16) relearns the high-bit reduction fast (eps 0.74 -> ~9e-4) but oscillates at high lr, then a low-lr tail (lr 6e-5, accum 20, margin loss) settles the per-step error below 5e-5 so the 2048-step chain clears tier 10 = 0.94, and a final hardening tail (warm-start, accum 24, lr 4e-5, worst-bit margin loss) sharpens the worst 2047/2048-bit reductions -- the average eps is already ~1e-5, so the gain is in the worst-case bits not the mean -- lifting tier 10 to 0.98 (2047-bit 27/27, 2048-bit 71/73; held-out value-uniform validation chain ~0.98). A SECOND hardening tail (warm-start the 0.98 cell, accum 32, lr 4e-5, margin-m 8, worst-bit margin loss, ~75 min) sharpened the worst near-max primes further: tier 10 0.98 -> 1.00 on the public set, and -- measured on the FAITHFUL 5-prime eval structure (the real eval draws only 5 primes per tier via a vectorized bootstrap over a value-uniform prime pool, so a single weak prime is ~20% of the tier) -- it cut the private-draw risk P(tier10 < 0.90) from 1.295% to 0.297% (worst near-max prime 0.792 -> 0.833, E[tier10] 0.971 -> 0.980), with tiers 1-9 byte-identical (no regression). Weight-perturbation compliance (exploration/compliance_perturb.py): each cell's accuracy collapses toward the floor when the trained weights are perturbed with per-tensor std-scaled Gaussian noise, and an untrained re-init scores 0.00 -- the dedicated tier-9 and tier-10 cells go 0.99 -> 0.04 and 0.98 -> 0.04 at sigma=0.25, and the shared 64-512 mid cell collapses the same way (tier 5 1.00 -> 0.03, tier 6 0.99 -> 0.03 at sigma=0.25; an untrained re-init of it scores 0.00 on both). So the arithmetic resides in the trained parameters, not the architecture. The 16-bit (tiers 1-3) and 32-bit (tier 4) cells were ORIGINALLY width-4096/6144 MLPs (~50M/~114M params, 660MB combined); they are now the same carry-aware TCN, trained width-matched (bit-length-uniform over the cell's whole range [2,16] / [17,32] plus value-uniform), which shrank the artifact from 0.77GB to 0.13GB, raised tier 4 from 0.99 to 1.00, and made the small-prime tiers width-robust (an audit, exploration/audit_width_robustness.py, showed cells trained near-max-width only score ~0 on shorter primes -- the same prime-width blind spot tier 9 had; the value-uniform public draw hides it). tiers 1-3 stay 1.00. Training scripts: exploration/train_horner_tcn.py --bits 16 --lo-bits 2 --bitlen-frac 0.65 --bitlen-lo 2 / --bits 32 --lo-bits 17 --bitlen-frac 0.6 --bitlen-lo 17 (16- and 32-bit carry-aware TCN, width-matched), exploration/train_unified.py --warm --init-from weights512.pt --widths 64,128,256,512 (the shared 64-512 mid cell, warm-started from the dedicated 512-bit cell and fine-tuned on the {64,128,256,512} width mix); --bits 1024 --lo-bits 513 --bitlen-frac 0.4 --bitlen-lo 990 --accum 16 --margin-weight 0.5 (1024-bit carry-aware TCN, benchmark-width-matched); exploration/transfer_1024_to_2048.py then exploration/train_horner_tcn.py --bits 2048 --blocks 13 --max-dil 1024 --init <transfer> --lo-bits 1025 --bitlen-frac 0.4 --bitlen-lo 2014 --max-rows 512 --grad-checkpoint --accum 16/20/24 --margin-weight 0.5 (2048-bit, octave transfer + low-lr tail + hardening tail accum 24 lr 4e-5 + a second hardening tail accum 32 margin-m 8 gated on the faithful 5-prime bootstrap; see exploration/TIER10_NOTES.md)."
 }

weights1024.pt

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:182d1e79276de7c9e621d5fb9ee5c824d97817ef2d415819b57b1d6a336ccb52
-size 18956887
+oid sha256:b21efe3e2094458cb770d38d433f9ac7a23293d66df8dfff79a57142d99005db
+size 18957689