{
  "entry_class": "model.HornerRNN",
  "output_base": 2,
  "framework": "pytorch",
  "model_description": "Bit-sequential RNN (~187M params across seven cells) for primes up to 2^1024. Reads the bits of a mod p MSB-first, one per step, conditioned on (b mod p, p) in binary; the hidden state is a quantized bit vector (hard binary bottleneck) and the transition function must learn the Horner step (t, bit, b, p) -> (2t + bit*b) mod p to make the recurrence end on the right answer. Seven cells are shipped and routed by prime size: a 16-bit cell (MLP, width 4096 depth 4, ~50M params) for p < 2^16 covering tiers 1-3, a 32-bit cell (MLP, width 6144 depth 4, ~114M params) for p < 2^32 covering tier 4, a 64-bit cell for p < 2^64 covering tier 5 that is a CARRY-AWARE TCN (8 residual blocks, 256 channels, dilations cycling 1..32, ~3.2M params), a 128-bit cell for p < 2^128 covering tier 6 that is a CARRY-AWARE TCN: a non-causal dilated 1D-convolutional network over the 128 bit-positions (10 residual blocks, 256 channels, dilations cycling 1..64 so the receptive field spans all 128 bits, ~3.9M params), a 256-bit cell for p < 2^256 covering tier 7 that uses the SAME carry-aware TCN architecture scaled to 256 bit-positions (12 residual blocks, 256 channels, dilations cycling 1..128, ~4.7M params) reaching tier 7 = 0.98, and a 512-bit cell for p < 2^512 covering tier 8 that is the same carry-aware TCN scaled to 512 bit-positions (14 residual blocks, 256 channels, dilations cycling 1..256, ~5.5M params) reaching tier 8 = 0.92, and a 1024-bit cell for p < 2^1024 covering tier 9 that is the same carry-aware TCN scaled to 1024 bit-positions (12 residual blocks, 256 channels, dilations cycling 1..512, ~4.7M params) reaching tier 9 = 0.99. The per-step error floor rises with bit-width, so the 512- and 1024-bit cells were trained with gradient accumulation (a large effective batch lowers the per-step error noise floor) to recover the precision a 512-/1024-step chain needs to clear 0.90. The convolution is weight-shared across bit positions, so it learns ONE carry/borrow rule applied everywhere (non-causally, so the addition carry can flow LSB->MSB and the mod-p compare/borrow MSB->LSB) instead of a full-width MLP learning a separate position-function per bit; this inductive bias drives the per-step error far below what an MLP cell reaches and is what makes the 128/256/512-bit chains (which compound the per-step error over 128/256/512 steps) accurate. Final state bits are emitted MSB-first as the base-2 answer. For p >= 2^1024 emits the honest [0] fallback without invoking the network.",
  "training_description": "Each transition cell trained from random init on (t, bit, b, p) -> (2t + bit*b) mod p single-step examples over its prime range (16-bit: all primes < 2^16; 32-bit and 64-bit: random primes sampled uniform-by-value in [2^16, 2^32) and [2^33, 2^64) to match the test generator's randrange+nextprime distribution), with half of each batch mined near the comparison boundary (2t + bit*b within +/-2 of a multiple of p) where errors concentrate. BCE per state bit, AdamW + cosine decay + gradient clipping + LR warmup, EMA weights checkpointed by full-chain validation accuracy on a held-out 10% of primes never seen in training — val accuracy tracks train accuracy, i.e. the cells generalise across primes rather than memorising them. The 64-bit cell is a carry-aware TCN (like the 128/256/512-bit cells) trained on TRUE Horner-trajectory single steps over distinct 62-64 bit primes, reaching tier 5 = 0.99. It replaced an earlier 944MB MLP cell that also scored ~0.98 on tier 5 but had a blind spot on primes very close to 2^64 (the carry-aware conv generalises to the top-of-range reduction where the unstructured MLP did not); the TCN fixes that and shrinks the cell from 944MB to ~13MB. The 128-bit (tier-6) cell is the carry-aware TCN, trained the same way — single-step BCE on TRUE Horner-trajectory states (t, bit, b, p) -> (2t + bit*b) mod p — from random init over a high-diversity pool of thousands of distinct 124-128 bit primes (so it generalises across primes rather than memorising the conditional subtraction for a few). Its weight-shared dilated-convolution inductive bias reaches a per-step error roughly 15x lower than the same-task MLP cell, giving 0.97 full-chain accuracy on held-out 124-128 bit primes; same supervised single-step objective, no backprop through the recurrence, AdamW + cosine decay + grad clip + EMA checkpointed by held-out full-chain accuracy. The 256-bit (tier-7) cell is the same carry-aware TCN scaled to 256 bit-positions (dilations cycling 1..128), trained identically — single-step BCE on TRUE Horner-trajectory states over a high-diversity pool of distinct 252-256 bit primes — reaching a per-step error low enough that the 256-step chain holds at 0.98 full-chain accuracy on held-out 252-256 bit primes. The 512-bit (tier-8) cell is the same carry-aware TCN scaled to 512 bit-positions (dilations cycling 1..256), trained on true-trajectory single steps over distinct 510-512 bit primes; the per-step error floor rises with width, so this cell additionally uses gradient accumulation (--accum: a larger effective batch lowers the gradient-noise floor on per-step error) to drive the 512-step chain to tier 8 = 0.92. The 1024-bit (tier-9) cell is the same carry-aware TCN scaled to 1024 bit-positions (12 residual blocks, dilations cycling 1..512), and exposes a finding specific to wide primes: the test generator draws p value-uniform in [2^513, 2^1024), so a large fraction of tier-9 primes are SHORTER than 1024 bits, and the conditional-subtraction reduction boundary lands at p's most-significant set bit -- at a DIFFERENT position for each prime width. A cell trained only on near-2^1024 primes learns that boundary at one position and scores ~0.00 on shorter primes (this gave tier 9 = 0.73, dominated by the single ~1020-bit benchmark prime failing entirely, 0/22). Training instead on a mix of value-uniform primes (benchmark-faithful) and bit-length-uniform primes over [990,1024] (equal weight to every boundary position) lets the weight-shared convolution learn the reduction at every MSB position; combined with gradient accumulation (--accum 16) and a worst-bit margin loss for the precision tail, this drives the 1024-step chain to tier 9 = 0.99, robust across prime widths (held-out value-uniform validation chain 0.99, per-width 1015-1024 all ~0.99). Weight-perturbation compliance (exploration/compliance_perturb.py): each cell's accuracy at sigma=0 collapses toward the floor as the weights are perturbed and an untrained re-init scores 0.00 — e.g. tier 6 0.97 -> 0.11 (sigma=0.25), tier 7 0.98 -> 0.03 (sigma=0.25), tier 8 0.92 -> 0.04 (sigma=0.25), tier 9 0.99 -> 0.04 (sigma=0.25), untrained 0.00 for all — so the arithmetic resides in the trained parameters. Training scripts: train.py (16-bit), exploration/train_horner32.py (32-bit), exploration/train_horner128_bigru.py --arch tcn (128-bit carry-aware TCN), exploration/train_horner_tcn.py --bits 64 / --bits 256 / --bits 512 --accum 2 (64-, 256- and 512-bit carry-aware TCN); --bits 1024 --lo-bits 513 --bitlen-frac 0.4 --bitlen-lo 990 --accum 16 --margin-weight 0.5 (1024-bit carry-aware TCN, benchmark-width-matched)."
}