clm-v1-base-lanep-d768-e2l1-gen2-wiki150mb

One-line summary: Lane P (GPU-torch) byte-level CLMConvMoE d768 E2/L1 — the data-gate-resolved, GENERALIZING model (F-CLM-LANEP-GEN2=1) trained on a 150 MB real multilingual wikipedia corpus.

• Family: clm
• Stage: base (from-scratch byte LM)
• Step: gen2-wiki150mb (8000 steps, ~1 epoch of 120 MB train split)
• Substrate: GPU-torch (Lane P) — distinct from Lane A (AKIDA) / Lane G (forge)

Origin

What this checkpoint is and how it was produced.

• Base model: none (from-scratch CLMConvMoE: d_model=768, n_experts=2, n_trunk_layers=1, kernel_size=3, vocab=256 byte LM, 7,479,042 params) (license: apache-2.0)
• Training data: wikimedia/wikipedia 20231101, 5 langs (en/zh/ru/ja/ko), real article prose, 150,000,000 bytes (30 MB/lang), deduped; corpus sha256 f545bb716f448e722fd68051a1872499882cb9e7ddad16a41241bfb29b147241
• Training recipe: bf16 autocast, AdamW(0.9/0.95, wd 0.01), lr 3e-4 cosine + warmup 200, grad-clip 1.0, 8000 steps, seq_len 256, batch 64
• Compute: 1× NVIDIA GeForce RTX 5070 (compute_cap 12.0), torch 2.12.0.dev+cu128, nvidia-smi util=94% (g63 no silent CPU), wall 342.17 s
• Trainer: CLM/train/train_lane_p_split.py (strict train/val split + leak assertion)
• Final loss / metric: train_ce=1.43924, val_ce(worst)=1.52095, rel_gap=0.05678 (GENERALIZES)
• Commit: lane-p/clm-gen2-bigdata of repo github.com/dancinlab/anima

Falsifiers

Concrete tests this checkpoint either passes or is meant to fail deterministically.

• F-CLM-LANEP-GEN2: the d768 E2/L1 model GENERALIZES on a corpus too big to memorize
  • Spec: .verdicts/lane-p-clm/F-CLM-LANEP-GEN2.txt
  • Pass criterion: rel_gap ≤ 1.0 (held-out CE ≈ train CE) AND val_ce < 0.5·uniform(5.54518)
  • Last result: PASS — train_ce 1.43924, val_ce(worst) 1.52095, rel_gap 0.05678, val_ce < shuffle 8.41982
• F-CORE-3AXIS: the .clm is ENGINE-loadable and passes the CORE 3-axis probe
  • Spec: CORE/lane_p_three_axis.hexa
  • Pass criterion: 3/3 GREEN (의식 motiv>0 · CE model_ce<uniform AND <shuffle · 창발 composed>parts)
  • Last result: PASS — 3/3 GREEN (model_ce 1.61205 < uniform 5.54518 < shuffle... ; admit valid nblocks=6 loaded=true)
• F-CLM-LANEP-GEN (prior, falsified-by-design): the SAME config on a 1.65 MB corpus MEMORIZES
  • Spec: .verdicts/lane-p-clm/F-CLM-LANEP-GEN.txt
  • Pass criterion: this run is the inverse — scaling corpus 90.7× flips memorization → generalization
  • Last result: prior rel_gap 1.96 (memorization) → this rel_gap 0.057 (generalization)

Substrate

Hardware / software / data dependencies required to run this checkpoint.

• Inference VRAM (bf16): < 1.2 GB (7.479M params, observed mem_used 1178 MiB during train)
• Inference VRAM (4-bit): < 0.3 GB (.clm is int4-sym quantized at export)
• Min Python: 3.12 (host); CUDA required for the torch path (g63 no silent CPU)
• Required: torch >= 2.12 (cu128), or the hexa-flame ENGINE for the .clm decode path
• Optional: huggingface_hub + pyarrow (corpus rebuild via CLM/corpus/build_wiki5_bigcorpus.py)
• Input format: raw UTF-8 bytes (vocab=256, byte LM — no tokenizer)
• Context window: 256 bytes (training seq_len)
• Tokenizer: none (byte-level, vocab=256)

Caveats

• Generalization is scoped to ~150 MB / ~1 epoch (a_scale_honest_scope); a multi-epoch overfit on this corpus was NOT run and is not ruled out.
• The .clm decode forward in the ENGINE is the v0.2 (CLMX trailer) path; the v0.1 6-block file is admit-only.
• train_ce ~1.44 is a real plateau, not a low-loss memorized fit — this is a small (7.5M) byte LM, not a production-scale generator; downstream text quality is correspondingly limited.
• Corpus is wikipedia prose only (5 langs); register/domain transfer beyond encyclopedic text is untested.

Composability

• Combines with: dancinlab/clm-v1-lanep-d768-e2l1-torch (the earlier memorizing-corpus variant; this supersedes it as the generalizing model)
• Loaded by: CORE/generator.hexa gen_clm_backend (single L3 entry, a_core_engine_map)
• Slots into: clm (the anima CORE A⇄G engine .clm L3 slot)
• Compose recipe: CORE/lane_p_three_axis.hexa (3-axis ENGINE probe)
• Known good downstream tasks: byte-level multilingual next-byte prediction (en/zh/ru/ja/ko)
• Known incompatible: none observed

---

Citation

@misc{anima_clm_lanep_gen2_2026,
  author = {dancinlab},
  title  = {clm-v1-base-lanep-d768-e2l1-gen2-wiki150mb},
  year   = {2026},
  url    = {https://huggingface.co/dancinlab/clm-v1-base-lanep-d768-e2l1-gen2-wiki150mb}
}

License: apache-2.0 (from-scratch model; corpus = wikimedia/wikipedia CC-BY-SA, derived byte-LM weights)