Upload README.md with huggingface_hub

README.md

@@ -22,7 +22,7 @@ v20 design: the expert capacity is unchanged (~507M) but it is reorganised from
 **16 private pools into ONE shared pool read by 3 layers**, following
 *Memory Layers at Scale* (arXiv:2412.09764).
 
-## Headline result: 1.48x faster training at 16x the expert granularity
+## Headline result: 1.44x faster training at 16x the expert granularity
 
 Both runs used **the same GPU model (RTX 5060 Ti 16 GB)**, the same streamed
 FineWeb-Edu+Cosmopedia corpus, the same codon table and the same optimizer stack.
@@ -35,10 +35,18 @@ FineWeb-Edu+Cosmopedia corpus, the same codon table and the same optimizer stack
 | Expert params | 507.5M | 507.5M (identical) |
 | Controller params | 569.4M | 598.1M |
 | Gathers per token | 16 | **3** |
-| **Avg throughput** | **22,616 tok/s** | **33,451 tok/s** |
-| Median / max tok/s | 22,904 / 24,040 | 33,676 / 34,768 |
+| **Avg throughput** | **22,566 tok/s** | **32,488 tok/s** |
+| Median / max tok/s | 22,871 / 24,040 | 32,858 / 34,768 |
+| Tokens trained | 1.553B | 1.162B |
+| Final valid CE | 4.318 | **4.0317** |
+| **Mean valid CE, matched 572-1145M tokens** | 4.1665 | **4.0213** |
 | Dense DNA-2B reference | 45,700 tok/s | 45,700 tok/s |
 
+**v21 is 1.44x faster AND better per token.** Over the token window both runs
+covered (572-1145M), v21's mean valid CE is
+**0.145 lower** than v20's - so the
+throughput win did not cost sample efficiency, it improved it.
+
 ![throughput](throughput_v20_vs_v21.png)
 ![summary](summary_v20_vs_v21.png)
 ![loss](loss_v20_vs_v21.png)
@@ -62,7 +70,7 @@ FineWeb-Edu+Cosmopedia corpus, the same codon table and the same optimizer stack
   recurrence, the dense FFNs and the PEER *router*, and leaves only the gather eager.
 - **The projected 44k tok/s was not reached.** CPU-proxy benchmarking overestimated the
   win; on the real GPU the recurrence and the eager gather region dominate. The honest
-  measured number is **33,451 tok/s**.
+  measured number is **32,488 tok/s**.
 
 ## Architecture
 
@@ -77,8 +85,8 @@ optimizer           : Muon (recurrence) + fused AdamW (dense) + sparse SGD (pool
 
 - Total experts: **495,616** per PEER layer (shared)
 - Active params/token: **598M** controller total, ~70M active
-- Tokens seen (this checkpoint): **101M**
-- Final valid CE: **4.802**
+- Tokens seen: **1.162B** (113,738 steps, run completed)
+- Final valid CE: **3.7246** (last logged periodic checkpoint 4.0317)
 
 ## Files
 
@@ -100,10 +108,9 @@ python train_peer_v21.py --batch 20 --seq 512 --nk 704 --topk 8 --pheads 1 \
 ```
 
 ## Honest limitations
-- This is a **systems/throughput** artifact. It is an early-training checkpoint, not a
-  chat model, and has had no SFT.
-- v21 and v20 are at **different token counts**, so the CE-vs-tokens curve is the only
-  fair quality comparison; the throughput comparison is the load-bearing result.
+- This is a **systems/throughput** artifact, not a chat model: no SFT has been done.
+- The two runs stopped at different token counts (v20 1.553B, v21 1.162B), so the
+  quality claim is made only over the **matched token window** both runs covered.
 - The codon memory table is inherited unchanged from DNA-DiskChat-2B.
 
 ## References