Update README.md

README.md

@@ -105,6 +105,16 @@ The softmax bottleneck is caused by projecting a narrow hidden vector (d = 176)
 Our attention architecture uses **Grouped-Query Attention (GQA)** with 4 query heads and 1 key-value head (a 4:1 head ratio). 
 In a 1.5M parameter model, memory bandwidth is already very fast, but the GQA layout forces the model to compress its keys and values into a shared latent subspace. This acts as a regularizer, preventing individual attention heads from developing isolated representations that lead to early overfitting on the pre-training corpus.
 
+## 📚 The Data & Training Volume: High-Density Saturation
+
+To get this level of coherence out of 1.49M parameters, I had to ensure the pre-training data was highly dense. I streamed and packed roughly **300,000 unique stories** from the TinyStoriesV2 dataset, creating a clean base corpus of about **54 million unique tokens**. 
+
+While the full training run was scheduled for 113,400 steps, I found the optimal weights at **Step 101,000**. 
+
+At this release point, with an effective batch size of **64** and a context window of **320 tokens**, the model had processed exactly **2,068,480,000 tokens** (~2.07 Billion tokens). 
+
+This means the model cycled through the 300,000-story corpus for approximately **38 full epochs**. This massive overtraining density—averaging over 1,300 tokens processed for every single parameter in the model—is what allowed the Nano version to achieve its final **1.5177 validation loss** (4.56 Perplexity) and lock in its stable grammatical footing.
+
 ---
 
 ## ⚓ The Attention Sink Anchor