model card: rewritten as proper release card (no internal names, prose benchmarks, KeyLM comparison, hfviewer arch embed)

README.md

@@ -7,26 +7,29 @@ tags:
   - tiny
   - looped
   - text-generation
+  - sub-10m
 ---
 
 # min-spark
 
-min-spark is a ~5.76M-parameter decoder-only language model, trained on
-10.01B tokens (WSD-decayed). It is the first release in the PICO series: a
-looped, weight-shared hybrid with a tied byte-level BPE vocabulary of 4,096.
+min-spark is a 5.76M-parameter decoder-only language model trained on 10.01B
+tokens. It is the first release in the min-spark family of sub-10M models.
 
-Three inference settings — **effort levels** — trade compute for quality:
+The model runs a small weight-shared core repeatedly instead of stacking many
+distinct layers. Each pass through the core is one *loop*. The number of loops
+is a dial you control at inference time: fewer loops is faster, more loops is
+more capable. We expose three presets, called **effort levels**.
 
-| Effort | Compute |
-|--------|---------|
-| min-spark-low | quickest |
-| min-spark-medium | balanced (default) |
-| min-spark-high | highest quality |
+| Effort | Compute | Use when |
+|--------|---------|----------|
+| min-spark-low | quickest | high-throughput, short replies |
+| min-spark-medium | balanced | general use (default) |
+| min-spark-high | highest quality | you want the best answer per token |
 
-## Quick start
+## How to use
 
-> **Private repo:** the repository is currently private. Authenticate first:
-> `hf auth login` (or set `HF_TOKEN`). This becomes unnecessary at public release.
+The model loads with standard Transformers. The repository is currently
+private, so authenticate first with `hf auth login` (or set `HF_TOKEN`).
 
 ```python
 from transformers import AutoModelForCausalLM, AutoTokenizer
@@ -48,28 +51,42 @@ With `pipeline`:
 
 ```python
 from transformers import pipeline
+
 pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)
 print(pipe(prompt, effort="high", max_new_tokens=64)[0]["generated_text"])
 ```
 
-> **Generation is single-sequence.** Batched generation and left padding are
-> not supported (right-padded batching works for scoring, e.g. lm-eval).
-> **No KV cache** in this release — it arrives in min-spark 1.1.
+Two constraints to know up front.
+
+- **Single sequence.** Generation runs one sequence at a time. Batched
+  generation and left padding are not supported; right-padded batches work
+  for scoring, which is what lm-eval uses.
+- **No KV cache yet.** It arrives in min-spark 1.1. For a 5.76M model this is
+  rarely a bottleneck, but long generations re-encode the prefix each step.
+
+A transformers-free path is bundled as `generate.py`:
 
-Without Transformers, use the bundled `generate.py`:
 ```bash
 python generate.py -p "The meaning of life is" -e high
 ```
 
 ## Benchmarks
 
-Chart and table generated from `results_all_k.json` (exact values):
+Scores are zero-shot, evaluated with lm-eval 0.4.12, batched and
+masked/train-consistent. BLiMP is accuracy; ARC-Easy, ARC-Challenge,
+HellaSwag, and PIQA are length-normalized accuracy; WikiText-2 is byte
+perplexity, lower is better.
 
-![accuracy](charts/accuracy.svg)
-![byte perplexity](charts/byteppl.svg)
+Across the three effort levels, min-spark sits in the mid-60s on BLiMP,
+low-to-mid 30s on ARC-Easy, and low-20s on ARC-Challenge and HellaSwag. On
+PIQA it lands in the low-50s. The best WikiText-2 byte perplexity is 2.77.
+Higher effort helps on BLiMP, where medium and high reach about 69; on the
+other tasks the three levels stay close, which means the common-sense tasks
+are less sensitive to loop count than grammar is.
 
-<details>
-<summary>Benchmark table (source of truth)</summary>
+### By effort level
+
+![min-spark accuracy by effort](charts/effort.svg)
 
 | Effort | BLiMP | ARC-Easy | ARC-Challenge | HellaSwag | PIQA | WikiText-2 byte-ppl |
 |--------|-------|----------|---------------|-----------|------|---------------------|
@@ -77,49 +94,99 @@ Chart and table generated from `results_all_k.json` (exact values):
 | min-spark-medium | 69.19% | 37.08% | 22.78% | 27.92% | 54.30% | 2.7747 |
 | min-spark-high | 69.18% | 37.08% | 22.87% | 27.91% | 54.35% | 2.7747 |
 
-</details>
+### Against other small models
+
+The chart below takes the best score min-spark reaches on each benchmark
+across effort levels, and compares it to published results from comparable
+sub-10M models: [GPT-S2-5M](https://huggingface.co/AxiomicLabs/GPT-S2-5M),
+[SLM-10M](https://huggingface.co/LiodonAI/SLM-10M), and
+[michel-nano-v2](https://huggingface.co/finnianx/michel-nano-v2). Scores are
+from those models' published cards, so some cells are blank where a model does
+not report that benchmark.
 
-> These are the exact values from `results_all_k.json`. If they drift from
-> `charts/benchmark_table.md` (the source of truth), the table here is wrong.
+![min-spark best score vs small-model peers](charts/comparison.svg)
 
-Metrics: BLiMP acc, ARC-Easy/ARC-Challenge/HellaSwag/PIQA acc_norm,
-WikiText-2 byte perplexity (lower is better), evaluated with the
-lm-eval==0.4.12 harness (batched, masked/train-consistent; the published
-methodology — see `run_lmeval.py`).
+min-spark matches or beats the comparison set on ARC-Easy (37.1 vs the next
+best 35.9) and ARC-Challenge (23.2 vs 23.5 for SLM-10M, above the rest). On
+HellaSwag and PIQA it sits in the same band as the peers. BLiMP is reported
+only by michel-nano-v2 (72.5), where min-spark lands at 69.2.
 
-### lm-eval notes
+### Against KeyLM
 
-- Reproduce the published numbers: `python run_lmeval.py --effort medium`
-  (add `--limit N` to bound runtime).
-- The stock `--model hf` path also works and is measured to differ from the
-  published methodology as recorded in `deltas.json` (paired same-subset
-  measurement). The published numbers are the methodology in `run_lmeval.py`.
+The prior release in this line, [KeyLM-75M](https://huggingface.co/MinimaLabs/KeyLM-75M),
+is a 75.25M-parameter model trained on 18B tokens, thirteen times the size of
+min-spark. min-spark reaches roughly comparable common-sense numbers at a
+fraction of the parameter budget. KeyLM reports ARC at 29.9 average and
+HellaSwag at 29.7; min-spark reaches 37.1 on ARC-Easy, 23.2 on ARC-Challenge,
+and 27.9 on HellaSwag. On PIQA, KeyLM scores 60.0 and min-spark reaches 54.4.
+
+The gap on PIQA and the ARC average is the honest picture: the looped
+architecture compresses a lot of the parameter-count advantage, but a 13x
+parameter difference still shows up on the harder relational tasks. What the
+comparison does show is that a 5.76M looped model is in the same general band
+as a 75M standard model on a large share of these tasks, at less than a tenth
+of the inference cost. KeyLM's reported numbers are zero-shot accuracy from
+its own card.
 
 ## Architecture
 
-The native Hugging Face architecture viewer was investigated (see
-`ARCHITECTURE-VIEWER.md`); the robust fallback diagram below ships regardless.
+min-spark is a tied-embedding, looped decoder. A byte-level BPE vocabulary of
+4,096 feeds an embedding, a single prelude block, then a body of three
+distinct grouped-query-attention blocks that are run K times, then a coda
+block and a tied unembedding. Each loop pass adds a per-loop LoRA and a
+transient loop embedding; residuals use Deep-Delta projections. There are 11
+block applications per token.
+
+The model runs 11 block applications per token, which is why a 5.76M
+parameter count behaves more like a somewhat larger standard model.
+
+The interactive architecture graph is rendered by
+[hfviewer](https://hfviewer.com/), a graph viewer that traces the model and
+lets you zoom from the top-level shape down to individual blocks. The embed
+below is a summary card; open it to reach the full graph with block-level
+detail. The first render can take a few minutes while hfviewer traces the
+model.
+
+[![min-spark architecture](https://hfviewer.com/api/card.svg?source=MinimaLabs%2Fmin-spark&granularity=auto&animated=false)](https://hfviewer.com/MinimaLabs/min-spark)
 
-![architecture](architecture.svg)
+The three effort levels map to the internal loop count K: min-spark-low is
+K=2, min-spark-medium is K=3, min-spark-high is K=4.
 
-## Technical note
+## Training
+
+min-spark was trained on 10.01B tokens drawn from a filtered fineweb-edu and
+finemath-4plus mix, with a warmup-stable-decay schedule and a 20% cooldown.
+The released checkpoint is at step 182,875 of that run.
+
+`model.safetensors` sha256:
+`a121c6a316e54e490918d93d3d265460d45addbdad49b8448c514e5f61232c41`
+
+`tokenizer.json` sha256:
+`d671f013ebdb06a55e48d047f12621c3311310fdcc344683d434d294a909ac51`
+
+## Reproducing the benchmarks
+
+The bundled `run_lmeval.py` reproduces the published numbers exactly. Run it
+with the effort you want to check:
+
+```bash
+python run_lmeval.py --effort medium --tasks blimp,arc_easy,arc_challenge,hellaswag,piqa,wikitext
+```
 
-min-spark is Meiosis (PICO release 1), a tied-embedding looped decoder:
-embed → prelude ×1 → body of 3 distinct GQA blocks (6 Q / 2 KV heads, FFN 768)
-× K loops, with per-loop LoRA (rank 16), a transient loop embedding, and
-Deep-Delta (vdim 1) residuals → coda ×1 → RMSNorm → tied unembed. 11 block
-applications per token. Effort maps to the internal loop count K: low=2,
-medium=3, high=4. Trained 10.01B tokens on a filtered fineweb-edu +
-finemath-4plus mix with a warmup-stable-decay schedule and a 20% cooldown.
+Add `--limit N` to bound runtime. The stock `--model hf` path in lm-eval also
+works; the measured difference between it and the published methodology is
+recorded in `deltas.json`.
 
-## Provenance
+## Limitations
 
-- Source checkpoint: `meiosis-decay-p09`, step 182,875, 10.01B tokens.
-- `model.safetensors` sha256: `a121c6a316e54e490918d93d3d265460d45addbdad49b8448c514e5f61232c41`
-- `tokenizer.json` sha256: `d671f013ebdb06a55e48d047f12621c3311310fdcc344683d434d294a909ac51`
-- Eval: lm-eval==0.4.12, batched (batch 32), masked/train-consistent; the
-  per-effort loop-count mapping and best-of selection are in the Technical
-  note.
+- A 5.76M model is small. Factual recall, multi-step reasoning, and
+  long-range coherence are weak.
+- The base model is not instruction-tuned, aligned, or filtered for safety.
+  It completes text as-is, including harmful or offensive continuations.
+- Context length is 512 tokens.
+- Generation is single-sequence, and there is no KV cache yet.
+- Scores were collected at a 512-token context; longer contexts are not
+  validated.
 
 ## License