release: pona-nanogpt-1 (checkpoint + ONNX + tokenizer + model card)

README.md

@@ -0,0 +1,105 @@
+---
+license: mit
+language:
+  - tok
+library_name: nanogpt
+pipeline_tag: text-generation
+tags:
+  - pona
+  - nanogpt
+  - word-level
+  - gpt
+  - toki-pona
+  - chat
+---
+
+# Model Card — `pona-nanogpt-1` (v1, the conversational word arm)
+
+> A [sup computer](https://www.supcpu.com) release — a small language model studio. [Model page](https://www.supcpu.com/models/pona-nanogpt-1/) · [monorepo](https://github.com/romellogoodman/sup-computer) (frozen code: [`projects/pona/models/pona-nanogpt-1/`](https://github.com/romellogoodman/sup-computer/tree/main/projects/pona/models/pona-nanogpt-1), tag `pona-nanogpt-1`) · runs in your browser at [www.supcpu.com/interfaces](https://www.supcpu.com/interfaces/).
+
+The full experiment — whether span, not grammar class, predicts what a small
+model finds hard — is
+[experiment 11](https://www.supcpu.com/research/a-language-small-enough-to-get-right/).
+
+## What it is
+
+A word-level Toki Pona GPT you can hold a conversation with. 2.73M params
+(6L/6H/192E, block 128) over a 370-token vocabulary — the ~130-word lexicon
+plus names, punctuation, and specials — small enough that the website's
+`/pona` chat renders the entire vocabulary as its keyboard. Trained from
+scratch on 6.93M characters of filtered Toki Pona (Toki Pona Wikipedia, the
+permissive subset of poki Lapo, Tatoeba) interleaved ×12 with 122
+LLM-generated dialogues that survived an oracle filter: a dialogue entered
+the corpus only if every sentence passed the same grammar checker the model
+is scored by. Dialogue turns are `- `-prefixed lines. The model learns that
+a dash line answers the dash line before it — that convention is the whole
+chat contract the keyboard UI relies on.
+
+## Numbers that matter
+
+| Metric | pona-nanogpt-1 | context |
+|---|---|---|
+| first-try grammaticality (error-only) | **96.9%** [95.6, 97.8] | its own corpus scores 96.0% → 101.0% corpus-relative |
+| strict grammaticality (all issue classes) | 90.4% | corpus strict: 90.9% |
+| error hazard per character | 0.078% | char arm 0.194%, glyph omni-xl 0.171% |
+| replies grammatical, t = 0.8 | 160/160 (100.0%) | word arm without dialogue data: 96.2%, 1.2% empty |
+| mean reply length | 6.35 words | the no-dialogue ablation drifts to 11.6-word non-sequiturs |
+| unique replies / echoes | 91.9% / 0% | 20 oracle-verified prompts × 8 replies |
+| memorization | 5.6% exact sentences, 1.45% 8-gram overlap | exact matches concentrate in stock lines ("mi pona.") |
+| val loss | 2.457 per word token | word arm without dialogue data: 2.512 |
+
+Free-prose protocol, pinned before any model was scored: 1,000 raw
+unconditional sentences at t = 1.0, no top-k, the model's own punctuation as
+segment boundaries. The headline number is above the corpus's own pass rate —
+training denoised the data. The denominator exists because register drift is
+real: Wikipedia passes the oracle at only 86.9%.
+
+## Scoring
+
+The oracle is [telo misikeke](https://telo-misikeke.gitlab.io/) (MIT),
+vendored at pinned commit `0a1852d`, driven via node with the Linku word
+list. It passed a trust gate — 16 known-good pu sentences accepted, 7
+known-bad flagged — before any number was reported. The headline metric
+fails a sentence on `error`-category issues only; the strict rate rides
+alongside. The oracle judges grammar, not meaning: an on-topic reply and a
+fluent non-sequitur can score the same.
+
+## Sampling: use temperature 0.8
+
+Replies are 100.0% grammatical at t = 0.8 and 98.8% at 1.0, so the `/pona`
+interface and the reply numbers above both use 0.8. The trade is repetition:
+at low temperature the model reuses phrases within a reply. Free-prose
+benchmark numbers stay measured at t = 1.0.
+
+## Training
+
+6 layers, 6 heads, 192 embed, block 128, dropout 0.1, batch 64; 1,500 steps
+at lr 3e-4 (beta2 0.99, warmup 50) on an M4 Mac (MPS), ~86ms/step — about
+three minutes. Best-val checkpointing. The dialogue mix cost nothing on the
+prose objective: val loss improved over the identical-recipe word arm
+(2.457 vs 2.512) while adding the reply behavior.
+
+## Limitations
+
+- **Pronoun deixis slips.** `mi`/`sina` swaps — grammatical, wrong person.
+  The oracle cannot see person errors, so the 100% reply number does not
+  certify deixis; this is v1's known conversational tic.
+- **Question machinery is the weakest grammar.** `illFormedQuestion` is the
+  top reply nitpick (11 of 160 at t = 0.8); it is nitpick-class, so it rides
+  outside the headline metric.
+- **Grammatical ≠ sensible.** Every score is a grammar checker's; no eval
+  here measures whether a reply is true, kind, or coherent beyond topic.
+- One seed, one run; the char/word hazard comparison rests on single runs
+  per arm.
+- The corpus ceiling is 96.0% — scores approaching it say as much about the
+  register mix as about the model.
+
+## Reproduce
+
+The frozen folder (`projects/pona/models/pona-nanogpt-1/`) rebuilds
+everything in place: `fetch_*.py → build_corpus.py → build_chat_corpus.py
+--reps 12 → prepare.py → train.py config.py`, then `harness.py` for the
+oracle eval and `chat_eval.py` for replies. The 122-dialogue set ships
+pinned in the folder (`dialogue.txt`, sha1-verified against its committed
+manifest) because LLM sampling cannot regenerate it. Weights ship via the
+artifact URLs in `registry.json`, never in the tree.

ckpt.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:46eab96cc34a4fd910258168b4ecc20dfc5a0c51b02b2eef0d43e911ef381466
+size 32779750

pona-nanogpt-1.int8.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b53ff458a3aecc1ce83b11b89faa990d113890622f69174a6a0397abb359ac31
+size 3133152

pona-nanogpt-1.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f1dbf5cc545de3078c5200a9ef6357c9fff820112f7230dac3908203bbea04ef
+size 11406320

pona-nanogpt-1.vocab.json

@@ -0,0 +1 @@
+{"stoi": {"<unk>": 0, "<name>": 1, "\n": 2, "!": 3, "\"": 4, "'": 5, "(": 6, ")": 7, ",": 8, "-": 9, ".": 10, ":": 11, ";": 12, "?": 13, "a": 14, "aa": 15, "aaa": 16, "aaaaa": 17, "akesi": 18, "aku": 19, "al": 20, "ala": 21, "alasa": 22, "ale": 23, "ali": 24, "alo": 25, "amiko": 26, "an": 27, "and": 28, "anpa": 29, "ante": 30, "anu": 31, "apeja": 32, "asa": 33, "awen": 34, "b": 35, "brillig": 36, "c": 37, "co": 38, "com": 39, "counties": 40, "de": 41, "di": 42, "e": 43, "en": 44, "enepi": 45, "enko": 46, "epansa": 47, "epiku": 48, "er": 49, "es": 50, "esun": 51, "ete": 52, "h": 53, "had": 54, "hij": 55, "i": 56, "ijo": 57, "ike": 58, "iki": 59, "il": 60, "ilo": 61, "in": 62, "info": 63, "inli": 64, "insa": 65, "ipi": 66, "iseki": 67, "isipin": 68, "j": 69, "ja": 70, "jaki": 71, "jalan": 72, "jami": 73, "jan": 74, "jasima": 75, "jelo": 76, "jetesantakalu": 77, "jo": 78, "josuta": 79, "jule": 80, "k": 81, "ka": 82, "kala": 83, "kalama": 84, "kalu": 85, "kama": 86, "kamala": 87, "kamalawala": 88, "kan": 89, "kapesi": 90, "kasi": 91, "ke": 92, "ken": 93, "kepeken": 94, "kese": 95, "ki": 96, "kijetenitakalu": 97, "kijetesanataklu": 98, "kijetesantakalu": 99, "kiki": 100, "kili": 101, "kin": 102, "kipisi": 103, "kisa": 104, "kiwen": 105, "ko": 106, "kokosila": 107, "kon": 108, "konwe": 109, "ku": 110, "kule": 111, "kulu": 112, "kulupu": 113, "kuntu": 114, "kute": 115, "l": 116, "la": 117, "lanpan": 118, "lape": 119, "laso": 120, "lawa": 121, "leko": 122, "len": 123, "lete": 124, "li": 125, "lili": 126, "lin": 127, "linja": 128, "linluwi": 129, "lipamanka": 130, "lipu": 131, "lo": 132, "loje": 133, "lon": 134, "lu": 135, "luka": 136, "lukin": 137, "luna": 138, "lupa": 139, "m": 140, "ma": 141, "majuna": 142, "makan": 143, "mama": 144, "man": 145, "mani": 146, "meli": 147, "melome": 148, "meso": 149, "mi": 150, "mije": 151, "mijomi": 152, "misa": 153, "misikeke": 154, "misu": 155, "moku": 156, "mola": 157, "moli": 158, "monsi": 159, "monsuta": 160, "monsutatesu": 161, "msa": 162, "mu": 163, "mulapisu": 164, "mun": 165, "musi": 166, "musu": 167, "mute": 168, "muu": 169, "n": 170, "na": 171, "nala": 172, "nalanja": 173, "namako": 174, "nanpa": 175, "nasa": 176, "nasin": 177, "nata": 178, "nena": 179, "ni": 180, "nimi": 181, "nn": 182, "nnn": 183, "noka": 184, "nu": 185, "o": 186, "of": 187, "oji": 188, "oke": 189, "oko": 190, "olin": 191, "omekapo": 192, "on": 193, "ona": 194, "open": 195, "org": 196, "owe": 197, "p": 198, "paka": 199, "pakala": 200, "pake": 201, "pali": 202, "palisa": 203, "pan": 204, "pana": 205, "pasila": 206, "pata": 207, "peta": 208, "pi": 209, "pilin": 210, "pimeja": 211, "pimjea": 212, "pini": 213, "pipi": 214, "po": 215, "poka": 216, "poki": 217, "polinpin": 218, "pona": 219, "powe": 220, "print": 221, "pu": 222, "puwa": 223, "r": 224, "remove": 225, "s": 226, "salam": 227, "sama": 228, "san": 229, "sapiens": 230, "seki": 231, "seli": 232, "selo": 233, "seme": 234, "sen": 235, "sewi": 236, "si": 237, "sijelo": 238, "sike": 239, "sin": 240, "sina": 241, "sinpin": 242, "sipi": 243, "sitelen": 244, "soko": 245, "sona": 246, "soto": 247, "soweli": 248, "sowo": 249, "su": 250, "suli": 251, "sulo": 252, "sun": 253, "suno": 254, "supa": 255, "super": 256, "sutopatikuna": 257, "suwi": 258, "t": 259, "ta": 260, "taki": 261, "tan": 262, "taso": 263, "tawa": 264, "te": 265, "teje": 266, "telo": 267, "tempo": 268, "ten": 269, "tenpo": 270, "test": 271, "the": 272, "tiki": 273, "to": 274, "tok": 275, "toki": 276, "tokipona": 277, "tokiponido": 278, "tomo": 279, "tonsi": 280, "tu": 281, "tuki": 282, "tuli": 283, "tuta": 284, "u": 285, "unpa": 286, "unu": 287, "usawi": 288, "uta": 289, "utala": 290, "utula": 291, "w": 292, "wa": 293, "waleja": 294, "walo": 295, "wan": 296, "waso": 297, "wawa": 298, "we": 299, "weka": 300, "whiffling": 301, "wikipedia": 302, "wile": 303, "y": 304, "yupekosi": 305, "Alapi": 306, "Amelika": 307, "Anku": 308, "Apeja": 309, "Apika": 310, "Asija": 311, "Elena": 312, "Elopa": 313, "Ensi": 314, "Epanja": 315, "Epelanto": 316, "Inli": 317, "Intonesija": 318, "Isale": 319, "Isilami": 320, "Italija": 321, "Itan": 322, "JAN": 323, "Jesu": 324, "Juke": 325, "KUNTULI": 326, "Kanata": 327, "Kanse": 328, "Ke": 329, "Lakuse": 330, "Lasina": 331, "Loma": 332, "Losi": 333, "Luke": 334, "Lusi": 335, "Mali": 336, "Masu": 337, "Mawi": 338, "Mesiko": 339, "Mewi": 340, "Mewika": 341, "Mupa": 342, "Nasi": 343, "Netelan": 344, "Nijon": 345, "Oselija": 346, "Paki": 347, "Palata": 348, "Pali": 349, "Pasen": 350, "Pasiju": 351, "Posuka": 352, "Potuke": 353, "Puta": 354, "Sami": 355, "San": 356, "Sesesele": 357, "Sili": 358, "Sonja": 359, "Sonko": 360, "TU": 361, "Tami": 362, "Tan": 363, "Tatowepa": 364, "Tepo": 365, "Ton": 366, "Tosi": 367, "Tuki": 368, "Wikipesija": 369}, "itos": {"0": "<unk>", "1": "<name>", "2": "\n", "3": "!", "4": "\"", "5": "'", "6": "(", "7": ")", "8": ",", "9": "-", "10": ".", "11": ":", "12": ";", "13": "?", "14": "a", "15": "aa", "16": "aaa", "17": "aaaaa", "18": "akesi", "19": "aku", "20": "al", "21": "ala", "22": "alasa", "23": "ale", "24": "ali", "25": "alo", "26": "amiko", "27": "an", "28": "and", "29": "anpa", "30": "ante", "31": "anu", "32": "apeja", "33": "asa", "34": "awen", "35": "b", "36": "brillig", "37": "c", "38": "co", "39": "com", "40": "counties", "41": "de", "42": "di", "43": "e", "44": "en", "45": "enepi", "46": "enko", "47": "epansa", "48": "epiku", "49": "er", "50": "es", "51": "esun", "52": "ete", "53": "h", "54": "had", "55": "hij", "56": "i", "57": "ijo", "58": "ike", "59": "iki", "60": "il", "61": "ilo", "62": "in", "63": "info", "64": "inli", "65": "insa", "66": "ipi", "67": "iseki", "68": "isipin", "69": "j", "70": "ja", "71": "jaki", "72": "jalan", "73": "jami", "74": "jan", "75": "jasima", "76": "jelo", "77": "jetesantakalu", "78": "jo", "79": "josuta", "80": "jule", "81": "k", "82": "ka", "83": "kala", "84": "kalama", "85": "kalu", "86": "kama", "87": "kamala", "88": "kamalawala", "89": "kan", "90": "kapesi", "91": "kasi", "92": "ke", "93": "ken", "94": "kepeken", "95": "kese", "96": "ki", "97": "kijetenitakalu", "98": "kijetesanataklu", "99": "kijetesantakalu", "100": "kiki", "101": "kili", "102": "kin", "103": "kipisi", "104": "kisa", "105": "kiwen", "106": "ko", "107": "kokosila", "108": "kon", "109": "konwe", "110": "ku", "111": "kule", "112": "kulu", "113": "kulupu", "114": "kuntu", "115": "kute", "116": "l", "117": "la", "118": "lanpan", "119": "lape", "120": "laso", "121": "lawa", "122": "leko", "123": "len", "124": "lete", "125": "li", "126": "lili", "127": "lin", "128": "linja", "129": "linluwi", "130": "lipamanka", "131": "lipu", "132": "lo", "133": "loje", "134": "lon", "135": "lu", "136": "luka", "137": "lukin", "138": "luna", "139": "lupa", "140": "m", "141": "ma", "142": "majuna", "143": "makan", "144": "mama", "145": "man", "146": "mani", "147": "meli", "148": "melome", "149": "meso", "150": "mi", "151": "mije", "152": "mijomi", "153": "misa", "154": "misikeke", "155": "misu", "156": "moku", "157": "mola", "158": "moli", "159": "monsi", "160": "monsuta", "161": "monsutatesu", "162": "msa", "163": "mu", "164": "mulapisu", "165": "mun", "166": "musi", "167": "musu", "168": "mute", "169": "muu", "170": "n", "171": "na", "172": "nala", "173": "nalanja", "174": "namako", "175": "nanpa", "176": "nasa", "177": "nasin", "178": "nata", "179": "nena", "180": "ni", "181": "nimi", "182": "nn", "183": "nnn", "184": "noka", "185": "nu", "186": "o", "187": "of", "188": "oji", "189": "oke", "190": "oko", "191": "olin", "192": "omekapo", "193": "on", "194": "ona", "195": "open", "196": "org", "197": "owe", "198": "p", "199": "paka", "200": "pakala", "201": "pake", "202": "pali", "203": "palisa", "204": "pan", "205": "pana", "206": "pasila", "207": "pata", "208": "peta", "209": "pi", "210": "pilin", "211": "pimeja", "212": "pimjea", "213": "pini", "214": "pipi", "215": "po", "216": "poka", "217": "poki", "218": "polinpin", "219": "pona", "220": "powe", "221": "print", "222": "pu", "223": "puwa", "224": "r", "225": "remove", "226": "s", "227": "salam", "228": "sama", "229": "san", "230": "sapiens", "231": "seki", "232": "seli", "233": "selo", "234": "seme", "235": "sen", "236": "sewi", "237": "si", "238": "sijelo", "239": "sike", "240": "sin", "241": "sina", "242": "sinpin", "243": "sipi", "244": "sitelen", "245": "soko", "246": "sona", "247": "soto", "248": "soweli", "249": "sowo", "250": "su", "251": "suli", "252": "sulo", "253": "sun", "254": "suno", "255": "supa", "256": "super", "257": "sutopatikuna", "258": "suwi", "259": "t", "260": "ta", "261": "taki", "262": "tan", "263": "taso", "264": "tawa", "265": "te", "266": "teje", "267": "telo", "268": "tempo", "269": "ten", "270": "tenpo", "271": "test", "272": "the", "273": "tiki", "274": "to", "275": "tok", "276": "toki", "277": "tokipona", "278": "tokiponido", "279": "tomo", "280": "tonsi", "281": "tu", "282": "tuki", "283": "tuli", "284": "tuta", "285": "u", "286": "unpa", "287": "unu", "288": "usawi", "289": "uta", "290": "utala", "291": "utula", "292": "w", "293": "wa", "294": "waleja", "295": "walo", "296": "wan", "297": "waso", "298": "wawa", "299": "we", "300": "weka", "301": "whiffling", "302": "wikipedia", "303": "wile", "304": "y", "305": "yupekosi", "306": "Alapi", "307": "Amelika", "308": "Anku", "309": "Apeja", "310": "Apika", "311": "Asija", "312": "Elena", "313": "Elopa", "314": "Ensi", "315": "Epanja", "316": "Epelanto", "317": "Inli", "318": "Intonesija", "319": "Isale", "320": "Isilami", "321": "Italija", "322": "Itan", "323": "JAN", "324": "Jesu", "325": "Juke", "326": "KUNTULI", "327": "Kanata", "328": "Kanse", "329": "Ke", "330": "Lakuse", "331": "Lasina", "332": "Loma", "333": "Losi", "334": "Luke", "335": "Lusi", "336": "Mali", "337": "Masu", "338": "Mawi", "339": "Mesiko", "340": "Mewi", "341": "Mewika", "342": "Mupa", "343": "Nasi", "344": "Netelan", "345": "Nijon", "346": "Oselija", "347": "Paki", "348": "Palata", "349": "Pali", "350": "Pasen", "351": "Pasiju", "352": "Posuka", "353": "Potuke", "354": "Puta", "355": "Sami", "356": "San", "357": "Sesesele", "358": "Sili", "359": "Sonja", "360": "Sonko", "361": "TU", "362": "Tami", "363": "Tan", "364": "Tatowepa", "365": "Tepo", "366": "Ton", "367": "Tosi", "368": "Tuki", "369": "Wikipesija"}}