Upload folder using huggingface_hub

README.md

@@ -1,3 +1,122 @@
 ---
-license: apache-2.0
+license: mit
+tags:
+  - educational
+  - from-scratch
+  - character-level
+  - transformer
+  - pytorch
 ---
+
+# SIL-v01 — Tiny From-Scratch Language Model
+
+**This is an educational project, not a practical language model.** It is a
+from-scratch, char-RNN-scale transformer built to learn the mechanics of
+language modeling end-to-end: tokenization → embeddings → the
+language-modeling objective → training loop → self-attention → subword
+tokenization. It is not intended for any downstream or production use —
+treat it as a worked example, not a tool.
+
+The published checkpoint here is the **best-performing configuration found
+in a controlled scaling sweep**: a 4-layer, 4-head, causal self-attention
+transformer with `n_embd=128`, trained on a character-level tokenizer.
+
+## Model architecture
+
+Token + position embeddings → 4 stacked pre-norm transformer blocks
+(multi-head causal self-attention + feedforward, residual connections) →
+final LayerNorm → linear head to vocab logits. Same family as a minimal
+GPT, at a scale that trains on a CPU in minutes. Full hyperparameters are
+in `config.json`; nothing about the architecture is hardcoded in the loader
+— see `load_model.py`.
+
+| | |
+|---|---|
+| Parameters | 812,609 |
+| n_embd | 128 |
+| n_head | 4 |
+| n_layer | 4 |
+| block_size | 32 |
+| Tokenizer | character-level, vocab_size=65 |
+
+## Training corpus
+
+Trained on **TinyShakespeare** (~1.1M characters), the classic small corpus
+for character-level language modeling. It is fetched directly from Andrej
+Karpathy's `char-rnn` GitHub repository
+(`raw.githubusercontent.com/karpathy/char-rnn/master/data/tinyshakespeare/input.txt`)
+— a concatenation of Shakespeare's plays, ultimately public-domain text, but
+sourced from that repo directly rather than downloaded from Project
+Gutenberg. (This project's data pipeline also includes a Project Gutenberg
+boilerplate-stripper for *other* corpora it experiments with — Dostoevsky,
+Nietzsche, Suetonius — but that path was not used for this checkpoint.)
+
+90/10 train/val split, held fixed across every run below.
+
+## Results
+
+Four runs, all on TinyShakespeare, same architecture family (`n_layer=4`,
+`n_head=4`, `block_size=32`, `lr=1e-3`, `max_iters=8000`, `batch_size=32`,
+seed=1337) — a controlled sweep over embedding width, plus one run swapping
+in a trained BPE subword tokenizer instead of character-level.
+
+**Raw cross-entropy (nats/token) is not comparable across tokenizers** —
+a 1000-token BPE vocabulary and a 65-token char vocabulary have different
+random-guessing floors (ln(1000) ≈ 6.9 vs ln(65) ≈ 4.2), so a higher raw
+loss on BPE does not mean a worse model. **Bits-per-character (BPC)**
+normalizes both onto the same unit — bits of model surprise per character
+of the *original* text — and is the number to compare across the table.
+
+| Tokenizer | n_embd | Params | Train loss (nats) | Train BPC | Val loss (nats) | Val BPC | Train/val gap (BPC) | Wall clock |
+|---|---|---|---|---|---|---|---|---|
+| char | 32  | 55,745  | 1.7159 | 2.4755 | 1.8849 | 2.7193 | 0.2438 | 4m48.8s |
+| char | 64  | 209,729 | 1.5247 | 2.1997 | 1.7220 | 2.4843 | 0.2846 | 7m19.4s |
+| char | **128** | **812,609** | **1.4043** | **2.0260** | **1.6257** | **2.3454** | 0.3194 | 16m37.6s |
+| bpe (vocab=1000) | 32 | 116,520 | 3.5683 | 2.1272 | 3.8843 | 2.4189 | 0.2917 | 6m29.5s |
+
+**Published checkpoint: char, n_embd=128** (bolded row) — lowest val BPC of
+the char-level sweep.
+
+Scaling observation: doubling `n_embd` costs a roughly constant ~3.8×
+parameters each step (32→64, 64→128), but the val-BPC improvement shrinks
+each time (0.235 → 0.139 in BPC terms), and the train/val gap widens —
+diminishing, saturating returns from embedding width alone once depth
+(`n_layer=4`) and context (`block_size=32`) are held fixed.
+
+Tokenizer observation: char-level (BPC 2.35) still edges out this one BPE
+run (BPC 2.42) at comparable parameter count, but BPE achieves that with
+~7x fewer parameters than the 812K char model and visibly more coherent
+generated chunks — a fair head-to-head at matched parameter count wasn't
+run here.
+
+## Files
+
+| File | Purpose |
+|---|---|
+| `model.pt` | `state_dict()` of the trained `BigramLanguageModel` |
+| `config.json` | Every hyperparameter needed to reconstruct the architecture and tokenizer |
+| `tokenizer_char.json` | The character-level tokenizer's `stoi` vocabulary |
+| `bigram.py` | Model architecture source (vendored so this repo is self-sufficient) |
+| `tokenizer.py` | Character tokenizer source (vendored, same reason) |
+| `load_model.py` | Reconstructs the model + tokenizer from `config.json` and generates a sample |
+
+## Usage
+
+```bash
+pip install torch
+python load_model.py
+```
+
+`load_model.py` reads `config.json` for every architectural parameter
+(`vocab_size`, `n_embd`, `n_head`, `n_layer`, `block_size`, `tokenizer_type`)
+— it does not assume or hardcode them — loads `model.pt` into a freshly
+constructed model, and samples 400 characters to prove the checkpoint and
+its config agree.
+
+## Limitations
+
+This is a ~65-character-vocabulary, 32-token-context toy model trained for
+8,000 steps on 1MB of text. It reproduces surface Shakespeare-ish texture
+(names, verse-like line breaks, archaic diction) but not coherent meaning,
+plot, or factual content. **Do not use this for anything beyond studying
+how these mechanics fit together.**

bigram.py

@@ -0,0 +1,266 @@
+"""
+Stage 2: The bigram language model — the simplest possible neural LM.
+
+Originally this was ONE embedding table of shape (vocab_size, vocab_size):
+row i a vector of raw next-token scores for "current token is i." Then a
+second table added position information. Then a single self-attention Head.
+Now the head is split into several narrower heads that run in parallel.
+
+  - token_embedding_table:    one learned vector per vocab entry — the
+                               token's identity, regardless of where it sits.
+  - position_embedding_table: one learned vector per slot in the context
+                               window — "this is position 3," regardless of
+                               which token is there.
+  - MultiHeadAttention:       n_head independent Heads, each working in a
+                               narrower head_size = n_embd // n_head space,
+                               run in parallel and concatenated back to
+                               n_embd. Same total width as one big head, but
+                               each head can specialize on a different kind
+                               of relationship between positions instead of
+                               averaging all of them into one.
+
+    predict the next token; measure error with cross-entropy loss.
+
+Everything a GPT does is this same objective with a smarter architecture.
+Note the connection to the Word2vec slide from your lecture: like CBOW /
+Skip-gram, this model learns embeddings. Position embeddings were a crude,
+static fix for word order; self-attention is the real one — instead of a
+fixed "position 3" vector, each token computes a data-dependent, weighted
+average of what came before it.
+"""
+
+from __future__ import annotations
+
+import torch
+import torch.nn as nn
+from torch.nn import functional as F
+
+
+class Head(nn.Module):
+    """One head of causal self-attention."""
+
+    def __init__(self, n_embd: int, head_size: int, block_size: int) -> None:
+        super().__init__()
+        # Three separate linear projections of the same input x, each
+        # (n_embd, head_size). No bias — these are pure projections, like
+        # rotating/rescaling x into a new space, not adding an offset.
+        self.key = nn.Linear(n_embd, head_size, bias=False)
+        self.query = nn.Linear(n_embd, head_size, bias=False)
+        self.value = nn.Linear(n_embd, head_size, bias=False)
+        # A (block_size, block_size) lower-triangular matrix of 1s, e.g. for
+        # block_size=4:
+        #   1 0 0 0
+        #   1 1 0 0
+        #   1 1 1 0
+        #   1 1 1 1
+        # Row t has 1s in columns 0..t: "position t may attend to positions
+        # 0..t." Registered as a buffer (not a parameter) so it moves with
+        # .to(device) and is saved in state_dict, but is never trained.
+        self.register_buffer("tril", torch.tril(torch.ones(block_size, block_size)))
+        self.head_size = head_size
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        """
+        x: (B, T, n_embd) — the token+position embeddings for one batch.
+        Returns (B, T, head_size): each position's attention-weighted
+        summary of itself and everything before it.
+        """
+        B, T, _ = x.shape
+
+        # Every position asks a question (query) and advertises an answer
+        # (key). Both (B, T, n_embd) -> (B, T, head_size).
+        q = self.query(x)
+        k = self.key(x)
+
+        # Compare every query to every key via dot product:
+        # (B, T, head_size) @ (B, head_size, T) -> (B, T, T).
+        # wei[b, i, j] = "how much position i's query matches position j's
+        # key" — raw, unnormalized affinity, before masking or softmax.
+        wei = q @ k.transpose(-2, -1)
+        # Divide by sqrt(head_size) before softmax — see the explanation
+        # printed by the demo script; in short, it keeps the variance of
+        # these dot products at ~1 instead of growing with head_size, so
+        # softmax doesn't saturate into a near one-hot distribution.
+        wei = wei * self.head_size**-0.5
+
+        # Causal mask: position i must not see position j > i (the future).
+        # tril[:T, :T] == 0 marks the upper triangle (j > i); those entries
+        # become -inf so softmax turns them into exactly 0.
+        wei = wei.masked_fill(self.tril[:T, :T] == 0, float("-inf"))
+        # Softmax over the last dim (T, the "which key" axis): every row i
+        # becomes a probability distribution over positions 0..i that sums
+        # to 1. Still (B, T, T).
+        wei = F.softmax(wei, dim=-1)
+
+        # Every position also advertises a value: (B, T, n_embd) ->
+        # (B, T, head_size). This is what actually gets mixed together,
+        # as opposed to key/query which only decide the mixing weights.
+        v = self.value(x)
+        # (B, T, T) @ (B, T, head_size) -> (B, T, head_size): position i's
+        # output is the wei[i, :]-weighted average of every value vector at
+        # positions 0..i.
+        out = wei @ v
+        return out
+
+
+class MultiHeadAttention(nn.Module):
+    """Several causal self-attention heads run in parallel, then combined."""
+
+    def __init__(self, n_embd: int, n_head: int, block_size: int) -> None:
+        super().__init__()
+        # n_embd must split evenly across n_head so the concatenated output
+        # lands back at exactly n_embd (e.g. 32 = 4 heads * 8).
+        assert n_embd % n_head == 0, "n_embd must be divisible by n_head"
+        head_size = n_embd // n_head
+        self.heads = nn.ModuleList(
+            [Head(n_embd, head_size, block_size) for _ in range(n_head)]
+        )
+        # Each head reads the same x (n_embd-wide) and produces its own
+        # head_size-wide output. Concatenating n_head of those gets back to
+        # n_embd; this projection lets the heads' outputs mix before moving
+        # on, rather than just being stapled together.
+        self.proj = nn.Linear(n_embd, n_embd)
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        # Each h(x): (B, T, head_size). Concatenate along the last dim ->
+        # (B, T, n_head * head_size) == (B, T, n_embd).
+        out = torch.cat([h(x) for h in self.heads], dim=-1)
+        return self.proj(out)
+
+
+class FeedForward(nn.Module):
+    """Per-position MLP: the "thinking" step after attention's "looking."
+
+    Attention only moves information between positions via weighted
+    averages of value vectors — it can't recombine features nonlinearly.
+    This module runs independently on every position (no mixing across T)
+    and gives the model a place to do that recombination. The inner
+    dimension is widened 4x, as in the original Transformer paper, then
+    projected back to n_embd so it can be added into the residual stream.
+    """
+
+    def __init__(self, n_embd: int) -> None:
+        super().__init__()
+        self.net = nn.Sequential(
+            nn.Linear(n_embd, 4 * n_embd),
+            nn.ReLU(),
+            nn.Linear(4 * n_embd, n_embd),
+        )
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        return self.net(x)
+
+
+class Block(nn.Module):
+    """One transformer block: communicate (attention), then compute (FFN).
+
+    Pre-norm residual design: x = x + sublayer(LN(x)) for both sublayers.
+    The "+" is what makes this residual — each sublayer adds a correction
+    to x rather than replacing it, so during backprop gradients have a
+    direct addition path (derivative 1) back through every block, no matter
+    how deep the stack gets. Without it, gradients must flow through each
+    block's full nonlinear transform in series, and repeated multiplication
+    by small Jacobians is exactly the vanishing-gradient failure mode from
+    the RNN slide. LayerNorm before each sublayer keeps that sublayer's
+    input at a stable, zero-mean/unit-variance scale regardless of how
+    large the residual stream has grown by this depth.
+    """
+
+    def __init__(self, n_embd: int, n_head: int, block_size: int) -> None:
+        super().__init__()
+        self.sa = MultiHeadAttention(n_embd, n_head, block_size)
+        self.ffwd = FeedForward(n_embd)
+        self.ln1 = nn.LayerNorm(n_embd)
+        self.ln2 = nn.LayerNorm(n_embd)
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        x = x + self.sa(self.ln1(x))
+        x = x + self.ffwd(self.ln2(x))
+        return x
+
+
+class BigramLanguageModel(nn.Module):
+    def __init__(
+        self,
+        vocab_size: int,
+        block_size: int,
+        n_embd: int,
+        n_head: int,
+        n_layer: int,
+    ) -> None:
+        super().__init__()
+        self.block_size = block_size
+        # (vocab_size, n_embd): row i is token i's identity vector.
+        self.token_embedding_table = nn.Embedding(vocab_size, n_embd)
+        # (block_size, n_embd): row t is position t's vector. There are
+        # only block_size rows because that's the longest context this
+        # model will ever be asked about — see the crop in generate().
+        self.position_embedding_table = nn.Embedding(block_size, n_embd)
+        # n_layer stacked (attention, feedforward) blocks with residual
+        # connections and layernorm — see Block's docstring for why those
+        # two aren't decoration.
+        self.blocks = nn.Sequential(
+            *[Block(n_embd, n_head, block_size) for _ in range(n_layer)]
+        )
+        # One more LayerNorm after the last block. The residual stream's
+        # scale is unconstrained as it accumulates additions across n_layer
+        # blocks; this normalizes it once before the final projection reads
+        # it, matching the scale lm_head was initialized to expect.
+        self.ln_f = nn.LayerNorm(n_embd)
+        # Project the final n_embd representation to vocab-sized logits.
+        self.lm_head = nn.Linear(n_embd, vocab_size)
+
+    def forward(
+        self, idx: torch.Tensor, targets: torch.Tensor | None = None
+    ) -> tuple[torch.Tensor, torch.Tensor | None]:
+        """
+        idx:     (B, T) batch of token-id sequences
+        targets: (B, T) the same sequences shifted one position left,
+                 i.e. "the correct next token at every position"
+
+        Returns (logits, loss). Loss is None during generation.
+        """
+        B, T = idx.shape
+
+        # (B, T) -> (B, T, n_embd): each token id looks up its identity vector.
+        tok_emb = self.token_embedding_table(idx)
+        # (T,) -> (T, n_embd): one vector per position, shared across the batch.
+        pos_emb = self.position_embedding_table(torch.arange(T, device=idx.device))
+        # (B, T, n_embd) + (T, n_embd): the position vectors broadcast over
+        # the batch dim, so every sequence gets its own token vectors plus
+        # the same T position vectors. Result: (B, T, n_embd).
+        x = tok_emb + pos_emb
+        # (B, T, n_embd) -> (B, T, n_embd): n_layer rounds of (attend, then
+        # think), each wrapped in a residual connection.
+        x = self.blocks(x)
+        # Normalize the accumulated residual stream once before reading it.
+        x = self.ln_f(x)
+        # (B, T, n_embd) -> (B, T, vocab_size): project back to next-token scores.
+        logits = self.lm_head(x)
+
+        loss = None
+        if targets is not None:
+            B, T, C = logits.shape
+            # cross_entropy wants (N, C) vs (N,), so flatten batch+time.
+            loss = F.cross_entropy(logits.view(B * T, C), targets.view(B * T))
+        return logits, loss
+
+    @torch.no_grad()
+    def generate(self, idx: torch.Tensor, max_new_tokens: int) -> torch.Tensor:
+        """
+        Autoregressive sampling: feed the sequence in, take the logits at
+        the LAST position, turn them into probabilities, sample one token,
+        append it, repeat. This loop is identical in shape to how a full
+        GPT generates text — only the model in the middle changes.
+        """
+        for _ in range(max_new_tokens):
+            # position_embedding_table has only block_size rows, so a
+            # sequence longer than that would index out of range — crop to
+            # the last block_size tokens before every forward pass.
+            idx_cond = idx[:, -self.block_size :]
+            logits, _ = self(idx_cond)         # (B, T, C)
+            logits = logits[:, -1, :]          # last time step: (B, C)
+            probs = F.softmax(logits, dim=-1)  # scores -> probabilities
+            next_id = torch.multinomial(probs, num_samples=1)  # sample
+            idx = torch.cat((idx, next_id), dim=1)
+        return idx

config.json

@@ -0,0 +1,20 @@
+{
+  "architecture": "BigramLanguageModel",
+  "architecture_description": "Token + position embeddings -> n_layer stacked transformer blocks (pre-norm multi-head causal self-attention + feedforward, residual connections) -> final LayerNorm -> linear head to vocab logits.",
+  "vocab_size": 65,
+  "n_embd": 128,
+  "n_head": 4,
+  "n_layer": 4,
+  "block_size": 32,
+  "tokenizer_type": "char",
+  "tokenizer_file": "tokenizer_char.json",
+  "weights_file": "model.pt",
+  "training": {
+    "dataset": "tinyshakespeare.txt",
+    "batch_size": 32,
+    "learning_rate": 1e-3,
+    "max_iters": 8000,
+    "seed": 1337
+  },
+  "parameter_count": 812609
+}

load_model.py

@@ -0,0 +1,64 @@
+"""
+Self-contained loader for this checkpoint.
+
+Reconstructs the model architecture and tokenizer entirely from config.json
+(no hardcoded hyperparameters here), loads the trained weights, and
+generates a sample -- proving the artifact set (config.json + model.pt +
+tokenizer_char.json + bigram.py + tokenizer.py) is sufficient on its own,
+with no dependency on the original training repo or its train.py.
+
+Usage:
+    python load_model.py
+"""
+
+from __future__ import annotations
+
+import json
+from pathlib import Path
+
+import torch
+
+from bigram import BigramLanguageModel
+from tokenizer import CharTokenizer
+
+HERE = Path(__file__).resolve().parent
+
+
+def load_model_and_tokenizer(dir_path: Path = HERE):
+    config = json.loads((dir_path / "config.json").read_text())
+
+    if config["tokenizer_type"] != "char":
+        raise ValueError(
+            f"This loader only wires up 'char'; config says {config['tokenizer_type']!r}"
+        )
+    tokenizer = CharTokenizer.load(dir_path / config["tokenizer_file"])
+
+    model = BigramLanguageModel(
+        vocab_size=config["vocab_size"],
+        block_size=config["block_size"],
+        n_embd=config["n_embd"],
+        n_head=config["n_head"],
+        n_layer=config["n_layer"],
+    )
+    state_dict = torch.load(dir_path / config["weights_file"], map_location="cpu")
+    model.load_state_dict(state_dict)
+    model.eval()
+
+    return model, tokenizer, config
+
+
+def main() -> None:
+    model, tokenizer, config = load_model_and_tokenizer()
+
+    n_params = sum(p.numel() for p in model.parameters())
+    print(f"Loaded model: {n_params:,} parameters (config.json says {config['parameter_count']:,})")
+    assert n_params == config["parameter_count"], "Reconstructed model doesn't match config!"
+
+    context = torch.zeros((1, 1), dtype=torch.long)
+    print("\n----- generated sample -----")
+    sample = tokenizer.decode(model.generate(context, max_new_tokens=400)[0].tolist())
+    print(sample)
+
+
+if __name__ == "__main__":
+    main()

model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c13668656eb1fc5c87780a22b34dcb0ca4fdfe63e6e6c87a81235a95e5ffdc78
+size 3348079

tokenizer.py

@@ -0,0 +1,79 @@
+"""
+Stage 1: Tokenization (character-level).
+
+This maps directly onto the "Tokenization summary" slide you saw:
+we're starting at the CHARACTER level because it's the simplest possible
+scheme — near-zero risk of out-of-vocabulary tokens, trivially
+interpretable, and the vocab is tiny (~65 symbols for Shakespeare).
+The cost is longer sequences and less meaningful individual tokens.
+
+Later (Stage 4) you'll swap this out for a subword tokenizer (BPE) and
+watch how vocab size, sequence length, and sample quality change.
+That contrast is one of the best lessons in this whole exercise.
+
+Key idea: a tokenizer is just two lookup tables.
+    encode: string  -> list of integers
+    decode: list of integers -> string
+Everything downstream (embeddings, attention) operates on the integers.
+"""
+
+from __future__ import annotations
+
+import json
+from pathlib import Path
+
+
+class CharTokenizer:
+    """A character-level tokenizer built from a training corpus."""
+
+    def __init__(self, text: str) -> None:
+        # The vocabulary is simply every unique character, sorted so the
+        # mapping is deterministic across runs.
+        chars = sorted(set(text))
+        self.vocab_size = len(chars)
+
+        # stoi = "string to integer", itos = "integer to string".
+        # These two dicts ARE the tokenizer. That's it.
+        self.stoi: dict[str, int] = {ch: i for i, ch in enumerate(chars)}
+        self.itos: dict[int, str] = {i: ch for i, ch in enumerate(chars)}
+
+    def encode(self, text: str) -> list[int]:
+        """Convert a string into a list of token ids."""
+        return [self.stoi[ch] for ch in text]
+
+    def decode(self, ids: list[int]) -> str:
+        """Convert a list of token ids back into a string."""
+        return "".join(self.itos[i] for i in ids)
+
+    # --- persistence, so a trained model ships with its tokenizer ---
+
+    def save(self, path: str | Path) -> None:
+        Path(path).write_text(
+            json.dumps({"stoi": self.stoi}, indent=2), encoding="utf-8"
+        )
+
+    @classmethod
+    def load(cls, path: str | Path) -> "CharTokenizer":
+        stoi = json.loads(Path(path).read_text(encoding="utf-8"))["stoi"]
+        tok = cls.__new__(cls)
+        tok.stoi = {k: int(v) for k, v in stoi.items()}
+        tok.itos = {int(v): k for k, v in stoi.items()}
+        tok.vocab_size = len(tok.stoi)
+        return tok
+
+
+if __name__ == "__main__":
+    # Quick self-test / demo
+    data_path = Path(__file__).resolve().parent.parent / "data" / "input.txt"
+    text = data_path.read_text(encoding="utf-8")
+
+    tok = CharTokenizer(text)
+    print(f"Vocab size: {tok.vocab_size}")
+    print(f"Vocabulary: {''.join(tok.itos[i] for i in range(tok.vocab_size))!r}")
+
+    sample = "To be, or not to be"
+    ids = tok.encode(sample)
+    print(f"\nencode({sample!r})\n  -> {ids}")
+    print(f"decode(...)\n  -> {tok.decode(ids)!r}")
+    assert tok.decode(ids) == sample, "Round-trip failed!"
+    print("\nRound-trip OK ✔")

tokenizer_char.json

@@ -0,0 +1,69 @@
+{
+  "stoi": {
+    "\n": 0,
+    " ": 1,
+    "!": 2,
+    "$": 3,
+    "&": 4,
+    "'": 5,
+    ",": 6,
+    "-": 7,
+    ".": 8,
+    "3": 9,
+    ":": 10,
+    ";": 11,
+    "?": 12,
+    "A": 13,
+    "B": 14,
+    "C": 15,
+    "D": 16,
+    "E": 17,
+    "F": 18,
+    "G": 19,
+    "H": 20,
+    "I": 21,
+    "J": 22,
+    "K": 23,
+    "L": 24,
+    "M": 25,
+    "N": 26,
+    "O": 27,
+    "P": 28,
+    "Q": 29,
+    "R": 30,
+    "S": 31,
+    "T": 32,
+    "U": 33,
+    "V": 34,
+    "W": 35,
+    "X": 36,
+    "Y": 37,
+    "Z": 38,
+    "a": 39,
+    "b": 40,
+    "c": 41,
+    "d": 42,
+    "e": 43,
+    "f": 44,
+    "g": 45,
+    "h": 46,
+    "i": 47,
+    "j": 48,
+    "k": 49,
+    "l": 50,
+    "m": 51,
+    "n": 52,
+    "o": 53,
+    "p": 54,
+    "q": 55,
+    "r": 56,
+    "s": 57,
+    "t": 58,
+    "u": 59,
+    "v": 60,
+    "w": 61,
+    "x": 62,
+    "y": 63,
+    "z": 64
+  }
+}