atrost/nanochat-d24-matformer-h16-natural-20260506-m

This is a standalone nanochat-native checkpoint extracted from atrost/nanochat-d24-matformer-h16-natural-20260506/d24_matformer_h16_natural_20260506 at step 003624.

• Family: matformer
• Submodel: M
• Config: {"attn_alpha_init_value": 1.0, "dec_alpha_init_value": 1.0, "ffn_alpha_init_value": 1.0, "head_dim": 80, "matformer_ffn_hidden_dims": [1280], "matformer_head_counts": [4], "n_embd": 1280, "n_head": 4, "n_kv_head": 4, "n_layer": 24, "norm_type": "rmsnorm", "sequence_len": 2048, "vocab_size": 32768, "window_pattern": "L"}

Load it into the local nanochat checkpoint layout before evaluation:

from nanochat.hf_utils import stage_hf_checkpoint

stage_hf_checkpoint(
    repo_id="atrost/nanochat-d24-matformer-h16-natural-20260506-m",
    path_in_repo="d24_matformer_h16_natural_20260506_m",
    model_tag="d24_matformer_h16_natural_20260506_m",
    step=3624,
    base_dir="/tmp/nanochat-eval",
)

Then run eval with:

NANOCHAT_BASE_DIR=/tmp/nanochat-eval python -m scripts.base_eval --model-tag d24_matformer_h16_natural_20260506_m --step 3624