Recor2d commited on
Commit
803906e
·
verified ·
1 Parent(s): ccb0eb8

Add BAM-B0 model card

Browse files
Files changed (1) hide show
  1. README.md +33 -0
README.md ADDED
@@ -0,0 +1,33 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ tags:
4
+ - babylm
5
+ - masked-language-model
6
+ - gpt-bert
7
+ - strict
8
+ ---
9
+
10
+ # BAM-B0
11
+
12
+ BAM-B0 (BabyLM Aligned MNTP) is a 96.3M-parameter GPT-BERT/MNTP masked language
13
+ model trained for the BabyLM 2026 Strict track.
14
+
15
+ ## Model
16
+
17
+ - Architecture: custom GPT-BERT masked language model
18
+ - Parameters: 96,333,280
19
+ - Tokenizer: 16k BabyLM Strict tokenizer
20
+ - Objective: 15 masked-next-token-prediction steps to 1 causal language modeling step
21
+ - Initialization: random initialization
22
+
23
+ ## Evaluation
24
+
25
+ The B0-only Strict submission artifact contains final predictions, B0's own
26
+ 28-checkpoint fast-evaluation trajectory, GlobalPIQA, filtered Entity
27
+ Tracking predictions, Reading predictions, and 28-stage AoA surprisals. It
28
+ was checked with the 2026 leaderboard validation code. The JSON is retained
29
+ separately for the official BabyLM submission and is intentionally not
30
+ included in this model repository.
31
+
32
+ The model uses custom configuration and modeling files included in this
33
+ repository; load it with `trust_remote_code=True`.