KSP-NMAI
/

Text Generation
Transformers
Safetensors
English
gpt2
boris
nmai
250M
text-generation-inference
KlondikeDev commited on
Commit
41fb011
·
verified ·
1 Parent(s): 5030b35

Fix model card: correct title, document prompt format, add pipeline_tag/base_model metadata

Browse files
Files changed (1) hide show
  1. README.md +47 -21
README.md CHANGED
@@ -1,58 +1,84 @@
1
  ---
2
  license: apache-2.0
 
 
 
 
3
  datasets:
4
  - HuggingFaceFW/fineweb-edu
5
  - mlfoundations/dclm-baseline-1.0-parquet
6
- language:
7
- - en
8
  tags:
9
  - boris
10
  - nmai
 
11
  - 250M
12
  ---
13
 
14
  ![Boris](Boris-250M.png)
15
 
16
  # Boris-250M
17
- Boris-250M is a Large Language Model created by New Millennium Artificial Intelligence, trained on 5.01B tokens and contains 250 million parameters.
18
 
19
- Trained for ~105:56:17 on one RTX 3060.
 
20
 
21
- # Final Stats:
22
- ### Loss - 3.0693
23
- ### Grad Norm - 0.225
24
- ### Learning Rate - 6.00e-05
25
 
 
26
 
27
- ![Benchmarks](benchmarks.png)
 
 
 
 
28
 
 
 
 
 
29
 
30
  ## Details
31
 
32
  | | |
33
  |---|---|
34
  | Architecture | GPT-2 (pre-LN, learned positional embeddings, tied embeddings) |
35
- | Parameters | ~250.4M |
36
  | Layers / heads / d_model | 12 / 18 / 1152 |
37
  | Context length | 1024 |
38
- | Training tokens | 5.01B (20.0 tokens/param, Chinchilla-optimal) |
39
- | Training data | 60% FineWeb-Edu / 40% DCLM |
40
  | Vocab | 50304 (GPT-NeoX-20B BPE, padded) |
41
  | Tokenizer | `EleutherAI/gpt-neox-20b` |
42
  | Precision | trained in bf16 autocast with fp32 master weights |
43
 
44
- Base model; not instruction-tuned or aligned. Outputs may be inaccurate or
45
- offensive.
46
 
47
- ## Copyright & License:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
48
 
49
  *Copyright 2026 Joseph Jones*
50
 
51
- This project and all associated files (the "Work") are licensed under the Apache License, Version 2.0 (the "License"); you may not use this project except in compliance with the License. You may obtain a copy of the License at:
 
 
52
 
53
  http://www.apache.org/licenses/LICENSE-2.0
54
- Unless required by applicable law or agreed to in writing, software
55
- distributed under the License is distributed on an "AS IS" BASIS,
56
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
57
- See the License for the specific language governing permissions and
58
- limitations under the License.
 
1
  ---
2
  license: apache-2.0
3
+ language:
4
+ - en
5
+ library_name: transformers
6
+ pipeline_tag: text-generation
7
  datasets:
8
  - HuggingFaceFW/fineweb-edu
9
  - mlfoundations/dclm-baseline-1.0-parquet
 
 
10
  tags:
11
  - boris
12
  - nmai
13
+ - gpt2
14
  - 250M
15
  ---
16
 
17
  ![Boris](Boris-250M.png)
18
 
19
  # Boris-250M
 
20
 
21
+ Boris-250M is a 250 million-parameter language model created by New Millennium
22
+ Artificial Intelligence (NMAI).
23
 
24
+ This is a **base (pretrained) model**. It has not been instruction-tuned and does
25
+ not follow instructions or hold a conversation — it continues text. For an
26
+ instruction-following version, see
27
+ [KSP-NMAI/Boris-250M-Instruct](https://huggingface.co/KSP-NMAI/Boris-250M-Instruct).
28
 
29
+ ## Usage
30
 
31
+ ```python
32
+ from transformers import AutoModelForCausalLM, AutoTokenizer
33
+
34
+ tok = AutoTokenizer.from_pretrained("KSP-NMAI/Boris-250M")
35
+ model = AutoModelForCausalLM.from_pretrained("KSP-NMAI/Boris-250M")
36
 
37
+ ids = tok("The ocean is", return_tensors="pt").input_ids
38
+ out = model.generate(ids, max_new_tokens=40, do_sample=True, top_p=0.95)
39
+ print(tok.decode(out[0], skip_special_tokens=True))
40
+ ```
41
 
42
  ## Details
43
 
44
  | | |
45
  |---|---|
46
  | Architecture | GPT-2 (pre-LN, learned positional embeddings, tied embeddings) |
 
47
  | Layers / heads / d_model | 12 / 18 / 1152 |
48
  | Context length | 1024 |
 
 
49
  | Vocab | 50304 (GPT-NeoX-20B BPE, padded) |
50
  | Tokenizer | `EleutherAI/gpt-neox-20b` |
51
  | Precision | trained in bf16 autocast with fp32 master weights |
52
 
53
+ ## Base model training
 
54
 
55
+ Trained on 5.01B tokens for ~105:56:17 on one RTX 3060.
56
+
57
+ | | |
58
+ |---|---|
59
+ | Final loss | 3.0693 |
60
+ | Final grad norm | 0.225 |
61
+ | Final learning rate | 6.00e-05 |
62
+
63
+ ![Benchmarks](benchmarks.png)
64
+
65
+ ## Limitations
66
+
67
+ A base model of this size will produce text that is frequently inaccurate,
68
+ inconsistent, or offensive. It has received no alignment or safety tuning and
69
+ should not be used for factual reference or deployed without supervision.
70
+
71
+ ## Copyright & License
72
 
73
  *Copyright 2026 Joseph Jones*
74
 
75
+ This project and all associated files (the "Work") are licensed under the Apache
76
+ License, Version 2.0 (the "License"); you may not use this project except in
77
+ compliance with the License. You may obtain a copy of the License at:
78
 
79
  http://www.apache.org/licenses/LICENSE-2.0
80
+
81
+ Unless required by applicable law or agreed to in writing, software distributed
82
+ under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
83
+ CONDITIONS OF ANY KIND, either express or implied. See the License for the
84
+ specific language governing permissions and limitations under the License.