Hugging Face's logo

Lambent
/
cosmo-rag-1b-v0.1

PEFT
PyTorch
Safetensors
llama
Generated from Trainer
Model card Files
xet
 Community
1
Lambent commited on
Commit
ed2ddc1
·
1 Parent(s): f9b4c2f

merged model only

Browse files
Files changed (26) hide show
  1. adapter_config.json +0 -33
  2. adapter_model.bin +0 -3
  3. checkpoint-145/README.md +0 -202
  4. checkpoint-145/adapter_config.json +0 -33
  5. checkpoint-145/adapter_model.safetensors +0 -3
  6. checkpoint-145/optimizer.pt +0 -3
  7. checkpoint-145/rng_state.pth +0 -3
  8. checkpoint-145/scheduler.pt +0 -3
  9. checkpoint-145/trainer_state.json +0 -1036
  10. checkpoint-145/training_args.bin +0 -3
  11. checkpoint-290/README.md +0 -202
  12. checkpoint-290/adapter_config.json +0 -33
  13. checkpoint-290/adapter_model.safetensors +0 -3
  14. checkpoint-290/optimizer.pt +0 -3
  15. checkpoint-290/rng_state.pth +0 -3
  16. checkpoint-290/scheduler.pt +0 -3
  17. checkpoint-290/trainer_state.json +0 -2059
  18. checkpoint-290/training_args.bin +0 -3
  19. checkpoint-435/README.md +0 -202
  20. checkpoint-435/adapter_config.json +0 -33
  21. checkpoint-435/adapter_model.safetensors +0 -3
  22. checkpoint-435/optimizer.pt +0 -3
  23. checkpoint-435/rng_state.pth +0 -3
  24. checkpoint-435/scheduler.pt +0 -3
  25. checkpoint-435/trainer_state.json +0 -3082
  26. checkpoint-435/training_args.bin +0 -3
adapter_config.json DELETED
@@ -1,33 +0,0 @@
1
- {
2
- "alpha_pattern": {},
3
- "auto_mapping": null,
4
- "base_model_name_or_path": "HuggingFaceTB/cosmo-1b",
5
- "bias": "none",
6
- "fan_in_fan_out": null,
7
- "inference_mode": true,
8
- "init_lora_weights": true,
9
- "layers_pattern": null,
10
- "layers_to_transform": null,
11
- "loftq_config": {},
12
- "lora_alpha": 32,
13
- "lora_dropout": 0.1,
14
- "megatron_config": null,
15
- "megatron_core": "megatron.core",
16
- "modules_to_save": null,
17
- "peft_type": "LORA",
18
- "r": 32,
19
- "rank_pattern": {},
20
- "revision": null,
21
- "target_modules": [
22
- "v_proj",
23
- "k_proj",
24
- "o_proj",
25
- "up_proj",
26
- "down_proj",
27
- "gate_proj",
28
- "q_proj"
29
- ],
30
- "task_type": "CAUSAL_LM",
31
- "use_dora": false,
32
- "use_rslora": false
33
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
adapter_model.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:24bd5232d3bbaf5cb3924dc87739532ef34b4a256997033cea388c17750e57ca
3
- size 144824970
 
 
 
 
checkpoint-145/README.md DELETED
@@ -1,202 +0,0 @@
1
- ---
2
- library_name: peft
3
- base_model: HuggingFaceTB/cosmo-1b
4
- ---
5
-
6
- # Model Card for Model ID
7
-
8
- <!-- Provide a quick summary of what the model is/does. -->
9
-
10
-
11
-
12
- ## Model Details
13
-
14
- ### Model Description
15
-
16
- <!-- Provide a longer summary of what this model is. -->
17
-
18
-
19
-
20
- - **Developed by:** [More Information Needed]
21
- - **Funded by [optional]:** [More Information Needed]
22
- - **Shared by [optional]:** [More Information Needed]
23
- - **Model type:** [More Information Needed]
24
- - **Language(s) (NLP):** [More Information Needed]
25
- - **License:** [More Information Needed]
26
- - **Finetuned from model [optional]:** [More Information Needed]
27
-
28
- ### Model Sources [optional]
29
-
30
- <!-- Provide the basic links for the model. -->
31
-
32
- - **Repository:** [More Information Needed]
33
- - **Paper [optional]:** [More Information Needed]
34
- - **Demo [optional]:** [More Information Needed]
35
-
36
- ## Uses
37
-
38
- <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
39
-
40
- ### Direct Use
41
-
42
- <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
43
-
44
- [More Information Needed]
45
-
46
- ### Downstream Use [optional]
47
-
48
- <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
49
-
50
- [More Information Needed]
51
-
52
- ### Out-of-Scope Use
53
-
54
- <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
55
-
56
- [More Information Needed]
57
-
58
- ## Bias, Risks, and Limitations
59
-
60
- <!-- This section is meant to convey both technical and sociotechnical limitations. -->
61
-
62
- [More Information Needed]
63
-
64
- ### Recommendations
65
-
66
- <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
67
-
68
- Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
69
-
70
- ## How to Get Started with the Model
71
-
72
- Use the code below to get started with the model.
73
-
74
- [More Information Needed]
75
-
76
- ## Training Details
77
-
78
- ### Training Data
79
-
80
- <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
81
-
82
- [More Information Needed]
83
-
84
- ### Training Procedure
85
-
86
- <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
87
-
88
- #### Preprocessing [optional]
89
-
90
- [More Information Needed]
91
-
92
-
93
- #### Training Hyperparameters
94
-
95
- - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
96
-
97
- #### Speeds, Sizes, Times [optional]
98
-
99
- <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
100
-
101
- [More Information Needed]
102
-
103
- ## Evaluation
104
-
105
- <!-- This section describes the evaluation protocols and provides the results. -->
106
-
107
- ### Testing Data, Factors & Metrics
108
-
109
- #### Testing Data
110
-
111
- <!-- This should link to a Dataset Card if possible. -->
112
-
113
- [More Information Needed]
114
-
115
- #### Factors
116
-
117
- <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
118
-
119
- [More Information Needed]
120
-
121
- #### Metrics
122
-
123
- <!-- These are the evaluation metrics being used, ideally with a description of why. -->
124
-
125
- [More Information Needed]
126
-
127
- ### Results
128
-
129
- [More Information Needed]
130
-
131
- #### Summary
132
-
133
-
134
-
135
- ## Model Examination [optional]
136
-
137
- <!-- Relevant interpretability work for the model goes here -->
138
-
139
- [More Information Needed]
140
-
141
- ## Environmental Impact
142
-
143
- <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
144
-
145
- Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
146
-
147
- - **Hardware Type:** [More Information Needed]
148
- - **Hours used:** [More Information Needed]
149
- - **Cloud Provider:** [More Information Needed]
150
- - **Compute Region:** [More Information Needed]
151
- - **Carbon Emitted:** [More Information Needed]
152
-
153
- ## Technical Specifications [optional]
154
-
155
- ### Model Architecture and Objective
156
-
157
- [More Information Needed]
158
-
159
- ### Compute Infrastructure
160
-
161
- [More Information Needed]
162
-
163
- #### Hardware
164
-
165
- [More Information Needed]
166
-
167
- #### Software
168
-
169
- [More Information Needed]
170
-
171
- ## Citation [optional]
172
-
173
- <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
174
-
175
- **BibTeX:**
176
-
177
- [More Information Needed]
178
-
179
- **APA:**
180
-
181
- [More Information Needed]
182
-
183
- ## Glossary [optional]
184
-
185
- <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
186
-
187
- [More Information Needed]
188
-
189
- ## More Information [optional]
190
-
191
- [More Information Needed]
192
-
193
- ## Model Card Authors [optional]
194
-
195
- [More Information Needed]
196
-
197
- ## Model Card Contact
198
-
199
- [More Information Needed]
200
- ### Framework versions
201
-
202
- - PEFT 0.9.1.dev0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
checkpoint-145/adapter_config.json DELETED
@@ -1,33 +0,0 @@
1
- {
2
- "alpha_pattern": {},
3
- "auto_mapping": null,
4
- "base_model_name_or_path": "HuggingFaceTB/cosmo-1b",
5
- "bias": "none",
6
- "fan_in_fan_out": null,
7
- "inference_mode": true,
8
- "init_lora_weights": true,
9
- "layers_pattern": null,
10
- "layers_to_transform": null,
11
- "loftq_config": {},
12
- "lora_alpha": 32,
13
- "lora_dropout": 0.1,
14
- "megatron_config": null,
15
- "megatron_core": "megatron.core",
16
- "modules_to_save": null,
17
- "peft_type": "LORA",
18
- "r": 32,
19
- "rank_pattern": {},
20
- "revision": null,
21
- "target_modules": [
22
- "v_proj",
23
- "k_proj",
24
- "o_proj",
25
- "up_proj",
26
- "down_proj",
27
- "gate_proj",
28
- "q_proj"
29
- ],
30
- "task_type": "CAUSAL_LM",
31
- "use_dora": false,
32
- "use_rslora": false
33
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
checkpoint-145/adapter_model.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:e15aca1ee7543e55af667d28321a0b59a74e90f2f5e2b15430aec7bdc747d62c
3
- size 144748392
 
 
 
 
checkpoint-145/optimizer.pt DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:8bc4f1a539ae93886c989ceba123acdd796887ea67fd2510d7c7a04d2fb463ba
3
- size 72886996
 
 
 
 
checkpoint-145/rng_state.pth DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:d7d682e16e2d1eff31639821332e33972e3bbb8ddc928d8042f7015d3f47be01
3
- size 14244
 
 
 
 
checkpoint-145/scheduler.pt DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:dadc53b42f15a836e5c06cb606100a044276c39771c5eca3bb88745ea4ea20dd
3
- size 1064
 
 
 
 
checkpoint-145/trainer_state.json DELETED
@@ -1,1036 +0,0 @@
1
- {
2
- "best_metric": null,
3
- "best_model_checkpoint": null,
4
- "epoch": 0.9965635738831615,
5
- "eval_steps": 500,
6
- "global_step": 145,
7
- "is_hyper_param_search": false,
8
- "is_local_process_zero": true,
9
- "is_world_process_zero": true,
10
- "log_history": [
11
- {
12
- "epoch": 0.01,
13
- "grad_norm": 0.37809523940086365,
14
- "learning_rate": 2e-05,
15
- "loss": 1.7814,
16
- "step": 1
17
- },
18
- {
19
- "epoch": 0.01,
20
- "grad_norm": 0.366623193025589,
21
- "learning_rate": 4e-05,
22
- "loss": 1.69,
23
- "step": 2
24
- },
25
- {
26
- "epoch": 0.02,
27
- "grad_norm": 0.3546508848667145,
28
- "learning_rate": 6e-05,
29
- "loss": 1.7151,
30
- "step": 3
31
- },
32
- {
33
- "epoch": 0.03,
34
- "grad_norm": 0.36381417512893677,
35
- "learning_rate": 8e-05,
36
- "loss": 1.6609,
37
- "step": 4
38
- },
39
- {
40
- "epoch": 0.03,
41
- "grad_norm": 0.4049898684024811,
42
- "learning_rate": 0.0001,
43
- "loss": 1.752,
44
- "step": 5
45
- },
46
- {
47
- "epoch": 0.04,
48
- "grad_norm": 0.4318874478340149,
49
- "learning_rate": 0.00012,
50
- "loss": 1.747,
51
- "step": 6
52
- },
53
- {
54
- "epoch": 0.05,
55
- "grad_norm": 0.39768990874290466,
56
- "learning_rate": 0.00014,
57
- "loss": 1.4432,
58
- "step": 7
59
- },
60
- {
61
- "epoch": 0.05,
62
- "grad_norm": 0.5056629180908203,
63
- "learning_rate": 0.00016,
64
- "loss": 1.7657,
65
- "step": 8
66
- },
67
- {
68
- "epoch": 0.06,
69
- "grad_norm": 0.4276069104671478,
70
- "learning_rate": 0.00018,
71
- "loss": 1.4221,
72
- "step": 9
73
- },
74
- {
75
- "epoch": 0.07,
76
- "grad_norm": 0.433552622795105,
77
- "learning_rate": 0.0002,
78
- "loss": 1.3117,
79
- "step": 10
80
- },
81
- {
82
- "epoch": 0.08,
83
- "grad_norm": 0.42256900668144226,
84
- "learning_rate": 0.0001999972679420179,
85
- "loss": 1.2687,
86
- "step": 11
87
- },
88
- {
89
- "epoch": 0.08,
90
- "grad_norm": 0.3066452145576477,
91
- "learning_rate": 0.00019998907191735434,
92
- "loss": 1.0398,
93
- "step": 12
94
- },
95
- {
96
- "epoch": 0.09,
97
- "grad_norm": 0.3393724858760834,
98
- "learning_rate": 0.00019997541237384966,
99
- "loss": 1.0063,
100
- "step": 13
101
- },
102
- {
103
- "epoch": 0.1,
104
- "grad_norm": 0.49836328625679016,
105
- "learning_rate": 0.00019995629005787713,
106
- "loss": 1.2251,
107
- "step": 14
108
- },
109
- {
110
- "epoch": 0.1,
111
- "grad_norm": 0.3539387285709381,
112
- "learning_rate": 0.0001999317060143023,
113
- "loss": 0.9454,
114
- "step": 15
115
- },
116
- {
117
- "epoch": 0.11,
118
- "grad_norm": 0.35584160685539246,
119
- "learning_rate": 0.0001999016615864258,
120
- "loss": 1.0105,
121
- "step": 16
122
- },
123
- {
124
- "epoch": 0.12,
125
- "grad_norm": 0.33191052079200745,
126
- "learning_rate": 0.00019986615841591002,
127
- "loss": 1.1322,
128
- "step": 17
129
- },
130
- {
131
- "epoch": 0.12,
132
- "grad_norm": 0.3499913513660431,
133
- "learning_rate": 0.00019982519844268933,
134
- "loss": 0.8495,
135
- "step": 18
136
- },
137
- {
138
- "epoch": 0.13,
139
- "grad_norm": 0.2582085132598877,
140
- "learning_rate": 0.0001997787839048642,
141
- "loss": 0.9725,
142
- "step": 19
143
- },
144
- {
145
- "epoch": 0.14,
146
- "grad_norm": 0.23511624336242676,
147
- "learning_rate": 0.00019972691733857883,
148
- "loss": 0.9233,
149
- "step": 20
150
- },
151
- {
152
- "epoch": 0.14,
153
- "grad_norm": 0.273019939661026,
154
- "learning_rate": 0.00019966960157788248,
155
- "loss": 1.0366,
156
- "step": 21
157
- },
158
- {
159
- "epoch": 0.15,
160
- "grad_norm": 0.283527135848999,
161
- "learning_rate": 0.0001996068397545748,
162
- "loss": 0.9944,
163
- "step": 22
164
- },
165
- {
166
- "epoch": 0.16,
167
- "grad_norm": 0.24311360716819763,
168
- "learning_rate": 0.00019953863529803466,
169
- "loss": 1.0308,
170
- "step": 23
171
- },
172
- {
173
- "epoch": 0.16,
174
- "grad_norm": 0.24368223547935486,
175
- "learning_rate": 0.00019946499193503262,
176
- "loss": 1.0051,
177
- "step": 24
178
- },
179
- {
180
- "epoch": 0.17,
181
- "grad_norm": 0.2269805669784546,
182
- "learning_rate": 0.0001993859136895274,
183
- "loss": 0.835,
184
- "step": 25
185
- },
186
- {
187
- "epoch": 0.18,
188
- "grad_norm": 0.21915565431118011,
189
- "learning_rate": 0.00019930140488244602,
190
- "loss": 0.9997,
191
- "step": 26
192
- },
193
- {
194
- "epoch": 0.19,
195
- "grad_norm": 0.21885964274406433,
196
- "learning_rate": 0.0001992114701314478,
197
- "loss": 1.0013,
198
- "step": 27
199
- },
200
- {
201
- "epoch": 0.19,
202
- "grad_norm": 0.22426557540893555,
203
- "learning_rate": 0.00019911611435067172,
204
- "loss": 0.8633,
205
- "step": 28
206
- },
207
- {
208
- "epoch": 0.2,
209
- "grad_norm": 0.21587449312210083,
210
- "learning_rate": 0.0001990153427504683,
211
- "loss": 0.8479,
212
- "step": 29
213
- },
214
- {
215
- "epoch": 0.21,
216
- "grad_norm": 0.24985891580581665,
217
- "learning_rate": 0.0001989091608371146,
218
- "loss": 0.8936,
219
- "step": 30
220
- },
221
- {
222
- "epoch": 0.21,
223
- "grad_norm": 0.19733141362667084,
224
- "learning_rate": 0.0001987975744125135,
225
- "loss": 0.7743,
226
- "step": 31
227
- },
228
- {
229
- "epoch": 0.22,
230
- "grad_norm": 0.1997266411781311,
231
- "learning_rate": 0.00019868058957387663,
232
- "loss": 0.9078,
233
- "step": 32
234
- },
235
- {
236
- "epoch": 0.23,
237
- "grad_norm": 0.24114945530891418,
238
- "learning_rate": 0.00019855821271339125,
239
- "loss": 1.0413,
240
- "step": 33
241
- },
242
- {
243
- "epoch": 0.23,
244
- "grad_norm": 0.2427120953798294,
245
- "learning_rate": 0.00019843045051787096,
246
- "loss": 0.8308,
247
- "step": 34
248
- },
249
- {
250
- "epoch": 0.24,
251
- "grad_norm": 0.1968858540058136,
252
- "learning_rate": 0.0001982973099683902,
253
- "loss": 0.818,
254
- "step": 35
255
- },
256
- {
257
- "epoch": 0.25,
258
- "grad_norm": 0.2783421576023102,
259
- "learning_rate": 0.00019815879833990304,
260
- "loss": 0.8504,
261
- "step": 36
262
- },
263
- {
264
- "epoch": 0.25,
265
- "grad_norm": 0.23332269489765167,
266
- "learning_rate": 0.00019801492320084546,
267
- "loss": 0.9826,
268
- "step": 37
269
- },
270
- {
271
- "epoch": 0.26,
272
- "grad_norm": 0.22864645719528198,
273
- "learning_rate": 0.00019786569241272197,
274
- "loss": 0.8705,
275
- "step": 38
276
- },
277
- {
278
- "epoch": 0.27,
279
- "grad_norm": 0.20620585978031158,
280
- "learning_rate": 0.00019771111412967583,
281
- "loss": 0.8264,
282
- "step": 39
283
- },
284
- {
285
- "epoch": 0.27,
286
- "grad_norm": 0.20590244233608246,
287
- "learning_rate": 0.00019755119679804367,
288
- "loss": 0.8455,
289
- "step": 40
290
- },
291
- {
292
- "epoch": 0.28,
293
- "grad_norm": 0.20340456068515778,
294
- "learning_rate": 0.00019738594915589397,
295
- "loss": 0.871,
296
- "step": 41
297
- },
298
- {
299
- "epoch": 0.29,
300
- "grad_norm": 0.2383277714252472,
301
- "learning_rate": 0.0001972153802325495,
302
- "loss": 0.6833,
303
- "step": 42
304
- },
305
- {
306
- "epoch": 0.3,
307
- "grad_norm": 0.21479888260364532,
308
- "learning_rate": 0.00019703949934809408,
309
- "loss": 0.8355,
310
- "step": 43
311
- },
312
- {
313
- "epoch": 0.3,
314
- "grad_norm": 0.23538872599601746,
315
- "learning_rate": 0.0001968583161128631,
316
- "loss": 0.8479,
317
- "step": 44
318
- },
319
- {
320
- "epoch": 0.31,
321
- "grad_norm": 0.2260449230670929,
322
- "learning_rate": 0.00019667184042691875,
323
- "loss": 0.779,
324
- "step": 45
325
- },
326
- {
327
- "epoch": 0.32,
328
- "grad_norm": 0.20232993364334106,
329
- "learning_rate": 0.0001964800824795087,
330
- "loss": 0.7293,
331
- "step": 46
332
- },
333
- {
334
- "epoch": 0.32,
335
- "grad_norm": 0.19780154526233673,
336
- "learning_rate": 0.00019628305274850956,
337
- "loss": 0.7709,
338
- "step": 47
339
- },
340
- {
341
- "epoch": 0.33,
342
- "grad_norm": 0.24822202324867249,
343
- "learning_rate": 0.00019608076199985433,
344
- "loss": 0.8885,
345
- "step": 48
346
- },
347
- {
348
- "epoch": 0.34,
349
- "grad_norm": 0.20940056443214417,
350
- "learning_rate": 0.0001958732212869441,
351
- "loss": 0.6916,
352
- "step": 49
353
- },
354
- {
355
- "epoch": 0.34,
356
- "grad_norm": 0.18944858014583588,
357
- "learning_rate": 0.0001956604419500441,
358
- "loss": 0.7326,
359
- "step": 50
360
- },
361
- {
362
- "epoch": 0.35,
363
- "grad_norm": 0.201398104429245,
364
- "learning_rate": 0.00019544243561566403,
365
- "loss": 0.6542,
366
- "step": 51
367
- },
368
- {
369
- "epoch": 0.36,
370
- "grad_norm": 0.22385160624980927,
371
- "learning_rate": 0.00019521921419592283,
372
- "loss": 0.8036,
373
- "step": 52
374
- },
375
- {
376
- "epoch": 0.36,
377
- "grad_norm": 0.19502398371696472,
378
- "learning_rate": 0.0001949907898878977,
379
- "loss": 0.7579,
380
- "step": 53
381
- },
382
- {
383
- "epoch": 0.37,
384
- "grad_norm": 0.21633867919445038,
385
- "learning_rate": 0.00019475717517295778,
386
- "loss": 0.7803,
387
- "step": 54
388
- },
389
- {
390
- "epoch": 0.38,
391
- "grad_norm": 0.24109330773353577,
392
- "learning_rate": 0.00019451838281608197,
393
- "loss": 0.7745,
394
- "step": 55
395
- },
396
- {
397
- "epoch": 0.38,
398
- "grad_norm": 0.20285621285438538,
399
- "learning_rate": 0.00019427442586516155,
400
- "loss": 0.7306,
401
- "step": 56
402
- },
403
- {
404
- "epoch": 0.39,
405
- "grad_norm": 0.23838144540786743,
406
- "learning_rate": 0.00019402531765028722,
407
- "loss": 0.7606,
408
- "step": 57
409
- },
410
- {
411
- "epoch": 0.4,
412
- "grad_norm": 0.25280559062957764,
413
- "learning_rate": 0.00019377107178302074,
414
- "loss": 0.7928,
415
- "step": 58
416
- },
417
- {
418
- "epoch": 0.41,
419
- "grad_norm": 0.24614648520946503,
420
- "learning_rate": 0.00019351170215565114,
421
- "loss": 0.739,
422
- "step": 59
423
- },
424
- {
425
- "epoch": 0.41,
426
- "grad_norm": 0.20391589403152466,
427
- "learning_rate": 0.00019324722294043558,
428
- "loss": 0.7636,
429
- "step": 60
430
- },
431
- {
432
- "epoch": 0.42,
433
- "grad_norm": 0.23704086244106293,
434
- "learning_rate": 0.00019297764858882514,
435
- "loss": 0.6919,
436
- "step": 61
437
- },
438
- {
439
- "epoch": 0.43,
440
- "grad_norm": 0.22057901322841644,
441
- "learning_rate": 0.00019270299383067498,
442
- "loss": 0.6035,
443
- "step": 62
444
- },
445
- {
446
- "epoch": 0.43,
447
- "grad_norm": 0.22797846794128418,
448
- "learning_rate": 0.0001924232736734396,
449
- "loss": 0.7087,
450
- "step": 63
451
- },
452
- {
453
- "epoch": 0.44,
454
- "grad_norm": 0.22181542217731476,
455
- "learning_rate": 0.00019213850340135276,
456
- "loss": 0.8088,
457
- "step": 64
458
- },
459
- {
460
- "epoch": 0.45,
461
- "grad_norm": 0.2096872180700302,
462
- "learning_rate": 0.00019184869857459232,
463
- "loss": 0.7073,
464
- "step": 65
465
- },
466
- {
467
- "epoch": 0.45,
468
- "grad_norm": 0.192199245095253,
469
- "learning_rate": 0.00019155387502843013,
470
- "loss": 0.6605,
471
- "step": 66
472
- },
473
- {
474
- "epoch": 0.46,
475
- "grad_norm": 0.2242266982793808,
476
- "learning_rate": 0.00019125404887236663,
477
- "loss": 0.7287,
478
- "step": 67
479
- },
480
- {
481
- "epoch": 0.47,
482
- "grad_norm": 0.24139907956123352,
483
- "learning_rate": 0.00019094923648925067,
484
- "loss": 0.6611,
485
- "step": 68
486
- },
487
- {
488
- "epoch": 0.47,
489
- "grad_norm": 0.22859904170036316,
490
- "learning_rate": 0.00019063945453438432,
491
- "loss": 0.6189,
492
- "step": 69
493
- },
494
- {
495
- "epoch": 0.48,
496
- "grad_norm": 0.27671098709106445,
497
- "learning_rate": 0.0001903247199346129,
498
- "loss": 0.8595,
499
- "step": 70
500
- },
501
- {
502
- "epoch": 0.49,
503
- "grad_norm": 0.23342834413051605,
504
- "learning_rate": 0.00019000504988739986,
505
- "loss": 0.6718,
506
- "step": 71
507
- },
508
- {
509
- "epoch": 0.49,
510
- "grad_norm": 0.2517896890640259,
511
- "learning_rate": 0.00018968046185988732,
512
- "loss": 0.722,
513
- "step": 72
514
- },
515
- {
516
- "epoch": 0.5,
517
- "grad_norm": 0.25217679142951965,
518
- "learning_rate": 0.00018935097358794144,
519
- "loss": 0.824,
520
- "step": 73
521
- },
522
- {
523
- "epoch": 0.51,
524
- "grad_norm": 0.2682301998138428,
525
- "learning_rate": 0.00018901660307518354,
526
- "loss": 0.7837,
527
- "step": 74
528
- },
529
- {
530
- "epoch": 0.52,
531
- "grad_norm": 0.28926676511764526,
532
- "learning_rate": 0.0001886773685920062,
533
- "loss": 0.7364,
534
- "step": 75
535
- },
536
- {
537
- "epoch": 0.52,
538
- "grad_norm": 0.25249186158180237,
539
- "learning_rate": 0.00018833328867457497,
540
- "loss": 0.8566,
541
- "step": 76
542
- },
543
- {
544
- "epoch": 0.53,
545
- "grad_norm": 0.21752771735191345,
546
- "learning_rate": 0.0001879843821238155,
547
- "loss": 0.7077,
548
- "step": 77
549
- },
550
- {
551
- "epoch": 0.54,
552
- "grad_norm": 0.22346794605255127,
553
- "learning_rate": 0.00018763066800438636,
554
- "loss": 0.6522,
555
- "step": 78
556
- },
557
- {
558
- "epoch": 0.54,
559
- "grad_norm": 0.23172937333583832,
560
- "learning_rate": 0.00018727216564363723,
561
- "loss": 0.6099,
562
- "step": 79
563
- },
564
- {
565
- "epoch": 0.55,
566
- "grad_norm": 0.20895916223526,
567
- "learning_rate": 0.00018690889463055283,
568
- "loss": 0.7242,
569
- "step": 80
570
- },
571
- {
572
- "epoch": 0.56,
573
- "grad_norm": 0.26868510246276855,
574
- "learning_rate": 0.0001865408748146826,
575
- "loss": 0.705,
576
- "step": 81
577
- },
578
- {
579
- "epoch": 0.56,
580
- "grad_norm": 0.243666410446167,
581
- "learning_rate": 0.00018616812630505597,
582
- "loss": 0.8512,
583
- "step": 82
584
- },
585
- {
586
- "epoch": 0.57,
587
- "grad_norm": 0.21445880830287933,
588
- "learning_rate": 0.00018579066946908384,
589
- "loss": 0.7092,
590
- "step": 83
591
- },
592
- {
593
- "epoch": 0.58,
594
- "grad_norm": 0.22805824875831604,
595
- "learning_rate": 0.00018540852493144545,
596
- "loss": 0.6757,
597
- "step": 84
598
- },
599
- {
600
- "epoch": 0.58,
601
- "grad_norm": 0.25106164813041687,
602
- "learning_rate": 0.00018502171357296144,
603
- "loss": 0.7171,
604
- "step": 85
605
- },
606
- {
607
- "epoch": 0.59,
608
- "grad_norm": 0.2290162742137909,
609
- "learning_rate": 0.000184630256529453,
610
- "loss": 0.7053,
611
- "step": 86
612
- },
613
- {
614
- "epoch": 0.6,
615
- "grad_norm": 0.21118655800819397,
616
- "learning_rate": 0.00018423417519058694,
617
- "loss": 0.5827,
618
- "step": 87
619
- },
620
- {
621
- "epoch": 0.6,
622
- "grad_norm": 0.23761862516403198,
623
- "learning_rate": 0.00018383349119870695,
624
- "loss": 0.5827,
625
- "step": 88
626
- },
627
- {
628
- "epoch": 0.61,
629
- "grad_norm": 0.23689566552639008,
630
- "learning_rate": 0.00018342822644765104,
631
- "loss": 0.6401,
632
- "step": 89
633
- },
634
- {
635
- "epoch": 0.62,
636
- "grad_norm": 0.26510947942733765,
637
- "learning_rate": 0.00018301840308155507,
638
- "loss": 0.5963,
639
- "step": 90
640
- },
641
- {
642
- "epoch": 0.63,
643
- "grad_norm": 0.2142990678548813,
644
- "learning_rate": 0.0001826040434936431,
645
- "loss": 0.6258,
646
- "step": 91
647
- },
648
- {
649
- "epoch": 0.63,
650
- "grad_norm": 0.20709159970283508,
651
- "learning_rate": 0.00018218517032500344,
652
- "loss": 0.5685,
653
- "step": 92
654
- },
655
- {
656
- "epoch": 0.64,
657
- "grad_norm": 0.2485310137271881,
658
- "learning_rate": 0.0001817618064633518,
659
- "loss": 0.5901,
660
- "step": 93
661
- },
662
- {
663
- "epoch": 0.65,
664
- "grad_norm": 0.2432468980550766,
665
- "learning_rate": 0.00018133397504178057,
666
- "loss": 0.6891,
667
- "step": 94
668
- },
669
- {
670
- "epoch": 0.65,
671
- "grad_norm": 0.2318205088376999,
672
- "learning_rate": 0.00018090169943749476,
673
- "loss": 0.7339,
674
- "step": 95
675
- },
676
- {
677
- "epoch": 0.66,
678
- "grad_norm": 0.25346460938453674,
679
- "learning_rate": 0.00018046500327053463,
680
- "loss": 0.6606,
681
- "step": 96
682
- },
683
- {
684
- "epoch": 0.67,
685
- "grad_norm": 0.29421111941337585,
686
- "learning_rate": 0.0001800239104024851,
687
- "loss": 0.6698,
688
- "step": 97
689
- },
690
- {
691
- "epoch": 0.67,
692
- "grad_norm": 0.2356603443622589,
693
- "learning_rate": 0.00017957844493517213,
694
- "loss": 0.6225,
695
- "step": 98
696
- },
697
- {
698
- "epoch": 0.68,
699
- "grad_norm": 0.2331649512052536,
700
- "learning_rate": 0.00017912863120934534,
701
- "loss": 0.6838,
702
- "step": 99
703
- },
704
- {
705
- "epoch": 0.69,
706
- "grad_norm": 0.23008978366851807,
707
- "learning_rate": 0.00017867449380334834,
708
- "loss": 0.6303,
709
- "step": 100
710
- },
711
- {
712
- "epoch": 0.69,
713
- "grad_norm": 0.21604081988334656,
714
- "learning_rate": 0.00017821605753177562,
715
- "loss": 0.4923,
716
- "step": 101
717
- },
718
- {
719
- "epoch": 0.7,
720
- "grad_norm": 0.2283111810684204,
721
- "learning_rate": 0.00017775334744411678,
722
- "loss": 0.6344,
723
- "step": 102
724
- },
725
- {
726
- "epoch": 0.71,
727
- "grad_norm": 0.2448730617761612,
728
- "learning_rate": 0.00017728638882338746,
729
- "loss": 0.6789,
730
- "step": 103
731
- },
732
- {
733
- "epoch": 0.71,
734
- "grad_norm": 0.22479577362537384,
735
- "learning_rate": 0.00017681520718474823,
736
- "loss": 0.5841,
737
- "step": 104
738
- },
739
- {
740
- "epoch": 0.72,
741
- "grad_norm": 0.21999216079711914,
742
- "learning_rate": 0.00017633982827411032,
743
- "loss": 0.6402,
744
- "step": 105
745
- },
746
- {
747
- "epoch": 0.73,
748
- "grad_norm": 0.2296195775270462,
749
- "learning_rate": 0.00017586027806672857,
750
- "loss": 0.6148,
751
- "step": 106
752
- },
753
- {
754
- "epoch": 0.74,
755
- "grad_norm": 0.22455835342407227,
756
- "learning_rate": 0.00017537658276578247,
757
- "loss": 0.6418,
758
- "step": 107
759
- },
760
- {
761
- "epoch": 0.74,
762
- "grad_norm": 0.23929741978645325,
763
- "learning_rate": 0.00017488876880094413,
764
- "loss": 0.5459,
765
- "step": 108
766
- },
767
- {
768
- "epoch": 0.75,
769
- "grad_norm": 0.229934424161911,
770
- "learning_rate": 0.00017439686282693436,
771
- "loss": 0.6049,
772
- "step": 109
773
- },
774
- {
775
- "epoch": 0.76,
776
- "grad_norm": 0.24801373481750488,
777
- "learning_rate": 0.00017390089172206592,
778
- "loss": 0.7663,
779
- "step": 110
780
- },
781
- {
782
- "epoch": 0.76,
783
- "grad_norm": 0.2523267865180969,
784
- "learning_rate": 0.00017340088258677522,
785
- "loss": 0.6642,
786
- "step": 111
787
- },
788
- {
789
- "epoch": 0.77,
790
- "grad_norm": 0.22252008318901062,
791
- "learning_rate": 0.00017289686274214118,
792
- "loss": 0.6325,
793
- "step": 112
794
- },
795
- {
796
- "epoch": 0.78,
797
- "grad_norm": 0.27619731426239014,
798
- "learning_rate": 0.0001723888597283926,
799
- "loss": 0.655,
800
- "step": 113
801
- },
802
- {
803
- "epoch": 0.78,
804
- "grad_norm": 0.25090277194976807,
805
- "learning_rate": 0.00017187690130340328,
806
- "loss": 0.6251,
807
- "step": 114
808
- },
809
- {
810
- "epoch": 0.79,
811
- "grad_norm": 0.2236175239086151,
812
- "learning_rate": 0.00017136101544117525,
813
- "loss": 0.5959,
814
- "step": 115
815
- },
816
- {
817
- "epoch": 0.8,
818
- "grad_norm": 0.21940240263938904,
819
- "learning_rate": 0.00017084123033031024,
820
- "loss": 0.6028,
821
- "step": 116
822
- },
823
- {
824
- "epoch": 0.8,
825
- "grad_norm": 0.2741597294807434,
826
- "learning_rate": 0.00017031757437246947,
827
- "loss": 0.4947,
828
- "step": 117
829
- },
830
- {
831
- "epoch": 0.81,
832
- "grad_norm": 0.23343035578727722,
833
- "learning_rate": 0.00016979007618082175,
834
- "loss": 0.4415,
835
- "step": 118
836
- },
837
- {
838
- "epoch": 0.82,
839
- "grad_norm": 0.2116251140832901,
840
- "learning_rate": 0.00016925876457848,
841
- "loss": 0.5283,
842
- "step": 119
843
- },
844
- {
845
- "epoch": 0.82,
846
- "grad_norm": 0.25621068477630615,
847
- "learning_rate": 0.00016872366859692627,
848
- "loss": 0.7334,
849
- "step": 120
850
- },
851
- {
852
- "epoch": 0.83,
853
- "grad_norm": 0.22588759660720825,
854
- "learning_rate": 0.00016818481747442554,
855
- "loss": 0.6514,
856
- "step": 121
857
- },
858
- {
859
- "epoch": 0.84,
860
- "grad_norm": 0.24995574355125427,
861
- "learning_rate": 0.00016764224065442796,
862
- "loss": 0.5632,
863
- "step": 122
864
- },
865
- {
866
- "epoch": 0.85,
867
- "grad_norm": 0.2382134348154068,
868
- "learning_rate": 0.00016709596778396026,
869
- "loss": 0.5954,
870
- "step": 123
871
- },
872
- {
873
- "epoch": 0.85,
874
- "grad_norm": 0.25166618824005127,
875
- "learning_rate": 0.00016654602871200546,
876
- "loss": 0.6092,
877
- "step": 124
878
- },
879
- {
880
- "epoch": 0.86,
881
- "grad_norm": 0.23028074204921722,
882
- "learning_rate": 0.0001659924534878723,
883
- "loss": 0.576,
884
- "step": 125
885
- },
886
- {
887
- "epoch": 0.87,
888
- "grad_norm": 0.2527604103088379,
889
- "learning_rate": 0.00016543527235955282,
890
- "loss": 0.5985,
891
- "step": 126
892
- },
893
- {
894
- "epoch": 0.87,
895
- "grad_norm": 0.25870686769485474,
896
- "learning_rate": 0.00016487451577207018,
897
- "loss": 0.5767,
898
- "step": 127
899
- },
900
- {
901
- "epoch": 0.88,
902
- "grad_norm": 0.24936434626579285,
903
- "learning_rate": 0.0001643102143658145,
904
- "loss": 0.7494,
905
- "step": 128
906
- },
907
- {
908
- "epoch": 0.89,
909
- "grad_norm": 0.25158020853996277,
910
- "learning_rate": 0.000163742398974869,
911
- "loss": 0.578,
912
- "step": 129
913
- },
914
- {
915
- "epoch": 0.89,
916
- "grad_norm": 0.22290346026420593,
917
- "learning_rate": 0.0001631711006253251,
918
- "loss": 0.6489,
919
- "step": 130
920
- },
921
- {
922
- "epoch": 0.9,
923
- "grad_norm": 0.26421919465065,
924
- "learning_rate": 0.00016259635053358717,
925
- "loss": 0.5669,
926
- "step": 131
927
- },
928
- {
929
- "epoch": 0.91,
930
- "grad_norm": 0.2573990225791931,
931
- "learning_rate": 0.0001620181801046667,
932
- "loss": 0.6405,
933
- "step": 132
934
- },
935
- {
936
- "epoch": 0.91,
937
- "grad_norm": 0.19883453845977783,
938
- "learning_rate": 0.00016143662093046638,
939
- "loss": 0.4731,
940
- "step": 133
941
- },
942
- {
943
- "epoch": 0.92,
944
- "grad_norm": 0.24181459844112396,
945
- "learning_rate": 0.00016085170478805395,
946
- "loss": 0.6819,
947
- "step": 134
948
- },
949
- {
950
- "epoch": 0.93,
951
- "grad_norm": 0.2319963425397873,
952
- "learning_rate": 0.00016026346363792567,
953
- "loss": 0.6373,
954
- "step": 135
955
- },
956
- {
957
- "epoch": 0.93,
958
- "grad_norm": 0.2552095055580139,
959
- "learning_rate": 0.0001596719296222601,
960
- "loss": 0.7151,
961
- "step": 136
962
- },
963
- {
964
- "epoch": 0.94,
965
- "grad_norm": 0.27191445231437683,
966
- "learning_rate": 0.00015907713506316192,
967
- "loss": 0.611,
968
- "step": 137
969
- },
970
- {
971
- "epoch": 0.95,
972
- "grad_norm": 0.26981061697006226,
973
- "learning_rate": 0.0001584791124608955,
974
- "loss": 0.5923,
975
- "step": 138
976
- },
977
- {
978
- "epoch": 0.96,
979
- "grad_norm": 0.23907916247844696,
980
- "learning_rate": 0.00015787789449210938,
981
- "loss": 0.5655,
982
- "step": 139
983
- },
984
- {
985
- "epoch": 0.96,
986
- "grad_norm": 0.23364509642124176,
987
- "learning_rate": 0.00015727351400805052,
988
- "loss": 0.6019,
989
- "step": 140
990
- },
991
- {
992
- "epoch": 0.97,
993
- "grad_norm": 0.23096178472042084,
994
- "learning_rate": 0.0001566660040327695,
995
- "loss": 0.602,
996
- "step": 141
997
- },
998
- {
999
- "epoch": 0.98,
1000
- "grad_norm": 0.2656322717666626,
1001
- "learning_rate": 0.0001560553977613158,
1002
- "loss": 0.6195,
1003
- "step": 142
1004
- },
1005
- {
1006
- "epoch": 0.98,
1007
- "grad_norm": 0.23051652312278748,
1008
- "learning_rate": 0.00015544172855792423,
1009
- "loss": 0.6132,
1010
- "step": 143
1011
- },
1012
- {
1013
- "epoch": 0.99,
1014
- "grad_norm": 0.2443581074476242,
1015
- "learning_rate": 0.00015482502995419167,
1016
- "loss": 0.6144,
1017
- "step": 144
1018
- },
1019
- {
1020
- "epoch": 1.0,
1021
- "grad_norm": 0.2075706273317337,
1022
- "learning_rate": 0.00015420533564724495,
1023
- "loss": 0.5556,
1024
- "step": 145
1025
- }
1026
- ],
1027
- "logging_steps": 1,
1028
- "max_steps": 435,
1029
- "num_input_tokens_seen": 0,
1030
- "num_train_epochs": 3,
1031
- "save_steps": 145,
1032
- "total_flos": 9.76361694953472e+16,
1033
- "train_batch_size": 8,
1034
- "trial_name": null,
1035
- "trial_params": null
1036
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
checkpoint-145/training_args.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:e0f4e74f968ef592cbe5e646087e575fbeada43b9dc36d459d99d6a9ebc528b7
3
- size 5560
 
 
 
 
checkpoint-290/README.md DELETED
@@ -1,202 +0,0 @@
1
- ---
2
- library_name: peft
3
- base_model: HuggingFaceTB/cosmo-1b
4
- ---
5
-
6
- # Model Card for Model ID
7
-
8
- <!-- Provide a quick summary of what the model is/does. -->
9
-
10
-
11
-
12
- ## Model Details
13
-
14
- ### Model Description
15
-
16
- <!-- Provide a longer summary of what this model is. -->
17
-
18
-
19
-
20
- - **Developed by:** [More Information Needed]
21
- - **Funded by [optional]:** [More Information Needed]
22
- - **Shared by [optional]:** [More Information Needed]
23
- - **Model type:** [More Information Needed]
24
- - **Language(s) (NLP):** [More Information Needed]
25
- - **License:** [More Information Needed]
26
- - **Finetuned from model [optional]:** [More Information Needed]
27
-
28
- ### Model Sources [optional]
29
-
30
- <!-- Provide the basic links for the model. -->
31
-
32
- - **Repository:** [More Information Needed]
33
- - **Paper [optional]:** [More Information Needed]
34
- - **Demo [optional]:** [More Information Needed]
35
-
36
- ## Uses
37
-
38
- <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
39
-
40
- ### Direct Use
41
-
42
- <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
43
-
44
- [More Information Needed]
45
-
46
- ### Downstream Use [optional]
47
-
48
- <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
49
-
50
- [More Information Needed]
51
-
52
- ### Out-of-Scope Use
53
-
54
- <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
55
-
56
- [More Information Needed]
57
-
58
- ## Bias, Risks, and Limitations
59
-
60
- <!-- This section is meant to convey both technical and sociotechnical limitations. -->
61
-
62
- [More Information Needed]
63
-
64
- ### Recommendations
65
-
66
- <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
67
-
68
- Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
69
-
70
- ## How to Get Started with the Model
71
-
72
- Use the code below to get started with the model.
73
-
74
- [More Information Needed]
75
-
76
- ## Training Details
77
-
78
- ### Training Data
79
-
80
- <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
81
-
82
- [More Information Needed]
83
-
84
- ### Training Procedure
85
-
86
- <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
87
-
88
- #### Preprocessing [optional]
89
-
90
- [More Information Needed]
91
-
92
-
93
- #### Training Hyperparameters
94
-
95
- - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
96
-
97
- #### Speeds, Sizes, Times [optional]
98
-
99
- <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
100
-
101
- [More Information Needed]
102
-
103
- ## Evaluation
104
-
105
- <!-- This section describes the evaluation protocols and provides the results. -->
106
-
107
- ### Testing Data, Factors & Metrics
108
-
109
- #### Testing Data
110
-
111
- <!-- This should link to a Dataset Card if possible. -->
112
-
113
- [More Information Needed]
114
-
115
- #### Factors
116
-
117
- <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
118
-
119
- [More Information Needed]
120
-
121
- #### Metrics
122
-
123
- <!-- These are the evaluation metrics being used, ideally with a description of why. -->
124
-
125
- [More Information Needed]
126
-
127
- ### Results
128
-
129
- [More Information Needed]
130
-
131
- #### Summary
132
-
133
-
134
-
135
- ## Model Examination [optional]
136
-
137
- <!-- Relevant interpretability work for the model goes here -->
138
-
139
- [More Information Needed]
140
-
141
- ## Environmental Impact
142
-
143
- <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
144
-
145
- Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
146
-
147
- - **Hardware Type:** [More Information Needed]
148
- - **Hours used:** [More Information Needed]
149
- - **Cloud Provider:** [More Information Needed]
150
- - **Compute Region:** [More Information Needed]
151
- - **Carbon Emitted:** [More Information Needed]
152
-
153
- ## Technical Specifications [optional]
154
-
155
- ### Model Architecture and Objective
156
-
157
- [More Information Needed]
158
-
159
- ### Compute Infrastructure
160
-
161
- [More Information Needed]
162
-
163
- #### Hardware
164
-
165
- [More Information Needed]
166
-
167
- #### Software
168
-
169
- [More Information Needed]
170
-
171
- ## Citation [optional]
172
-
173
- <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
174
-
175
- **BibTeX:**
176
-
177
- [More Information Needed]
178
-
179
- **APA:**
180
-
181
- [More Information Needed]
182
-
183
- ## Glossary [optional]
184
-
185
- <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
186
-
187
- [More Information Needed]
188
-
189
- ## More Information [optional]
190
-
191
- [More Information Needed]
192
-
193
- ## Model Card Authors [optional]
194
-
195
- [More Information Needed]
196
-
197
- ## Model Card Contact
198
-
199
- [More Information Needed]
200
- ### Framework versions
201
-
202
- - PEFT 0.9.1.dev0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
checkpoint-290/adapter_config.json DELETED
@@ -1,33 +0,0 @@
1
- {
2
- "alpha_pattern": {},
3
- "auto_mapping": null,
4
- "base_model_name_or_path": "HuggingFaceTB/cosmo-1b",
5
- "bias": "none",
6
- "fan_in_fan_out": null,
7
- "inference_mode": true,
8
- "init_lora_weights": true,
9
- "layers_pattern": null,
10
- "layers_to_transform": null,
11
- "loftq_config": {},
12
- "lora_alpha": 32,
13
- "lora_dropout": 0.1,
14
- "megatron_config": null,
15
- "megatron_core": "megatron.core",
16
- "modules_to_save": null,
17
- "peft_type": "LORA",
18
- "r": 32,
19
- "rank_pattern": {},
20
- "revision": null,
21
- "target_modules": [
22
- "v_proj",
23
- "k_proj",
24
- "o_proj",
25
- "up_proj",
26
- "down_proj",
27
- "gate_proj",
28
- "q_proj"
29
- ],
30
- "task_type": "CAUSAL_LM",
31
- "use_dora": false,
32
- "use_rslora": false
33
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
checkpoint-290/adapter_model.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:c3c3beef805b9b3a73aa826333b30c1f8b651be54657b72a6274b31ac5be49be
3
- size 144748392
 
 
 
 
checkpoint-290/optimizer.pt DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:f46f8a2c6fccf8486194b3dca83a0baaec044d0dba76b81f549546e0c11df636
3
- size 72887380
 
 
 
 
checkpoint-290/rng_state.pth DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:c8b7fb51585bedb5cfc596fed320770581a502ad993a98decb3f0b397ca635e8
3
- size 14244
 
 
 
 
checkpoint-290/scheduler.pt DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:6a977aa284b2579061f2f683a13b0da8d3cef821bd533e41970eab743abe4618
3
- size 1064
 
 
 
 
checkpoint-290/trainer_state.json DELETED
@@ -1,2059 +0,0 @@
1
- {
2
- "best_metric": null,
3
- "best_model_checkpoint": null,
4
- "epoch": 1.9742268041237114,
5
- "eval_steps": 500,
6
- "global_step": 290,
7
- "is_hyper_param_search": false,
8
- "is_local_process_zero": true,
9
- "is_world_process_zero": true,
10
- "log_history": [
11
- {
12
- "epoch": 0.01,
13
- "grad_norm": 0.37809523940086365,
14
- "learning_rate": 2e-05,
15
- "loss": 1.7814,
16
- "step": 1
17
- },
18
- {
19
- "epoch": 0.01,
20
- "grad_norm": 0.366623193025589,
21
- "learning_rate": 4e-05,
22
- "loss": 1.69,
23
- "step": 2
24
- },
25
- {
26
- "epoch": 0.02,
27
- "grad_norm": 0.3546508848667145,
28
- "learning_rate": 6e-05,
29
- "loss": 1.7151,
30
- "step": 3
31
- },
32
- {
33
- "epoch": 0.03,
34
- "grad_norm": 0.36381417512893677,
35
- "learning_rate": 8e-05,
36
- "loss": 1.6609,
37
- "step": 4
38
- },
39
- {
40
- "epoch": 0.03,
41
- "grad_norm": 0.4049898684024811,
42
- "learning_rate": 0.0001,
43
- "loss": 1.752,
44
- "step": 5
45
- },
46
- {
47
- "epoch": 0.04,
48
- "grad_norm": 0.4318874478340149,
49
- "learning_rate": 0.00012,
50
- "loss": 1.747,
51
- "step": 6
52
- },
53
- {
54
- "epoch": 0.05,
55
- "grad_norm": 0.39768990874290466,
56
- "learning_rate": 0.00014,
57
- "loss": 1.4432,
58
- "step": 7
59
- },
60
- {
61
- "epoch": 0.05,
62
- "grad_norm": 0.5056629180908203,
63
- "learning_rate": 0.00016,
64
- "loss": 1.7657,
65
- "step": 8
66
- },
67
- {
68
- "epoch": 0.06,
69
- "grad_norm": 0.4276069104671478,
70
- "learning_rate": 0.00018,
71
- "loss": 1.4221,
72
- "step": 9
73
- },
74
- {
75
- "epoch": 0.07,
76
- "grad_norm": 0.433552622795105,
77
- "learning_rate": 0.0002,
78
- "loss": 1.3117,
79
- "step": 10
80
- },
81
- {
82
- "epoch": 0.08,
83
- "grad_norm": 0.42256900668144226,
84
- "learning_rate": 0.0001999972679420179,
85
- "loss": 1.2687,
86
- "step": 11
87
- },
88
- {
89
- "epoch": 0.08,
90
- "grad_norm": 0.3066452145576477,
91
- "learning_rate": 0.00019998907191735434,
92
- "loss": 1.0398,
93
- "step": 12
94
- },
95
- {
96
- "epoch": 0.09,
97
- "grad_norm": 0.3393724858760834,
98
- "learning_rate": 0.00019997541237384966,
99
- "loss": 1.0063,
100
- "step": 13
101
- },
102
- {
103
- "epoch": 0.1,
104
- "grad_norm": 0.49836328625679016,
105
- "learning_rate": 0.00019995629005787713,
106
- "loss": 1.2251,
107
- "step": 14
108
- },
109
- {
110
- "epoch": 0.1,
111
- "grad_norm": 0.3539387285709381,
112
- "learning_rate": 0.0001999317060143023,
113
- "loss": 0.9454,
114
- "step": 15
115
- },
116
- {
117
- "epoch": 0.11,
118
- "grad_norm": 0.35584160685539246,
119
- "learning_rate": 0.0001999016615864258,
120
- "loss": 1.0105,
121
- "step": 16
122
- },
123
- {
124
- "epoch": 0.12,
125
- "grad_norm": 0.33191052079200745,
126
- "learning_rate": 0.00019986615841591002,
127
- "loss": 1.1322,
128
- "step": 17
129
- },
130
- {
131
- "epoch": 0.12,
132
- "grad_norm": 0.3499913513660431,
133
- "learning_rate": 0.00019982519844268933,
134
- "loss": 0.8495,
135
- "step": 18
136
- },
137
- {
138
- "epoch": 0.13,
139
- "grad_norm": 0.2582085132598877,
140
- "learning_rate": 0.0001997787839048642,
141
- "loss": 0.9725,
142
- "step": 19
143
- },
144
- {
145
- "epoch": 0.14,
146
- "grad_norm": 0.23511624336242676,
147
- "learning_rate": 0.00019972691733857883,
148
- "loss": 0.9233,
149
- "step": 20
150
- },
151
- {
152
- "epoch": 0.14,
153
- "grad_norm": 0.273019939661026,
154
- "learning_rate": 0.00019966960157788248,
155
- "loss": 1.0366,
156
- "step": 21
157
- },
158
- {
159
- "epoch": 0.15,
160
- "grad_norm": 0.283527135848999,
161
- "learning_rate": 0.0001996068397545748,
162
- "loss": 0.9944,
163
- "step": 22
164
- },
165
- {
166
- "epoch": 0.16,
167
- "grad_norm": 0.24311360716819763,
168
- "learning_rate": 0.00019953863529803466,
169
- "loss": 1.0308,
170
- "step": 23
171
- },
172
- {
173
- "epoch": 0.16,
174
- "grad_norm": 0.24368223547935486,
175
- "learning_rate": 0.00019946499193503262,
176
- "loss": 1.0051,
177
- "step": 24
178
- },
179
- {
180
- "epoch": 0.17,
181
- "grad_norm": 0.2269805669784546,
182
- "learning_rate": 0.0001993859136895274,
183
- "loss": 0.835,
184
- "step": 25
185
- },
186
- {
187
- "epoch": 0.18,
188
- "grad_norm": 0.21915565431118011,
189
- "learning_rate": 0.00019930140488244602,
190
- "loss": 0.9997,
191
- "step": 26
192
- },
193
- {
194
- "epoch": 0.19,
195
- "grad_norm": 0.21885964274406433,
196
- "learning_rate": 0.0001992114701314478,
197
- "loss": 1.0013,
198
- "step": 27
199
- },
200
- {
201
- "epoch": 0.19,
202
- "grad_norm": 0.22426557540893555,
203
- "learning_rate": 0.00019911611435067172,
204
- "loss": 0.8633,
205
- "step": 28
206
- },
207
- {
208
- "epoch": 0.2,
209
- "grad_norm": 0.21587449312210083,
210
- "learning_rate": 0.0001990153427504683,
211
- "loss": 0.8479,
212
- "step": 29
213
- },
214
- {
215
- "epoch": 0.21,
216
- "grad_norm": 0.24985891580581665,
217
- "learning_rate": 0.0001989091608371146,
218
- "loss": 0.8936,
219
- "step": 30
220
- },
221
- {
222
- "epoch": 0.21,
223
- "grad_norm": 0.19733141362667084,
224
- "learning_rate": 0.0001987975744125135,
225
- "loss": 0.7743,
226
- "step": 31
227
- },
228
- {
229
- "epoch": 0.22,
230
- "grad_norm": 0.1997266411781311,
231
- "learning_rate": 0.00019868058957387663,
232
- "loss": 0.9078,
233
- "step": 32
234
- },
235
- {
236
- "epoch": 0.23,
237
- "grad_norm": 0.24114945530891418,
238
- "learning_rate": 0.00019855821271339125,
239
- "loss": 1.0413,
240
- "step": 33
241
- },
242
- {
243
- "epoch": 0.23,
244
- "grad_norm": 0.2427120953798294,
245
- "learning_rate": 0.00019843045051787096,
246
- "loss": 0.8308,
247
- "step": 34
248
- },
249
- {
250
- "epoch": 0.24,
251
- "grad_norm": 0.1968858540058136,
252
- "learning_rate": 0.0001982973099683902,
253
- "loss": 0.818,
254
- "step": 35
255
- },
256
- {
257
- "epoch": 0.25,
258
- "grad_norm": 0.2783421576023102,
259
- "learning_rate": 0.00019815879833990304,
260
- "loss": 0.8504,
261
- "step": 36
262
- },
263
- {
264
- "epoch": 0.25,
265
- "grad_norm": 0.23332269489765167,
266
- "learning_rate": 0.00019801492320084546,
267
- "loss": 0.9826,
268
- "step": 37
269
- },
270
- {
271
- "epoch": 0.26,
272
- "grad_norm": 0.22864645719528198,
273
- "learning_rate": 0.00019786569241272197,
274
- "loss": 0.8705,
275
- "step": 38
276
- },
277
- {
278
- "epoch": 0.27,
279
- "grad_norm": 0.20620585978031158,
280
- "learning_rate": 0.00019771111412967583,
281
- "loss": 0.8264,
282
- "step": 39
283
- },
284
- {
285
- "epoch": 0.27,
286
- "grad_norm": 0.20590244233608246,
287
- "learning_rate": 0.00019755119679804367,
288
- "loss": 0.8455,
289
- "step": 40
290
- },
291
- {
292
- "epoch": 0.28,
293
- "grad_norm": 0.20340456068515778,
294
- "learning_rate": 0.00019738594915589397,
295
- "loss": 0.871,
296
- "step": 41
297
- },
298
- {
299
- "epoch": 0.29,
300
- "grad_norm": 0.2383277714252472,
301
- "learning_rate": 0.0001972153802325495,
302
- "loss": 0.6833,
303
- "step": 42
304
- },
305
- {
306
- "epoch": 0.3,
307
- "grad_norm": 0.21479888260364532,
308
- "learning_rate": 0.00019703949934809408,
309
- "loss": 0.8355,
310
- "step": 43
311
- },
312
- {
313
- "epoch": 0.3,
314
- "grad_norm": 0.23538872599601746,
315
- "learning_rate": 0.0001968583161128631,
316
- "loss": 0.8479,
317
- "step": 44
318
- },
319
- {
320
- "epoch": 0.31,
321
- "grad_norm": 0.2260449230670929,
322
- "learning_rate": 0.00019667184042691875,
323
- "loss": 0.779,
324
- "step": 45
325
- },
326
- {
327
- "epoch": 0.32,
328
- "grad_norm": 0.20232993364334106,
329
- "learning_rate": 0.0001964800824795087,
330
- "loss": 0.7293,
331
- "step": 46
332
- },
333
- {
334
- "epoch": 0.32,
335
- "grad_norm": 0.19780154526233673,
336
- "learning_rate": 0.00019628305274850956,
337
- "loss": 0.7709,
338
- "step": 47
339
- },
340
- {
341
- "epoch": 0.33,
342
- "grad_norm": 0.24822202324867249,
343
- "learning_rate": 0.00019608076199985433,
344
- "loss": 0.8885,
345
- "step": 48
346
- },
347
- {
348
- "epoch": 0.34,
349
- "grad_norm": 0.20940056443214417,
350
- "learning_rate": 0.0001958732212869441,
351
- "loss": 0.6916,
352
- "step": 49
353
- },
354
- {
355
- "epoch": 0.34,
356
- "grad_norm": 0.18944858014583588,
357
- "learning_rate": 0.0001956604419500441,
358
- "loss": 0.7326,
359
- "step": 50
360
- },
361
- {
362
- "epoch": 0.35,
363
- "grad_norm": 0.201398104429245,
364
- "learning_rate": 0.00019544243561566403,
365
- "loss": 0.6542,
366
- "step": 51
367
- },
368
- {
369
- "epoch": 0.36,
370
- "grad_norm": 0.22385160624980927,
371
- "learning_rate": 0.00019521921419592283,
372
- "loss": 0.8036,
373
- "step": 52
374
- },
375
- {
376
- "epoch": 0.36,
377
- "grad_norm": 0.19502398371696472,
378
- "learning_rate": 0.0001949907898878977,
379
- "loss": 0.7579,
380
- "step": 53
381
- },
382
- {
383
- "epoch": 0.37,
384
- "grad_norm": 0.21633867919445038,
385
- "learning_rate": 0.00019475717517295778,
386
- "loss": 0.7803,
387
- "step": 54
388
- },
389
- {
390
- "epoch": 0.38,
391
- "grad_norm": 0.24109330773353577,
392
- "learning_rate": 0.00019451838281608197,
393
- "loss": 0.7745,
394
- "step": 55
395
- },
396
- {
397
- "epoch": 0.38,
398
- "grad_norm": 0.20285621285438538,
399
- "learning_rate": 0.00019427442586516155,
400
- "loss": 0.7306,
401
- "step": 56
402
- },
403
- {
404
- "epoch": 0.39,
405
- "grad_norm": 0.23838144540786743,
406
- "learning_rate": 0.00019402531765028722,
407
- "loss": 0.7606,
408
- "step": 57
409
- },
410
- {
411
- "epoch": 0.4,
412
- "grad_norm": 0.25280559062957764,
413
- "learning_rate": 0.00019377107178302074,
414
- "loss": 0.7928,
415
- "step": 58
416
- },
417
- {
418
- "epoch": 0.41,
419
- "grad_norm": 0.24614648520946503,
420
- "learning_rate": 0.00019351170215565114,
421
- "loss": 0.739,
422
- "step": 59
423
- },
424
- {
425
- "epoch": 0.41,
426
- "grad_norm": 0.20391589403152466,
427
- "learning_rate": 0.00019324722294043558,
428
- "loss": 0.7636,
429
- "step": 60
430
- },
431
- {
432
- "epoch": 0.42,
433
- "grad_norm": 0.23704086244106293,
434
- "learning_rate": 0.00019297764858882514,
435
- "loss": 0.6919,
436
- "step": 61
437
- },
438
- {
439
- "epoch": 0.43,
440
- "grad_norm": 0.22057901322841644,
441
- "learning_rate": 0.00019270299383067498,
442
- "loss": 0.6035,
443
- "step": 62
444
- },
445
- {
446
- "epoch": 0.43,
447
- "grad_norm": 0.22797846794128418,
448
- "learning_rate": 0.0001924232736734396,
449
- "loss": 0.7087,
450
- "step": 63
451
- },
452
- {
453
- "epoch": 0.44,
454
- "grad_norm": 0.22181542217731476,
455
- "learning_rate": 0.00019213850340135276,
456
- "loss": 0.8088,
457
- "step": 64
458
- },
459
- {
460
- "epoch": 0.45,
461
- "grad_norm": 0.2096872180700302,
462
- "learning_rate": 0.00019184869857459232,
463
- "loss": 0.7073,
464
- "step": 65
465
- },
466
- {
467
- "epoch": 0.45,
468
- "grad_norm": 0.192199245095253,
469
- "learning_rate": 0.00019155387502843013,
470
- "loss": 0.6605,
471
- "step": 66
472
- },
473
- {
474
- "epoch": 0.46,
475
- "grad_norm": 0.2242266982793808,
476
- "learning_rate": 0.00019125404887236663,
477
- "loss": 0.7287,
478
- "step": 67
479
- },
480
- {
481
- "epoch": 0.47,
482
- "grad_norm": 0.24139907956123352,
483
- "learning_rate": 0.00019094923648925067,
484
- "loss": 0.6611,
485
- "step": 68
486
- },
487
- {
488
- "epoch": 0.47,
489
- "grad_norm": 0.22859904170036316,
490
- "learning_rate": 0.00019063945453438432,
491
- "loss": 0.6189,
492
- "step": 69
493
- },
494
- {
495
- "epoch": 0.48,
496
- "grad_norm": 0.27671098709106445,
497
- "learning_rate": 0.0001903247199346129,
498
- "loss": 0.8595,
499
- "step": 70
500
- },
501
- {
502
- "epoch": 0.49,
503
- "grad_norm": 0.23342834413051605,
504
- "learning_rate": 0.00019000504988739986,
505
- "loss": 0.6718,
506
- "step": 71
507
- },
508
- {
509
- "epoch": 0.49,
510
- "grad_norm": 0.2517896890640259,
511
- "learning_rate": 0.00018968046185988732,
512
- "loss": 0.722,
513
- "step": 72
514
- },
515
- {
516
- "epoch": 0.5,
517
- "grad_norm": 0.25217679142951965,
518
- "learning_rate": 0.00018935097358794144,
519
- "loss": 0.824,
520
- "step": 73
521
- },
522
- {
523
- "epoch": 0.51,
524
- "grad_norm": 0.2682301998138428,
525
- "learning_rate": 0.00018901660307518354,
526
- "loss": 0.7837,
527
- "step": 74
528
- },
529
- {
530
- "epoch": 0.52,
531
- "grad_norm": 0.28926676511764526,
532
- "learning_rate": 0.0001886773685920062,
533
- "loss": 0.7364,
534
- "step": 75
535
- },
536
- {
537
- "epoch": 0.52,
538
- "grad_norm": 0.25249186158180237,
539
- "learning_rate": 0.00018833328867457497,
540
- "loss": 0.8566,
541
- "step": 76
542
- },
543
- {
544
- "epoch": 0.53,
545
- "grad_norm": 0.21752771735191345,
546
- "learning_rate": 0.0001879843821238155,
547
- "loss": 0.7077,
548
- "step": 77
549
- },
550
- {
551
- "epoch": 0.54,
552
- "grad_norm": 0.22346794605255127,
553
- "learning_rate": 0.00018763066800438636,
554
- "loss": 0.6522,
555
- "step": 78
556
- },
557
- {
558
- "epoch": 0.54,
559
- "grad_norm": 0.23172937333583832,
560
- "learning_rate": 0.00018727216564363723,
561
- "loss": 0.6099,
562
- "step": 79
563
- },
564
- {
565
- "epoch": 0.55,
566
- "grad_norm": 0.20895916223526,
567
- "learning_rate": 0.00018690889463055283,
568
- "loss": 0.7242,
569
- "step": 80
570
- },
571
- {
572
- "epoch": 0.56,
573
- "grad_norm": 0.26868510246276855,
574
- "learning_rate": 0.0001865408748146826,
575
- "loss": 0.705,
576
- "step": 81
577
- },
578
- {
579
- "epoch": 0.56,
580
- "grad_norm": 0.243666410446167,
581
- "learning_rate": 0.00018616812630505597,
582
- "loss": 0.8512,
583
- "step": 82
584
- },
585
- {
586
- "epoch": 0.57,
587
- "grad_norm": 0.21445880830287933,
588
- "learning_rate": 0.00018579066946908384,
589
- "loss": 0.7092,
590
- "step": 83
591
- },
592
- {
593
- "epoch": 0.58,
594
- "grad_norm": 0.22805824875831604,
595
- "learning_rate": 0.00018540852493144545,
596
- "loss": 0.6757,
597
- "step": 84
598
- },
599
- {
600
- "epoch": 0.58,
601
- "grad_norm": 0.25106164813041687,
602
- "learning_rate": 0.00018502171357296144,
603
- "loss": 0.7171,
604
- "step": 85
605
- },
606
- {
607
- "epoch": 0.59,
608
- "grad_norm": 0.2290162742137909,
609
- "learning_rate": 0.000184630256529453,
610
- "loss": 0.7053,
611
- "step": 86
612
- },
613
- {
614
- "epoch": 0.6,
615
- "grad_norm": 0.21118655800819397,
616
- "learning_rate": 0.00018423417519058694,
617
- "loss": 0.5827,
618
- "step": 87
619
- },
620
- {
621
- "epoch": 0.6,
622
- "grad_norm": 0.23761862516403198,
623
- "learning_rate": 0.00018383349119870695,
624
- "loss": 0.5827,
625
- "step": 88
626
- },
627
- {
628
- "epoch": 0.61,
629
- "grad_norm": 0.23689566552639008,
630
- "learning_rate": 0.00018342822644765104,
631
- "loss": 0.6401,
632
- "step": 89
633
- },
634
- {
635
- "epoch": 0.62,
636
- "grad_norm": 0.26510947942733765,
637
- "learning_rate": 0.00018301840308155507,
638
- "loss": 0.5963,
639
- "step": 90
640
- },
641
- {
642
- "epoch": 0.63,
643
- "grad_norm": 0.2142990678548813,
644
- "learning_rate": 0.0001826040434936431,
645
- "loss": 0.6258,
646
- "step": 91
647
- },
648
- {
649
- "epoch": 0.63,
650
- "grad_norm": 0.20709159970283508,
651
- "learning_rate": 0.00018218517032500344,
652
- "loss": 0.5685,
653
- "step": 92
654
- },
655
- {
656
- "epoch": 0.64,
657
- "grad_norm": 0.2485310137271881,
658
- "learning_rate": 0.0001817618064633518,
659
- "loss": 0.5901,
660
- "step": 93
661
- },
662
- {
663
- "epoch": 0.65,
664
- "grad_norm": 0.2432468980550766,
665
- "learning_rate": 0.00018133397504178057,
666
- "loss": 0.6891,
667
- "step": 94
668
- },
669
- {
670
- "epoch": 0.65,
671
- "grad_norm": 0.2318205088376999,
672
- "learning_rate": 0.00018090169943749476,
673
- "loss": 0.7339,
674
- "step": 95
675
- },
676
- {
677
- "epoch": 0.66,
678
- "grad_norm": 0.25346460938453674,
679
- "learning_rate": 0.00018046500327053463,
680
- "loss": 0.6606,
681
- "step": 96
682
- },
683
- {
684
- "epoch": 0.67,
685
- "grad_norm": 0.29421111941337585,
686
- "learning_rate": 0.0001800239104024851,
687
- "loss": 0.6698,
688
- "step": 97
689
- },
690
- {
691
- "epoch": 0.67,
692
- "grad_norm": 0.2356603443622589,
693
- "learning_rate": 0.00017957844493517213,
694
- "loss": 0.6225,
695
- "step": 98
696
- },
697
- {
698
- "epoch": 0.68,
699
- "grad_norm": 0.2331649512052536,
700
- "learning_rate": 0.00017912863120934534,
701
- "loss": 0.6838,
702
- "step": 99
703
- },
704
- {
705
- "epoch": 0.69,
706
- "grad_norm": 0.23008978366851807,
707
- "learning_rate": 0.00017867449380334834,
708
- "loss": 0.6303,
709
- "step": 100
710
- },
711
- {
712
- "epoch": 0.69,
713
- "grad_norm": 0.21604081988334656,
714
- "learning_rate": 0.00017821605753177562,
715
- "loss": 0.4923,
716
- "step": 101
717
- },
718
- {
719
- "epoch": 0.7,
720
- "grad_norm": 0.2283111810684204,
721
- "learning_rate": 0.00017775334744411678,
722
- "loss": 0.6344,
723
- "step": 102
724
- },
725
- {
726
- "epoch": 0.71,
727
- "grad_norm": 0.2448730617761612,
728
- "learning_rate": 0.00017728638882338746,
729
- "loss": 0.6789,
730
- "step": 103
731
- },
732
- {
733
- "epoch": 0.71,
734
- "grad_norm": 0.22479577362537384,
735
- "learning_rate": 0.00017681520718474823,
736
- "loss": 0.5841,
737
- "step": 104
738
- },
739
- {
740
- "epoch": 0.72,
741
- "grad_norm": 0.21999216079711914,
742
- "learning_rate": 0.00017633982827411032,
743
- "loss": 0.6402,
744
- "step": 105
745
- },
746
- {
747
- "epoch": 0.73,
748
- "grad_norm": 0.2296195775270462,
749
- "learning_rate": 0.00017586027806672857,
750
- "loss": 0.6148,
751
- "step": 106
752
- },
753
- {
754
- "epoch": 0.74,
755
- "grad_norm": 0.22455835342407227,
756
- "learning_rate": 0.00017537658276578247,
757
- "loss": 0.6418,
758
- "step": 107
759
- },
760
- {
761
- "epoch": 0.74,
762
- "grad_norm": 0.23929741978645325,
763
- "learning_rate": 0.00017488876880094413,
764
- "loss": 0.5459,
765
- "step": 108
766
- },
767
- {
768
- "epoch": 0.75,
769
- "grad_norm": 0.229934424161911,
770
- "learning_rate": 0.00017439686282693436,
771
- "loss": 0.6049,
772
- "step": 109
773
- },
774
- {
775
- "epoch": 0.76,
776
- "grad_norm": 0.24801373481750488,
777
- "learning_rate": 0.00017390089172206592,
778
- "loss": 0.7663,
779
- "step": 110
780
- },
781
- {
782
- "epoch": 0.76,
783
- "grad_norm": 0.2523267865180969,
784
- "learning_rate": 0.00017340088258677522,
785
- "loss": 0.6642,
786
- "step": 111
787
- },
788
- {
789
- "epoch": 0.77,
790
- "grad_norm": 0.22252008318901062,
791
- "learning_rate": 0.00017289686274214118,
792
- "loss": 0.6325,
793
- "step": 112
794
- },
795
- {
796
- "epoch": 0.78,
797
- "grad_norm": 0.27619731426239014,
798
- "learning_rate": 0.0001723888597283926,
799
- "loss": 0.655,
800
- "step": 113
801
- },
802
- {
803
- "epoch": 0.78,
804
- "grad_norm": 0.25090277194976807,
805
- "learning_rate": 0.00017187690130340328,
806
- "loss": 0.6251,
807
- "step": 114
808
- },
809
- {
810
- "epoch": 0.79,
811
- "grad_norm": 0.2236175239086151,
812
- "learning_rate": 0.00017136101544117525,
813
- "loss": 0.5959,
814
- "step": 115
815
- },
816
- {
817
- "epoch": 0.8,
818
- "grad_norm": 0.21940240263938904,
819
- "learning_rate": 0.00017084123033031024,
820
- "loss": 0.6028,
821
- "step": 116
822
- },
823
- {
824
- "epoch": 0.8,
825
- "grad_norm": 0.2741597294807434,
826
- "learning_rate": 0.00017031757437246947,
827
- "loss": 0.4947,
828
- "step": 117
829
- },
830
- {
831
- "epoch": 0.81,
832
- "grad_norm": 0.23343035578727722,
833
- "learning_rate": 0.00016979007618082175,
834
- "loss": 0.4415,
835
- "step": 118
836
- },
837
- {
838
- "epoch": 0.82,
839
- "grad_norm": 0.2116251140832901,
840
- "learning_rate": 0.00016925876457848,
841
- "loss": 0.5283,
842
- "step": 119
843
- },
844
- {
845
- "epoch": 0.82,
846
- "grad_norm": 0.25621068477630615,
847
- "learning_rate": 0.00016872366859692627,
848
- "loss": 0.7334,
849
- "step": 120
850
- },
851
- {
852
- "epoch": 0.83,
853
- "grad_norm": 0.22588759660720825,
854
- "learning_rate": 0.00016818481747442554,
855
- "loss": 0.6514,
856
- "step": 121
857
- },
858
- {
859
- "epoch": 0.84,
860
- "grad_norm": 0.24995574355125427,
861
- "learning_rate": 0.00016764224065442796,
862
- "loss": 0.5632,
863
- "step": 122
864
- },
865
- {
866
- "epoch": 0.85,
867
- "grad_norm": 0.2382134348154068,
868
- "learning_rate": 0.00016709596778396026,
869
- "loss": 0.5954,
870
- "step": 123
871
- },
872
- {
873
- "epoch": 0.85,
874
- "grad_norm": 0.25166618824005127,
875
- "learning_rate": 0.00016654602871200546,
876
- "loss": 0.6092,
877
- "step": 124
878
- },
879
- {
880
- "epoch": 0.86,
881
- "grad_norm": 0.23028074204921722,
882
- "learning_rate": 0.0001659924534878723,
883
- "loss": 0.576,
884
- "step": 125
885
- },
886
- {
887
- "epoch": 0.87,
888
- "grad_norm": 0.2527604103088379,
889
- "learning_rate": 0.00016543527235955282,
890
- "loss": 0.5985,
891
- "step": 126
892
- },
893
- {
894
- "epoch": 0.87,
895
- "grad_norm": 0.25870686769485474,
896
- "learning_rate": 0.00016487451577207018,
897
- "loss": 0.5767,
898
- "step": 127
899
- },
900
- {
901
- "epoch": 0.88,
902
- "grad_norm": 0.24936434626579285,
903
- "learning_rate": 0.0001643102143658145,
904
- "loss": 0.7494,
905
- "step": 128
906
- },
907
- {
908
- "epoch": 0.89,
909
- "grad_norm": 0.25158020853996277,
910
- "learning_rate": 0.000163742398974869,
911
- "loss": 0.578,
912
- "step": 129
913
- },
914
- {
915
- "epoch": 0.89,
916
- "grad_norm": 0.22290346026420593,
917
- "learning_rate": 0.0001631711006253251,
918
- "loss": 0.6489,
919
- "step": 130
920
- },
921
- {
922
- "epoch": 0.9,
923
- "grad_norm": 0.26421919465065,
924
- "learning_rate": 0.00016259635053358717,
925
- "loss": 0.5669,
926
- "step": 131
927
- },
928
- {
929
- "epoch": 0.91,
930
- "grad_norm": 0.2573990225791931,
931
- "learning_rate": 0.0001620181801046667,
932
- "loss": 0.6405,
933
- "step": 132
934
- },
935
- {
936
- "epoch": 0.91,
937
- "grad_norm": 0.19883453845977783,
938
- "learning_rate": 0.00016143662093046638,
939
- "loss": 0.4731,
940
- "step": 133
941
- },
942
- {
943
- "epoch": 0.92,
944
- "grad_norm": 0.24181459844112396,
945
- "learning_rate": 0.00016085170478805395,
946
- "loss": 0.6819,
947
- "step": 134
948
- },
949
- {
950
- "epoch": 0.93,
951
- "grad_norm": 0.2319963425397873,
952
- "learning_rate": 0.00016026346363792567,
953
- "loss": 0.6373,
954
- "step": 135
955
- },
956
- {
957
- "epoch": 0.93,
958
- "grad_norm": 0.2552095055580139,
959
- "learning_rate": 0.0001596719296222601,
960
- "loss": 0.7151,
961
- "step": 136
962
- },
963
- {
964
- "epoch": 0.94,
965
- "grad_norm": 0.27191445231437683,
966
- "learning_rate": 0.00015907713506316192,
967
- "loss": 0.611,
968
- "step": 137
969
- },
970
- {
971
- "epoch": 0.95,
972
- "grad_norm": 0.26981061697006226,
973
- "learning_rate": 0.0001584791124608955,
974
- "loss": 0.5923,
975
- "step": 138
976
- },
977
- {
978
- "epoch": 0.96,
979
- "grad_norm": 0.23907916247844696,
980
- "learning_rate": 0.00015787789449210938,
981
- "loss": 0.5655,
982
- "step": 139
983
- },
984
- {
985
- "epoch": 0.96,
986
- "grad_norm": 0.23364509642124176,
987
- "learning_rate": 0.00015727351400805052,
988
- "loss": 0.6019,
989
- "step": 140
990
- },
991
- {
992
- "epoch": 0.97,
993
- "grad_norm": 0.23096178472042084,
994
- "learning_rate": 0.0001566660040327695,
995
- "loss": 0.602,
996
- "step": 141
997
- },
998
- {
999
- "epoch": 0.98,
1000
- "grad_norm": 0.2656322717666626,
1001
- "learning_rate": 0.0001560553977613158,
1002
- "loss": 0.6195,
1003
- "step": 142
1004
- },
1005
- {
1006
- "epoch": 0.98,
1007
- "grad_norm": 0.23051652312278748,
1008
- "learning_rate": 0.00015544172855792423,
1009
- "loss": 0.6132,
1010
- "step": 143
1011
- },
1012
- {
1013
- "epoch": 0.99,
1014
- "grad_norm": 0.2443581074476242,
1015
- "learning_rate": 0.00015482502995419167,
1016
- "loss": 0.6144,
1017
- "step": 144
1018
- },
1019
- {
1020
- "epoch": 1.0,
1021
- "grad_norm": 0.2075706273317337,
1022
- "learning_rate": 0.00015420533564724495,
1023
- "loss": 0.5556,
1024
- "step": 145
1025
- },
1026
- {
1027
- "epoch": 1.0,
1028
- "grad_norm": 0.23596633970737457,
1029
- "learning_rate": 0.00015358267949789966,
1030
- "loss": 0.5926,
1031
- "step": 146
1032
- },
1033
- {
1034
- "epoch": 1.01,
1035
- "grad_norm": 0.25716009736061096,
1036
- "learning_rate": 0.0001529570955288099,
1037
- "loss": 0.6569,
1038
- "step": 147
1039
- },
1040
- {
1041
- "epoch": 1.02,
1042
- "grad_norm": 0.21337899565696716,
1043
- "learning_rate": 0.0001523286179226091,
1044
- "loss": 0.5873,
1045
- "step": 148
1046
- },
1047
- {
1048
- "epoch": 1.02,
1049
- "eval_loss": 0.6392110586166382,
1050
- "eval_runtime": 57.9435,
1051
- "eval_samples_per_second": 8.974,
1052
- "eval_steps_per_second": 1.122,
1053
- "step": 148
1054
- },
1055
- {
1056
- "epoch": 1.01,
1057
- "grad_norm": 0.20899337530136108,
1058
- "learning_rate": 0.00015169728102004256,
1059
- "loss": 0.5699,
1060
- "step": 149
1061
- },
1062
- {
1063
- "epoch": 1.01,
1064
- "grad_norm": 0.2038021683692932,
1065
- "learning_rate": 0.0001510631193180907,
1066
- "loss": 0.6119,
1067
- "step": 150
1068
- },
1069
- {
1070
- "epoch": 1.02,
1071
- "grad_norm": 0.19867417216300964,
1072
- "learning_rate": 0.00015042616746808435,
1073
- "loss": 0.5437,
1074
- "step": 151
1075
- },
1076
- {
1077
- "epoch": 1.03,
1078
- "grad_norm": 0.22234468162059784,
1079
- "learning_rate": 0.00014978646027381123,
1080
- "loss": 0.461,
1081
- "step": 152
1082
- },
1083
- {
1084
- "epoch": 1.03,
1085
- "grad_norm": 0.2240789234638214,
1086
- "learning_rate": 0.00014914403268961426,
1087
- "loss": 0.5873,
1088
- "step": 153
1089
- },
1090
- {
1091
- "epoch": 1.04,
1092
- "grad_norm": 0.1992528736591339,
1093
- "learning_rate": 0.00014849891981848158,
1094
- "loss": 0.4627,
1095
- "step": 154
1096
- },
1097
- {
1098
- "epoch": 1.05,
1099
- "grad_norm": 0.23049698770046234,
1100
- "learning_rate": 0.00014785115691012864,
1101
- "loss": 0.439,
1102
- "step": 155
1103
- },
1104
- {
1105
- "epoch": 1.05,
1106
- "grad_norm": 0.22458627820014954,
1107
- "learning_rate": 0.00014720077935907196,
1108
- "loss": 0.5893,
1109
- "step": 156
1110
- },
1111
- {
1112
- "epoch": 1.06,
1113
- "grad_norm": 0.21547600626945496,
1114
- "learning_rate": 0.00014654782270269497,
1115
- "loss": 0.6106,
1116
- "step": 157
1117
- },
1118
- {
1119
- "epoch": 1.07,
1120
- "grad_norm": 0.21248772740364075,
1121
- "learning_rate": 0.00014589232261930674,
1122
- "loss": 0.5599,
1123
- "step": 158
1124
- },
1125
- {
1126
- "epoch": 1.07,
1127
- "grad_norm": 0.22486011683940887,
1128
- "learning_rate": 0.0001452343149261919,
1129
- "loss": 0.5421,
1130
- "step": 159
1131
- },
1132
- {
1133
- "epoch": 1.08,
1134
- "grad_norm": 0.2963809072971344,
1135
- "learning_rate": 0.00014457383557765386,
1136
- "loss": 0.5321,
1137
- "step": 160
1138
- },
1139
- {
1140
- "epoch": 1.09,
1141
- "grad_norm": 0.2344168871641159,
1142
- "learning_rate": 0.0001439109206630501,
1143
- "loss": 0.5582,
1144
- "step": 161
1145
- },
1146
- {
1147
- "epoch": 1.09,
1148
- "grad_norm": 0.21608802676200867,
1149
- "learning_rate": 0.0001432456064048204,
1150
- "loss": 0.5249,
1151
- "step": 162
1152
- },
1153
- {
1154
- "epoch": 1.1,
1155
- "grad_norm": 0.3204934895038605,
1156
- "learning_rate": 0.00014257792915650728,
1157
- "loss": 0.5718,
1158
- "step": 163
1159
- },
1160
- {
1161
- "epoch": 1.11,
1162
- "grad_norm": 0.2391967922449112,
1163
- "learning_rate": 0.00014190792540076986,
1164
- "loss": 0.4687,
1165
- "step": 164
1166
- },
1167
- {
1168
- "epoch": 1.12,
1169
- "grad_norm": 0.20837116241455078,
1170
- "learning_rate": 0.00014123563174739037,
1171
- "loss": 0.437,
1172
- "step": 165
1173
- },
1174
- {
1175
- "epoch": 1.12,
1176
- "grad_norm": 0.25774210691452026,
1177
- "learning_rate": 0.0001405610849312736,
1178
- "loss": 0.6387,
1179
- "step": 166
1180
- },
1181
- {
1182
- "epoch": 1.13,
1183
- "grad_norm": 0.23517344892024994,
1184
- "learning_rate": 0.00013988432181043982,
1185
- "loss": 0.5234,
1186
- "step": 167
1187
- },
1188
- {
1189
- "epoch": 1.14,
1190
- "grad_norm": 0.24470289051532745,
1191
- "learning_rate": 0.00013920537936401077,
1192
- "loss": 0.5389,
1193
- "step": 168
1194
- },
1195
- {
1196
- "epoch": 1.14,
1197
- "grad_norm": 0.289318323135376,
1198
- "learning_rate": 0.000138524294690189,
1199
- "loss": 0.5869,
1200
- "step": 169
1201
- },
1202
- {
1203
- "epoch": 1.15,
1204
- "grad_norm": 0.2550831437110901,
1205
- "learning_rate": 0.00013784110500423104,
1206
- "loss": 0.5825,
1207
- "step": 170
1208
- },
1209
- {
1210
- "epoch": 1.16,
1211
- "grad_norm": 0.24041080474853516,
1212
- "learning_rate": 0.00013715584763641345,
1213
- "loss": 0.6241,
1214
- "step": 171
1215
- },
1216
- {
1217
- "epoch": 1.16,
1218
- "grad_norm": 0.25661879777908325,
1219
- "learning_rate": 0.00013646856002999354,
1220
- "loss": 0.5429,
1221
- "step": 172
1222
- },
1223
- {
1224
- "epoch": 1.17,
1225
- "grad_norm": 0.26754602789878845,
1226
- "learning_rate": 0.00013577927973916306,
1227
- "loss": 0.6131,
1228
- "step": 173
1229
- },
1230
- {
1231
- "epoch": 1.18,
1232
- "grad_norm": 0.22897249460220337,
1233
- "learning_rate": 0.00013508804442699648,
1234
- "loss": 0.4834,
1235
- "step": 174
1236
- },
1237
- {
1238
- "epoch": 1.18,
1239
- "grad_norm": 0.22126823663711548,
1240
- "learning_rate": 0.00013439489186339282,
1241
- "loss": 0.4735,
1242
- "step": 175
1243
- },
1244
- {
1245
- "epoch": 1.19,
1246
- "grad_norm": 0.2490013688802719,
1247
- "learning_rate": 0.00013369985992301198,
1248
- "loss": 0.5248,
1249
- "step": 176
1250
- },
1251
- {
1252
- "epoch": 1.2,
1253
- "grad_norm": 0.29270628094673157,
1254
- "learning_rate": 0.00013300298658320517,
1255
- "loss": 0.643,
1256
- "step": 177
1257
- },
1258
- {
1259
- "epoch": 1.2,
1260
- "grad_norm": 0.21294419467449188,
1261
- "learning_rate": 0.00013230430992193973,
1262
- "loss": 0.4513,
1263
- "step": 178
1264
- },
1265
- {
1266
- "epoch": 1.21,
1267
- "grad_norm": 0.20743757486343384,
1268
- "learning_rate": 0.00013160386811571876,
1269
- "loss": 0.5091,
1270
- "step": 179
1271
- },
1272
- {
1273
- "epoch": 1.22,
1274
- "grad_norm": 0.2378052920103073,
1275
- "learning_rate": 0.00013090169943749476,
1276
- "loss": 0.5357,
1277
- "step": 180
1278
- },
1279
- {
1280
- "epoch": 1.23,
1281
- "grad_norm": 0.24781295657157898,
1282
- "learning_rate": 0.00013019784225457855,
1283
- "loss": 0.5801,
1284
- "step": 181
1285
- },
1286
- {
1287
- "epoch": 1.23,
1288
- "grad_norm": 0.23349957168102264,
1289
- "learning_rate": 0.00012949233502654284,
1290
- "loss": 0.5168,
1291
- "step": 182
1292
- },
1293
- {
1294
- "epoch": 1.24,
1295
- "grad_norm": 0.2096189707517624,
1296
- "learning_rate": 0.00012878521630312078,
1297
- "loss": 0.4809,
1298
- "step": 183
1299
- },
1300
- {
1301
- "epoch": 1.25,
1302
- "grad_norm": 0.21930260956287384,
1303
- "learning_rate": 0.0001280765247220993,
1304
- "loss": 0.45,
1305
- "step": 184
1306
- },
1307
- {
1308
- "epoch": 1.25,
1309
- "grad_norm": 0.28573113679885864,
1310
- "learning_rate": 0.0001273662990072083,
1311
- "loss": 0.6648,
1312
- "step": 185
1313
- },
1314
- {
1315
- "epoch": 1.26,
1316
- "grad_norm": 0.23740755021572113,
1317
- "learning_rate": 0.00012665457796600443,
1318
- "loss": 0.5403,
1319
- "step": 186
1320
- },
1321
- {
1322
- "epoch": 1.27,
1323
- "grad_norm": 0.23442810773849487,
1324
- "learning_rate": 0.0001259414004877507,
1325
- "loss": 0.5332,
1326
- "step": 187
1327
- },
1328
- {
1329
- "epoch": 1.27,
1330
- "grad_norm": 0.22515830397605896,
1331
- "learning_rate": 0.00012522680554129156,
1332
- "loss": 0.4567,
1333
- "step": 188
1334
- },
1335
- {
1336
- "epoch": 1.28,
1337
- "grad_norm": 0.23571448028087616,
1338
- "learning_rate": 0.00012451083217292357,
1339
- "loss": 0.4068,
1340
- "step": 189
1341
- },
1342
- {
1343
- "epoch": 1.29,
1344
- "grad_norm": 0.1914808303117752,
1345
- "learning_rate": 0.00012379351950426187,
1346
- "loss": 0.4618,
1347
- "step": 190
1348
- },
1349
- {
1350
- "epoch": 1.29,
1351
- "grad_norm": 0.2481367439031601,
1352
- "learning_rate": 0.0001230749067301025,
1353
- "loss": 0.4055,
1354
- "step": 191
1355
- },
1356
- {
1357
- "epoch": 1.3,
1358
- "grad_norm": 0.22704686224460602,
1359
- "learning_rate": 0.00012235503311628073,
1360
- "loss": 0.4984,
1361
- "step": 192
1362
- },
1363
- {
1364
- "epoch": 1.31,
1365
- "grad_norm": 0.2515145242214203,
1366
- "learning_rate": 0.00012163393799752565,
1367
- "loss": 0.5463,
1368
- "step": 193
1369
- },
1370
- {
1371
- "epoch": 1.31,
1372
- "grad_norm": 0.23563461005687714,
1373
- "learning_rate": 0.00012091166077531075,
1374
- "loss": 0.5143,
1375
- "step": 194
1376
- },
1377
- {
1378
- "epoch": 1.32,
1379
- "grad_norm": 0.22308993339538574,
1380
- "learning_rate": 0.00012018824091570103,
1381
- "loss": 0.5242,
1382
- "step": 195
1383
- },
1384
- {
1385
- "epoch": 1.33,
1386
- "grad_norm": 0.2774996757507324,
1387
- "learning_rate": 0.00011946371794719656,
1388
- "loss": 0.5629,
1389
- "step": 196
1390
- },
1391
- {
1392
- "epoch": 1.34,
1393
- "grad_norm": 0.20966477692127228,
1394
- "learning_rate": 0.00011873813145857249,
1395
- "loss": 0.4812,
1396
- "step": 197
1397
- },
1398
- {
1399
- "epoch": 1.34,
1400
- "grad_norm": 0.2643747925758362,
1401
- "learning_rate": 0.00011801152109671595,
1402
- "loss": 0.4194,
1403
- "step": 198
1404
- },
1405
- {
1406
- "epoch": 1.35,
1407
- "grad_norm": 0.23725077509880066,
1408
- "learning_rate": 0.00011728392656445981,
1409
- "loss": 0.5517,
1410
- "step": 199
1411
- },
1412
- {
1413
- "epoch": 1.36,
1414
- "grad_norm": 0.25400522351264954,
1415
- "learning_rate": 0.000116555387618413,
1416
- "loss": 0.5144,
1417
- "step": 200
1418
- },
1419
- {
1420
- "epoch": 1.36,
1421
- "grad_norm": 0.24393105506896973,
1422
- "learning_rate": 0.00011582594406678839,
1423
- "loss": 0.5923,
1424
- "step": 201
1425
- },
1426
- {
1427
- "epoch": 1.37,
1428
- "grad_norm": 0.25045904517173767,
1429
- "learning_rate": 0.00011509563576722753,
1430
- "loss": 0.5662,
1431
- "step": 202
1432
- },
1433
- {
1434
- "epoch": 1.38,
1435
- "grad_norm": 0.20616577565670013,
1436
- "learning_rate": 0.00011436450262462284,
1437
- "loss": 0.4704,
1438
- "step": 203
1439
- },
1440
- {
1441
- "epoch": 1.38,
1442
- "grad_norm": 0.2518722712993622,
1443
- "learning_rate": 0.00011363258458893699,
1444
- "loss": 0.5764,
1445
- "step": 204
1446
- },
1447
- {
1448
- "epoch": 1.39,
1449
- "grad_norm": 0.2739958167076111,
1450
- "learning_rate": 0.00011289992165302035,
1451
- "loss": 0.5376,
1452
- "step": 205
1453
- },
1454
- {
1455
- "epoch": 1.4,
1456
- "grad_norm": 0.2688806354999542,
1457
- "learning_rate": 0.00011216655385042525,
1458
- "loss": 0.5916,
1459
- "step": 206
1460
- },
1461
- {
1462
- "epoch": 1.4,
1463
- "grad_norm": 0.22472062706947327,
1464
- "learning_rate": 0.00011143252125321892,
1465
- "loss": 0.4668,
1466
- "step": 207
1467
- },
1468
- {
1469
- "epoch": 1.41,
1470
- "grad_norm": 0.2582922577857971,
1471
- "learning_rate": 0.00011069786396979367,
1472
- "loss": 0.6268,
1473
- "step": 208
1474
- },
1475
- {
1476
- "epoch": 1.42,
1477
- "grad_norm": 0.2291630357503891,
1478
- "learning_rate": 0.0001099626221426754,
1479
- "loss": 0.498,
1480
- "step": 209
1481
- },
1482
- {
1483
- "epoch": 1.42,
1484
- "grad_norm": 0.24377630650997162,
1485
- "learning_rate": 0.00010922683594633021,
1486
- "loss": 0.5157,
1487
- "step": 210
1488
- },
1489
- {
1490
- "epoch": 1.43,
1491
- "grad_norm": 0.2518634498119354,
1492
- "learning_rate": 0.00010849054558496905,
1493
- "loss": 0.4786,
1494
- "step": 211
1495
- },
1496
- {
1497
- "epoch": 1.44,
1498
- "grad_norm": 0.23311232030391693,
1499
- "learning_rate": 0.00010775379129035116,
1500
- "loss": 0.4653,
1501
- "step": 212
1502
- },
1503
- {
1504
- "epoch": 1.45,
1505
- "grad_norm": 0.3206501007080078,
1506
- "learning_rate": 0.00010701661331958553,
1507
- "loss": 0.4863,
1508
- "step": 213
1509
- },
1510
- {
1511
- "epoch": 1.45,
1512
- "grad_norm": 0.23678985238075256,
1513
- "learning_rate": 0.00010627905195293135,
1514
- "loss": 0.5854,
1515
- "step": 214
1516
- },
1517
- {
1518
- "epoch": 1.46,
1519
- "grad_norm": 0.2651495933532715,
1520
- "learning_rate": 0.000105541147491597,
1521
- "loss": 0.4881,
1522
- "step": 215
1523
- },
1524
- {
1525
- "epoch": 1.47,
1526
- "grad_norm": 0.2638210952281952,
1527
- "learning_rate": 0.00010480294025553798,
1528
- "loss": 0.4984,
1529
- "step": 216
1530
- },
1531
- {
1532
- "epoch": 1.47,
1533
- "grad_norm": 0.28642264008522034,
1534
- "learning_rate": 0.00010406447058125368,
1535
- "loss": 0.6304,
1536
- "step": 217
1537
- },
1538
- {
1539
- "epoch": 1.48,
1540
- "grad_norm": 0.29784247279167175,
1541
- "learning_rate": 0.0001033257788195835,
1542
- "loss": 0.6045,
1543
- "step": 218
1544
- },
1545
- {
1546
- "epoch": 1.49,
1547
- "grad_norm": 0.28990456461906433,
1548
- "learning_rate": 0.0001025869053335019,
1549
- "loss": 0.6472,
1550
- "step": 219
1551
- },
1552
- {
1553
- "epoch": 1.49,
1554
- "grad_norm": 0.2643013894557953,
1555
- "learning_rate": 0.00010184789049591299,
1556
- "loss": 0.5676,
1557
- "step": 220
1558
- },
1559
- {
1560
- "epoch": 1.5,
1561
- "grad_norm": 0.23880311846733093,
1562
- "learning_rate": 0.0001011087746874445,
1563
- "loss": 0.4708,
1564
- "step": 221
1565
- },
1566
- {
1567
- "epoch": 1.51,
1568
- "grad_norm": 0.2558128833770752,
1569
- "learning_rate": 0.00010036959829424131,
1570
- "loss": 0.5367,
1571
- "step": 222
1572
- },
1573
- {
1574
- "epoch": 1.51,
1575
- "grad_norm": 0.2300223559141159,
1576
- "learning_rate": 9.96304017057587e-05,
1577
- "loss": 0.4626,
1578
- "step": 223
1579
- },
1580
- {
1581
- "epoch": 1.52,
1582
- "grad_norm": 0.2416905015707016,
1583
- "learning_rate": 9.889122531255552e-05,
1584
- "loss": 0.4464,
1585
- "step": 224
1586
- },
1587
- {
1588
- "epoch": 1.53,
1589
- "grad_norm": 0.26283279061317444,
1590
- "learning_rate": 9.815210950408704e-05,
1591
- "loss": 0.6352,
1592
- "step": 225
1593
- },
1594
- {
1595
- "epoch": 1.53,
1596
- "grad_norm": 0.24686123430728912,
1597
- "learning_rate": 9.741309466649813e-05,
1598
- "loss": 0.5295,
1599
- "step": 226
1600
- },
1601
- {
1602
- "epoch": 1.54,
1603
- "grad_norm": 0.2653113305568695,
1604
- "learning_rate": 9.667422118041651e-05,
1605
- "loss": 0.4808,
1606
- "step": 227
1607
- },
1608
- {
1609
- "epoch": 1.55,
1610
- "grad_norm": 0.26245051622390747,
1611
- "learning_rate": 9.593552941874635e-05,
1612
- "loss": 0.542,
1613
- "step": 228
1614
- },
1615
- {
1616
- "epoch": 1.55,
1617
- "grad_norm": 0.29881128668785095,
1618
- "learning_rate": 9.519705974446207e-05,
1619
- "loss": 0.6013,
1620
- "step": 229
1621
- },
1622
- {
1623
- "epoch": 1.56,
1624
- "grad_norm": 0.2243887037038803,
1625
- "learning_rate": 9.4458852508403e-05,
1626
- "loss": 0.474,
1627
- "step": 230
1628
- },
1629
- {
1630
- "epoch": 1.57,
1631
- "grad_norm": 0.2504364252090454,
1632
- "learning_rate": 9.372094804706867e-05,
1633
- "loss": 0.472,
1634
- "step": 231
1635
- },
1636
- {
1637
- "epoch": 1.58,
1638
- "grad_norm": 0.27919840812683105,
1639
- "learning_rate": 9.298338668041451e-05,
1640
- "loss": 0.4604,
1641
- "step": 232
1642
- },
1643
- {
1644
- "epoch": 1.58,
1645
- "grad_norm": 0.23338647186756134,
1646
- "learning_rate": 9.224620870964886e-05,
1647
- "loss": 0.4342,
1648
- "step": 233
1649
- },
1650
- {
1651
- "epoch": 1.59,
1652
- "grad_norm": 0.2783346474170685,
1653
- "learning_rate": 9.150945441503093e-05,
1654
- "loss": 0.5666,
1655
- "step": 234
1656
- },
1657
- {
1658
- "epoch": 1.6,
1659
- "grad_norm": 0.2518472373485565,
1660
- "learning_rate": 9.077316405366981e-05,
1661
- "loss": 0.4972,
1662
- "step": 235
1663
- },
1664
- {
1665
- "epoch": 1.6,
1666
- "grad_norm": 0.2644898295402527,
1667
- "learning_rate": 9.00373778573246e-05,
1668
- "loss": 0.4328,
1669
- "step": 236
1670
- },
1671
- {
1672
- "epoch": 1.61,
1673
- "grad_norm": 0.34149104356765747,
1674
- "learning_rate": 8.930213603020638e-05,
1675
- "loss": 0.6514,
1676
- "step": 237
1677
- },
1678
- {
1679
- "epoch": 1.62,
1680
- "grad_norm": 0.2584168612957001,
1681
- "learning_rate": 8.85674787467811e-05,
1682
- "loss": 0.5692,
1683
- "step": 238
1684
- },
1685
- {
1686
- "epoch": 1.62,
1687
- "grad_norm": 0.22137361764907837,
1688
- "learning_rate": 8.783344614957477e-05,
1689
- "loss": 0.5334,
1690
- "step": 239
1691
- },
1692
- {
1693
- "epoch": 1.63,
1694
- "grad_norm": 0.26125484704971313,
1695
- "learning_rate": 8.710007834697969e-05,
1696
- "loss": 0.5633,
1697
- "step": 240
1698
- },
1699
- {
1700
- "epoch": 1.64,
1701
- "grad_norm": 0.263741672039032,
1702
- "learning_rate": 8.636741541106299e-05,
1703
- "loss": 0.525,
1704
- "step": 241
1705
- },
1706
- {
1707
- "epoch": 1.64,
1708
- "grad_norm": 0.2840401232242584,
1709
- "learning_rate": 8.563549737537719e-05,
1710
- "loss": 0.5634,
1711
- "step": 242
1712
- },
1713
- {
1714
- "epoch": 1.65,
1715
- "grad_norm": 0.25339609384536743,
1716
- "learning_rate": 8.490436423277248e-05,
1717
- "loss": 0.5634,
1718
- "step": 243
1719
- },
1720
- {
1721
- "epoch": 1.66,
1722
- "grad_norm": 0.26044854521751404,
1723
- "learning_rate": 8.417405593321163e-05,
1724
- "loss": 0.5488,
1725
- "step": 244
1726
- },
1727
- {
1728
- "epoch": 1.66,
1729
- "grad_norm": 0.23519276082515717,
1730
- "learning_rate": 8.344461238158699e-05,
1731
- "loss": 0.4145,
1732
- "step": 245
1733
- },
1734
- {
1735
- "epoch": 1.67,
1736
- "grad_norm": 0.2189461588859558,
1737
- "learning_rate": 8.271607343554021e-05,
1738
- "loss": 0.518,
1739
- "step": 246
1740
- },
1741
- {
1742
- "epoch": 1.68,
1743
- "grad_norm": 0.2877447009086609,
1744
- "learning_rate": 8.198847890328406e-05,
1745
- "loss": 0.4556,
1746
- "step": 247
1747
- },
1748
- {
1749
- "epoch": 1.69,
1750
- "grad_norm": 0.2689398229122162,
1751
- "learning_rate": 8.126186854142752e-05,
1752
- "loss": 0.4426,
1753
- "step": 248
1754
- },
1755
- {
1756
- "epoch": 1.69,
1757
- "grad_norm": 0.25999996066093445,
1758
- "learning_rate": 8.053628205280347e-05,
1759
- "loss": 0.4472,
1760
- "step": 249
1761
- },
1762
- {
1763
- "epoch": 1.7,
1764
- "grad_norm": 0.2660202980041504,
1765
- "learning_rate": 7.9811759084299e-05,
1766
- "loss": 0.4789,
1767
- "step": 250
1768
- },
1769
- {
1770
- "epoch": 1.71,
1771
- "grad_norm": 0.2570638060569763,
1772
- "learning_rate": 7.908833922468927e-05,
1773
- "loss": 0.4245,
1774
- "step": 251
1775
- },
1776
- {
1777
- "epoch": 1.71,
1778
- "grad_norm": 0.22138474881649017,
1779
- "learning_rate": 7.836606200247436e-05,
1780
- "loss": 0.4377,
1781
- "step": 252
1782
- },
1783
- {
1784
- "epoch": 1.72,
1785
- "grad_norm": 0.2643532454967499,
1786
- "learning_rate": 7.764496688371929e-05,
1787
- "loss": 0.5689,
1788
- "step": 253
1789
- },
1790
- {
1791
- "epoch": 1.73,
1792
- "grad_norm": 0.3166361153125763,
1793
- "learning_rate": 7.692509326989753e-05,
1794
- "loss": 0.559,
1795
- "step": 254
1796
- },
1797
- {
1798
- "epoch": 1.73,
1799
- "grad_norm": 0.26772719621658325,
1800
- "learning_rate": 7.620648049573815e-05,
1801
- "loss": 0.5051,
1802
- "step": 255
1803
- },
1804
- {
1805
- "epoch": 1.74,
1806
- "grad_norm": 0.31207817792892456,
1807
- "learning_rate": 7.548916782707642e-05,
1808
- "loss": 0.5906,
1809
- "step": 256
1810
- },
1811
- {
1812
- "epoch": 1.75,
1813
- "grad_norm": 0.3109002113342285,
1814
- "learning_rate": 7.477319445870845e-05,
1815
- "loss": 0.5926,
1816
- "step": 257
1817
- },
1818
- {
1819
- "epoch": 1.75,
1820
- "grad_norm": 0.24690654873847961,
1821
- "learning_rate": 7.405859951224933e-05,
1822
- "loss": 0.5132,
1823
- "step": 258
1824
- },
1825
- {
1826
- "epoch": 1.76,
1827
- "grad_norm": 0.2552764117717743,
1828
- "learning_rate": 7.33454220339956e-05,
1829
- "loss": 0.6071,
1830
- "step": 259
1831
- },
1832
- {
1833
- "epoch": 1.77,
1834
- "grad_norm": 0.22326481342315674,
1835
- "learning_rate": 7.263370099279172e-05,
1836
- "loss": 0.5135,
1837
- "step": 260
1838
- },
1839
- {
1840
- "epoch": 1.77,
1841
- "grad_norm": 0.22871443629264832,
1842
- "learning_rate": 7.192347527790073e-05,
1843
- "loss": 0.4972,
1844
- "step": 261
1845
- },
1846
- {
1847
- "epoch": 1.78,
1848
- "grad_norm": 0.25344693660736084,
1849
- "learning_rate": 7.121478369687926e-05,
1850
- "loss": 0.4668,
1851
- "step": 262
1852
- },
1853
- {
1854
- "epoch": 1.79,
1855
- "grad_norm": 0.24425604939460754,
1856
- "learning_rate": 7.050766497345714e-05,
1857
- "loss": 0.5499,
1858
- "step": 263
1859
- },
1860
- {
1861
- "epoch": 1.8,
1862
- "grad_norm": 0.24569104611873627,
1863
- "learning_rate": 6.980215774542147e-05,
1864
- "loss": 0.5668,
1865
- "step": 264
1866
- },
1867
- {
1868
- "epoch": 1.8,
1869
- "grad_norm": 0.20475324988365173,
1870
- "learning_rate": 6.909830056250527e-05,
1871
- "loss": 0.5205,
1872
- "step": 265
1873
- },
1874
- {
1875
- "epoch": 1.81,
1876
- "grad_norm": 0.2739812731742859,
1877
- "learning_rate": 6.839613188428126e-05,
1878
- "loss": 0.4883,
1879
- "step": 266
1880
- },
1881
- {
1882
- "epoch": 1.82,
1883
- "grad_norm": 0.24493646621704102,
1884
- "learning_rate": 6.769569007806027e-05,
1885
- "loss": 0.5187,
1886
- "step": 267
1887
- },
1888
- {
1889
- "epoch": 1.82,
1890
- "grad_norm": 0.25336140394210815,
1891
- "learning_rate": 6.699701341679488e-05,
1892
- "loss": 0.5467,
1893
- "step": 268
1894
- },
1895
- {
1896
- "epoch": 1.83,
1897
- "grad_norm": 0.3002632260322571,
1898
- "learning_rate": 6.630014007698807e-05,
1899
- "loss": 0.613,
1900
- "step": 269
1901
- },
1902
- {
1903
- "epoch": 1.84,
1904
- "grad_norm": 0.23223187029361725,
1905
- "learning_rate": 6.560510813660719e-05,
1906
- "loss": 0.4893,
1907
- "step": 270
1908
- },
1909
- {
1910
- "epoch": 1.84,
1911
- "grad_norm": 0.22384615242481232,
1912
- "learning_rate": 6.491195557300353e-05,
1913
- "loss": 0.4206,
1914
- "step": 271
1915
- },
1916
- {
1917
- "epoch": 1.85,
1918
- "grad_norm": 0.2999289333820343,
1919
- "learning_rate": 6.422072026083697e-05,
1920
- "loss": 0.5848,
1921
- "step": 272
1922
- },
1923
- {
1924
- "epoch": 1.86,
1925
- "grad_norm": 0.230120450258255,
1926
- "learning_rate": 6.35314399700065e-05,
1927
- "loss": 0.4872,
1928
- "step": 273
1929
- },
1930
- {
1931
- "epoch": 1.86,
1932
- "grad_norm": 0.2404165416955948,
1933
- "learning_rate": 6.284415236358653e-05,
1934
- "loss": 0.5125,
1935
- "step": 274
1936
- },
1937
- {
1938
- "epoch": 1.87,
1939
- "grad_norm": 0.2641153037548065,
1940
- "learning_rate": 6.215889499576898e-05,
1941
- "loss": 0.5132,
1942
- "step": 275
1943
- },
1944
- {
1945
- "epoch": 1.88,
1946
- "grad_norm": 0.2617990970611572,
1947
- "learning_rate": 6.147570530981099e-05,
1948
- "loss": 0.4281,
1949
- "step": 276
1950
- },
1951
- {
1952
- "epoch": 1.88,
1953
- "grad_norm": 0.2755200266838074,
1954
- "learning_rate": 6.0794620635989244e-05,
1955
- "loss": 0.4031,
1956
- "step": 277
1957
- },
1958
- {
1959
- "epoch": 1.89,
1960
- "grad_norm": 0.32721757888793945,
1961
- "learning_rate": 6.011567818956021e-05,
1962
- "loss": 0.4956,
1963
- "step": 278
1964
- },
1965
- {
1966
- "epoch": 1.9,
1967
- "grad_norm": 0.3732713758945465,
1968
- "learning_rate": 5.943891506872645e-05,
1969
- "loss": 0.5509,
1970
- "step": 279
1971
- },
1972
- {
1973
- "epoch": 1.91,
1974
- "grad_norm": 0.27332013845443726,
1975
- "learning_rate": 5.876436825260967e-05,
1976
- "loss": 0.5633,
1977
- "step": 280
1978
- },
1979
- {
1980
- "epoch": 1.91,
1981
- "grad_norm": 0.23139792680740356,
1982
- "learning_rate": 5.809207459923016e-05,
1983
- "loss": 0.3776,
1984
- "step": 281
1985
- },
1986
- {
1987
- "epoch": 1.92,
1988
- "grad_norm": 0.29461902379989624,
1989
- "learning_rate": 5.7422070843492734e-05,
1990
- "loss": 0.438,
1991
- "step": 282
1992
- },
1993
- {
1994
- "epoch": 1.93,
1995
- "grad_norm": 0.27846580743789673,
1996
- "learning_rate": 5.675439359517962e-05,
1997
- "loss": 0.4784,
1998
- "step": 283
1999
- },
2000
- {
2001
- "epoch": 1.93,
2002
- "grad_norm": 0.2910459637641907,
2003
- "learning_rate": 5.608907933694994e-05,
2004
- "loss": 0.5903,
2005
- "step": 284
2006
- },
2007
- {
2008
- "epoch": 1.94,
2009
- "grad_norm": 0.2746853828430176,
2010
- "learning_rate": 5.542616442234618e-05,
2011
- "loss": 0.4604,
2012
- "step": 285
2013
- },
2014
- {
2015
- "epoch": 1.95,
2016
- "grad_norm": 0.2670218348503113,
2017
- "learning_rate": 5.476568507380815e-05,
2018
- "loss": 0.4851,
2019
- "step": 286
2020
- },
2021
- {
2022
- "epoch": 1.95,
2023
- "grad_norm": 0.289640873670578,
2024
- "learning_rate": 5.410767738069328e-05,
2025
- "loss": 0.4749,
2026
- "step": 287
2027
- },
2028
- {
2029
- "epoch": 1.96,
2030
- "grad_norm": 0.2754352390766144,
2031
- "learning_rate": 5.345217729730501e-05,
2032
- "loss": 0.5822,
2033
- "step": 288
2034
- },
2035
- {
2036
- "epoch": 1.97,
2037
- "grad_norm": 0.29798761010169983,
2038
- "learning_rate": 5.279922064092808e-05,
2039
- "loss": 0.5244,
2040
- "step": 289
2041
- },
2042
- {
2043
- "epoch": 1.97,
2044
- "grad_norm": 0.2856166660785675,
2045
- "learning_rate": 5.214884308987136e-05,
2046
- "loss": 0.5525,
2047
- "step": 290
2048
- }
2049
- ],
2050
- "logging_steps": 1,
2051
- "max_steps": 435,
2052
- "num_input_tokens_seen": 0,
2053
- "num_train_epochs": 3,
2054
- "save_steps": 145,
2055
- "total_flos": 1.9525129671278592e+17,
2056
- "train_batch_size": 8,
2057
- "trial_name": null,
2058
- "trial_params": null
2059
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
checkpoint-290/training_args.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:e0f4e74f968ef592cbe5e646087e575fbeada43b9dc36d459d99d6a9ebc528b7
3
- size 5560
 
 
 
 
checkpoint-435/README.md DELETED
@@ -1,202 +0,0 @@
1
- ---
2
- library_name: peft
3
- base_model: HuggingFaceTB/cosmo-1b
4
- ---
5
-
6
- # Model Card for Model ID
7
-
8
- <!-- Provide a quick summary of what the model is/does. -->
9
-
10
-
11
-
12
- ## Model Details
13
-
14
- ### Model Description
15
-
16
- <!-- Provide a longer summary of what this model is. -->
17
-
18
-
19
-
20
- - **Developed by:** [More Information Needed]
21
- - **Funded by [optional]:** [More Information Needed]
22
- - **Shared by [optional]:** [More Information Needed]
23
- - **Model type:** [More Information Needed]
24
- - **Language(s) (NLP):** [More Information Needed]
25
- - **License:** [More Information Needed]
26
- - **Finetuned from model [optional]:** [More Information Needed]
27
-
28
- ### Model Sources [optional]
29
-
30
- <!-- Provide the basic links for the model. -->
31
-
32
- - **Repository:** [More Information Needed]
33
- - **Paper [optional]:** [More Information Needed]
34
- - **Demo [optional]:** [More Information Needed]
35
-
36
- ## Uses
37
-
38
- <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
39
-
40
- ### Direct Use
41
-
42
- <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
43
-
44
- [More Information Needed]
45
-
46
- ### Downstream Use [optional]
47
-
48
- <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
49
-
50
- [More Information Needed]
51
-
52
- ### Out-of-Scope Use
53
-
54
- <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
55
-
56
- [More Information Needed]
57
-
58
- ## Bias, Risks, and Limitations
59
-
60
- <!-- This section is meant to convey both technical and sociotechnical limitations. -->
61
-
62
- [More Information Needed]
63
-
64
- ### Recommendations
65
-
66
- <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
67
-
68
- Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
69
-
70
- ## How to Get Started with the Model
71
-
72
- Use the code below to get started with the model.
73
-
74
- [More Information Needed]
75
-
76
- ## Training Details
77
-
78
- ### Training Data
79
-
80
- <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
81
-
82
- [More Information Needed]
83
-
84
- ### Training Procedure
85
-
86
- <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
87
-
88
- #### Preprocessing [optional]
89
-
90
- [More Information Needed]
91
-
92
-
93
- #### Training Hyperparameters
94
-
95
- - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
96
-
97
- #### Speeds, Sizes, Times [optional]
98
-
99
- <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
100
-
101
- [More Information Needed]
102
-
103
- ## Evaluation
104
-
105
- <!-- This section describes the evaluation protocols and provides the results. -->
106
-
107
- ### Testing Data, Factors & Metrics
108
-
109
- #### Testing Data
110
-
111
- <!-- This should link to a Dataset Card if possible. -->
112
-
113
- [More Information Needed]
114
-
115
- #### Factors
116
-
117
- <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
118
-
119
- [More Information Needed]
120
-
121
- #### Metrics
122
-
123
- <!-- These are the evaluation metrics being used, ideally with a description of why. -->
124
-
125
- [More Information Needed]
126
-
127
- ### Results
128
-
129
- [More Information Needed]
130
-
131
- #### Summary
132
-
133
-
134
-
135
- ## Model Examination [optional]
136
-
137
- <!-- Relevant interpretability work for the model goes here -->
138
-
139
- [More Information Needed]
140
-
141
- ## Environmental Impact
142
-
143
- <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
144
-
145
- Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
146
-
147
- - **Hardware Type:** [More Information Needed]
148
- - **Hours used:** [More Information Needed]
149
- - **Cloud Provider:** [More Information Needed]
150
- - **Compute Region:** [More Information Needed]
151
- - **Carbon Emitted:** [More Information Needed]
152
-
153
- ## Technical Specifications [optional]
154
-
155
- ### Model Architecture and Objective
156
-
157
- [More Information Needed]
158
-
159
- ### Compute Infrastructure
160
-
161
- [More Information Needed]
162
-
163
- #### Hardware
164
-
165
- [More Information Needed]
166
-
167
- #### Software
168
-
169
- [More Information Needed]
170
-
171
- ## Citation [optional]
172
-
173
- <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
174
-
175
- **BibTeX:**
176
-
177
- [More Information Needed]
178
-
179
- **APA:**
180
-
181
- [More Information Needed]
182
-
183
- ## Glossary [optional]
184
-
185
- <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
186
-
187
- [More Information Needed]
188
-
189
- ## More Information [optional]
190
-
191
- [More Information Needed]
192
-
193
- ## Model Card Authors [optional]
194
-
195
- [More Information Needed]
196
-
197
- ## Model Card Contact
198
-
199
- [More Information Needed]
200
- ### Framework versions
201
-
202
- - PEFT 0.9.1.dev0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
checkpoint-435/adapter_config.json DELETED
@@ -1,33 +0,0 @@
1
- {
2
- "alpha_pattern": {},
3
- "auto_mapping": null,
4
- "base_model_name_or_path": "HuggingFaceTB/cosmo-1b",
5
- "bias": "none",
6
- "fan_in_fan_out": null,
7
- "inference_mode": true,
8
- "init_lora_weights": true,
9
- "layers_pattern": null,
10
- "layers_to_transform": null,
11
- "loftq_config": {},
12
- "lora_alpha": 32,
13
- "lora_dropout": 0.1,
14
- "megatron_config": null,
15
- "megatron_core": "megatron.core",
16
- "modules_to_save": null,
17
- "peft_type": "LORA",
18
- "r": 32,
19
- "rank_pattern": {},
20
- "revision": null,
21
- "target_modules": [
22
- "v_proj",
23
- "k_proj",
24
- "o_proj",
25
- "up_proj",
26
- "down_proj",
27
- "gate_proj",
28
- "q_proj"
29
- ],
30
- "task_type": "CAUSAL_LM",
31
- "use_dora": false,
32
- "use_rslora": false
33
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
checkpoint-435/adapter_model.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:359774f6b53f026ff6ce012c633ef131eb058dac10ad79492ade359d7303cf4c
3
- size 144748392
 
 
 
 
checkpoint-435/optimizer.pt DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:a01a74a078b74a1a44b1e41ecc2c3e3ea0287ce5a785afe95be90b285e8b7fb4
3
- size 72887380
 
 
 
 
checkpoint-435/rng_state.pth DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:19a63f3d4ca1eb6cd7c0cdbb9b50e83d2608862f542c93e95054dd1a0d4d70f4
3
- size 14244
 
 
 
 
checkpoint-435/scheduler.pt DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:a301035e0f353aca3f770342289bdb6e15ca0abb2f650431eca332188b0898af
3
- size 1064
 
 
 
 
checkpoint-435/trainer_state.json DELETED
@@ -1,3082 +0,0 @@
1
- {
2
- "best_metric": null,
3
- "best_model_checkpoint": null,
4
- "epoch": 2.9518900343642613,
5
- "eval_steps": 500,
6
- "global_step": 435,
7
- "is_hyper_param_search": false,
8
- "is_local_process_zero": true,
9
- "is_world_process_zero": true,
10
- "log_history": [
11
- {
12
- "epoch": 0.01,
13
- "grad_norm": 0.37809523940086365,
14
- "learning_rate": 2e-05,
15
- "loss": 1.7814,
16
- "step": 1
17
- },
18
- {
19
- "epoch": 0.01,
20
- "grad_norm": 0.366623193025589,
21
- "learning_rate": 4e-05,
22
- "loss": 1.69,
23
- "step": 2
24
- },
25
- {
26
- "epoch": 0.02,
27
- "grad_norm": 0.3546508848667145,
28
- "learning_rate": 6e-05,
29
- "loss": 1.7151,
30
- "step": 3
31
- },
32
- {
33
- "epoch": 0.03,
34
- "grad_norm": 0.36381417512893677,
35
- "learning_rate": 8e-05,
36
- "loss": 1.6609,
37
- "step": 4
38
- },
39
- {
40
- "epoch": 0.03,
41
- "grad_norm": 0.4049898684024811,
42
- "learning_rate": 0.0001,
43
- "loss": 1.752,
44
- "step": 5
45
- },
46
- {
47
- "epoch": 0.04,
48
- "grad_norm": 0.4318874478340149,
49
- "learning_rate": 0.00012,
50
- "loss": 1.747,
51
- "step": 6
52
- },
53
- {
54
- "epoch": 0.05,
55
- "grad_norm": 0.39768990874290466,
56
- "learning_rate": 0.00014,
57
- "loss": 1.4432,
58
- "step": 7
59
- },
60
- {
61
- "epoch": 0.05,
62
- "grad_norm": 0.5056629180908203,
63
- "learning_rate": 0.00016,
64
- "loss": 1.7657,
65
- "step": 8
66
- },
67
- {
68
- "epoch": 0.06,
69
- "grad_norm": 0.4276069104671478,
70
- "learning_rate": 0.00018,
71
- "loss": 1.4221,
72
- "step": 9
73
- },
74
- {
75
- "epoch": 0.07,
76
- "grad_norm": 0.433552622795105,
77
- "learning_rate": 0.0002,
78
- "loss": 1.3117,
79
- "step": 10
80
- },
81
- {
82
- "epoch": 0.08,
83
- "grad_norm": 0.42256900668144226,
84
- "learning_rate": 0.0001999972679420179,
85
- "loss": 1.2687,
86
- "step": 11
87
- },
88
- {
89
- "epoch": 0.08,
90
- "grad_norm": 0.3066452145576477,
91
- "learning_rate": 0.00019998907191735434,
92
- "loss": 1.0398,
93
- "step": 12
94
- },
95
- {
96
- "epoch": 0.09,
97
- "grad_norm": 0.3393724858760834,
98
- "learning_rate": 0.00019997541237384966,
99
- "loss": 1.0063,
100
- "step": 13
101
- },
102
- {
103
- "epoch": 0.1,
104
- "grad_norm": 0.49836328625679016,
105
- "learning_rate": 0.00019995629005787713,
106
- "loss": 1.2251,
107
- "step": 14
108
- },
109
- {
110
- "epoch": 0.1,
111
- "grad_norm": 0.3539387285709381,
112
- "learning_rate": 0.0001999317060143023,
113
- "loss": 0.9454,
114
- "step": 15
115
- },
116
- {
117
- "epoch": 0.11,
118
- "grad_norm": 0.35584160685539246,
119
- "learning_rate": 0.0001999016615864258,
120
- "loss": 1.0105,
121
- "step": 16
122
- },
123
- {
124
- "epoch": 0.12,
125
- "grad_norm": 0.33191052079200745,
126
- "learning_rate": 0.00019986615841591002,
127
- "loss": 1.1322,
128
- "step": 17
129
- },
130
- {
131
- "epoch": 0.12,
132
- "grad_norm": 0.3499913513660431,
133
- "learning_rate": 0.00019982519844268933,
134
- "loss": 0.8495,
135
- "step": 18
136
- },
137
- {
138
- "epoch": 0.13,
139
- "grad_norm": 0.2582085132598877,
140
- "learning_rate": 0.0001997787839048642,
141
- "loss": 0.9725,
142
- "step": 19
143
- },
144
- {
145
- "epoch": 0.14,
146
- "grad_norm": 0.23511624336242676,
147
- "learning_rate": 0.00019972691733857883,
148
- "loss": 0.9233,
149
- "step": 20
150
- },
151
- {
152
- "epoch": 0.14,
153
- "grad_norm": 0.273019939661026,
154
- "learning_rate": 0.00019966960157788248,
155
- "loss": 1.0366,
156
- "step": 21
157
- },
158
- {
159
- "epoch": 0.15,
160
- "grad_norm": 0.283527135848999,
161
- "learning_rate": 0.0001996068397545748,
162
- "loss": 0.9944,
163
- "step": 22
164
- },
165
- {
166
- "epoch": 0.16,
167
- "grad_norm": 0.24311360716819763,
168
- "learning_rate": 0.00019953863529803466,
169
- "loss": 1.0308,
170
- "step": 23
171
- },
172
- {
173
- "epoch": 0.16,
174
- "grad_norm": 0.24368223547935486,
175
- "learning_rate": 0.00019946499193503262,
176
- "loss": 1.0051,
177
- "step": 24
178
- },
179
- {
180
- "epoch": 0.17,
181
- "grad_norm": 0.2269805669784546,
182
- "learning_rate": 0.0001993859136895274,
183
- "loss": 0.835,
184
- "step": 25
185
- },
186
- {
187
- "epoch": 0.18,
188
- "grad_norm": 0.21915565431118011,
189
- "learning_rate": 0.00019930140488244602,
190
- "loss": 0.9997,
191
- "step": 26
192
- },
193
- {
194
- "epoch": 0.19,
195
- "grad_norm": 0.21885964274406433,
196
- "learning_rate": 0.0001992114701314478,
197
- "loss": 1.0013,
198
- "step": 27
199
- },
200
- {
201
- "epoch": 0.19,
202
- "grad_norm": 0.22426557540893555,
203
- "learning_rate": 0.00019911611435067172,
204
- "loss": 0.8633,
205
- "step": 28
206
- },
207
- {
208
- "epoch": 0.2,
209
- "grad_norm": 0.21587449312210083,
210
- "learning_rate": 0.0001990153427504683,
211
- "loss": 0.8479,
212
- "step": 29
213
- },
214
- {
215
- "epoch": 0.21,
216
- "grad_norm": 0.24985891580581665,
217
- "learning_rate": 0.0001989091608371146,
218
- "loss": 0.8936,
219
- "step": 30
220
- },
221
- {
222
- "epoch": 0.21,
223
- "grad_norm": 0.19733141362667084,
224
- "learning_rate": 0.0001987975744125135,
225
- "loss": 0.7743,
226
- "step": 31
227
- },
228
- {
229
- "epoch": 0.22,
230
- "grad_norm": 0.1997266411781311,
231
- "learning_rate": 0.00019868058957387663,
232
- "loss": 0.9078,
233
- "step": 32
234
- },
235
- {
236
- "epoch": 0.23,
237
- "grad_norm": 0.24114945530891418,
238
- "learning_rate": 0.00019855821271339125,
239
- "loss": 1.0413,
240
- "step": 33
241
- },
242
- {
243
- "epoch": 0.23,
244
- "grad_norm": 0.2427120953798294,
245
- "learning_rate": 0.00019843045051787096,
246
- "loss": 0.8308,
247
- "step": 34
248
- },
249
- {
250
- "epoch": 0.24,
251
- "grad_norm": 0.1968858540058136,
252
- "learning_rate": 0.0001982973099683902,
253
- "loss": 0.818,
254
- "step": 35
255
- },
256
- {
257
- "epoch": 0.25,
258
- "grad_norm": 0.2783421576023102,
259
- "learning_rate": 0.00019815879833990304,
260
- "loss": 0.8504,
261
- "step": 36
262
- },
263
- {
264
- "epoch": 0.25,
265
- "grad_norm": 0.23332269489765167,
266
- "learning_rate": 0.00019801492320084546,
267
- "loss": 0.9826,
268
- "step": 37
269
- },
270
- {
271
- "epoch": 0.26,
272
- "grad_norm": 0.22864645719528198,
273
- "learning_rate": 0.00019786569241272197,
274
- "loss": 0.8705,
275
- "step": 38
276
- },
277
- {
278
- "epoch": 0.27,
279
- "grad_norm": 0.20620585978031158,
280
- "learning_rate": 0.00019771111412967583,
281
- "loss": 0.8264,
282
- "step": 39
283
- },
284
- {
285
- "epoch": 0.27,
286
- "grad_norm": 0.20590244233608246,
287
- "learning_rate": 0.00019755119679804367,
288
- "loss": 0.8455,
289
- "step": 40
290
- },
291
- {
292
- "epoch": 0.28,
293
- "grad_norm": 0.20340456068515778,
294
- "learning_rate": 0.00019738594915589397,
295
- "loss": 0.871,
296
- "step": 41
297
- },
298
- {
299
- "epoch": 0.29,
300
- "grad_norm": 0.2383277714252472,
301
- "learning_rate": 0.0001972153802325495,
302
- "loss": 0.6833,
303
- "step": 42
304
- },
305
- {
306
- "epoch": 0.3,
307
- "grad_norm": 0.21479888260364532,
308
- "learning_rate": 0.00019703949934809408,
309
- "loss": 0.8355,
310
- "step": 43
311
- },
312
- {
313
- "epoch": 0.3,
314
- "grad_norm": 0.23538872599601746,
315
- "learning_rate": 0.0001968583161128631,
316
- "loss": 0.8479,
317
- "step": 44
318
- },
319
- {
320
- "epoch": 0.31,
321
- "grad_norm": 0.2260449230670929,
322
- "learning_rate": 0.00019667184042691875,
323
- "loss": 0.779,
324
- "step": 45
325
- },
326
- {
327
- "epoch": 0.32,
328
- "grad_norm": 0.20232993364334106,
329
- "learning_rate": 0.0001964800824795087,
330
- "loss": 0.7293,
331
- "step": 46
332
- },
333
- {
334
- "epoch": 0.32,
335
- "grad_norm": 0.19780154526233673,
336
- "learning_rate": 0.00019628305274850956,
337
- "loss": 0.7709,
338
- "step": 47
339
- },
340
- {
341
- "epoch": 0.33,
342
- "grad_norm": 0.24822202324867249,
343
- "learning_rate": 0.00019608076199985433,
344
- "loss": 0.8885,
345
- "step": 48
346
- },
347
- {
348
- "epoch": 0.34,
349
- "grad_norm": 0.20940056443214417,
350
- "learning_rate": 0.0001958732212869441,
351
- "loss": 0.6916,
352
- "step": 49
353
- },
354
- {
355
- "epoch": 0.34,
356
- "grad_norm": 0.18944858014583588,
357
- "learning_rate": 0.0001956604419500441,
358
- "loss": 0.7326,
359
- "step": 50
360
- },
361
- {
362
- "epoch": 0.35,
363
- "grad_norm": 0.201398104429245,
364
- "learning_rate": 0.00019544243561566403,
365
- "loss": 0.6542,
366
- "step": 51
367
- },
368
- {
369
- "epoch": 0.36,
370
- "grad_norm": 0.22385160624980927,
371
- "learning_rate": 0.00019521921419592283,
372
- "loss": 0.8036,
373
- "step": 52
374
- },
375
- {
376
- "epoch": 0.36,
377
- "grad_norm": 0.19502398371696472,
378
- "learning_rate": 0.0001949907898878977,
379
- "loss": 0.7579,
380
- "step": 53
381
- },
382
- {
383
- "epoch": 0.37,
384
- "grad_norm": 0.21633867919445038,
385
- "learning_rate": 0.00019475717517295778,
386
- "loss": 0.7803,
387
- "step": 54
388
- },
389
- {
390
- "epoch": 0.38,
391
- "grad_norm": 0.24109330773353577,
392
- "learning_rate": 0.00019451838281608197,
393
- "loss": 0.7745,
394
- "step": 55
395
- },
396
- {
397
- "epoch": 0.38,
398
- "grad_norm": 0.20285621285438538,
399
- "learning_rate": 0.00019427442586516155,
400
- "loss": 0.7306,
401
- "step": 56
402
- },
403
- {
404
- "epoch": 0.39,
405
- "grad_norm": 0.23838144540786743,
406
- "learning_rate": 0.00019402531765028722,
407
- "loss": 0.7606,
408
- "step": 57
409
- },
410
- {
411
- "epoch": 0.4,
412
- "grad_norm": 0.25280559062957764,
413
- "learning_rate": 0.00019377107178302074,
414
- "loss": 0.7928,
415
- "step": 58
416
- },
417
- {
418
- "epoch": 0.41,
419
- "grad_norm": 0.24614648520946503,
420
- "learning_rate": 0.00019351170215565114,
421
- "loss": 0.739,
422
- "step": 59
423
- },
424
- {
425
- "epoch": 0.41,
426
- "grad_norm": 0.20391589403152466,
427
- "learning_rate": 0.00019324722294043558,
428
- "loss": 0.7636,
429
- "step": 60
430
- },
431
- {
432
- "epoch": 0.42,
433
- "grad_norm": 0.23704086244106293,
434
- "learning_rate": 0.00019297764858882514,
435
- "loss": 0.6919,
436
- "step": 61
437
- },
438
- {
439
- "epoch": 0.43,
440
- "grad_norm": 0.22057901322841644,
441
- "learning_rate": 0.00019270299383067498,
442
- "loss": 0.6035,
443
- "step": 62
444
- },
445
- {
446
- "epoch": 0.43,
447
- "grad_norm": 0.22797846794128418,
448
- "learning_rate": 0.0001924232736734396,
449
- "loss": 0.7087,
450
- "step": 63
451
- },
452
- {
453
- "epoch": 0.44,
454
- "grad_norm": 0.22181542217731476,
455
- "learning_rate": 0.00019213850340135276,
456
- "loss": 0.8088,
457
- "step": 64
458
- },
459
- {
460
- "epoch": 0.45,
461
- "grad_norm": 0.2096872180700302,
462
- "learning_rate": 0.00019184869857459232,
463
- "loss": 0.7073,
464
- "step": 65
465
- },
466
- {
467
- "epoch": 0.45,
468
- "grad_norm": 0.192199245095253,
469
- "learning_rate": 0.00019155387502843013,
470
- "loss": 0.6605,
471
- "step": 66
472
- },
473
- {
474
- "epoch": 0.46,
475
- "grad_norm": 0.2242266982793808,
476
- "learning_rate": 0.00019125404887236663,
477
- "loss": 0.7287,
478
- "step": 67
479
- },
480
- {
481
- "epoch": 0.47,
482
- "grad_norm": 0.24139907956123352,
483
- "learning_rate": 0.00019094923648925067,
484
- "loss": 0.6611,
485
- "step": 68
486
- },
487
- {
488
- "epoch": 0.47,
489
- "grad_norm": 0.22859904170036316,
490
- "learning_rate": 0.00019063945453438432,
491
- "loss": 0.6189,
492
- "step": 69
493
- },
494
- {
495
- "epoch": 0.48,
496
- "grad_norm": 0.27671098709106445,
497
- "learning_rate": 0.0001903247199346129,
498
- "loss": 0.8595,
499
- "step": 70
500
- },
501
- {
502
- "epoch": 0.49,
503
- "grad_norm": 0.23342834413051605,
504
- "learning_rate": 0.00019000504988739986,
505
- "loss": 0.6718,
506
- "step": 71
507
- },
508
- {
509
- "epoch": 0.49,
510
- "grad_norm": 0.2517896890640259,
511
- "learning_rate": 0.00018968046185988732,
512
- "loss": 0.722,
513
- "step": 72
514
- },
515
- {
516
- "epoch": 0.5,
517
- "grad_norm": 0.25217679142951965,
518
- "learning_rate": 0.00018935097358794144,
519
- "loss": 0.824,
520
- "step": 73
521
- },
522
- {
523
- "epoch": 0.51,
524
- "grad_norm": 0.2682301998138428,
525
- "learning_rate": 0.00018901660307518354,
526
- "loss": 0.7837,
527
- "step": 74
528
- },
529
- {
530
- "epoch": 0.52,
531
- "grad_norm": 0.28926676511764526,
532
- "learning_rate": 0.0001886773685920062,
533
- "loss": 0.7364,
534
- "step": 75
535
- },
536
- {
537
- "epoch": 0.52,
538
- "grad_norm": 0.25249186158180237,
539
- "learning_rate": 0.00018833328867457497,
540
- "loss": 0.8566,
541
- "step": 76
542
- },
543
- {
544
- "epoch": 0.53,
545
- "grad_norm": 0.21752771735191345,
546
- "learning_rate": 0.0001879843821238155,
547
- "loss": 0.7077,
548
- "step": 77
549
- },
550
- {
551
- "epoch": 0.54,
552
- "grad_norm": 0.22346794605255127,
553
- "learning_rate": 0.00018763066800438636,
554
- "loss": 0.6522,
555
- "step": 78
556
- },
557
- {
558
- "epoch": 0.54,
559
- "grad_norm": 0.23172937333583832,
560
- "learning_rate": 0.00018727216564363723,
561
- "loss": 0.6099,
562
- "step": 79
563
- },
564
- {
565
- "epoch": 0.55,
566
- "grad_norm": 0.20895916223526,
567
- "learning_rate": 0.00018690889463055283,
568
- "loss": 0.7242,
569
- "step": 80
570
- },
571
- {
572
- "epoch": 0.56,
573
- "grad_norm": 0.26868510246276855,
574
- "learning_rate": 0.0001865408748146826,
575
- "loss": 0.705,
576
- "step": 81
577
- },
578
- {
579
- "epoch": 0.56,
580
- "grad_norm": 0.243666410446167,
581
- "learning_rate": 0.00018616812630505597,
582
- "loss": 0.8512,
583
- "step": 82
584
- },
585
- {
586
- "epoch": 0.57,
587
- "grad_norm": 0.21445880830287933,
588
- "learning_rate": 0.00018579066946908384,
589
- "loss": 0.7092,
590
- "step": 83
591
- },
592
- {
593
- "epoch": 0.58,
594
- "grad_norm": 0.22805824875831604,
595
- "learning_rate": 0.00018540852493144545,
596
- "loss": 0.6757,
597
- "step": 84
598
- },
599
- {
600
- "epoch": 0.58,
601
- "grad_norm": 0.25106164813041687,
602
- "learning_rate": 0.00018502171357296144,
603
- "loss": 0.7171,
604
- "step": 85
605
- },
606
- {
607
- "epoch": 0.59,
608
- "grad_norm": 0.2290162742137909,
609
- "learning_rate": 0.000184630256529453,
610
- "loss": 0.7053,
611
- "step": 86
612
- },
613
- {
614
- "epoch": 0.6,
615
- "grad_norm": 0.21118655800819397,
616
- "learning_rate": 0.00018423417519058694,
617
- "loss": 0.5827,
618
- "step": 87
619
- },
620
- {
621
- "epoch": 0.6,
622
- "grad_norm": 0.23761862516403198,
623
- "learning_rate": 0.00018383349119870695,
624
- "loss": 0.5827,
625
- "step": 88
626
- },
627
- {
628
- "epoch": 0.61,
629
- "grad_norm": 0.23689566552639008,
630
- "learning_rate": 0.00018342822644765104,
631
- "loss": 0.6401,
632
- "step": 89
633
- },
634
- {
635
- "epoch": 0.62,
636
- "grad_norm": 0.26510947942733765,
637
- "learning_rate": 0.00018301840308155507,
638
- "loss": 0.5963,
639
- "step": 90
640
- },
641
- {
642
- "epoch": 0.63,
643
- "grad_norm": 0.2142990678548813,
644
- "learning_rate": 0.0001826040434936431,
645
- "loss": 0.6258,
646
- "step": 91
647
- },
648
- {
649
- "epoch": 0.63,
650
- "grad_norm": 0.20709159970283508,
651
- "learning_rate": 0.00018218517032500344,
652
- "loss": 0.5685,
653
- "step": 92
654
- },
655
- {
656
- "epoch": 0.64,
657
- "grad_norm": 0.2485310137271881,
658
- "learning_rate": 0.0001817618064633518,
659
- "loss": 0.5901,
660
- "step": 93
661
- },
662
- {
663
- "epoch": 0.65,
664
- "grad_norm": 0.2432468980550766,
665
- "learning_rate": 0.00018133397504178057,
666
- "loss": 0.6891,
667
- "step": 94
668
- },
669
- {
670
- "epoch": 0.65,
671
- "grad_norm": 0.2318205088376999,
672
- "learning_rate": 0.00018090169943749476,
673
- "loss": 0.7339,
674
- "step": 95
675
- },
676
- {
677
- "epoch": 0.66,
678
- "grad_norm": 0.25346460938453674,
679
- "learning_rate": 0.00018046500327053463,
680
- "loss": 0.6606,
681
- "step": 96
682
- },
683
- {
684
- "epoch": 0.67,
685
- "grad_norm": 0.29421111941337585,
686
- "learning_rate": 0.0001800239104024851,
687
- "loss": 0.6698,
688
- "step": 97
689
- },
690
- {
691
- "epoch": 0.67,
692
- "grad_norm": 0.2356603443622589,
693
- "learning_rate": 0.00017957844493517213,
694
- "loss": 0.6225,
695
- "step": 98
696
- },
697
- {
698
- "epoch": 0.68,
699
- "grad_norm": 0.2331649512052536,
700
- "learning_rate": 0.00017912863120934534,
701
- "loss": 0.6838,
702
- "step": 99
703
- },
704
- {
705
- "epoch": 0.69,
706
- "grad_norm": 0.23008978366851807,
707
- "learning_rate": 0.00017867449380334834,
708
- "loss": 0.6303,
709
- "step": 100
710
- },
711
- {
712
- "epoch": 0.69,
713
- "grad_norm": 0.21604081988334656,
714
- "learning_rate": 0.00017821605753177562,
715
- "loss": 0.4923,
716
- "step": 101
717
- },
718
- {
719
- "epoch": 0.7,
720
- "grad_norm": 0.2283111810684204,
721
- "learning_rate": 0.00017775334744411678,
722
- "loss": 0.6344,
723
- "step": 102
724
- },
725
- {
726
- "epoch": 0.71,
727
- "grad_norm": 0.2448730617761612,
728
- "learning_rate": 0.00017728638882338746,
729
- "loss": 0.6789,
730
- "step": 103
731
- },
732
- {
733
- "epoch": 0.71,
734
- "grad_norm": 0.22479577362537384,
735
- "learning_rate": 0.00017681520718474823,
736
- "loss": 0.5841,
737
- "step": 104
738
- },
739
- {
740
- "epoch": 0.72,
741
- "grad_norm": 0.21999216079711914,
742
- "learning_rate": 0.00017633982827411032,
743
- "loss": 0.6402,
744
- "step": 105
745
- },
746
- {
747
- "epoch": 0.73,
748
- "grad_norm": 0.2296195775270462,
749
- "learning_rate": 0.00017586027806672857,
750
- "loss": 0.6148,
751
- "step": 106
752
- },
753
- {
754
- "epoch": 0.74,
755
- "grad_norm": 0.22455835342407227,
756
- "learning_rate": 0.00017537658276578247,
757
- "loss": 0.6418,
758
- "step": 107
759
- },
760
- {
761
- "epoch": 0.74,
762
- "grad_norm": 0.23929741978645325,
763
- "learning_rate": 0.00017488876880094413,
764
- "loss": 0.5459,
765
- "step": 108
766
- },
767
- {
768
- "epoch": 0.75,
769
- "grad_norm": 0.229934424161911,
770
- "learning_rate": 0.00017439686282693436,
771
- "loss": 0.6049,
772
- "step": 109
773
- },
774
- {
775
- "epoch": 0.76,
776
- "grad_norm": 0.24801373481750488,
777
- "learning_rate": 0.00017390089172206592,
778
- "loss": 0.7663,
779
- "step": 110
780
- },
781
- {
782
- "epoch": 0.76,
783
- "grad_norm": 0.2523267865180969,
784
- "learning_rate": 0.00017340088258677522,
785
- "loss": 0.6642,
786
- "step": 111
787
- },
788
- {
789
- "epoch": 0.77,
790
- "grad_norm": 0.22252008318901062,
791
- "learning_rate": 0.00017289686274214118,
792
- "loss": 0.6325,
793
- "step": 112
794
- },
795
- {
796
- "epoch": 0.78,
797
- "grad_norm": 0.27619731426239014,
798
- "learning_rate": 0.0001723888597283926,
799
- "loss": 0.655,
800
- "step": 113
801
- },
802
- {
803
- "epoch": 0.78,
804
- "grad_norm": 0.25090277194976807,
805
- "learning_rate": 0.00017187690130340328,
806
- "loss": 0.6251,
807
- "step": 114
808
- },
809
- {
810
- "epoch": 0.79,
811
- "grad_norm": 0.2236175239086151,
812
- "learning_rate": 0.00017136101544117525,
813
- "loss": 0.5959,
814
- "step": 115
815
- },
816
- {
817
- "epoch": 0.8,
818
- "grad_norm": 0.21940240263938904,
819
- "learning_rate": 0.00017084123033031024,
820
- "loss": 0.6028,
821
- "step": 116
822
- },
823
- {
824
- "epoch": 0.8,
825
- "grad_norm": 0.2741597294807434,
826
- "learning_rate": 0.00017031757437246947,
827
- "loss": 0.4947,
828
- "step": 117
829
- },
830
- {
831
- "epoch": 0.81,
832
- "grad_norm": 0.23343035578727722,
833
- "learning_rate": 0.00016979007618082175,
834
- "loss": 0.4415,
835
- "step": 118
836
- },
837
- {
838
- "epoch": 0.82,
839
- "grad_norm": 0.2116251140832901,
840
- "learning_rate": 0.00016925876457848,
841
- "loss": 0.5283,
842
- "step": 119
843
- },
844
- {
845
- "epoch": 0.82,
846
- "grad_norm": 0.25621068477630615,
847
- "learning_rate": 0.00016872366859692627,
848
- "loss": 0.7334,
849
- "step": 120
850
- },
851
- {
852
- "epoch": 0.83,
853
- "grad_norm": 0.22588759660720825,
854
- "learning_rate": 0.00016818481747442554,
855
- "loss": 0.6514,
856
- "step": 121
857
- },
858
- {
859
- "epoch": 0.84,
860
- "grad_norm": 0.24995574355125427,
861
- "learning_rate": 0.00016764224065442796,
862
- "loss": 0.5632,
863
- "step": 122
864
- },
865
- {
866
- "epoch": 0.85,
867
- "grad_norm": 0.2382134348154068,
868
- "learning_rate": 0.00016709596778396026,
869
- "loss": 0.5954,
870
- "step": 123
871
- },
872
- {
873
- "epoch": 0.85,
874
- "grad_norm": 0.25166618824005127,
875
- "learning_rate": 0.00016654602871200546,
876
- "loss": 0.6092,
877
- "step": 124
878
- },
879
- {
880
- "epoch": 0.86,
881
- "grad_norm": 0.23028074204921722,
882
- "learning_rate": 0.0001659924534878723,
883
- "loss": 0.576,
884
- "step": 125
885
- },
886
- {
887
- "epoch": 0.87,
888
- "grad_norm": 0.2527604103088379,
889
- "learning_rate": 0.00016543527235955282,
890
- "loss": 0.5985,
891
- "step": 126
892
- },
893
- {
894
- "epoch": 0.87,
895
- "grad_norm": 0.25870686769485474,
896
- "learning_rate": 0.00016487451577207018,
897
- "loss": 0.5767,
898
- "step": 127
899
- },
900
- {
901
- "epoch": 0.88,
902
- "grad_norm": 0.24936434626579285,
903
- "learning_rate": 0.0001643102143658145,
904
- "loss": 0.7494,
905
- "step": 128
906
- },
907
- {
908
- "epoch": 0.89,
909
- "grad_norm": 0.25158020853996277,
910
- "learning_rate": 0.000163742398974869,
911
- "loss": 0.578,
912
- "step": 129
913
- },
914
- {
915
- "epoch": 0.89,
916
- "grad_norm": 0.22290346026420593,
917
- "learning_rate": 0.0001631711006253251,
918
- "loss": 0.6489,
919
- "step": 130
920
- },
921
- {
922
- "epoch": 0.9,
923
- "grad_norm": 0.26421919465065,
924
- "learning_rate": 0.00016259635053358717,
925
- "loss": 0.5669,
926
- "step": 131
927
- },
928
- {
929
- "epoch": 0.91,
930
- "grad_norm": 0.2573990225791931,
931
- "learning_rate": 0.0001620181801046667,
932
- "loss": 0.6405,
933
- "step": 132
934
- },
935
- {
936
- "epoch": 0.91,
937
- "grad_norm": 0.19883453845977783,
938
- "learning_rate": 0.00016143662093046638,
939
- "loss": 0.4731,
940
- "step": 133
941
- },
942
- {
943
- "epoch": 0.92,
944
- "grad_norm": 0.24181459844112396,
945
- "learning_rate": 0.00016085170478805395,
946
- "loss": 0.6819,
947
- "step": 134
948
- },
949
- {
950
- "epoch": 0.93,
951
- "grad_norm": 0.2319963425397873,
952
- "learning_rate": 0.00016026346363792567,
953
- "loss": 0.6373,
954
- "step": 135
955
- },
956
- {
957
- "epoch": 0.93,
958
- "grad_norm": 0.2552095055580139,
959
- "learning_rate": 0.0001596719296222601,
960
- "loss": 0.7151,
961
- "step": 136
962
- },
963
- {
964
- "epoch": 0.94,
965
- "grad_norm": 0.27191445231437683,
966
- "learning_rate": 0.00015907713506316192,
967
- "loss": 0.611,
968
- "step": 137
969
- },
970
- {
971
- "epoch": 0.95,
972
- "grad_norm": 0.26981061697006226,
973
- "learning_rate": 0.0001584791124608955,
974
- "loss": 0.5923,
975
- "step": 138
976
- },
977
- {
978
- "epoch": 0.96,
979
- "grad_norm": 0.23907916247844696,
980
- "learning_rate": 0.00015787789449210938,
981
- "loss": 0.5655,
982
- "step": 139
983
- },
984
- {
985
- "epoch": 0.96,
986
- "grad_norm": 0.23364509642124176,
987
- "learning_rate": 0.00015727351400805052,
988
- "loss": 0.6019,
989
- "step": 140
990
- },
991
- {
992
- "epoch": 0.97,
993
- "grad_norm": 0.23096178472042084,
994
- "learning_rate": 0.0001566660040327695,
995
- "loss": 0.602,
996
- "step": 141
997
- },
998
- {
999
- "epoch": 0.98,
1000
- "grad_norm": 0.2656322717666626,
1001
- "learning_rate": 0.0001560553977613158,
1002
- "loss": 0.6195,
1003
- "step": 142
1004
- },
1005
- {
1006
- "epoch": 0.98,
1007
- "grad_norm": 0.23051652312278748,
1008
- "learning_rate": 0.00015544172855792423,
1009
- "loss": 0.6132,
1010
- "step": 143
1011
- },
1012
- {
1013
- "epoch": 0.99,
1014
- "grad_norm": 0.2443581074476242,
1015
- "learning_rate": 0.00015482502995419167,
1016
- "loss": 0.6144,
1017
- "step": 144
1018
- },
1019
- {
1020
- "epoch": 1.0,
1021
- "grad_norm": 0.2075706273317337,
1022
- "learning_rate": 0.00015420533564724495,
1023
- "loss": 0.5556,
1024
- "step": 145
1025
- },
1026
- {
1027
- "epoch": 1.0,
1028
- "grad_norm": 0.23596633970737457,
1029
- "learning_rate": 0.00015358267949789966,
1030
- "loss": 0.5926,
1031
- "step": 146
1032
- },
1033
- {
1034
- "epoch": 1.01,
1035
- "grad_norm": 0.25716009736061096,
1036
- "learning_rate": 0.0001529570955288099,
1037
- "loss": 0.6569,
1038
- "step": 147
1039
- },
1040
- {
1041
- "epoch": 1.02,
1042
- "grad_norm": 0.21337899565696716,
1043
- "learning_rate": 0.0001523286179226091,
1044
- "loss": 0.5873,
1045
- "step": 148
1046
- },
1047
- {
1048
- "epoch": 1.02,
1049
- "eval_loss": 0.6392110586166382,
1050
- "eval_runtime": 57.9435,
1051
- "eval_samples_per_second": 8.974,
1052
- "eval_steps_per_second": 1.122,
1053
- "step": 148
1054
- },
1055
- {
1056
- "epoch": 1.01,
1057
- "grad_norm": 0.20899337530136108,
1058
- "learning_rate": 0.00015169728102004256,
1059
- "loss": 0.5699,
1060
- "step": 149
1061
- },
1062
- {
1063
- "epoch": 1.01,
1064
- "grad_norm": 0.2038021683692932,
1065
- "learning_rate": 0.0001510631193180907,
1066
- "loss": 0.6119,
1067
- "step": 150
1068
- },
1069
- {
1070
- "epoch": 1.02,
1071
- "grad_norm": 0.19867417216300964,
1072
- "learning_rate": 0.00015042616746808435,
1073
- "loss": 0.5437,
1074
- "step": 151
1075
- },
1076
- {
1077
- "epoch": 1.03,
1078
- "grad_norm": 0.22234468162059784,
1079
- "learning_rate": 0.00014978646027381123,
1080
- "loss": 0.461,
1081
- "step": 152
1082
- },
1083
- {
1084
- "epoch": 1.03,
1085
- "grad_norm": 0.2240789234638214,
1086
- "learning_rate": 0.00014914403268961426,
1087
- "loss": 0.5873,
1088
- "step": 153
1089
- },
1090
- {
1091
- "epoch": 1.04,
1092
- "grad_norm": 0.1992528736591339,
1093
- "learning_rate": 0.00014849891981848158,
1094
- "loss": 0.4627,
1095
- "step": 154
1096
- },
1097
- {
1098
- "epoch": 1.05,
1099
- "grad_norm": 0.23049698770046234,
1100
- "learning_rate": 0.00014785115691012864,
1101
- "loss": 0.439,
1102
- "step": 155
1103
- },
1104
- {
1105
- "epoch": 1.05,
1106
- "grad_norm": 0.22458627820014954,
1107
- "learning_rate": 0.00014720077935907196,
1108
- "loss": 0.5893,
1109
- "step": 156
1110
- },
1111
- {
1112
- "epoch": 1.06,
1113
- "grad_norm": 0.21547600626945496,
1114
- "learning_rate": 0.00014654782270269497,
1115
- "loss": 0.6106,
1116
- "step": 157
1117
- },
1118
- {
1119
- "epoch": 1.07,
1120
- "grad_norm": 0.21248772740364075,
1121
- "learning_rate": 0.00014589232261930674,
1122
- "loss": 0.5599,
1123
- "step": 158
1124
- },
1125
- {
1126
- "epoch": 1.07,
1127
- "grad_norm": 0.22486011683940887,
1128
- "learning_rate": 0.0001452343149261919,
1129
- "loss": 0.5421,
1130
- "step": 159
1131
- },
1132
- {
1133
- "epoch": 1.08,
1134
- "grad_norm": 0.2963809072971344,
1135
- "learning_rate": 0.00014457383557765386,
1136
- "loss": 0.5321,
1137
- "step": 160
1138
- },
1139
- {
1140
- "epoch": 1.09,
1141
- "grad_norm": 0.2344168871641159,
1142
- "learning_rate": 0.0001439109206630501,
1143
- "loss": 0.5582,
1144
- "step": 161
1145
- },
1146
- {
1147
- "epoch": 1.09,
1148
- "grad_norm": 0.21608802676200867,
1149
- "learning_rate": 0.0001432456064048204,
1150
- "loss": 0.5249,
1151
- "step": 162
1152
- },
1153
- {
1154
- "epoch": 1.1,
1155
- "grad_norm": 0.3204934895038605,
1156
- "learning_rate": 0.00014257792915650728,
1157
- "loss": 0.5718,
1158
- "step": 163
1159
- },
1160
- {
1161
- "epoch": 1.11,
1162
- "grad_norm": 0.2391967922449112,
1163
- "learning_rate": 0.00014190792540076986,
1164
- "loss": 0.4687,
1165
- "step": 164
1166
- },
1167
- {
1168
- "epoch": 1.12,
1169
- "grad_norm": 0.20837116241455078,
1170
- "learning_rate": 0.00014123563174739037,
1171
- "loss": 0.437,
1172
- "step": 165
1173
- },
1174
- {
1175
- "epoch": 1.12,
1176
- "grad_norm": 0.25774210691452026,
1177
- "learning_rate": 0.0001405610849312736,
1178
- "loss": 0.6387,
1179
- "step": 166
1180
- },
1181
- {
1182
- "epoch": 1.13,
1183
- "grad_norm": 0.23517344892024994,
1184
- "learning_rate": 0.00013988432181043982,
1185
- "loss": 0.5234,
1186
- "step": 167
1187
- },
1188
- {
1189
- "epoch": 1.14,
1190
- "grad_norm": 0.24470289051532745,
1191
- "learning_rate": 0.00013920537936401077,
1192
- "loss": 0.5389,
1193
- "step": 168
1194
- },
1195
- {
1196
- "epoch": 1.14,
1197
- "grad_norm": 0.289318323135376,
1198
- "learning_rate": 0.000138524294690189,
1199
- "loss": 0.5869,
1200
- "step": 169
1201
- },
1202
- {
1203
- "epoch": 1.15,
1204
- "grad_norm": 0.2550831437110901,
1205
- "learning_rate": 0.00013784110500423104,
1206
- "loss": 0.5825,
1207
- "step": 170
1208
- },
1209
- {
1210
- "epoch": 1.16,
1211
- "grad_norm": 0.24041080474853516,
1212
- "learning_rate": 0.00013715584763641345,
1213
- "loss": 0.6241,
1214
- "step": 171
1215
- },
1216
- {
1217
- "epoch": 1.16,
1218
- "grad_norm": 0.25661879777908325,
1219
- "learning_rate": 0.00013646856002999354,
1220
- "loss": 0.5429,
1221
- "step": 172
1222
- },
1223
- {
1224
- "epoch": 1.17,
1225
- "grad_norm": 0.26754602789878845,
1226
- "learning_rate": 0.00013577927973916306,
1227
- "loss": 0.6131,
1228
- "step": 173
1229
- },
1230
- {
1231
- "epoch": 1.18,
1232
- "grad_norm": 0.22897249460220337,
1233
- "learning_rate": 0.00013508804442699648,
1234
- "loss": 0.4834,
1235
- "step": 174
1236
- },
1237
- {
1238
- "epoch": 1.18,
1239
- "grad_norm": 0.22126823663711548,
1240
- "learning_rate": 0.00013439489186339282,
1241
- "loss": 0.4735,
1242
- "step": 175
1243
- },
1244
- {
1245
- "epoch": 1.19,
1246
- "grad_norm": 0.2490013688802719,
1247
- "learning_rate": 0.00013369985992301198,
1248
- "loss": 0.5248,
1249
- "step": 176
1250
- },
1251
- {
1252
- "epoch": 1.2,
1253
- "grad_norm": 0.29270628094673157,
1254
- "learning_rate": 0.00013300298658320517,
1255
- "loss": 0.643,
1256
- "step": 177
1257
- },
1258
- {
1259
- "epoch": 1.2,
1260
- "grad_norm": 0.21294419467449188,
1261
- "learning_rate": 0.00013230430992193973,
1262
- "loss": 0.4513,
1263
- "step": 178
1264
- },
1265
- {
1266
- "epoch": 1.21,
1267
- "grad_norm": 0.20743757486343384,
1268
- "learning_rate": 0.00013160386811571876,
1269
- "loss": 0.5091,
1270
- "step": 179
1271
- },
1272
- {
1273
- "epoch": 1.22,
1274
- "grad_norm": 0.2378052920103073,
1275
- "learning_rate": 0.00013090169943749476,
1276
- "loss": 0.5357,
1277
- "step": 180
1278
- },
1279
- {
1280
- "epoch": 1.23,
1281
- "grad_norm": 0.24781295657157898,
1282
- "learning_rate": 0.00013019784225457855,
1283
- "loss": 0.5801,
1284
- "step": 181
1285
- },
1286
- {
1287
- "epoch": 1.23,
1288
- "grad_norm": 0.23349957168102264,
1289
- "learning_rate": 0.00012949233502654284,
1290
- "loss": 0.5168,
1291
- "step": 182
1292
- },
1293
- {
1294
- "epoch": 1.24,
1295
- "grad_norm": 0.2096189707517624,
1296
- "learning_rate": 0.00012878521630312078,
1297
- "loss": 0.4809,
1298
- "step": 183
1299
- },
1300
- {
1301
- "epoch": 1.25,
1302
- "grad_norm": 0.21930260956287384,
1303
- "learning_rate": 0.0001280765247220993,
1304
- "loss": 0.45,
1305
- "step": 184
1306
- },
1307
- {
1308
- "epoch": 1.25,
1309
- "grad_norm": 0.28573113679885864,
1310
- "learning_rate": 0.0001273662990072083,
1311
- "loss": 0.6648,
1312
- "step": 185
1313
- },
1314
- {
1315
- "epoch": 1.26,
1316
- "grad_norm": 0.23740755021572113,
1317
- "learning_rate": 0.00012665457796600443,
1318
- "loss": 0.5403,
1319
- "step": 186
1320
- },
1321
- {
1322
- "epoch": 1.27,
1323
- "grad_norm": 0.23442810773849487,
1324
- "learning_rate": 0.0001259414004877507,
1325
- "loss": 0.5332,
1326
- "step": 187
1327
- },
1328
- {
1329
- "epoch": 1.27,
1330
- "grad_norm": 0.22515830397605896,
1331
- "learning_rate": 0.00012522680554129156,
1332
- "loss": 0.4567,
1333
- "step": 188
1334
- },
1335
- {
1336
- "epoch": 1.28,
1337
- "grad_norm": 0.23571448028087616,
1338
- "learning_rate": 0.00012451083217292357,
1339
- "loss": 0.4068,
1340
- "step": 189
1341
- },
1342
- {
1343
- "epoch": 1.29,
1344
- "grad_norm": 0.1914808303117752,
1345
- "learning_rate": 0.00012379351950426187,
1346
- "loss": 0.4618,
1347
- "step": 190
1348
- },
1349
- {
1350
- "epoch": 1.29,
1351
- "grad_norm": 0.2481367439031601,
1352
- "learning_rate": 0.0001230749067301025,
1353
- "loss": 0.4055,
1354
- "step": 191
1355
- },
1356
- {
1357
- "epoch": 1.3,
1358
- "grad_norm": 0.22704686224460602,
1359
- "learning_rate": 0.00012235503311628073,
1360
- "loss": 0.4984,
1361
- "step": 192
1362
- },
1363
- {
1364
- "epoch": 1.31,
1365
- "grad_norm": 0.2515145242214203,
1366
- "learning_rate": 0.00012163393799752565,
1367
- "loss": 0.5463,
1368
- "step": 193
1369
- },
1370
- {
1371
- "epoch": 1.31,
1372
- "grad_norm": 0.23563461005687714,
1373
- "learning_rate": 0.00012091166077531075,
1374
- "loss": 0.5143,
1375
- "step": 194
1376
- },
1377
- {
1378
- "epoch": 1.32,
1379
- "grad_norm": 0.22308993339538574,
1380
- "learning_rate": 0.00012018824091570103,
1381
- "loss": 0.5242,
1382
- "step": 195
1383
- },
1384
- {
1385
- "epoch": 1.33,
1386
- "grad_norm": 0.2774996757507324,
1387
- "learning_rate": 0.00011946371794719656,
1388
- "loss": 0.5629,
1389
- "step": 196
1390
- },
1391
- {
1392
- "epoch": 1.34,
1393
- "grad_norm": 0.20966477692127228,
1394
- "learning_rate": 0.00011873813145857249,
1395
- "loss": 0.4812,
1396
- "step": 197
1397
- },
1398
- {
1399
- "epoch": 1.34,
1400
- "grad_norm": 0.2643747925758362,
1401
- "learning_rate": 0.00011801152109671595,
1402
- "loss": 0.4194,
1403
- "step": 198
1404
- },
1405
- {
1406
- "epoch": 1.35,
1407
- "grad_norm": 0.23725077509880066,
1408
- "learning_rate": 0.00011728392656445981,
1409
- "loss": 0.5517,
1410
- "step": 199
1411
- },
1412
- {
1413
- "epoch": 1.36,
1414
- "grad_norm": 0.25400522351264954,
1415
- "learning_rate": 0.000116555387618413,
1416
- "loss": 0.5144,
1417
- "step": 200
1418
- },
1419
- {
1420
- "epoch": 1.36,
1421
- "grad_norm": 0.24393105506896973,
1422
- "learning_rate": 0.00011582594406678839,
1423
- "loss": 0.5923,
1424
- "step": 201
1425
- },
1426
- {
1427
- "epoch": 1.37,
1428
- "grad_norm": 0.25045904517173767,
1429
- "learning_rate": 0.00011509563576722753,
1430
- "loss": 0.5662,
1431
- "step": 202
1432
- },
1433
- {
1434
- "epoch": 1.38,
1435
- "grad_norm": 0.20616577565670013,
1436
- "learning_rate": 0.00011436450262462284,
1437
- "loss": 0.4704,
1438
- "step": 203
1439
- },
1440
- {
1441
- "epoch": 1.38,
1442
- "grad_norm": 0.2518722712993622,
1443
- "learning_rate": 0.00011363258458893699,
1444
- "loss": 0.5764,
1445
- "step": 204
1446
- },
1447
- {
1448
- "epoch": 1.39,
1449
- "grad_norm": 0.2739958167076111,
1450
- "learning_rate": 0.00011289992165302035,
1451
- "loss": 0.5376,
1452
- "step": 205
1453
- },
1454
- {
1455
- "epoch": 1.4,
1456
- "grad_norm": 0.2688806354999542,
1457
- "learning_rate": 0.00011216655385042525,
1458
- "loss": 0.5916,
1459
- "step": 206
1460
- },
1461
- {
1462
- "epoch": 1.4,
1463
- "grad_norm": 0.22472062706947327,
1464
- "learning_rate": 0.00011143252125321892,
1465
- "loss": 0.4668,
1466
- "step": 207
1467
- },
1468
- {
1469
- "epoch": 1.41,
1470
- "grad_norm": 0.2582922577857971,
1471
- "learning_rate": 0.00011069786396979367,
1472
- "loss": 0.6268,
1473
- "step": 208
1474
- },
1475
- {
1476
- "epoch": 1.42,
1477
- "grad_norm": 0.2291630357503891,
1478
- "learning_rate": 0.0001099626221426754,
1479
- "loss": 0.498,
1480
- "step": 209
1481
- },
1482
- {
1483
- "epoch": 1.42,
1484
- "grad_norm": 0.24377630650997162,
1485
- "learning_rate": 0.00010922683594633021,
1486
- "loss": 0.5157,
1487
- "step": 210
1488
- },
1489
- {
1490
- "epoch": 1.43,
1491
- "grad_norm": 0.2518634498119354,
1492
- "learning_rate": 0.00010849054558496905,
1493
- "loss": 0.4786,
1494
- "step": 211
1495
- },
1496
- {
1497
- "epoch": 1.44,
1498
- "grad_norm": 0.23311232030391693,
1499
- "learning_rate": 0.00010775379129035116,
1500
- "loss": 0.4653,
1501
- "step": 212
1502
- },
1503
- {
1504
- "epoch": 1.45,
1505
- "grad_norm": 0.3206501007080078,
1506
- "learning_rate": 0.00010701661331958553,
1507
- "loss": 0.4863,
1508
- "step": 213
1509
- },
1510
- {
1511
- "epoch": 1.45,
1512
- "grad_norm": 0.23678985238075256,
1513
- "learning_rate": 0.00010627905195293135,
1514
- "loss": 0.5854,
1515
- "step": 214
1516
- },
1517
- {
1518
- "epoch": 1.46,
1519
- "grad_norm": 0.2651495933532715,
1520
- "learning_rate": 0.000105541147491597,
1521
- "loss": 0.4881,
1522
- "step": 215
1523
- },
1524
- {
1525
- "epoch": 1.47,
1526
- "grad_norm": 0.2638210952281952,
1527
- "learning_rate": 0.00010480294025553798,
1528
- "loss": 0.4984,
1529
- "step": 216
1530
- },
1531
- {
1532
- "epoch": 1.47,
1533
- "grad_norm": 0.28642264008522034,
1534
- "learning_rate": 0.00010406447058125368,
1535
- "loss": 0.6304,
1536
- "step": 217
1537
- },
1538
- {
1539
- "epoch": 1.48,
1540
- "grad_norm": 0.29784247279167175,
1541
- "learning_rate": 0.0001033257788195835,
1542
- "loss": 0.6045,
1543
- "step": 218
1544
- },
1545
- {
1546
- "epoch": 1.49,
1547
- "grad_norm": 0.28990456461906433,
1548
- "learning_rate": 0.0001025869053335019,
1549
- "loss": 0.6472,
1550
- "step": 219
1551
- },
1552
- {
1553
- "epoch": 1.49,
1554
- "grad_norm": 0.2643013894557953,
1555
- "learning_rate": 0.00010184789049591299,
1556
- "loss": 0.5676,
1557
- "step": 220
1558
- },
1559
- {
1560
- "epoch": 1.5,
1561
- "grad_norm": 0.23880311846733093,
1562
- "learning_rate": 0.0001011087746874445,
1563
- "loss": 0.4708,
1564
- "step": 221
1565
- },
1566
- {
1567
- "epoch": 1.51,
1568
- "grad_norm": 0.2558128833770752,
1569
- "learning_rate": 0.00010036959829424131,
1570
- "loss": 0.5367,
1571
- "step": 222
1572
- },
1573
- {
1574
- "epoch": 1.51,
1575
- "grad_norm": 0.2300223559141159,
1576
- "learning_rate": 9.96304017057587e-05,
1577
- "loss": 0.4626,
1578
- "step": 223
1579
- },
1580
- {
1581
- "epoch": 1.52,
1582
- "grad_norm": 0.2416905015707016,
1583
- "learning_rate": 9.889122531255552e-05,
1584
- "loss": 0.4464,
1585
- "step": 224
1586
- },
1587
- {
1588
- "epoch": 1.53,
1589
- "grad_norm": 0.26283279061317444,
1590
- "learning_rate": 9.815210950408704e-05,
1591
- "loss": 0.6352,
1592
- "step": 225
1593
- },
1594
- {
1595
- "epoch": 1.53,
1596
- "grad_norm": 0.24686123430728912,
1597
- "learning_rate": 9.741309466649813e-05,
1598
- "loss": 0.5295,
1599
- "step": 226
1600
- },
1601
- {
1602
- "epoch": 1.54,
1603
- "grad_norm": 0.2653113305568695,
1604
- "learning_rate": 9.667422118041651e-05,
1605
- "loss": 0.4808,
1606
- "step": 227
1607
- },
1608
- {
1609
- "epoch": 1.55,
1610
- "grad_norm": 0.26245051622390747,
1611
- "learning_rate": 9.593552941874635e-05,
1612
- "loss": 0.542,
1613
- "step": 228
1614
- },
1615
- {
1616
- "epoch": 1.55,
1617
- "grad_norm": 0.29881128668785095,
1618
- "learning_rate": 9.519705974446207e-05,
1619
- "loss": 0.6013,
1620
- "step": 229
1621
- },
1622
- {
1623
- "epoch": 1.56,
1624
- "grad_norm": 0.2243887037038803,
1625
- "learning_rate": 9.4458852508403e-05,
1626
- "loss": 0.474,
1627
- "step": 230
1628
- },
1629
- {
1630
- "epoch": 1.57,
1631
- "grad_norm": 0.2504364252090454,
1632
- "learning_rate": 9.372094804706867e-05,
1633
- "loss": 0.472,
1634
- "step": 231
1635
- },
1636
- {
1637
- "epoch": 1.58,
1638
- "grad_norm": 0.27919840812683105,
1639
- "learning_rate": 9.298338668041451e-05,
1640
- "loss": 0.4604,
1641
- "step": 232
1642
- },
1643
- {
1644
- "epoch": 1.58,
1645
- "grad_norm": 0.23338647186756134,
1646
- "learning_rate": 9.224620870964886e-05,
1647
- "loss": 0.4342,
1648
- "step": 233
1649
- },
1650
- {
1651
- "epoch": 1.59,
1652
- "grad_norm": 0.2783346474170685,
1653
- "learning_rate": 9.150945441503093e-05,
1654
- "loss": 0.5666,
1655
- "step": 234
1656
- },
1657
- {
1658
- "epoch": 1.6,
1659
- "grad_norm": 0.2518472373485565,
1660
- "learning_rate": 9.077316405366981e-05,
1661
- "loss": 0.4972,
1662
- "step": 235
1663
- },
1664
- {
1665
- "epoch": 1.6,
1666
- "grad_norm": 0.2644898295402527,
1667
- "learning_rate": 9.00373778573246e-05,
1668
- "loss": 0.4328,
1669
- "step": 236
1670
- },
1671
- {
1672
- "epoch": 1.61,
1673
- "grad_norm": 0.34149104356765747,
1674
- "learning_rate": 8.930213603020638e-05,
1675
- "loss": 0.6514,
1676
- "step": 237
1677
- },
1678
- {
1679
- "epoch": 1.62,
1680
- "grad_norm": 0.2584168612957001,
1681
- "learning_rate": 8.85674787467811e-05,
1682
- "loss": 0.5692,
1683
- "step": 238
1684
- },
1685
- {
1686
- "epoch": 1.62,
1687
- "grad_norm": 0.22137361764907837,
1688
- "learning_rate": 8.783344614957477e-05,
1689
- "loss": 0.5334,
1690
- "step": 239
1691
- },
1692
- {
1693
- "epoch": 1.63,
1694
- "grad_norm": 0.26125484704971313,
1695
- "learning_rate": 8.710007834697969e-05,
1696
- "loss": 0.5633,
1697
- "step": 240
1698
- },
1699
- {
1700
- "epoch": 1.64,
1701
- "grad_norm": 0.263741672039032,
1702
- "learning_rate": 8.636741541106299e-05,
1703
- "loss": 0.525,
1704
- "step": 241
1705
- },
1706
- {
1707
- "epoch": 1.64,
1708
- "grad_norm": 0.2840401232242584,
1709
- "learning_rate": 8.563549737537719e-05,
1710
- "loss": 0.5634,
1711
- "step": 242
1712
- },
1713
- {
1714
- "epoch": 1.65,
1715
- "grad_norm": 0.25339609384536743,
1716
- "learning_rate": 8.490436423277248e-05,
1717
- "loss": 0.5634,
1718
- "step": 243
1719
- },
1720
- {
1721
- "epoch": 1.66,
1722
- "grad_norm": 0.26044854521751404,
1723
- "learning_rate": 8.417405593321163e-05,
1724
- "loss": 0.5488,
1725
- "step": 244
1726
- },
1727
- {
1728
- "epoch": 1.66,
1729
- "grad_norm": 0.23519276082515717,
1730
- "learning_rate": 8.344461238158699e-05,
1731
- "loss": 0.4145,
1732
- "step": 245
1733
- },
1734
- {
1735
- "epoch": 1.67,
1736
- "grad_norm": 0.2189461588859558,
1737
- "learning_rate": 8.271607343554021e-05,
1738
- "loss": 0.518,
1739
- "step": 246
1740
- },
1741
- {
1742
- "epoch": 1.68,
1743
- "grad_norm": 0.2877447009086609,
1744
- "learning_rate": 8.198847890328406e-05,
1745
- "loss": 0.4556,
1746
- "step": 247
1747
- },
1748
- {
1749
- "epoch": 1.69,
1750
- "grad_norm": 0.2689398229122162,
1751
- "learning_rate": 8.126186854142752e-05,
1752
- "loss": 0.4426,
1753
- "step": 248
1754
- },
1755
- {
1756
- "epoch": 1.69,
1757
- "grad_norm": 0.25999996066093445,
1758
- "learning_rate": 8.053628205280347e-05,
1759
- "loss": 0.4472,
1760
- "step": 249
1761
- },
1762
- {
1763
- "epoch": 1.7,
1764
- "grad_norm": 0.2660202980041504,
1765
- "learning_rate": 7.9811759084299e-05,
1766
- "loss": 0.4789,
1767
- "step": 250
1768
- },
1769
- {
1770
- "epoch": 1.71,
1771
- "grad_norm": 0.2570638060569763,
1772
- "learning_rate": 7.908833922468927e-05,
1773
- "loss": 0.4245,
1774
- "step": 251
1775
- },
1776
- {
1777
- "epoch": 1.71,
1778
- "grad_norm": 0.22138474881649017,
1779
- "learning_rate": 7.836606200247436e-05,
1780
- "loss": 0.4377,
1781
- "step": 252
1782
- },
1783
- {
1784
- "epoch": 1.72,
1785
- "grad_norm": 0.2643532454967499,
1786
- "learning_rate": 7.764496688371929e-05,
1787
- "loss": 0.5689,
1788
- "step": 253
1789
- },
1790
- {
1791
- "epoch": 1.73,
1792
- "grad_norm": 0.3166361153125763,
1793
- "learning_rate": 7.692509326989753e-05,
1794
- "loss": 0.559,
1795
- "step": 254
1796
- },
1797
- {
1798
- "epoch": 1.73,
1799
- "grad_norm": 0.26772719621658325,
1800
- "learning_rate": 7.620648049573815e-05,
1801
- "loss": 0.5051,
1802
- "step": 255
1803
- },
1804
- {
1805
- "epoch": 1.74,
1806
- "grad_norm": 0.31207817792892456,
1807
- "learning_rate": 7.548916782707642e-05,
1808
- "loss": 0.5906,
1809
- "step": 256
1810
- },
1811
- {
1812
- "epoch": 1.75,
1813
- "grad_norm": 0.3109002113342285,
1814
- "learning_rate": 7.477319445870845e-05,
1815
- "loss": 0.5926,
1816
- "step": 257
1817
- },
1818
- {
1819
- "epoch": 1.75,
1820
- "grad_norm": 0.24690654873847961,
1821
- "learning_rate": 7.405859951224933e-05,
1822
- "loss": 0.5132,
1823
- "step": 258
1824
- },
1825
- {
1826
- "epoch": 1.76,
1827
- "grad_norm": 0.2552764117717743,
1828
- "learning_rate": 7.33454220339956e-05,
1829
- "loss": 0.6071,
1830
- "step": 259
1831
- },
1832
- {
1833
- "epoch": 1.77,
1834
- "grad_norm": 0.22326481342315674,
1835
- "learning_rate": 7.263370099279172e-05,
1836
- "loss": 0.5135,
1837
- "step": 260
1838
- },
1839
- {
1840
- "epoch": 1.77,
1841
- "grad_norm": 0.22871443629264832,
1842
- "learning_rate": 7.192347527790073e-05,
1843
- "loss": 0.4972,
1844
- "step": 261
1845
- },
1846
- {
1847
- "epoch": 1.78,
1848
- "grad_norm": 0.25344693660736084,
1849
- "learning_rate": 7.121478369687926e-05,
1850
- "loss": 0.4668,
1851
- "step": 262
1852
- },
1853
- {
1854
- "epoch": 1.79,
1855
- "grad_norm": 0.24425604939460754,
1856
- "learning_rate": 7.050766497345714e-05,
1857
- "loss": 0.5499,
1858
- "step": 263
1859
- },
1860
- {
1861
- "epoch": 1.8,
1862
- "grad_norm": 0.24569104611873627,
1863
- "learning_rate": 6.980215774542147e-05,
1864
- "loss": 0.5668,
1865
- "step": 264
1866
- },
1867
- {
1868
- "epoch": 1.8,
1869
- "grad_norm": 0.20475324988365173,
1870
- "learning_rate": 6.909830056250527e-05,
1871
- "loss": 0.5205,
1872
- "step": 265
1873
- },
1874
- {
1875
- "epoch": 1.81,
1876
- "grad_norm": 0.2739812731742859,
1877
- "learning_rate": 6.839613188428126e-05,
1878
- "loss": 0.4883,
1879
- "step": 266
1880
- },
1881
- {
1882
- "epoch": 1.82,
1883
- "grad_norm": 0.24493646621704102,
1884
- "learning_rate": 6.769569007806027e-05,
1885
- "loss": 0.5187,
1886
- "step": 267
1887
- },
1888
- {
1889
- "epoch": 1.82,
1890
- "grad_norm": 0.25336140394210815,
1891
- "learning_rate": 6.699701341679488e-05,
1892
- "loss": 0.5467,
1893
- "step": 268
1894
- },
1895
- {
1896
- "epoch": 1.83,
1897
- "grad_norm": 0.3002632260322571,
1898
- "learning_rate": 6.630014007698807e-05,
1899
- "loss": 0.613,
1900
- "step": 269
1901
- },
1902
- {
1903
- "epoch": 1.84,
1904
- "grad_norm": 0.23223187029361725,
1905
- "learning_rate": 6.560510813660719e-05,
1906
- "loss": 0.4893,
1907
- "step": 270
1908
- },
1909
- {
1910
- "epoch": 1.84,
1911
- "grad_norm": 0.22384615242481232,
1912
- "learning_rate": 6.491195557300353e-05,
1913
- "loss": 0.4206,
1914
- "step": 271
1915
- },
1916
- {
1917
- "epoch": 1.85,
1918
- "grad_norm": 0.2999289333820343,
1919
- "learning_rate": 6.422072026083697e-05,
1920
- "loss": 0.5848,
1921
- "step": 272
1922
- },
1923
- {
1924
- "epoch": 1.86,
1925
- "grad_norm": 0.230120450258255,
1926
- "learning_rate": 6.35314399700065e-05,
1927
- "loss": 0.4872,
1928
- "step": 273
1929
- },
1930
- {
1931
- "epoch": 1.86,
1932
- "grad_norm": 0.2404165416955948,
1933
- "learning_rate": 6.284415236358653e-05,
1934
- "loss": 0.5125,
1935
- "step": 274
1936
- },
1937
- {
1938
- "epoch": 1.87,
1939
- "grad_norm": 0.2641153037548065,
1940
- "learning_rate": 6.215889499576898e-05,
1941
- "loss": 0.5132,
1942
- "step": 275
1943
- },
1944
- {
1945
- "epoch": 1.88,
1946
- "grad_norm": 0.2617990970611572,
1947
- "learning_rate": 6.147570530981099e-05,
1948
- "loss": 0.4281,
1949
- "step": 276
1950
- },
1951
- {
1952
- "epoch": 1.88,
1953
- "grad_norm": 0.2755200266838074,
1954
- "learning_rate": 6.0794620635989244e-05,
1955
- "loss": 0.4031,
1956
- "step": 277
1957
- },
1958
- {
1959
- "epoch": 1.89,
1960
- "grad_norm": 0.32721757888793945,
1961
- "learning_rate": 6.011567818956021e-05,
1962
- "loss": 0.4956,
1963
- "step": 278
1964
- },
1965
- {
1966
- "epoch": 1.9,
1967
- "grad_norm": 0.3732713758945465,
1968
- "learning_rate": 5.943891506872645e-05,
1969
- "loss": 0.5509,
1970
- "step": 279
1971
- },
1972
- {
1973
- "epoch": 1.91,
1974
- "grad_norm": 0.27332013845443726,
1975
- "learning_rate": 5.876436825260967e-05,
1976
- "loss": 0.5633,
1977
- "step": 280
1978
- },
1979
- {
1980
- "epoch": 1.91,
1981
- "grad_norm": 0.23139792680740356,
1982
- "learning_rate": 5.809207459923016e-05,
1983
- "loss": 0.3776,
1984
- "step": 281
1985
- },
1986
- {
1987
- "epoch": 1.92,
1988
- "grad_norm": 0.29461902379989624,
1989
- "learning_rate": 5.7422070843492734e-05,
1990
- "loss": 0.438,
1991
- "step": 282
1992
- },
1993
- {
1994
- "epoch": 1.93,
1995
- "grad_norm": 0.27846580743789673,
1996
- "learning_rate": 5.675439359517962e-05,
1997
- "loss": 0.4784,
1998
- "step": 283
1999
- },
2000
- {
2001
- "epoch": 1.93,
2002
- "grad_norm": 0.2910459637641907,
2003
- "learning_rate": 5.608907933694994e-05,
2004
- "loss": 0.5903,
2005
- "step": 284
2006
- },
2007
- {
2008
- "epoch": 1.94,
2009
- "grad_norm": 0.2746853828430176,
2010
- "learning_rate": 5.542616442234618e-05,
2011
- "loss": 0.4604,
2012
- "step": 285
2013
- },
2014
- {
2015
- "epoch": 1.95,
2016
- "grad_norm": 0.2670218348503113,
2017
- "learning_rate": 5.476568507380815e-05,
2018
- "loss": 0.4851,
2019
- "step": 286
2020
- },
2021
- {
2022
- "epoch": 1.95,
2023
- "grad_norm": 0.289640873670578,
2024
- "learning_rate": 5.410767738069328e-05,
2025
- "loss": 0.4749,
2026
- "step": 287
2027
- },
2028
- {
2029
- "epoch": 1.96,
2030
- "grad_norm": 0.2754352390766144,
2031
- "learning_rate": 5.345217729730501e-05,
2032
- "loss": 0.5822,
2033
- "step": 288
2034
- },
2035
- {
2036
- "epoch": 1.97,
2037
- "grad_norm": 0.29798761010169983,
2038
- "learning_rate": 5.279922064092808e-05,
2039
- "loss": 0.5244,
2040
- "step": 289
2041
- },
2042
- {
2043
- "epoch": 1.97,
2044
- "grad_norm": 0.2856166660785675,
2045
- "learning_rate": 5.214884308987136e-05,
2046
- "loss": 0.5525,
2047
- "step": 290
2048
- },
2049
- {
2050
- "epoch": 1.98,
2051
- "grad_norm": 0.2690298557281494,
2052
- "learning_rate": 5.150108018151845e-05,
2053
- "loss": 0.5414,
2054
- "step": 291
2055
- },
2056
- {
2057
- "epoch": 1.99,
2058
- "grad_norm": 0.29631486535072327,
2059
- "learning_rate": 5.0855967310385776e-05,
2060
- "loss": 0.5426,
2061
- "step": 292
2062
- },
2063
- {
2064
- "epoch": 1.99,
2065
- "grad_norm": 0.2700301706790924,
2066
- "learning_rate": 5.021353972618877e-05,
2067
- "loss": 0.5618,
2068
- "step": 293
2069
- },
2070
- {
2071
- "epoch": 2.0,
2072
- "grad_norm": 0.283239483833313,
2073
- "learning_rate": 4.957383253191567e-05,
2074
- "loss": 0.5658,
2075
- "step": 294
2076
- },
2077
- {
2078
- "epoch": 2.01,
2079
- "grad_norm": 0.33372044563293457,
2080
- "learning_rate": 4.893688068190932e-05,
2081
- "loss": 0.6722,
2082
- "step": 295
2083
- },
2084
- {
2085
- "epoch": 2.02,
2086
- "grad_norm": 0.28842851519584656,
2087
- "learning_rate": 4.8302718979957465e-05,
2088
- "loss": 0.4513,
2089
- "step": 296
2090
- },
2091
- {
2092
- "epoch": 2.02,
2093
- "eval_loss": 0.6006384491920471,
2094
- "eval_runtime": 57.9742,
2095
- "eval_samples_per_second": 8.97,
2096
- "eval_steps_per_second": 1.121,
2097
- "step": 296
2098
- },
2099
- {
2100
- "epoch": 2.0,
2101
- "grad_norm": 0.2431992143392563,
2102
- "learning_rate": 4.7671382077390923e-05,
2103
- "loss": 0.4022,
2104
- "step": 297
2105
- },
2106
- {
2107
- "epoch": 2.01,
2108
- "grad_norm": 0.20665951073169708,
2109
- "learning_rate": 4.704290447119013e-05,
2110
- "loss": 0.3693,
2111
- "step": 298
2112
- },
2113
- {
2114
- "epoch": 2.02,
2115
- "grad_norm": 0.21755042672157288,
2116
- "learning_rate": 4.6417320502100316e-05,
2117
- "loss": 0.4507,
2118
- "step": 299
2119
- },
2120
- {
2121
- "epoch": 2.02,
2122
- "grad_norm": 0.25068557262420654,
2123
- "learning_rate": 4.5794664352755055e-05,
2124
- "loss": 0.4027,
2125
- "step": 300
2126
- },
2127
- {
2128
- "epoch": 2.03,
2129
- "grad_norm": 0.21365563571453094,
2130
- "learning_rate": 4.5174970045808373e-05,
2131
- "loss": 0.447,
2132
- "step": 301
2133
- },
2134
- {
2135
- "epoch": 2.04,
2136
- "grad_norm": 0.20687268674373627,
2137
- "learning_rate": 4.4558271442075817e-05,
2138
- "loss": 0.3979,
2139
- "step": 302
2140
- },
2141
- {
2142
- "epoch": 2.04,
2143
- "grad_norm": 0.24014657735824585,
2144
- "learning_rate": 4.394460223868422e-05,
2145
- "loss": 0.494,
2146
- "step": 303
2147
- },
2148
- {
2149
- "epoch": 2.05,
2150
- "grad_norm": 0.20596379041671753,
2151
- "learning_rate": 4.333399596723054e-05,
2152
- "loss": 0.3708,
2153
- "step": 304
2154
- },
2155
- {
2156
- "epoch": 2.06,
2157
- "grad_norm": 0.23077203333377838,
2158
- "learning_rate": 4.272648599194948e-05,
2159
- "loss": 0.4383,
2160
- "step": 305
2161
- },
2162
- {
2163
- "epoch": 2.07,
2164
- "grad_norm": 0.2302219718694687,
2165
- "learning_rate": 4.212210550789066e-05,
2166
- "loss": 0.437,
2167
- "step": 306
2168
- },
2169
- {
2170
- "epoch": 2.07,
2171
- "grad_norm": 0.23546743392944336,
2172
- "learning_rate": 4.1520887539104516e-05,
2173
- "loss": 0.5266,
2174
- "step": 307
2175
- },
2176
- {
2177
- "epoch": 2.08,
2178
- "grad_norm": 0.20262156426906586,
2179
- "learning_rate": 4.092286493683812e-05,
2180
- "loss": 0.4039,
2181
- "step": 308
2182
- },
2183
- {
2184
- "epoch": 2.09,
2185
- "grad_norm": 0.2310255765914917,
2186
- "learning_rate": 4.0328070377739936e-05,
2187
- "loss": 0.4723,
2188
- "step": 309
2189
- },
2190
- {
2191
- "epoch": 2.09,
2192
- "grad_norm": 0.22382591664791107,
2193
- "learning_rate": 3.973653636207437e-05,
2194
- "loss": 0.4599,
2195
- "step": 310
2196
- },
2197
- {
2198
- "epoch": 2.1,
2199
- "grad_norm": 0.2400328814983368,
2200
- "learning_rate": 3.914829521194606e-05,
2201
- "loss": 0.4837,
2202
- "step": 311
2203
- },
2204
- {
2205
- "epoch": 2.11,
2206
- "grad_norm": 0.23558029532432556,
2207
- "learning_rate": 3.8563379069533626e-05,
2208
- "loss": 0.4706,
2209
- "step": 312
2210
- },
2211
- {
2212
- "epoch": 2.11,
2213
- "grad_norm": 0.24073629081249237,
2214
- "learning_rate": 3.7981819895333336e-05,
2215
- "loss": 0.4662,
2216
- "step": 313
2217
- },
2218
- {
2219
- "epoch": 2.12,
2220
- "grad_norm": 0.23906177282333374,
2221
- "learning_rate": 3.740364946641284e-05,
2222
- "loss": 0.4566,
2223
- "step": 314
2224
- },
2225
- {
2226
- "epoch": 2.13,
2227
- "grad_norm": 0.23790515959262848,
2228
- "learning_rate": 3.682889937467493e-05,
2229
- "loss": 0.4365,
2230
- "step": 315
2231
- },
2232
- {
2233
- "epoch": 2.13,
2234
- "grad_norm": 0.2540919780731201,
2235
- "learning_rate": 3.6257601025131026e-05,
2236
- "loss": 0.4546,
2237
- "step": 316
2238
- },
2239
- {
2240
- "epoch": 2.14,
2241
- "grad_norm": 0.28762128949165344,
2242
- "learning_rate": 3.568978563418551e-05,
2243
- "loss": 0.4624,
2244
- "step": 317
2245
- },
2246
- {
2247
- "epoch": 2.15,
2248
- "grad_norm": 0.24216574430465698,
2249
- "learning_rate": 3.512548422792983e-05,
2250
- "loss": 0.4387,
2251
- "step": 318
2252
- },
2253
- {
2254
- "epoch": 2.15,
2255
- "grad_norm": 0.2672511637210846,
2256
- "learning_rate": 3.456472764044718e-05,
2257
- "loss": 0.503,
2258
- "step": 319
2259
- },
2260
- {
2261
- "epoch": 2.16,
2262
- "grad_norm": 0.26805609464645386,
2263
- "learning_rate": 3.400754651212776e-05,
2264
- "loss": 0.5428,
2265
- "step": 320
2266
- },
2267
- {
2268
- "epoch": 2.17,
2269
- "grad_norm": 0.2592112123966217,
2270
- "learning_rate": 3.3453971287994545e-05,
2271
- "loss": 0.4206,
2272
- "step": 321
2273
- },
2274
- {
2275
- "epoch": 2.18,
2276
- "grad_norm": 0.25570839643478394,
2277
- "learning_rate": 3.290403221603976e-05,
2278
- "loss": 0.5237,
2279
- "step": 322
2280
- },
2281
- {
2282
- "epoch": 2.18,
2283
- "grad_norm": 0.2656814157962799,
2284
- "learning_rate": 3.235775934557204e-05,
2285
- "loss": 0.4655,
2286
- "step": 323
2287
- },
2288
- {
2289
- "epoch": 2.19,
2290
- "grad_norm": 0.2883979380130768,
2291
- "learning_rate": 3.1815182525574495e-05,
2292
- "loss": 0.5198,
2293
- "step": 324
2294
- },
2295
- {
2296
- "epoch": 2.2,
2297
- "grad_norm": 0.35970887541770935,
2298
- "learning_rate": 3.1276331403073735e-05,
2299
- "loss": 0.4448,
2300
- "step": 325
2301
- },
2302
- {
2303
- "epoch": 2.2,
2304
- "grad_norm": 0.25861579179763794,
2305
- "learning_rate": 3.074123542152001e-05,
2306
- "loss": 0.5068,
2307
- "step": 326
2308
- },
2309
- {
2310
- "epoch": 2.21,
2311
- "grad_norm": 0.2832384407520294,
2312
- "learning_rate": 3.020992381917823e-05,
2313
- "loss": 0.4679,
2314
- "step": 327
2315
- },
2316
- {
2317
- "epoch": 2.22,
2318
- "grad_norm": 0.2728095054626465,
2319
- "learning_rate": 2.968242562753051e-05,
2320
- "loss": 0.4211,
2321
- "step": 328
2322
- },
2323
- {
2324
- "epoch": 2.22,
2325
- "grad_norm": 0.27320149540901184,
2326
- "learning_rate": 2.915876966968978e-05,
2327
- "loss": 0.5132,
2328
- "step": 329
2329
- },
2330
- {
2331
- "epoch": 2.23,
2332
- "grad_norm": 0.24840450286865234,
2333
- "learning_rate": 2.8638984558824777e-05,
2334
- "loss": 0.4337,
2335
- "step": 330
2336
- },
2337
- {
2338
- "epoch": 2.24,
2339
- "grad_norm": 0.24390266835689545,
2340
- "learning_rate": 2.812309869659675e-05,
2341
- "loss": 0.4411,
2342
- "step": 331
2343
- },
2344
- {
2345
- "epoch": 2.24,
2346
- "grad_norm": 0.23822544515132904,
2347
- "learning_rate": 2.7611140271607417e-05,
2348
- "loss": 0.4178,
2349
- "step": 332
2350
- },
2351
- {
2352
- "epoch": 2.25,
2353
- "grad_norm": 0.2783829867839813,
2354
- "learning_rate": 2.7103137257858868e-05,
2355
- "loss": 0.4428,
2356
- "step": 333
2357
- },
2358
- {
2359
- "epoch": 2.26,
2360
- "grad_norm": 0.2625970244407654,
2361
- "learning_rate": 2.6599117413224817e-05,
2362
- "loss": 0.4076,
2363
- "step": 334
2364
- },
2365
- {
2366
- "epoch": 2.26,
2367
- "grad_norm": 0.2514960765838623,
2368
- "learning_rate": 2.6099108277934103e-05,
2369
- "loss": 0.3783,
2370
- "step": 335
2371
- },
2372
- {
2373
- "epoch": 2.27,
2374
- "grad_norm": 0.2869970202445984,
2375
- "learning_rate": 2.5603137173065674e-05,
2376
- "loss": 0.3544,
2377
- "step": 336
2378
- },
2379
- {
2380
- "epoch": 2.28,
2381
- "grad_norm": 0.24134644865989685,
2382
- "learning_rate": 2.5111231199055896e-05,
2383
- "loss": 0.3664,
2384
- "step": 337
2385
- },
2386
- {
2387
- "epoch": 2.29,
2388
- "grad_norm": 0.30167311429977417,
2389
- "learning_rate": 2.462341723421758e-05,
2390
- "loss": 0.4425,
2391
- "step": 338
2392
- },
2393
- {
2394
- "epoch": 2.29,
2395
- "grad_norm": 0.24307532608509064,
2396
- "learning_rate": 2.4139721933271465e-05,
2397
- "loss": 0.4078,
2398
- "step": 339
2399
- },
2400
- {
2401
- "epoch": 2.3,
2402
- "grad_norm": 0.2623879015445709,
2403
- "learning_rate": 2.36601717258897e-05,
2404
- "loss": 0.4477,
2405
- "step": 340
2406
- },
2407
- {
2408
- "epoch": 2.31,
2409
- "grad_norm": 0.2870636582374573,
2410
- "learning_rate": 2.3184792815251766e-05,
2411
- "loss": 0.5101,
2412
- "step": 341
2413
- },
2414
- {
2415
- "epoch": 2.31,
2416
- "grad_norm": 0.36139050126075745,
2417
- "learning_rate": 2.2713611176612582e-05,
2418
- "loss": 0.3755,
2419
- "step": 342
2420
- },
2421
- {
2422
- "epoch": 2.32,
2423
- "grad_norm": 0.2712129056453705,
2424
- "learning_rate": 2.224665255588325e-05,
2425
- "loss": 0.4178,
2426
- "step": 343
2427
- },
2428
- {
2429
- "epoch": 2.33,
2430
- "grad_norm": 0.2709867060184479,
2431
- "learning_rate": 2.1783942468224382e-05,
2432
- "loss": 0.4848,
2433
- "step": 344
2434
- },
2435
- {
2436
- "epoch": 2.33,
2437
- "grad_norm": 0.24954509735107422,
2438
- "learning_rate": 2.132550619665168e-05,
2439
- "loss": 0.4352,
2440
- "step": 345
2441
- },
2442
- {
2443
- "epoch": 2.34,
2444
- "grad_norm": 0.23785164952278137,
2445
- "learning_rate": 2.08713687906547e-05,
2446
- "loss": 0.4731,
2447
- "step": 346
2448
- },
2449
- {
2450
- "epoch": 2.35,
2451
- "grad_norm": 0.22254765033721924,
2452
- "learning_rate": 2.0421555064827878e-05,
2453
- "loss": 0.3899,
2454
- "step": 347
2455
- },
2456
- {
2457
- "epoch": 2.35,
2458
- "grad_norm": 0.2513158619403839,
2459
- "learning_rate": 1.9976089597514903e-05,
2460
- "loss": 0.3748,
2461
- "step": 348
2462
- },
2463
- {
2464
- "epoch": 2.36,
2465
- "grad_norm": 0.26037031412124634,
2466
- "learning_rate": 1.9534996729465426e-05,
2467
- "loss": 0.4183,
2468
- "step": 349
2469
- },
2470
- {
2471
- "epoch": 2.37,
2472
- "grad_norm": 0.2564360499382019,
2473
- "learning_rate": 1.9098300562505266e-05,
2474
- "loss": 0.4153,
2475
- "step": 350
2476
- },
2477
- {
2478
- "epoch": 2.37,
2479
- "grad_norm": 0.2211247682571411,
2480
- "learning_rate": 1.8666024958219408e-05,
2481
- "loss": 0.4147,
2482
- "step": 351
2483
- },
2484
- {
2485
- "epoch": 2.38,
2486
- "grad_norm": 0.265408992767334,
2487
- "learning_rate": 1.8238193536648195e-05,
2488
- "loss": 0.4655,
2489
- "step": 352
2490
- },
2491
- {
2492
- "epoch": 2.39,
2493
- "grad_norm": 0.24408482015132904,
2494
- "learning_rate": 1.7814829674996592e-05,
2495
- "loss": 0.4488,
2496
- "step": 353
2497
- },
2498
- {
2499
- "epoch": 2.4,
2500
- "grad_norm": 0.2215915322303772,
2501
- "learning_rate": 1.7395956506356937e-05,
2502
- "loss": 0.4214,
2503
- "step": 354
2504
- },
2505
- {
2506
- "epoch": 2.4,
2507
- "grad_norm": 0.2894541919231415,
2508
- "learning_rate": 1.6981596918444953e-05,
2509
- "loss": 0.4201,
2510
- "step": 355
2511
- },
2512
- {
2513
- "epoch": 2.41,
2514
- "grad_norm": 0.24868462979793549,
2515
- "learning_rate": 1.6571773552349e-05,
2516
- "loss": 0.4273,
2517
- "step": 356
2518
- },
2519
- {
2520
- "epoch": 2.42,
2521
- "grad_norm": 0.26563265919685364,
2522
- "learning_rate": 1.6166508801293013e-05,
2523
- "loss": 0.5204,
2524
- "step": 357
2525
- },
2526
- {
2527
- "epoch": 2.42,
2528
- "grad_norm": 0.27713629603385925,
2529
- "learning_rate": 1.5765824809413056e-05,
2530
- "loss": 0.424,
2531
- "step": 358
2532
- },
2533
- {
2534
- "epoch": 2.43,
2535
- "grad_norm": 0.290195107460022,
2536
- "learning_rate": 1.5369743470547027e-05,
2537
- "loss": 0.3887,
2538
- "step": 359
2539
- },
2540
- {
2541
- "epoch": 2.44,
2542
- "grad_norm": 0.27845337986946106,
2543
- "learning_rate": 1.4978286427038601e-05,
2544
- "loss": 0.4918,
2545
- "step": 360
2546
- },
2547
- {
2548
- "epoch": 2.44,
2549
- "grad_norm": 0.2511301040649414,
2550
- "learning_rate": 1.4591475068554572e-05,
2551
- "loss": 0.3305,
2552
- "step": 361
2553
- },
2554
- {
2555
- "epoch": 2.45,
2556
- "grad_norm": 0.28970545530319214,
2557
- "learning_rate": 1.4209330530916165e-05,
2558
- "loss": 0.484,
2559
- "step": 362
2560
- },
2561
- {
2562
- "epoch": 2.46,
2563
- "grad_norm": 0.27981922030448914,
2564
- "learning_rate": 1.3831873694944031e-05,
2565
- "loss": 0.487,
2566
- "step": 363
2567
- },
2568
- {
2569
- "epoch": 2.46,
2570
- "grad_norm": 0.23398059606552124,
2571
- "learning_rate": 1.3459125185317434e-05,
2572
- "loss": 0.4202,
2573
- "step": 364
2574
- },
2575
- {
2576
- "epoch": 2.47,
2577
- "grad_norm": 0.265634685754776,
2578
- "learning_rate": 1.3091105369447165e-05,
2579
- "loss": 0.4261,
2580
- "step": 365
2581
- },
2582
- {
2583
- "epoch": 2.48,
2584
- "grad_norm": 0.25916439294815063,
2585
- "learning_rate": 1.2727834356362778e-05,
2586
- "loss": 0.4882,
2587
- "step": 366
2588
- },
2589
- {
2590
- "epoch": 2.48,
2591
- "grad_norm": 0.29924753308296204,
2592
- "learning_rate": 1.2369331995613665e-05,
2593
- "loss": 0.4131,
2594
- "step": 367
2595
- },
2596
- {
2597
- "epoch": 2.49,
2598
- "grad_norm": 0.2214874029159546,
2599
- "learning_rate": 1.2015617876184527e-05,
2600
- "loss": 0.3283,
2601
- "step": 368
2602
- },
2603
- {
2604
- "epoch": 2.5,
2605
- "grad_norm": 0.24375221133232117,
2606
- "learning_rate": 1.166671132542505e-05,
2607
- "loss": 0.38,
2608
- "step": 369
2609
- },
2610
- {
2611
- "epoch": 2.51,
2612
- "grad_norm": 0.25932660698890686,
2613
- "learning_rate": 1.1322631407993811e-05,
2614
- "loss": 0.4511,
2615
- "step": 370
2616
- },
2617
- {
2618
- "epoch": 2.51,
2619
- "grad_norm": 0.24242925643920898,
2620
- "learning_rate": 1.098339692481648e-05,
2621
- "loss": 0.3851,
2622
- "step": 371
2623
- },
2624
- {
2625
- "epoch": 2.52,
2626
- "grad_norm": 0.24607916176319122,
2627
- "learning_rate": 1.0649026412058583e-05,
2628
- "loss": 0.4515,
2629
- "step": 372
2630
- },
2631
- {
2632
- "epoch": 2.53,
2633
- "grad_norm": 0.2514863610267639,
2634
- "learning_rate": 1.0319538140112728e-05,
2635
- "loss": 0.4129,
2636
- "step": 373
2637
- },
2638
- {
2639
- "epoch": 2.53,
2640
- "grad_norm": 0.23908530175685883,
2641
- "learning_rate": 9.994950112600154e-06,
2642
- "loss": 0.3945,
2643
- "step": 374
2644
- },
2645
- {
2646
- "epoch": 2.54,
2647
- "grad_norm": 0.27527761459350586,
2648
- "learning_rate": 9.675280065387116e-06,
2649
- "loss": 0.4138,
2650
- "step": 375
2651
- },
2652
- {
2653
- "epoch": 2.55,
2654
- "grad_norm": 0.24933388829231262,
2655
- "learning_rate": 9.360545465615667e-06,
2656
- "loss": 0.494,
2657
- "step": 376
2658
- },
2659
- {
2660
- "epoch": 2.55,
2661
- "grad_norm": 0.23585304617881775,
2662
- "learning_rate": 9.05076351074936e-06,
2663
- "loss": 0.4261,
2664
- "step": 377
2665
- },
2666
- {
2667
- "epoch": 2.56,
2668
- "grad_norm": 0.26296553015708923,
2669
- "learning_rate": 8.745951127633411e-06,
2670
- "loss": 0.4748,
2671
- "step": 378
2672
- },
2673
- {
2674
- "epoch": 2.57,
2675
- "grad_norm": 0.3045956790447235,
2676
- "learning_rate": 8.44612497156989e-06,
2677
- "loss": 0.4836,
2678
- "step": 379
2679
- },
2680
- {
2681
- "epoch": 2.57,
2682
- "grad_norm": 0.2398134022951126,
2683
- "learning_rate": 8.151301425407699e-06,
2684
- "loss": 0.3921,
2685
- "step": 380
2686
- },
2687
- {
2688
- "epoch": 2.58,
2689
- "grad_norm": 0.24965588748455048,
2690
- "learning_rate": 7.861496598647278e-06,
2691
- "loss": 0.3871,
2692
- "step": 381
2693
- },
2694
- {
2695
- "epoch": 2.59,
2696
- "grad_norm": 0.24710305035114288,
2697
- "learning_rate": 7.576726326560424e-06,
2698
- "loss": 0.417,
2699
- "step": 382
2700
- },
2701
- {
2702
- "epoch": 2.59,
2703
- "grad_norm": 0.26674723625183105,
2704
- "learning_rate": 7.2970061693250154e-06,
2705
- "loss": 0.371,
2706
- "step": 383
2707
- },
2708
- {
2709
- "epoch": 2.6,
2710
- "grad_norm": 0.25980159640312195,
2711
- "learning_rate": 7.022351411174866e-06,
2712
- "loss": 0.4956,
2713
- "step": 384
2714
- },
2715
- {
2716
- "epoch": 2.61,
2717
- "grad_norm": 0.2925579249858856,
2718
- "learning_rate": 6.75277705956443e-06,
2719
- "loss": 0.4367,
2720
- "step": 385
2721
- },
2722
- {
2723
- "epoch": 2.62,
2724
- "grad_norm": 0.3107716739177704,
2725
- "learning_rate": 6.48829784434889e-06,
2726
- "loss": 0.5735,
2727
- "step": 386
2728
- },
2729
- {
2730
- "epoch": 2.62,
2731
- "grad_norm": 0.32108625769615173,
2732
- "learning_rate": 6.228928216979257e-06,
2733
- "loss": 0.4012,
2734
- "step": 387
2735
- },
2736
- {
2737
- "epoch": 2.63,
2738
- "grad_norm": 0.24750036001205444,
2739
- "learning_rate": 5.97468234971279e-06,
2740
- "loss": 0.3773,
2741
- "step": 388
2742
- },
2743
- {
2744
- "epoch": 2.64,
2745
- "grad_norm": 0.30614009499549866,
2746
- "learning_rate": 5.725574134838474e-06,
2747
- "loss": 0.4044,
2748
- "step": 389
2749
- },
2750
- {
2751
- "epoch": 2.64,
2752
- "grad_norm": 0.24482354521751404,
2753
- "learning_rate": 5.481617183918053e-06,
2754
- "loss": 0.4081,
2755
- "step": 390
2756
- },
2757
- {
2758
- "epoch": 2.65,
2759
- "grad_norm": 0.3191269636154175,
2760
- "learning_rate": 5.242824827042237e-06,
2761
- "loss": 0.4647,
2762
- "step": 391
2763
- },
2764
- {
2765
- "epoch": 2.66,
2766
- "grad_norm": 0.2543604373931885,
2767
- "learning_rate": 5.009210112102292e-06,
2768
- "loss": 0.3955,
2769
- "step": 392
2770
- },
2771
- {
2772
- "epoch": 2.66,
2773
- "grad_norm": 0.2701224684715271,
2774
- "learning_rate": 4.7807858040771924e-06,
2775
- "loss": 0.4296,
2776
- "step": 393
2777
- },
2778
- {
2779
- "epoch": 2.67,
2780
- "grad_norm": 0.21666711568832397,
2781
- "learning_rate": 4.557564384335977e-06,
2782
- "loss": 0.3457,
2783
- "step": 394
2784
- },
2785
- {
2786
- "epoch": 2.68,
2787
- "grad_norm": 0.2999736964702606,
2788
- "learning_rate": 4.339558049955927e-06,
2789
- "loss": 0.4869,
2790
- "step": 395
2791
- },
2792
- {
2793
- "epoch": 2.68,
2794
- "grad_norm": 0.2725822925567627,
2795
- "learning_rate": 4.126778713055923e-06,
2796
- "loss": 0.4391,
2797
- "step": 396
2798
- },
2799
- {
2800
- "epoch": 2.69,
2801
- "grad_norm": 0.2413054257631302,
2802
- "learning_rate": 3.919238000145687e-06,
2803
- "loss": 0.3792,
2804
- "step": 397
2805
- },
2806
- {
2807
- "epoch": 2.7,
2808
- "grad_norm": 0.3631000518798828,
2809
- "learning_rate": 3.7169472514904544e-06,
2810
- "loss": 0.5391,
2811
- "step": 398
2812
- },
2813
- {
2814
- "epoch": 2.7,
2815
- "grad_norm": 0.2513572573661804,
2816
- "learning_rate": 3.5199175204913117e-06,
2817
- "loss": 0.4417,
2818
- "step": 399
2819
- },
2820
- {
2821
- "epoch": 2.71,
2822
- "grad_norm": 0.30949896574020386,
2823
- "learning_rate": 3.3281595730812575e-06,
2824
- "loss": 0.5317,
2825
- "step": 400
2826
- },
2827
- {
2828
- "epoch": 2.72,
2829
- "grad_norm": 0.23445190489292145,
2830
- "learning_rate": 3.1416838871368924e-06,
2831
- "loss": 0.3915,
2832
- "step": 401
2833
- },
2834
- {
2835
- "epoch": 2.73,
2836
- "grad_norm": 0.2802431583404541,
2837
- "learning_rate": 2.9605006519059507e-06,
2838
- "loss": 0.5729,
2839
- "step": 402
2840
- },
2841
- {
2842
- "epoch": 2.73,
2843
- "grad_norm": 0.2695799171924591,
2844
- "learning_rate": 2.7846197674504913e-06,
2845
- "loss": 0.444,
2846
- "step": 403
2847
- },
2848
- {
2849
- "epoch": 2.74,
2850
- "grad_norm": 0.31457579135894775,
2851
- "learning_rate": 2.614050844106042e-06,
2852
- "loss": 0.4572,
2853
- "step": 404
2854
- },
2855
- {
2856
- "epoch": 2.75,
2857
- "grad_norm": 0.29997965693473816,
2858
- "learning_rate": 2.4488032019563402e-06,
2859
- "loss": 0.4318,
2860
- "step": 405
2861
- },
2862
- {
2863
- "epoch": 2.75,
2864
- "grad_norm": 0.3099231719970703,
2865
- "learning_rate": 2.2888858703241866e-06,
2866
- "loss": 0.4432,
2867
- "step": 406
2868
- },
2869
- {
2870
- "epoch": 2.76,
2871
- "grad_norm": 0.29187706112861633,
2872
- "learning_rate": 2.1343075872780396e-06,
2873
- "loss": 0.5138,
2874
- "step": 407
2875
- },
2876
- {
2877
- "epoch": 2.77,
2878
- "grad_norm": 0.26461467146873474,
2879
- "learning_rate": 1.985076799154528e-06,
2880
- "loss": 0.423,
2881
- "step": 408
2882
- },
2883
- {
2884
- "epoch": 2.77,
2885
- "grad_norm": 0.3078599274158478,
2886
- "learning_rate": 1.8412016600969695e-06,
2887
- "loss": 0.5092,
2888
- "step": 409
2889
- },
2890
- {
2891
- "epoch": 2.78,
2892
- "grad_norm": 0.27981695532798767,
2893
- "learning_rate": 1.7026900316098215e-06,
2894
- "loss": 0.4187,
2895
- "step": 410
2896
- },
2897
- {
2898
- "epoch": 2.79,
2899
- "grad_norm": 0.25746870040893555,
2900
- "learning_rate": 1.5695494821290735e-06,
2901
- "loss": 0.349,
2902
- "step": 411
2903
- },
2904
- {
2905
- "epoch": 2.79,
2906
- "grad_norm": 0.3197019398212433,
2907
- "learning_rate": 1.4417872866087534e-06,
2908
- "loss": 0.4377,
2909
- "step": 412
2910
- },
2911
- {
2912
- "epoch": 2.8,
2913
- "grad_norm": 0.26146501302719116,
2914
- "learning_rate": 1.3194104261233798e-06,
2915
- "loss": 0.4026,
2916
- "step": 413
2917
- },
2918
- {
2919
- "epoch": 2.81,
2920
- "grad_norm": 0.2363249659538269,
2921
- "learning_rate": 1.2024255874865108e-06,
2922
- "loss": 0.4289,
2923
- "step": 414
2924
- },
2925
- {
2926
- "epoch": 2.81,
2927
- "grad_norm": 0.24949558079242706,
2928
- "learning_rate": 1.0908391628854041e-06,
2929
- "loss": 0.4798,
2930
- "step": 415
2931
- },
2932
- {
2933
- "epoch": 2.82,
2934
- "grad_norm": 0.24620747566223145,
2935
- "learning_rate": 9.846572495316952e-07,
2936
- "loss": 0.4676,
2937
- "step": 416
2938
- },
2939
- {
2940
- "epoch": 2.83,
2941
- "grad_norm": 0.3077957034111023,
2942
- "learning_rate": 8.838856493282754e-07,
2943
- "loss": 0.392,
2944
- "step": 417
2945
- },
2946
- {
2947
- "epoch": 2.84,
2948
- "grad_norm": 0.3206171691417694,
2949
- "learning_rate": 7.885298685522235e-07,
2950
- "loss": 0.512,
2951
- "step": 418
2952
- },
2953
- {
2954
- "epoch": 2.84,
2955
- "grad_norm": 0.27576300501823425,
2956
- "learning_rate": 6.985951175539796e-07,
2957
- "loss": 0.4261,
2958
- "step": 419
2959
- },
2960
- {
2961
- "epoch": 2.85,
2962
- "grad_norm": 0.32109883427619934,
2963
- "learning_rate": 6.140863104726391e-07,
2964
- "loss": 0.4245,
2965
- "step": 420
2966
- },
2967
- {
2968
- "epoch": 2.86,
2969
- "grad_norm": 0.24693267047405243,
2970
- "learning_rate": 5.350080649674127e-07,
2971
- "loss": 0.455,
2972
- "step": 421
2973
- },
2974
- {
2975
- "epoch": 2.86,
2976
- "grad_norm": 0.2874516248703003,
2977
- "learning_rate": 4.613647019653389e-07,
2978
- "loss": 0.4103,
2979
- "step": 422
2980
- },
2981
- {
2982
- "epoch": 2.87,
2983
- "grad_norm": 0.24350453913211823,
2984
- "learning_rate": 3.931602454251837e-07,
2985
- "loss": 0.3772,
2986
- "step": 423
2987
- },
2988
- {
2989
- "epoch": 2.88,
2990
- "grad_norm": 0.2467474788427353,
2991
- "learning_rate": 3.3039842211752824e-07,
2992
- "loss": 0.4136,
2993
- "step": 424
2994
- },
2995
- {
2996
- "epoch": 2.88,
2997
- "grad_norm": 0.2721412479877472,
2998
- "learning_rate": 2.7308266142119785e-07,
2999
- "loss": 0.3993,
3000
- "step": 425
3001
- },
3002
- {
3003
- "epoch": 2.89,
3004
- "grad_norm": 0.29503345489501953,
3005
- "learning_rate": 2.212160951358011e-07,
3006
- "loss": 0.4101,
3007
- "step": 426
3008
- },
3009
- {
3010
- "epoch": 2.9,
3011
- "grad_norm": 0.23621775209903717,
3012
- "learning_rate": 1.748015573106887e-07,
3013
- "loss": 0.4014,
3014
- "step": 427
3015
- },
3016
- {
3017
- "epoch": 2.9,
3018
- "grad_norm": 0.2449731081724167,
3019
- "learning_rate": 1.338415840900109e-07,
3020
- "loss": 0.3782,
3021
- "step": 428
3022
- },
3023
- {
3024
- "epoch": 2.91,
3025
- "grad_norm": 0.26130935549736023,
3026
- "learning_rate": 9.833841357421714e-08,
3027
- "loss": 0.4822,
3028
- "step": 429
3029
- },
3030
- {
3031
- "epoch": 2.92,
3032
- "grad_norm": 0.21016739308834076,
3033
- "learning_rate": 6.829398569770939e-08,
3034
- "loss": 0.3927,
3035
- "step": 430
3036
- },
3037
- {
3038
- "epoch": 2.92,
3039
- "grad_norm": 0.27280792593955994,
3040
- "learning_rate": 4.370994212287149e-08,
3041
- "loss": 0.5188,
3042
- "step": 431
3043
- },
3044
- {
3045
- "epoch": 2.93,
3046
- "grad_norm": 0.28947770595550537,
3047
- "learning_rate": 2.4587626150351926e-08,
3048
- "loss": 0.486,
3049
- "step": 432
3050
- },
3051
- {
3052
- "epoch": 2.94,
3053
- "grad_norm": 0.2234964370727539,
3054
- "learning_rate": 1.0928082645667025e-08,
3055
- "loss": 0.3444,
3056
- "step": 433
3057
- },
3058
- {
3059
- "epoch": 2.95,
3060
- "grad_norm": 0.2797339856624603,
3061
- "learning_rate": 2.732057982124392e-09,
3062
- "loss": 0.4264,
3063
- "step": 434
3064
- },
3065
- {
3066
- "epoch": 2.95,
3067
- "grad_norm": 0.2575642168521881,
3068
- "learning_rate": 0.0,
3069
- "loss": 0.422,
3070
- "step": 435
3071
- }
3072
- ],
3073
- "logging_steps": 1,
3074
- "max_steps": 435,
3075
- "num_input_tokens_seen": 0,
3076
- "num_train_epochs": 3,
3077
- "save_steps": 145,
3078
- "total_flos": 2.9274017026277376e+17,
3079
- "train_batch_size": 8,
3080
- "trial_name": null,
3081
- "trial_params": null
3082
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
checkpoint-435/training_args.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:e0f4e74f968ef592cbe5e646087e575fbeada43b9dc36d459d99d6a9ebc528b7
3
- size 5560