Training in progress, step 250, checkpoint

.gitattributes

@@ -41,3 +41,4 @@ checkpoint-600/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 checkpoint-50/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 checkpoint-100/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 checkpoint-150/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-250/tokenizer.json filter=lfs diff=lfs merge=lfs -text

checkpoint-250/README.md

@@ -0,0 +1,202 @@
+---
+base_model: unsloth/Qwen2.5-0.5B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.2

checkpoint-250/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "unsloth/Qwen2.5-0.5B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "k_proj",
+    "gate_proj",
+    "v_proj",
+    "down_proj",
+    "up_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": true
+}

checkpoint-250/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:414e5127c42a1dda0ecca134d07d9a20098d5418f0f7e6b83b18e4ce994602f5
+size 70430032

checkpoint-250/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

checkpoint-250/chat_template.jinja

@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

checkpoint-250/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:02e96020bee901466aa35a856ba4d8926a6f4c1ec810e82c966d8e19c2de1d9e
+size 36136293

checkpoint-250/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:181a477a6decc447b8502ae584b4d267ee1822572222c2a703d82e1a6c8f3e34
+size 14581

checkpoint-250/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a6d2aeb7abae3ab3d7eea62015bfd18ff3f1951f6774c9bee39cbcac9fd8e531
+size 1465

checkpoint-250/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|vision_pad|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-250/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:913950e4971737031da511cdd1b410daae4566f62eb845b3975bca5a102323d8
+size 11421995

checkpoint-250/tokenizer_config.json

@@ -0,0 +1,208 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 32768,
+  "pad_token": "<|vision_pad|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

checkpoint-250/trainer_state.json

@@ -0,0 +1,1824 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.6305170239596469,
+  "eval_steps": 50,
+  "global_step": 250,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0025220680958385876,
+      "grad_norm": 83.50857543945312,
+      "learning_rate": 0.0,
+      "loss": 3.3887,
+      "step": 1
+    },
+    {
+      "epoch": 0.005044136191677175,
+      "grad_norm": 71.31573486328125,
+      "learning_rate": 2.0000000000000003e-06,
+      "loss": 3.4305,
+      "step": 2
+    },
+    {
+      "epoch": 0.007566204287515763,
+      "grad_norm": 66.55958557128906,
+      "learning_rate": 4.000000000000001e-06,
+      "loss": 3.0425,
+      "step": 3
+    },
+    {
+      "epoch": 0.01008827238335435,
+      "grad_norm": 67.16094207763672,
+      "learning_rate": 6e-06,
+      "loss": 3.1193,
+      "step": 4
+    },
+    {
+      "epoch": 0.012610340479192938,
+      "grad_norm": 58.159088134765625,
+      "learning_rate": 8.000000000000001e-06,
+      "loss": 2.7595,
+      "step": 5
+    },
+    {
+      "epoch": 0.015132408575031526,
+      "grad_norm": 39.46601867675781,
+      "learning_rate": 1e-05,
+      "loss": 2.3184,
+      "step": 6
+    },
+    {
+      "epoch": 0.017654476670870115,
+      "grad_norm": 45.5927734375,
+      "learning_rate": 9.987325728770595e-06,
+      "loss": 2.1763,
+      "step": 7
+    },
+    {
+      "epoch": 0.0201765447667087,
+      "grad_norm": 20.798906326293945,
+      "learning_rate": 9.974651457541193e-06,
+      "loss": 2.1835,
+      "step": 8
+    },
+    {
+      "epoch": 0.02269861286254729,
+      "grad_norm": 15.517325401306152,
+      "learning_rate": 9.961977186311787e-06,
+      "loss": 2.1354,
+      "step": 9
+    },
+    {
+      "epoch": 0.025220680958385876,
+      "grad_norm": 15.667418479919434,
+      "learning_rate": 9.949302915082384e-06,
+      "loss": 1.9287,
+      "step": 10
+    },
+    {
+      "epoch": 0.027742749054224466,
+      "grad_norm": 13.178864479064941,
+      "learning_rate": 9.93662864385298e-06,
+      "loss": 1.9898,
+      "step": 11
+    },
+    {
+      "epoch": 0.03026481715006305,
+      "grad_norm": 10.053223609924316,
+      "learning_rate": 9.923954372623576e-06,
+      "loss": 2.0013,
+      "step": 12
+    },
+    {
+      "epoch": 0.03278688524590164,
+      "grad_norm": 14.007344245910645,
+      "learning_rate": 9.91128010139417e-06,
+      "loss": 2.0559,
+      "step": 13
+    },
+    {
+      "epoch": 0.03530895334174023,
+      "grad_norm": 11.14796257019043,
+      "learning_rate": 9.898605830164766e-06,
+      "loss": 2.0379,
+      "step": 14
+    },
+    {
+      "epoch": 0.03783102143757881,
+      "grad_norm": 13.28112506866455,
+      "learning_rate": 9.885931558935362e-06,
+      "loss": 1.9854,
+      "step": 15
+    },
+    {
+      "epoch": 0.0403530895334174,
+      "grad_norm": 11.45155143737793,
+      "learning_rate": 9.873257287705957e-06,
+      "loss": 2.1884,
+      "step": 16
+    },
+    {
+      "epoch": 0.04287515762925599,
+      "grad_norm": 10.636513710021973,
+      "learning_rate": 9.860583016476553e-06,
+      "loss": 1.9676,
+      "step": 17
+    },
+    {
+      "epoch": 0.04539722572509458,
+      "grad_norm": 10.700081825256348,
+      "learning_rate": 9.847908745247149e-06,
+      "loss": 1.9426,
+      "step": 18
+    },
+    {
+      "epoch": 0.04791929382093316,
+      "grad_norm": 10.350312232971191,
+      "learning_rate": 9.835234474017745e-06,
+      "loss": 1.931,
+      "step": 19
+    },
+    {
+      "epoch": 0.05044136191677175,
+      "grad_norm": 11.60483455657959,
+      "learning_rate": 9.822560202788341e-06,
+      "loss": 1.9397,
+      "step": 20
+    },
+    {
+      "epoch": 0.05296343001261034,
+      "grad_norm": 10.923288345336914,
+      "learning_rate": 9.809885931558936e-06,
+      "loss": 1.8861,
+      "step": 21
+    },
+    {
+      "epoch": 0.05548549810844893,
+      "grad_norm": 10.222322463989258,
+      "learning_rate": 9.797211660329532e-06,
+      "loss": 1.8154,
+      "step": 22
+    },
+    {
+      "epoch": 0.058007566204287514,
+      "grad_norm": 9.665061950683594,
+      "learning_rate": 9.784537389100128e-06,
+      "loss": 1.9885,
+      "step": 23
+    },
+    {
+      "epoch": 0.0605296343001261,
+      "grad_norm": 10.544888496398926,
+      "learning_rate": 9.771863117870724e-06,
+      "loss": 1.9728,
+      "step": 24
+    },
+    {
+      "epoch": 0.06305170239596469,
+      "grad_norm": 9.924367904663086,
+      "learning_rate": 9.759188846641318e-06,
+      "loss": 1.9665,
+      "step": 25
+    },
+    {
+      "epoch": 0.06557377049180328,
+      "grad_norm": 9.470415115356445,
+      "learning_rate": 9.746514575411914e-06,
+      "loss": 1.8872,
+      "step": 26
+    },
+    {
+      "epoch": 0.06809583858764187,
+      "grad_norm": 9.330588340759277,
+      "learning_rate": 9.73384030418251e-06,
+      "loss": 1.9458,
+      "step": 27
+    },
+    {
+      "epoch": 0.07061790668348046,
+      "grad_norm": 9.101158142089844,
+      "learning_rate": 9.721166032953105e-06,
+      "loss": 2.0452,
+      "step": 28
+    },
+    {
+      "epoch": 0.07313997477931904,
+      "grad_norm": 9.63065242767334,
+      "learning_rate": 9.708491761723701e-06,
+      "loss": 1.9648,
+      "step": 29
+    },
+    {
+      "epoch": 0.07566204287515763,
+      "grad_norm": 9.038743019104004,
+      "learning_rate": 9.695817490494297e-06,
+      "loss": 1.9469,
+      "step": 30
+    },
+    {
+      "epoch": 0.07818411097099622,
+      "grad_norm": 9.375469207763672,
+      "learning_rate": 9.683143219264893e-06,
+      "loss": 2.0387,
+      "step": 31
+    },
+    {
+      "epoch": 0.0807061790668348,
+      "grad_norm": 9.170827865600586,
+      "learning_rate": 9.670468948035488e-06,
+      "loss": 2.0396,
+      "step": 32
+    },
+    {
+      "epoch": 0.0832282471626734,
+      "grad_norm": 9.155048370361328,
+      "learning_rate": 9.657794676806086e-06,
+      "loss": 2.1346,
+      "step": 33
+    },
+    {
+      "epoch": 0.08575031525851198,
+      "grad_norm": 9.660133361816406,
+      "learning_rate": 9.64512040557668e-06,
+      "loss": 1.814,
+      "step": 34
+    },
+    {
+      "epoch": 0.08827238335435057,
+      "grad_norm": 9.577939987182617,
+      "learning_rate": 9.632446134347276e-06,
+      "loss": 1.9353,
+      "step": 35
+    },
+    {
+      "epoch": 0.09079445145018916,
+      "grad_norm": 9.347375869750977,
+      "learning_rate": 9.619771863117872e-06,
+      "loss": 1.8748,
+      "step": 36
+    },
+    {
+      "epoch": 0.09331651954602774,
+      "grad_norm": 9.635443687438965,
+      "learning_rate": 9.607097591888467e-06,
+      "loss": 2.0299,
+      "step": 37
+    },
+    {
+      "epoch": 0.09583858764186633,
+      "grad_norm": 9.470548629760742,
+      "learning_rate": 9.594423320659063e-06,
+      "loss": 1.966,
+      "step": 38
+    },
+    {
+      "epoch": 0.09836065573770492,
+      "grad_norm": 9.292247772216797,
+      "learning_rate": 9.581749049429659e-06,
+      "loss": 1.8779,
+      "step": 39
+    },
+    {
+      "epoch": 0.1008827238335435,
+      "grad_norm": 9.822443962097168,
+      "learning_rate": 9.569074778200255e-06,
+      "loss": 1.8484,
+      "step": 40
+    },
+    {
+      "epoch": 0.1034047919293821,
+      "grad_norm": 8.519198417663574,
+      "learning_rate": 9.55640050697085e-06,
+      "loss": 1.7622,
+      "step": 41
+    },
+    {
+      "epoch": 0.10592686002522068,
+      "grad_norm": 9.220460891723633,
+      "learning_rate": 9.543726235741445e-06,
+      "loss": 1.8659,
+      "step": 42
+    },
+    {
+      "epoch": 0.10844892812105927,
+      "grad_norm": 8.564704895019531,
+      "learning_rate": 9.531051964512042e-06,
+      "loss": 1.8478,
+      "step": 43
+    },
+    {
+      "epoch": 0.11097099621689786,
+      "grad_norm": 9.696242332458496,
+      "learning_rate": 9.518377693282636e-06,
+      "loss": 1.8625,
+      "step": 44
+    },
+    {
+      "epoch": 0.11349306431273644,
+      "grad_norm": 9.121529579162598,
+      "learning_rate": 9.505703422053234e-06,
+      "loss": 1.9736,
+      "step": 45
+    },
+    {
+      "epoch": 0.11601513240857503,
+      "grad_norm": 9.0239896774292,
+      "learning_rate": 9.493029150823828e-06,
+      "loss": 1.8687,
+      "step": 46
+    },
+    {
+      "epoch": 0.11853720050441362,
+      "grad_norm": 8.566268920898438,
+      "learning_rate": 9.480354879594424e-06,
+      "loss": 1.7855,
+      "step": 47
+    },
+    {
+      "epoch": 0.1210592686002522,
+      "grad_norm": 9.728326797485352,
+      "learning_rate": 9.46768060836502e-06,
+      "loss": 2.0394,
+      "step": 48
+    },
+    {
+      "epoch": 0.1235813366960908,
+      "grad_norm": 8.644201278686523,
+      "learning_rate": 9.455006337135616e-06,
+      "loss": 1.8265,
+      "step": 49
+    },
+    {
+      "epoch": 0.12610340479192939,
+      "grad_norm": 9.00877857208252,
+      "learning_rate": 9.442332065906211e-06,
+      "loss": 1.9655,
+      "step": 50
+    },
+    {
+      "epoch": 0.12610340479192939,
+      "eval_loss": 1.8593257665634155,
+      "eval_runtime": 18.8102,
+      "eval_samples_per_second": 37.48,
+      "eval_steps_per_second": 18.766,
+      "step": 50
+    },
+    {
+      "epoch": 0.12862547288776796,
+      "grad_norm": 8.626359939575195,
+      "learning_rate": 9.429657794676807e-06,
+      "loss": 1.8767,
+      "step": 51
+    },
+    {
+      "epoch": 0.13114754098360656,
+      "grad_norm": 8.989786148071289,
+      "learning_rate": 9.416983523447403e-06,
+      "loss": 1.9197,
+      "step": 52
+    },
+    {
+      "epoch": 0.13366960907944514,
+      "grad_norm": 8.265259742736816,
+      "learning_rate": 9.404309252217997e-06,
+      "loss": 1.7882,
+      "step": 53
+    },
+    {
+      "epoch": 0.13619167717528374,
+      "grad_norm": 9.578763008117676,
+      "learning_rate": 9.391634980988594e-06,
+      "loss": 2.0616,
+      "step": 54
+    },
+    {
+      "epoch": 0.13871374527112232,
+      "grad_norm": 9.27950382232666,
+      "learning_rate": 9.37896070975919e-06,
+      "loss": 1.9043,
+      "step": 55
+    },
+    {
+      "epoch": 0.14123581336696092,
+      "grad_norm": 8.789308547973633,
+      "learning_rate": 9.366286438529786e-06,
+      "loss": 1.8745,
+      "step": 56
+    },
+    {
+      "epoch": 0.1437578814627995,
+      "grad_norm": 8.460257530212402,
+      "learning_rate": 9.35361216730038e-06,
+      "loss": 1.9719,
+      "step": 57
+    },
+    {
+      "epoch": 0.14627994955863807,
+      "grad_norm": 8.686505317687988,
+      "learning_rate": 9.340937896070976e-06,
+      "loss": 1.8288,
+      "step": 58
+    },
+    {
+      "epoch": 0.14880201765447668,
+      "grad_norm": 9.074113845825195,
+      "learning_rate": 9.328263624841572e-06,
+      "loss": 1.8212,
+      "step": 59
+    },
+    {
+      "epoch": 0.15132408575031525,
+      "grad_norm": 8.919495582580566,
+      "learning_rate": 9.315589353612169e-06,
+      "loss": 1.9267,
+      "step": 60
+    },
+    {
+      "epoch": 0.15384615384615385,
+      "grad_norm": 9.349574089050293,
+      "learning_rate": 9.302915082382765e-06,
+      "loss": 1.8566,
+      "step": 61
+    },
+    {
+      "epoch": 0.15636822194199243,
+      "grad_norm": 10.293651580810547,
+      "learning_rate": 9.290240811153359e-06,
+      "loss": 1.7412,
+      "step": 62
+    },
+    {
+      "epoch": 0.15889029003783103,
+      "grad_norm": 9.373260498046875,
+      "learning_rate": 9.277566539923955e-06,
+      "loss": 1.9118,
+      "step": 63
+    },
+    {
+      "epoch": 0.1614123581336696,
+      "grad_norm": 9.066275596618652,
+      "learning_rate": 9.264892268694551e-06,
+      "loss": 1.89,
+      "step": 64
+    },
+    {
+      "epoch": 0.16393442622950818,
+      "grad_norm": 9.459461212158203,
+      "learning_rate": 9.252217997465146e-06,
+      "loss": 1.8408,
+      "step": 65
+    },
+    {
+      "epoch": 0.1664564943253468,
+      "grad_norm": 9.49438190460205,
+      "learning_rate": 9.239543726235742e-06,
+      "loss": 1.9826,
+      "step": 66
+    },
+    {
+      "epoch": 0.16897856242118536,
+      "grad_norm": 9.088374137878418,
+      "learning_rate": 9.226869455006338e-06,
+      "loss": 1.8442,
+      "step": 67
+    },
+    {
+      "epoch": 0.17150063051702397,
+      "grad_norm": 8.981496810913086,
+      "learning_rate": 9.214195183776934e-06,
+      "loss": 1.8115,
+      "step": 68
+    },
+    {
+      "epoch": 0.17402269861286254,
+      "grad_norm": 9.342341423034668,
+      "learning_rate": 9.201520912547528e-06,
+      "loss": 1.849,
+      "step": 69
+    },
+    {
+      "epoch": 0.17654476670870115,
+      "grad_norm": 9.18710708618164,
+      "learning_rate": 9.188846641318126e-06,
+      "loss": 1.7475,
+      "step": 70
+    },
+    {
+      "epoch": 0.17906683480453972,
+      "grad_norm": 9.204767227172852,
+      "learning_rate": 9.17617237008872e-06,
+      "loss": 1.8912,
+      "step": 71
+    },
+    {
+      "epoch": 0.18158890290037832,
+      "grad_norm": 9.105266571044922,
+      "learning_rate": 9.163498098859315e-06,
+      "loss": 1.8394,
+      "step": 72
+    },
+    {
+      "epoch": 0.1841109709962169,
+      "grad_norm": 9.192793846130371,
+      "learning_rate": 9.150823827629913e-06,
+      "loss": 1.9242,
+      "step": 73
+    },
+    {
+      "epoch": 0.18663303909205547,
+      "grad_norm": 8.911624908447266,
+      "learning_rate": 9.138149556400507e-06,
+      "loss": 1.9121,
+      "step": 74
+    },
+    {
+      "epoch": 0.18915510718789408,
+      "grad_norm": 9.20001220703125,
+      "learning_rate": 9.125475285171103e-06,
+      "loss": 1.9752,
+      "step": 75
+    },
+    {
+      "epoch": 0.19167717528373265,
+      "grad_norm": 8.967398643493652,
+      "learning_rate": 9.1128010139417e-06,
+      "loss": 1.7422,
+      "step": 76
+    },
+    {
+      "epoch": 0.19419924337957126,
+      "grad_norm": 8.867136001586914,
+      "learning_rate": 9.100126742712296e-06,
+      "loss": 1.7391,
+      "step": 77
+    },
+    {
+      "epoch": 0.19672131147540983,
+      "grad_norm": 9.721291542053223,
+      "learning_rate": 9.08745247148289e-06,
+      "loss": 1.7955,
+      "step": 78
+    },
+    {
+      "epoch": 0.19924337957124844,
+      "grad_norm": 8.936488151550293,
+      "learning_rate": 9.074778200253486e-06,
+      "loss": 1.602,
+      "step": 79
+    },
+    {
+      "epoch": 0.201765447667087,
+      "grad_norm": 8.965775489807129,
+      "learning_rate": 9.062103929024082e-06,
+      "loss": 1.9918,
+      "step": 80
+    },
+    {
+      "epoch": 0.2042875157629256,
+      "grad_norm": 9.03048324584961,
+      "learning_rate": 9.049429657794677e-06,
+      "loss": 1.8228,
+      "step": 81
+    },
+    {
+      "epoch": 0.2068095838587642,
+      "grad_norm": 9.195337295532227,
+      "learning_rate": 9.036755386565273e-06,
+      "loss": 1.9259,
+      "step": 82
+    },
+    {
+      "epoch": 0.20933165195460277,
+      "grad_norm": 9.696627616882324,
+      "learning_rate": 9.024081115335869e-06,
+      "loss": 1.8497,
+      "step": 83
+    },
+    {
+      "epoch": 0.21185372005044137,
+      "grad_norm": 9.298614501953125,
+      "learning_rate": 9.011406844106465e-06,
+      "loss": 1.8685,
+      "step": 84
+    },
+    {
+      "epoch": 0.21437578814627994,
+      "grad_norm": 8.810253143310547,
+      "learning_rate": 8.998732572877061e-06,
+      "loss": 1.7147,
+      "step": 85
+    },
+    {
+      "epoch": 0.21689785624211855,
+      "grad_norm": 8.825125694274902,
+      "learning_rate": 8.986058301647655e-06,
+      "loss": 1.8466,
+      "step": 86
+    },
+    {
+      "epoch": 0.21941992433795712,
+      "grad_norm": 8.869908332824707,
+      "learning_rate": 8.973384030418252e-06,
+      "loss": 1.9044,
+      "step": 87
+    },
+    {
+      "epoch": 0.22194199243379573,
+      "grad_norm": 9.151841163635254,
+      "learning_rate": 8.960709759188848e-06,
+      "loss": 1.8922,
+      "step": 88
+    },
+    {
+      "epoch": 0.2244640605296343,
+      "grad_norm": 9.193544387817383,
+      "learning_rate": 8.948035487959444e-06,
+      "loss": 1.8199,
+      "step": 89
+    },
+    {
+      "epoch": 0.22698612862547288,
+      "grad_norm": 8.479183197021484,
+      "learning_rate": 8.935361216730038e-06,
+      "loss": 1.847,
+      "step": 90
+    },
+    {
+      "epoch": 0.22950819672131148,
+      "grad_norm": 9.11871337890625,
+      "learning_rate": 8.922686945500634e-06,
+      "loss": 1.8631,
+      "step": 91
+    },
+    {
+      "epoch": 0.23203026481715006,
+      "grad_norm": 9.080836296081543,
+      "learning_rate": 8.91001267427123e-06,
+      "loss": 1.9849,
+      "step": 92
+    },
+    {
+      "epoch": 0.23455233291298866,
+      "grad_norm": 10.182774543762207,
+      "learning_rate": 8.897338403041825e-06,
+      "loss": 1.933,
+      "step": 93
+    },
+    {
+      "epoch": 0.23707440100882723,
+      "grad_norm": 9.077899932861328,
+      "learning_rate": 8.884664131812421e-06,
+      "loss": 1.7692,
+      "step": 94
+    },
+    {
+      "epoch": 0.23959646910466584,
+      "grad_norm": 8.86284065246582,
+      "learning_rate": 8.871989860583017e-06,
+      "loss": 1.7995,
+      "step": 95
+    },
+    {
+      "epoch": 0.2421185372005044,
+      "grad_norm": 9.612898826599121,
+      "learning_rate": 8.859315589353613e-06,
+      "loss": 2.045,
+      "step": 96
+    },
+    {
+      "epoch": 0.244640605296343,
+      "grad_norm": 9.372990608215332,
+      "learning_rate": 8.846641318124208e-06,
+      "loss": 1.8652,
+      "step": 97
+    },
+    {
+      "epoch": 0.2471626733921816,
+      "grad_norm": 9.045456886291504,
+      "learning_rate": 8.833967046894805e-06,
+      "loss": 1.8405,
+      "step": 98
+    },
+    {
+      "epoch": 0.24968474148802017,
+      "grad_norm": 8.836333274841309,
+      "learning_rate": 8.8212927756654e-06,
+      "loss": 1.7977,
+      "step": 99
+    },
+    {
+      "epoch": 0.25220680958385877,
+      "grad_norm": 8.542088508605957,
+      "learning_rate": 8.808618504435996e-06,
+      "loss": 1.7449,
+      "step": 100
+    },
+    {
+      "epoch": 0.25220680958385877,
+      "eval_loss": 1.821334719657898,
+      "eval_runtime": 18.8896,
+      "eval_samples_per_second": 37.322,
+      "eval_steps_per_second": 18.688,
+      "step": 100
+    },
+    {
+      "epoch": 0.2547288776796974,
+      "grad_norm": 9.596298217773438,
+      "learning_rate": 8.795944233206592e-06,
+      "loss": 1.8074,
+      "step": 101
+    },
+    {
+      "epoch": 0.2572509457755359,
+      "grad_norm": 8.635272026062012,
+      "learning_rate": 8.783269961977186e-06,
+      "loss": 2.0499,
+      "step": 102
+    },
+    {
+      "epoch": 0.2597730138713745,
+      "grad_norm": 8.884391784667969,
+      "learning_rate": 8.770595690747782e-06,
+      "loss": 1.8544,
+      "step": 103
+    },
+    {
+      "epoch": 0.26229508196721313,
+      "grad_norm": 8.27906322479248,
+      "learning_rate": 8.757921419518379e-06,
+      "loss": 1.6638,
+      "step": 104
+    },
+    {
+      "epoch": 0.2648171500630517,
+      "grad_norm": 8.792101860046387,
+      "learning_rate": 8.745247148288975e-06,
+      "loss": 1.8727,
+      "step": 105
+    },
+    {
+      "epoch": 0.2673392181588903,
+      "grad_norm": 8.464868545532227,
+      "learning_rate": 8.732572877059569e-06,
+      "loss": 1.7947,
+      "step": 106
+    },
+    {
+      "epoch": 0.2698612862547289,
+      "grad_norm": 9.57347583770752,
+      "learning_rate": 8.719898605830165e-06,
+      "loss": 2.0333,
+      "step": 107
+    },
+    {
+      "epoch": 0.2723833543505675,
+      "grad_norm": 8.682156562805176,
+      "learning_rate": 8.707224334600761e-06,
+      "loss": 1.7808,
+      "step": 108
+    },
+    {
+      "epoch": 0.27490542244640603,
+      "grad_norm": 9.720832824707031,
+      "learning_rate": 8.694550063371356e-06,
+      "loss": 1.9601,
+      "step": 109
+    },
+    {
+      "epoch": 0.27742749054224464,
+      "grad_norm": 8.890487670898438,
+      "learning_rate": 8.681875792141954e-06,
+      "loss": 1.9123,
+      "step": 110
+    },
+    {
+      "epoch": 0.27994955863808324,
+      "grad_norm": 9.121131896972656,
+      "learning_rate": 8.669201520912548e-06,
+      "loss": 1.8399,
+      "step": 111
+    },
+    {
+      "epoch": 0.28247162673392184,
+      "grad_norm": 8.931142807006836,
+      "learning_rate": 8.656527249683144e-06,
+      "loss": 1.88,
+      "step": 112
+    },
+    {
+      "epoch": 0.2849936948297604,
+      "grad_norm": 9.079472541809082,
+      "learning_rate": 8.64385297845374e-06,
+      "loss": 1.7766,
+      "step": 113
+    },
+    {
+      "epoch": 0.287515762925599,
+      "grad_norm": 9.532262802124023,
+      "learning_rate": 8.631178707224336e-06,
+      "loss": 1.7715,
+      "step": 114
+    },
+    {
+      "epoch": 0.2900378310214376,
+      "grad_norm": 8.928796768188477,
+      "learning_rate": 8.61850443599493e-06,
+      "loss": 1.6179,
+      "step": 115
+    },
+    {
+      "epoch": 0.29255989911727615,
+      "grad_norm": 8.79500961303711,
+      "learning_rate": 8.605830164765527e-06,
+      "loss": 1.7653,
+      "step": 116
+    },
+    {
+      "epoch": 0.29508196721311475,
+      "grad_norm": 9.038860321044922,
+      "learning_rate": 8.593155893536123e-06,
+      "loss": 1.8273,
+      "step": 117
+    },
+    {
+      "epoch": 0.29760403530895335,
+      "grad_norm": 9.18289852142334,
+      "learning_rate": 8.580481622306717e-06,
+      "loss": 1.945,
+      "step": 118
+    },
+    {
+      "epoch": 0.30012610340479196,
+      "grad_norm": 9.104387283325195,
+      "learning_rate": 8.567807351077313e-06,
+      "loss": 1.8444,
+      "step": 119
+    },
+    {
+      "epoch": 0.3026481715006305,
+      "grad_norm": 9.251449584960938,
+      "learning_rate": 8.55513307984791e-06,
+      "loss": 1.7502,
+      "step": 120
+    },
+    {
+      "epoch": 0.3051702395964691,
+      "grad_norm": 9.526406288146973,
+      "learning_rate": 8.542458808618506e-06,
+      "loss": 1.8277,
+      "step": 121
+    },
+    {
+      "epoch": 0.3076923076923077,
+      "grad_norm": 9.494799613952637,
+      "learning_rate": 8.5297845373891e-06,
+      "loss": 1.9802,
+      "step": 122
+    },
+    {
+      "epoch": 0.31021437578814626,
+      "grad_norm": 9.093886375427246,
+      "learning_rate": 8.517110266159696e-06,
+      "loss": 1.7598,
+      "step": 123
+    },
+    {
+      "epoch": 0.31273644388398486,
+      "grad_norm": 9.275187492370605,
+      "learning_rate": 8.504435994930292e-06,
+      "loss": 1.7269,
+      "step": 124
+    },
+    {
+      "epoch": 0.31525851197982346,
+      "grad_norm": 8.311441421508789,
+      "learning_rate": 8.491761723700888e-06,
+      "loss": 1.8353,
+      "step": 125
+    },
+    {
+      "epoch": 0.31778058007566207,
+      "grad_norm": 8.989309310913086,
+      "learning_rate": 8.479087452471484e-06,
+      "loss": 1.8799,
+      "step": 126
+    },
+    {
+      "epoch": 0.3203026481715006,
+      "grad_norm": 9.124334335327148,
+      "learning_rate": 8.466413181242079e-06,
+      "loss": 1.8709,
+      "step": 127
+    },
+    {
+      "epoch": 0.3228247162673392,
+      "grad_norm": 9.008078575134277,
+      "learning_rate": 8.453738910012675e-06,
+      "loss": 1.7002,
+      "step": 128
+    },
+    {
+      "epoch": 0.3253467843631778,
+      "grad_norm": 8.43070125579834,
+      "learning_rate": 8.441064638783271e-06,
+      "loss": 1.8012,
+      "step": 129
+    },
+    {
+      "epoch": 0.32786885245901637,
+      "grad_norm": 9.406371116638184,
+      "learning_rate": 8.428390367553865e-06,
+      "loss": 1.8611,
+      "step": 130
+    },
+    {
+      "epoch": 0.33039092055485497,
+      "grad_norm": 8.853006362915039,
+      "learning_rate": 8.415716096324462e-06,
+      "loss": 1.7709,
+      "step": 131
+    },
+    {
+      "epoch": 0.3329129886506936,
+      "grad_norm": 9.170083045959473,
+      "learning_rate": 8.403041825095058e-06,
+      "loss": 2.0669,
+      "step": 132
+    },
+    {
+      "epoch": 0.3354350567465322,
+      "grad_norm": 9.382063865661621,
+      "learning_rate": 8.390367553865654e-06,
+      "loss": 1.8012,
+      "step": 133
+    },
+    {
+      "epoch": 0.3379571248423707,
+      "grad_norm": 9.157647132873535,
+      "learning_rate": 8.377693282636248e-06,
+      "loss": 1.8584,
+      "step": 134
+    },
+    {
+      "epoch": 0.34047919293820933,
+      "grad_norm": 9.179341316223145,
+      "learning_rate": 8.365019011406846e-06,
+      "loss": 1.8589,
+      "step": 135
+    },
+    {
+      "epoch": 0.34300126103404793,
+      "grad_norm": 9.133096694946289,
+      "learning_rate": 8.35234474017744e-06,
+      "loss": 1.9222,
+      "step": 136
+    },
+    {
+      "epoch": 0.3455233291298865,
+      "grad_norm": 8.78919792175293,
+      "learning_rate": 8.339670468948035e-06,
+      "loss": 1.839,
+      "step": 137
+    },
+    {
+      "epoch": 0.3480453972257251,
+      "grad_norm": 8.94666576385498,
+      "learning_rate": 8.326996197718633e-06,
+      "loss": 1.7538,
+      "step": 138
+    },
+    {
+      "epoch": 0.3505674653215637,
+      "grad_norm": 9.572528839111328,
+      "learning_rate": 8.314321926489227e-06,
+      "loss": 1.9778,
+      "step": 139
+    },
+    {
+      "epoch": 0.3530895334174023,
+      "grad_norm": 9.474174499511719,
+      "learning_rate": 8.301647655259823e-06,
+      "loss": 1.9579,
+      "step": 140
+    },
+    {
+      "epoch": 0.35561160151324084,
+      "grad_norm": 9.462223052978516,
+      "learning_rate": 8.28897338403042e-06,
+      "loss": 1.7689,
+      "step": 141
+    },
+    {
+      "epoch": 0.35813366960907944,
+      "grad_norm": 8.904463768005371,
+      "learning_rate": 8.276299112801015e-06,
+      "loss": 1.7506,
+      "step": 142
+    },
+    {
+      "epoch": 0.36065573770491804,
+      "grad_norm": 9.692485809326172,
+      "learning_rate": 8.26362484157161e-06,
+      "loss": 2.0011,
+      "step": 143
+    },
+    {
+      "epoch": 0.36317780580075665,
+      "grad_norm": 8.99770450592041,
+      "learning_rate": 8.250950570342206e-06,
+      "loss": 1.7689,
+      "step": 144
+    },
+    {
+      "epoch": 0.3656998738965952,
+      "grad_norm": 8.849214553833008,
+      "learning_rate": 8.238276299112802e-06,
+      "loss": 1.845,
+      "step": 145
+    },
+    {
+      "epoch": 0.3682219419924338,
+      "grad_norm": 9.16205883026123,
+      "learning_rate": 8.225602027883396e-06,
+      "loss": 1.9285,
+      "step": 146
+    },
+    {
+      "epoch": 0.3707440100882724,
+      "grad_norm": 9.169661521911621,
+      "learning_rate": 8.212927756653993e-06,
+      "loss": 1.8835,
+      "step": 147
+    },
+    {
+      "epoch": 0.37326607818411095,
+      "grad_norm": 9.326395988464355,
+      "learning_rate": 8.200253485424589e-06,
+      "loss": 1.8162,
+      "step": 148
+    },
+    {
+      "epoch": 0.37578814627994955,
+      "grad_norm": 8.962848663330078,
+      "learning_rate": 8.187579214195185e-06,
+      "loss": 1.8775,
+      "step": 149
+    },
+    {
+      "epoch": 0.37831021437578816,
+      "grad_norm": 8.53931713104248,
+      "learning_rate": 8.17490494296578e-06,
+      "loss": 1.7893,
+      "step": 150
+    },
+    {
+      "epoch": 0.37831021437578816,
+      "eval_loss": 1.8017936944961548,
+      "eval_runtime": 18.8835,
+      "eval_samples_per_second": 37.334,
+      "eval_steps_per_second": 18.694,
+      "step": 150
+    },
+    {
+      "epoch": 0.38083228247162676,
+      "grad_norm": 8.814638137817383,
+      "learning_rate": 8.162230671736375e-06,
+      "loss": 1.7401,
+      "step": 151
+    },
+    {
+      "epoch": 0.3833543505674653,
+      "grad_norm": 9.316661834716797,
+      "learning_rate": 8.149556400506971e-06,
+      "loss": 1.9799,
+      "step": 152
+    },
+    {
+      "epoch": 0.3858764186633039,
+      "grad_norm": 9.053354263305664,
+      "learning_rate": 8.136882129277567e-06,
+      "loss": 1.6721,
+      "step": 153
+    },
+    {
+      "epoch": 0.3883984867591425,
+      "grad_norm": 9.15576457977295,
+      "learning_rate": 8.124207858048164e-06,
+      "loss": 1.9065,
+      "step": 154
+    },
+    {
+      "epoch": 0.39092055485498106,
+      "grad_norm": 8.911511421203613,
+      "learning_rate": 8.111533586818758e-06,
+      "loss": 1.9229,
+      "step": 155
+    },
+    {
+      "epoch": 0.39344262295081966,
+      "grad_norm": 8.312164306640625,
+      "learning_rate": 8.098859315589354e-06,
+      "loss": 1.7699,
+      "step": 156
+    },
+    {
+      "epoch": 0.39596469104665827,
+      "grad_norm": 9.155098915100098,
+      "learning_rate": 8.08618504435995e-06,
+      "loss": 1.9292,
+      "step": 157
+    },
+    {
+      "epoch": 0.39848675914249687,
+      "grad_norm": 8.932412147521973,
+      "learning_rate": 8.073510773130546e-06,
+      "loss": 1.8102,
+      "step": 158
+    },
+    {
+      "epoch": 0.4010088272383354,
+      "grad_norm": 8.135700225830078,
+      "learning_rate": 8.06083650190114e-06,
+      "loss": 1.72,
+      "step": 159
+    },
+    {
+      "epoch": 0.403530895334174,
+      "grad_norm": 9.693431854248047,
+      "learning_rate": 8.048162230671737e-06,
+      "loss": 1.8558,
+      "step": 160
+    },
+    {
+      "epoch": 0.4060529634300126,
+      "grad_norm": 8.98935317993164,
+      "learning_rate": 8.035487959442333e-06,
+      "loss": 1.9407,
+      "step": 161
+    },
+    {
+      "epoch": 0.4085750315258512,
+      "grad_norm": 8.735937118530273,
+      "learning_rate": 8.022813688212927e-06,
+      "loss": 1.8333,
+      "step": 162
+    },
+    {
+      "epoch": 0.4110970996216898,
+      "grad_norm": 9.313026428222656,
+      "learning_rate": 8.010139416983525e-06,
+      "loss": 1.9925,
+      "step": 163
+    },
+    {
+      "epoch": 0.4136191677175284,
+      "grad_norm": 9.607177734375,
+      "learning_rate": 7.99746514575412e-06,
+      "loss": 1.8164,
+      "step": 164
+    },
+    {
+      "epoch": 0.416141235813367,
+      "grad_norm": 9.086844444274902,
+      "learning_rate": 7.984790874524716e-06,
+      "loss": 1.8585,
+      "step": 165
+    },
+    {
+      "epoch": 0.41866330390920553,
+      "grad_norm": 9.212312698364258,
+      "learning_rate": 7.972116603295312e-06,
+      "loss": 1.8332,
+      "step": 166
+    },
+    {
+      "epoch": 0.42118537200504413,
+      "grad_norm": 9.246332168579102,
+      "learning_rate": 7.959442332065906e-06,
+      "loss": 1.8488,
+      "step": 167
+    },
+    {
+      "epoch": 0.42370744010088274,
+      "grad_norm": 9.077427864074707,
+      "learning_rate": 7.946768060836502e-06,
+      "loss": 1.7442,
+      "step": 168
+    },
+    {
+      "epoch": 0.4262295081967213,
+      "grad_norm": 9.019879341125488,
+      "learning_rate": 7.934093789607098e-06,
+      "loss": 1.9638,
+      "step": 169
+    },
+    {
+      "epoch": 0.4287515762925599,
+      "grad_norm": 8.645258903503418,
+      "learning_rate": 7.921419518377694e-06,
+      "loss": 1.8532,
+      "step": 170
+    },
+    {
+      "epoch": 0.4312736443883985,
+      "grad_norm": 8.863842010498047,
+      "learning_rate": 7.908745247148289e-06,
+      "loss": 1.7754,
+      "step": 171
+    },
+    {
+      "epoch": 0.4337957124842371,
+      "grad_norm": 9.303889274597168,
+      "learning_rate": 7.896070975918885e-06,
+      "loss": 1.774,
+      "step": 172
+    },
+    {
+      "epoch": 0.43631778058007564,
+      "grad_norm": 9.072324752807617,
+      "learning_rate": 7.883396704689481e-06,
+      "loss": 1.9916,
+      "step": 173
+    },
+    {
+      "epoch": 0.43883984867591425,
+      "grad_norm": 8.935220718383789,
+      "learning_rate": 7.870722433460076e-06,
+      "loss": 1.7988,
+      "step": 174
+    },
+    {
+      "epoch": 0.44136191677175285,
+      "grad_norm": 8.469002723693848,
+      "learning_rate": 7.858048162230673e-06,
+      "loss": 1.7943,
+      "step": 175
+    },
+    {
+      "epoch": 0.44388398486759145,
+      "grad_norm": 8.663657188415527,
+      "learning_rate": 7.845373891001268e-06,
+      "loss": 1.9424,
+      "step": 176
+    },
+    {
+      "epoch": 0.44640605296343,
+      "grad_norm": 9.067521095275879,
+      "learning_rate": 7.832699619771864e-06,
+      "loss": 1.8128,
+      "step": 177
+    },
+    {
+      "epoch": 0.4489281210592686,
+      "grad_norm": 8.450661659240723,
+      "learning_rate": 7.82002534854246e-06,
+      "loss": 1.7287,
+      "step": 178
+    },
+    {
+      "epoch": 0.4514501891551072,
+      "grad_norm": 8.83759593963623,
+      "learning_rate": 7.807351077313056e-06,
+      "loss": 1.7602,
+      "step": 179
+    },
+    {
+      "epoch": 0.45397225725094575,
+      "grad_norm": 8.693680763244629,
+      "learning_rate": 7.79467680608365e-06,
+      "loss": 2.0174,
+      "step": 180
+    },
+    {
+      "epoch": 0.45649432534678436,
+      "grad_norm": 9.432674407958984,
+      "learning_rate": 7.782002534854247e-06,
+      "loss": 1.9696,
+      "step": 181
+    },
+    {
+      "epoch": 0.45901639344262296,
+      "grad_norm": 8.901416778564453,
+      "learning_rate": 7.769328263624843e-06,
+      "loss": 1.8075,
+      "step": 182
+    },
+    {
+      "epoch": 0.46153846153846156,
+      "grad_norm": 8.675703048706055,
+      "learning_rate": 7.756653992395437e-06,
+      "loss": 1.8696,
+      "step": 183
+    },
+    {
+      "epoch": 0.4640605296343001,
+      "grad_norm": 9.176361083984375,
+      "learning_rate": 7.743979721166033e-06,
+      "loss": 1.8572,
+      "step": 184
+    },
+    {
+      "epoch": 0.4665825977301387,
+      "grad_norm": 8.568829536437988,
+      "learning_rate": 7.73130544993663e-06,
+      "loss": 1.638,
+      "step": 185
+    },
+    {
+      "epoch": 0.4691046658259773,
+      "grad_norm": 8.751213073730469,
+      "learning_rate": 7.718631178707225e-06,
+      "loss": 1.7869,
+      "step": 186
+    },
+    {
+      "epoch": 0.47162673392181587,
+      "grad_norm": 8.986238479614258,
+      "learning_rate": 7.70595690747782e-06,
+      "loss": 1.831,
+      "step": 187
+    },
+    {
+      "epoch": 0.47414880201765447,
+      "grad_norm": 9.22641372680664,
+      "learning_rate": 7.693282636248416e-06,
+      "loss": 1.7918,
+      "step": 188
+    },
+    {
+      "epoch": 0.4766708701134931,
+      "grad_norm": 9.145533561706543,
+      "learning_rate": 7.680608365019012e-06,
+      "loss": 1.6651,
+      "step": 189
+    },
+    {
+      "epoch": 0.4791929382093317,
+      "grad_norm": 8.419795989990234,
+      "learning_rate": 7.667934093789608e-06,
+      "loss": 1.6928,
+      "step": 190
+    },
+    {
+      "epoch": 0.4817150063051702,
+      "grad_norm": 9.782970428466797,
+      "learning_rate": 7.655259822560204e-06,
+      "loss": 1.9865,
+      "step": 191
+    },
+    {
+      "epoch": 0.4842370744010088,
+      "grad_norm": 8.885458946228027,
+      "learning_rate": 7.642585551330799e-06,
+      "loss": 1.9461,
+      "step": 192
+    },
+    {
+      "epoch": 0.48675914249684743,
+      "grad_norm": 10.114888191223145,
+      "learning_rate": 7.629911280101395e-06,
+      "loss": 1.6703,
+      "step": 193
+    },
+    {
+      "epoch": 0.489281210592686,
+      "grad_norm": 9.351268768310547,
+      "learning_rate": 7.617237008871991e-06,
+      "loss": 1.8674,
+      "step": 194
+    },
+    {
+      "epoch": 0.4918032786885246,
+      "grad_norm": 9.144396781921387,
+      "learning_rate": 7.604562737642585e-06,
+      "loss": 2.0062,
+      "step": 195
+    },
+    {
+      "epoch": 0.4943253467843632,
+      "grad_norm": 9.084301948547363,
+      "learning_rate": 7.591888466413182e-06,
+      "loss": 2.015,
+      "step": 196
+    },
+    {
+      "epoch": 0.4968474148802018,
+      "grad_norm": 8.582064628601074,
+      "learning_rate": 7.5792141951837775e-06,
+      "loss": 1.8281,
+      "step": 197
+    },
+    {
+      "epoch": 0.49936948297604034,
+      "grad_norm": 8.800094604492188,
+      "learning_rate": 7.566539923954374e-06,
+      "loss": 1.7135,
+      "step": 198
+    },
+    {
+      "epoch": 0.501891551071879,
+      "grad_norm": 9.495306015014648,
+      "learning_rate": 7.553865652724969e-06,
+      "loss": 1.8995,
+      "step": 199
+    },
+    {
+      "epoch": 0.5044136191677175,
+      "grad_norm": 9.089255332946777,
+      "learning_rate": 7.541191381495565e-06,
+      "loss": 1.8346,
+      "step": 200
+    },
+    {
+      "epoch": 0.5044136191677175,
+      "eval_loss": 1.7910906076431274,
+      "eval_runtime": 18.8348,
+      "eval_samples_per_second": 37.431,
+      "eval_steps_per_second": 18.742,
+      "step": 200
+    },
+    {
+      "epoch": 0.5069356872635561,
+      "grad_norm": 9.345693588256836,
+      "learning_rate": 7.52851711026616e-06,
+      "loss": 1.7234,
+      "step": 201
+    },
+    {
+      "epoch": 0.5094577553593947,
+      "grad_norm": 9.04035758972168,
+      "learning_rate": 7.5158428390367555e-06,
+      "loss": 1.9198,
+      "step": 202
+    },
+    {
+      "epoch": 0.5119798234552333,
+      "grad_norm": 8.787433624267578,
+      "learning_rate": 7.503168567807352e-06,
+      "loss": 1.8651,
+      "step": 203
+    },
+    {
+      "epoch": 0.5145018915510718,
+      "grad_norm": 8.92418098449707,
+      "learning_rate": 7.490494296577947e-06,
+      "loss": 1.8808,
+      "step": 204
+    },
+    {
+      "epoch": 0.5170239596469105,
+      "grad_norm": 9.472183227539062,
+      "learning_rate": 7.477820025348543e-06,
+      "loss": 1.8794,
+      "step": 205
+    },
+    {
+      "epoch": 0.519546027742749,
+      "grad_norm": 8.935132026672363,
+      "learning_rate": 7.465145754119138e-06,
+      "loss": 1.9661,
+      "step": 206
+    },
+    {
+      "epoch": 0.5220680958385876,
+      "grad_norm": 8.539314270019531,
+      "learning_rate": 7.452471482889735e-06,
+      "loss": 1.7449,
+      "step": 207
+    },
+    {
+      "epoch": 0.5245901639344263,
+      "grad_norm": 9.224102020263672,
+      "learning_rate": 7.4397972116603304e-06,
+      "loss": 1.7893,
+      "step": 208
+    },
+    {
+      "epoch": 0.5271122320302648,
+      "grad_norm": 8.306479454040527,
+      "learning_rate": 7.427122940430926e-06,
+      "loss": 1.6752,
+      "step": 209
+    },
+    {
+      "epoch": 0.5296343001261034,
+      "grad_norm": 8.810216903686523,
+      "learning_rate": 7.414448669201522e-06,
+      "loss": 1.9381,
+      "step": 210
+    },
+    {
+      "epoch": 0.532156368221942,
+      "grad_norm": 8.660805702209473,
+      "learning_rate": 7.401774397972117e-06,
+      "loss": 1.7787,
+      "step": 211
+    },
+    {
+      "epoch": 0.5346784363177806,
+      "grad_norm": 8.914039611816406,
+      "learning_rate": 7.389100126742713e-06,
+      "loss": 1.9574,
+      "step": 212
+    },
+    {
+      "epoch": 0.5372005044136192,
+      "grad_norm": 8.298178672790527,
+      "learning_rate": 7.376425855513308e-06,
+      "loss": 1.7711,
+      "step": 213
+    },
+    {
+      "epoch": 0.5397225725094578,
+      "grad_norm": 9.578728675842285,
+      "learning_rate": 7.3637515842839045e-06,
+      "loss": 1.8558,
+      "step": 214
+    },
+    {
+      "epoch": 0.5422446406052963,
+      "grad_norm": 9.25766658782959,
+      "learning_rate": 7.3510773130545e-06,
+      "loss": 1.887,
+      "step": 215
+    },
+    {
+      "epoch": 0.544766708701135,
+      "grad_norm": 9.36644458770752,
+      "learning_rate": 7.338403041825095e-06,
+      "loss": 1.8779,
+      "step": 216
+    },
+    {
+      "epoch": 0.5472887767969735,
+      "grad_norm": 9.4396333694458,
+      "learning_rate": 7.325728770595691e-06,
+      "loss": 1.8936,
+      "step": 217
+    },
+    {
+      "epoch": 0.5498108448928121,
+      "grad_norm": 8.876384735107422,
+      "learning_rate": 7.313054499366286e-06,
+      "loss": 1.8349,
+      "step": 218
+    },
+    {
+      "epoch": 0.5523329129886507,
+      "grad_norm": 9.552757263183594,
+      "learning_rate": 7.300380228136883e-06,
+      "loss": 1.9562,
+      "step": 219
+    },
+    {
+      "epoch": 0.5548549810844893,
+      "grad_norm": 8.633830070495605,
+      "learning_rate": 7.287705956907478e-06,
+      "loss": 1.7676,
+      "step": 220
+    },
+    {
+      "epoch": 0.5573770491803278,
+      "grad_norm": 9.029250144958496,
+      "learning_rate": 7.275031685678075e-06,
+      "loss": 1.8633,
+      "step": 221
+    },
+    {
+      "epoch": 0.5598991172761665,
+      "grad_norm": 9.104911804199219,
+      "learning_rate": 7.26235741444867e-06,
+      "loss": 1.9103,
+      "step": 222
+    },
+    {
+      "epoch": 0.562421185372005,
+      "grad_norm": 8.998291015625,
+      "learning_rate": 7.249683143219266e-06,
+      "loss": 1.6631,
+      "step": 223
+    },
+    {
+      "epoch": 0.5649432534678437,
+      "grad_norm": 9.102141380310059,
+      "learning_rate": 7.237008871989861e-06,
+      "loss": 1.8491,
+      "step": 224
+    },
+    {
+      "epoch": 0.5674653215636822,
+      "grad_norm": 8.94286823272705,
+      "learning_rate": 7.224334600760457e-06,
+      "loss": 1.6546,
+      "step": 225
+    },
+    {
+      "epoch": 0.5699873896595208,
+      "grad_norm": 9.074462890625,
+      "learning_rate": 7.211660329531053e-06,
+      "loss": 1.8034,
+      "step": 226
+    },
+    {
+      "epoch": 0.5725094577553594,
+      "grad_norm": 8.364324569702148,
+      "learning_rate": 7.198986058301648e-06,
+      "loss": 1.8561,
+      "step": 227
+    },
+    {
+      "epoch": 0.575031525851198,
+      "grad_norm": 8.235718727111816,
+      "learning_rate": 7.186311787072244e-06,
+      "loss": 1.6534,
+      "step": 228
+    },
+    {
+      "epoch": 0.5775535939470365,
+      "grad_norm": 8.920160293579102,
+      "learning_rate": 7.173637515842839e-06,
+      "loss": 1.8104,
+      "step": 229
+    },
+    {
+      "epoch": 0.5800756620428752,
+      "grad_norm": 9.526931762695312,
+      "learning_rate": 7.1609632446134354e-06,
+      "loss": 1.8956,
+      "step": 230
+    },
+    {
+      "epoch": 0.5825977301387137,
+      "grad_norm": 10.063536643981934,
+      "learning_rate": 7.148288973384031e-06,
+      "loss": 1.8132,
+      "step": 231
+    },
+    {
+      "epoch": 0.5851197982345523,
+      "grad_norm": 8.38437271118164,
+      "learning_rate": 7.135614702154626e-06,
+      "loss": 1.9233,
+      "step": 232
+    },
+    {
+      "epoch": 0.587641866330391,
+      "grad_norm": 10.298331260681152,
+      "learning_rate": 7.122940430925223e-06,
+      "loss": 1.8351,
+      "step": 233
+    },
+    {
+      "epoch": 0.5901639344262295,
+      "grad_norm": 8.098287582397461,
+      "learning_rate": 7.110266159695818e-06,
+      "loss": 1.672,
+      "step": 234
+    },
+    {
+      "epoch": 0.592686002522068,
+      "grad_norm": 9.341371536254883,
+      "learning_rate": 7.097591888466414e-06,
+      "loss": 1.7139,
+      "step": 235
+    },
+    {
+      "epoch": 0.5952080706179067,
+      "grad_norm": 9.039361953735352,
+      "learning_rate": 7.0849176172370095e-06,
+      "loss": 1.6284,
+      "step": 236
+    },
+    {
+      "epoch": 0.5977301387137453,
+      "grad_norm": 9.125579833984375,
+      "learning_rate": 7.072243346007606e-06,
+      "loss": 1.8376,
+      "step": 237
+    },
+    {
+      "epoch": 0.6002522068095839,
+      "grad_norm": 8.501140594482422,
+      "learning_rate": 7.059569074778201e-06,
+      "loss": 1.7236,
+      "step": 238
+    },
+    {
+      "epoch": 0.6027742749054225,
+      "grad_norm": 8.870810508728027,
+      "learning_rate": 7.046894803548796e-06,
+      "loss": 1.724,
+      "step": 239
+    },
+    {
+      "epoch": 0.605296343001261,
+      "grad_norm": 9.259521484375,
+      "learning_rate": 7.034220532319392e-06,
+      "loss": 1.7792,
+      "step": 240
+    },
+    {
+      "epoch": 0.6078184110970997,
+      "grad_norm": 10.289566993713379,
+      "learning_rate": 7.0215462610899875e-06,
+      "loss": 1.8516,
+      "step": 241
+    },
+    {
+      "epoch": 0.6103404791929382,
+      "grad_norm": 10.193709373474121,
+      "learning_rate": 7.008871989860584e-06,
+      "loss": 1.8237,
+      "step": 242
+    },
+    {
+      "epoch": 0.6128625472887768,
+      "grad_norm": 9.346511840820312,
+      "learning_rate": 6.996197718631179e-06,
+      "loss": 1.7183,
+      "step": 243
+    },
+    {
+      "epoch": 0.6153846153846154,
+      "grad_norm": 9.695211410522461,
+      "learning_rate": 6.983523447401776e-06,
+      "loss": 1.8324,
+      "step": 244
+    },
+    {
+      "epoch": 0.617906683480454,
+      "grad_norm": 8.961493492126465,
+      "learning_rate": 6.97084917617237e-06,
+      "loss": 1.8801,
+      "step": 245
+    },
+    {
+      "epoch": 0.6204287515762925,
+      "grad_norm": 9.273719787597656,
+      "learning_rate": 6.9581749049429655e-06,
+      "loss": 1.7128,
+      "step": 246
+    },
+    {
+      "epoch": 0.6229508196721312,
+      "grad_norm": 8.997072219848633,
+      "learning_rate": 6.9455006337135625e-06,
+      "loss": 1.8675,
+      "step": 247
+    },
+    {
+      "epoch": 0.6254728877679697,
+      "grad_norm": 8.742542266845703,
+      "learning_rate": 6.932826362484158e-06,
+      "loss": 1.853,
+      "step": 248
+    },
+    {
+      "epoch": 0.6279949558638083,
+      "grad_norm": 9.45918083190918,
+      "learning_rate": 6.920152091254754e-06,
+      "loss": 1.8785,
+      "step": 249
+    },
+    {
+      "epoch": 0.6305170239596469,
+      "grad_norm": 9.404273986816406,
+      "learning_rate": 6.907477820025349e-06,
+      "loss": 1.7804,
+      "step": 250
+    },
+    {
+      "epoch": 0.6305170239596469,
+      "eval_loss": 1.7830545902252197,
+      "eval_runtime": 18.8692,
+      "eval_samples_per_second": 37.362,
+      "eval_steps_per_second": 18.708,
+      "step": 250
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 794,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 2,
+  "save_steps": 50,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1346152388814336.0,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-250/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ec3d8df95ba642eec10e761db819efa4c74dbd8edeed27cfd2cb36ccafa00305
+size 6161