Add files using upload-large-folder tool

.gitattributes

@@ -33,3 +33,7 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-2865/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+merged/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-2750/tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,207 @@
+---
+base_model: Qwen/Qwen3.5-2B
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-2B
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.0

adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-2B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.0",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "k_proj",
+    "gate_proj",
+    "o_proj",
+    "down_proj",
+    "up_proj",
+    "q_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e54460326b97c66aced3c8ec3a50427b59111b42282d8638b4bbbe132d510518
+size 87319256

chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is true %}
+        {{- '<think>\n' }}
+    {%- else %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- endif %}
+{%- endif %}

checkpoint-2750/README.md

@@ -0,0 +1,207 @@
+---
+base_model: Qwen/Qwen3.5-2B
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-2B
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.0

checkpoint-2750/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-2B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.0",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "k_proj",
+    "gate_proj",
+    "o_proj",
+    "down_proj",
+    "up_proj",
+    "q_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-2750/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6b49ebcad405adb694c065950e282615812e3981bdca4202f2a79e151d3c1ec2
+size 87319256

checkpoint-2750/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is true %}
+        {{- '<think>\n' }}
+    {%- else %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- endif %}
+{%- endif %}

checkpoint-2750/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9b823bb8d08dc7f3e67974d7373180327c0c3b3f484279111011eeccb193952e
+size 174750283

checkpoint-2750/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:35b19cca0c77fd3faf9cb577574ebff9d16240a5010f338c8fd848717050f145
+size 14645

checkpoint-2750/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e4246573c193d4338561dd7c638ea83197ef8cbd56a1d02b874104194a5175da
+size 1465

checkpoint-2750/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:87a7830d63fcf43bf241c3c5242e96e62dd3fdc29224ca26fed8ea333db72de4
+size 19989343

checkpoint-2750/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": true,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

checkpoint-2750/trainer_state.json

@@ -0,0 +1,2047 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.8802724652868745,
+  "eval_steps": 250,
+  "global_step": 2750,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.010479434110558029,
+      "grad_norm": 0.19915591180324554,
+      "learning_rate": 1.0465116279069768e-05,
+      "loss": 1.1350045204162598,
+      "step": 10
+    },
+    {
+      "epoch": 0.020958868221116058,
+      "grad_norm": 0.18158815801143646,
+      "learning_rate": 2.2093023255813955e-05,
+      "loss": 1.0580164909362793,
+      "step": 20
+    },
+    {
+      "epoch": 0.03143830233167409,
+      "grad_norm": 0.16481591761112213,
+      "learning_rate": 3.372093023255814e-05,
+      "loss": 0.9252842903137207,
+      "step": 30
+    },
+    {
+      "epoch": 0.041917736442232116,
+      "grad_norm": 0.15599584579467773,
+      "learning_rate": 4.5348837209302326e-05,
+      "loss": 0.8342072486877441,
+      "step": 40
+    },
+    {
+      "epoch": 0.05239717055279015,
+      "grad_norm": 0.1804327368736267,
+      "learning_rate": 5.697674418604652e-05,
+      "loss": 0.7955524921417236,
+      "step": 50
+    },
+    {
+      "epoch": 0.06287660466334818,
+      "grad_norm": 0.16934047639369965,
+      "learning_rate": 6.86046511627907e-05,
+      "loss": 0.7358035087585449,
+      "step": 60
+    },
+    {
+      "epoch": 0.07335603877390622,
+      "grad_norm": 0.2234930843114853,
+      "learning_rate": 8.023255813953489e-05,
+      "loss": 0.6985861301422119,
+      "step": 70
+    },
+    {
+      "epoch": 0.08383547288446423,
+      "grad_norm": 0.16290400922298431,
+      "learning_rate": 9.186046511627907e-05,
+      "loss": 0.599607515335083,
+      "step": 80
+    },
+    {
+      "epoch": 0.09431490699502226,
+      "grad_norm": 0.1660464107990265,
+      "learning_rate": 9.999971245570617e-05,
+      "loss": 0.5886398315429687,
+      "step": 90
+    },
+    {
+      "epoch": 0.1047943411055803,
+      "grad_norm": 0.16978025436401367,
+      "learning_rate": 9.999460064915317e-05,
+      "loss": 0.5450529098510742,
+      "step": 100
+    },
+    {
+      "epoch": 0.11527377521613832,
+      "grad_norm": 0.21447990834712982,
+      "learning_rate": 9.998309972134645e-05,
+      "loss": 0.5072262287139893,
+      "step": 110
+    },
+    {
+      "epoch": 0.12575320932669637,
+      "grad_norm": 0.17418669164180756,
+      "learning_rate": 9.996521114206116e-05,
+      "loss": 0.49445347785949706,
+      "step": 120
+    },
+    {
+      "epoch": 0.13623264343725439,
+      "grad_norm": 0.22226351499557495,
+      "learning_rate": 9.994093719739023e-05,
+      "loss": 0.47142682075500486,
+      "step": 130
+    },
+    {
+      "epoch": 0.14671207754781243,
+      "grad_norm": 0.1745530068874359,
+      "learning_rate": 9.991028098945215e-05,
+      "loss": 0.46663532257080076,
+      "step": 140
+    },
+    {
+      "epoch": 0.15719151165837045,
+      "grad_norm": 0.17074695229530334,
+      "learning_rate": 9.987324643599459e-05,
+      "loss": 0.4508847236633301,
+      "step": 150
+    },
+    {
+      "epoch": 0.16767094576892846,
+      "grad_norm": 0.13428406417369843,
+      "learning_rate": 9.982983826989367e-05,
+      "loss": 0.40740265846252444,
+      "step": 160
+    },
+    {
+      "epoch": 0.1781503798794865,
+      "grad_norm": 0.17766578495502472,
+      "learning_rate": 9.978006203854918e-05,
+      "loss": 0.3998516321182251,
+      "step": 170
+    },
+    {
+      "epoch": 0.18862981399004453,
+      "grad_norm": 0.1672629565000534,
+      "learning_rate": 9.972392410317562e-05,
+      "loss": 0.41658673286437986,
+      "step": 180
+    },
+    {
+      "epoch": 0.19910924810060257,
+      "grad_norm": 0.1333673745393753,
+      "learning_rate": 9.96614316379892e-05,
+      "loss": 0.37024455070495604,
+      "step": 190
+    },
+    {
+      "epoch": 0.2095886822111606,
+      "grad_norm": 0.18037110567092896,
+      "learning_rate": 9.959259262929113e-05,
+      "loss": 0.35086841583251954,
+      "step": 200
+    },
+    {
+      "epoch": 0.22006811632171863,
+      "grad_norm": 0.14616410434246063,
+      "learning_rate": 9.951741587444683e-05,
+      "loss": 0.37918968200683595,
+      "step": 210
+    },
+    {
+      "epoch": 0.23054755043227665,
+      "grad_norm": 0.14523574709892273,
+      "learning_rate": 9.943591098076184e-05,
+      "loss": 0.32804527282714846,
+      "step": 220
+    },
+    {
+      "epoch": 0.2410269845428347,
+      "grad_norm": 0.14667049050331116,
+      "learning_rate": 9.934808836425393e-05,
+      "loss": 0.3480507850646973,
+      "step": 230
+    },
+    {
+      "epoch": 0.25150641865339274,
+      "grad_norm": 0.18156558275222778,
+      "learning_rate": 9.925395924832198e-05,
+      "loss": 0.3300448179244995,
+      "step": 240
+    },
+    {
+      "epoch": 0.26198585276395076,
+      "grad_norm": 0.13806430995464325,
+      "learning_rate": 9.91535356623117e-05,
+      "loss": 0.3127591609954834,
+      "step": 250
+    },
+    {
+      "epoch": 0.26198585276395076,
+      "eval_loss": 0.3132782578468323,
+      "eval_runtime": 94.8848,
+      "eval_samples_per_second": 3.278,
+      "eval_steps_per_second": 3.278,
+      "step": 250
+    },
+    {
+      "epoch": 0.27246528687450877,
+      "grad_norm": 0.17205959558486938,
+      "learning_rate": 9.904683043997835e-05,
+      "loss": 0.3306673288345337,
+      "step": 260
+    },
+    {
+      "epoch": 0.2829447209850668,
+      "grad_norm": 0.12620031833648682,
+      "learning_rate": 9.893385721784656e-05,
+      "loss": 0.3011106729507446,
+      "step": 270
+    },
+    {
+      "epoch": 0.29342415509562486,
+      "grad_norm": 0.11466006934642792,
+      "learning_rate": 9.881463043346768e-05,
+      "loss": 0.2951968669891357,
+      "step": 280
+    },
+    {
+      "epoch": 0.3039035892061829,
+      "grad_norm": 0.1671207845211029,
+      "learning_rate": 9.868916532357475e-05,
+      "loss": 0.2910990953445435,
+      "step": 290
+    },
+    {
+      "epoch": 0.3143830233167409,
+      "grad_norm": 0.1683349907398224,
+      "learning_rate": 9.855747792213521e-05,
+      "loss": 0.31409192085266113,
+      "step": 300
+    },
+    {
+      "epoch": 0.3248624574272989,
+      "grad_norm": 0.12934699654579163,
+      "learning_rate": 9.84195850583019e-05,
+      "loss": 0.27755858898162844,
+      "step": 310
+    },
+    {
+      "epoch": 0.33534189153785693,
+      "grad_norm": 0.13784605264663696,
+      "learning_rate": 9.827550435426234e-05,
+      "loss": 0.2809821605682373,
+      "step": 320
+    },
+    {
+      "epoch": 0.345821325648415,
+      "grad_norm": 0.18590271472930908,
+      "learning_rate": 9.812525422298664e-05,
+      "loss": 0.28698866367340087,
+      "step": 330
+    },
+    {
+      "epoch": 0.356300759758973,
+      "grad_norm": 0.1704522967338562,
+      "learning_rate": 9.796885386587447e-05,
+      "loss": 0.250814414024353,
+      "step": 340
+    },
+    {
+      "epoch": 0.36678019386953103,
+      "grad_norm": 0.1316167265176773,
+      "learning_rate": 9.780632327030112e-05,
+      "loss": 0.25458922386169436,
+      "step": 350
+    },
+    {
+      "epoch": 0.37725962798008905,
+      "grad_norm": 0.16226200759410858,
+      "learning_rate": 9.763768320706319e-05,
+      "loss": 0.26563262939453125,
+      "step": 360
+    },
+    {
+      "epoch": 0.3877390620906471,
+      "grad_norm": 0.1297195851802826,
+      "learning_rate": 9.746295522772424e-05,
+      "loss": 0.2632328748703003,
+      "step": 370
+    },
+    {
+      "epoch": 0.39821849620120514,
+      "grad_norm": 0.1286139190196991,
+      "learning_rate": 9.728216166186049e-05,
+      "loss": 0.2624588251113892,
+      "step": 380
+    },
+    {
+      "epoch": 0.40869793031176316,
+      "grad_norm": 0.1587965339422226,
+      "learning_rate": 9.709532561420725e-05,
+      "loss": 0.24741590023040771,
+      "step": 390
+    },
+    {
+      "epoch": 0.4191773644223212,
+      "grad_norm": 0.11963177472352982,
+      "learning_rate": 9.690247096170615e-05,
+      "loss": 0.22777397632598878,
+      "step": 400
+    },
+    {
+      "epoch": 0.42965679853287925,
+      "grad_norm": 0.13638927042484283,
+      "learning_rate": 9.670362235045387e-05,
+      "loss": 0.23324952125549317,
+      "step": 410
+    },
+    {
+      "epoch": 0.44013623264343726,
+      "grad_norm": 0.1514088362455368,
+      "learning_rate": 9.649880519255232e-05,
+      "loss": 0.2505915880203247,
+      "step": 420
+    },
+    {
+      "epoch": 0.4506156667539953,
+      "grad_norm": 0.10994207113981247,
+      "learning_rate": 9.62880456628612e-05,
+      "loss": 0.2078850269317627,
+      "step": 430
+    },
+    {
+      "epoch": 0.4610951008645533,
+      "grad_norm": 0.11983369290828705,
+      "learning_rate": 9.607137069565288e-05,
+      "loss": 0.21452484130859376,
+      "step": 440
+    },
+    {
+      "epoch": 0.47157453497511137,
+      "grad_norm": 0.12684305012226105,
+      "learning_rate": 9.58488079811703e-05,
+      "loss": 0.22002685070037842,
+      "step": 450
+    },
+    {
+      "epoch": 0.4820539690856694,
+      "grad_norm": 0.16841623187065125,
+      "learning_rate": 9.562038596208828e-05,
+      "loss": 0.21405396461486817,
+      "step": 460
+    },
+    {
+      "epoch": 0.4925334031962274,
+      "grad_norm": 0.1498555839061737,
+      "learning_rate": 9.538613382987865e-05,
+      "loss": 0.20534911155700683,
+      "step": 470
+    },
+    {
+      "epoch": 0.5030128373067855,
+      "grad_norm": 0.13913628458976746,
+      "learning_rate": 9.514608152107974e-05,
+      "loss": 0.22248730659484864,
+      "step": 480
+    },
+    {
+      "epoch": 0.5134922714173434,
+      "grad_norm": 0.14408951997756958,
+      "learning_rate": 9.490025971347047e-05,
+      "loss": 0.214866042137146,
+      "step": 490
+    },
+    {
+      "epoch": 0.5239717055279015,
+      "grad_norm": 0.1649770438671112,
+      "learning_rate": 9.464869982215001e-05,
+      "loss": 0.19965900182724,
+      "step": 500
+    },
+    {
+      "epoch": 0.5239717055279015,
+      "eval_loss": 0.19267401099205017,
+      "eval_runtime": 95.3374,
+      "eval_samples_per_second": 3.262,
+      "eval_steps_per_second": 3.262,
+      "step": 500
+    },
+    {
+      "epoch": 0.5344511396384595,
+      "grad_norm": 0.1305568665266037,
+      "learning_rate": 9.439143399552291e-05,
+      "loss": 0.21112546920776368,
+      "step": 510
+    },
+    {
+      "epoch": 0.5449305737490175,
+      "grad_norm": 0.11998175084590912,
+      "learning_rate": 9.412849511119074e-05,
+      "loss": 0.21422922611236572,
+      "step": 520
+    },
+    {
+      "epoch": 0.5554100078595756,
+      "grad_norm": 0.15220341086387634,
+      "learning_rate": 9.385991677175046e-05,
+      "loss": 0.20999882221221924,
+      "step": 530
+    },
+    {
+      "epoch": 0.5658894419701336,
+      "grad_norm": 0.13170023262500763,
+      "learning_rate": 9.358573330050004e-05,
+      "loss": 0.20208392143249512,
+      "step": 540
+    },
+    {
+      "epoch": 0.5763688760806917,
+      "grad_norm": 0.10457764565944672,
+      "learning_rate": 9.330597973705219e-05,
+      "loss": 0.1908803701400757,
+      "step": 550
+    },
+    {
+      "epoch": 0.5868483101912497,
+      "grad_norm": 0.12568537890911102,
+      "learning_rate": 9.302069183285637e-05,
+      "loss": 0.19316340684890748,
+      "step": 560
+    },
+    {
+      "epoch": 0.5973277443018077,
+      "grad_norm": 0.14824528992176056,
+      "learning_rate": 9.272990604662988e-05,
+      "loss": 0.18987581729888917,
+      "step": 570
+    },
+    {
+      "epoch": 0.6078071784123658,
+      "grad_norm": 0.14521734416484833,
+      "learning_rate": 9.243365953969861e-05,
+      "loss": 0.19232832193374633,
+      "step": 580
+    },
+    {
+      "epoch": 0.6182866125229237,
+      "grad_norm": 0.1335408091545105,
+      "learning_rate": 9.213199017124793e-05,
+      "loss": 0.1758212924003601,
+      "step": 590
+    },
+    {
+      "epoch": 0.6287660466334818,
+      "grad_norm": 0.11143071949481964,
+      "learning_rate": 9.182493649348447e-05,
+      "loss": 0.19117680788040162,
+      "step": 600
+    },
+    {
+      "epoch": 0.6392454807440399,
+      "grad_norm": 0.14789296686649323,
+      "learning_rate": 9.151253774670921e-05,
+      "loss": 0.184559965133667,
+      "step": 610
+    },
+    {
+      "epoch": 0.6497249148545978,
+      "grad_norm": 0.10541336238384247,
+      "learning_rate": 9.119483385430283e-05,
+      "loss": 0.1720304846763611,
+      "step": 620
+    },
+    {
+      "epoch": 0.6602043489651559,
+      "grad_norm": 0.12105975300073624,
+      "learning_rate": 9.087186541762358e-05,
+      "loss": 0.17654836177825928,
+      "step": 630
+    },
+    {
+      "epoch": 0.6706837830757139,
+      "grad_norm": 0.13114669919013977,
+      "learning_rate": 9.054367371081858e-05,
+      "loss": 0.1696592688560486,
+      "step": 640
+    },
+    {
+      "epoch": 0.6811632171862719,
+      "grad_norm": 0.13745592534542084,
+      "learning_rate": 9.021030067554919e-05,
+      "loss": 0.15404462814331055,
+      "step": 650
+    },
+    {
+      "epoch": 0.69164265129683,
+      "grad_norm": 0.15927442908287048,
+      "learning_rate": 8.987178891563094e-05,
+      "loss": 0.17024366855621337,
+      "step": 660
+    },
+    {
+      "epoch": 0.702122085407388,
+      "grad_norm": 0.13737429678440094,
+      "learning_rate": 8.952818169158903e-05,
+      "loss": 0.1602048397064209,
+      "step": 670
+    },
+    {
+      "epoch": 0.712601519517946,
+      "grad_norm": 0.13941751420497894,
+      "learning_rate": 8.91795229151297e-05,
+      "loss": 0.18057082891464232,
+      "step": 680
+    },
+    {
+      "epoch": 0.7230809536285041,
+      "grad_norm": 0.14242954552173615,
+      "learning_rate": 8.882585714352856e-05,
+      "loss": 0.14863334894180297,
+      "step": 690
+    },
+    {
+      "epoch": 0.7335603877390621,
+      "grad_norm": 0.15553542971611023,
+      "learning_rate": 8.846722957393626e-05,
+      "loss": 0.15701137781143187,
+      "step": 700
+    },
+    {
+      "epoch": 0.7440398218496201,
+      "grad_norm": 0.12901411950588226,
+      "learning_rate": 8.810368603760249e-05,
+      "loss": 0.15571318864822387,
+      "step": 710
+    },
+    {
+      "epoch": 0.7545192559601781,
+      "grad_norm": 0.13449430465698242,
+      "learning_rate": 8.773527299401902e-05,
+      "loss": 0.16418551206588744,
+      "step": 720
+    },
+    {
+      "epoch": 0.7649986900707362,
+      "grad_norm": 0.10630270838737488,
+      "learning_rate": 8.736203752498218e-05,
+      "loss": 0.16800801753997802,
+      "step": 730
+    },
+    {
+      "epoch": 0.7754781241812942,
+      "grad_norm": 0.11299935728311539,
+      "learning_rate": 8.698402732857611e-05,
+      "loss": 0.15700833797454833,
+      "step": 740
+    },
+    {
+      "epoch": 0.7859575582918522,
+      "grad_norm": 0.11920930445194244,
+      "learning_rate": 8.660129071307707e-05,
+      "loss": 0.15091001987457275,
+      "step": 750
+    },
+    {
+      "epoch": 0.7859575582918522,
+      "eval_loss": 0.1356429010629654,
+      "eval_runtime": 94.0557,
+      "eval_samples_per_second": 3.307,
+      "eval_steps_per_second": 3.307,
+      "step": 750
+    },
+    {
+      "epoch": 0.7964369924024103,
+      "grad_norm": 0.13870343565940857,
+      "learning_rate": 8.621387659077986e-05,
+      "loss": 0.1422027826309204,
+      "step": 760
+    },
+    {
+      "epoch": 0.8069164265129684,
+      "grad_norm": 0.12753477692604065,
+      "learning_rate": 8.582183447174697e-05,
+      "loss": 0.142450213432312,
+      "step": 770
+    },
+    {
+      "epoch": 0.8173958606235263,
+      "grad_norm": 0.11877496540546417,
+      "learning_rate": 8.542521445748141e-05,
+      "loss": 0.15361062288284302,
+      "step": 780
+    },
+    {
+      "epoch": 0.8278752947340844,
+      "grad_norm": 0.1200249195098877,
+      "learning_rate": 8.502406723452392e-05,
+      "loss": 0.14647477865219116,
+      "step": 790
+    },
+    {
+      "epoch": 0.8383547288446423,
+      "grad_norm": 0.12913794815540314,
+      "learning_rate": 8.461844406797543e-05,
+      "loss": 0.1591552734375,
+      "step": 800
+    },
+    {
+      "epoch": 0.8488341629552004,
+      "grad_norm": 0.17270176112651825,
+      "learning_rate": 8.420839679494558e-05,
+      "loss": 0.1495436668395996,
+      "step": 810
+    },
+    {
+      "epoch": 0.8593135970657585,
+      "grad_norm": 0.15545596182346344,
+      "learning_rate": 8.379397781792808e-05,
+      "loss": 0.15377395153045653,
+      "step": 820
+    },
+    {
+      "epoch": 0.8697930311763165,
+      "grad_norm": 0.12941111624240875,
+      "learning_rate": 8.337524009810395e-05,
+      "loss": 0.14733861684799193,
+      "step": 830
+    },
+    {
+      "epoch": 0.8802724652868745,
+      "grad_norm": 0.13152749836444855,
+      "learning_rate": 8.295223714857319e-05,
+      "loss": 0.13980752229690552,
+      "step": 840
+    },
+    {
+      "epoch": 0.8907518993974325,
+      "grad_norm": 0.11208872497081757,
+      "learning_rate": 8.252502302751612e-05,
+      "loss": 0.12019969224929809,
+      "step": 850
+    },
+    {
+      "epoch": 0.9012313335079906,
+      "grad_norm": 0.11118603497743607,
+      "learning_rate": 8.209365233128482e-05,
+      "loss": 0.13822466135025024,
+      "step": 860
+    },
+    {
+      "epoch": 0.9117107676185486,
+      "grad_norm": 0.11705653369426727,
+      "learning_rate": 8.165818018742605e-05,
+      "loss": 0.1439664840698242,
+      "step": 870
+    },
+    {
+      "epoch": 0.9221902017291066,
+      "grad_norm": 0.08817730098962784,
+      "learning_rate": 8.121866224763606e-05,
+      "loss": 0.13380355834960939,
+      "step": 880
+    },
+    {
+      "epoch": 0.9326696358396647,
+      "grad_norm": 0.1092257872223854,
+      "learning_rate": 8.077515468064851e-05,
+      "loss": 0.12982802391052245,
+      "step": 890
+    },
+    {
+      "epoch": 0.9431490699502227,
+      "grad_norm": 0.12680962681770325,
+      "learning_rate": 8.032771416505647e-05,
+      "loss": 0.1489071011543274,
+      "step": 900
+    },
+    {
+      "epoch": 0.9536285040607807,
+      "grad_norm": 0.11953219771385193,
+      "learning_rate": 7.987639788206888e-05,
+      "loss": 0.14020267724990845,
+      "step": 910
+    },
+    {
+      "epoch": 0.9641079381713388,
+      "grad_norm": 0.1041467934846878,
+      "learning_rate": 7.942126350820318e-05,
+      "loss": 0.1439213275909424,
+      "step": 920
+    },
+    {
+      "epoch": 0.9745873722818967,
+      "grad_norm": 0.1277916431427002,
+      "learning_rate": 7.896236920791442e-05,
+      "loss": 0.1468779683113098,
+      "step": 930
+    },
+    {
+      "epoch": 0.9850668063924548,
+      "grad_norm": 0.11245205253362656,
+      "learning_rate": 7.849977362616201e-05,
+      "loss": 0.12012372016906739,
+      "step": 940
+    },
+    {
+      "epoch": 0.9955462405030129,
+      "grad_norm": 0.12230483442544937,
+      "learning_rate": 7.803353588091522e-05,
+      "loss": 0.1488939881324768,
+      "step": 950
+    },
+    {
+      "epoch": 1.005239717055279,
+      "grad_norm": 0.14185865223407745,
+      "learning_rate": 7.7563715555598e-05,
+      "loss": 0.11488113403320313,
+      "step": 960
+    },
+    {
+      "epoch": 1.015719151165837,
+      "grad_norm": 0.10545773804187775,
+      "learning_rate": 7.709037269147459e-05,
+      "loss": 0.10712549686431885,
+      "step": 970
+    },
+    {
+      "epoch": 1.026198585276395,
+      "grad_norm": 0.10376274585723877,
+      "learning_rate": 7.661356777997631e-05,
+      "loss": 0.11428828239440918,
+      "step": 980
+    },
+    {
+      "epoch": 1.0366780193869531,
+      "grad_norm": 0.09950564056634903,
+      "learning_rate": 7.613336175497111e-05,
+      "loss": 0.09823058247566223,
+      "step": 990
+    },
+    {
+      "epoch": 1.0471574534975112,
+      "grad_norm": 0.10412753373384476,
+      "learning_rate": 7.564981598497643e-05,
+      "loss": 0.1106558084487915,
+      "step": 1000
+    },
+    {
+      "epoch": 1.0471574534975112,
+      "eval_loss": 0.11185819655656815,
+      "eval_runtime": 93.808,
+      "eval_samples_per_second": 3.315,
+      "eval_steps_per_second": 3.315,
+      "step": 1000
+    },
+    {
+      "epoch": 1.057636887608069,
+      "grad_norm": 0.10430868715047836,
+      "learning_rate": 7.516299226531645e-05,
+      "loss": 0.11168640851974487,
+      "step": 1010
+    },
+    {
+      "epoch": 1.0681163217186271,
+      "grad_norm": 0.09646806865930557,
+      "learning_rate": 7.467295281022501e-05,
+      "loss": 0.10711305141448975,
+      "step": 1020
+    },
+    {
+      "epoch": 1.0785957558291852,
+      "grad_norm": 0.13060614466667175,
+      "learning_rate": 7.417976024489474e-05,
+      "loss": 0.10001810789108276,
+      "step": 1030
+    },
+    {
+      "epoch": 1.0890751899397433,
+      "grad_norm": 0.10389085114002228,
+      "learning_rate": 7.368347759747393e-05,
+      "loss": 0.11893858909606933,
+      "step": 1040
+    },
+    {
+      "epoch": 1.0995546240503014,
+      "grad_norm": 0.11291550099849701,
+      "learning_rate": 7.318416829101164e-05,
+      "loss": 0.1079628586769104,
+      "step": 1050
+    },
+    {
+      "epoch": 1.1100340581608594,
+      "grad_norm": 0.10372598469257355,
+      "learning_rate": 7.268189613535255e-05,
+      "loss": 0.10332397222518921,
+      "step": 1060
+    },
+    {
+      "epoch": 1.1205134922714173,
+      "grad_norm": 0.12971536815166473,
+      "learning_rate": 7.217672531898225e-05,
+      "loss": 0.10804877281188965,
+      "step": 1070
+    },
+    {
+      "epoch": 1.1309929263819753,
+      "grad_norm": 0.10902425646781921,
+      "learning_rate": 7.166872040082431e-05,
+      "loss": 0.09947454929351807,
+      "step": 1080
+    },
+    {
+      "epoch": 1.1414723604925334,
+      "grad_norm": 0.09305932372808456,
+      "learning_rate": 7.11579463019897e-05,
+      "loss": 0.09406971335411071,
+      "step": 1090
+    },
+    {
+      "epoch": 1.1519517946030915,
+      "grad_norm": 0.11485275626182556,
+      "learning_rate": 7.064446829748034e-05,
+      "loss": 0.09943979978561401,
+      "step": 1100
+    },
+    {
+      "epoch": 1.1624312287136496,
+      "grad_norm": 0.09556467831134796,
+      "learning_rate": 7.0128352007847e-05,
+      "loss": 0.10862170457839966,
+      "step": 1110
+    },
+    {
+      "epoch": 1.1729106628242074,
+      "grad_norm": 0.11937833577394485,
+      "learning_rate": 6.96096633908034e-05,
+      "loss": 0.10385221242904663,
+      "step": 1120
+    },
+    {
+      "epoch": 1.1833900969347655,
+      "grad_norm": 0.11560507863759995,
+      "learning_rate": 6.908846873279691e-05,
+      "loss": 0.09252402186393738,
+      "step": 1130
+    },
+    {
+      "epoch": 1.1938695310453236,
+      "grad_norm": 0.11119654029607773,
+      "learning_rate": 6.856483464053758e-05,
+      "loss": 0.09637172818183899,
+      "step": 1140
+    },
+    {
+      "epoch": 1.2043489651558816,
+      "grad_norm": 0.11722644418478012,
+      "learning_rate": 6.803882803248585e-05,
+      "loss": 0.09078751802444458,
+      "step": 1150
+    },
+    {
+      "epoch": 1.2148283992664397,
+      "grad_norm": 0.10487739741802216,
+      "learning_rate": 6.751051613030082e-05,
+      "loss": 0.10334972143173218,
+      "step": 1160
+    },
+    {
+      "epoch": 1.2253078333769976,
+      "grad_norm": 0.10202383995056152,
+      "learning_rate": 6.697996645024937e-05,
+      "loss": 0.08661433458328247,
+      "step": 1170
+    },
+    {
+      "epoch": 1.2357872674875556,
+      "grad_norm": 0.11801143735647202,
+      "learning_rate": 6.644724679457804e-05,
+      "loss": 0.0997927188873291,
+      "step": 1180
+    },
+    {
+      "epoch": 1.2462667015981137,
+      "grad_norm": 0.10949107259511948,
+      "learning_rate": 6.591242524284802e-05,
+      "loss": 0.0977592945098877,
+      "step": 1190
+    },
+    {
+      "epoch": 1.2567461357086718,
+      "grad_norm": 0.10221222043037415,
+      "learning_rate": 6.537557014323487e-05,
+      "loss": 0.0970361053943634,
+      "step": 1200
+    },
+    {
+      "epoch": 1.2672255698192298,
+      "grad_norm": 0.10554748773574829,
+      "learning_rate": 6.483675010379393e-05,
+      "loss": 0.09007551074028015,
+      "step": 1210
+    },
+    {
+      "epoch": 1.2777050039297877,
+      "grad_norm": 0.11625627428293228,
+      "learning_rate": 6.429603398369242e-05,
+      "loss": 0.08734490275382996,
+      "step": 1220
+    },
+    {
+      "epoch": 1.2881844380403458,
+      "grad_norm": 0.10624277591705322,
+      "learning_rate": 6.37534908844095e-05,
+      "loss": 0.09858485460281372,
+      "step": 1230
+    },
+    {
+      "epoch": 1.2986638721509038,
+      "grad_norm": 0.10184557735919952,
+      "learning_rate": 6.320919014090534e-05,
+      "loss": 0.09335023164749146,
+      "step": 1240
+    },
+    {
+      "epoch": 1.309143306261462,
+      "grad_norm": 0.10787283629179001,
+      "learning_rate": 6.266320131276051e-05,
+      "loss": 0.08665563464164734,
+      "step": 1250
+    },
+    {
+      "epoch": 1.309143306261462,
+      "eval_loss": 0.08951585739850998,
+      "eval_runtime": 94.0567,
+      "eval_samples_per_second": 3.307,
+      "eval_steps_per_second": 3.307,
+      "step": 1250
+    },
+    {
+      "epoch": 1.31962274037202,
+      "grad_norm": 0.10836981981992722,
+      "learning_rate": 6.211559417528631e-05,
+      "loss": 0.0933380126953125,
+      "step": 1260
+    },
+    {
+      "epoch": 1.3301021744825778,
+      "grad_norm": 0.1397171914577484,
+      "learning_rate": 6.156643871060795e-05,
+      "loss": 0.09835371971130372,
+      "step": 1270
+    },
+    {
+      "epoch": 1.340581608593136,
+      "grad_norm": 0.11242218315601349,
+      "learning_rate": 6.101580509872097e-05,
+      "loss": 0.09398673176765442,
+      "step": 1280
+    },
+    {
+      "epoch": 1.351061042703694,
+      "grad_norm": 0.10235017538070679,
+      "learning_rate": 6.0463763708522536e-05,
+      "loss": 0.10350929498672486,
+      "step": 1290
+    },
+    {
+      "epoch": 1.361540476814252,
+      "grad_norm": 0.09327106177806854,
+      "learning_rate": 5.99103850888186e-05,
+      "loss": 0.09580238461494446,
+      "step": 1300
+    },
+    {
+      "epoch": 1.3720199109248101,
+      "grad_norm": 0.12995658814907074,
+      "learning_rate": 5.9355739959307976e-05,
+      "loss": 0.08437412977218628,
+      "step": 1310
+    },
+    {
+      "epoch": 1.382499345035368,
+      "grad_norm": 0.11962983757257462,
+      "learning_rate": 5.879989920154466e-05,
+      "loss": 0.08409937620162963,
+      "step": 1320
+    },
+    {
+      "epoch": 1.392978779145926,
+      "grad_norm": 0.09431737661361694,
+      "learning_rate": 5.824293384987941e-05,
+      "loss": 0.09504773020744324,
+      "step": 1330
+    },
+    {
+      "epoch": 1.4034582132564841,
+      "grad_norm": 0.13824374973773956,
+      "learning_rate": 5.768491508238188e-05,
+      "loss": 0.09193333983421326,
+      "step": 1340
+    },
+    {
+      "epoch": 1.4139376473670422,
+      "grad_norm": 0.10595858097076416,
+      "learning_rate": 5.712591421174422e-05,
+      "loss": 0.08976472616195678,
+      "step": 1350
+    },
+    {
+      "epoch": 1.4244170814776003,
+      "grad_norm": 0.09911809861660004,
+      "learning_rate": 5.6566002676167725e-05,
+      "loss": 0.07597061395645141,
+      "step": 1360
+    },
+    {
+      "epoch": 1.4348965155881581,
+      "grad_norm": 0.09723466634750366,
+      "learning_rate": 5.60052520302332e-05,
+      "loss": 0.10513757467269898,
+      "step": 1370
+    },
+    {
+      "epoch": 1.4453759496987162,
+      "grad_norm": 0.11331687867641449,
+      "learning_rate": 5.5443733935756615e-05,
+      "loss": 0.09019948840141297,
+      "step": 1380
+    },
+    {
+      "epoch": 1.4558553838092743,
+      "grad_norm": 0.13363589346408844,
+      "learning_rate": 5.4881520152630886e-05,
+      "loss": 0.08314153552055359,
+      "step": 1390
+    },
+    {
+      "epoch": 1.4663348179198323,
+      "grad_norm": 0.14111892879009247,
+      "learning_rate": 5.4318682529655404e-05,
+      "loss": 0.07892010807991028,
+      "step": 1400
+    },
+    {
+      "epoch": 1.4768142520303904,
+      "grad_norm": 0.13948485255241394,
+      "learning_rate": 5.3755292995353913e-05,
+      "loss": 0.0840128481388092,
+      "step": 1410
+    },
+    {
+      "epoch": 1.4872936861409483,
+      "grad_norm": 0.12535949051380157,
+      "learning_rate": 5.31914235487823e-05,
+      "loss": 0.07869629859924317,
+      "step": 1420
+    },
+    {
+      "epoch": 1.4977731202515066,
+      "grad_norm": 0.10041694343090057,
+      "learning_rate": 5.2627146250327484e-05,
+      "loss": 0.08074848055839538,
+      "step": 1430
+    },
+    {
+      "epoch": 1.5082525543620644,
+      "grad_norm": 0.10112891346216202,
+      "learning_rate": 5.2062533212498275e-05,
+      "loss": 0.0860810935497284,
+      "step": 1440
+    },
+    {
+      "epoch": 1.5187319884726225,
+      "grad_norm": 0.11297477036714554,
+      "learning_rate": 5.149765659070973e-05,
+      "loss": 0.08794642686843872,
+      "step": 1450
+    },
+    {
+      "epoch": 1.5292114225831805,
+      "grad_norm": 0.10511091351509094,
+      "learning_rate": 5.0932588574061945e-05,
+      "loss": 0.07854819297790527,
+      "step": 1460
+    },
+    {
+      "epoch": 1.5396908566937384,
+      "grad_norm": 0.09333530068397522,
+      "learning_rate": 5.036740137611453e-05,
+      "loss": 0.08821435570716858,
+      "step": 1470
+    },
+    {
+      "epoch": 1.5501702908042967,
+      "grad_norm": 0.11480343341827393,
+      "learning_rate": 4.980216722565804e-05,
+      "loss": 0.08062278628349304,
+      "step": 1480
+    },
+    {
+      "epoch": 1.5606497249148545,
+      "grad_norm": 0.08406255394220352,
+      "learning_rate": 4.923695835748338e-05,
+      "loss": 0.0940588355064392,
+      "step": 1490
+    },
+    {
+      "epoch": 1.5711291590254126,
+      "grad_norm": 0.12927693128585815,
+      "learning_rate": 4.8671847003150447e-05,
+      "loss": 0.0775177538394928,
+      "step": 1500
+    },
+    {
+      "epoch": 1.5711291590254126,
+      "eval_loss": 0.07877222448587418,
+      "eval_runtime": 34.4389,
+      "eval_samples_per_second": 9.03,
+      "eval_steps_per_second": 9.03,
+      "step": 1500
+    },
+    {
+      "epoch": 1.5816085931359707,
+      "grad_norm": 0.1255076378583908,
+      "learning_rate": 4.810690538175728e-05,
+      "loss": 0.09362970590591431,
+      "step": 1510
+    },
+    {
+      "epoch": 1.5920880272465285,
+      "grad_norm": 0.1326853185892105,
+      "learning_rate": 4.754220569071068e-05,
+      "loss": 0.08364834189414978,
+      "step": 1520
+    },
+    {
+      "epoch": 1.6025674613570868,
+      "grad_norm": 0.10229979455471039,
+      "learning_rate": 4.697782009649962e-05,
+      "loss": 0.0725843846797943,
+      "step": 1530
+    },
+    {
+      "epoch": 1.6130468954676447,
+      "grad_norm": 0.11407258361577988,
+      "learning_rate": 4.641382072547272e-05,
+      "loss": 0.07566151022911072,
+      "step": 1540
+    },
+    {
+      "epoch": 1.6235263295782028,
+      "grad_norm": 0.09398165345191956,
+      "learning_rate": 4.585027965462075e-05,
+      "loss": 0.087736576795578,
+      "step": 1550
+    },
+    {
+      "epoch": 1.6340057636887608,
+      "grad_norm": 0.11289424449205399,
+      "learning_rate": 4.528726890236544e-05,
+      "loss": 0.08366051316261292,
+      "step": 1560
+    },
+    {
+      "epoch": 1.6444851977993187,
+      "grad_norm": 0.09478718787431717,
+      "learning_rate": 4.4724860419355746e-05,
+      "loss": 0.0885531723499298,
+      "step": 1570
+    },
+    {
+      "epoch": 1.654964631909877,
+      "grad_norm": 0.09163404256105423,
+      "learning_rate": 4.416312607927295e-05,
+      "loss": 0.08392030596733094,
+      "step": 1580
+    },
+    {
+      "epoch": 1.6654440660204348,
+      "grad_norm": 0.11422222852706909,
+      "learning_rate": 4.360213766964542e-05,
+      "loss": 0.08059985041618348,
+      "step": 1590
+    },
+    {
+      "epoch": 1.675923500130993,
+      "grad_norm": 0.08131479471921921,
+      "learning_rate": 4.304196688267438e-05,
+      "loss": 0.07613803148269653,
+      "step": 1600
+    },
+    {
+      "epoch": 1.686402934241551,
+      "grad_norm": 0.09615079313516617,
+      "learning_rate": 4.248268530607199e-05,
+      "loss": 0.07764078378677368,
+      "step": 1610
+    },
+    {
+      "epoch": 1.696882368352109,
+      "grad_norm": 0.09730526059865952,
+      "learning_rate": 4.192436441391271e-05,
+      "loss": 0.07644452452659607,
+      "step": 1620
+    },
+    {
+      "epoch": 1.707361802462667,
+      "grad_norm": 0.09649327397346497,
+      "learning_rate": 4.136707555749907e-05,
+      "loss": 0.07866159081459045,
+      "step": 1630
+    },
+    {
+      "epoch": 1.717841236573225,
+      "grad_norm": 0.11804413050413132,
+      "learning_rate": 4.0810889956243415e-05,
+      "loss": 0.06996130347251892,
+      "step": 1640
+    },
+    {
+      "epoch": 1.728320670683783,
+      "grad_norm": 0.09874672442674637,
+      "learning_rate": 4.025587868856622e-05,
+      "loss": 0.07877404093742371,
+      "step": 1650
+    },
+    {
+      "epoch": 1.738800104794341,
+      "grad_norm": 0.11149467527866364,
+      "learning_rate": 3.9702112682812544e-05,
+      "loss": 0.07241421341896057,
+      "step": 1660
+    },
+    {
+      "epoch": 1.7492795389048992,
+      "grad_norm": 0.08748896420001984,
+      "learning_rate": 3.914966270818766e-05,
+      "loss": 0.07336459755897522,
+      "step": 1670
+    },
+    {
+      "epoch": 1.7597589730154573,
+      "grad_norm": 0.1172696202993393,
+      "learning_rate": 3.859859936571307e-05,
+      "loss": 0.07742337584495544,
+      "step": 1680
+    },
+    {
+      "epoch": 1.770238407126015,
+      "grad_norm": 0.0719197615981102,
+      "learning_rate": 3.8048993079203925e-05,
+      "loss": 0.06242966651916504,
+      "step": 1690
+    },
+    {
+      "epoch": 1.7807178412365732,
+      "grad_norm": 0.12380168586969376,
+      "learning_rate": 3.750091408626907e-05,
+      "loss": 0.07270430326461792,
+      "step": 1700
+    },
+    {
+      "epoch": 1.7911972753471312,
+      "grad_norm": 0.1587221622467041,
+      "learning_rate": 3.6954432429335015e-05,
+      "loss": 0.06409866213798524,
+      "step": 1710
+    },
+    {
+      "epoch": 1.8016767094576893,
+      "grad_norm": 0.10983912646770477,
+      "learning_rate": 3.640961794669482e-05,
+      "loss": 0.06610031127929687,
+      "step": 1720
+    },
+    {
+      "epoch": 1.8121561435682474,
+      "grad_norm": 0.11023026704788208,
+      "learning_rate": 3.586654026358287e-05,
+      "loss": 0.06866579055786133,
+      "step": 1730
+    },
+    {
+      "epoch": 1.8226355776788052,
+      "grad_norm": 0.11857719719409943,
+      "learning_rate": 3.532526878327719e-05,
+      "loss": 0.06734356880187989,
+      "step": 1740
+    },
+    {
+      "epoch": 1.8331150117893635,
+      "grad_norm": 0.09280339628458023,
+      "learning_rate": 3.478587267822987e-05,
+      "loss": 0.06897796392440796,
+      "step": 1750
+    },
+    {
+      "epoch": 1.8331150117893635,
+      "eval_loss": 0.06596127897500992,
+      "eval_runtime": 35.5001,
+      "eval_samples_per_second": 8.761,
+      "eval_steps_per_second": 8.761,
+      "step": 1750
+    },
+    {
+      "epoch": 1.8435944458999214,
+      "grad_norm": 0.1175367683172226,
+      "learning_rate": 3.424842088122716e-05,
+      "loss": 0.08288194537162781,
+      "step": 1760
+    },
+    {
+      "epoch": 1.8540738800104795,
+      "grad_norm": 0.10271462798118591,
+      "learning_rate": 3.371298207658003e-05,
+      "loss": 0.05643013119697571,
+      "step": 1770
+    },
+    {
+      "epoch": 1.8645533141210375,
+      "grad_norm": 0.11965195834636688,
+      "learning_rate": 3.3179624691346654e-05,
+      "loss": 0.07403092980384826,
+      "step": 1780
+    },
+    {
+      "epoch": 1.8750327482315954,
+      "grad_norm": 0.09981680661439896,
+      "learning_rate": 3.2648416886587686e-05,
+      "loss": 0.07118859887123108,
+      "step": 1790
+    },
+    {
+      "epoch": 1.8855121823421537,
+      "grad_norm": 0.07787375897169113,
+      "learning_rate": 3.2119426548655435e-05,
+      "loss": 0.07219682335853576,
+      "step": 1800
+    },
+    {
+      "epoch": 1.8959916164527115,
+      "grad_norm": 0.1303507387638092,
+      "learning_rate": 3.1592721280518404e-05,
+      "loss": 0.07636030912399291,
+      "step": 1810
+    },
+    {
+      "epoch": 1.9064710505632696,
+      "grad_norm": 0.09162267297506332,
+      "learning_rate": 3.106836839312175e-05,
+      "loss": 0.06230143308639526,
+      "step": 1820
+    },
+    {
+      "epoch": 1.9169504846738277,
+      "grad_norm": 0.11375878751277924,
+      "learning_rate": 3.054643489678526e-05,
+      "loss": 0.060506826639175414,
+      "step": 1830
+    },
+    {
+      "epoch": 1.9274299187843855,
+      "grad_norm": 0.1377716213464737,
+      "learning_rate": 3.0026987492639668e-05,
+      "loss": 0.08148540854454041,
+      "step": 1840
+    },
+    {
+      "epoch": 1.9379093528949438,
+      "grad_norm": 0.10483554750680923,
+      "learning_rate": 2.951009256410255e-05,
+      "loss": 0.07040726542472839,
+      "step": 1850
+    },
+    {
+      "epoch": 1.9483887870055017,
+      "grad_norm": 0.08736151456832886,
+      "learning_rate": 2.8995816168394702e-05,
+      "loss": 0.04931557774543762,
+      "step": 1860
+    },
+    {
+      "epoch": 1.9588682211160597,
+      "grad_norm": 0.11461569368839264,
+      "learning_rate": 2.848422402809828e-05,
+      "loss": 0.057559752464294435,
+      "step": 1870
+    },
+    {
+      "epoch": 1.9693476552266178,
+      "grad_norm": 0.09060918539762497,
+      "learning_rate": 2.7975381522757803e-05,
+      "loss": 0.06379705667495728,
+      "step": 1880
+    },
+    {
+      "epoch": 1.9798270893371757,
+      "grad_norm": 0.07104971259832382,
+      "learning_rate": 2.746935368052477e-05,
+      "loss": 0.05813115239143372,
+      "step": 1890
+    },
+    {
+      "epoch": 1.990306523447734,
+      "grad_norm": 0.10802938044071198,
+      "learning_rate": 2.696620516984733e-05,
+      "loss": 0.07732833027839661,
+      "step": 1900
+    },
+    {
+      "epoch": 2.0,
+      "grad_norm": 0.16884952783584595,
+      "learning_rate": 2.6466000291206004e-05,
+      "loss": 0.06166202425956726,
+      "step": 1910
+    },
+    {
+      "epoch": 2.010479434110558,
+      "grad_norm": 0.08582179993391037,
+      "learning_rate": 2.5968802968896228e-05,
+      "loss": 0.04766199886798859,
+      "step": 1920
+    },
+    {
+      "epoch": 2.020958868221116,
+      "grad_norm": 0.1457364708185196,
+      "learning_rate": 2.5474676742859048e-05,
+      "loss": 0.03826354146003723,
+      "step": 1930
+    },
+    {
+      "epoch": 2.031438302331674,
+      "grad_norm": 0.09275342524051666,
+      "learning_rate": 2.4983684760561023e-05,
+      "loss": 0.045059433579444884,
+      "step": 1940
+    },
+    {
+      "epoch": 2.0419177364422323,
+      "grad_norm": 0.09085927903652191,
+      "learning_rate": 2.44958897689242e-05,
+      "loss": 0.04904903173446655,
+      "step": 1950
+    },
+    {
+      "epoch": 2.05239717055279,
+      "grad_norm": 0.11733179539442062,
+      "learning_rate": 2.401135410630731e-05,
+      "loss": 0.05008396506309509,
+      "step": 1960
+    },
+    {
+      "epoch": 2.062876604663348,
+      "grad_norm": 0.0894237607717514,
+      "learning_rate": 2.3530139694539095e-05,
+      "loss": 0.04057626128196716,
+      "step": 1970
+    },
+    {
+      "epoch": 2.0733560387739063,
+      "grad_norm": 0.08560927212238312,
+      "learning_rate": 2.305230803100496e-05,
+      "loss": 0.04843136668205261,
+      "step": 1980
+    },
+    {
+      "epoch": 2.083835472884464,
+      "grad_norm": 0.07991836220026016,
+      "learning_rate": 2.257792018078793e-05,
+      "loss": 0.0544127106666565,
+      "step": 1990
+    },
+    {
+      "epoch": 2.0943149069950224,
+      "grad_norm": 0.08846250921487808,
+      "learning_rate": 2.210703676886461e-05,
+      "loss": 0.0459000825881958,
+      "step": 2000
+    },
+    {
+      "epoch": 2.0943149069950224,
+      "eval_loss": 0.060011014342308044,
+      "eval_runtime": 36.3755,
+      "eval_samples_per_second": 8.55,
+      "eval_steps_per_second": 8.55,
+      "step": 2000
+    },
+    {
+      "epoch": 2.1047943411055803,
+      "grad_norm": 0.10082945972681046,
+      "learning_rate": 2.1639717972357678e-05,
+      "loss": 0.038090622425079344,
+      "step": 2010
+    },
+    {
+      "epoch": 2.115273775216138,
+      "grad_norm": 0.05712248757481575,
+      "learning_rate": 2.1176023512845376e-05,
+      "loss": 0.04598597884178161,
+      "step": 2020
+    },
+    {
+      "epoch": 2.1257532093266964,
+      "grad_norm": 0.11628362536430359,
+      "learning_rate": 2.0716012648729353e-05,
+      "loss": 0.04984880685806274,
+      "step": 2030
+    },
+    {
+      "epoch": 2.1362326434372543,
+      "grad_norm": 0.10635484755039215,
+      "learning_rate": 2.025974416766171e-05,
+      "loss": 0.04293925166130066,
+      "step": 2040
+    },
+    {
+      "epoch": 2.1467120775478126,
+      "grad_norm": 0.1017381027340889,
+      "learning_rate": 1.9807276379032113e-05,
+      "loss": 0.04305694401264191,
+      "step": 2050
+    },
+    {
+      "epoch": 2.1571915116583704,
+      "grad_norm": 0.13550882041454315,
+      "learning_rate": 1.9358667106516055e-05,
+      "loss": 0.04478869140148163,
+      "step": 2060
+    },
+    {
+      "epoch": 2.1676709457689283,
+      "grad_norm": 0.08526366949081421,
+      "learning_rate": 1.8913973680685226e-05,
+      "loss": 0.036646312475204466,
+      "step": 2070
+    },
+    {
+      "epoch": 2.1781503798794866,
+      "grad_norm": 0.10932011157274246,
+      "learning_rate": 1.8473252931680928e-05,
+      "loss": 0.042200219631195066,
+      "step": 2080
+    },
+    {
+      "epoch": 2.1886298139900444,
+      "grad_norm": 0.08768360316753387,
+      "learning_rate": 1.803656118195136e-05,
+      "loss": 0.0437488317489624,
+      "step": 2090
+    },
+    {
+      "epoch": 2.1991092481006027,
+      "grad_norm": 0.08362651616334915,
+      "learning_rate": 1.760395423905379e-05,
+      "loss": 0.04669668078422547,
+      "step": 2100
+    },
+    {
+      "epoch": 2.2095886822111606,
+      "grad_norm": 0.08554034680128098,
+      "learning_rate": 1.7175487388522588e-05,
+      "loss": 0.034989356994628906,
+      "step": 2110
+    },
+    {
+      "epoch": 2.220068116321719,
+      "grad_norm": 0.08215561509132385,
+      "learning_rate": 1.6751215386803986e-05,
+      "loss": 0.040298929810523985,
+      "step": 2120
+    },
+    {
+      "epoch": 2.2305475504322767,
+      "grad_norm": 0.0840689167380333,
+      "learning_rate": 1.6331192454258337e-05,
+      "loss": 0.041704925894737246,
+      "step": 2130
+    },
+    {
+      "epoch": 2.2410269845428346,
+      "grad_norm": 0.06530614197254181,
+      "learning_rate": 1.5915472268231018e-05,
+      "loss": 0.03651900887489319,
+      "step": 2140
+    },
+    {
+      "epoch": 2.251506418653393,
+      "grad_norm": 0.12431822717189789,
+      "learning_rate": 1.550410795619261e-05,
+      "loss": 0.04806804955005646,
+      "step": 2150
+    },
+    {
+      "epoch": 2.2619858527639507,
+      "grad_norm": 0.09592410176992416,
+      "learning_rate": 1.509715208894949e-05,
+      "loss": 0.0454313725233078,
+      "step": 2160
+    },
+    {
+      "epoch": 2.2724652868745085,
+      "grad_norm": 0.07589780539274216,
+      "learning_rate": 1.469465667392536e-05,
+      "loss": 0.03574602603912354,
+      "step": 2170
+    },
+    {
+      "epoch": 2.282944720985067,
+      "grad_norm": 0.09734483063220978,
+      "learning_rate": 1.4296673148515038e-05,
+      "loss": 0.04358702301979065,
+      "step": 2180
+    },
+    {
+      "epoch": 2.2934241550956247,
+      "grad_norm": 0.0974339172244072,
+      "learning_rate": 1.3903252373510838e-05,
+      "loss": 0.04603351950645447,
+      "step": 2190
+    },
+    {
+      "epoch": 2.303903589206183,
+      "grad_norm": 0.09025271981954575,
+      "learning_rate": 1.3514444626602773e-05,
+      "loss": 0.040065237879753114,
+      "step": 2200
+    },
+    {
+      "epoch": 2.314383023316741,
+      "grad_norm": 0.07625086605548859,
+      "learning_rate": 1.3130299595953338e-05,
+      "loss": 0.044061675667762756,
+      "step": 2210
+    },
+    {
+      "epoch": 2.324862457427299,
+      "grad_norm": 0.07306221127510071,
+      "learning_rate": 1.2750866373847465e-05,
+      "loss": 0.03366467654705048,
+      "step": 2220
+    },
+    {
+      "epoch": 2.335341891537857,
+      "grad_norm": 0.08357638120651245,
+      "learning_rate": 1.2376193450418715e-05,
+      "loss": 0.041424044966697694,
+      "step": 2230
+    },
+    {
+      "epoch": 2.345821325648415,
+      "grad_norm": 0.09153921157121658,
+      "learning_rate": 1.2006328707452459e-05,
+      "loss": 0.03938372135162353,
+      "step": 2240
+    },
+    {
+      "epoch": 2.356300759758973,
+      "grad_norm": 0.09109660983085632,
+      "learning_rate": 1.1641319412266765e-05,
+      "loss": 0.04015985131263733,
+      "step": 2250
+    },
+    {
+      "epoch": 2.356300759758973,
+      "eval_loss": 0.05486458167433739,
+      "eval_runtime": 36.8119,
+      "eval_samples_per_second": 8.448,
+      "eval_steps_per_second": 8.448,
+      "step": 2250
+    },
+    {
+      "epoch": 2.366780193869531,
+      "grad_norm": 0.052502721548080444,
+      "learning_rate": 1.1281212211671822e-05,
+      "loss": 0.0270554780960083,
+      "step": 2260
+    },
+    {
+      "epoch": 2.377259627980089,
+      "grad_norm": 0.07931812107563019,
+      "learning_rate": 1.0926053126008584e-05,
+      "loss": 0.0417300134897232,
+      "step": 2270
+    },
+    {
+      "epoch": 2.387739062090647,
+      "grad_norm": 0.08996254205703735,
+      "learning_rate": 1.0575887543267609e-05,
+      "loss": 0.037659955024719236,
+      "step": 2280
+    },
+    {
+      "epoch": 2.398218496201205,
+      "grad_norm": 0.08800788223743439,
+      "learning_rate": 1.023076021328867e-05,
+      "loss": 0.048437944054603575,
+      "step": 2290
+    },
+    {
+      "epoch": 2.4086979303117633,
+      "grad_norm": 0.10572271049022675,
+      "learning_rate": 9.890715242041787e-06,
+      "loss": 0.04166909456253052,
+      "step": 2300
+    },
+    {
+      "epoch": 2.419177364422321,
+      "grad_norm": 0.10573071986436844,
+      "learning_rate": 9.555796085990781e-06,
+      "loss": 0.03919607996940613,
+      "step": 2310
+    },
+    {
+      "epoch": 2.4296567985328794,
+      "grad_norm": 0.09714583307504654,
+      "learning_rate": 9.226045546539608e-06,
+      "loss": 0.03530588150024414,
+      "step": 2320
+    },
+    {
+      "epoch": 2.4401362326434373,
+      "grad_norm": 0.09436199069023132,
+      "learning_rate": 8.901505764562518e-06,
+      "loss": 0.05111382007598877,
+      "step": 2330
+    },
+    {
+      "epoch": 2.450615666753995,
+      "grad_norm": 0.06353961676359177,
+      "learning_rate": 8.582218215018656e-06,
+      "loss": 0.03805697858333588,
+      "step": 2340
+    },
+    {
+      "epoch": 2.4610951008645534,
+      "grad_norm": 0.08853815495967865,
+      "learning_rate": 8.268223701651684e-06,
+      "loss": 0.04815975427627563,
+      "step": 2350
+    },
+    {
+      "epoch": 2.4715745349751113,
+      "grad_norm": 0.07472016662359238,
+      "learning_rate": 7.959562351775196e-06,
+      "loss": 0.042247459292411804,
+      "step": 2360
+    },
+    {
+      "epoch": 2.4820539690856696,
+      "grad_norm": 0.12121549248695374,
+      "learning_rate": 7.656273611144632e-06,
+      "loss": 0.040102115273475646,
+      "step": 2370
+    },
+    {
+      "epoch": 2.4925334031962274,
+      "grad_norm": 0.08667747676372528,
+      "learning_rate": 7.358396238916254e-06,
+      "loss": 0.03656341433525086,
+      "step": 2380
+    },
+    {
+      "epoch": 2.5030128373067857,
+      "grad_norm": 0.1162872165441513,
+      "learning_rate": 7.065968302693882e-06,
+      "loss": 0.04052766263484955,
+      "step": 2390
+    },
+    {
+      "epoch": 2.5134922714173435,
+      "grad_norm": 0.07924140989780426,
+      "learning_rate": 6.7790271736639595e-06,
+      "loss": 0.03394221067428589,
+      "step": 2400
+    },
+    {
+      "epoch": 2.5239717055279014,
+      "grad_norm": 0.09523408859968185,
+      "learning_rate": 6.497609521819681e-06,
+      "loss": 0.04119439423084259,
+      "step": 2410
+    },
+    {
+      "epoch": 2.5344511396384597,
+      "grad_norm": 0.12182598561048508,
+      "learning_rate": 6.221751311274731e-06,
+      "loss": 0.05154783725738525,
+      "step": 2420
+    },
+    {
+      "epoch": 2.5449305737490175,
+      "grad_norm": 0.09359873831272125,
+      "learning_rate": 5.951487795667149e-06,
+      "loss": 0.035483264923095705,
+      "step": 2430
+    },
+    {
+      "epoch": 2.5554100078595754,
+      "grad_norm": 0.08514095097780228,
+      "learning_rate": 5.686853513654117e-06,
+      "loss": 0.03830339312553406,
+      "step": 2440
+    },
+    {
+      "epoch": 2.5658894419701337,
+      "grad_norm": 0.10625084489583969,
+      "learning_rate": 5.4278822844979705e-06,
+      "loss": 0.034111028909683226,
+      "step": 2450
+    },
+    {
+      "epoch": 2.5763688760806915,
+      "grad_norm": 0.1004003956913948,
+      "learning_rate": 5.174607203744286e-06,
+      "loss": 0.04465605318546295,
+      "step": 2460
+    },
+    {
+      "epoch": 2.58684831019125,
+      "grad_norm": 0.0962519720196724,
+      "learning_rate": 4.927060638992382e-06,
+      "loss": 0.041056016087532045,
+      "step": 2470
+    },
+    {
+      "epoch": 2.5973277443018077,
+      "grad_norm": 0.06380607187747955,
+      "learning_rate": 4.685274225758846e-06,
+      "loss": 0.03880062401294708,
+      "step": 2480
+    },
+    {
+      "epoch": 2.607807178412366,
+      "grad_norm": 0.07326535880565643,
+      "learning_rate": 4.449278863434647e-06,
+      "loss": 0.03194461762905121,
+      "step": 2490
+    },
+    {
+      "epoch": 2.618286612522924,
+      "grad_norm": 0.12218596786260605,
+      "learning_rate": 4.2191047113362854e-06,
+      "loss": 0.04258840978145599,
+      "step": 2500
+    },
+    {
+      "epoch": 2.618286612522924,
+      "eval_loss": 0.05223666876554489,
+      "eval_runtime": 37.7234,
+      "eval_samples_per_second": 8.244,
+      "eval_steps_per_second": 8.244,
+      "step": 2500
+    },
+    {
+      "epoch": 2.6287660466334817,
+      "grad_norm": 0.08594664931297302,
+      "learning_rate": 3.994781184851598e-06,
+      "loss": 0.04302787780761719,
+      "step": 2510
+    },
+    {
+      "epoch": 2.63924548074404,
+      "grad_norm": 0.08187596499919891,
+      "learning_rate": 3.776336951680548e-06,
+      "loss": 0.0341387003660202,
+      "step": 2520
+    },
+    {
+      "epoch": 2.649724914854598,
+      "grad_norm": 0.10216796398162842,
+      "learning_rate": 3.563799928171596e-06,
+      "loss": 0.04289879500865936,
+      "step": 2530
+    },
+    {
+      "epoch": 2.6602043489651557,
+      "grad_norm": 0.11215174198150635,
+      "learning_rate": 3.3571972757540814e-06,
+      "loss": 0.04055049121379852,
+      "step": 2540
+    },
+    {
+      "epoch": 2.670683783075714,
+      "grad_norm": 0.07941269129514694,
+      "learning_rate": 3.156555397467176e-06,
+      "loss": 0.04118689000606537,
+      "step": 2550
+    },
+    {
+      "epoch": 2.681163217186272,
+      "grad_norm": 0.09404437988996506,
+      "learning_rate": 2.9618999345855547e-06,
+      "loss": 0.03079705536365509,
+      "step": 2560
+    },
+    {
+      "epoch": 2.69164265129683,
+      "grad_norm": 0.1109817698597908,
+      "learning_rate": 2.773255763342647e-06,
+      "loss": 0.038885954022407535,
+      "step": 2570
+    },
+    {
+      "epoch": 2.702122085407388,
+      "grad_norm": 0.09431962668895721,
+      "learning_rate": 2.590646991751472e-06,
+      "loss": 0.043543145060539246,
+      "step": 2580
+    },
+    {
+      "epoch": 2.7126015195179463,
+      "grad_norm": 0.08184763044118881,
+      "learning_rate": 2.414096956523776e-06,
+      "loss": 0.03256987631320953,
+      "step": 2590
+    },
+    {
+      "epoch": 2.723080953628504,
+      "grad_norm": 0.08390141278505325,
+      "learning_rate": 2.2436282200876458e-06,
+      "loss": 0.03908055424690247,
+      "step": 2600
+    },
+    {
+      "epoch": 2.733560387739062,
+      "grad_norm": 0.0762532502412796,
+      "learning_rate": 2.07926256770416e-06,
+      "loss": 0.04899201393127441,
+      "step": 2610
+    },
+    {
+      "epoch": 2.7440398218496203,
+      "grad_norm": 0.08239631354808807,
+      "learning_rate": 1.9210210046832768e-06,
+      "loss": 0.048707082867622375,
+      "step": 2620
+    },
+    {
+      "epoch": 2.754519255960178,
+      "grad_norm": 0.09619107842445374,
+      "learning_rate": 1.7689237536994364e-06,
+      "loss": 0.0372231125831604,
+      "step": 2630
+    },
+    {
+      "epoch": 2.764998690070736,
+      "grad_norm": 0.07099667191505432,
+      "learning_rate": 1.6229902522072293e-06,
+      "loss": 0.03421170711517334,
+      "step": 2640
+    },
+    {
+      "epoch": 2.7754781241812942,
+      "grad_norm": 0.10154753923416138,
+      "learning_rate": 1.4832391499572996e-06,
+      "loss": 0.03656705319881439,
+      "step": 2650
+    },
+    {
+      "epoch": 2.785957558291852,
+      "grad_norm": 0.09349387139081955,
+      "learning_rate": 1.3496883066130173e-06,
+      "loss": 0.03710306882858276,
+      "step": 2660
+    },
+    {
+      "epoch": 2.7964369924024104,
+      "grad_norm": 0.061091430485248566,
+      "learning_rate": 1.2223547894680443e-06,
+      "loss": 0.0308389812707901,
+      "step": 2670
+    },
+    {
+      "epoch": 2.8069164265129682,
+      "grad_norm": 0.09838075935840607,
+      "learning_rate": 1.101254871265256e-06,
+      "loss": 0.03703555166721344,
+      "step": 2680
+    },
+    {
+      "epoch": 2.8173958606235265,
+      "grad_norm": 0.10046928375959396,
+      "learning_rate": 9.864040281170938e-07,
+      "loss": 0.04500553905963898,
+      "step": 2690
+    },
+    {
+      "epoch": 2.8278752947340844,
+      "grad_norm": 0.06770773977041245,
+      "learning_rate": 8.778169375277978e-07,
+      "loss": 0.03823737502098083,
+      "step": 2700
+    },
+    {
+      "epoch": 2.8383547288446422,
+      "grad_norm": 0.08373535424470901,
+      "learning_rate": 7.755074765176618e-07,
+      "loss": 0.03961678743362427,
+      "step": 2710
+    },
+    {
+      "epoch": 2.8488341629552005,
+      "grad_norm": 0.07590050995349884,
+      "learning_rate": 6.794887198496413e-07,
+      "loss": 0.03221273124217987,
+      "step": 2720
+    },
+    {
+      "epoch": 2.8593135970657584,
+      "grad_norm": 0.08507678657770157,
+      "learning_rate": 5.897729383583906e-07,
+      "loss": 0.04571912884712219,
+      "step": 2730
+    },
+    {
+      "epoch": 2.8697930311763162,
+      "grad_norm": 0.06584763526916504,
+      "learning_rate": 5.063715973821659e-07,
+      "loss": 0.03794914484024048,
+      "step": 2740
+    },
+    {
+      "epoch": 2.8802724652868745,
+      "grad_norm": 0.07312892377376556,
+      "learning_rate": 4.292953552975154e-07,
+      "loss": 0.036365586519241336,
+      "step": 2750
+    },
+    {
+      "epoch": 2.8802724652868745,
+      "eval_loss": 0.05090421438217163,
+      "eval_runtime": 85.293,
+      "eval_samples_per_second": 3.646,
+      "eval_steps_per_second": 3.646,
+      "step": 2750
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 2865,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 250,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 8.668152022199163e+17,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-2750/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e8fc737554ff6f82c4ea137b5313611e3b2b3b63fd69b3926d6b1fe9da14c0a6
+size 5201

checkpoint-2865/README.md

@@ -0,0 +1,207 @@
+---
+base_model: Qwen/Qwen3.5-2B
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3.5-2B
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.18.0

checkpoint-2865/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-2B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.0",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "k_proj",
+    "gate_proj",
+    "o_proj",
+    "down_proj",
+    "up_proj",
+    "q_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-2865/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e54460326b97c66aced3c8ec3a50427b59111b42282d8638b4bbbe132d510518
+size 87319256

checkpoint-2865/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is true %}
+        {{- '<think>\n' }}
+    {%- else %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- endif %}
+{%- endif %}

checkpoint-2865/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a4e94fd092ed2523d6e2e9f17a72149ce8dc0997b192119b210e5713146e635f
+size 174750283

checkpoint-2865/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cad28a71806e0eabf48ed08b2dec44bd87e88427c15cd75dc56fa5f7a84126dd
+size 14645

checkpoint-2865/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8d5623d9d5ab3d6dfaf03bedc8ed63928ffd6ae34c2b75efbaf2c90b81268293
+size 1465

checkpoint-2865/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:87a7830d63fcf43bf241c3c5242e96e62dd3fdc29224ca26fed8ea333db72de4
+size 19989343

checkpoint-2865/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": true,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

checkpoint-2865/trainer_state.json

@@ -0,0 +1,2124 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 3.0,
+  "eval_steps": 250,
+  "global_step": 2865,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.010479434110558029,
+      "grad_norm": 0.19915591180324554,
+      "learning_rate": 1.0465116279069768e-05,
+      "loss": 1.1350045204162598,
+      "step": 10
+    },
+    {
+      "epoch": 0.020958868221116058,
+      "grad_norm": 0.18158815801143646,
+      "learning_rate": 2.2093023255813955e-05,
+      "loss": 1.0580164909362793,
+      "step": 20
+    },
+    {
+      "epoch": 0.03143830233167409,
+      "grad_norm": 0.16481591761112213,
+      "learning_rate": 3.372093023255814e-05,
+      "loss": 0.9252842903137207,
+      "step": 30
+    },
+    {
+      "epoch": 0.041917736442232116,
+      "grad_norm": 0.15599584579467773,
+      "learning_rate": 4.5348837209302326e-05,
+      "loss": 0.8342072486877441,
+      "step": 40
+    },
+    {
+      "epoch": 0.05239717055279015,
+      "grad_norm": 0.1804327368736267,
+      "learning_rate": 5.697674418604652e-05,
+      "loss": 0.7955524921417236,
+      "step": 50
+    },
+    {
+      "epoch": 0.06287660466334818,
+      "grad_norm": 0.16934047639369965,
+      "learning_rate": 6.86046511627907e-05,
+      "loss": 0.7358035087585449,
+      "step": 60
+    },
+    {
+      "epoch": 0.07335603877390622,
+      "grad_norm": 0.2234930843114853,
+      "learning_rate": 8.023255813953489e-05,
+      "loss": 0.6985861301422119,
+      "step": 70
+    },
+    {
+      "epoch": 0.08383547288446423,
+      "grad_norm": 0.16290400922298431,
+      "learning_rate": 9.186046511627907e-05,
+      "loss": 0.599607515335083,
+      "step": 80
+    },
+    {
+      "epoch": 0.09431490699502226,
+      "grad_norm": 0.1660464107990265,
+      "learning_rate": 9.999971245570617e-05,
+      "loss": 0.5886398315429687,
+      "step": 90
+    },
+    {
+      "epoch": 0.1047943411055803,
+      "grad_norm": 0.16978025436401367,
+      "learning_rate": 9.999460064915317e-05,
+      "loss": 0.5450529098510742,
+      "step": 100
+    },
+    {
+      "epoch": 0.11527377521613832,
+      "grad_norm": 0.21447990834712982,
+      "learning_rate": 9.998309972134645e-05,
+      "loss": 0.5072262287139893,
+      "step": 110
+    },
+    {
+      "epoch": 0.12575320932669637,
+      "grad_norm": 0.17418669164180756,
+      "learning_rate": 9.996521114206116e-05,
+      "loss": 0.49445347785949706,
+      "step": 120
+    },
+    {
+      "epoch": 0.13623264343725439,
+      "grad_norm": 0.22226351499557495,
+      "learning_rate": 9.994093719739023e-05,
+      "loss": 0.47142682075500486,
+      "step": 130
+    },
+    {
+      "epoch": 0.14671207754781243,
+      "grad_norm": 0.1745530068874359,
+      "learning_rate": 9.991028098945215e-05,
+      "loss": 0.46663532257080076,
+      "step": 140
+    },
+    {
+      "epoch": 0.15719151165837045,
+      "grad_norm": 0.17074695229530334,
+      "learning_rate": 9.987324643599459e-05,
+      "loss": 0.4508847236633301,
+      "step": 150
+    },
+    {
+      "epoch": 0.16767094576892846,
+      "grad_norm": 0.13428406417369843,
+      "learning_rate": 9.982983826989367e-05,
+      "loss": 0.40740265846252444,
+      "step": 160
+    },
+    {
+      "epoch": 0.1781503798794865,
+      "grad_norm": 0.17766578495502472,
+      "learning_rate": 9.978006203854918e-05,
+      "loss": 0.3998516321182251,
+      "step": 170
+    },
+    {
+      "epoch": 0.18862981399004453,
+      "grad_norm": 0.1672629565000534,
+      "learning_rate": 9.972392410317562e-05,
+      "loss": 0.41658673286437986,
+      "step": 180
+    },
+    {
+      "epoch": 0.19910924810060257,
+      "grad_norm": 0.1333673745393753,
+      "learning_rate": 9.96614316379892e-05,
+      "loss": 0.37024455070495604,
+      "step": 190
+    },
+    {
+      "epoch": 0.2095886822111606,
+      "grad_norm": 0.18037110567092896,
+      "learning_rate": 9.959259262929113e-05,
+      "loss": 0.35086841583251954,
+      "step": 200
+    },
+    {
+      "epoch": 0.22006811632171863,
+      "grad_norm": 0.14616410434246063,
+      "learning_rate": 9.951741587444683e-05,
+      "loss": 0.37918968200683595,
+      "step": 210
+    },
+    {
+      "epoch": 0.23054755043227665,
+      "grad_norm": 0.14523574709892273,
+      "learning_rate": 9.943591098076184e-05,
+      "loss": 0.32804527282714846,
+      "step": 220
+    },
+    {
+      "epoch": 0.2410269845428347,
+      "grad_norm": 0.14667049050331116,
+      "learning_rate": 9.934808836425393e-05,
+      "loss": 0.3480507850646973,
+      "step": 230
+    },
+    {
+      "epoch": 0.25150641865339274,
+      "grad_norm": 0.18156558275222778,
+      "learning_rate": 9.925395924832198e-05,
+      "loss": 0.3300448179244995,
+      "step": 240
+    },
+    {
+      "epoch": 0.26198585276395076,
+      "grad_norm": 0.13806430995464325,
+      "learning_rate": 9.91535356623117e-05,
+      "loss": 0.3127591609954834,
+      "step": 250
+    },
+    {
+      "epoch": 0.26198585276395076,
+      "eval_loss": 0.3132782578468323,
+      "eval_runtime": 94.8848,
+      "eval_samples_per_second": 3.278,
+      "eval_steps_per_second": 3.278,
+      "step": 250
+    },
+    {
+      "epoch": 0.27246528687450877,
+      "grad_norm": 0.17205959558486938,
+      "learning_rate": 9.904683043997835e-05,
+      "loss": 0.3306673288345337,
+      "step": 260
+    },
+    {
+      "epoch": 0.2829447209850668,
+      "grad_norm": 0.12620031833648682,
+      "learning_rate": 9.893385721784656e-05,
+      "loss": 0.3011106729507446,
+      "step": 270
+    },
+    {
+      "epoch": 0.29342415509562486,
+      "grad_norm": 0.11466006934642792,
+      "learning_rate": 9.881463043346768e-05,
+      "loss": 0.2951968669891357,
+      "step": 280
+    },
+    {
+      "epoch": 0.3039035892061829,
+      "grad_norm": 0.1671207845211029,
+      "learning_rate": 9.868916532357475e-05,
+      "loss": 0.2910990953445435,
+      "step": 290
+    },
+    {
+      "epoch": 0.3143830233167409,
+      "grad_norm": 0.1683349907398224,
+      "learning_rate": 9.855747792213521e-05,
+      "loss": 0.31409192085266113,
+      "step": 300
+    },
+    {
+      "epoch": 0.3248624574272989,
+      "grad_norm": 0.12934699654579163,
+      "learning_rate": 9.84195850583019e-05,
+      "loss": 0.27755858898162844,
+      "step": 310
+    },
+    {
+      "epoch": 0.33534189153785693,
+      "grad_norm": 0.13784605264663696,
+      "learning_rate": 9.827550435426234e-05,
+      "loss": 0.2809821605682373,
+      "step": 320
+    },
+    {
+      "epoch": 0.345821325648415,
+      "grad_norm": 0.18590271472930908,
+      "learning_rate": 9.812525422298664e-05,
+      "loss": 0.28698866367340087,
+      "step": 330
+    },
+    {
+      "epoch": 0.356300759758973,
+      "grad_norm": 0.1704522967338562,
+      "learning_rate": 9.796885386587447e-05,
+      "loss": 0.250814414024353,
+      "step": 340
+    },
+    {
+      "epoch": 0.36678019386953103,
+      "grad_norm": 0.1316167265176773,
+      "learning_rate": 9.780632327030112e-05,
+      "loss": 0.25458922386169436,
+      "step": 350
+    },
+    {
+      "epoch": 0.37725962798008905,
+      "grad_norm": 0.16226200759410858,
+      "learning_rate": 9.763768320706319e-05,
+      "loss": 0.26563262939453125,
+      "step": 360
+    },
+    {
+      "epoch": 0.3877390620906471,
+      "grad_norm": 0.1297195851802826,
+      "learning_rate": 9.746295522772424e-05,
+      "loss": 0.2632328748703003,
+      "step": 370
+    },
+    {
+      "epoch": 0.39821849620120514,
+      "grad_norm": 0.1286139190196991,
+      "learning_rate": 9.728216166186049e-05,
+      "loss": 0.2624588251113892,
+      "step": 380
+    },
+    {
+      "epoch": 0.40869793031176316,
+      "grad_norm": 0.1587965339422226,
+      "learning_rate": 9.709532561420725e-05,
+      "loss": 0.24741590023040771,
+      "step": 390
+    },
+    {
+      "epoch": 0.4191773644223212,
+      "grad_norm": 0.11963177472352982,
+      "learning_rate": 9.690247096170615e-05,
+      "loss": 0.22777397632598878,
+      "step": 400
+    },
+    {
+      "epoch": 0.42965679853287925,
+      "grad_norm": 0.13638927042484283,
+      "learning_rate": 9.670362235045387e-05,
+      "loss": 0.23324952125549317,
+      "step": 410
+    },
+    {
+      "epoch": 0.44013623264343726,
+      "grad_norm": 0.1514088362455368,
+      "learning_rate": 9.649880519255232e-05,
+      "loss": 0.2505915880203247,
+      "step": 420
+    },
+    {
+      "epoch": 0.4506156667539953,
+      "grad_norm": 0.10994207113981247,
+      "learning_rate": 9.62880456628612e-05,
+      "loss": 0.2078850269317627,
+      "step": 430
+    },
+    {
+      "epoch": 0.4610951008645533,
+      "grad_norm": 0.11983369290828705,
+      "learning_rate": 9.607137069565288e-05,
+      "loss": 0.21452484130859376,
+      "step": 440
+    },
+    {
+      "epoch": 0.47157453497511137,
+      "grad_norm": 0.12684305012226105,
+      "learning_rate": 9.58488079811703e-05,
+      "loss": 0.22002685070037842,
+      "step": 450
+    },
+    {
+      "epoch": 0.4820539690856694,
+      "grad_norm": 0.16841623187065125,
+      "learning_rate": 9.562038596208828e-05,
+      "loss": 0.21405396461486817,
+      "step": 460
+    },
+    {
+      "epoch": 0.4925334031962274,
+      "grad_norm": 0.1498555839061737,
+      "learning_rate": 9.538613382987865e-05,
+      "loss": 0.20534911155700683,
+      "step": 470
+    },
+    {
+      "epoch": 0.5030128373067855,
+      "grad_norm": 0.13913628458976746,
+      "learning_rate": 9.514608152107974e-05,
+      "loss": 0.22248730659484864,
+      "step": 480
+    },
+    {
+      "epoch": 0.5134922714173434,
+      "grad_norm": 0.14408951997756958,
+      "learning_rate": 9.490025971347047e-05,
+      "loss": 0.214866042137146,
+      "step": 490
+    },
+    {
+      "epoch": 0.5239717055279015,
+      "grad_norm": 0.1649770438671112,
+      "learning_rate": 9.464869982215001e-05,
+      "loss": 0.19965900182724,
+      "step": 500
+    },
+    {
+      "epoch": 0.5239717055279015,
+      "eval_loss": 0.19267401099205017,
+      "eval_runtime": 95.3374,
+      "eval_samples_per_second": 3.262,
+      "eval_steps_per_second": 3.262,
+      "step": 500
+    },
+    {
+      "epoch": 0.5344511396384595,
+      "grad_norm": 0.1305568665266037,
+      "learning_rate": 9.439143399552291e-05,
+      "loss": 0.21112546920776368,
+      "step": 510
+    },
+    {
+      "epoch": 0.5449305737490175,
+      "grad_norm": 0.11998175084590912,
+      "learning_rate": 9.412849511119074e-05,
+      "loss": 0.21422922611236572,
+      "step": 520
+    },
+    {
+      "epoch": 0.5554100078595756,
+      "grad_norm": 0.15220341086387634,
+      "learning_rate": 9.385991677175046e-05,
+      "loss": 0.20999882221221924,
+      "step": 530
+    },
+    {
+      "epoch": 0.5658894419701336,
+      "grad_norm": 0.13170023262500763,
+      "learning_rate": 9.358573330050004e-05,
+      "loss": 0.20208392143249512,
+      "step": 540
+    },
+    {
+      "epoch": 0.5763688760806917,
+      "grad_norm": 0.10457764565944672,
+      "learning_rate": 9.330597973705219e-05,
+      "loss": 0.1908803701400757,
+      "step": 550
+    },
+    {
+      "epoch": 0.5868483101912497,
+      "grad_norm": 0.12568537890911102,
+      "learning_rate": 9.302069183285637e-05,
+      "loss": 0.19316340684890748,
+      "step": 560
+    },
+    {
+      "epoch": 0.5973277443018077,
+      "grad_norm": 0.14824528992176056,
+      "learning_rate": 9.272990604662988e-05,
+      "loss": 0.18987581729888917,
+      "step": 570
+    },
+    {
+      "epoch": 0.6078071784123658,
+      "grad_norm": 0.14521734416484833,
+      "learning_rate": 9.243365953969861e-05,
+      "loss": 0.19232832193374633,
+      "step": 580
+    },
+    {
+      "epoch": 0.6182866125229237,
+      "grad_norm": 0.1335408091545105,
+      "learning_rate": 9.213199017124793e-05,
+      "loss": 0.1758212924003601,
+      "step": 590
+    },
+    {
+      "epoch": 0.6287660466334818,
+      "grad_norm": 0.11143071949481964,
+      "learning_rate": 9.182493649348447e-05,
+      "loss": 0.19117680788040162,
+      "step": 600
+    },
+    {
+      "epoch": 0.6392454807440399,
+      "grad_norm": 0.14789296686649323,
+      "learning_rate": 9.151253774670921e-05,
+      "loss": 0.184559965133667,
+      "step": 610
+    },
+    {
+      "epoch": 0.6497249148545978,
+      "grad_norm": 0.10541336238384247,
+      "learning_rate": 9.119483385430283e-05,
+      "loss": 0.1720304846763611,
+      "step": 620
+    },
+    {
+      "epoch": 0.6602043489651559,
+      "grad_norm": 0.12105975300073624,
+      "learning_rate": 9.087186541762358e-05,
+      "loss": 0.17654836177825928,
+      "step": 630
+    },
+    {
+      "epoch": 0.6706837830757139,
+      "grad_norm": 0.13114669919013977,
+      "learning_rate": 9.054367371081858e-05,
+      "loss": 0.1696592688560486,
+      "step": 640
+    },
+    {
+      "epoch": 0.6811632171862719,
+      "grad_norm": 0.13745592534542084,
+      "learning_rate": 9.021030067554919e-05,
+      "loss": 0.15404462814331055,
+      "step": 650
+    },
+    {
+      "epoch": 0.69164265129683,
+      "grad_norm": 0.15927442908287048,
+      "learning_rate": 8.987178891563094e-05,
+      "loss": 0.17024366855621337,
+      "step": 660
+    },
+    {
+      "epoch": 0.702122085407388,
+      "grad_norm": 0.13737429678440094,
+      "learning_rate": 8.952818169158903e-05,
+      "loss": 0.1602048397064209,
+      "step": 670
+    },
+    {
+      "epoch": 0.712601519517946,
+      "grad_norm": 0.13941751420497894,
+      "learning_rate": 8.91795229151297e-05,
+      "loss": 0.18057082891464232,
+      "step": 680
+    },
+    {
+      "epoch": 0.7230809536285041,
+      "grad_norm": 0.14242954552173615,
+      "learning_rate": 8.882585714352856e-05,
+      "loss": 0.14863334894180297,
+      "step": 690
+    },
+    {
+      "epoch": 0.7335603877390621,
+      "grad_norm": 0.15553542971611023,
+      "learning_rate": 8.846722957393626e-05,
+      "loss": 0.15701137781143187,
+      "step": 700
+    },
+    {
+      "epoch": 0.7440398218496201,
+      "grad_norm": 0.12901411950588226,
+      "learning_rate": 8.810368603760249e-05,
+      "loss": 0.15571318864822387,
+      "step": 710
+    },
+    {
+      "epoch": 0.7545192559601781,
+      "grad_norm": 0.13449430465698242,
+      "learning_rate": 8.773527299401902e-05,
+      "loss": 0.16418551206588744,
+      "step": 720
+    },
+    {
+      "epoch": 0.7649986900707362,
+      "grad_norm": 0.10630270838737488,
+      "learning_rate": 8.736203752498218e-05,
+      "loss": 0.16800801753997802,
+      "step": 730
+    },
+    {
+      "epoch": 0.7754781241812942,
+      "grad_norm": 0.11299935728311539,
+      "learning_rate": 8.698402732857611e-05,
+      "loss": 0.15700833797454833,
+      "step": 740
+    },
+    {
+      "epoch": 0.7859575582918522,
+      "grad_norm": 0.11920930445194244,
+      "learning_rate": 8.660129071307707e-05,
+      "loss": 0.15091001987457275,
+      "step": 750
+    },
+    {
+      "epoch": 0.7859575582918522,
+      "eval_loss": 0.1356429010629654,
+      "eval_runtime": 94.0557,
+      "eval_samples_per_second": 3.307,
+      "eval_steps_per_second": 3.307,
+      "step": 750
+    },
+    {
+      "epoch": 0.7964369924024103,
+      "grad_norm": 0.13870343565940857,
+      "learning_rate": 8.621387659077986e-05,
+      "loss": 0.1422027826309204,
+      "step": 760
+    },
+    {
+      "epoch": 0.8069164265129684,
+      "grad_norm": 0.12753477692604065,
+      "learning_rate": 8.582183447174697e-05,
+      "loss": 0.142450213432312,
+      "step": 770
+    },
+    {
+      "epoch": 0.8173958606235263,
+      "grad_norm": 0.11877496540546417,
+      "learning_rate": 8.542521445748141e-05,
+      "loss": 0.15361062288284302,
+      "step": 780
+    },
+    {
+      "epoch": 0.8278752947340844,
+      "grad_norm": 0.1200249195098877,
+      "learning_rate": 8.502406723452392e-05,
+      "loss": 0.14647477865219116,
+      "step": 790
+    },
+    {
+      "epoch": 0.8383547288446423,
+      "grad_norm": 0.12913794815540314,
+      "learning_rate": 8.461844406797543e-05,
+      "loss": 0.1591552734375,
+      "step": 800
+    },
+    {
+      "epoch": 0.8488341629552004,
+      "grad_norm": 0.17270176112651825,
+      "learning_rate": 8.420839679494558e-05,
+      "loss": 0.1495436668395996,
+      "step": 810
+    },
+    {
+      "epoch": 0.8593135970657585,
+      "grad_norm": 0.15545596182346344,
+      "learning_rate": 8.379397781792808e-05,
+      "loss": 0.15377395153045653,
+      "step": 820
+    },
+    {
+      "epoch": 0.8697930311763165,
+      "grad_norm": 0.12941111624240875,
+      "learning_rate": 8.337524009810395e-05,
+      "loss": 0.14733861684799193,
+      "step": 830
+    },
+    {
+      "epoch": 0.8802724652868745,
+      "grad_norm": 0.13152749836444855,
+      "learning_rate": 8.295223714857319e-05,
+      "loss": 0.13980752229690552,
+      "step": 840
+    },
+    {
+      "epoch": 0.8907518993974325,
+      "grad_norm": 0.11208872497081757,
+      "learning_rate": 8.252502302751612e-05,
+      "loss": 0.12019969224929809,
+      "step": 850
+    },
+    {
+      "epoch": 0.9012313335079906,
+      "grad_norm": 0.11118603497743607,
+      "learning_rate": 8.209365233128482e-05,
+      "loss": 0.13822466135025024,
+      "step": 860
+    },
+    {
+      "epoch": 0.9117107676185486,
+      "grad_norm": 0.11705653369426727,
+      "learning_rate": 8.165818018742605e-05,
+      "loss": 0.1439664840698242,
+      "step": 870
+    },
+    {
+      "epoch": 0.9221902017291066,
+      "grad_norm": 0.08817730098962784,
+      "learning_rate": 8.121866224763606e-05,
+      "loss": 0.13380355834960939,
+      "step": 880
+    },
+    {
+      "epoch": 0.9326696358396647,
+      "grad_norm": 0.1092257872223854,
+      "learning_rate": 8.077515468064851e-05,
+      "loss": 0.12982802391052245,
+      "step": 890
+    },
+    {
+      "epoch": 0.9431490699502227,
+      "grad_norm": 0.12680962681770325,
+      "learning_rate": 8.032771416505647e-05,
+      "loss": 0.1489071011543274,
+      "step": 900
+    },
+    {
+      "epoch": 0.9536285040607807,
+      "grad_norm": 0.11953219771385193,
+      "learning_rate": 7.987639788206888e-05,
+      "loss": 0.14020267724990845,
+      "step": 910
+    },
+    {
+      "epoch": 0.9641079381713388,
+      "grad_norm": 0.1041467934846878,
+      "learning_rate": 7.942126350820318e-05,
+      "loss": 0.1439213275909424,
+      "step": 920
+    },
+    {
+      "epoch": 0.9745873722818967,
+      "grad_norm": 0.1277916431427002,
+      "learning_rate": 7.896236920791442e-05,
+      "loss": 0.1468779683113098,
+      "step": 930
+    },
+    {
+      "epoch": 0.9850668063924548,
+      "grad_norm": 0.11245205253362656,
+      "learning_rate": 7.849977362616201e-05,
+      "loss": 0.12012372016906739,
+      "step": 940
+    },
+    {
+      "epoch": 0.9955462405030129,
+      "grad_norm": 0.12230483442544937,
+      "learning_rate": 7.803353588091522e-05,
+      "loss": 0.1488939881324768,
+      "step": 950
+    },
+    {
+      "epoch": 1.005239717055279,
+      "grad_norm": 0.14185865223407745,
+      "learning_rate": 7.7563715555598e-05,
+      "loss": 0.11488113403320313,
+      "step": 960
+    },
+    {
+      "epoch": 1.015719151165837,
+      "grad_norm": 0.10545773804187775,
+      "learning_rate": 7.709037269147459e-05,
+      "loss": 0.10712549686431885,
+      "step": 970
+    },
+    {
+      "epoch": 1.026198585276395,
+      "grad_norm": 0.10376274585723877,
+      "learning_rate": 7.661356777997631e-05,
+      "loss": 0.11428828239440918,
+      "step": 980
+    },
+    {
+      "epoch": 1.0366780193869531,
+      "grad_norm": 0.09950564056634903,
+      "learning_rate": 7.613336175497111e-05,
+      "loss": 0.09823058247566223,
+      "step": 990
+    },
+    {
+      "epoch": 1.0471574534975112,
+      "grad_norm": 0.10412753373384476,
+      "learning_rate": 7.564981598497643e-05,
+      "loss": 0.1106558084487915,
+      "step": 1000
+    },
+    {
+      "epoch": 1.0471574534975112,
+      "eval_loss": 0.11185819655656815,
+      "eval_runtime": 93.808,
+      "eval_samples_per_second": 3.315,
+      "eval_steps_per_second": 3.315,
+      "step": 1000
+    },
+    {
+      "epoch": 1.057636887608069,
+      "grad_norm": 0.10430868715047836,
+      "learning_rate": 7.516299226531645e-05,
+      "loss": 0.11168640851974487,
+      "step": 1010
+    },
+    {
+      "epoch": 1.0681163217186271,
+      "grad_norm": 0.09646806865930557,
+      "learning_rate": 7.467295281022501e-05,
+      "loss": 0.10711305141448975,
+      "step": 1020
+    },
+    {
+      "epoch": 1.0785957558291852,
+      "grad_norm": 0.13060614466667175,
+      "learning_rate": 7.417976024489474e-05,
+      "loss": 0.10001810789108276,
+      "step": 1030
+    },
+    {
+      "epoch": 1.0890751899397433,
+      "grad_norm": 0.10389085114002228,
+      "learning_rate": 7.368347759747393e-05,
+      "loss": 0.11893858909606933,
+      "step": 1040
+    },
+    {
+      "epoch": 1.0995546240503014,
+      "grad_norm": 0.11291550099849701,
+      "learning_rate": 7.318416829101164e-05,
+      "loss": 0.1079628586769104,
+      "step": 1050
+    },
+    {
+      "epoch": 1.1100340581608594,
+      "grad_norm": 0.10372598469257355,
+      "learning_rate": 7.268189613535255e-05,
+      "loss": 0.10332397222518921,
+      "step": 1060
+    },
+    {
+      "epoch": 1.1205134922714173,
+      "grad_norm": 0.12971536815166473,
+      "learning_rate": 7.217672531898225e-05,
+      "loss": 0.10804877281188965,
+      "step": 1070
+    },
+    {
+      "epoch": 1.1309929263819753,
+      "grad_norm": 0.10902425646781921,
+      "learning_rate": 7.166872040082431e-05,
+      "loss": 0.09947454929351807,
+      "step": 1080
+    },
+    {
+      "epoch": 1.1414723604925334,
+      "grad_norm": 0.09305932372808456,
+      "learning_rate": 7.11579463019897e-05,
+      "loss": 0.09406971335411071,
+      "step": 1090
+    },
+    {
+      "epoch": 1.1519517946030915,
+      "grad_norm": 0.11485275626182556,
+      "learning_rate": 7.064446829748034e-05,
+      "loss": 0.09943979978561401,
+      "step": 1100
+    },
+    {
+      "epoch": 1.1624312287136496,
+      "grad_norm": 0.09556467831134796,
+      "learning_rate": 7.0128352007847e-05,
+      "loss": 0.10862170457839966,
+      "step": 1110
+    },
+    {
+      "epoch": 1.1729106628242074,
+      "grad_norm": 0.11937833577394485,
+      "learning_rate": 6.96096633908034e-05,
+      "loss": 0.10385221242904663,
+      "step": 1120
+    },
+    {
+      "epoch": 1.1833900969347655,
+      "grad_norm": 0.11560507863759995,
+      "learning_rate": 6.908846873279691e-05,
+      "loss": 0.09252402186393738,
+      "step": 1130
+    },
+    {
+      "epoch": 1.1938695310453236,
+      "grad_norm": 0.11119654029607773,
+      "learning_rate": 6.856483464053758e-05,
+      "loss": 0.09637172818183899,
+      "step": 1140
+    },
+    {
+      "epoch": 1.2043489651558816,
+      "grad_norm": 0.11722644418478012,
+      "learning_rate": 6.803882803248585e-05,
+      "loss": 0.09078751802444458,
+      "step": 1150
+    },
+    {
+      "epoch": 1.2148283992664397,
+      "grad_norm": 0.10487739741802216,
+      "learning_rate": 6.751051613030082e-05,
+      "loss": 0.10334972143173218,
+      "step": 1160
+    },
+    {
+      "epoch": 1.2253078333769976,
+      "grad_norm": 0.10202383995056152,
+      "learning_rate": 6.697996645024937e-05,
+      "loss": 0.08661433458328247,
+      "step": 1170
+    },
+    {
+      "epoch": 1.2357872674875556,
+      "grad_norm": 0.11801143735647202,
+      "learning_rate": 6.644724679457804e-05,
+      "loss": 0.0997927188873291,
+      "step": 1180
+    },
+    {
+      "epoch": 1.2462667015981137,
+      "grad_norm": 0.10949107259511948,
+      "learning_rate": 6.591242524284802e-05,
+      "loss": 0.0977592945098877,
+      "step": 1190
+    },
+    {
+      "epoch": 1.2567461357086718,
+      "grad_norm": 0.10221222043037415,
+      "learning_rate": 6.537557014323487e-05,
+      "loss": 0.0970361053943634,
+      "step": 1200
+    },
+    {
+      "epoch": 1.2672255698192298,
+      "grad_norm": 0.10554748773574829,
+      "learning_rate": 6.483675010379393e-05,
+      "loss": 0.09007551074028015,
+      "step": 1210
+    },
+    {
+      "epoch": 1.2777050039297877,
+      "grad_norm": 0.11625627428293228,
+      "learning_rate": 6.429603398369242e-05,
+      "loss": 0.08734490275382996,
+      "step": 1220
+    },
+    {
+      "epoch": 1.2881844380403458,
+      "grad_norm": 0.10624277591705322,
+      "learning_rate": 6.37534908844095e-05,
+      "loss": 0.09858485460281372,
+      "step": 1230
+    },
+    {
+      "epoch": 1.2986638721509038,
+      "grad_norm": 0.10184557735919952,
+      "learning_rate": 6.320919014090534e-05,
+      "loss": 0.09335023164749146,
+      "step": 1240
+    },
+    {
+      "epoch": 1.309143306261462,
+      "grad_norm": 0.10787283629179001,
+      "learning_rate": 6.266320131276051e-05,
+      "loss": 0.08665563464164734,
+      "step": 1250
+    },
+    {
+      "epoch": 1.309143306261462,
+      "eval_loss": 0.08951585739850998,
+      "eval_runtime": 94.0567,
+      "eval_samples_per_second": 3.307,
+      "eval_steps_per_second": 3.307,
+      "step": 1250
+    },
+    {
+      "epoch": 1.31962274037202,
+      "grad_norm": 0.10836981981992722,
+      "learning_rate": 6.211559417528631e-05,
+      "loss": 0.0933380126953125,
+      "step": 1260
+    },
+    {
+      "epoch": 1.3301021744825778,
+      "grad_norm": 0.1397171914577484,
+      "learning_rate": 6.156643871060795e-05,
+      "loss": 0.09835371971130372,
+      "step": 1270
+    },
+    {
+      "epoch": 1.340581608593136,
+      "grad_norm": 0.11242218315601349,
+      "learning_rate": 6.101580509872097e-05,
+      "loss": 0.09398673176765442,
+      "step": 1280
+    },
+    {
+      "epoch": 1.351061042703694,
+      "grad_norm": 0.10235017538070679,
+      "learning_rate": 6.0463763708522536e-05,
+      "loss": 0.10350929498672486,
+      "step": 1290
+    },
+    {
+      "epoch": 1.361540476814252,
+      "grad_norm": 0.09327106177806854,
+      "learning_rate": 5.99103850888186e-05,
+      "loss": 0.09580238461494446,
+      "step": 1300
+    },
+    {
+      "epoch": 1.3720199109248101,
+      "grad_norm": 0.12995658814907074,
+      "learning_rate": 5.9355739959307976e-05,
+      "loss": 0.08437412977218628,
+      "step": 1310
+    },
+    {
+      "epoch": 1.382499345035368,
+      "grad_norm": 0.11962983757257462,
+      "learning_rate": 5.879989920154466e-05,
+      "loss": 0.08409937620162963,
+      "step": 1320
+    },
+    {
+      "epoch": 1.392978779145926,
+      "grad_norm": 0.09431737661361694,
+      "learning_rate": 5.824293384987941e-05,
+      "loss": 0.09504773020744324,
+      "step": 1330
+    },
+    {
+      "epoch": 1.4034582132564841,
+      "grad_norm": 0.13824374973773956,
+      "learning_rate": 5.768491508238188e-05,
+      "loss": 0.09193333983421326,
+      "step": 1340
+    },
+    {
+      "epoch": 1.4139376473670422,
+      "grad_norm": 0.10595858097076416,
+      "learning_rate": 5.712591421174422e-05,
+      "loss": 0.08976472616195678,
+      "step": 1350
+    },
+    {
+      "epoch": 1.4244170814776003,
+      "grad_norm": 0.09911809861660004,
+      "learning_rate": 5.6566002676167725e-05,
+      "loss": 0.07597061395645141,
+      "step": 1360
+    },
+    {
+      "epoch": 1.4348965155881581,
+      "grad_norm": 0.09723466634750366,
+      "learning_rate": 5.60052520302332e-05,
+      "loss": 0.10513757467269898,
+      "step": 1370
+    },
+    {
+      "epoch": 1.4453759496987162,
+      "grad_norm": 0.11331687867641449,
+      "learning_rate": 5.5443733935756615e-05,
+      "loss": 0.09019948840141297,
+      "step": 1380
+    },
+    {
+      "epoch": 1.4558553838092743,
+      "grad_norm": 0.13363589346408844,
+      "learning_rate": 5.4881520152630886e-05,
+      "loss": 0.08314153552055359,
+      "step": 1390
+    },
+    {
+      "epoch": 1.4663348179198323,
+      "grad_norm": 0.14111892879009247,
+      "learning_rate": 5.4318682529655404e-05,
+      "loss": 0.07892010807991028,
+      "step": 1400
+    },
+    {
+      "epoch": 1.4768142520303904,
+      "grad_norm": 0.13948485255241394,
+      "learning_rate": 5.3755292995353913e-05,
+      "loss": 0.0840128481388092,
+      "step": 1410
+    },
+    {
+      "epoch": 1.4872936861409483,
+      "grad_norm": 0.12535949051380157,
+      "learning_rate": 5.31914235487823e-05,
+      "loss": 0.07869629859924317,
+      "step": 1420
+    },
+    {
+      "epoch": 1.4977731202515066,
+      "grad_norm": 0.10041694343090057,
+      "learning_rate": 5.2627146250327484e-05,
+      "loss": 0.08074848055839538,
+      "step": 1430
+    },
+    {
+      "epoch": 1.5082525543620644,
+      "grad_norm": 0.10112891346216202,
+      "learning_rate": 5.2062533212498275e-05,
+      "loss": 0.0860810935497284,
+      "step": 1440
+    },
+    {
+      "epoch": 1.5187319884726225,
+      "grad_norm": 0.11297477036714554,
+      "learning_rate": 5.149765659070973e-05,
+      "loss": 0.08794642686843872,
+      "step": 1450
+    },
+    {
+      "epoch": 1.5292114225831805,
+      "grad_norm": 0.10511091351509094,
+      "learning_rate": 5.0932588574061945e-05,
+      "loss": 0.07854819297790527,
+      "step": 1460
+    },
+    {
+      "epoch": 1.5396908566937384,
+      "grad_norm": 0.09333530068397522,
+      "learning_rate": 5.036740137611453e-05,
+      "loss": 0.08821435570716858,
+      "step": 1470
+    },
+    {
+      "epoch": 1.5501702908042967,
+      "grad_norm": 0.11480343341827393,
+      "learning_rate": 4.980216722565804e-05,
+      "loss": 0.08062278628349304,
+      "step": 1480
+    },
+    {
+      "epoch": 1.5606497249148545,
+      "grad_norm": 0.08406255394220352,
+      "learning_rate": 4.923695835748338e-05,
+      "loss": 0.0940588355064392,
+      "step": 1490
+    },
+    {
+      "epoch": 1.5711291590254126,
+      "grad_norm": 0.12927693128585815,
+      "learning_rate": 4.8671847003150447e-05,
+      "loss": 0.0775177538394928,
+      "step": 1500
+    },
+    {
+      "epoch": 1.5711291590254126,
+      "eval_loss": 0.07877222448587418,
+      "eval_runtime": 34.4389,
+      "eval_samples_per_second": 9.03,
+      "eval_steps_per_second": 9.03,
+      "step": 1500
+    },
+    {
+      "epoch": 1.5816085931359707,
+      "grad_norm": 0.1255076378583908,
+      "learning_rate": 4.810690538175728e-05,
+      "loss": 0.09362970590591431,
+      "step": 1510
+    },
+    {
+      "epoch": 1.5920880272465285,
+      "grad_norm": 0.1326853185892105,
+      "learning_rate": 4.754220569071068e-05,
+      "loss": 0.08364834189414978,
+      "step": 1520
+    },
+    {
+      "epoch": 1.6025674613570868,
+      "grad_norm": 0.10229979455471039,
+      "learning_rate": 4.697782009649962e-05,
+      "loss": 0.0725843846797943,
+      "step": 1530
+    },
+    {
+      "epoch": 1.6130468954676447,
+      "grad_norm": 0.11407258361577988,
+      "learning_rate": 4.641382072547272e-05,
+      "loss": 0.07566151022911072,
+      "step": 1540
+    },
+    {
+      "epoch": 1.6235263295782028,
+      "grad_norm": 0.09398165345191956,
+      "learning_rate": 4.585027965462075e-05,
+      "loss": 0.087736576795578,
+      "step": 1550
+    },
+    {
+      "epoch": 1.6340057636887608,
+      "grad_norm": 0.11289424449205399,
+      "learning_rate": 4.528726890236544e-05,
+      "loss": 0.08366051316261292,
+      "step": 1560
+    },
+    {
+      "epoch": 1.6444851977993187,
+      "grad_norm": 0.09478718787431717,
+      "learning_rate": 4.4724860419355746e-05,
+      "loss": 0.0885531723499298,
+      "step": 1570
+    },
+    {
+      "epoch": 1.654964631909877,
+      "grad_norm": 0.09163404256105423,
+      "learning_rate": 4.416312607927295e-05,
+      "loss": 0.08392030596733094,
+      "step": 1580
+    },
+    {
+      "epoch": 1.6654440660204348,
+      "grad_norm": 0.11422222852706909,
+      "learning_rate": 4.360213766964542e-05,
+      "loss": 0.08059985041618348,
+      "step": 1590
+    },
+    {
+      "epoch": 1.675923500130993,
+      "grad_norm": 0.08131479471921921,
+      "learning_rate": 4.304196688267438e-05,
+      "loss": 0.07613803148269653,
+      "step": 1600
+    },
+    {
+      "epoch": 1.686402934241551,
+      "grad_norm": 0.09615079313516617,
+      "learning_rate": 4.248268530607199e-05,
+      "loss": 0.07764078378677368,
+      "step": 1610
+    },
+    {
+      "epoch": 1.696882368352109,
+      "grad_norm": 0.09730526059865952,
+      "learning_rate": 4.192436441391271e-05,
+      "loss": 0.07644452452659607,
+      "step": 1620
+    },
+    {
+      "epoch": 1.707361802462667,
+      "grad_norm": 0.09649327397346497,
+      "learning_rate": 4.136707555749907e-05,
+      "loss": 0.07866159081459045,
+      "step": 1630
+    },
+    {
+      "epoch": 1.717841236573225,
+      "grad_norm": 0.11804413050413132,
+      "learning_rate": 4.0810889956243415e-05,
+      "loss": 0.06996130347251892,
+      "step": 1640
+    },
+    {
+      "epoch": 1.728320670683783,
+      "grad_norm": 0.09874672442674637,
+      "learning_rate": 4.025587868856622e-05,
+      "loss": 0.07877404093742371,
+      "step": 1650
+    },
+    {
+      "epoch": 1.738800104794341,
+      "grad_norm": 0.11149467527866364,
+      "learning_rate": 3.9702112682812544e-05,
+      "loss": 0.07241421341896057,
+      "step": 1660
+    },
+    {
+      "epoch": 1.7492795389048992,
+      "grad_norm": 0.08748896420001984,
+      "learning_rate": 3.914966270818766e-05,
+      "loss": 0.07336459755897522,
+      "step": 1670
+    },
+    {
+      "epoch": 1.7597589730154573,
+      "grad_norm": 0.1172696202993393,
+      "learning_rate": 3.859859936571307e-05,
+      "loss": 0.07742337584495544,
+      "step": 1680
+    },
+    {
+      "epoch": 1.770238407126015,
+      "grad_norm": 0.0719197615981102,
+      "learning_rate": 3.8048993079203925e-05,
+      "loss": 0.06242966651916504,
+      "step": 1690
+    },
+    {
+      "epoch": 1.7807178412365732,
+      "grad_norm": 0.12380168586969376,
+      "learning_rate": 3.750091408626907e-05,
+      "loss": 0.07270430326461792,
+      "step": 1700
+    },
+    {
+      "epoch": 1.7911972753471312,
+      "grad_norm": 0.1587221622467041,
+      "learning_rate": 3.6954432429335015e-05,
+      "loss": 0.06409866213798524,
+      "step": 1710
+    },
+    {
+      "epoch": 1.8016767094576893,
+      "grad_norm": 0.10983912646770477,
+      "learning_rate": 3.640961794669482e-05,
+      "loss": 0.06610031127929687,
+      "step": 1720
+    },
+    {
+      "epoch": 1.8121561435682474,
+      "grad_norm": 0.11023026704788208,
+      "learning_rate": 3.586654026358287e-05,
+      "loss": 0.06866579055786133,
+      "step": 1730
+    },
+    {
+      "epoch": 1.8226355776788052,
+      "grad_norm": 0.11857719719409943,
+      "learning_rate": 3.532526878327719e-05,
+      "loss": 0.06734356880187989,
+      "step": 1740
+    },
+    {
+      "epoch": 1.8331150117893635,
+      "grad_norm": 0.09280339628458023,
+      "learning_rate": 3.478587267822987e-05,
+      "loss": 0.06897796392440796,
+      "step": 1750
+    },
+    {
+      "epoch": 1.8331150117893635,
+      "eval_loss": 0.06596127897500992,
+      "eval_runtime": 35.5001,
+      "eval_samples_per_second": 8.761,
+      "eval_steps_per_second": 8.761,
+      "step": 1750
+    },
+    {
+      "epoch": 1.8435944458999214,
+      "grad_norm": 0.1175367683172226,
+      "learning_rate": 3.424842088122716e-05,
+      "loss": 0.08288194537162781,
+      "step": 1760
+    },
+    {
+      "epoch": 1.8540738800104795,
+      "grad_norm": 0.10271462798118591,
+      "learning_rate": 3.371298207658003e-05,
+      "loss": 0.05643013119697571,
+      "step": 1770
+    },
+    {
+      "epoch": 1.8645533141210375,
+      "grad_norm": 0.11965195834636688,
+      "learning_rate": 3.3179624691346654e-05,
+      "loss": 0.07403092980384826,
+      "step": 1780
+    },
+    {
+      "epoch": 1.8750327482315954,
+      "grad_norm": 0.09981680661439896,
+      "learning_rate": 3.2648416886587686e-05,
+      "loss": 0.07118859887123108,
+      "step": 1790
+    },
+    {
+      "epoch": 1.8855121823421537,
+      "grad_norm": 0.07787375897169113,
+      "learning_rate": 3.2119426548655435e-05,
+      "loss": 0.07219682335853576,
+      "step": 1800
+    },
+    {
+      "epoch": 1.8959916164527115,
+      "grad_norm": 0.1303507387638092,
+      "learning_rate": 3.1592721280518404e-05,
+      "loss": 0.07636030912399291,
+      "step": 1810
+    },
+    {
+      "epoch": 1.9064710505632696,
+      "grad_norm": 0.09162267297506332,
+      "learning_rate": 3.106836839312175e-05,
+      "loss": 0.06230143308639526,
+      "step": 1820
+    },
+    {
+      "epoch": 1.9169504846738277,
+      "grad_norm": 0.11375878751277924,
+      "learning_rate": 3.054643489678526e-05,
+      "loss": 0.060506826639175414,
+      "step": 1830
+    },
+    {
+      "epoch": 1.9274299187843855,
+      "grad_norm": 0.1377716213464737,
+      "learning_rate": 3.0026987492639668e-05,
+      "loss": 0.08148540854454041,
+      "step": 1840
+    },
+    {
+      "epoch": 1.9379093528949438,
+      "grad_norm": 0.10483554750680923,
+      "learning_rate": 2.951009256410255e-05,
+      "loss": 0.07040726542472839,
+      "step": 1850
+    },
+    {
+      "epoch": 1.9483887870055017,
+      "grad_norm": 0.08736151456832886,
+      "learning_rate": 2.8995816168394702e-05,
+      "loss": 0.04931557774543762,
+      "step": 1860
+    },
+    {
+      "epoch": 1.9588682211160597,
+      "grad_norm": 0.11461569368839264,
+      "learning_rate": 2.848422402809828e-05,
+      "loss": 0.057559752464294435,
+      "step": 1870
+    },
+    {
+      "epoch": 1.9693476552266178,
+      "grad_norm": 0.09060918539762497,
+      "learning_rate": 2.7975381522757803e-05,
+      "loss": 0.06379705667495728,
+      "step": 1880
+    },
+    {
+      "epoch": 1.9798270893371757,
+      "grad_norm": 0.07104971259832382,
+      "learning_rate": 2.746935368052477e-05,
+      "loss": 0.05813115239143372,
+      "step": 1890
+    },
+    {
+      "epoch": 1.990306523447734,
+      "grad_norm": 0.10802938044071198,
+      "learning_rate": 2.696620516984733e-05,
+      "loss": 0.07732833027839661,
+      "step": 1900
+    },
+    {
+      "epoch": 2.0,
+      "grad_norm": 0.16884952783584595,
+      "learning_rate": 2.6466000291206004e-05,
+      "loss": 0.06166202425956726,
+      "step": 1910
+    },
+    {
+      "epoch": 2.010479434110558,
+      "grad_norm": 0.08582179993391037,
+      "learning_rate": 2.5968802968896228e-05,
+      "loss": 0.04766199886798859,
+      "step": 1920
+    },
+    {
+      "epoch": 2.020958868221116,
+      "grad_norm": 0.1457364708185196,
+      "learning_rate": 2.5474676742859048e-05,
+      "loss": 0.03826354146003723,
+      "step": 1930
+    },
+    {
+      "epoch": 2.031438302331674,
+      "grad_norm": 0.09275342524051666,
+      "learning_rate": 2.4983684760561023e-05,
+      "loss": 0.045059433579444884,
+      "step": 1940
+    },
+    {
+      "epoch": 2.0419177364422323,
+      "grad_norm": 0.09085927903652191,
+      "learning_rate": 2.44958897689242e-05,
+      "loss": 0.04904903173446655,
+      "step": 1950
+    },
+    {
+      "epoch": 2.05239717055279,
+      "grad_norm": 0.11733179539442062,
+      "learning_rate": 2.401135410630731e-05,
+      "loss": 0.05008396506309509,
+      "step": 1960
+    },
+    {
+      "epoch": 2.062876604663348,
+      "grad_norm": 0.0894237607717514,
+      "learning_rate": 2.3530139694539095e-05,
+      "loss": 0.04057626128196716,
+      "step": 1970
+    },
+    {
+      "epoch": 2.0733560387739063,
+      "grad_norm": 0.08560927212238312,
+      "learning_rate": 2.305230803100496e-05,
+      "loss": 0.04843136668205261,
+      "step": 1980
+    },
+    {
+      "epoch": 2.083835472884464,
+      "grad_norm": 0.07991836220026016,
+      "learning_rate": 2.257792018078793e-05,
+      "loss": 0.0544127106666565,
+      "step": 1990
+    },
+    {
+      "epoch": 2.0943149069950224,
+      "grad_norm": 0.08846250921487808,
+      "learning_rate": 2.210703676886461e-05,
+      "loss": 0.0459000825881958,
+      "step": 2000
+    },
+    {
+      "epoch": 2.0943149069950224,
+      "eval_loss": 0.060011014342308044,
+      "eval_runtime": 36.3755,
+      "eval_samples_per_second": 8.55,
+      "eval_steps_per_second": 8.55,
+      "step": 2000
+    },
+    {
+      "epoch": 2.1047943411055803,
+      "grad_norm": 0.10082945972681046,
+      "learning_rate": 2.1639717972357678e-05,
+      "loss": 0.038090622425079344,
+      "step": 2010
+    },
+    {
+      "epoch": 2.115273775216138,
+      "grad_norm": 0.05712248757481575,
+      "learning_rate": 2.1176023512845376e-05,
+      "loss": 0.04598597884178161,
+      "step": 2020
+    },
+    {
+      "epoch": 2.1257532093266964,
+      "grad_norm": 0.11628362536430359,
+      "learning_rate": 2.0716012648729353e-05,
+      "loss": 0.04984880685806274,
+      "step": 2030
+    },
+    {
+      "epoch": 2.1362326434372543,
+      "grad_norm": 0.10635484755039215,
+      "learning_rate": 2.025974416766171e-05,
+      "loss": 0.04293925166130066,
+      "step": 2040
+    },
+    {
+      "epoch": 2.1467120775478126,
+      "grad_norm": 0.1017381027340889,
+      "learning_rate": 1.9807276379032113e-05,
+      "loss": 0.04305694401264191,
+      "step": 2050
+    },
+    {
+      "epoch": 2.1571915116583704,
+      "grad_norm": 0.13550882041454315,
+      "learning_rate": 1.9358667106516055e-05,
+      "loss": 0.04478869140148163,
+      "step": 2060
+    },
+    {
+      "epoch": 2.1676709457689283,
+      "grad_norm": 0.08526366949081421,
+      "learning_rate": 1.8913973680685226e-05,
+      "loss": 0.036646312475204466,
+      "step": 2070
+    },
+    {
+      "epoch": 2.1781503798794866,
+      "grad_norm": 0.10932011157274246,
+      "learning_rate": 1.8473252931680928e-05,
+      "loss": 0.042200219631195066,
+      "step": 2080
+    },
+    {
+      "epoch": 2.1886298139900444,
+      "grad_norm": 0.08768360316753387,
+      "learning_rate": 1.803656118195136e-05,
+      "loss": 0.0437488317489624,
+      "step": 2090
+    },
+    {
+      "epoch": 2.1991092481006027,
+      "grad_norm": 0.08362651616334915,
+      "learning_rate": 1.760395423905379e-05,
+      "loss": 0.04669668078422547,
+      "step": 2100
+    },
+    {
+      "epoch": 2.2095886822111606,
+      "grad_norm": 0.08554034680128098,
+      "learning_rate": 1.7175487388522588e-05,
+      "loss": 0.034989356994628906,
+      "step": 2110
+    },
+    {
+      "epoch": 2.220068116321719,
+      "grad_norm": 0.08215561509132385,
+      "learning_rate": 1.6751215386803986e-05,
+      "loss": 0.040298929810523985,
+      "step": 2120
+    },
+    {
+      "epoch": 2.2305475504322767,
+      "grad_norm": 0.0840689167380333,
+      "learning_rate": 1.6331192454258337e-05,
+      "loss": 0.041704925894737246,
+      "step": 2130
+    },
+    {
+      "epoch": 2.2410269845428346,
+      "grad_norm": 0.06530614197254181,
+      "learning_rate": 1.5915472268231018e-05,
+      "loss": 0.03651900887489319,
+      "step": 2140
+    },
+    {
+      "epoch": 2.251506418653393,
+      "grad_norm": 0.12431822717189789,
+      "learning_rate": 1.550410795619261e-05,
+      "loss": 0.04806804955005646,
+      "step": 2150
+    },
+    {
+      "epoch": 2.2619858527639507,
+      "grad_norm": 0.09592410176992416,
+      "learning_rate": 1.509715208894949e-05,
+      "loss": 0.0454313725233078,
+      "step": 2160
+    },
+    {
+      "epoch": 2.2724652868745085,
+      "grad_norm": 0.07589780539274216,
+      "learning_rate": 1.469465667392536e-05,
+      "loss": 0.03574602603912354,
+      "step": 2170
+    },
+    {
+      "epoch": 2.282944720985067,
+      "grad_norm": 0.09734483063220978,
+      "learning_rate": 1.4296673148515038e-05,
+      "loss": 0.04358702301979065,
+      "step": 2180
+    },
+    {
+      "epoch": 2.2934241550956247,
+      "grad_norm": 0.0974339172244072,
+      "learning_rate": 1.3903252373510838e-05,
+      "loss": 0.04603351950645447,
+      "step": 2190
+    },
+    {
+      "epoch": 2.303903589206183,
+      "grad_norm": 0.09025271981954575,
+      "learning_rate": 1.3514444626602773e-05,
+      "loss": 0.040065237879753114,
+      "step": 2200
+    },
+    {
+      "epoch": 2.314383023316741,
+      "grad_norm": 0.07625086605548859,
+      "learning_rate": 1.3130299595953338e-05,
+      "loss": 0.044061675667762756,
+      "step": 2210
+    },
+    {
+      "epoch": 2.324862457427299,
+      "grad_norm": 0.07306221127510071,
+      "learning_rate": 1.2750866373847465e-05,
+      "loss": 0.03366467654705048,
+      "step": 2220
+    },
+    {
+      "epoch": 2.335341891537857,
+      "grad_norm": 0.08357638120651245,
+      "learning_rate": 1.2376193450418715e-05,
+      "loss": 0.041424044966697694,
+      "step": 2230
+    },
+    {
+      "epoch": 2.345821325648415,
+      "grad_norm": 0.09153921157121658,
+      "learning_rate": 1.2006328707452459e-05,
+      "loss": 0.03938372135162353,
+      "step": 2240
+    },
+    {
+      "epoch": 2.356300759758973,
+      "grad_norm": 0.09109660983085632,
+      "learning_rate": 1.1641319412266765e-05,
+      "loss": 0.04015985131263733,
+      "step": 2250
+    },
+    {
+      "epoch": 2.356300759758973,
+      "eval_loss": 0.05486458167433739,
+      "eval_runtime": 36.8119,
+      "eval_samples_per_second": 8.448,
+      "eval_steps_per_second": 8.448,
+      "step": 2250
+    },
+    {
+      "epoch": 2.366780193869531,
+      "grad_norm": 0.052502721548080444,
+      "learning_rate": 1.1281212211671822e-05,
+      "loss": 0.0270554780960083,
+      "step": 2260
+    },
+    {
+      "epoch": 2.377259627980089,
+      "grad_norm": 0.07931812107563019,
+      "learning_rate": 1.0926053126008584e-05,
+      "loss": 0.0417300134897232,
+      "step": 2270
+    },
+    {
+      "epoch": 2.387739062090647,
+      "grad_norm": 0.08996254205703735,
+      "learning_rate": 1.0575887543267609e-05,
+      "loss": 0.037659955024719236,
+      "step": 2280
+    },
+    {
+      "epoch": 2.398218496201205,
+      "grad_norm": 0.08800788223743439,
+      "learning_rate": 1.023076021328867e-05,
+      "loss": 0.048437944054603575,
+      "step": 2290
+    },
+    {
+      "epoch": 2.4086979303117633,
+      "grad_norm": 0.10572271049022675,
+      "learning_rate": 9.890715242041787e-06,
+      "loss": 0.04166909456253052,
+      "step": 2300
+    },
+    {
+      "epoch": 2.419177364422321,
+      "grad_norm": 0.10573071986436844,
+      "learning_rate": 9.555796085990781e-06,
+      "loss": 0.03919607996940613,
+      "step": 2310
+    },
+    {
+      "epoch": 2.4296567985328794,
+      "grad_norm": 0.09714583307504654,
+      "learning_rate": 9.226045546539608e-06,
+      "loss": 0.03530588150024414,
+      "step": 2320
+    },
+    {
+      "epoch": 2.4401362326434373,
+      "grad_norm": 0.09436199069023132,
+      "learning_rate": 8.901505764562518e-06,
+      "loss": 0.05111382007598877,
+      "step": 2330
+    },
+    {
+      "epoch": 2.450615666753995,
+      "grad_norm": 0.06353961676359177,
+      "learning_rate": 8.582218215018656e-06,
+      "loss": 0.03805697858333588,
+      "step": 2340
+    },
+    {
+      "epoch": 2.4610951008645534,
+      "grad_norm": 0.08853815495967865,
+      "learning_rate": 8.268223701651684e-06,
+      "loss": 0.04815975427627563,
+      "step": 2350
+    },
+    {
+      "epoch": 2.4715745349751113,
+      "grad_norm": 0.07472016662359238,
+      "learning_rate": 7.959562351775196e-06,
+      "loss": 0.042247459292411804,
+      "step": 2360
+    },
+    {
+      "epoch": 2.4820539690856696,
+      "grad_norm": 0.12121549248695374,
+      "learning_rate": 7.656273611144632e-06,
+      "loss": 0.040102115273475646,
+      "step": 2370
+    },
+    {
+      "epoch": 2.4925334031962274,
+      "grad_norm": 0.08667747676372528,
+      "learning_rate": 7.358396238916254e-06,
+      "loss": 0.03656341433525086,
+      "step": 2380
+    },
+    {
+      "epoch": 2.5030128373067857,
+      "grad_norm": 0.1162872165441513,
+      "learning_rate": 7.065968302693882e-06,
+      "loss": 0.04052766263484955,
+      "step": 2390
+    },
+    {
+      "epoch": 2.5134922714173435,
+      "grad_norm": 0.07924140989780426,
+      "learning_rate": 6.7790271736639595e-06,
+      "loss": 0.03394221067428589,
+      "step": 2400
+    },
+    {
+      "epoch": 2.5239717055279014,
+      "grad_norm": 0.09523408859968185,
+      "learning_rate": 6.497609521819681e-06,
+      "loss": 0.04119439423084259,
+      "step": 2410
+    },
+    {
+      "epoch": 2.5344511396384597,
+      "grad_norm": 0.12182598561048508,
+      "learning_rate": 6.221751311274731e-06,
+      "loss": 0.05154783725738525,
+      "step": 2420
+    },
+    {
+      "epoch": 2.5449305737490175,
+      "grad_norm": 0.09359873831272125,
+      "learning_rate": 5.951487795667149e-06,
+      "loss": 0.035483264923095705,
+      "step": 2430
+    },
+    {
+      "epoch": 2.5554100078595754,
+      "grad_norm": 0.08514095097780228,
+      "learning_rate": 5.686853513654117e-06,
+      "loss": 0.03830339312553406,
+      "step": 2440
+    },
+    {
+      "epoch": 2.5658894419701337,
+      "grad_norm": 0.10625084489583969,
+      "learning_rate": 5.4278822844979705e-06,
+      "loss": 0.034111028909683226,
+      "step": 2450
+    },
+    {
+      "epoch": 2.5763688760806915,
+      "grad_norm": 0.1004003956913948,
+      "learning_rate": 5.174607203744286e-06,
+      "loss": 0.04465605318546295,
+      "step": 2460
+    },
+    {
+      "epoch": 2.58684831019125,
+      "grad_norm": 0.0962519720196724,
+      "learning_rate": 4.927060638992382e-06,
+      "loss": 0.041056016087532045,
+      "step": 2470
+    },
+    {
+      "epoch": 2.5973277443018077,
+      "grad_norm": 0.06380607187747955,
+      "learning_rate": 4.685274225758846e-06,
+      "loss": 0.03880062401294708,
+      "step": 2480
+    },
+    {
+      "epoch": 2.607807178412366,
+      "grad_norm": 0.07326535880565643,
+      "learning_rate": 4.449278863434647e-06,
+      "loss": 0.03194461762905121,
+      "step": 2490
+    },
+    {
+      "epoch": 2.618286612522924,
+      "grad_norm": 0.12218596786260605,
+      "learning_rate": 4.2191047113362854e-06,
+      "loss": 0.04258840978145599,
+      "step": 2500
+    },
+    {
+      "epoch": 2.618286612522924,
+      "eval_loss": 0.05223666876554489,
+      "eval_runtime": 37.7234,
+      "eval_samples_per_second": 8.244,
+      "eval_steps_per_second": 8.244,
+      "step": 2500
+    },
+    {
+      "epoch": 2.6287660466334817,
+      "grad_norm": 0.08594664931297302,
+      "learning_rate": 3.994781184851598e-06,
+      "loss": 0.04302787780761719,
+      "step": 2510
+    },
+    {
+      "epoch": 2.63924548074404,
+      "grad_norm": 0.08187596499919891,
+      "learning_rate": 3.776336951680548e-06,
+      "loss": 0.0341387003660202,
+      "step": 2520
+    },
+    {
+      "epoch": 2.649724914854598,
+      "grad_norm": 0.10216796398162842,
+      "learning_rate": 3.563799928171596e-06,
+      "loss": 0.04289879500865936,
+      "step": 2530
+    },
+    {
+      "epoch": 2.6602043489651557,
+      "grad_norm": 0.11215174198150635,
+      "learning_rate": 3.3571972757540814e-06,
+      "loss": 0.04055049121379852,
+      "step": 2540
+    },
+    {
+      "epoch": 2.670683783075714,
+      "grad_norm": 0.07941269129514694,
+      "learning_rate": 3.156555397467176e-06,
+      "loss": 0.04118689000606537,
+      "step": 2550
+    },
+    {
+      "epoch": 2.681163217186272,
+      "grad_norm": 0.09404437988996506,
+      "learning_rate": 2.9618999345855547e-06,
+      "loss": 0.03079705536365509,
+      "step": 2560
+    },
+    {
+      "epoch": 2.69164265129683,
+      "grad_norm": 0.1109817698597908,
+      "learning_rate": 2.773255763342647e-06,
+      "loss": 0.038885954022407535,
+      "step": 2570
+    },
+    {
+      "epoch": 2.702122085407388,
+      "grad_norm": 0.09431962668895721,
+      "learning_rate": 2.590646991751472e-06,
+      "loss": 0.043543145060539246,
+      "step": 2580
+    },
+    {
+      "epoch": 2.7126015195179463,
+      "grad_norm": 0.08184763044118881,
+      "learning_rate": 2.414096956523776e-06,
+      "loss": 0.03256987631320953,
+      "step": 2590
+    },
+    {
+      "epoch": 2.723080953628504,
+      "grad_norm": 0.08390141278505325,
+      "learning_rate": 2.2436282200876458e-06,
+      "loss": 0.03908055424690247,
+      "step": 2600
+    },
+    {
+      "epoch": 2.733560387739062,
+      "grad_norm": 0.0762532502412796,
+      "learning_rate": 2.07926256770416e-06,
+      "loss": 0.04899201393127441,
+      "step": 2610
+    },
+    {
+      "epoch": 2.7440398218496203,
+      "grad_norm": 0.08239631354808807,
+      "learning_rate": 1.9210210046832768e-06,
+      "loss": 0.048707082867622375,
+      "step": 2620
+    },
+    {
+      "epoch": 2.754519255960178,
+      "grad_norm": 0.09619107842445374,
+      "learning_rate": 1.7689237536994364e-06,
+      "loss": 0.0372231125831604,
+      "step": 2630
+    },
+    {
+      "epoch": 2.764998690070736,
+      "grad_norm": 0.07099667191505432,
+      "learning_rate": 1.6229902522072293e-06,
+      "loss": 0.03421170711517334,
+      "step": 2640
+    },
+    {
+      "epoch": 2.7754781241812942,
+      "grad_norm": 0.10154753923416138,
+      "learning_rate": 1.4832391499572996e-06,
+      "loss": 0.03656705319881439,
+      "step": 2650
+    },
+    {
+      "epoch": 2.785957558291852,
+      "grad_norm": 0.09349387139081955,
+      "learning_rate": 1.3496883066130173e-06,
+      "loss": 0.03710306882858276,
+      "step": 2660
+    },
+    {
+      "epoch": 2.7964369924024104,
+      "grad_norm": 0.061091430485248566,
+      "learning_rate": 1.2223547894680443e-06,
+      "loss": 0.0308389812707901,
+      "step": 2670
+    },
+    {
+      "epoch": 2.8069164265129682,
+      "grad_norm": 0.09838075935840607,
+      "learning_rate": 1.101254871265256e-06,
+      "loss": 0.03703555166721344,
+      "step": 2680
+    },
+    {
+      "epoch": 2.8173958606235265,
+      "grad_norm": 0.10046928375959396,
+      "learning_rate": 9.864040281170938e-07,
+      "loss": 0.04500553905963898,
+      "step": 2690
+    },
+    {
+      "epoch": 2.8278752947340844,
+      "grad_norm": 0.06770773977041245,
+      "learning_rate": 8.778169375277978e-07,
+      "loss": 0.03823737502098083,
+      "step": 2700
+    },
+    {
+      "epoch": 2.8383547288446422,
+      "grad_norm": 0.08373535424470901,
+      "learning_rate": 7.755074765176618e-07,
+      "loss": 0.03961678743362427,
+      "step": 2710
+    },
+    {
+      "epoch": 2.8488341629552005,
+      "grad_norm": 0.07590050995349884,
+      "learning_rate": 6.794887198496413e-07,
+      "loss": 0.03221273124217987,
+      "step": 2720
+    },
+    {
+      "epoch": 2.8593135970657584,
+      "grad_norm": 0.08507678657770157,
+      "learning_rate": 5.897729383583906e-07,
+      "loss": 0.04571912884712219,
+      "step": 2730
+    },
+    {
+      "epoch": 2.8697930311763162,
+      "grad_norm": 0.06584763526916504,
+      "learning_rate": 5.063715973821659e-07,
+      "loss": 0.03794914484024048,
+      "step": 2740
+    },
+    {
+      "epoch": 2.8802724652868745,
+      "grad_norm": 0.07312892377376556,
+      "learning_rate": 4.292953552975154e-07,
+      "loss": 0.036365586519241336,
+      "step": 2750
+    },
+    {
+      "epoch": 2.8802724652868745,
+      "eval_loss": 0.05090421438217163,
+      "eval_runtime": 85.293,
+      "eval_samples_per_second": 3.646,
+      "eval_steps_per_second": 3.646,
+      "step": 2750
+    },
+    {
+      "epoch": 2.8907518993974324,
+      "grad_norm": 0.08459606021642685,
+      "learning_rate": 3.5855406215725697e-07,
+      "loss": 0.03068857192993164,
+      "step": 2760
+    },
+    {
+      "epoch": 2.9012313335079907,
+      "grad_norm": 0.06866376101970673,
+      "learning_rate": 2.9415675843163515e-07,
+      "loss": 0.03265829384326935,
+      "step": 2770
+    },
+    {
+      "epoch": 2.9117107676185485,
+      "grad_norm": 0.09082643687725067,
+      "learning_rate": 2.361116738529956e-07,
+      "loss": 0.03418546915054321,
+      "step": 2780
+    },
+    {
+      "epoch": 2.922190201729107,
+      "grad_norm": 0.10772739350795746,
+      "learning_rate": 1.8442622636404284e-07,
+      "loss": 0.03810786008834839,
+      "step": 2790
+    },
+    {
+      "epoch": 2.9326696358396647,
+      "grad_norm": 0.08321297913789749,
+      "learning_rate": 1.391070211698764e-07,
+      "loss": 0.04068491756916046,
+      "step": 2800
+    },
+    {
+      "epoch": 2.9431490699502225,
+      "grad_norm": 0.11239277571439743,
+      "learning_rate": 1.0015984989385496e-07,
+      "loss": 0.041029155254364014,
+      "step": 2810
+    },
+    {
+      "epoch": 2.953628504060781,
+      "grad_norm": 0.07199843227863312,
+      "learning_rate": 6.758968983747171e-08,
+      "loss": 0.037902483344078065,
+      "step": 2820
+    },
+    {
+      "epoch": 2.9641079381713387,
+      "grad_norm": 0.08249279856681824,
+      "learning_rate": 4.140070334422985e-08,
+      "loss": 0.03996126651763916,
+      "step": 2830
+    },
+    {
+      "epoch": 2.9745873722818965,
+      "grad_norm": 0.0852220207452774,
+      "learning_rate": 2.1596237267751396e-08,
+      "loss": 0.04228667616844177,
+      "step": 2840
+    },
+    {
+      "epoch": 2.985066806392455,
+      "grad_norm": 0.0858582928776741,
+      "learning_rate": 8.178822544052666e-09,
+      "loss": 0.03813594281673431,
+      "step": 2850
+    },
+    {
+      "epoch": 2.995546240503013,
+      "grad_norm": 0.06642451137304306,
+      "learning_rate": 1.1501738680919084e-09,
+      "loss": 0.033472076058387756,
+      "step": 2860
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 2865,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 250,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 9.031737271887514e+17,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-2865/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e8fc737554ff6f82c4ea137b5313611e3b2b3b63fd69b3926d6b1fe9da14c0a6
+size 5201

merged/chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is true %}
+        {{- '<think>\n' }}
+    {%- else %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- endif %}
+{%- endif %}

merged/config.json

@@ -0,0 +1,75 @@
+{
+  "architectures": [
+    "Qwen3_5ForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "attn_output_gate": true,
+  "bos_token_id": null,
+  "dtype": "bfloat16",
+  "eos_token_id": 248046,
+  "full_attention_interval": 4,
+  "head_dim": 256,
+  "hidden_act": "silu",
+  "hidden_size": 2048,
+  "initializer_range": 0.02,
+  "intermediate_size": 6144,
+  "layer_types": [
+    "linear_attention",
+    "linear_attention",
+    "linear_attention",
+    "full_attention",
+    "linear_attention",
+    "linear_attention",
+    "linear_attention",
+    "full_attention",
+    "linear_attention",
+    "linear_attention",
+    "linear_attention",
+    "full_attention",
+    "linear_attention",
+    "linear_attention",
+    "linear_attention",
+    "full_attention",
+    "linear_attention",
+    "linear_attention",
+    "linear_attention",
+    "full_attention",
+    "linear_attention",
+    "linear_attention",
+    "linear_attention",
+    "full_attention"
+  ],
+  "linear_conv_kernel_dim": 4,
+  "linear_key_head_dim": 128,
+  "linear_num_key_heads": 16,
+  "linear_num_value_heads": 16,
+  "linear_value_head_dim": 128,
+  "mamba_ssm_dtype": "float32",
+  "max_position_embeddings": 262144,
+  "mlp_only_layers": [],
+  "model_type": "qwen3_5_text",
+  "mtp_num_hidden_layers": 1,
+  "mtp_use_dedicated_embeddings": false,
+  "num_attention_heads": 8,
+  "num_hidden_layers": 24,
+  "num_key_value_heads": 2,
+  "pad_token_id": 248044,
+  "partial_rotary_factor": 0.25,
+  "rms_norm_eps": 1e-06,
+  "rope_parameters": {
+    "mrope_interleaved": true,
+    "mrope_section": [
+      11,
+      11,
+      10
+    ],
+    "partial_rotary_factor": 0.25,
+    "rope_theta": 10000000,
+    "rope_type": "default"
+  },
+  "tie_word_embeddings": true,
+  "transformers_version": "5.3.0",
+  "use_cache": false,
+  "vocab_size": 248320
+}

merged/generation_config.json

@@ -0,0 +1,10 @@
+{
+  "_from_model_config": true,
+  "eos_token_id": [
+    248046,
+    248044
+  ],
+  "pad_token_id": 248044,
+  "transformers_version": "5.3.0",
+  "use_cache": true
+}

merged/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6cd7701ff97b7561e0dc158635941426e2a6b9fce824390e6995c9485605b2b7
+size 3763692048

merged/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:87a7830d63fcf43bf241c3c5242e96e62dd3fdc29224ca26fed8ea333db72de4
+size 19989343

merged/tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": true,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

run-config.txt

@@ -0,0 +1,12 @@
+model=/data/pretrained_models/Qwen3.5-2B
+data=/home/lg/workflow_tooluse/Flow_RL_luogan/temp/metamath/metamath-output/setmm-train-qwen35-4b-mixed-12000
+max_length=6144
+micro_batch_size=2
+gradient_accumulation_steps=8
+effective_batch_size=16
+learning_rate=1e-4
+num_train_epochs=3
+direct_ref_mode=same-file-distractors
+same_file_distractor_direct_refs=4
+distractor_seed=0
+gpu_ids=1

skipped-tokenization.jsonl

@@ -0,0 +1,422 @@
+{"label": "upgrwlkedg", "expanded_label": "upgriswlk", "source": "expanded", "tokens": 9508, "reason": "max_length"}
+{"label": "iiconn", "expanded_label": "1re", "source": "expanded", "tokens": 7147, "reason": "max_length"}
+{"label": "dprd2db", "expanded_label": "dprdspan", "source": "expanded", "tokens": 7426, "reason": "max_length"}
+{"label": "pm2mpf", "expanded_label": "pm2mpval", "source": "expanded", "tokens": 8963, "reason": "max_length"}
+{"label": "phllvec", "expanded_label": "isphl", "source": "expanded", "tokens": 10796, "reason": "max_length"}
+{"label": "xrlelttr", "expanded_label": "xrlttr", "source": "expanded", "tokens": 6891, "reason": "max_length"}
+{"label": "trlres", "expanded_label": "wlkres", "source": "expanded", "tokens": 17184, "reason": "max_length"}
+{"label": "pcorev", "expanded_label": "pcorevlem", "source": "expanded", "tokens": 53452, "reason": "max_length"}
+{"label": "clwlkclwwlkf1o", "expanded_label": "clwlkclwwlkfo", "source": "expanded", "tokens": 11298, "reason": "max_length"}
+{"label": "divcan7", "expanded_label": "divdivdiv", "source": "expanded", "tokens": 9114, "reason": "max_length"}
+{"label": "ghmfghm", "expanded_label": "ghmgrp", "source": "expanded", "tokens": 6287, "reason": "max_length"}
+{"label": "zrzeroorngc", "expanded_label": "zrtermorngc", "source": "expanded", "tokens": 9359, "reason": "max_length"}
+{"label": "srgmgp", "expanded_label": "issrg", "source": "expanded", "tokens": 12562, "reason": "max_length"}
+{"label": "m1bits", "expanded_label": "bitscmp", "source": "expanded", "tokens": 15084, "reason": "max_length"}
+{"label": "resinhcl", "expanded_label": "sinhval", "source": "expanded", "tokens": 6694, "reason": "max_length"}
+{"label": "orngogrp", "expanded_label": "isorng", "source": "expanded", "tokens": 9577, "reason": "max_length"}
+{"label": "psrasclcl", "expanded_label": "psrring", "source": "expanded", "tokens": 8539, "reason": "max_length"}
+{"label": "usgrexmpl1", "expanded_label": "usgrexmpl1lem", "source": "expanded", "tokens": 23243, "reason": "max_length"}
+{"label": "lssnvc", "expanded_label": "lssnlm", "source": "expanded", "tokens": 9394, "reason": "max_length"}
+{"label": "xlemul2a", "expanded_label": "xlemul1a", "source": "expanded", "tokens": 17516, "reason": "max_length"}
+{"label": "cphphl", "expanded_label": "iscph", "source": "expanded", "tokens": 8762, "reason": "max_length"}
+{"label": "fclstop", "expanded_label": "isfcls", "source": "expanded", "tokens": 7205, "reason": "max_length"}
+{"label": "lvecprop2d", "expanded_label": "lmodprop2d", "source": "expanded", "tokens": 33457, "reason": "max_length"}
+{"label": "drhmsubcALTV", "expanded_label": "srhmsubcALTV", "source": "expanded", "tokens": 13003, "reason": "max_length"}
+{"label": "nsmndex1", "expanded_label": "smndex1n0mnd", "source": "expanded", "tokens": 8470, "reason": "max_length"}
+{"label": "ringcidALTV", "expanded_label": "ringccatidALTV", "source": "expanded", "tokens": 31357, "reason": "max_length"}
+{"label": "ghmima", "expanded_label": "ghmrn", "source": "expanded", "tokens": 8915, "reason": "max_length"}
+{"label": "pmatassa", "expanded_label": "matassa", "source": "expanded", "tokens": 9716, "reason": "max_length"}
+{"label": "addgt0", "expanded_label": "00id", "source": "expanded", "tokens": 8215, "reason": "max_length"}
+{"label": "ser0", "expanded_label": "00id", "source": "expanded", "tokens": 8607, "reason": "max_length"}
+{"label": "mulgfn", "expanded_label": "mulgfval", "source": "expanded", "tokens": 15315, "reason": "max_length"}
+{"label": "usgrexmpl2", "expanded_label": "usgrexmpl2lem", "source": "expanded", "tokens": 22678, "reason": "max_length"}
+{"label": "indistps2ALT", "expanded_label": "indistopon", "source": "expanded", "tokens": 6910, "reason": "max_length"}
+{"label": "pfx0", "expanded_label": "swrd0", "source": "expanded", "tokens": 7535, "reason": "max_length"}
+{"label": "dprd2db", "expanded_label": "dprd2da", "source": "expanded", "tokens": 71940, "reason": "max_length"}
+{"label": "fmfg", "expanded_label": "fgcl", "source": "expanded", "tokens": 8599, "reason": "max_length"}
+{"label": "numufl", "expanded_label": "filssufilg", "source": "expanded", "tokens": 9093, "reason": "max_length"}
+{"label": "wlkiswwlkupgr", "expanded_label": "wlkiswwlksupgr2", "source": "expanded", "tokens": 12895, "reason": "max_length"}
+{"label": "addcomsr", "expanded_label": "addsrpr", "source": "expanded", "tokens": 14104, "reason": "max_length"}
+{"label": "mat1rhm", "expanded_label": "matring", "source": "expanded", "tokens": 18352, "reason": "max_length"}
+{"label": "dpjid", "expanded_label": "dpjidcl", "source": "expanded", "tokens": 38662, "reason": "max_length"}
+{"label": "metcn4", "expanded_label": "met1stc", "source": "expanded", "tokens": 13166, "reason": "max_length"}
+{"label": "mhmf", "expanded_label": "ismhm", "source": "expanded", "tokens": 7423, "reason": "max_length"}
+{"label": "pr01ssre", "expanded_label": "1re", "source": "expanded", "tokens": 6787, "reason": "max_length"}
+{"label": "zcld2", "expanded_label": "recld2", "source": "expanded", "tokens": 7912, "reason": "max_length"}
+{"label": "submgmmgm", "expanded_label": "issubmgm2", "source": "expanded", "tokens": 6349, "reason": "max_length"}
+{"label": "rehaus", "expanded_label": "tgioo", "source": "expanded", "tokens": 22231, "reason": "max_length"}
+{"label": "pmat1op", "expanded_label": "mat1", "source": "expanded", "tokens": 6209, "reason": "max_length"}
+{"label": "sincos1sgn", "expanded_label": "1re", "source": "expanded", "tokens": 9636, "reason": "max_length"}
+{"label": "wwlksnonfi", "expanded_label": "wwlksnfi", "source": "expanded", "tokens": 6363, "reason": "max_length"}
+{"label": "1rp", "expanded_label": "1re", "source": "expanded", "tokens": 6599, "reason": "max_length"}
+{"label": "qtopcmp", "expanded_label": "cncmp", "source": "expanded", "tokens": 16895, "reason": "max_length"}
+{"label": "sqrtle", "expanded_label": "resqrtcl", "source": "expanded", "tokens": 7775, "reason": "max_length"}
+{"label": "opsrsca", "expanded_label": "psrsca", "source": "expanded", "tokens": 6160, "reason": "max_length"}
+{"label": "pgpfi2", "expanded_label": "pgpfi", "source": "expanded", "tokens": 18037, "reason": "max_length"}
+{"label": "ere", "expanded_label": "1re", "source": "expanded", "tokens": 6736, "reason": "max_length"}
+{"label": "sgrp2nmnd", "expanded_label": "sgrp2nmndlem5", "source": "expanded", "tokens": 6277, "reason": "max_length"}
+{"label": "tgtopon", "expanded_label": "tgcl", "source": "expanded", "tokens": 9236, "reason": "max_length"}
+{"label": "cnptop1", "expanded_label": "iscnp2", "source": "expanded", "tokens": 9488, "reason": "max_length"}
+{"label": "fprod0diag", "expanded_label": "fsum0diaglem", "source": "expanded", "tokens": 6497, "reason": "max_length"}
+{"label": "zprodn0", "expanded_label": "zprod", "source": "expanded", "tokens": 18101, "reason": "max_length"}
+{"label": "ishtpyd", "expanded_label": "ishtpy", "source": "expanded", "tokens": 8698, "reason": "max_length"}
+{"label": "axlttrn", "expanded_label": "ltxrlt", "source": "expanded", "tokens": 8594, "reason": "max_length"}
+{"label": "xrltletr", "expanded_label": "xrlttr", "source": "expanded", "tokens": 6927, "reason": "max_length"}
+{"label": "m2cpmf1o", "expanded_label": "m2cpmfo", "source": "expanded", "tokens": 6756, "reason": "max_length"}
+{"label": "lgricngricex", "expanded_label": "gpg5grlic", "source": "expanded", "tokens": 6365, "reason": "max_length"}
+{"label": "nnrecre", "expanded_label": "1re", "source": "expanded", "tokens": 6790, "reason": "max_length"}
+{"label": "grimedgi", "expanded_label": "grimedg", "source": "expanded", "tokens": 21386, "reason": "max_length"}
+{"label": "neg1lt0", "expanded_label": "1re", "source": "expanded", "tokens": 6830, "reason": "max_length"}
+{"label": "pi1xfrgim", "expanded_label": "pi1xfrcnv", "source": "expanded", "tokens": 19046, "reason": "max_length"}
+{"label": "metreg", "expanded_label": "methaus", "source": "expanded", "tokens": 11319, "reason": "max_length"}
+{"label": "fldiv2", "expanded_label": "fldiv", "source": "expanded", "tokens": 15130, "reason": "max_length"}
+{"label": "erclwwlkn", "expanded_label": "erclwwlknsym", "source": "expanded", "tokens": 7474, "reason": "max_length"}
+{"label": "uvtx2vtx1edg", "expanded_label": "nbgr2vtx1edg", "source": "expanded", "tokens": 8190, "reason": "max_length"}
+{"label": "recms", "expanded_label": "recld2", "source": "expanded", "tokens": 7878, "reason": "max_length"}
+{"label": "iccordt", "expanded_label": "letsr", "source": "expanded", "tokens": 6694, "reason": "max_length"}
+{"label": "crhmsubc", "expanded_label": "srhmsubc", "source": "expanded", "tokens": 12634, "reason": "max_length"}
+{"label": "xmul02", "expanded_label": "xmulcom", "source": "expanded", "tokens": 8030, "reason": "max_length"}
+{"label": "pcabs", "expanded_label": "pcneg", "source": "expanded", "tokens": 8094, "reason": "max_length"}
+{"label": "indisuni", "expanded_label": "indistopon", "source": "expanded", "tokens": 7576, "reason": "max_length"}
+{"label": "rngqiprngho", "expanded_label": "rngqiprngghm", "source": "expanded", "tokens": 10189, "reason": "max_length"}
+{"label": "peano2re", "expanded_label": "1re", "source": "expanded", "tokens": 6658, "reason": "max_length"}
+{"label": "seqabs", "expanded_label": "fsumabs", "source": "expanded", "tokens": 19216, "reason": "max_length"}
+{"label": "prmrp", "expanded_label": "coprm", "source": "expanded", "tokens": 7707, "reason": "max_length"}
+{"label": "addsub", "expanded_label": "addcom", "source": "expanded", "tokens": 6963, "reason": "max_length"}
+{"label": "xmullid", "expanded_label": "xmulcom", "source": "expanded", "tokens": 8165, "reason": "max_length"}
+{"label": "lsslmod", "expanded_label": "islss3", "source": "expanded", "tokens": 11153, "reason": "max_length"}
+{"label": "mat0dimid", "expanded_label": "matring", "source": "expanded", "tokens": 16819, "reason": "max_length"}
+{"label": "1lt4", "expanded_label": "1re", "source": "expanded", "tokens": 6930, "reason": "max_length"}
+{"label": "2ndctop", "expanded_label": "tgcl", "source": "expanded", "tokens": 9706, "reason": "max_length"}
+{"label": "ordthmeo", "expanded_label": "isocnv", "source": "expanded", "tokens": 6246, "reason": "max_length"}
+{"label": "9re", "expanded_label": "1re", "source": "expanded", "tokens": 6826, "reason": "max_length"}
+{"label": "addgegt0", "expanded_label": "00id", "source": "expanded", "tokens": 8276, "reason": "max_length"}
+{"label": "mdetuni", "expanded_label": "mdetuni0", "source": "expanded", "tokens": 36640, "reason": "max_length"}
+{"label": "wlkswwlksen", "expanded_label": "wlkswwlksf1o", "source": "expanded", "tokens": 8077, "reason": "max_length"}
+{"label": "cphnlm", "expanded_label": "iscph", "source": "expanded", "tokens": 8530, "reason": "max_length"}
+{"label": "algcvgb", "expanded_label": "algcvgblem", "source": "expanded", "tokens": 7042, "reason": "max_length"}
+{"label": "neglcm", "expanded_label": "lcmneg", "source": "expanded", "tokens": 8580, "reason": "max_length"}
+{"label": "metflem", "expanded_label": "ismet", "source": "expanded", "tokens": 6299, "reason": "max_length"}
+{"label": "nmoleub2b", "expanded_label": "nmoleub2lem2", "source": "expanded", "tokens": 13552, "reason": "max_length"}
+{"label": "nbedgusgr", "expanded_label": "hasheqf1oi", "source": "expanded", "tokens": 7832, "reason": "max_length"}
+{"label": "drhmsubc", "expanded_label": "srhmsubc", "source": "expanded", "tokens": 12663, "reason": "max_length"}
+{"label": "nnpw2blenfzo2", "expanded_label": "elfzolborelfzop1", "source": "expanded", "tokens": 7424, "reason": "max_length"}
+{"label": "lssvancl2", "expanded_label": "lmodcom", "source": "expanded", "tokens": 11067, "reason": "max_length"}
+{"label": "ordthmeo", "expanded_label": "ordthmeolem", "source": "expanded", "tokens": 26266, "reason": "max_length"}
+{"label": "ismhmd", "expanded_label": "ismhm", "source": "expanded", "tokens": 7438, "reason": "max_length"}
+{"label": "crctcshwlk", "expanded_label": "crctcshlem4", "source": "expanded", "tokens": 6817, "reason": "max_length"}
+{"label": "0reALT", "expanded_label": "1re", "source": "expanded", "tokens": 8961, "reason": "max_length"}
+{"label": "fsumshft", "expanded_label": "mptfzshft", "source": "expanded", "tokens": 7431, "reason": "max_length"}
+{"label": "ltm1", "expanded_label": "1re", "source": "expanded", "tokens": 7209, "reason": "max_length"}
+{"label": "subcn", "expanded_label": "addcnlem", "source": "expanded", "tokens": 10706, "reason": "max_length"}
+{"label": "bastop", "expanded_label": "tgcl", "source": "expanded", "tokens": 9261, "reason": "max_length"}
+{"label": "frgrncvvdeqlem10", "expanded_label": "frgrncvvdeqlem9", "source": "expanded", "tokens": 9599, "reason": "max_length"}
+{"label": "upgrwlkupwlkb", "expanded_label": "upgrwlkupwlk", "source": "expanded", "tokens": 12711, "reason": "max_length"}
+{"label": "gexdvds2", "expanded_label": "oddvds", "source": "expanded", "tokens": 8924, "reason": "max_length"}
+{"label": "mat1ov", "expanded_label": "mat1", "source": "expanded", "tokens": 7234, "reason": "max_length"}
+{"label": "pgjsgr", "expanded_label": "gpgusgra", "source": "expanded", "tokens": 8100, "reason": "max_length"}
+{"label": "m2cpmghm", "expanded_label": "mat2pmatghm", "source": "expanded", "tokens": 14236, "reason": "max_length"}
+{"label": "cnconst", "expanded_label": "cnconst2", "source": "expanded", "tokens": 7114, "reason": "max_length"}
+{"label": "4re", "expanded_label": "1re", "source": "expanded", "tokens": 6779, "reason": "max_length"}
+{"label": "hausflf", "expanded_label": "hausflimi", "source": "expanded", "tokens": 6580, "reason": "max_length"}
+{"label": "gcdmodi", "expanded_label": "modgcd", "source": "expanded", "tokens": 7405, "reason": "max_length"}
+{"label": "xmetutop", "expanded_label": "psmetutop", "source": "expanded", "tokens": 14660, "reason": "max_length"}
+{"label": "issdrg2", "expanded_label": "issubdrg", "source": "expanded", "tokens": 12046, "reason": "max_length"}
+{"label": "uspgrsprf1o", "expanded_label": "uspgrsprfo", "source": "expanded", "tokens": 9662, "reason": "max_length"}
+{"label": "subrgsubrng", "expanded_label": "ringrng", "source": "expanded", "tokens": 6980, "reason": "max_length"}
+{"label": "symgextf1o", "expanded_label": "symgextf1", "source": "expanded", "tokens": 8019, "reason": "max_length"}
+{"label": "mulgp1", "expanded_label": "mulgdir", "source": "expanded", "tokens": 11090, "reason": "max_length"}
+{"label": "nbusgrvtx", "expanded_label": "nbumgrvtx", "source": "expanded", "tokens": 6729, "reason": "max_length"}
+{"label": "rlimcn1b", "expanded_label": "rlimcn1", "source": "expanded", "tokens": 7000, "reason": "max_length"}
+{"label": "subid", "expanded_label": "addrid", "source": "expanded", "tokens": 10979, "reason": "max_length"}
+{"label": "rngridlmcl", "expanded_label": "opprrng", "source": "expanded", "tokens": 7855, "reason": "max_length"}
+{"label": "fusgr1th", "expanded_label": "finsumvtxdg2size", "source": "expanded", "tokens": 15434, "reason": "max_length"}
+{"label": "cnmptc", "expanded_label": "cnconst2", "source": "expanded", "tokens": 6700, "reason": "max_length"}
+{"label": "divalgmodcl", "expanded_label": "divalgmod", "source": "expanded", "tokens": 7672, "reason": "max_length"}
+{"label": "cpmatsrgpmat", "expanded_label": "cpmatmcl", "source": "expanded", "tokens": 7671, "reason": "max_length"}
+{"label": "6re", "expanded_label": "1re", "source": "expanded", "tokens": 6977, "reason": "max_length"}
+{"label": "pn0sr", "expanded_label": "1idsr", "source": "expanded", "tokens": 6286, "reason": "max_length"}
+{"label": "mat1bas", "expanded_label": "matring", "source": "expanded", "tokens": 15980, "reason": "max_length"}
+{"label": "grlicer", "expanded_label": "grlictr", "source": "expanded", "tokens": 11391, "reason": "max_length"}
+{"label": "wlk1ewlk", "expanded_label": "wlk1walk", "source": "expanded", "tokens": 20611, "reason": "max_length"}
+{"label": "gexdvds2", "expanded_label": "gexdvds", "source": "expanded", "tokens": 17221, "reason": "max_length"}
+{"label": "expcncf", "expanded_label": "expcn", "source": "expanded", "tokens": 6499, "reason": "max_length"}
+{"label": "reexpcl", "expanded_label": "1re", "source": "expanded", "tokens": 6871, "reason": "max_length"}
+{"label": "erclwwlk", "expanded_label": "erclwwlktr", "source": "expanded", "tokens": 10470, "reason": "max_length"}
+{"label": "expp1z", "expanded_label": "expaddz", "source": "expanded", "tokens": 11654, "reason": "max_length"}
+{"label": "1wlkd", "expanded_label": "1wlkdlem4", "source": "expanded", "tokens": 7723, "reason": "max_length"}
+{"label": "symgfixf1o", "expanded_label": "symgfixf1", "source": "expanded", "tokens": 8441, "reason": "max_length"}
+{"label": "htpycn", "expanded_label": "ishtpy", "source": "expanded", "tokens": 8322, "reason": "max_length"}
+{"label": "ringccatALTV", "expanded_label": "ringccatidALTV", "source": "expanded", "tokens": 36133, "reason": "max_length"}
+{"label": "dfnbgrss2", "expanded_label": "dfnbgr6", "source": "expanded", "tokens": 7749, "reason": "max_length"}
+{"label": "locfintop", "expanded_label": "islocfin", "source": "expanded", "tokens": 7464, "reason": "max_length"}
+{"label": "rngcidALTV", "expanded_label": "rngccatidALTV", "source": "expanded", "tokens": 31264, "reason": "max_length"}
+{"label": "unitabl", "expanded_label": "unitgrp", "source": "expanded", "tokens": 19083, "reason": "max_length"}
+{"label": "kqt0", "expanded_label": "kqt0lem", "source": "expanded", "tokens": 9261, "reason": "max_length"}
+{"label": "recmet", "expanded_label": "recld2", "source": "expanded", "tokens": 8047, "reason": "max_length"}
+{"label": "gagrp", "expanded_label": "isga", "source": "expanded", "tokens": 9295, "reason": "max_length"}
+{"label": "rngccatALTV", "expanded_label": "rngccatidALTV", "source": "expanded", "tokens": 35837, "reason": "max_length"}
+{"label": "mpl1", "expanded_label": "mplsubrg", "source": "expanded", "tokens": 7919, "reason": "max_length"}
+{"label": "vtxduhgr0edgnel", "expanded_label": "vtxd0nedgb", "source": "expanded", "tokens": 7459, "reason": "max_length"}
+{"label": "pmtrfmvdn0", "expanded_label": "pmtrfrn", "source": "expanded", "tokens": 8866, "reason": "max_length"}
+{"label": "psrbagev2", "expanded_label": "psrbagev1", "source": "expanded", "tokens": 7226, "reason": "max_length"}
+{"label": "rnghmghm", "expanded_label": "isrnghm", "source": "expanded", "tokens": 8318, "reason": "max_length"}
+{"label": "erclwwlk", "expanded_label": "erclwwlksym", "source": "expanded", "tokens": 6259, "reason": "max_length"}
+{"label": "numclwwlk2lem3", "expanded_label": "numclwlk2lem2f1o", "source": "expanded", "tokens": 16930, "reason": "max_length"}
+{"label": "wwlksnonfi", "expanded_label": "iswwlksnon", "source": "expanded", "tokens": 7618, "reason": "max_length"}
+{"label": "erclwwlkn", "expanded_label": "erclwwlkntr", "source": "expanded", "tokens": 13187, "reason": "max_length"}
+{"label": "gcdn0cl", "expanded_label": "gcdcllem3", "source": "expanded", "tokens": 15171, "reason": "max_length"}
+{"label": "symggen2", "expanded_label": "symggen", "source": "expanded", "tokens": 40700, "reason": "max_length"}
+{"label": "mdet0f1o", "expanded_label": "mdet0pr", "source": "expanded", "tokens": 9891, "reason": "max_length"}
+{"label": "ringgrp", "expanded_label": "isring", "source": "expanded", "tokens": 8062, "reason": "max_length"}
+{"label": "mulginvinv", "expanded_label": "mulginvcom", "source": "expanded", "tokens": 10250, "reason": "max_length"}
+{"label": "2zrng0", "expanded_label": "cncrng", "source": "expanded", "tokens": 8315, "reason": "max_length"}
+{"label": "wlkv", "expanded_label": "wksfval", "source": "expanded", "tokens": 7814, "reason": "max_length"}
+{"label": "isncvsngpd", "expanded_label": "isncvsngp", "source": "expanded", "tokens": 7861, "reason": "max_length"}
+{"label": "rlimcn2", "expanded_label": "rlimcn3", "source": "expanded", "tokens": 11998, "reason": "max_length"}
+{"label": "ghmco", "expanded_label": "mhmco", "source": "expanded", "tokens": 8559, "reason": "max_length"}
+{"label": "qdensere2", "expanded_label": "tgioo", "source": "expanded", "tokens": 20132, "reason": "max_length"}
+{"label": "vdgn1frgrv3", "expanded_label": "vdgn1frgrv2", "source": "expanded", "tokens": 8714, "reason": "max_length"}
+{"label": "0le1", "expanded_label": "1re", "source": "expanded", "tokens": 6633, "reason": "max_length"}
+{"label": "wrdlen2", "expanded_label": "wrdlen2i", "source": "expanded", "tokens": 7494, "reason": "max_length"}
+{"label": "wlkiswwlkupgr", "expanded_label": "wlkiswwlks1", "source": "expanded", "tokens": 9327, "reason": "max_length"}
+{"label": "nzrpropd", "expanded_label": "ringpropd", "source": "expanded", "tokens": 19645, "reason": "max_length"}
+{"label": "unitlinv", "expanded_label": "unitgrp", "source": "expanded", "tokens": 13837, "reason": "max_length"}
+{"label": "fmfil", "expanded_label": "fbasrn", "source": "expanded", "tokens": 14221, "reason": "max_length"}
+{"label": "odhash2", "expanded_label": "odf1o2", "source": "expanded", "tokens": 12218, "reason": "max_length"}
+{"label": "xmulmnf2", "expanded_label": "xmulcom", "source": "expanded", "tokens": 8195, "reason": "max_length"}
+{"label": "2arymaptf1o", "expanded_label": "2arymaptfo", "source": "expanded", "tokens": 8887, "reason": "max_length"}
+{"label": "ply1plusgpropd", "expanded_label": "psrplusgpropd", "source": "expanded", "tokens": 9265, "reason": "max_length"}
+{"label": "1le2", "expanded_label": "1re", "source": "expanded", "tokens": 6748, "reason": "max_length"}
+{"label": "uspgrsprf1o", "expanded_label": "uspgrsprf1", "source": "expanded", "tokens": 8507, "reason": "max_length"}
+{"label": "numclwwlk1lem2f1o", "expanded_label": "numclwwlk1lem2fo", "source": "expanded", "tokens": 15086, "reason": "max_length"}
+{"label": "unben", "expanded_label": "unbenlem", "source": "expanded", "tokens": 13691, "reason": "max_length"}
+{"label": "frlmelbas", "expanded_label": "frlmbas", "source": "expanded", "tokens": 9104, "reason": "max_length"}
+{"label": "1arith2", "expanded_label": "1arith", "source": "expanded", "tokens": 27416, "reason": "max_length"}
+{"label": "1arymaptf1o", "expanded_label": "1arymaptf1", "source": "expanded", "tokens": 7210, "reason": "max_length"}
+{"label": "sgrpssmgm", "expanded_label": "mgmnsgrpex", "source": "expanded", "tokens": 7533, "reason": "max_length"}
+{"label": "pj1eq", "expanded_label": "pj1id", "source": "expanded", "tokens": 10282, "reason": "max_length"}
+{"label": "isghmd", "expanded_label": "isghm", "source": "expanded", "tokens": 7692, "reason": "max_length"}
+{"label": "ghmima", "expanded_label": "resghm", "source": "expanded", "tokens": 7045, "reason": "max_length"}
+{"label": "znle", "expanded_label": "znval", "source": "expanded", "tokens": 8606, "reason": "max_length"}
+{"label": "clnbfiusgrfi", "expanded_label": "fusgrfis", "source": "expanded", "tokens": 7030, "reason": "max_length"}
+{"label": "assalmod", "expanded_label": "isassa", "source": "expanded", "tokens": 6614, "reason": "max_length"}
+{"label": "xrs1cmn", "expanded_label": "xaddcom", "source": "expanded", "tokens": 6572, "reason": "max_length"}
+{"label": "pcelnn", "expanded_label": "pcdvdsb", "source": "expanded", "tokens": 10977, "reason": "max_length"}
+{"label": "indistpsALT", "expanded_label": "indistopon", "source": "expanded", "tokens": 7112, "reason": "max_length"}
+{"label": "infssuzle", "expanded_label": "uzwo", "source": "expanded", "tokens": 9520, "reason": "max_length"}
+{"label": "cpmatsrgpmat", "expanded_label": "1elcpmat", "source": "expanded", "tokens": 8164, "reason": "max_length"}
+{"label": "scmatsrng", "expanded_label": "scmatmulcl", "source": "expanded", "tokens": 9402, "reason": "max_length"}
+{"label": "oppgmndb", "expanded_label": "oppgmnd", "source": "expanded", "tokens": 7363, "reason": "max_length"}
+{"label": "2zrngaabl", "expanded_label": "2zrngagrp", "source": "expanded", "tokens": 6148, "reason": "max_length"}
+{"label": "lmodvaddsub4", "expanded_label": "abladdsub4", "source": "expanded", "tokens": 6739, "reason": "max_length"}
+{"label": "m1modnnsub1", "expanded_label": "1re", "source": "expanded", "tokens": 7710, "reason": "max_length"}
+{"label": "haushmph", "expanded_label": "cnhaus", "source": "expanded", "tokens": 13415, "reason": "max_length"}
+{"label": "cncms", "expanded_label": "cncmet", "source": "expanded", "tokens": 9751, "reason": "max_length"}
+{"label": "fprodxp", "expanded_label": "fprod2d", "source": "expanded", "tokens": 7010, "reason": "max_length"}
+{"label": "fsummulc1", "expanded_label": "fsummulc2", "source": "expanded", "tokens": 10995, "reason": "max_length"}
+{"label": "istop2g", "expanded_label": "fiint", "source": "expanded", "tokens": 17441, "reason": "max_length"}
+{"label": "opsrlmod", "expanded_label": "psrlmod", "source": "expanded", "tokens": 21192, "reason": "max_length"}
+{"label": "psradd", "expanded_label": "psrplusg", "source": "expanded", "tokens": 7929, "reason": "max_length"}
+{"label": "phtpyhtpy", "expanded_label": "isphtpy", "source": "expanded", "tokens": 6188, "reason": "max_length"}
+{"label": "refld", "expanded_label": "cncrng", "source": "expanded", "tokens": 8397, "reason": "max_length"}
+{"label": "kgenuni", "expanded_label": "kgentopon", "source": "expanded", "tokens": 9648, "reason": "max_length"}
+{"label": "clsdif", "expanded_label": "clsval2", "source": "expanded", "tokens": 9494, "reason": "max_length"}
+{"label": "pm2mpgrpiso", "expanded_label": "pm2mpghm", "source": "expanded", "tokens": 23647, "reason": "max_length"}
+{"label": "psrasclcl", "expanded_label": "psrlmod", "source": "expanded", "tokens": 18726, "reason": "max_length"}
+{"label": "hashge2el2difb", "expanded_label": "hashge2el2dif", "source": "expanded", "tokens": 8135, "reason": "max_length"}
+{"label": "relin01", "expanded_label": "1re", "source": "expanded", "tokens": 7726, "reason": "max_length"}
+{"label": "nqerid", "expanded_label": "nqerf", "source": "expanded", "tokens": 7228, "reason": "max_length"}
+{"label": "dsmmlmod", "expanded_label": "dsmmlss", "source": "expanded", "tokens": 11428, "reason": "max_length"}
+{"label": "rhmisrnghm", "expanded_label": "ringrng", "source": "expanded", "tokens": 6625, "reason": "max_length"}
+{"label": "orbstaval", "expanded_label": "gastacl", "source": "expanded", "tokens": 9593, "reason": "max_length"}
+{"label": "mat0dim0", "expanded_label": "matring", "source": "expanded", "tokens": 16862, "reason": "max_length"}
+{"label": "4pos", "expanded_label": "1re", "source": "expanded", "tokens": 6841, "reason": "max_length"}
+{"label": "uspgredgleord", "expanded_label": "uspgredg2v", "source": "expanded", "tokens": 6377, "reason": "max_length"}
+{"label": "restt1", "expanded_label": "cnt1", "source": "expanded", "tokens": 6228, "reason": "max_length"}
+{"label": "metres2", "expanded_label": "xmetres2", "source": "expanded", "tokens": 6309, "reason": "max_length"}
+{"label": "2pos", "expanded_label": "1re", "source": "expanded", "tokens": 8970, "reason": "max_length"}
+{"label": "uzfbas", "expanded_label": "uzrest", "source": "expanded", "tokens": 8616, "reason": "max_length"}
+{"label": "grpid", "expanded_label": "grprcan", "source": "expanded", "tokens": 10928, "reason": "max_length"}
+{"label": "wwlksnextbij0", "expanded_label": "wwlksnextinj", "source": "expanded", "tokens": 17544, "reason": "max_length"}
+{"label": "0le2", "expanded_label": "1re", "source": "expanded", "tokens": 9037, "reason": "max_length"}
+{"label": "pmatring", "expanded_label": "matring", "source": "expanded", "tokens": 15581, "reason": "max_length"}
+{"label": "elicc01", "expanded_label": "1re", "source": "expanded", "tokens": 6735, "reason": "max_length"}
+{"label": "gpgprismgr4cycl0", "expanded_label": "gpgprismgr4cycllem11", "source": "expanded", "tokens": 6170, "reason": "max_length"}
+{"label": "wlkiswwlks", "expanded_label": "wlkiswwlks1", "source": "expanded", "tokens": 9332, "reason": "max_length"}
+{"label": "orbstaval", "expanded_label": "eqger", "source": "expanded", "tokens": 16573, "reason": "max_length"}
+{"label": "nlmngp", "expanded_label": "isnlm", "source": "expanded", "tokens": 6526, "reason": "max_length"}
+{"label": "cncdrg", "expanded_label": "cnsubrg", "source": "expanded", "tokens": 11658, "reason": "max_length"}
+{"label": "txbasex", "expanded_label": "txuni2", "source": "expanded", "tokens": 6234, "reason": "max_length"}
+{"label": "neggcd", "expanded_label": "gcdneg", "source": "expanded", "tokens": 6270, "reason": "max_length"}
+{"label": "0cnALT2", "expanded_label": "cnegex", "source": "expanded", "tokens": 8302, "reason": "max_length"}
+{"label": "xrrest", "expanded_label": "xrtgioo", "source": "expanded", "tokens": 12124, "reason": "max_length"}
+{"label": "evls1scafv", "expanded_label": "evls1sca", "source": "expanded", "tokens": 9522, "reason": "max_length"}
+{"label": "assaring", "expanded_label": "isassa", "source": "expanded", "tokens": 6337, "reason": "max_length"}
+{"label": "ramtub", "expanded_label": "ramcl2lem", "source": "expanded", "tokens": 6234, "reason": "max_length"}
+{"label": "gaset", "expanded_label": "isga", "source": "expanded", "tokens": 8960, "reason": "max_length"}
+{"label": "ringmgp", "expanded_label": "isring", "source": "expanded", "tokens": 8047, "reason": "max_length"}
+{"label": "fusgrvtxdgonume", "expanded_label": "vtxdgoddnumeven", "source": "expanded", "tokens": 8717, "reason": "max_length"}
+{"label": "1lt6", "expanded_label": "1re", "source": "expanded", "tokens": 6929, "reason": "max_length"}
+{"label": "rngqiprngho", "expanded_label": "rngqiprnglin", "source": "expanded", "tokens": 9300, "reason": "max_length"}
+{"label": "addnqf", "expanded_label": "nqerf", "source": "expanded", "tokens": 7503, "reason": "max_length"}
+{"label": "1lt10", "expanded_label": "1re", "source": "expanded", "tokens": 6987, "reason": "max_length"}
+{"label": "qtopconn", "expanded_label": "cnconn", "source": "expanded", "tokens": 9265, "reason": "max_length"}
+{"label": "uvtx2vtx1edgb", "expanded_label": "nbuhgr2vtx1edgb", "source": "expanded", "tokens": 9405, "reason": "max_length"}
+{"label": "rngqiprng", "expanded_label": "ringrng", "source": "expanded", "tokens": 6666, "reason": "max_length"}
+{"label": "psgnfitr", "expanded_label": "symggrp", "source": "expanded", "tokens": 6239, "reason": "max_length"}
+{"label": "resttop", "expanded_label": "tgrest", "source": "expanded", "tokens": 9629, "reason": "max_length"}
+{"label": "frgpgrp", "expanded_label": "frgp0", "source": "expanded", "tokens": 17274, "reason": "max_length"}
+{"label": "sqrt2irr0", "expanded_label": "sqrt2irr", "source": "expanded", "tokens": 11416, "reason": "max_length"}
+{"label": "frgr2wsp1", "expanded_label": "wpthswwlks2on", "source": "expanded", "tokens": 10945, "reason": "max_length"}
+{"label": "metelcls", "expanded_label": "met1stc", "source": "expanded", "tokens": 13311, "reason": "max_length"}
+{"label": "crhmsubcALTV", "expanded_label": "srhmsubcALTV", "source": "expanded", "tokens": 12910, "reason": "max_length"}
+{"label": "sgrp2nmnd", "expanded_label": "sgrp2nmndlem4", "source": "expanded", "tokens": 17737, "reason": "max_length"}
+{"label": "phlsrng", "expanded_label": "isphl", "source": "expanded", "tokens": 10453, "reason": "max_length"}
+{"label": "subcn", "expanded_label": "subcn2", "source": "expanded", "tokens": 7032, "reason": "max_length"}
+{"label": "utoptopon", "expanded_label": "utoptop", "source": "expanded", "tokens": 10837, "reason": "max_length"}
+{"label": "metcnp4", "expanded_label": "met1stc", "source": "expanded", "tokens": 13317, "reason": "max_length"}
+{"label": "rhmpsrlem1", "expanded_label": "psrbaglefi", "source": "expanded", "tokens": 8707, "reason": "max_length"}
+{"label": "fcfelbas", "expanded_label": "fcfval", "source": "expanded", "tokens": 6815, "reason": "max_length"}
+{"label": "opprneg", "expanded_label": "grpinvfval", "source": "expanded", "tokens": 7126, "reason": "max_length"}
+{"label": "isgrpde", "expanded_label": "ismndd", "source": "expanded", "tokens": 6733, "reason": "max_length"}
+{"label": "1lt2", "expanded_label": "1re", "source": "expanded", "tokens": 6790, "reason": "max_length"}
+{"label": "hausnlly", "expanded_label": "restnlly", "source": "expanded", "tokens": 7658, "reason": "max_length"}
+{"label": "dvdsunit", "expanded_label": "dvdsrtr", "source": "expanded", "tokens": 6678, "reason": "max_length"}
+{"label": "nbfiusgrfi", "expanded_label": "fusgrfis", "source": "expanded", "tokens": 7133, "reason": "max_length"}
+{"label": "opsrring", "expanded_label": "psrring", "source": "expanded", "tokens": 8414, "reason": "max_length"}
+{"label": "dprd0", "expanded_label": "dprdz", "source": "expanded", "tokens": 14331, "reason": "max_length"}
+{"label": "recmet", "expanded_label": "cncmet", "source": "expanded", "tokens": 9197, "reason": "max_length"}
+{"label": "dvdseq", "expanded_label": "dvdsabseq", "source": "expanded", "tokens": 6996, "reason": "max_length"}
+{"label": "telgsumfz0s", "expanded_label": "telgsumfzs", "source": "expanded", "tokens": 12061, "reason": "max_length"}
+{"label": "evls1fvcl", "expanded_label": "ressply1evl", "source": "expanded", "tokens": 7459, "reason": "max_length"}
+{"label": "t0hmph", "expanded_label": "cnt0", "source": "expanded", "tokens": 8061, "reason": "max_length"}
+{"label": "1le3", "expanded_label": "1re", "source": "expanded", "tokens": 6650, "reason": "max_length"}
+{"label": "zring0", "expanded_label": "cncrng", "source": "expanded", "tokens": 7883, "reason": "max_length"}
+{"label": "ackval42a", "expanded_label": "ackval42", "source": "expanded", "tokens": 6435, "reason": "max_length"}
+{"label": "mplsca", "expanded_label": "psrsca", "source": "expanded", "tokens": 6278, "reason": "max_length"}
+{"label": "subid1", "expanded_label": "addrid", "source": "expanded", "tokens": 10958, "reason": "max_length"}
+{"label": "idmatidpmat", "expanded_label": "mat2pmat1", "source": "expanded", "tokens": 7186, "reason": "max_length"}
+{"label": "vtxduhgrun", "expanded_label": "vtxdun", "source": "expanded", "tokens": 13019, "reason": "max_length"}
+{"label": "nnwo", "expanded_label": "uzwo", "source": "expanded", "tokens": 8873, "reason": "max_length"}
+{"label": "peano2rem", "expanded_label": "1re", "source": "expanded", "tokens": 6713, "reason": "max_length"}
+{"label": "usgrlimprop", "expanded_label": "uspgrlim", "source": "expanded", "tokens": 16778, "reason": "max_length"}
+{"label": "vdgfrgrgt2", "expanded_label": "vdgn1frgrv2", "source": "expanded", "tokens": 9428, "reason": "max_length"}
+{"label": "evls1varsrng", "expanded_label": "evls1var", "source": "expanded", "tokens": 8899, "reason": "max_length"}
+{"label": "indf", "expanded_label": "1re", "source": "expanded", "tokens": 7405, "reason": "max_length"}
+{"label": "usgrn2cycl", "expanded_label": "uspgrn2crct", "source": "expanded", "tokens": 12349, "reason": "max_length"}
+{"label": "addgtge0", "expanded_label": "00id", "source": "expanded", "tokens": 8117, "reason": "max_length"}
+{"label": "zringcrng", "expanded_label": "cncrng", "source": "expanded", "tokens": 7480, "reason": "max_length"}
+{"label": "restt0", "expanded_label": "cnt0", "source": "expanded", "tokens": 10949, "reason": "max_length"}
+{"label": "ring1ne0", "expanded_label": "hashgt12el", "source": "expanded", "tokens": 6637, "reason": "max_length"}
+{"label": "cycsubggenodd", "expanded_label": "dfod2", "source": "expanded", "tokens": 22577, "reason": "max_length"}
+{"label": "0mat2pmat", "expanded_label": "mat2pmatghm", "source": "expanded", "tokens": 15693, "reason": "max_length"}
+{"label": "omndmnd", "expanded_label": "isomnd", "source": "expanded", "tokens": 8458, "reason": "max_length"}
+{"label": "nneop", "expanded_label": "nneo", "source": "expanded", "tokens": 6454, "reason": "max_length"}
+{"label": "eqg0subgecsn", "expanded_label": "eqg0subg", "source": "expanded", "tokens": 6571, "reason": "max_length"}
+{"label": "xltmul2", "expanded_label": "xmulcom", "source": "expanded", "tokens": 14328, "reason": "max_length"}
+{"label": "subrgnrg", "expanded_label": "subgngp", "source": "expanded", "tokens": 7882, "reason": "max_length"}
+{"label": "pi1buni", "expanded_label": "pi1blem", "source": "expanded", "tokens": 7231, "reason": "max_length"}
+{"label": "isoddgcd1", "expanded_label": "coprm", "source": "expanded", "tokens": 7107, "reason": "max_length"}
+{"label": "1lt5", "expanded_label": "1re", "source": "expanded", "tokens": 6962, "reason": "max_length"}
+{"label": "rellycmp", "expanded_label": "cnllycmp", "source": "expanded", "tokens": 22702, "reason": "max_length"}
+{"label": "pm2mprhm", "expanded_label": "matring", "source": "expanded", "tokens": 16674, "reason": "max_length"}
+{"label": "icccld", "expanded_label": "difreicc", "source": "expanded", "tokens": 10085, "reason": "max_length"}
+{"label": "m2cpmrhm", "expanded_label": "matring", "source": "expanded", "tokens": 17131, "reason": "max_length"}
+{"label": "coe1tmfv2", "expanded_label": "coe1tm", "source": "expanded", "tokens": 12134, "reason": "max_length"}
+{"label": "tsmscl", "expanded_label": "eltsms", "source": "expanded", "tokens": 8460, "reason": "max_length"}
+{"label": "eluzadd", "expanded_label": "zaddcl", "source": "expanded", "tokens": 6242, "reason": "max_length"}
+{"label": "sst0", "expanded_label": "cnt0", "source": "expanded", "tokens": 8434, "reason": "max_length"}
+{"label": "clwlkclwwlkf1o", "expanded_label": "clwlkclwwlkf1", "source": "expanded", "tokens": 11644, "reason": "max_length"}
+{"label": "indistop", "expanded_label": "indistopon", "source": "expanded", "tokens": 7204, "reason": "max_length"}
+{"label": "cphnmfval", "expanded_label": "iscph", "source": "expanded", "tokens": 8389, "reason": "max_length"}
+{"label": "fclssscls", "expanded_label": "isfcls", "source": "expanded", "tokens": 7916, "reason": "max_length"}
+{"label": "2zrng", "expanded_label": "2zlidl", "source": "expanded", "tokens": 9569, "reason": "max_length"}
+{"label": "sringcat", "expanded_label": "srhmsubc", "source": "expanded", "tokens": 12711, "reason": "max_length"}
+{"label": "xkotopon", "expanded_label": "xkouni", "source": "expanded", "tokens": 7374, "reason": "max_length"}
+{"label": "abs2dif2", "expanded_label": "abstri", "source": "expanded", "tokens": 7522, "reason": "max_length"}
+{"label": "nrmreg", "expanded_label": "nrmr0reg", "source": "expanded", "tokens": 6220, "reason": "max_length"}
+{"label": "evlsrhm", "expanded_label": "evlsval2", "source": "expanded", "tokens": 8301, "reason": "max_length"}
+{"label": "clwwlkf1o", "expanded_label": "clwwlkf1", "source": "expanded", "tokens": 16313, "reason": "max_length"}
+{"label": "mat2pmatrhm", "expanded_label": "matring", "source": "expanded", "tokens": 26359, "reason": "max_length"}
+{"label": "sqrt1", "expanded_label": "1re", "source": "expanded", "tokens": 7104, "reason": "max_length"}
+{"label": "sumhash", "expanded_label": "ssfi", "source": "expanded", "tokens": 12417, "reason": "max_length"}
+{"label": "pmatcollpw3", "expanded_label": "pmatcollpw", "source": "expanded", "tokens": 12192, "reason": "max_length"}
+{"label": "pcprecl", "expanded_label": "pclem", "source": "expanded", "tokens": 9618, "reason": "max_length"}
+{"label": "1ne2", "expanded_label": "1re", "source": "expanded", "tokens": 6611, "reason": "max_length"}
+{"label": "wlknwwlksnen", "expanded_label": "wlknwwlksnbij", "source": "expanded", "tokens": 7322, "reason": "max_length"}
+{"label": "pm2mpf", "expanded_label": "pm2mpcl", "source": "expanded", "tokens": 7149, "reason": "max_length"}
+{"label": "m2cpmf1", "expanded_label": "mat2pmatf1", "source": "expanded", "tokens": 7912, "reason": "max_length"}
+{"label": "lincsumscmcl", "expanded_label": "lincscmcl", "source": "expanded", "tokens": 12114, "reason": "max_length"}
+{"label": "obsrcl", "expanded_label": "isobs", "source": "expanded", "tokens": 7261, "reason": "max_length"}
+{"label": "evls1pw", "expanded_label": "evls1rhm", "source": "expanded", "tokens": 6659, "reason": "max_length"}
+{"label": "lcmfunsn", "expanded_label": "lcmfunsnlem", "source": "expanded", "tokens": 10953, "reason": "max_length"}
+{"label": "zrzeroorngc", "expanded_label": "zrinitorngc", "source": "expanded", "tokens": 9895, "reason": "max_length"}
+{"label": "rrxmetfi", "expanded_label": "rrxmet", "source": "expanded", "tokens": 47140, "reason": "max_length"}
+{"label": "iicmp", "expanded_label": "1re", "source": "expanded", "tokens": 7086, "reason": "max_length"}
+{"label": "rngabl", "expanded_label": "isrng", "source": "expanded", "tokens": 7974, "reason": "max_length"}
+{"label": "2re", "expanded_label": "1re", "source": "expanded", "tokens": 8834, "reason": "max_length"}
+{"label": "fprodshft", "expanded_label": "mptfzshft", "source": "expanded", "tokens": 7323, "reason": "max_length"}
+{"label": "lmhmlem", "expanded_label": "islmhm", "source": "expanded", "tokens": 7650, "reason": "max_length"}
+{"label": "cncongr", "expanded_label": "cncongr2", "source": "expanded", "tokens": 14989, "reason": "max_length"}
+{"label": "nnssre", "expanded_label": "1re", "source": "expanded", "tokens": 6912, "reason": "max_length"}
+{"label": "1elunit", "expanded_label": "1re", "source": "expanded", "tokens": 6868, "reason": "max_length"}
+{"label": "cnfldhaus", "expanded_label": "methaus", "source": "expanded", "tokens": 10873, "reason": "max_length"}
+{"label": "cnpf", "expanded_label": "iscnp2", "source": "expanded", "tokens": 9405, "reason": "max_length"}
+{"label": "sum2id", "expanded_label": "sumeq2ii", "source": "expanded", "tokens": 10306, "reason": "max_length"}
+{"label": "dsmmlmod", "expanded_label": "prdslmodd", "source": "expanded", "tokens": 27808, "reason": "max_length"}
+{"label": "unitinvcl", "expanded_label": "unitgrp", "source": "expanded", "tokens": 12866, "reason": "max_length"}
+{"label": "bcpascm1", "expanded_label": "bcpasc", "source": "expanded", "tokens": 24545, "reason": "max_length"}
+{"label": "2arymaptf1o", "expanded_label": "2arymaptf1", "source": "expanded", "tokens": 9368, "reason": "max_length"}
+{"label": "xmetf", "expanded_label": "isxmet", "source": "expanded", "tokens": 6591, "reason": "max_length"}
+{"label": "coe1tmfv1", "expanded_label": "coe1tm", "source": "expanded", "tokens": 11689, "reason": "max_length"}
+{"label": "rlimdmo1", "expanded_label": "rlimo1", "source": "expanded", "tokens": 7587, "reason": "max_length"}
+{"label": "nlmlmod", "expanded_label": "isnlm", "source": "expanded", "tokens": 6171, "reason": "max_length"}
+{"label": "7re", "expanded_label": "1re", "source": "expanded", "tokens": 6750, "reason": "max_length"}
+{"label": "omndtos", "expanded_label": "isomnd", "source": "expanded", "tokens": 8315, "reason": "max_length"}
+{"label": "scmatf1o", "expanded_label": "scmatf1", "source": "expanded", "tokens": 10246, "reason": "max_length"}
+{"label": "resthaus", "expanded_label": "cnhaus", "source": "expanded", "tokens": 17432, "reason": "max_length"}
+{"label": "grplactf1o", "expanded_label": "grplactcnv", "source": "expanded", "tokens": 6922, "reason": "max_length"}
+{"label": "bitsinv", "expanded_label": "bitsf1ocnv", "source": "expanded", "tokens": 6737, "reason": "max_length"}
+{"label": "cnmptkc", "expanded_label": "xkoccn", "source": "expanded", "tokens": 14863, "reason": "max_length"}
+{"label": "sringcatALTV", "expanded_label": "srhmsubcALTV", "source": "expanded", "tokens": 12861, "reason": "max_length"}
+{"label": "lcmgcdnn", "expanded_label": "lcmgcd", "source": "expanded", "tokens": 7128, "reason": "max_length"}
+{"label": "cncongr", "expanded_label": "cncongr1", "source": "expanded", "tokens": 25677, "reason": "max_length"}
+{"label": "re0g", "expanded_label": "cncrng", "source": "expanded", "tokens": 7900, "reason": "max_length"}
+{"label": "lmodring", "expanded_label": "islmod", "source": "expanded", "tokens": 22570, "reason": "max_length"}
+{"label": "phimul", "expanded_label": "phimullem", "source": "expanded", "tokens": 59881, "reason": "max_length"}
+{"label": "cycsubgcld", "expanded_label": "cycsubgcl", "source": "expanded", "tokens": 9718, "reason": "max_length"}
+{"label": "zndvds0", "expanded_label": "zndvds", "source": "expanded", "tokens": 8442, "reason": "max_length"}
+{"label": "ghmabl", "expanded_label": "ghmgrp", "source": "expanded", "tokens": 6213, "reason": "max_length"}
+{"label": "pm2mpghmlem1", "expanded_label": "matring", "source": "expanded", "tokens": 16380, "reason": "max_length"}
+{"label": "frgrncvvdeqlem10", "expanded_label": "frgrncvvdeqlem8", "source": "expanded", "tokens": 9143, "reason": "max_length"}
+{"label": "pi1xfrgim", "expanded_label": "pi1xfr", "source": "expanded", "tokens": 56581, "reason": "max_length"}
+{"label": "grlicer", "expanded_label": "grlicsym", "source": "expanded", "tokens": 9352, "reason": "max_length"}
+{"label": "o1const", "expanded_label": "rlimo1", "source": "expanded", "tokens": 8521, "reason": "max_length"}
+{"label": "cphssphl", "expanded_label": "cphsscph", "source": "expanded", "tokens": 10434, "reason": "max_length"}
+{"label": "islmhmd", "expanded_label": "islmhm", "source": "expanded", "tokens": 7502, "reason": "max_length"}
+{"label": "fsumsub", "expanded_label": "fsumadd", "source": "expanded", "tokens": 13138, "reason": "max_length"}
+{"label": "kgenftop", "expanded_label": "kgentopon", "source": "expanded", "tokens": 10064, "reason": "max_length"}
+{"label": "clwwlkf1o", "expanded_label": "clwwlkfo", "source": "expanded", "tokens": 7535, "reason": "max_length"}
+{"label": "mdet0fv0", "expanded_label": "mdet0pr", "source": "expanded", "tokens": 9903, "reason": "max_length"}
+{"label": "usgrexmpl", "expanded_label": "usgrexmplef", "source": "expanded", "tokens": 9478, "reason": "max_length"}
+{"label": "numclwwlk1lem2f1o", "expanded_label": "numclwwlk1lem2f1", "source": "expanded", "tokens": 19478, "reason": "max_length"}
+{"label": "odval2", "expanded_label": "odeq", "source": "expanded", "tokens": 6696, "reason": "max_length"}
+{"label": "phiprm", "expanded_label": "phiprmpw", "source": "expanded", "tokens": 17595, "reason": "max_length"}
+{"label": "3re", "expanded_label": "1re", "source": "expanded", "tokens": 6728, "reason": "max_length"}
+{"label": "ghmf", "expanded_label": "isghm", "source": "expanded", "tokens": 7750, "reason": "max_length"}
+{"label": "5re", "expanded_label": "1re", "source": "expanded", "tokens": 6884, "reason": "max_length"}
+{"label": "cnmgpabl", "expanded_label": "cncrng", "source": "expanded", "tokens": 7869, "reason": "max_length"}
+{"label": "ordtrestixx", "expanded_label": "letsr", "source": "expanded", "tokens": 6760, "reason": "max_length"}
+{"label": "filfinnfr", "expanded_label": "fbfinnfr", "source": "expanded", "tokens": 6699, "reason": "max_length"}
+{"label": "vtxdfiun", "expanded_label": "vtxdun", "source": "expanded", "tokens": 13496, "reason": "max_length"}
+{"label": "cnprcl", "expanded_label": "iscnp2", "source": "expanded", "tokens": 9418, "reason": "max_length"}
+{"label": "1arymaptf1o", "expanded_label": "1arymaptfo", "source": "expanded", "tokens": 6423, "reason": "max_length"}
+{"label": "pc1", "expanded_label": "pczpre", "source": "expanded", "tokens": 10054, "reason": "max_length"}
+{"label": "sshaus", "expanded_label": "cnhaus", "source": "expanded", "tokens": 13569, "reason": "max_length"}
+{"label": "8re", "expanded_label": "1re", "source": "expanded", "tokens": 6721, "reason": "max_length"}
+{"label": "nmoleub2a", "expanded_label": "nmoleub2lem2", "source": "expanded", "tokens": 13300, "reason": "max_length"}
+{"label": "nnesq", "expanded_label": "zesq", "source": "expanded", "tokens": 7574, "reason": "max_length"}
+{"label": "vtxdgfusgrf", "expanded_label": "fusgrfis", "source": "expanded", "tokens": 7818, "reason": "max_length"}
+{"label": "cphsca", "expanded_label": "iscph", "source": "expanded", "tokens": 7872, "reason": "max_length"}
+{"label": "pm2mpf1o", "expanded_label": "pm2mpf1", "source": "expanded", "tokens": 20808, "reason": "max_length"}

speed-estimate.md

@@ -0,0 +1,11 @@
+# Qwen3.5-2B-metamath 8192 speed estimate
+
+- Data: metamath-output/setmm-train-qwen35-4b-mixed-12000, 4000 original + 12000 expanded.
+- max_length=8192 keeps about 14492/16000 examples from tokenizer length scan.
+- Training config: Qwen3.5-2B base, FLA fast path available, LoRA rank 32/alpha 64/dropout 0.05, bf16, gradient checkpointing, lr=5e-4, 1 epoch.
+- Batch: per-device train batch size 2, gradient accumulation 8 on one GPU, effective batch size about 16.
+- Smoke run: 59 train examples, 4 optimizer steps, runtime 120.9s. First step was about 94s due to compile/init; later steps were about 8-11s/step on the short smoke sample.
+- Full 1-epoch steps: about 888 optimizer steps after 2% eval split.
+- Estimated full runtime: roughly 4-8 hours depending on length mix, checkpoint/eval cost, and whether the compiled kernels stay warm.
+- Log: /data/pretrained_models/Qwen3.5-2B-metamath/train-8192.log
+- PID file: /data/pretrained_models/Qwen3.5-2B-metamath/train-8192.pid

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:87a7830d63fcf43bf241c3c5242e96e62dd3fdc29224ca26fed8ea333db72de4
+size 19989343

tokenizer_config.json

@@ -0,0 +1,31 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": true,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

train-6144-mb2x8-gpu1.log

@@ -0,0 +1,11 @@
+Starting Qwen3.5-2B Metamath training
+Output: /data/pretrained_models/Qwen3.5-2B-metamath
+Effective batch size: 16
+/data/home/lg/.conda/envs/verl_qwen35/lib/python3.12/site-packages/torch/cuda/__init__.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
+  import pynvml  # type: ignore[import]
+`torch_dtype` is deprecated! Use `dtype` instead!
+
+warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
+The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'eos_token_id': 248046, 'pad_token_id': 248044}.
+trainable params: 21,823,488 || all params: 1,903,648,576 || trainable%: 1.1464
+
  0%|          | 0/955 [00:00<?, ?it/s]
  0%|          | 1/955 [00:08<2:08:48,  8.10s/it]
  0%|          | 2/955 [00:13<1:42:03,  6.43s/it]
  0%|          | 3/955 [00:18<1:33:39,  5.90s/it]
  0%|          | 4/955 [00:24<1:33:10,  5.88s/it]
  1%|          | 5/955 [00:29<1:26:47,  5.48s/it]
  1%|          | 6/955 [00:33<1:22:03,  5.19s/it]
  1%|          | 7/955 [00:39<1:22:53,  5.25s/it]
  1%|          | 8/955 [00:44<1:22:33,  5.23s/it]
  1%|          | 9/955 [00:49<1:22:53,  5.26s/it]
  1%|          | 10/955 [00:55<1:23:03,  5.27s/it]
                                                  
  1%|          | 10/955 [00:55<1:23:03,  5.27s/it]
  1%|          | 11/955 [01:00<1:21:48,  5.20s/it]
  1%|▏         | 12/955 [01:06<1:26:21,  5.49s/it]
  1%|▏         | 13/955 [01:11<1:26:44,  5.52s/it]
  1%|▏         | 14/955 [01:16<1:21:12,  5.18s/it]
  2%|▏         | 15/955 [01:23<1:32:44,  5.92s/it]
  2%|▏         | 16/955 [01:29<1:31:53,  5.87s/it]
  2%|▏         | 17/955 [01:34<1:27:09,  5.57s/it]
  2%|▏         | 18/955 [01:39<1:24:54,  5.44s/it]
  2%|▏         | 19/955 [01:47<1:35:33,  6.13s/it]
  2%|▏         | 20/955 [01:52<1:29:57,  5.77s/it]
                                                  
  2%|▏         | 20/955 [01:52<1:29:57,  5.77s/it]
  2%|▏         | 21/955 [01:58<1:29:40,  5.76s/it]
  2%|▏         | 22/955 [02:02<1:25:43,  5.51s/it]
  2%|▏         | 23/955 [02:10<1:36:31,  6.21s/it]
  3%|▎         | 24/955 [02:16<1:31:56,  5.92s/it]
  3%|▎         | 25/955 [02:22<1:33:55,  6.06s/it]
  3%|▎         | 26/955 [02:27<1:27:44,  5.67s/it]
  3%|▎         | 27/955 [02:32<1:27:30,  5.66s/it]
  3%|▎         | 28/955 [02:40<1:34:33,  6.12s/it]
  3%|▎         | 29/955 [02:48<1:47:23,  6.96s/it]
  3%|▎         | 30/955 [02:57<1:54:11,  7.41s/it]
                                                  
  3%|▎         | 30/955 [02:57<1:54:11,  7.41s/it]
  3%|▎         | 31/955 [03:04<1:51:03,  7.21s/it]
  3%|▎         | 32/955 [03:09<1:42:57,  6.69s/it]
  3%|▎         | 33/955 [03:19<1:56:30,  7.58s/it]
  4%|▎         | 34/955 [03:26<1:53:59,  7.43s/it]
  4%|▎         | 35/955 [03:31<1:44:08,  6.79s/it]
  4%|▍         | 36/955 [03:37<1:39:52,  6.52s/it]
  4%|▍         | 37/955 [03:43<1:36:02,  6.28s/it]
  4%|▍         | 38/955 [03:49<1:34:08,  6.16s/it]
  4%|▍         | 39/955 [03:58<1:47:32,  7.04s/it]
  4%|▍         | 40/955 [04:04<1:45:12,  6.90s/it]
                                                  
  4%|▍         | 40/955 [04:04<1:45:12,  6.90s/it]
  4%|▍         | 41/955 [04:11<1:42:05,  6.70s/it]
  4%|▍         | 42/955 [04:18<1:44:20,  6.86s/it]
  5%|▍         | 43/955 [04:28<1:58:59,  7.83s/it]
  5%|▍         | 44/955 [04:35<1:56:05,  7.65s/it]
  5%|▍         | 45/955 [04:43<1:57:25,  7.74s/it]
  5%|▍         | 46/955 [04:49<1:51:07,  7.33s/it]
  5%|▍         | 47/955 [04:56<1:46:18,  7.02s/it]
  5%|▌         | 48/955 [05:01<1:37:38,  6.46s/it]
  5%|▌         | 49/955 [05:11<1:51:55,  7.41s/it]
  5%|▌         | 50/955 [05:17<1:47:04,  7.10s/it]
                                                  
  5%|▌         | 50/955 [05:17<1:47:04,  7.10s/it]Terminated

train-8192-mb4x4-gpu1.log

@@ -0,0 +1,42 @@
+Starting Qwen3.5-2B Metamath training
+Output: /data/pretrained_models/Qwen3.5-2B-metamath
+Effective batch size: 16
+/data/home/lg/.conda/envs/verl_qwen35/lib/python3.12/site-packages/torch/cuda/__init__.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
+  import pynvml  # type: ignore[import]
+`torch_dtype` is deprecated! Use `dtype` instead!
+
+warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
+The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'eos_token_id': 248046, 'pad_token_id': 248044}.
+trainable params: 21,823,488 || all params: 1,903,648,576 || trainable%: 1.1464
+
  0%|          | 0/966 [00:00<?, ?it/s]
  0%|          | 1/966 [00:25<6:46:25, 25.27s/it]
  0%|          | 2/966 [00:30<3:33:54, 13.31s/it]
  0%|          | 3/966 [00:34<2:30:05,  9.35s/it]Traceback (most recent call last):
+  File "/home/lg/workflow_tooluse/Flow_RL_luogan/temp/metamath/tools/train_qwen35_metamath.py", line 381, in <module>
+    main()
+  File "/home/lg/workflow_tooluse/Flow_RL_luogan/temp/metamath/tools/train_qwen35_metamath.py", line 339, in main
+    trainer.train()
+  File "/data/home/lg/.conda/envs/verl_qwen35/lib/python3.12/site-packages/transformers/trainer.py", line 1424, in train
+    return inner_training_loop(
+           ^^^^^^^^^^^^^^^^^^^^
+  File "/data/home/lg/.conda/envs/verl_qwen35/lib/python3.12/site-packages/transformers/trainer.py", line 1506, in _inner_training_loop
+    self._run_epoch(
+  File "/data/home/lg/.conda/envs/verl_qwen35/lib/python3.12/site-packages/transformers/trainer.py", line 1734, in _run_epoch
+    tr_loss_step = self.training_step(model, inputs, num_items_in_batch)
+                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/data/home/lg/.conda/envs/verl_qwen35/lib/python3.12/site-packages/transformers/trainer.py", line 1934, in training_step
+    self.accelerator.backward(loss, **kwargs)
+  File "/home/lg/.local/lib/python3.12/site-packages/accelerate/accelerator.py", line 2329, in backward
+    loss.backward(**kwargs)
+  File "/data/home/lg/.conda/envs/verl_qwen35/lib/python3.12/site-packages/torch/_tensor.py", line 625, in backward
+    torch.autograd.backward(
+  File "/data/home/lg/.conda/envs/verl_qwen35/lib/python3.12/site-packages/torch/autograd/__init__.py", line 354, in backward
+    _engine_run_backward(
+  File "/data/home/lg/.conda/envs/verl_qwen35/lib/python3.12/site-packages/torch/autograd/graph.py", line 841, in _engine_run_backward
+    return Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
+           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 25.02 GiB. GPU 0 has a total capacity of 79.14 GiB of which 18.98 GiB is free. Including non-PyTorch memory, this process has 60.14 GiB memory in use. Of the allocated memory 56.74 GiB is allocated by PyTorch, and 2.36 GiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
+
  0%|          | 3/966 [00:45<4:00:55, 15.01s/it]
+Exception ignored in: <function ResourceTracker.__del__ at 0x7d8e4c558c20>
+Traceback (most recent call last):
+  File "/home/lg/.local/lib/python3.12/site-packages/multiprocess/resource_tracker.py", line 80, in __del__
+  File "/home/lg/.local/lib/python3.12/site-packages/multiprocess/resource_tracker.py", line 89, in _stop
+  File "/home/lg/.local/lib/python3.12/site-packages/multiprocess/resource_tracker.py", line 102, in _stop_locked
+AttributeError: '_thread.RLock' object has no attribute '_recursion_count'

train-8192.log

@@ -0,0 +1,8 @@
+/data/home/lg/.conda/envs/verl_qwen35/lib/python3.12/site-packages/torch/cuda/__init__.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
+  import pynvml  # type: ignore[import]
+`torch_dtype` is deprecated! Use `dtype` instead!
+
+warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
+The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'eos_token_id': 248046, 'pad_token_id': 248044}.
+trainable params: 21,823,488 || all params: 1,903,648,576 || trainable%: 1.1464
+
  0%|          | 0/888 [00:00<?, ?it/s]
  0%|          | 1/888 [00:11<2:53:39, 11.75s/it]
  0%|          | 2/888 [00:17<1:59:09,  8.07s/it]
  0%|          | 3/888 [00:28<2:21:20,  9.58s/it]
  0%|          | 4/888 [00:37<2:18:40,  9.41s/it]
  1%|          | 5/888 [00:47<2:19:10,  9.46s/it]
  1%|          | 6/888 [00:57<2:23:55,  9.79s/it]
  1%|          | 7/888 [01:04<2:09:23,  8.81s/it]
  1%|          | 8/888 [01:15<2:20:01,  9.55s/it]
  1%|          | 9/888 [01:23<2:10:23,  8.90s/it]
  1%|          | 10/888 [01:31<2:09:24,  8.84s/it]
                                                  
  1%|          | 10/888 [01:31<2:09:24,  8.84s/it]
  1%|          | 11/888 [01:42<2:15:42,  9.28s/it]
  1%|▏         | 12/888 [01:53<2:24:50,  9.92s/it]
  1%|▏         | 13/888 [02:03<2:24:07,  9.88s/it]
  2%|▏         | 14/888 [02:19<2:52:47, 11.86s/it]
  2%|▏         | 15/888 [02:36<3:11:57, 13.19s/it]
  2%|▏         | 16/888 [02:47<3:03:38, 12.64s/it]
  2%|▏         | 17/888 [02:59<3:00:47, 12.45s/it]
  2%|▏         | 18/888 [03:18<3:27:52, 14.34s/it]

train-8192.pid

@@ -0,0 +1 @@
+2564523

train-manifest.json

@@ -0,0 +1,17 @@
+{
+  "base_model": "/data/pretrained_models/Qwen3.5-2B",
+  "original_units": "/home/lg/workflow_tooluse/Flow_RL_luogan/temp/metamath/metamath-output/setmm-train-qwen35-4b-mixed-12000/setmm-proof-units.jsonl",
+  "expanded_units": "/home/lg/workflow_tooluse/Flow_RL_luogan/temp/metamath/metamath-output/setmm-train-qwen35-4b-mixed-12000/setmm-expanded-units.jsonl",
+  "output_dir": "/data/pretrained_models/Qwen3.5-2B-metamath",
+  "merged_dir": "/data/pretrained_models/Qwen3.5-2B-metamath/merged",
+  "train_examples": 15267,
+  "eval_examples": 311,
+  "skipped_examples": 422,
+  "max_length": 6144,
+  "direct_ref_mode": "same-file-distractors",
+  "same_file_distractor_direct_refs": 4,
+  "shuffle_direct_refs": true,
+  "learning_rate": 0.0001,
+  "lora_rank": 32,
+  "lora_alpha": 64
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e8fc737554ff6f82c4ea137b5313611e3b2b3b63fd69b3926d6b1fe9da14c0a6
+size 5201