diff --git a/.gitattributes b/.gitattributes
index a6344aac8c09253b3b630fb776ae94478aa0275b..2ce39ba75d79ffcc9452b163a31d3122985c4a1c 100644
--- a/.gitattributes
+++ b/.gitattributes
@@ -33,3 +33,7 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+checkpoint-1484/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-2226/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-742/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text
diff --git a/README.md b/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..d44d68a82ab48884e6afee364005e8a561c49c7c
--- /dev/null
+++ b/README.md
@@ -0,0 +1,202 @@
+---
+base_model: ModelSpace/GemmaX2-28-2B-v0.1
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.2
\ No newline at end of file
diff --git a/adapter_config.json b/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..191461bdeecd0575bb2844135920827ba4f2c74c
--- /dev/null
+++ b/adapter_config.json
@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "ModelSpace/GemmaX2-28-2B-v0.1",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}
\ No newline at end of file
diff --git a/adapter_model.safetensors b/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..bbea7f39df1ed43106a43966c997ee88d13e609b
--- /dev/null
+++ b/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:591a55ad10ccb0ac75ccca10719e3a31ab1ba1997afb246c2d0f2d2311f34f77
+size 6403448
diff --git a/chat_template.jinja b/chat_template.jinja
new file mode 100644
index 0000000000000000000000000000000000000000..127d9d1acfd2092b5b5d74ea3fe2626658923ada
--- /dev/null
+++ b/chat_template.jinja
@@ -0,0 +1 @@
+{% if messages[0]['role'] == 'system' %}{% set system_message = messages[0]['content'] %}{% endif %}{% if system_message is defined %}{{ system_message }}{% endif %}{% for message in messages %}{% set content = message['content'] %}{% if message['role'] == 'user' %}{{ content }}{% elif message['role'] == 'assistant' %}{{ content }}{% endif %}{% endfor %}
\ No newline at end of file
diff --git a/checkpoint-1484/README.md b/checkpoint-1484/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..d44d68a82ab48884e6afee364005e8a561c49c7c
--- /dev/null
+++ b/checkpoint-1484/README.md
@@ -0,0 +1,202 @@
+---
+base_model: ModelSpace/GemmaX2-28-2B-v0.1
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.2
\ No newline at end of file
diff --git a/checkpoint-1484/adapter_config.json b/checkpoint-1484/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..191461bdeecd0575bb2844135920827ba4f2c74c
--- /dev/null
+++ b/checkpoint-1484/adapter_config.json
@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "ModelSpace/GemmaX2-28-2B-v0.1",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}
\ No newline at end of file
diff --git a/checkpoint-1484/adapter_model.safetensors b/checkpoint-1484/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..08a09d8079299b3323b4e11942e4a5c844f8bc17
--- /dev/null
+++ b/checkpoint-1484/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a3ba6a691342e9054df6f2738d80d17a730ffd757a154b6dcd76f40d4951478f
+size 6403448
diff --git a/checkpoint-1484/chat_template.jinja b/checkpoint-1484/chat_template.jinja
new file mode 100644
index 0000000000000000000000000000000000000000..127d9d1acfd2092b5b5d74ea3fe2626658923ada
--- /dev/null
+++ b/checkpoint-1484/chat_template.jinja
@@ -0,0 +1 @@
+{% if messages[0]['role'] == 'system' %}{% set system_message = messages[0]['content'] %}{% endif %}{% if system_message is defined %}{{ system_message }}{% endif %}{% for message in messages %}{% set content = message['content'] %}{% if message['role'] == 'user' %}{{ content }}{% elif message['role'] == 'assistant' %}{{ content }}{% endif %}{% endfor %}
\ No newline at end of file
diff --git a/checkpoint-1484/optimizer.pt b/checkpoint-1484/optimizer.pt
new file mode 100644
index 0000000000000000000000000000000000000000..ad1e9fb482698a461cef797e8f03a37f77914b65
--- /dev/null
+++ b/checkpoint-1484/optimizer.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d80b36a032b1b39174952db805c8c8707eeb4493929952d63eb2ffb25f8a7cdb
+size 12867066
diff --git a/checkpoint-1484/rng_state.pth b/checkpoint-1484/rng_state.pth
new file mode 100644
index 0000000000000000000000000000000000000000..bb3203309c97b035278c43721a25da1b928ffe96
--- /dev/null
+++ b/checkpoint-1484/rng_state.pth
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b0d5e5fe873cb959a869124d0fde9fdd7607dd885ffd0e40c1204bdd38990e0a
+size 14244
diff --git a/checkpoint-1484/scaler.pt b/checkpoint-1484/scaler.pt
new file mode 100644
index 0000000000000000000000000000000000000000..c24f8a14ffe73dfd927257b617945d057dc590f5
--- /dev/null
+++ b/checkpoint-1484/scaler.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8034dcb9b17f8dbc5fc3d45ee64d1519a5453505777e6b99d18ec4870f4055ed
+size 988
diff --git a/checkpoint-1484/scheduler.pt b/checkpoint-1484/scheduler.pt
new file mode 100644
index 0000000000000000000000000000000000000000..810af6fcd50f9cfeb1a0accd064fcaf67abb0737
--- /dev/null
+++ b/checkpoint-1484/scheduler.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:17adb294b3f8906db57186475801763fe9be72f30360d16f2fcb79d24542560a
+size 1064
diff --git a/checkpoint-1484/special_tokens_map.json b/checkpoint-1484/special_tokens_map.json
new file mode 100644
index 0000000000000000000000000000000000000000..f6119589e367b2de0fc8cbd2f1217667532e3174
--- /dev/null
+++ b/checkpoint-1484/special_tokens_map.json
@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<bos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<eos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}
diff --git a/checkpoint-1484/tokenizer.json b/checkpoint-1484/tokenizer.json
new file mode 100644
index 0000000000000000000000000000000000000000..2b2d51a12a27cc5bd1baeaca3ec01d6f6f458f64
--- /dev/null
+++ b/checkpoint-1484/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:afe09ee4dcd6379dfa9f12c0178f86eab033fc9f97f4e1d605a44da3488b6c53
+size 34363138
diff --git a/checkpoint-1484/tokenizer.model b/checkpoint-1484/tokenizer.model
new file mode 100644
index 0000000000000000000000000000000000000000..796efe9ab515c15e146ce7588e6d7b9b8134dbf8
--- /dev/null
+++ b/checkpoint-1484/tokenizer.model
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61a7b147390c64585d6c3543dd6fc636906c9af3865a5548f27f31aee1d4c8e2
+size 4241003
diff --git a/checkpoint-1484/tokenizer_config.json b/checkpoint-1484/tokenizer_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..62e63dbcee5400f4f34987854723a5120c4cc076
--- /dev/null
+++ b/checkpoint-1484/tokenizer_config.json
@@ -0,0 +1,2011 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<eos>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "<bos>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "4": {
+      "content": "<mask>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "5": {
+      "content": "<2mass>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "6": {
+      "content": "[@BOS@]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "7": {
+      "content": "<unused0>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "8": {
+      "content": "<unused1>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "9": {
+      "content": "<unused2>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "10": {
+      "content": "<unused3>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "11": {
+      "content": "<unused4>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "12": {
+      "content": "<unused5>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "13": {
+      "content": "<unused6>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "14": {
+      "content": "<unused7>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "15": {
+      "content": "<unused8>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "16": {
+      "content": "<unused9>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "17": {
+      "content": "<unused10>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "18": {
+      "content": "<unused11>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "19": {
+      "content": "<unused12>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "20": {
+      "content": "<unused13>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "21": {
+      "content": "<unused14>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "22": {
+      "content": "<unused15>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "23": {
+      "content": "<unused16>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "24": {
+      "content": "<unused17>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "25": {
+      "content": "<unused18>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "26": {
+      "content": "<unused19>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "27": {
+      "content": "<unused20>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "28": {
+      "content": "<unused21>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "29": {
+      "content": "<unused22>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "30": {
+      "content": "<unused23>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "31": {
+      "content": "<unused24>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "32": {
+      "content": "<unused25>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "33": {
+      "content": "<unused26>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "34": {
+      "content": "<unused27>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "35": {
+      "content": "<unused28>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "36": {
+      "content": "<unused29>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "37": {
+      "content": "<unused30>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "38": {
+      "content": "<unused31>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "39": {
+      "content": "<unused32>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "40": {
+      "content": "<unused33>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "41": {
+      "content": "<unused34>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "42": {
+      "content": "<unused35>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "43": {
+      "content": "<unused36>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "44": {
+      "content": "<unused37>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "45": {
+      "content": "<unused38>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "46": {
+      "content": "<unused39>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "47": {
+      "content": "<unused40>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "48": {
+      "content": "<unused41>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "49": {
+      "content": "<unused42>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50": {
+      "content": "<unused43>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "51": {
+      "content": "<unused44>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "52": {
+      "content": "<unused45>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "53": {
+      "content": "<unused46>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "54": {
+      "content": "<unused47>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55": {
+      "content": "<unused48>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "56": {
+      "content": "<unused49>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "57": {
+      "content": "<unused50>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "58": {
+      "content": "<unused51>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "59": {
+      "content": "<unused52>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "60": {
+      "content": "<unused53>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "61": {
+      "content": "<unused54>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "62": {
+      "content": "<unused55>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "63": {
+      "content": "<unused56>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "64": {
+      "content": "<unused57>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "65": {
+      "content": "<unused58>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "66": {
+      "content": "<unused59>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "67": {
+      "content": "<unused60>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "68": {
+      "content": "<unused61>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "69": {
+      "content": "<unused62>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "70": {
+      "content": "<unused63>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "71": {
+      "content": "<unused64>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "72": {
+      "content": "<unused65>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "73": {
+      "content": "<unused66>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "74": {
+      "content": "<unused67>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "75": {
+      "content": "<unused68>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "76": {
+      "content": "<unused69>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "77": {
+      "content": "<unused70>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "78": {
+      "content": "<unused71>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "79": {
+      "content": "<unused72>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "80": {
+      "content": "<unused73>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "81": {
+      "content": "<unused74>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "82": {
+      "content": "<unused75>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "83": {
+      "content": "<unused76>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "84": {
+      "content": "<unused77>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "85": {
+      "content": "<unused78>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "86": {
+      "content": "<unused79>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "87": {
+      "content": "<unused80>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "88": {
+      "content": "<unused81>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "89": {
+      "content": "<unused82>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "90": {
+      "content": "<unused83>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "91": {
+      "content": "<unused84>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "92": {
+      "content": "<unused85>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "93": {
+      "content": "<unused86>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "94": {
+      "content": "<unused87>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95": {
+      "content": "<unused88>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "96": {
+      "content": "<unused89>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "97": {
+      "content": "<unused90>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "98": {
+      "content": "<unused91>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "99": {
+      "content": "<unused92>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "100": {
+      "content": "<unused93>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "101": {
+      "content": "<unused94>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "102": {
+      "content": "<unused95>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "103": {
+      "content": "<unused96>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "104": {
+      "content": "<unused97>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "105": {
+      "content": "<unused98>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "106": {
+      "content": "<start_of_turn>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "107": {
+      "content": "<end_of_turn>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "108": {
+      "content": "\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "109": {
+      "content": "\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "110": {
+      "content": "\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "111": {
+      "content": "\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "112": {
+      "content": "\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "113": {
+      "content": "\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "114": {
+      "content": "\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "115": {
+      "content": "\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "116": {
+      "content": "\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "117": {
+      "content": "\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "118": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "119": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "120": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "121": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "122": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "123": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "124": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "125": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "126": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "127": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "128": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "129": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "130": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "131": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "132": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "133": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "134": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "135": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "136": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "137": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "138": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "139": {
+      "content": "▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "140": {
+      "content": "▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "141": {
+      "content": "▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "142": {
+      "content": "▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "143": {
+      "content": "▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "144": {
+      "content": "▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "145": {
+      "content": "▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "146": {
+      "content": "▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "147": {
+      "content": "▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "148": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "149": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "150": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "152": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "153": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "154": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "155": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "156": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "157": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "158": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "159": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "160": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "161": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "162": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "163": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "164": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "165": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "166": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "167": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "168": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "169": {
+      "content": "<table>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "170": {
+      "content": "<caption>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "171": {
+      "content": "<thead>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "172": {
+      "content": "<tbody>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "173": {
+      "content": "<tfoot>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "174": {
+      "content": "<tr>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "175": {
+      "content": "<th>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "176": {
+      "content": "<td>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "177": {
+      "content": "</table>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "178": {
+      "content": "</caption>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "179": {
+      "content": "</thead>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "180": {
+      "content": "</tbody>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "181": {
+      "content": "</tfoot>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "182": {
+      "content": "</tr>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "183": {
+      "content": "</th>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "184": {
+      "content": "</td>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "185": {
+      "content": "<h1>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "186": {
+      "content": "<h2>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "187": {
+      "content": "<h3>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "188": {
+      "content": "<h4>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "189": {
+      "content": "<h5>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "190": {
+      "content": "<h6>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "191": {
+      "content": "<blockquote>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "192": {
+      "content": "</h1>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "193": {
+      "content": "</h2>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "194": {
+      "content": "</h3>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "195": {
+      "content": "</h4>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "196": {
+      "content": "</h5>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "197": {
+      "content": "</h6>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "198": {
+      "content": "</blockquote>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "199": {
+      "content": "<strong>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "200": {
+      "content": "<em>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "201": {
+      "content": "<b>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "202": {
+      "content": "<i>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "203": {
+      "content": "<u>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "204": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "205": {
+      "content": "<sub>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "206": {
+      "content": "<sup>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "207": {
+      "content": "<code>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "208": {
+      "content": "</strong>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "209": {
+      "content": "</em>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "210": {
+      "content": "</b>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "211": {
+      "content": "</i>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "212": {
+      "content": "</u>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "213": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "214": {
+      "content": "</sub>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "215": {
+      "content": "</sup>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "216": {
+      "content": "</code>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255968": {
+      "content": "[toxicity=0]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255969": {
+      "content": "\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255970": {
+      "content": "\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255971": {
+      "content": "\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255972": {
+      "content": "\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255973": {
+      "content": "\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255974": {
+      "content": "\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255975": {
+      "content": "\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255976": {
+      "content": "\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255977": {
+      "content": "\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255978": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255979": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255980": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255981": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255982": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255983": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255984": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255985": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255986": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255987": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255988": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255989": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255990": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255991": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255992": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255993": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255994": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255995": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255996": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255997": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255998": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255999": {
+      "content": "<unused99>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "bos_token": "<bos>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<eos>",
+  "extra_special_tokens": {},
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<pad>",
+  "padding_side": "right",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "split_special_tokens": false,
+  "tokenizer_class": "GemmaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}
diff --git a/checkpoint-1484/trainer_state.json b/checkpoint-1484/trainer_state.json
new file mode 100644
index 0000000000000000000000000000000000000000..ba7af49a9324d0e26dcc73a5b47f4b398d5c3a9f
--- /dev/null
+++ b/checkpoint-1484/trainer_state.json
@@ -0,0 +1,1070 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.0,
+  "eval_steps": 500,
+  "global_step": 1484,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.013490725126475547,
+      "grad_norm": 65.45332336425781,
+      "learning_rate": 0.00019964061096136568,
+      "loss": 22.1843,
+      "step": 10
+    },
+    {
+      "epoch": 0.026981450252951095,
+      "grad_norm": 6.102770805358887,
+      "learning_rate": 0.00019874213836477988,
+      "loss": 7.6527,
+      "step": 20
+    },
+    {
+      "epoch": 0.04047217537942664,
+      "grad_norm": 2.982379913330078,
+      "learning_rate": 0.00019784366576819408,
+      "loss": 3.4532,
+      "step": 30
+    },
+    {
+      "epoch": 0.05396290050590219,
+      "grad_norm": 3.4628562927246094,
+      "learning_rate": 0.00019694519317160826,
+      "loss": 2.4467,
+      "step": 40
+    },
+    {
+      "epoch": 0.06745362563237774,
+      "grad_norm": 3.0008349418640137,
+      "learning_rate": 0.00019604672057502246,
+      "loss": 1.6481,
+      "step": 50
+    },
+    {
+      "epoch": 0.08094435075885328,
+      "grad_norm": 1.767865777015686,
+      "learning_rate": 0.00019514824797843666,
+      "loss": 1.2119,
+      "step": 60
+    },
+    {
+      "epoch": 0.09443507588532883,
+      "grad_norm": 3.416968584060669,
+      "learning_rate": 0.00019424977538185087,
+      "loss": 0.967,
+      "step": 70
+    },
+    {
+      "epoch": 0.10792580101180438,
+      "grad_norm": 3.718262195587158,
+      "learning_rate": 0.00019335130278526507,
+      "loss": 0.8783,
+      "step": 80
+    },
+    {
+      "epoch": 0.12141652613827993,
+      "grad_norm": 3.689655065536499,
+      "learning_rate": 0.00019245283018867927,
+      "loss": 0.7619,
+      "step": 90
+    },
+    {
+      "epoch": 0.13490725126475547,
+      "grad_norm": 6.728384017944336,
+      "learning_rate": 0.00019155435759209347,
+      "loss": 0.7244,
+      "step": 100
+    },
+    {
+      "epoch": 0.14839797639123103,
+      "grad_norm": 4.880130290985107,
+      "learning_rate": 0.00019065588499550765,
+      "loss": 0.6931,
+      "step": 110
+    },
+    {
+      "epoch": 0.16188870151770657,
+      "grad_norm": 10.14767074584961,
+      "learning_rate": 0.00018975741239892185,
+      "loss": 0.6995,
+      "step": 120
+    },
+    {
+      "epoch": 0.17537942664418213,
+      "grad_norm": 10.897403717041016,
+      "learning_rate": 0.00018885893980233605,
+      "loss": 0.69,
+      "step": 130
+    },
+    {
+      "epoch": 0.18887015177065766,
+      "grad_norm": 7.759915351867676,
+      "learning_rate": 0.00018796046720575023,
+      "loss": 0.6899,
+      "step": 140
+    },
+    {
+      "epoch": 0.20236087689713322,
+      "grad_norm": 7.006256580352783,
+      "learning_rate": 0.00018706199460916443,
+      "loss": 0.6739,
+      "step": 150
+    },
+    {
+      "epoch": 0.21585160202360876,
+      "grad_norm": 5.777213096618652,
+      "learning_rate": 0.00018616352201257863,
+      "loss": 0.6731,
+      "step": 160
+    },
+    {
+      "epoch": 0.22934232715008432,
+      "grad_norm": 10.224111557006836,
+      "learning_rate": 0.00018526504941599283,
+      "loss": 0.6417,
+      "step": 170
+    },
+    {
+      "epoch": 0.24283305227655985,
+      "grad_norm": 6.332294464111328,
+      "learning_rate": 0.000184366576819407,
+      "loss": 0.6452,
+      "step": 180
+    },
+    {
+      "epoch": 0.2563237774030354,
+      "grad_norm": 3.9472005367279053,
+      "learning_rate": 0.0001834681042228212,
+      "loss": 0.6271,
+      "step": 190
+    },
+    {
+      "epoch": 0.26981450252951095,
+      "grad_norm": 6.640835762023926,
+      "learning_rate": 0.0001825696316262354,
+      "loss": 0.6161,
+      "step": 200
+    },
+    {
+      "epoch": 0.28330522765598654,
+      "grad_norm": 5.6504316329956055,
+      "learning_rate": 0.00018167115902964959,
+      "loss": 0.6452,
+      "step": 210
+    },
+    {
+      "epoch": 0.29679595278246207,
+      "grad_norm": 3.4219472408294678,
+      "learning_rate": 0.0001807726864330638,
+      "loss": 0.6228,
+      "step": 220
+    },
+    {
+      "epoch": 0.3102866779089376,
+      "grad_norm": 7.479050636291504,
+      "learning_rate": 0.000179874213836478,
+      "loss": 0.6035,
+      "step": 230
+    },
+    {
+      "epoch": 0.32377740303541314,
+      "grad_norm": 6.84128475189209,
+      "learning_rate": 0.0001789757412398922,
+      "loss": 0.5763,
+      "step": 240
+    },
+    {
+      "epoch": 0.3372681281618887,
+      "grad_norm": 3.407183885574341,
+      "learning_rate": 0.0001780772686433064,
+      "loss": 0.5981,
+      "step": 250
+    },
+    {
+      "epoch": 0.35075885328836426,
+      "grad_norm": 6.634527683258057,
+      "learning_rate": 0.0001771787960467206,
+      "loss": 0.6313,
+      "step": 260
+    },
+    {
+      "epoch": 0.3642495784148398,
+      "grad_norm": 6.274175643920898,
+      "learning_rate": 0.0001762803234501348,
+      "loss": 0.5971,
+      "step": 270
+    },
+    {
+      "epoch": 0.3777403035413153,
+      "grad_norm": 5.464880466461182,
+      "learning_rate": 0.00017538185085354897,
+      "loss": 0.629,
+      "step": 280
+    },
+    {
+      "epoch": 0.3912310286677909,
+      "grad_norm": 7.9421234130859375,
+      "learning_rate": 0.00017448337825696318,
+      "loss": 0.5604,
+      "step": 290
+    },
+    {
+      "epoch": 0.40472175379426645,
+      "grad_norm": 6.513714790344238,
+      "learning_rate": 0.00017358490566037738,
+      "loss": 0.5442,
+      "step": 300
+    },
+    {
+      "epoch": 0.418212478920742,
+      "grad_norm": 4.856393337249756,
+      "learning_rate": 0.00017268643306379155,
+      "loss": 0.5444,
+      "step": 310
+    },
+    {
+      "epoch": 0.4317032040472175,
+      "grad_norm": 7.252712249755859,
+      "learning_rate": 0.00017178796046720576,
+      "loss": 0.6071,
+      "step": 320
+    },
+    {
+      "epoch": 0.4451939291736931,
+      "grad_norm": 6.8953328132629395,
+      "learning_rate": 0.00017088948787061996,
+      "loss": 0.5622,
+      "step": 330
+    },
+    {
+      "epoch": 0.45868465430016864,
+      "grad_norm": 6.220505237579346,
+      "learning_rate": 0.00016999101527403416,
+      "loss": 0.6213,
+      "step": 340
+    },
+    {
+      "epoch": 0.47217537942664417,
+      "grad_norm": 1.9960452318191528,
+      "learning_rate": 0.00016909254267744833,
+      "loss": 0.5752,
+      "step": 350
+    },
+    {
+      "epoch": 0.4856661045531197,
+      "grad_norm": 4.8189778327941895,
+      "learning_rate": 0.00016819407008086254,
+      "loss": 0.5695,
+      "step": 360
+    },
+    {
+      "epoch": 0.4991568296795953,
+      "grad_norm": 2.157482624053955,
+      "learning_rate": 0.00016729559748427674,
+      "loss": 0.5818,
+      "step": 370
+    },
+    {
+      "epoch": 0.5126475548060708,
+      "grad_norm": 6.034278392791748,
+      "learning_rate": 0.00016639712488769091,
+      "loss": 0.5632,
+      "step": 380
+    },
+    {
+      "epoch": 0.5261382799325464,
+      "grad_norm": 3.6643030643463135,
+      "learning_rate": 0.00016549865229110512,
+      "loss": 0.5149,
+      "step": 390
+    },
+    {
+      "epoch": 0.5396290050590219,
+      "grad_norm": 3.4874682426452637,
+      "learning_rate": 0.00016460017969451932,
+      "loss": 0.5712,
+      "step": 400
+    },
+    {
+      "epoch": 0.5531197301854974,
+      "grad_norm": 5.94505500793457,
+      "learning_rate": 0.00016370170709793352,
+      "loss": 0.5579,
+      "step": 410
+    },
+    {
+      "epoch": 0.5666104553119731,
+      "grad_norm": 2.6858978271484375,
+      "learning_rate": 0.00016280323450134772,
+      "loss": 0.5601,
+      "step": 420
+    },
+    {
+      "epoch": 0.5801011804384486,
+      "grad_norm": 0.5919632315635681,
+      "learning_rate": 0.00016190476190476192,
+      "loss": 0.5122,
+      "step": 430
+    },
+    {
+      "epoch": 0.5935919055649241,
+      "grad_norm": 9.723355293273926,
+      "learning_rate": 0.00016100628930817613,
+      "loss": 0.5408,
+      "step": 440
+    },
+    {
+      "epoch": 0.6070826306913997,
+      "grad_norm": 3.421534776687622,
+      "learning_rate": 0.0001601078167115903,
+      "loss": 0.5125,
+      "step": 450
+    },
+    {
+      "epoch": 0.6205733558178752,
+      "grad_norm": 1.350819706916809,
+      "learning_rate": 0.0001592093441150045,
+      "loss": 0.5443,
+      "step": 460
+    },
+    {
+      "epoch": 0.6340640809443507,
+      "grad_norm": 2.9618473052978516,
+      "learning_rate": 0.0001583108715184187,
+      "loss": 0.5792,
+      "step": 470
+    },
+    {
+      "epoch": 0.6475548060708263,
+      "grad_norm": 11.16501522064209,
+      "learning_rate": 0.00015741239892183288,
+      "loss": 0.5499,
+      "step": 480
+    },
+    {
+      "epoch": 0.6610455311973018,
+      "grad_norm": 2.7122786045074463,
+      "learning_rate": 0.00015651392632524708,
+      "loss": 0.589,
+      "step": 490
+    },
+    {
+      "epoch": 0.6745362563237775,
+      "grad_norm": 6.2460408210754395,
+      "learning_rate": 0.00015561545372866128,
+      "loss": 0.5422,
+      "step": 500
+    },
+    {
+      "epoch": 0.688026981450253,
+      "grad_norm": 12.943184852600098,
+      "learning_rate": 0.0001547169811320755,
+      "loss": 0.5431,
+      "step": 510
+    },
+    {
+      "epoch": 0.7015177065767285,
+      "grad_norm": 2.4169883728027344,
+      "learning_rate": 0.00015381850853548966,
+      "loss": 0.4984,
+      "step": 520
+    },
+    {
+      "epoch": 0.715008431703204,
+      "grad_norm": 5.108154773712158,
+      "learning_rate": 0.00015292003593890386,
+      "loss": 0.4795,
+      "step": 530
+    },
+    {
+      "epoch": 0.7284991568296796,
+      "grad_norm": 3.268669605255127,
+      "learning_rate": 0.00015202156334231807,
+      "loss": 0.4961,
+      "step": 540
+    },
+    {
+      "epoch": 0.7419898819561551,
+      "grad_norm": 1.7926982641220093,
+      "learning_rate": 0.00015112309074573224,
+      "loss": 0.5339,
+      "step": 550
+    },
+    {
+      "epoch": 0.7554806070826307,
+      "grad_norm": 3.6382663249969482,
+      "learning_rate": 0.00015022461814914644,
+      "loss": 0.4855,
+      "step": 560
+    },
+    {
+      "epoch": 0.7689713322091062,
+      "grad_norm": 8.025957107543945,
+      "learning_rate": 0.00014932614555256067,
+      "loss": 0.4901,
+      "step": 570
+    },
+    {
+      "epoch": 0.7824620573355818,
+      "grad_norm": 4.220882892608643,
+      "learning_rate": 0.00014842767295597485,
+      "loss": 0.5448,
+      "step": 580
+    },
+    {
+      "epoch": 0.7959527824620574,
+      "grad_norm": 6.42817497253418,
+      "learning_rate": 0.00014752920035938905,
+      "loss": 0.4969,
+      "step": 590
+    },
+    {
+      "epoch": 0.8094435075885329,
+      "grad_norm": 4.182610988616943,
+      "learning_rate": 0.00014663072776280325,
+      "loss": 0.4863,
+      "step": 600
+    },
+    {
+      "epoch": 0.8229342327150084,
+      "grad_norm": 3.687934637069702,
+      "learning_rate": 0.00014573225516621745,
+      "loss": 0.5162,
+      "step": 610
+    },
+    {
+      "epoch": 0.836424957841484,
+      "grad_norm": 3.8240537643432617,
+      "learning_rate": 0.00014483378256963163,
+      "loss": 0.5089,
+      "step": 620
+    },
+    {
+      "epoch": 0.8499156829679595,
+      "grad_norm": 3.3823423385620117,
+      "learning_rate": 0.00014393530997304583,
+      "loss": 0.5447,
+      "step": 630
+    },
+    {
+      "epoch": 0.863406408094435,
+      "grad_norm": 4.591256618499756,
+      "learning_rate": 0.00014303683737646003,
+      "loss": 0.5067,
+      "step": 640
+    },
+    {
+      "epoch": 0.8768971332209107,
+      "grad_norm": 4.414494514465332,
+      "learning_rate": 0.0001421383647798742,
+      "loss": 0.4858,
+      "step": 650
+    },
+    {
+      "epoch": 0.8903878583473862,
+      "grad_norm": 3.4649970531463623,
+      "learning_rate": 0.0001412398921832884,
+      "loss": 0.5243,
+      "step": 660
+    },
+    {
+      "epoch": 0.9038785834738617,
+      "grad_norm": 4.813696384429932,
+      "learning_rate": 0.0001403414195867026,
+      "loss": 0.51,
+      "step": 670
+    },
+    {
+      "epoch": 0.9173693086003373,
+      "grad_norm": 4.597005844116211,
+      "learning_rate": 0.00013944294699011681,
+      "loss": 0.5079,
+      "step": 680
+    },
+    {
+      "epoch": 0.9308600337268128,
+      "grad_norm": 4.291003704071045,
+      "learning_rate": 0.000138544474393531,
+      "loss": 0.4966,
+      "step": 690
+    },
+    {
+      "epoch": 0.9443507588532883,
+      "grad_norm": 3.8949058055877686,
+      "learning_rate": 0.0001376460017969452,
+      "loss": 0.5086,
+      "step": 700
+    },
+    {
+      "epoch": 0.9578414839797639,
+      "grad_norm": 4.685634613037109,
+      "learning_rate": 0.0001367475292003594,
+      "loss": 0.5005,
+      "step": 710
+    },
+    {
+      "epoch": 0.9713322091062394,
+      "grad_norm": 3.9277801513671875,
+      "learning_rate": 0.0001358490566037736,
+      "loss": 0.4759,
+      "step": 720
+    },
+    {
+      "epoch": 0.984822934232715,
+      "grad_norm": 4.3011345863342285,
+      "learning_rate": 0.0001349505840071878,
+      "loss": 0.4371,
+      "step": 730
+    },
+    {
+      "epoch": 0.9983136593591906,
+      "grad_norm": 3.370706796646118,
+      "learning_rate": 0.000134052111410602,
+      "loss": 0.4811,
+      "step": 740
+    },
+    {
+      "epoch": 1.0107925801011803,
+      "grad_norm": 5.607315540313721,
+      "learning_rate": 0.00013315363881401617,
+      "loss": 0.4648,
+      "step": 750
+    },
+    {
+      "epoch": 1.024283305227656,
+      "grad_norm": 3.9047558307647705,
+      "learning_rate": 0.00013225516621743038,
+      "loss": 0.4813,
+      "step": 760
+    },
+    {
+      "epoch": 1.0377740303541316,
+      "grad_norm": 3.686000108718872,
+      "learning_rate": 0.00013135669362084458,
+      "loss": 0.5189,
+      "step": 770
+    },
+    {
+      "epoch": 1.0512647554806072,
+      "grad_norm": 3.8528940677642822,
+      "learning_rate": 0.00013045822102425878,
+      "loss": 0.4761,
+      "step": 780
+    },
+    {
+      "epoch": 1.0647554806070827,
+      "grad_norm": 3.5714054107666016,
+      "learning_rate": 0.00012955974842767296,
+      "loss": 0.4845,
+      "step": 790
+    },
+    {
+      "epoch": 1.0782462057335582,
+      "grad_norm": 3.3482840061187744,
+      "learning_rate": 0.00012866127583108716,
+      "loss": 0.487,
+      "step": 800
+    },
+    {
+      "epoch": 1.0917369308600338,
+      "grad_norm": 4.3598127365112305,
+      "learning_rate": 0.00012776280323450136,
+      "loss": 0.4918,
+      "step": 810
+    },
+    {
+      "epoch": 1.1052276559865093,
+      "grad_norm": 3.4356210231781006,
+      "learning_rate": 0.00012686433063791554,
+      "loss": 0.5042,
+      "step": 820
+    },
+    {
+      "epoch": 1.1187183811129848,
+      "grad_norm": 3.015895366668701,
+      "learning_rate": 0.00012596585804132974,
+      "loss": 0.4911,
+      "step": 830
+    },
+    {
+      "epoch": 1.1322091062394604,
+      "grad_norm": 3.4911601543426514,
+      "learning_rate": 0.00012506738544474394,
+      "loss": 0.507,
+      "step": 840
+    },
+    {
+      "epoch": 1.145699831365936,
+      "grad_norm": 5.718015670776367,
+      "learning_rate": 0.00012416891284815814,
+      "loss": 0.4668,
+      "step": 850
+    },
+    {
+      "epoch": 1.1591905564924114,
+      "grad_norm": 4.448844909667969,
+      "learning_rate": 0.00012327044025157232,
+      "loss": 0.465,
+      "step": 860
+    },
+    {
+      "epoch": 1.172681281618887,
+      "grad_norm": 3.243500232696533,
+      "learning_rate": 0.00012237196765498652,
+      "loss": 0.4922,
+      "step": 870
+    },
+    {
+      "epoch": 1.1861720067453625,
+      "grad_norm": 3.25435471534729,
+      "learning_rate": 0.00012147349505840073,
+      "loss": 0.4825,
+      "step": 880
+    },
+    {
+      "epoch": 1.199662731871838,
+      "grad_norm": 3.0708346366882324,
+      "learning_rate": 0.00012057502246181491,
+      "loss": 0.4777,
+      "step": 890
+    },
+    {
+      "epoch": 1.2131534569983136,
+      "grad_norm": 3.274693727493286,
+      "learning_rate": 0.00011967654986522911,
+      "loss": 0.4727,
+      "step": 900
+    },
+    {
+      "epoch": 1.2266441821247893,
+      "grad_norm": 5.614912986755371,
+      "learning_rate": 0.00011877807726864331,
+      "loss": 0.4914,
+      "step": 910
+    },
+    {
+      "epoch": 1.2401349072512649,
+      "grad_norm": 3.7809228897094727,
+      "learning_rate": 0.0001178796046720575,
+      "loss": 0.4727,
+      "step": 920
+    },
+    {
+      "epoch": 1.2536256323777404,
+      "grad_norm": 3.46541428565979,
+      "learning_rate": 0.0001169811320754717,
+      "loss": 0.4799,
+      "step": 930
+    },
+    {
+      "epoch": 1.267116357504216,
+      "grad_norm": 3.8525471687316895,
+      "learning_rate": 0.0001160826594788859,
+      "loss": 0.5079,
+      "step": 940
+    },
+    {
+      "epoch": 1.2806070826306915,
+      "grad_norm": 3.959237575531006,
+      "learning_rate": 0.00011518418688230011,
+      "loss": 0.4689,
+      "step": 950
+    },
+    {
+      "epoch": 1.294097807757167,
+      "grad_norm": 3.873572826385498,
+      "learning_rate": 0.00011428571428571428,
+      "loss": 0.4688,
+      "step": 960
+    },
+    {
+      "epoch": 1.3075885328836425,
+      "grad_norm": 3.813100814819336,
+      "learning_rate": 0.00011338724168912849,
+      "loss": 0.4762,
+      "step": 970
+    },
+    {
+      "epoch": 1.321079258010118,
+      "grad_norm": 3.517953634262085,
+      "learning_rate": 0.00011248876909254269,
+      "loss": 0.4595,
+      "step": 980
+    },
+    {
+      "epoch": 1.3345699831365936,
+      "grad_norm": 3.5959513187408447,
+      "learning_rate": 0.00011159029649595688,
+      "loss": 0.4959,
+      "step": 990
+    },
+    {
+      "epoch": 1.3480607082630691,
+      "grad_norm": 3.9473278522491455,
+      "learning_rate": 0.00011069182389937108,
+      "loss": 0.4552,
+      "step": 1000
+    },
+    {
+      "epoch": 1.3615514333895447,
+      "grad_norm": 3.5634729862213135,
+      "learning_rate": 0.00010979335130278528,
+      "loss": 0.4765,
+      "step": 1010
+    },
+    {
+      "epoch": 1.3750421585160202,
+      "grad_norm": 3.88102126121521,
+      "learning_rate": 0.00010889487870619948,
+      "loss": 0.4282,
+      "step": 1020
+    },
+    {
+      "epoch": 1.3885328836424957,
+      "grad_norm": 4.591094970703125,
+      "learning_rate": 0.00010799640610961366,
+      "loss": 0.483,
+      "step": 1030
+    },
+    {
+      "epoch": 1.4020236087689713,
+      "grad_norm": 2.726210355758667,
+      "learning_rate": 0.00010709793351302786,
+      "loss": 0.5491,
+      "step": 1040
+    },
+    {
+      "epoch": 1.4155143338954468,
+      "grad_norm": 2.8899638652801514,
+      "learning_rate": 0.00010619946091644206,
+      "loss": 0.4494,
+      "step": 1050
+    },
+    {
+      "epoch": 1.4290050590219225,
+      "grad_norm": 3.1624226570129395,
+      "learning_rate": 0.00010530098831985624,
+      "loss": 0.4861,
+      "step": 1060
+    },
+    {
+      "epoch": 1.4424957841483979,
+      "grad_norm": 4.227782726287842,
+      "learning_rate": 0.00010440251572327044,
+      "loss": 0.4771,
+      "step": 1070
+    },
+    {
+      "epoch": 1.4559865092748736,
+      "grad_norm": 3.1170289516448975,
+      "learning_rate": 0.00010350404312668464,
+      "loss": 0.4608,
+      "step": 1080
+    },
+    {
+      "epoch": 1.4694772344013491,
+      "grad_norm": 3.7711753845214844,
+      "learning_rate": 0.00010260557053009883,
+      "loss": 0.4553,
+      "step": 1090
+    },
+    {
+      "epoch": 1.4829679595278247,
+      "grad_norm": 3.723172187805176,
+      "learning_rate": 0.00010170709793351303,
+      "loss": 0.4347,
+      "step": 1100
+    },
+    {
+      "epoch": 1.4964586846543002,
+      "grad_norm": 3.469057083129883,
+      "learning_rate": 0.00010080862533692723,
+      "loss": 0.4747,
+      "step": 1110
+    },
+    {
+      "epoch": 1.5099494097807757,
+      "grad_norm": 3.3788669109344482,
+      "learning_rate": 9.991015274034142e-05,
+      "loss": 0.4788,
+      "step": 1120
+    },
+    {
+      "epoch": 1.5234401349072513,
+      "grad_norm": 5.321389198303223,
+      "learning_rate": 9.901168014375562e-05,
+      "loss": 0.4424,
+      "step": 1130
+    },
+    {
+      "epoch": 1.5369308600337268,
+      "grad_norm": 3.616950511932373,
+      "learning_rate": 9.811320754716981e-05,
+      "loss": 0.4765,
+      "step": 1140
+    },
+    {
+      "epoch": 1.5504215851602023,
+      "grad_norm": 6.907515525817871,
+      "learning_rate": 9.7214734950584e-05,
+      "loss": 0.4797,
+      "step": 1150
+    },
+    {
+      "epoch": 1.5639123102866779,
+      "grad_norm": 17.670085906982422,
+      "learning_rate": 9.63162623539982e-05,
+      "loss": 0.4531,
+      "step": 1160
+    },
+    {
+      "epoch": 1.5774030354131534,
+      "grad_norm": 2.698268413543701,
+      "learning_rate": 9.54177897574124e-05,
+      "loss": 0.493,
+      "step": 1170
+    },
+    {
+      "epoch": 1.590893760539629,
+      "grad_norm": 6.46640682220459,
+      "learning_rate": 9.451931716082661e-05,
+      "loss": 0.4616,
+      "step": 1180
+    },
+    {
+      "epoch": 1.6043844856661047,
+      "grad_norm": 2.7934491634368896,
+      "learning_rate": 9.36208445642408e-05,
+      "loss": 0.4819,
+      "step": 1190
+    },
+    {
+      "epoch": 1.61787521079258,
+      "grad_norm": 4.599141597747803,
+      "learning_rate": 9.272237196765498e-05,
+      "loss": 0.4506,
+      "step": 1200
+    },
+    {
+      "epoch": 1.6313659359190558,
+      "grad_norm": 3.3172645568847656,
+      "learning_rate": 9.182389937106919e-05,
+      "loss": 0.477,
+      "step": 1210
+    },
+    {
+      "epoch": 1.644856661045531,
+      "grad_norm": 3.431093215942383,
+      "learning_rate": 9.092542677448338e-05,
+      "loss": 0.4594,
+      "step": 1220
+    },
+    {
+      "epoch": 1.6583473861720068,
+      "grad_norm": 3.2188451290130615,
+      "learning_rate": 9.002695417789758e-05,
+      "loss": 0.4311,
+      "step": 1230
+    },
+    {
+      "epoch": 1.6718381112984821,
+      "grad_norm": 4.311905384063721,
+      "learning_rate": 8.912848158131178e-05,
+      "loss": 0.4608,
+      "step": 1240
+    },
+    {
+      "epoch": 1.685328836424958,
+      "grad_norm": 2.9855377674102783,
+      "learning_rate": 8.823000898472597e-05,
+      "loss": 0.5016,
+      "step": 1250
+    },
+    {
+      "epoch": 1.6988195615514334,
+      "grad_norm": 4.071963310241699,
+      "learning_rate": 8.733153638814017e-05,
+      "loss": 0.4684,
+      "step": 1260
+    },
+    {
+      "epoch": 1.712310286677909,
+      "grad_norm": 3.417339563369751,
+      "learning_rate": 8.643306379155436e-05,
+      "loss": 0.4364,
+      "step": 1270
+    },
+    {
+      "epoch": 1.7258010118043845,
+      "grad_norm": 3.9395434856414795,
+      "learning_rate": 8.553459119496856e-05,
+      "loss": 0.5122,
+      "step": 1280
+    },
+    {
+      "epoch": 1.73929173693086,
+      "grad_norm": 2.934011220932007,
+      "learning_rate": 8.463611859838275e-05,
+      "loss": 0.4572,
+      "step": 1290
+    },
+    {
+      "epoch": 1.7527824620573356,
+      "grad_norm": 3.452453851699829,
+      "learning_rate": 8.373764600179695e-05,
+      "loss": 0.4674,
+      "step": 1300
+    },
+    {
+      "epoch": 1.766273187183811,
+      "grad_norm": 3.5324788093566895,
+      "learning_rate": 8.283917340521114e-05,
+      "loss": 0.4678,
+      "step": 1310
+    },
+    {
+      "epoch": 1.7797639123102866,
+      "grad_norm": 3.4225003719329834,
+      "learning_rate": 8.194070080862534e-05,
+      "loss": 0.5027,
+      "step": 1320
+    },
+    {
+      "epoch": 1.7932546374367622,
+      "grad_norm": 1.9840672016143799,
+      "learning_rate": 8.104222821203954e-05,
+      "loss": 0.4977,
+      "step": 1330
+    },
+    {
+      "epoch": 1.806745362563238,
+      "grad_norm": 2.443516492843628,
+      "learning_rate": 8.014375561545373e-05,
+      "loss": 0.455,
+      "step": 1340
+    },
+    {
+      "epoch": 1.8202360876897132,
+      "grad_norm": 2.689674139022827,
+      "learning_rate": 7.924528301886794e-05,
+      "loss": 0.4665,
+      "step": 1350
+    },
+    {
+      "epoch": 1.833726812816189,
+      "grad_norm": 3.2184505462646484,
+      "learning_rate": 7.834681042228212e-05,
+      "loss": 0.475,
+      "step": 1360
+    },
+    {
+      "epoch": 1.8472175379426643,
+      "grad_norm": 3.5544068813323975,
+      "learning_rate": 7.744833782569631e-05,
+      "loss": 0.4921,
+      "step": 1370
+    },
+    {
+      "epoch": 1.86070826306914,
+      "grad_norm": 3.2145285606384277,
+      "learning_rate": 7.654986522911051e-05,
+      "loss": 0.4825,
+      "step": 1380
+    },
+    {
+      "epoch": 1.8741989881956154,
+      "grad_norm": 3.448352813720703,
+      "learning_rate": 7.56513926325247e-05,
+      "loss": 0.4442,
+      "step": 1390
+    },
+    {
+      "epoch": 1.8876897133220911,
+      "grad_norm": 3.329658269882202,
+      "learning_rate": 7.47529200359389e-05,
+      "loss": 0.4498,
+      "step": 1400
+    },
+    {
+      "epoch": 1.9011804384485667,
+      "grad_norm": 2.8206887245178223,
+      "learning_rate": 7.385444743935311e-05,
+      "loss": 0.4525,
+      "step": 1410
+    },
+    {
+      "epoch": 1.9146711635750422,
+      "grad_norm": 3.067368507385254,
+      "learning_rate": 7.29559748427673e-05,
+      "loss": 0.4652,
+      "step": 1420
+    },
+    {
+      "epoch": 1.9281618887015177,
+      "grad_norm": 2.902029275894165,
+      "learning_rate": 7.20575022461815e-05,
+      "loss": 0.462,
+      "step": 1430
+    },
+    {
+      "epoch": 1.9416526138279933,
+      "grad_norm": 2.9392433166503906,
+      "learning_rate": 7.115902964959569e-05,
+      "loss": 0.4505,
+      "step": 1440
+    },
+    {
+      "epoch": 1.9551433389544688,
+      "grad_norm": 3.328568935394287,
+      "learning_rate": 7.026055705300989e-05,
+      "loss": 0.4688,
+      "step": 1450
+    },
+    {
+      "epoch": 1.9686340640809443,
+      "grad_norm": 3.0431363582611084,
+      "learning_rate": 6.936208445642408e-05,
+      "loss": 0.4749,
+      "step": 1460
+    },
+    {
+      "epoch": 1.9821247892074199,
+      "grad_norm": 3.3194100856781006,
+      "learning_rate": 6.846361185983828e-05,
+      "loss": 0.4364,
+      "step": 1470
+    },
+    {
+      "epoch": 1.9956155143338954,
+      "grad_norm": 3.563108205795288,
+      "learning_rate": 6.756513926325248e-05,
+      "loss": 0.4635,
+      "step": 1480
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 2226,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 7.380687531855053e+16,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}
diff --git a/checkpoint-1484/training_args.bin b/checkpoint-1484/training_args.bin
new file mode 100644
index 0000000000000000000000000000000000000000..597bbc184c73a1ca565444bfc5a1a6c505b47b83
--- /dev/null
+++ b/checkpoint-1484/training_args.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:160a1045610f1057c5a1c808e36986bd0969b8e2c1163c803dfd9b9a2891eb12
+size 5304
diff --git a/checkpoint-2226/README.md b/checkpoint-2226/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..d44d68a82ab48884e6afee364005e8a561c49c7c
--- /dev/null
+++ b/checkpoint-2226/README.md
@@ -0,0 +1,202 @@
+---
+base_model: ModelSpace/GemmaX2-28-2B-v0.1
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.2
\ No newline at end of file
diff --git a/checkpoint-2226/adapter_config.json b/checkpoint-2226/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..191461bdeecd0575bb2844135920827ba4f2c74c
--- /dev/null
+++ b/checkpoint-2226/adapter_config.json
@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "ModelSpace/GemmaX2-28-2B-v0.1",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}
\ No newline at end of file
diff --git a/checkpoint-2226/adapter_model.safetensors b/checkpoint-2226/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..bbea7f39df1ed43106a43966c997ee88d13e609b
--- /dev/null
+++ b/checkpoint-2226/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:591a55ad10ccb0ac75ccca10719e3a31ab1ba1997afb246c2d0f2d2311f34f77
+size 6403448
diff --git a/checkpoint-2226/chat_template.jinja b/checkpoint-2226/chat_template.jinja
new file mode 100644
index 0000000000000000000000000000000000000000..127d9d1acfd2092b5b5d74ea3fe2626658923ada
--- /dev/null
+++ b/checkpoint-2226/chat_template.jinja
@@ -0,0 +1 @@
+{% if messages[0]['role'] == 'system' %}{% set system_message = messages[0]['content'] %}{% endif %}{% if system_message is defined %}{{ system_message }}{% endif %}{% for message in messages %}{% set content = message['content'] %}{% if message['role'] == 'user' %}{{ content }}{% elif message['role'] == 'assistant' %}{{ content }}{% endif %}{% endfor %}
\ No newline at end of file
diff --git a/checkpoint-2226/optimizer.pt b/checkpoint-2226/optimizer.pt
new file mode 100644
index 0000000000000000000000000000000000000000..54a60714d92b345035bdbfeda215286d99e631f3
--- /dev/null
+++ b/checkpoint-2226/optimizer.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4688443b29c8de6bce11791a5b00ac52201007b1b926dde1637cc91f0a417d36
+size 12867066
diff --git a/checkpoint-2226/rng_state.pth b/checkpoint-2226/rng_state.pth
new file mode 100644
index 0000000000000000000000000000000000000000..811a3aaf185b4a55bd0ef93ef54a31c850dc0118
--- /dev/null
+++ b/checkpoint-2226/rng_state.pth
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4bf9ffa91da5fad995fd93616a54812b1b78f262d6ac580e252c7bc80bf9c616
+size 14244
diff --git a/checkpoint-2226/scaler.pt b/checkpoint-2226/scaler.pt
new file mode 100644
index 0000000000000000000000000000000000000000..b6e8bcd665b88deea03b5022b8e209ffe4d9b4a0
--- /dev/null
+++ b/checkpoint-2226/scaler.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4fb0bd919e48b1871ef9779ddc4b93a423bc1ffe9ec6b4fd5e3705e37dacf353
+size 988
diff --git a/checkpoint-2226/scheduler.pt b/checkpoint-2226/scheduler.pt
new file mode 100644
index 0000000000000000000000000000000000000000..a09317921d47aea5b652d56faa0fe4bc6f30a834
--- /dev/null
+++ b/checkpoint-2226/scheduler.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e75ee1510b91b7ac6e02fe6fc4b04b0785775bdcf405691d8023c17656cc4270
+size 1064
diff --git a/checkpoint-2226/special_tokens_map.json b/checkpoint-2226/special_tokens_map.json
new file mode 100644
index 0000000000000000000000000000000000000000..f6119589e367b2de0fc8cbd2f1217667532e3174
--- /dev/null
+++ b/checkpoint-2226/special_tokens_map.json
@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<bos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<eos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}
diff --git a/checkpoint-2226/tokenizer.json b/checkpoint-2226/tokenizer.json
new file mode 100644
index 0000000000000000000000000000000000000000..2b2d51a12a27cc5bd1baeaca3ec01d6f6f458f64
--- /dev/null
+++ b/checkpoint-2226/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:afe09ee4dcd6379dfa9f12c0178f86eab033fc9f97f4e1d605a44da3488b6c53
+size 34363138
diff --git a/checkpoint-2226/tokenizer.model b/checkpoint-2226/tokenizer.model
new file mode 100644
index 0000000000000000000000000000000000000000..796efe9ab515c15e146ce7588e6d7b9b8134dbf8
--- /dev/null
+++ b/checkpoint-2226/tokenizer.model
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61a7b147390c64585d6c3543dd6fc636906c9af3865a5548f27f31aee1d4c8e2
+size 4241003
diff --git a/checkpoint-2226/tokenizer_config.json b/checkpoint-2226/tokenizer_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..62e63dbcee5400f4f34987854723a5120c4cc076
--- /dev/null
+++ b/checkpoint-2226/tokenizer_config.json
@@ -0,0 +1,2011 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<eos>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "<bos>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "4": {
+      "content": "<mask>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "5": {
+      "content": "<2mass>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "6": {
+      "content": "[@BOS@]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "7": {
+      "content": "<unused0>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "8": {
+      "content": "<unused1>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "9": {
+      "content": "<unused2>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "10": {
+      "content": "<unused3>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "11": {
+      "content": "<unused4>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "12": {
+      "content": "<unused5>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "13": {
+      "content": "<unused6>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "14": {
+      "content": "<unused7>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "15": {
+      "content": "<unused8>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "16": {
+      "content": "<unused9>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "17": {
+      "content": "<unused10>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "18": {
+      "content": "<unused11>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "19": {
+      "content": "<unused12>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "20": {
+      "content": "<unused13>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "21": {
+      "content": "<unused14>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "22": {
+      "content": "<unused15>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "23": {
+      "content": "<unused16>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "24": {
+      "content": "<unused17>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "25": {
+      "content": "<unused18>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "26": {
+      "content": "<unused19>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "27": {
+      "content": "<unused20>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "28": {
+      "content": "<unused21>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "29": {
+      "content": "<unused22>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "30": {
+      "content": "<unused23>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "31": {
+      "content": "<unused24>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "32": {
+      "content": "<unused25>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "33": {
+      "content": "<unused26>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "34": {
+      "content": "<unused27>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "35": {
+      "content": "<unused28>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "36": {
+      "content": "<unused29>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "37": {
+      "content": "<unused30>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "38": {
+      "content": "<unused31>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "39": {
+      "content": "<unused32>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "40": {
+      "content": "<unused33>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "41": {
+      "content": "<unused34>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "42": {
+      "content": "<unused35>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "43": {
+      "content": "<unused36>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "44": {
+      "content": "<unused37>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "45": {
+      "content": "<unused38>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "46": {
+      "content": "<unused39>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "47": {
+      "content": "<unused40>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "48": {
+      "content": "<unused41>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "49": {
+      "content": "<unused42>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50": {
+      "content": "<unused43>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "51": {
+      "content": "<unused44>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "52": {
+      "content": "<unused45>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "53": {
+      "content": "<unused46>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "54": {
+      "content": "<unused47>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55": {
+      "content": "<unused48>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "56": {
+      "content": "<unused49>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "57": {
+      "content": "<unused50>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "58": {
+      "content": "<unused51>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "59": {
+      "content": "<unused52>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "60": {
+      "content": "<unused53>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "61": {
+      "content": "<unused54>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "62": {
+      "content": "<unused55>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "63": {
+      "content": "<unused56>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "64": {
+      "content": "<unused57>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "65": {
+      "content": "<unused58>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "66": {
+      "content": "<unused59>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "67": {
+      "content": "<unused60>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "68": {
+      "content": "<unused61>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "69": {
+      "content": "<unused62>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "70": {
+      "content": "<unused63>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "71": {
+      "content": "<unused64>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "72": {
+      "content": "<unused65>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "73": {
+      "content": "<unused66>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "74": {
+      "content": "<unused67>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "75": {
+      "content": "<unused68>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "76": {
+      "content": "<unused69>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "77": {
+      "content": "<unused70>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "78": {
+      "content": "<unused71>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "79": {
+      "content": "<unused72>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "80": {
+      "content": "<unused73>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "81": {
+      "content": "<unused74>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "82": {
+      "content": "<unused75>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "83": {
+      "content": "<unused76>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "84": {
+      "content": "<unused77>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "85": {
+      "content": "<unused78>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "86": {
+      "content": "<unused79>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "87": {
+      "content": "<unused80>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "88": {
+      "content": "<unused81>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "89": {
+      "content": "<unused82>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "90": {
+      "content": "<unused83>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "91": {
+      "content": "<unused84>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "92": {
+      "content": "<unused85>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "93": {
+      "content": "<unused86>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "94": {
+      "content": "<unused87>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95": {
+      "content": "<unused88>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "96": {
+      "content": "<unused89>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "97": {
+      "content": "<unused90>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "98": {
+      "content": "<unused91>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "99": {
+      "content": "<unused92>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "100": {
+      "content": "<unused93>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "101": {
+      "content": "<unused94>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "102": {
+      "content": "<unused95>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "103": {
+      "content": "<unused96>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "104": {
+      "content": "<unused97>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "105": {
+      "content": "<unused98>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "106": {
+      "content": "<start_of_turn>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "107": {
+      "content": "<end_of_turn>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "108": {
+      "content": "\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "109": {
+      "content": "\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "110": {
+      "content": "\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "111": {
+      "content": "\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "112": {
+      "content": "\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "113": {
+      "content": "\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "114": {
+      "content": "\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "115": {
+      "content": "\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "116": {
+      "content": "\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "117": {
+      "content": "\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "118": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "119": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "120": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "121": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "122": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "123": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "124": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "125": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "126": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "127": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "128": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "129": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "130": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "131": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "132": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "133": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "134": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "135": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "136": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "137": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "138": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "139": {
+      "content": "▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "140": {
+      "content": "▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "141": {
+      "content": "▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "142": {
+      "content": "▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "143": {
+      "content": "▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "144": {
+      "content": "▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "145": {
+      "content": "▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "146": {
+      "content": "▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "147": {
+      "content": "▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "148": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "149": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "150": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "152": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "153": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "154": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "155": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "156": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "157": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "158": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "159": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "160": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "161": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "162": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "163": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "164": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "165": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "166": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "167": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "168": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "169": {
+      "content": "<table>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "170": {
+      "content": "<caption>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "171": {
+      "content": "<thead>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "172": {
+      "content": "<tbody>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "173": {
+      "content": "<tfoot>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "174": {
+      "content": "<tr>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "175": {
+      "content": "<th>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "176": {
+      "content": "<td>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "177": {
+      "content": "</table>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "178": {
+      "content": "</caption>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "179": {
+      "content": "</thead>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "180": {
+      "content": "</tbody>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "181": {
+      "content": "</tfoot>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "182": {
+      "content": "</tr>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "183": {
+      "content": "</th>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "184": {
+      "content": "</td>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "185": {
+      "content": "<h1>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "186": {
+      "content": "<h2>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "187": {
+      "content": "<h3>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "188": {
+      "content": "<h4>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "189": {
+      "content": "<h5>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "190": {
+      "content": "<h6>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "191": {
+      "content": "<blockquote>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "192": {
+      "content": "</h1>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "193": {
+      "content": "</h2>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "194": {
+      "content": "</h3>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "195": {
+      "content": "</h4>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "196": {
+      "content": "</h5>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "197": {
+      "content": "</h6>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "198": {
+      "content": "</blockquote>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "199": {
+      "content": "<strong>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "200": {
+      "content": "<em>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "201": {
+      "content": "<b>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "202": {
+      "content": "<i>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "203": {
+      "content": "<u>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "204": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "205": {
+      "content": "<sub>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "206": {
+      "content": "<sup>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "207": {
+      "content": "<code>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "208": {
+      "content": "</strong>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "209": {
+      "content": "</em>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "210": {
+      "content": "</b>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "211": {
+      "content": "</i>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "212": {
+      "content": "</u>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "213": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "214": {
+      "content": "</sub>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "215": {
+      "content": "</sup>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "216": {
+      "content": "</code>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255968": {
+      "content": "[toxicity=0]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255969": {
+      "content": "\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255970": {
+      "content": "\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255971": {
+      "content": "\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255972": {
+      "content": "\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255973": {
+      "content": "\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255974": {
+      "content": "\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255975": {
+      "content": "\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255976": {
+      "content": "\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255977": {
+      "content": "\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255978": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255979": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255980": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255981": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255982": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255983": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255984": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255985": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255986": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255987": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255988": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255989": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255990": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255991": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255992": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255993": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255994": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255995": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255996": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255997": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255998": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255999": {
+      "content": "<unused99>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "bos_token": "<bos>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<eos>",
+  "extra_special_tokens": {},
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<pad>",
+  "padding_side": "right",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "split_special_tokens": false,
+  "tokenizer_class": "GemmaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}
diff --git a/checkpoint-2226/trainer_state.json b/checkpoint-2226/trainer_state.json
new file mode 100644
index 0000000000000000000000000000000000000000..d4f028184f0a83ffbbbb0f4a12ea5eef0f08be65
--- /dev/null
+++ b/checkpoint-2226/trainer_state.json
@@ -0,0 +1,1588 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 3.0,
+  "eval_steps": 500,
+  "global_step": 2226,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.013490725126475547,
+      "grad_norm": 65.45332336425781,
+      "learning_rate": 0.00019964061096136568,
+      "loss": 22.1843,
+      "step": 10
+    },
+    {
+      "epoch": 0.026981450252951095,
+      "grad_norm": 6.102770805358887,
+      "learning_rate": 0.00019874213836477988,
+      "loss": 7.6527,
+      "step": 20
+    },
+    {
+      "epoch": 0.04047217537942664,
+      "grad_norm": 2.982379913330078,
+      "learning_rate": 0.00019784366576819408,
+      "loss": 3.4532,
+      "step": 30
+    },
+    {
+      "epoch": 0.05396290050590219,
+      "grad_norm": 3.4628562927246094,
+      "learning_rate": 0.00019694519317160826,
+      "loss": 2.4467,
+      "step": 40
+    },
+    {
+      "epoch": 0.06745362563237774,
+      "grad_norm": 3.0008349418640137,
+      "learning_rate": 0.00019604672057502246,
+      "loss": 1.6481,
+      "step": 50
+    },
+    {
+      "epoch": 0.08094435075885328,
+      "grad_norm": 1.767865777015686,
+      "learning_rate": 0.00019514824797843666,
+      "loss": 1.2119,
+      "step": 60
+    },
+    {
+      "epoch": 0.09443507588532883,
+      "grad_norm": 3.416968584060669,
+      "learning_rate": 0.00019424977538185087,
+      "loss": 0.967,
+      "step": 70
+    },
+    {
+      "epoch": 0.10792580101180438,
+      "grad_norm": 3.718262195587158,
+      "learning_rate": 0.00019335130278526507,
+      "loss": 0.8783,
+      "step": 80
+    },
+    {
+      "epoch": 0.12141652613827993,
+      "grad_norm": 3.689655065536499,
+      "learning_rate": 0.00019245283018867927,
+      "loss": 0.7619,
+      "step": 90
+    },
+    {
+      "epoch": 0.13490725126475547,
+      "grad_norm": 6.728384017944336,
+      "learning_rate": 0.00019155435759209347,
+      "loss": 0.7244,
+      "step": 100
+    },
+    {
+      "epoch": 0.14839797639123103,
+      "grad_norm": 4.880130290985107,
+      "learning_rate": 0.00019065588499550765,
+      "loss": 0.6931,
+      "step": 110
+    },
+    {
+      "epoch": 0.16188870151770657,
+      "grad_norm": 10.14767074584961,
+      "learning_rate": 0.00018975741239892185,
+      "loss": 0.6995,
+      "step": 120
+    },
+    {
+      "epoch": 0.17537942664418213,
+      "grad_norm": 10.897403717041016,
+      "learning_rate": 0.00018885893980233605,
+      "loss": 0.69,
+      "step": 130
+    },
+    {
+      "epoch": 0.18887015177065766,
+      "grad_norm": 7.759915351867676,
+      "learning_rate": 0.00018796046720575023,
+      "loss": 0.6899,
+      "step": 140
+    },
+    {
+      "epoch": 0.20236087689713322,
+      "grad_norm": 7.006256580352783,
+      "learning_rate": 0.00018706199460916443,
+      "loss": 0.6739,
+      "step": 150
+    },
+    {
+      "epoch": 0.21585160202360876,
+      "grad_norm": 5.777213096618652,
+      "learning_rate": 0.00018616352201257863,
+      "loss": 0.6731,
+      "step": 160
+    },
+    {
+      "epoch": 0.22934232715008432,
+      "grad_norm": 10.224111557006836,
+      "learning_rate": 0.00018526504941599283,
+      "loss": 0.6417,
+      "step": 170
+    },
+    {
+      "epoch": 0.24283305227655985,
+      "grad_norm": 6.332294464111328,
+      "learning_rate": 0.000184366576819407,
+      "loss": 0.6452,
+      "step": 180
+    },
+    {
+      "epoch": 0.2563237774030354,
+      "grad_norm": 3.9472005367279053,
+      "learning_rate": 0.0001834681042228212,
+      "loss": 0.6271,
+      "step": 190
+    },
+    {
+      "epoch": 0.26981450252951095,
+      "grad_norm": 6.640835762023926,
+      "learning_rate": 0.0001825696316262354,
+      "loss": 0.6161,
+      "step": 200
+    },
+    {
+      "epoch": 0.28330522765598654,
+      "grad_norm": 5.6504316329956055,
+      "learning_rate": 0.00018167115902964959,
+      "loss": 0.6452,
+      "step": 210
+    },
+    {
+      "epoch": 0.29679595278246207,
+      "grad_norm": 3.4219472408294678,
+      "learning_rate": 0.0001807726864330638,
+      "loss": 0.6228,
+      "step": 220
+    },
+    {
+      "epoch": 0.3102866779089376,
+      "grad_norm": 7.479050636291504,
+      "learning_rate": 0.000179874213836478,
+      "loss": 0.6035,
+      "step": 230
+    },
+    {
+      "epoch": 0.32377740303541314,
+      "grad_norm": 6.84128475189209,
+      "learning_rate": 0.0001789757412398922,
+      "loss": 0.5763,
+      "step": 240
+    },
+    {
+      "epoch": 0.3372681281618887,
+      "grad_norm": 3.407183885574341,
+      "learning_rate": 0.0001780772686433064,
+      "loss": 0.5981,
+      "step": 250
+    },
+    {
+      "epoch": 0.35075885328836426,
+      "grad_norm": 6.634527683258057,
+      "learning_rate": 0.0001771787960467206,
+      "loss": 0.6313,
+      "step": 260
+    },
+    {
+      "epoch": 0.3642495784148398,
+      "grad_norm": 6.274175643920898,
+      "learning_rate": 0.0001762803234501348,
+      "loss": 0.5971,
+      "step": 270
+    },
+    {
+      "epoch": 0.3777403035413153,
+      "grad_norm": 5.464880466461182,
+      "learning_rate": 0.00017538185085354897,
+      "loss": 0.629,
+      "step": 280
+    },
+    {
+      "epoch": 0.3912310286677909,
+      "grad_norm": 7.9421234130859375,
+      "learning_rate": 0.00017448337825696318,
+      "loss": 0.5604,
+      "step": 290
+    },
+    {
+      "epoch": 0.40472175379426645,
+      "grad_norm": 6.513714790344238,
+      "learning_rate": 0.00017358490566037738,
+      "loss": 0.5442,
+      "step": 300
+    },
+    {
+      "epoch": 0.418212478920742,
+      "grad_norm": 4.856393337249756,
+      "learning_rate": 0.00017268643306379155,
+      "loss": 0.5444,
+      "step": 310
+    },
+    {
+      "epoch": 0.4317032040472175,
+      "grad_norm": 7.252712249755859,
+      "learning_rate": 0.00017178796046720576,
+      "loss": 0.6071,
+      "step": 320
+    },
+    {
+      "epoch": 0.4451939291736931,
+      "grad_norm": 6.8953328132629395,
+      "learning_rate": 0.00017088948787061996,
+      "loss": 0.5622,
+      "step": 330
+    },
+    {
+      "epoch": 0.45868465430016864,
+      "grad_norm": 6.220505237579346,
+      "learning_rate": 0.00016999101527403416,
+      "loss": 0.6213,
+      "step": 340
+    },
+    {
+      "epoch": 0.47217537942664417,
+      "grad_norm": 1.9960452318191528,
+      "learning_rate": 0.00016909254267744833,
+      "loss": 0.5752,
+      "step": 350
+    },
+    {
+      "epoch": 0.4856661045531197,
+      "grad_norm": 4.8189778327941895,
+      "learning_rate": 0.00016819407008086254,
+      "loss": 0.5695,
+      "step": 360
+    },
+    {
+      "epoch": 0.4991568296795953,
+      "grad_norm": 2.157482624053955,
+      "learning_rate": 0.00016729559748427674,
+      "loss": 0.5818,
+      "step": 370
+    },
+    {
+      "epoch": 0.5126475548060708,
+      "grad_norm": 6.034278392791748,
+      "learning_rate": 0.00016639712488769091,
+      "loss": 0.5632,
+      "step": 380
+    },
+    {
+      "epoch": 0.5261382799325464,
+      "grad_norm": 3.6643030643463135,
+      "learning_rate": 0.00016549865229110512,
+      "loss": 0.5149,
+      "step": 390
+    },
+    {
+      "epoch": 0.5396290050590219,
+      "grad_norm": 3.4874682426452637,
+      "learning_rate": 0.00016460017969451932,
+      "loss": 0.5712,
+      "step": 400
+    },
+    {
+      "epoch": 0.5531197301854974,
+      "grad_norm": 5.94505500793457,
+      "learning_rate": 0.00016370170709793352,
+      "loss": 0.5579,
+      "step": 410
+    },
+    {
+      "epoch": 0.5666104553119731,
+      "grad_norm": 2.6858978271484375,
+      "learning_rate": 0.00016280323450134772,
+      "loss": 0.5601,
+      "step": 420
+    },
+    {
+      "epoch": 0.5801011804384486,
+      "grad_norm": 0.5919632315635681,
+      "learning_rate": 0.00016190476190476192,
+      "loss": 0.5122,
+      "step": 430
+    },
+    {
+      "epoch": 0.5935919055649241,
+      "grad_norm": 9.723355293273926,
+      "learning_rate": 0.00016100628930817613,
+      "loss": 0.5408,
+      "step": 440
+    },
+    {
+      "epoch": 0.6070826306913997,
+      "grad_norm": 3.421534776687622,
+      "learning_rate": 0.0001601078167115903,
+      "loss": 0.5125,
+      "step": 450
+    },
+    {
+      "epoch": 0.6205733558178752,
+      "grad_norm": 1.350819706916809,
+      "learning_rate": 0.0001592093441150045,
+      "loss": 0.5443,
+      "step": 460
+    },
+    {
+      "epoch": 0.6340640809443507,
+      "grad_norm": 2.9618473052978516,
+      "learning_rate": 0.0001583108715184187,
+      "loss": 0.5792,
+      "step": 470
+    },
+    {
+      "epoch": 0.6475548060708263,
+      "grad_norm": 11.16501522064209,
+      "learning_rate": 0.00015741239892183288,
+      "loss": 0.5499,
+      "step": 480
+    },
+    {
+      "epoch": 0.6610455311973018,
+      "grad_norm": 2.7122786045074463,
+      "learning_rate": 0.00015651392632524708,
+      "loss": 0.589,
+      "step": 490
+    },
+    {
+      "epoch": 0.6745362563237775,
+      "grad_norm": 6.2460408210754395,
+      "learning_rate": 0.00015561545372866128,
+      "loss": 0.5422,
+      "step": 500
+    },
+    {
+      "epoch": 0.688026981450253,
+      "grad_norm": 12.943184852600098,
+      "learning_rate": 0.0001547169811320755,
+      "loss": 0.5431,
+      "step": 510
+    },
+    {
+      "epoch": 0.7015177065767285,
+      "grad_norm": 2.4169883728027344,
+      "learning_rate": 0.00015381850853548966,
+      "loss": 0.4984,
+      "step": 520
+    },
+    {
+      "epoch": 0.715008431703204,
+      "grad_norm": 5.108154773712158,
+      "learning_rate": 0.00015292003593890386,
+      "loss": 0.4795,
+      "step": 530
+    },
+    {
+      "epoch": 0.7284991568296796,
+      "grad_norm": 3.268669605255127,
+      "learning_rate": 0.00015202156334231807,
+      "loss": 0.4961,
+      "step": 540
+    },
+    {
+      "epoch": 0.7419898819561551,
+      "grad_norm": 1.7926982641220093,
+      "learning_rate": 0.00015112309074573224,
+      "loss": 0.5339,
+      "step": 550
+    },
+    {
+      "epoch": 0.7554806070826307,
+      "grad_norm": 3.6382663249969482,
+      "learning_rate": 0.00015022461814914644,
+      "loss": 0.4855,
+      "step": 560
+    },
+    {
+      "epoch": 0.7689713322091062,
+      "grad_norm": 8.025957107543945,
+      "learning_rate": 0.00014932614555256067,
+      "loss": 0.4901,
+      "step": 570
+    },
+    {
+      "epoch": 0.7824620573355818,
+      "grad_norm": 4.220882892608643,
+      "learning_rate": 0.00014842767295597485,
+      "loss": 0.5448,
+      "step": 580
+    },
+    {
+      "epoch": 0.7959527824620574,
+      "grad_norm": 6.42817497253418,
+      "learning_rate": 0.00014752920035938905,
+      "loss": 0.4969,
+      "step": 590
+    },
+    {
+      "epoch": 0.8094435075885329,
+      "grad_norm": 4.182610988616943,
+      "learning_rate": 0.00014663072776280325,
+      "loss": 0.4863,
+      "step": 600
+    },
+    {
+      "epoch": 0.8229342327150084,
+      "grad_norm": 3.687934637069702,
+      "learning_rate": 0.00014573225516621745,
+      "loss": 0.5162,
+      "step": 610
+    },
+    {
+      "epoch": 0.836424957841484,
+      "grad_norm": 3.8240537643432617,
+      "learning_rate": 0.00014483378256963163,
+      "loss": 0.5089,
+      "step": 620
+    },
+    {
+      "epoch": 0.8499156829679595,
+      "grad_norm": 3.3823423385620117,
+      "learning_rate": 0.00014393530997304583,
+      "loss": 0.5447,
+      "step": 630
+    },
+    {
+      "epoch": 0.863406408094435,
+      "grad_norm": 4.591256618499756,
+      "learning_rate": 0.00014303683737646003,
+      "loss": 0.5067,
+      "step": 640
+    },
+    {
+      "epoch": 0.8768971332209107,
+      "grad_norm": 4.414494514465332,
+      "learning_rate": 0.0001421383647798742,
+      "loss": 0.4858,
+      "step": 650
+    },
+    {
+      "epoch": 0.8903878583473862,
+      "grad_norm": 3.4649970531463623,
+      "learning_rate": 0.0001412398921832884,
+      "loss": 0.5243,
+      "step": 660
+    },
+    {
+      "epoch": 0.9038785834738617,
+      "grad_norm": 4.813696384429932,
+      "learning_rate": 0.0001403414195867026,
+      "loss": 0.51,
+      "step": 670
+    },
+    {
+      "epoch": 0.9173693086003373,
+      "grad_norm": 4.597005844116211,
+      "learning_rate": 0.00013944294699011681,
+      "loss": 0.5079,
+      "step": 680
+    },
+    {
+      "epoch": 0.9308600337268128,
+      "grad_norm": 4.291003704071045,
+      "learning_rate": 0.000138544474393531,
+      "loss": 0.4966,
+      "step": 690
+    },
+    {
+      "epoch": 0.9443507588532883,
+      "grad_norm": 3.8949058055877686,
+      "learning_rate": 0.0001376460017969452,
+      "loss": 0.5086,
+      "step": 700
+    },
+    {
+      "epoch": 0.9578414839797639,
+      "grad_norm": 4.685634613037109,
+      "learning_rate": 0.0001367475292003594,
+      "loss": 0.5005,
+      "step": 710
+    },
+    {
+      "epoch": 0.9713322091062394,
+      "grad_norm": 3.9277801513671875,
+      "learning_rate": 0.0001358490566037736,
+      "loss": 0.4759,
+      "step": 720
+    },
+    {
+      "epoch": 0.984822934232715,
+      "grad_norm": 4.3011345863342285,
+      "learning_rate": 0.0001349505840071878,
+      "loss": 0.4371,
+      "step": 730
+    },
+    {
+      "epoch": 0.9983136593591906,
+      "grad_norm": 3.370706796646118,
+      "learning_rate": 0.000134052111410602,
+      "loss": 0.4811,
+      "step": 740
+    },
+    {
+      "epoch": 1.0107925801011803,
+      "grad_norm": 5.607315540313721,
+      "learning_rate": 0.00013315363881401617,
+      "loss": 0.4648,
+      "step": 750
+    },
+    {
+      "epoch": 1.024283305227656,
+      "grad_norm": 3.9047558307647705,
+      "learning_rate": 0.00013225516621743038,
+      "loss": 0.4813,
+      "step": 760
+    },
+    {
+      "epoch": 1.0377740303541316,
+      "grad_norm": 3.686000108718872,
+      "learning_rate": 0.00013135669362084458,
+      "loss": 0.5189,
+      "step": 770
+    },
+    {
+      "epoch": 1.0512647554806072,
+      "grad_norm": 3.8528940677642822,
+      "learning_rate": 0.00013045822102425878,
+      "loss": 0.4761,
+      "step": 780
+    },
+    {
+      "epoch": 1.0647554806070827,
+      "grad_norm": 3.5714054107666016,
+      "learning_rate": 0.00012955974842767296,
+      "loss": 0.4845,
+      "step": 790
+    },
+    {
+      "epoch": 1.0782462057335582,
+      "grad_norm": 3.3482840061187744,
+      "learning_rate": 0.00012866127583108716,
+      "loss": 0.487,
+      "step": 800
+    },
+    {
+      "epoch": 1.0917369308600338,
+      "grad_norm": 4.3598127365112305,
+      "learning_rate": 0.00012776280323450136,
+      "loss": 0.4918,
+      "step": 810
+    },
+    {
+      "epoch": 1.1052276559865093,
+      "grad_norm": 3.4356210231781006,
+      "learning_rate": 0.00012686433063791554,
+      "loss": 0.5042,
+      "step": 820
+    },
+    {
+      "epoch": 1.1187183811129848,
+      "grad_norm": 3.015895366668701,
+      "learning_rate": 0.00012596585804132974,
+      "loss": 0.4911,
+      "step": 830
+    },
+    {
+      "epoch": 1.1322091062394604,
+      "grad_norm": 3.4911601543426514,
+      "learning_rate": 0.00012506738544474394,
+      "loss": 0.507,
+      "step": 840
+    },
+    {
+      "epoch": 1.145699831365936,
+      "grad_norm": 5.718015670776367,
+      "learning_rate": 0.00012416891284815814,
+      "loss": 0.4668,
+      "step": 850
+    },
+    {
+      "epoch": 1.1591905564924114,
+      "grad_norm": 4.448844909667969,
+      "learning_rate": 0.00012327044025157232,
+      "loss": 0.465,
+      "step": 860
+    },
+    {
+      "epoch": 1.172681281618887,
+      "grad_norm": 3.243500232696533,
+      "learning_rate": 0.00012237196765498652,
+      "loss": 0.4922,
+      "step": 870
+    },
+    {
+      "epoch": 1.1861720067453625,
+      "grad_norm": 3.25435471534729,
+      "learning_rate": 0.00012147349505840073,
+      "loss": 0.4825,
+      "step": 880
+    },
+    {
+      "epoch": 1.199662731871838,
+      "grad_norm": 3.0708346366882324,
+      "learning_rate": 0.00012057502246181491,
+      "loss": 0.4777,
+      "step": 890
+    },
+    {
+      "epoch": 1.2131534569983136,
+      "grad_norm": 3.274693727493286,
+      "learning_rate": 0.00011967654986522911,
+      "loss": 0.4727,
+      "step": 900
+    },
+    {
+      "epoch": 1.2266441821247893,
+      "grad_norm": 5.614912986755371,
+      "learning_rate": 0.00011877807726864331,
+      "loss": 0.4914,
+      "step": 910
+    },
+    {
+      "epoch": 1.2401349072512649,
+      "grad_norm": 3.7809228897094727,
+      "learning_rate": 0.0001178796046720575,
+      "loss": 0.4727,
+      "step": 920
+    },
+    {
+      "epoch": 1.2536256323777404,
+      "grad_norm": 3.46541428565979,
+      "learning_rate": 0.0001169811320754717,
+      "loss": 0.4799,
+      "step": 930
+    },
+    {
+      "epoch": 1.267116357504216,
+      "grad_norm": 3.8525471687316895,
+      "learning_rate": 0.0001160826594788859,
+      "loss": 0.5079,
+      "step": 940
+    },
+    {
+      "epoch": 1.2806070826306915,
+      "grad_norm": 3.959237575531006,
+      "learning_rate": 0.00011518418688230011,
+      "loss": 0.4689,
+      "step": 950
+    },
+    {
+      "epoch": 1.294097807757167,
+      "grad_norm": 3.873572826385498,
+      "learning_rate": 0.00011428571428571428,
+      "loss": 0.4688,
+      "step": 960
+    },
+    {
+      "epoch": 1.3075885328836425,
+      "grad_norm": 3.813100814819336,
+      "learning_rate": 0.00011338724168912849,
+      "loss": 0.4762,
+      "step": 970
+    },
+    {
+      "epoch": 1.321079258010118,
+      "grad_norm": 3.517953634262085,
+      "learning_rate": 0.00011248876909254269,
+      "loss": 0.4595,
+      "step": 980
+    },
+    {
+      "epoch": 1.3345699831365936,
+      "grad_norm": 3.5959513187408447,
+      "learning_rate": 0.00011159029649595688,
+      "loss": 0.4959,
+      "step": 990
+    },
+    {
+      "epoch": 1.3480607082630691,
+      "grad_norm": 3.9473278522491455,
+      "learning_rate": 0.00011069182389937108,
+      "loss": 0.4552,
+      "step": 1000
+    },
+    {
+      "epoch": 1.3615514333895447,
+      "grad_norm": 3.5634729862213135,
+      "learning_rate": 0.00010979335130278528,
+      "loss": 0.4765,
+      "step": 1010
+    },
+    {
+      "epoch": 1.3750421585160202,
+      "grad_norm": 3.88102126121521,
+      "learning_rate": 0.00010889487870619948,
+      "loss": 0.4282,
+      "step": 1020
+    },
+    {
+      "epoch": 1.3885328836424957,
+      "grad_norm": 4.591094970703125,
+      "learning_rate": 0.00010799640610961366,
+      "loss": 0.483,
+      "step": 1030
+    },
+    {
+      "epoch": 1.4020236087689713,
+      "grad_norm": 2.726210355758667,
+      "learning_rate": 0.00010709793351302786,
+      "loss": 0.5491,
+      "step": 1040
+    },
+    {
+      "epoch": 1.4155143338954468,
+      "grad_norm": 2.8899638652801514,
+      "learning_rate": 0.00010619946091644206,
+      "loss": 0.4494,
+      "step": 1050
+    },
+    {
+      "epoch": 1.4290050590219225,
+      "grad_norm": 3.1624226570129395,
+      "learning_rate": 0.00010530098831985624,
+      "loss": 0.4861,
+      "step": 1060
+    },
+    {
+      "epoch": 1.4424957841483979,
+      "grad_norm": 4.227782726287842,
+      "learning_rate": 0.00010440251572327044,
+      "loss": 0.4771,
+      "step": 1070
+    },
+    {
+      "epoch": 1.4559865092748736,
+      "grad_norm": 3.1170289516448975,
+      "learning_rate": 0.00010350404312668464,
+      "loss": 0.4608,
+      "step": 1080
+    },
+    {
+      "epoch": 1.4694772344013491,
+      "grad_norm": 3.7711753845214844,
+      "learning_rate": 0.00010260557053009883,
+      "loss": 0.4553,
+      "step": 1090
+    },
+    {
+      "epoch": 1.4829679595278247,
+      "grad_norm": 3.723172187805176,
+      "learning_rate": 0.00010170709793351303,
+      "loss": 0.4347,
+      "step": 1100
+    },
+    {
+      "epoch": 1.4964586846543002,
+      "grad_norm": 3.469057083129883,
+      "learning_rate": 0.00010080862533692723,
+      "loss": 0.4747,
+      "step": 1110
+    },
+    {
+      "epoch": 1.5099494097807757,
+      "grad_norm": 3.3788669109344482,
+      "learning_rate": 9.991015274034142e-05,
+      "loss": 0.4788,
+      "step": 1120
+    },
+    {
+      "epoch": 1.5234401349072513,
+      "grad_norm": 5.321389198303223,
+      "learning_rate": 9.901168014375562e-05,
+      "loss": 0.4424,
+      "step": 1130
+    },
+    {
+      "epoch": 1.5369308600337268,
+      "grad_norm": 3.616950511932373,
+      "learning_rate": 9.811320754716981e-05,
+      "loss": 0.4765,
+      "step": 1140
+    },
+    {
+      "epoch": 1.5504215851602023,
+      "grad_norm": 6.907515525817871,
+      "learning_rate": 9.7214734950584e-05,
+      "loss": 0.4797,
+      "step": 1150
+    },
+    {
+      "epoch": 1.5639123102866779,
+      "grad_norm": 17.670085906982422,
+      "learning_rate": 9.63162623539982e-05,
+      "loss": 0.4531,
+      "step": 1160
+    },
+    {
+      "epoch": 1.5774030354131534,
+      "grad_norm": 2.698268413543701,
+      "learning_rate": 9.54177897574124e-05,
+      "loss": 0.493,
+      "step": 1170
+    },
+    {
+      "epoch": 1.590893760539629,
+      "grad_norm": 6.46640682220459,
+      "learning_rate": 9.451931716082661e-05,
+      "loss": 0.4616,
+      "step": 1180
+    },
+    {
+      "epoch": 1.6043844856661047,
+      "grad_norm": 2.7934491634368896,
+      "learning_rate": 9.36208445642408e-05,
+      "loss": 0.4819,
+      "step": 1190
+    },
+    {
+      "epoch": 1.61787521079258,
+      "grad_norm": 4.599141597747803,
+      "learning_rate": 9.272237196765498e-05,
+      "loss": 0.4506,
+      "step": 1200
+    },
+    {
+      "epoch": 1.6313659359190558,
+      "grad_norm": 3.3172645568847656,
+      "learning_rate": 9.182389937106919e-05,
+      "loss": 0.477,
+      "step": 1210
+    },
+    {
+      "epoch": 1.644856661045531,
+      "grad_norm": 3.431093215942383,
+      "learning_rate": 9.092542677448338e-05,
+      "loss": 0.4594,
+      "step": 1220
+    },
+    {
+      "epoch": 1.6583473861720068,
+      "grad_norm": 3.2188451290130615,
+      "learning_rate": 9.002695417789758e-05,
+      "loss": 0.4311,
+      "step": 1230
+    },
+    {
+      "epoch": 1.6718381112984821,
+      "grad_norm": 4.311905384063721,
+      "learning_rate": 8.912848158131178e-05,
+      "loss": 0.4608,
+      "step": 1240
+    },
+    {
+      "epoch": 1.685328836424958,
+      "grad_norm": 2.9855377674102783,
+      "learning_rate": 8.823000898472597e-05,
+      "loss": 0.5016,
+      "step": 1250
+    },
+    {
+      "epoch": 1.6988195615514334,
+      "grad_norm": 4.071963310241699,
+      "learning_rate": 8.733153638814017e-05,
+      "loss": 0.4684,
+      "step": 1260
+    },
+    {
+      "epoch": 1.712310286677909,
+      "grad_norm": 3.417339563369751,
+      "learning_rate": 8.643306379155436e-05,
+      "loss": 0.4364,
+      "step": 1270
+    },
+    {
+      "epoch": 1.7258010118043845,
+      "grad_norm": 3.9395434856414795,
+      "learning_rate": 8.553459119496856e-05,
+      "loss": 0.5122,
+      "step": 1280
+    },
+    {
+      "epoch": 1.73929173693086,
+      "grad_norm": 2.934011220932007,
+      "learning_rate": 8.463611859838275e-05,
+      "loss": 0.4572,
+      "step": 1290
+    },
+    {
+      "epoch": 1.7527824620573356,
+      "grad_norm": 3.452453851699829,
+      "learning_rate": 8.373764600179695e-05,
+      "loss": 0.4674,
+      "step": 1300
+    },
+    {
+      "epoch": 1.766273187183811,
+      "grad_norm": 3.5324788093566895,
+      "learning_rate": 8.283917340521114e-05,
+      "loss": 0.4678,
+      "step": 1310
+    },
+    {
+      "epoch": 1.7797639123102866,
+      "grad_norm": 3.4225003719329834,
+      "learning_rate": 8.194070080862534e-05,
+      "loss": 0.5027,
+      "step": 1320
+    },
+    {
+      "epoch": 1.7932546374367622,
+      "grad_norm": 1.9840672016143799,
+      "learning_rate": 8.104222821203954e-05,
+      "loss": 0.4977,
+      "step": 1330
+    },
+    {
+      "epoch": 1.806745362563238,
+      "grad_norm": 2.443516492843628,
+      "learning_rate": 8.014375561545373e-05,
+      "loss": 0.455,
+      "step": 1340
+    },
+    {
+      "epoch": 1.8202360876897132,
+      "grad_norm": 2.689674139022827,
+      "learning_rate": 7.924528301886794e-05,
+      "loss": 0.4665,
+      "step": 1350
+    },
+    {
+      "epoch": 1.833726812816189,
+      "grad_norm": 3.2184505462646484,
+      "learning_rate": 7.834681042228212e-05,
+      "loss": 0.475,
+      "step": 1360
+    },
+    {
+      "epoch": 1.8472175379426643,
+      "grad_norm": 3.5544068813323975,
+      "learning_rate": 7.744833782569631e-05,
+      "loss": 0.4921,
+      "step": 1370
+    },
+    {
+      "epoch": 1.86070826306914,
+      "grad_norm": 3.2145285606384277,
+      "learning_rate": 7.654986522911051e-05,
+      "loss": 0.4825,
+      "step": 1380
+    },
+    {
+      "epoch": 1.8741989881956154,
+      "grad_norm": 3.448352813720703,
+      "learning_rate": 7.56513926325247e-05,
+      "loss": 0.4442,
+      "step": 1390
+    },
+    {
+      "epoch": 1.8876897133220911,
+      "grad_norm": 3.329658269882202,
+      "learning_rate": 7.47529200359389e-05,
+      "loss": 0.4498,
+      "step": 1400
+    },
+    {
+      "epoch": 1.9011804384485667,
+      "grad_norm": 2.8206887245178223,
+      "learning_rate": 7.385444743935311e-05,
+      "loss": 0.4525,
+      "step": 1410
+    },
+    {
+      "epoch": 1.9146711635750422,
+      "grad_norm": 3.067368507385254,
+      "learning_rate": 7.29559748427673e-05,
+      "loss": 0.4652,
+      "step": 1420
+    },
+    {
+      "epoch": 1.9281618887015177,
+      "grad_norm": 2.902029275894165,
+      "learning_rate": 7.20575022461815e-05,
+      "loss": 0.462,
+      "step": 1430
+    },
+    {
+      "epoch": 1.9416526138279933,
+      "grad_norm": 2.9392433166503906,
+      "learning_rate": 7.115902964959569e-05,
+      "loss": 0.4505,
+      "step": 1440
+    },
+    {
+      "epoch": 1.9551433389544688,
+      "grad_norm": 3.328568935394287,
+      "learning_rate": 7.026055705300989e-05,
+      "loss": 0.4688,
+      "step": 1450
+    },
+    {
+      "epoch": 1.9686340640809443,
+      "grad_norm": 3.0431363582611084,
+      "learning_rate": 6.936208445642408e-05,
+      "loss": 0.4749,
+      "step": 1460
+    },
+    {
+      "epoch": 1.9821247892074199,
+      "grad_norm": 3.3194100856781006,
+      "learning_rate": 6.846361185983828e-05,
+      "loss": 0.4364,
+      "step": 1470
+    },
+    {
+      "epoch": 1.9956155143338954,
+      "grad_norm": 3.563108205795288,
+      "learning_rate": 6.756513926325248e-05,
+      "loss": 0.4635,
+      "step": 1480
+    },
+    {
+      "epoch": 2.0080944350758854,
+      "grad_norm": 2.6294658184051514,
+      "learning_rate": 6.666666666666667e-05,
+      "loss": 0.4253,
+      "step": 1490
+    },
+    {
+      "epoch": 2.0215851602023607,
+      "grad_norm": 2.6223340034484863,
+      "learning_rate": 6.576819407008087e-05,
+      "loss": 0.4534,
+      "step": 1500
+    },
+    {
+      "epoch": 2.0350758853288364,
+      "grad_norm": 2.769744396209717,
+      "learning_rate": 6.486972147349506e-05,
+      "loss": 0.4168,
+      "step": 1510
+    },
+    {
+      "epoch": 2.048566610455312,
+      "grad_norm": 2.9616827964782715,
+      "learning_rate": 6.397124887690926e-05,
+      "loss": 0.4464,
+      "step": 1520
+    },
+    {
+      "epoch": 2.0620573355817875,
+      "grad_norm": 2.952134132385254,
+      "learning_rate": 6.307277628032345e-05,
+      "loss": 0.4814,
+      "step": 1530
+    },
+    {
+      "epoch": 2.0755480607082633,
+      "grad_norm": 3.6748759746551514,
+      "learning_rate": 6.217430368373764e-05,
+      "loss": 0.4638,
+      "step": 1540
+    },
+    {
+      "epoch": 2.0890387858347386,
+      "grad_norm": 1.927171230316162,
+      "learning_rate": 6.127583108715184e-05,
+      "loss": 0.4512,
+      "step": 1550
+    },
+    {
+      "epoch": 2.1025295109612143,
+      "grad_norm": 2.6556429862976074,
+      "learning_rate": 6.037735849056604e-05,
+      "loss": 0.4752,
+      "step": 1560
+    },
+    {
+      "epoch": 2.1160202360876896,
+      "grad_norm": 1.954484462738037,
+      "learning_rate": 5.947888589398024e-05,
+      "loss": 0.4347,
+      "step": 1570
+    },
+    {
+      "epoch": 2.1295109612141654,
+      "grad_norm": 2.986450433731079,
+      "learning_rate": 5.8580413297394435e-05,
+      "loss": 0.4278,
+      "step": 1580
+    },
+    {
+      "epoch": 2.1430016863406407,
+      "grad_norm": 2.7762913703918457,
+      "learning_rate": 5.768194070080862e-05,
+      "loss": 0.4496,
+      "step": 1590
+    },
+    {
+      "epoch": 2.1564924114671165,
+      "grad_norm": 3.0169496536254883,
+      "learning_rate": 5.6783468104222825e-05,
+      "loss": 0.4359,
+      "step": 1600
+    },
+    {
+      "epoch": 2.1699831365935918,
+      "grad_norm": 2.6418251991271973,
+      "learning_rate": 5.5884995507637014e-05,
+      "loss": 0.4512,
+      "step": 1610
+    },
+    {
+      "epoch": 2.1834738617200675,
+      "grad_norm": 2.8345439434051514,
+      "learning_rate": 5.4986522911051216e-05,
+      "loss": 0.4339,
+      "step": 1620
+    },
+    {
+      "epoch": 2.196964586846543,
+      "grad_norm": 2.8585574626922607,
+      "learning_rate": 5.408805031446541e-05,
+      "loss": 0.4072,
+      "step": 1630
+    },
+    {
+      "epoch": 2.2104553119730186,
+      "grad_norm": 1.8751145601272583,
+      "learning_rate": 5.31895777178796e-05,
+      "loss": 0.4526,
+      "step": 1640
+    },
+    {
+      "epoch": 2.223946037099494,
+      "grad_norm": 2.441622734069824,
+      "learning_rate": 5.22911051212938e-05,
+      "loss": 0.4758,
+      "step": 1650
+    },
+    {
+      "epoch": 2.2374367622259697,
+      "grad_norm": 3.11625337600708,
+      "learning_rate": 5.1392632524708e-05,
+      "loss": 0.4705,
+      "step": 1660
+    },
+    {
+      "epoch": 2.2509274873524454,
+      "grad_norm": 2.761462688446045,
+      "learning_rate": 5.04941599281222e-05,
+      "loss": 0.4439,
+      "step": 1670
+    },
+    {
+      "epoch": 2.2644182124789207,
+      "grad_norm": 2.8460707664489746,
+      "learning_rate": 4.959568733153639e-05,
+      "loss": 0.4595,
+      "step": 1680
+    },
+    {
+      "epoch": 2.2779089376053965,
+      "grad_norm": 2.73833966255188,
+      "learning_rate": 4.869721473495058e-05,
+      "loss": 0.436,
+      "step": 1690
+    },
+    {
+      "epoch": 2.291399662731872,
+      "grad_norm": 3.1314399242401123,
+      "learning_rate": 4.7798742138364785e-05,
+      "loss": 0.4302,
+      "step": 1700
+    },
+    {
+      "epoch": 2.3048903878583475,
+      "grad_norm": 2.203801155090332,
+      "learning_rate": 4.690026954177898e-05,
+      "loss": 0.4353,
+      "step": 1710
+    },
+    {
+      "epoch": 2.318381112984823,
+      "grad_norm": 1.5654208660125732,
+      "learning_rate": 4.6001796945193176e-05,
+      "loss": 0.4436,
+      "step": 1720
+    },
+    {
+      "epoch": 2.3318718381112986,
+      "grad_norm": 1.8968826532363892,
+      "learning_rate": 4.5103324348607365e-05,
+      "loss": 0.4617,
+      "step": 1730
+    },
+    {
+      "epoch": 2.345362563237774,
+      "grad_norm": 3.4319088459014893,
+      "learning_rate": 4.420485175202157e-05,
+      "loss": 0.4951,
+      "step": 1740
+    },
+    {
+      "epoch": 2.3588532883642497,
+      "grad_norm": 0.9367527365684509,
+      "learning_rate": 4.330637915543576e-05,
+      "loss": 0.4469,
+      "step": 1750
+    },
+    {
+      "epoch": 2.372344013490725,
+      "grad_norm": 1.3041300773620605,
+      "learning_rate": 4.240790655884996e-05,
+      "loss": 0.4696,
+      "step": 1760
+    },
+    {
+      "epoch": 2.3858347386172007,
+      "grad_norm": 1.0405113697052002,
+      "learning_rate": 4.150943396226415e-05,
+      "loss": 0.4121,
+      "step": 1770
+    },
+    {
+      "epoch": 2.399325463743676,
+      "grad_norm": 1.1513006687164307,
+      "learning_rate": 4.061096136567835e-05,
+      "loss": 0.4134,
+      "step": 1780
+    },
+    {
+      "epoch": 2.412816188870152,
+      "grad_norm": 1.3874084949493408,
+      "learning_rate": 3.971248876909254e-05,
+      "loss": 0.4375,
+      "step": 1790
+    },
+    {
+      "epoch": 2.426306913996627,
+      "grad_norm": 1.29698646068573,
+      "learning_rate": 3.881401617250674e-05,
+      "loss": 0.4752,
+      "step": 1800
+    },
+    {
+      "epoch": 2.439797639123103,
+      "grad_norm": 0.8246452808380127,
+      "learning_rate": 3.7915543575920934e-05,
+      "loss": 0.4738,
+      "step": 1810
+    },
+    {
+      "epoch": 2.4532883642495786,
+      "grad_norm": 1.246367335319519,
+      "learning_rate": 3.7017070979335136e-05,
+      "loss": 0.4583,
+      "step": 1820
+    },
+    {
+      "epoch": 2.466779089376054,
+      "grad_norm": 2.288325071334839,
+      "learning_rate": 3.611859838274933e-05,
+      "loss": 0.4643,
+      "step": 1830
+    },
+    {
+      "epoch": 2.4802698145025297,
+      "grad_norm": 2.6386337280273438,
+      "learning_rate": 3.522012578616352e-05,
+      "loss": 0.4748,
+      "step": 1840
+    },
+    {
+      "epoch": 2.493760539629005,
+      "grad_norm": 2.788741111755371,
+      "learning_rate": 3.4321653189577715e-05,
+      "loss": 0.4623,
+      "step": 1850
+    },
+    {
+      "epoch": 2.5072512647554808,
+      "grad_norm": 3.7216169834136963,
+      "learning_rate": 3.342318059299192e-05,
+      "loss": 0.441,
+      "step": 1860
+    },
+    {
+      "epoch": 2.520741989881956,
+      "grad_norm": 2.007139205932617,
+      "learning_rate": 3.252470799640611e-05,
+      "loss": 0.4383,
+      "step": 1870
+    },
+    {
+      "epoch": 2.534232715008432,
+      "grad_norm": 0.6199157238006592,
+      "learning_rate": 3.162623539982031e-05,
+      "loss": 0.4197,
+      "step": 1880
+    },
+    {
+      "epoch": 2.547723440134907,
+      "grad_norm": 0.895648181438446,
+      "learning_rate": 3.0727762803234503e-05,
+      "loss": 0.4403,
+      "step": 1890
+    },
+    {
+      "epoch": 2.561214165261383,
+      "grad_norm": 0.6603314280509949,
+      "learning_rate": 2.9829290206648695e-05,
+      "loss": 0.4794,
+      "step": 1900
+    },
+    {
+      "epoch": 2.574704890387858,
+      "grad_norm": 1.2536957263946533,
+      "learning_rate": 2.8930817610062894e-05,
+      "loss": 0.4778,
+      "step": 1910
+    },
+    {
+      "epoch": 2.588195615514334,
+      "grad_norm": 0.6780521869659424,
+      "learning_rate": 2.803234501347709e-05,
+      "loss": 0.4672,
+      "step": 1920
+    },
+    {
+      "epoch": 2.6016863406408093,
+      "grad_norm": 0.6663461327552795,
+      "learning_rate": 2.7133872416891288e-05,
+      "loss": 0.4212,
+      "step": 1930
+    },
+    {
+      "epoch": 2.615177065767285,
+      "grad_norm": 1.3189208507537842,
+      "learning_rate": 2.6235399820305483e-05,
+      "loss": 0.4366,
+      "step": 1940
+    },
+    {
+      "epoch": 2.6286677908937603,
+      "grad_norm": 0.6126547455787659,
+      "learning_rate": 2.5336927223719675e-05,
+      "loss": 0.4467,
+      "step": 1950
+    },
+    {
+      "epoch": 2.642158516020236,
+      "grad_norm": 0.8297685980796814,
+      "learning_rate": 2.4438454627133874e-05,
+      "loss": 0.4203,
+      "step": 1960
+    },
+    {
+      "epoch": 2.655649241146712,
+      "grad_norm": 0.81744784116745,
+      "learning_rate": 2.353998203054807e-05,
+      "loss": 0.4714,
+      "step": 1970
+    },
+    {
+      "epoch": 2.669139966273187,
+      "grad_norm": 1.0496429204940796,
+      "learning_rate": 2.2641509433962265e-05,
+      "loss": 0.4517,
+      "step": 1980
+    },
+    {
+      "epoch": 2.6826306913996625,
+      "grad_norm": 0.843941330909729,
+      "learning_rate": 2.174303683737646e-05,
+      "loss": 0.4361,
+      "step": 1990
+    },
+    {
+      "epoch": 2.6961214165261382,
+      "grad_norm": 0.7860977649688721,
+      "learning_rate": 2.0844564240790655e-05,
+      "loss": 0.4061,
+      "step": 2000
+    },
+    {
+      "epoch": 2.709612141652614,
+      "grad_norm": 1.3650184869766235,
+      "learning_rate": 1.9946091644204854e-05,
+      "loss": 0.4641,
+      "step": 2010
+    },
+    {
+      "epoch": 2.7231028667790893,
+      "grad_norm": 0.623533308506012,
+      "learning_rate": 1.9047619047619046e-05,
+      "loss": 0.4296,
+      "step": 2020
+    },
+    {
+      "epoch": 2.736593591905565,
+      "grad_norm": 1.0236366987228394,
+      "learning_rate": 1.8149146451033245e-05,
+      "loss": 0.449,
+      "step": 2030
+    },
+    {
+      "epoch": 2.7500843170320404,
+      "grad_norm": 0.6305617690086365,
+      "learning_rate": 1.725067385444744e-05,
+      "loss": 0.4888,
+      "step": 2040
+    },
+    {
+      "epoch": 2.763575042158516,
+      "grad_norm": 0.5004448294639587,
+      "learning_rate": 1.6352201257861635e-05,
+      "loss": 0.4227,
+      "step": 2050
+    },
+    {
+      "epoch": 2.7770657672849914,
+      "grad_norm": 0.6496240496635437,
+      "learning_rate": 1.545372866127583e-05,
+      "loss": 0.4499,
+      "step": 2060
+    },
+    {
+      "epoch": 2.790556492411467,
+      "grad_norm": 0.9616618752479553,
+      "learning_rate": 1.455525606469003e-05,
+      "loss": 0.4368,
+      "step": 2070
+    },
+    {
+      "epoch": 2.8040472175379425,
+      "grad_norm": 0.5831645727157593,
+      "learning_rate": 1.3656783468104223e-05,
+      "loss": 0.4452,
+      "step": 2080
+    },
+    {
+      "epoch": 2.8175379426644183,
+      "grad_norm": 0.8784683346748352,
+      "learning_rate": 1.275831087151842e-05,
+      "loss": 0.4698,
+      "step": 2090
+    },
+    {
+      "epoch": 2.8310286677908936,
+      "grad_norm": 1.3440258502960205,
+      "learning_rate": 1.1859838274932616e-05,
+      "loss": 0.4195,
+      "step": 2100
+    },
+    {
+      "epoch": 2.8445193929173693,
+      "grad_norm": 1.2359696626663208,
+      "learning_rate": 1.0961365678346811e-05,
+      "loss": 0.4305,
+      "step": 2110
+    },
+    {
+      "epoch": 2.858010118043845,
+      "grad_norm": 1.1054702997207642,
+      "learning_rate": 1.0062893081761008e-05,
+      "loss": 0.4649,
+      "step": 2120
+    },
+    {
+      "epoch": 2.8715008431703204,
+      "grad_norm": 1.379691481590271,
+      "learning_rate": 9.164420485175203e-06,
+      "loss": 0.4876,
+      "step": 2130
+    },
+    {
+      "epoch": 2.8849915682967957,
+      "grad_norm": 1.6011934280395508,
+      "learning_rate": 8.265947888589399e-06,
+      "loss": 0.4759,
+      "step": 2140
+    },
+    {
+      "epoch": 2.8984822934232715,
+      "grad_norm": 1.1230287551879883,
+      "learning_rate": 7.367475292003594e-06,
+      "loss": 0.4569,
+      "step": 2150
+    },
+    {
+      "epoch": 2.911973018549747,
+      "grad_norm": 0.5642672181129456,
+      "learning_rate": 6.46900269541779e-06,
+      "loss": 0.4302,
+      "step": 2160
+    },
+    {
+      "epoch": 2.9254637436762225,
+      "grad_norm": 0.5524741411209106,
+      "learning_rate": 5.570530098831986e-06,
+      "loss": 0.4062,
+      "step": 2170
+    },
+    {
+      "epoch": 2.9389544688026983,
+      "grad_norm": 0.5504957437515259,
+      "learning_rate": 4.6720575022461816e-06,
+      "loss": 0.4336,
+      "step": 2180
+    },
+    {
+      "epoch": 2.9524451939291736,
+      "grad_norm": 0.9138615131378174,
+      "learning_rate": 3.7735849056603773e-06,
+      "loss": 0.4463,
+      "step": 2190
+    },
+    {
+      "epoch": 2.9659359190556494,
+      "grad_norm": 0.5804033875465393,
+      "learning_rate": 2.8751123090745735e-06,
+      "loss": 0.4323,
+      "step": 2200
+    },
+    {
+      "epoch": 2.9794266441821247,
+      "grad_norm": 0.8893300890922546,
+      "learning_rate": 1.9766397124887693e-06,
+      "loss": 0.4244,
+      "step": 2210
+    },
+    {
+      "epoch": 2.9929173693086004,
+      "grad_norm": 0.6132921576499939,
+      "learning_rate": 1.078167115902965e-06,
+      "loss": 0.4562,
+      "step": 2220
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 2226,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.107103129778258e+17,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}
diff --git a/checkpoint-2226/training_args.bin b/checkpoint-2226/training_args.bin
new file mode 100644
index 0000000000000000000000000000000000000000..597bbc184c73a1ca565444bfc5a1a6c505b47b83
--- /dev/null
+++ b/checkpoint-2226/training_args.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:160a1045610f1057c5a1c808e36986bd0969b8e2c1163c803dfd9b9a2891eb12
+size 5304
diff --git a/checkpoint-742/README.md b/checkpoint-742/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..d44d68a82ab48884e6afee364005e8a561c49c7c
--- /dev/null
+++ b/checkpoint-742/README.md
@@ -0,0 +1,202 @@
+---
+base_model: ModelSpace/GemmaX2-28-2B-v0.1
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.2
\ No newline at end of file
diff --git a/checkpoint-742/adapter_config.json b/checkpoint-742/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..191461bdeecd0575bb2844135920827ba4f2c74c
--- /dev/null
+++ b/checkpoint-742/adapter_config.json
@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "ModelSpace/GemmaX2-28-2B-v0.1",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}
\ No newline at end of file
diff --git a/checkpoint-742/adapter_model.safetensors b/checkpoint-742/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..12078f732d3dd1d7133ef243cea682c52ab07cb4
--- /dev/null
+++ b/checkpoint-742/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4569b6378fc065d43d2e895f058c775e07841121f72f9236007dee5d12802634
+size 6403448
diff --git a/checkpoint-742/chat_template.jinja b/checkpoint-742/chat_template.jinja
new file mode 100644
index 0000000000000000000000000000000000000000..127d9d1acfd2092b5b5d74ea3fe2626658923ada
--- /dev/null
+++ b/checkpoint-742/chat_template.jinja
@@ -0,0 +1 @@
+{% if messages[0]['role'] == 'system' %}{% set system_message = messages[0]['content'] %}{% endif %}{% if system_message is defined %}{{ system_message }}{% endif %}{% for message in messages %}{% set content = message['content'] %}{% if message['role'] == 'user' %}{{ content }}{% elif message['role'] == 'assistant' %}{{ content }}{% endif %}{% endfor %}
\ No newline at end of file
diff --git a/checkpoint-742/optimizer.pt b/checkpoint-742/optimizer.pt
new file mode 100644
index 0000000000000000000000000000000000000000..8d909451c9a7b9396d7d49c309fcfeddc9f1969d
--- /dev/null
+++ b/checkpoint-742/optimizer.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:26a1661207537db42a03524ec42b93f89291cc99a8e986b291e93e33045dc9d7
+size 12867066
diff --git a/checkpoint-742/rng_state.pth b/checkpoint-742/rng_state.pth
new file mode 100644
index 0000000000000000000000000000000000000000..b8b3f988b8fe1a0feb8eba9985e23809c26320e5
--- /dev/null
+++ b/checkpoint-742/rng_state.pth
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9fef553d726535b4661f010d4f55724073c08281643c9c52f60b8a855d23f2a5
+size 14244
diff --git a/checkpoint-742/scaler.pt b/checkpoint-742/scaler.pt
new file mode 100644
index 0000000000000000000000000000000000000000..0009aa47acb6272ecb5e40164deee2ac2822d988
--- /dev/null
+++ b/checkpoint-742/scaler.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:27e67248c927f121ec94d7ff69d9f92e941962df3005605b41ae8b40e0846cd3
+size 988
diff --git a/checkpoint-742/scheduler.pt b/checkpoint-742/scheduler.pt
new file mode 100644
index 0000000000000000000000000000000000000000..2c69c5b0ba6d9a7ca9eb86dc8b593d4c16ee11c9
--- /dev/null
+++ b/checkpoint-742/scheduler.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f84affbd25205884be9c0ccaffe26202497bd16a41e0d391fd97202f234b36
+size 1064
diff --git a/checkpoint-742/special_tokens_map.json b/checkpoint-742/special_tokens_map.json
new file mode 100644
index 0000000000000000000000000000000000000000..f6119589e367b2de0fc8cbd2f1217667532e3174
--- /dev/null
+++ b/checkpoint-742/special_tokens_map.json
@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<bos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<eos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}
diff --git a/checkpoint-742/tokenizer.json b/checkpoint-742/tokenizer.json
new file mode 100644
index 0000000000000000000000000000000000000000..2b2d51a12a27cc5bd1baeaca3ec01d6f6f458f64
--- /dev/null
+++ b/checkpoint-742/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:afe09ee4dcd6379dfa9f12c0178f86eab033fc9f97f4e1d605a44da3488b6c53
+size 34363138
diff --git a/checkpoint-742/tokenizer.model b/checkpoint-742/tokenizer.model
new file mode 100644
index 0000000000000000000000000000000000000000..796efe9ab515c15e146ce7588e6d7b9b8134dbf8
--- /dev/null
+++ b/checkpoint-742/tokenizer.model
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61a7b147390c64585d6c3543dd6fc636906c9af3865a5548f27f31aee1d4c8e2
+size 4241003
diff --git a/checkpoint-742/tokenizer_config.json b/checkpoint-742/tokenizer_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..62e63dbcee5400f4f34987854723a5120c4cc076
--- /dev/null
+++ b/checkpoint-742/tokenizer_config.json
@@ -0,0 +1,2011 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<eos>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "<bos>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "4": {
+      "content": "<mask>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "5": {
+      "content": "<2mass>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "6": {
+      "content": "[@BOS@]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "7": {
+      "content": "<unused0>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "8": {
+      "content": "<unused1>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "9": {
+      "content": "<unused2>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "10": {
+      "content": "<unused3>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "11": {
+      "content": "<unused4>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "12": {
+      "content": "<unused5>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "13": {
+      "content": "<unused6>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "14": {
+      "content": "<unused7>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "15": {
+      "content": "<unused8>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "16": {
+      "content": "<unused9>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "17": {
+      "content": "<unused10>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "18": {
+      "content": "<unused11>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "19": {
+      "content": "<unused12>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "20": {
+      "content": "<unused13>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "21": {
+      "content": "<unused14>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "22": {
+      "content": "<unused15>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "23": {
+      "content": "<unused16>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "24": {
+      "content": "<unused17>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "25": {
+      "content": "<unused18>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "26": {
+      "content": "<unused19>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "27": {
+      "content": "<unused20>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "28": {
+      "content": "<unused21>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "29": {
+      "content": "<unused22>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "30": {
+      "content": "<unused23>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "31": {
+      "content": "<unused24>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "32": {
+      "content": "<unused25>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "33": {
+      "content": "<unused26>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "34": {
+      "content": "<unused27>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "35": {
+      "content": "<unused28>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "36": {
+      "content": "<unused29>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "37": {
+      "content": "<unused30>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "38": {
+      "content": "<unused31>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "39": {
+      "content": "<unused32>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "40": {
+      "content": "<unused33>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "41": {
+      "content": "<unused34>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "42": {
+      "content": "<unused35>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "43": {
+      "content": "<unused36>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "44": {
+      "content": "<unused37>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "45": {
+      "content": "<unused38>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "46": {
+      "content": "<unused39>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "47": {
+      "content": "<unused40>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "48": {
+      "content": "<unused41>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "49": {
+      "content": "<unused42>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50": {
+      "content": "<unused43>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "51": {
+      "content": "<unused44>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "52": {
+      "content": "<unused45>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "53": {
+      "content": "<unused46>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "54": {
+      "content": "<unused47>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55": {
+      "content": "<unused48>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "56": {
+      "content": "<unused49>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "57": {
+      "content": "<unused50>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "58": {
+      "content": "<unused51>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "59": {
+      "content": "<unused52>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "60": {
+      "content": "<unused53>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "61": {
+      "content": "<unused54>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "62": {
+      "content": "<unused55>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "63": {
+      "content": "<unused56>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "64": {
+      "content": "<unused57>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "65": {
+      "content": "<unused58>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "66": {
+      "content": "<unused59>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "67": {
+      "content": "<unused60>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "68": {
+      "content": "<unused61>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "69": {
+      "content": "<unused62>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "70": {
+      "content": "<unused63>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "71": {
+      "content": "<unused64>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "72": {
+      "content": "<unused65>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "73": {
+      "content": "<unused66>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "74": {
+      "content": "<unused67>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "75": {
+      "content": "<unused68>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "76": {
+      "content": "<unused69>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "77": {
+      "content": "<unused70>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "78": {
+      "content": "<unused71>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "79": {
+      "content": "<unused72>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "80": {
+      "content": "<unused73>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "81": {
+      "content": "<unused74>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "82": {
+      "content": "<unused75>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "83": {
+      "content": "<unused76>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "84": {
+      "content": "<unused77>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "85": {
+      "content": "<unused78>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "86": {
+      "content": "<unused79>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "87": {
+      "content": "<unused80>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "88": {
+      "content": "<unused81>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "89": {
+      "content": "<unused82>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "90": {
+      "content": "<unused83>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "91": {
+      "content": "<unused84>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "92": {
+      "content": "<unused85>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "93": {
+      "content": "<unused86>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "94": {
+      "content": "<unused87>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95": {
+      "content": "<unused88>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "96": {
+      "content": "<unused89>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "97": {
+      "content": "<unused90>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "98": {
+      "content": "<unused91>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "99": {
+      "content": "<unused92>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "100": {
+      "content": "<unused93>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "101": {
+      "content": "<unused94>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "102": {
+      "content": "<unused95>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "103": {
+      "content": "<unused96>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "104": {
+      "content": "<unused97>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "105": {
+      "content": "<unused98>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "106": {
+      "content": "<start_of_turn>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "107": {
+      "content": "<end_of_turn>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "108": {
+      "content": "\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "109": {
+      "content": "\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "110": {
+      "content": "\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "111": {
+      "content": "\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "112": {
+      "content": "\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "113": {
+      "content": "\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "114": {
+      "content": "\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "115": {
+      "content": "\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "116": {
+      "content": "\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "117": {
+      "content": "\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "118": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "119": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "120": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "121": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "122": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "123": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "124": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "125": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "126": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "127": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "128": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "129": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "130": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "131": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "132": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "133": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "134": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "135": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "136": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "137": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "138": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "139": {
+      "content": "▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "140": {
+      "content": "▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "141": {
+      "content": "▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "142": {
+      "content": "▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "143": {
+      "content": "▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "144": {
+      "content": "▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "145": {
+      "content": "▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "146": {
+      "content": "▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "147": {
+      "content": "▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "148": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "149": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "150": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "152": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "153": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "154": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "155": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "156": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "157": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "158": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "159": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "160": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "161": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "162": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "163": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "164": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "165": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "166": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "167": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "168": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "169": {
+      "content": "<table>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "170": {
+      "content": "<caption>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "171": {
+      "content": "<thead>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "172": {
+      "content": "<tbody>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "173": {
+      "content": "<tfoot>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "174": {
+      "content": "<tr>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "175": {
+      "content": "<th>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "176": {
+      "content": "<td>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "177": {
+      "content": "</table>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "178": {
+      "content": "</caption>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "179": {
+      "content": "</thead>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "180": {
+      "content": "</tbody>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "181": {
+      "content": "</tfoot>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "182": {
+      "content": "</tr>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "183": {
+      "content": "</th>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "184": {
+      "content": "</td>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "185": {
+      "content": "<h1>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "186": {
+      "content": "<h2>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "187": {
+      "content": "<h3>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "188": {
+      "content": "<h4>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "189": {
+      "content": "<h5>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "190": {
+      "content": "<h6>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "191": {
+      "content": "<blockquote>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "192": {
+      "content": "</h1>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "193": {
+      "content": "</h2>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "194": {
+      "content": "</h3>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "195": {
+      "content": "</h4>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "196": {
+      "content": "</h5>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "197": {
+      "content": "</h6>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "198": {
+      "content": "</blockquote>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "199": {
+      "content": "<strong>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "200": {
+      "content": "<em>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "201": {
+      "content": "<b>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "202": {
+      "content": "<i>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "203": {
+      "content": "<u>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "204": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "205": {
+      "content": "<sub>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "206": {
+      "content": "<sup>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "207": {
+      "content": "<code>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "208": {
+      "content": "</strong>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "209": {
+      "content": "</em>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "210": {
+      "content": "</b>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "211": {
+      "content": "</i>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "212": {
+      "content": "</u>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "213": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "214": {
+      "content": "</sub>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "215": {
+      "content": "</sup>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "216": {
+      "content": "</code>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255968": {
+      "content": "[toxicity=0]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255969": {
+      "content": "\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255970": {
+      "content": "\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255971": {
+      "content": "\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255972": {
+      "content": "\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255973": {
+      "content": "\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255974": {
+      "content": "\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255975": {
+      "content": "\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255976": {
+      "content": "\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255977": {
+      "content": "\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255978": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255979": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255980": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255981": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255982": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255983": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255984": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255985": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255986": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255987": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255988": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255989": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255990": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255991": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255992": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255993": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255994": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255995": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255996": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255997": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255998": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255999": {
+      "content": "<unused99>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "bos_token": "<bos>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<eos>",
+  "extra_special_tokens": {},
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<pad>",
+  "padding_side": "right",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "split_special_tokens": false,
+  "tokenizer_class": "GemmaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}
diff --git a/checkpoint-742/trainer_state.json b/checkpoint-742/trainer_state.json
new file mode 100644
index 0000000000000000000000000000000000000000..876dc41bd81b47dbda9a03d6eb497f47fa7a5f60
--- /dev/null
+++ b/checkpoint-742/trainer_state.json
@@ -0,0 +1,552 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 1.0,
+  "eval_steps": 500,
+  "global_step": 742,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.013490725126475547,
+      "grad_norm": 65.45332336425781,
+      "learning_rate": 0.00019964061096136568,
+      "loss": 22.1843,
+      "step": 10
+    },
+    {
+      "epoch": 0.026981450252951095,
+      "grad_norm": 6.102770805358887,
+      "learning_rate": 0.00019874213836477988,
+      "loss": 7.6527,
+      "step": 20
+    },
+    {
+      "epoch": 0.04047217537942664,
+      "grad_norm": 2.982379913330078,
+      "learning_rate": 0.00019784366576819408,
+      "loss": 3.4532,
+      "step": 30
+    },
+    {
+      "epoch": 0.05396290050590219,
+      "grad_norm": 3.4628562927246094,
+      "learning_rate": 0.00019694519317160826,
+      "loss": 2.4467,
+      "step": 40
+    },
+    {
+      "epoch": 0.06745362563237774,
+      "grad_norm": 3.0008349418640137,
+      "learning_rate": 0.00019604672057502246,
+      "loss": 1.6481,
+      "step": 50
+    },
+    {
+      "epoch": 0.08094435075885328,
+      "grad_norm": 1.767865777015686,
+      "learning_rate": 0.00019514824797843666,
+      "loss": 1.2119,
+      "step": 60
+    },
+    {
+      "epoch": 0.09443507588532883,
+      "grad_norm": 3.416968584060669,
+      "learning_rate": 0.00019424977538185087,
+      "loss": 0.967,
+      "step": 70
+    },
+    {
+      "epoch": 0.10792580101180438,
+      "grad_norm": 3.718262195587158,
+      "learning_rate": 0.00019335130278526507,
+      "loss": 0.8783,
+      "step": 80
+    },
+    {
+      "epoch": 0.12141652613827993,
+      "grad_norm": 3.689655065536499,
+      "learning_rate": 0.00019245283018867927,
+      "loss": 0.7619,
+      "step": 90
+    },
+    {
+      "epoch": 0.13490725126475547,
+      "grad_norm": 6.728384017944336,
+      "learning_rate": 0.00019155435759209347,
+      "loss": 0.7244,
+      "step": 100
+    },
+    {
+      "epoch": 0.14839797639123103,
+      "grad_norm": 4.880130290985107,
+      "learning_rate": 0.00019065588499550765,
+      "loss": 0.6931,
+      "step": 110
+    },
+    {
+      "epoch": 0.16188870151770657,
+      "grad_norm": 10.14767074584961,
+      "learning_rate": 0.00018975741239892185,
+      "loss": 0.6995,
+      "step": 120
+    },
+    {
+      "epoch": 0.17537942664418213,
+      "grad_norm": 10.897403717041016,
+      "learning_rate": 0.00018885893980233605,
+      "loss": 0.69,
+      "step": 130
+    },
+    {
+      "epoch": 0.18887015177065766,
+      "grad_norm": 7.759915351867676,
+      "learning_rate": 0.00018796046720575023,
+      "loss": 0.6899,
+      "step": 140
+    },
+    {
+      "epoch": 0.20236087689713322,
+      "grad_norm": 7.006256580352783,
+      "learning_rate": 0.00018706199460916443,
+      "loss": 0.6739,
+      "step": 150
+    },
+    {
+      "epoch": 0.21585160202360876,
+      "grad_norm": 5.777213096618652,
+      "learning_rate": 0.00018616352201257863,
+      "loss": 0.6731,
+      "step": 160
+    },
+    {
+      "epoch": 0.22934232715008432,
+      "grad_norm": 10.224111557006836,
+      "learning_rate": 0.00018526504941599283,
+      "loss": 0.6417,
+      "step": 170
+    },
+    {
+      "epoch": 0.24283305227655985,
+      "grad_norm": 6.332294464111328,
+      "learning_rate": 0.000184366576819407,
+      "loss": 0.6452,
+      "step": 180
+    },
+    {
+      "epoch": 0.2563237774030354,
+      "grad_norm": 3.9472005367279053,
+      "learning_rate": 0.0001834681042228212,
+      "loss": 0.6271,
+      "step": 190
+    },
+    {
+      "epoch": 0.26981450252951095,
+      "grad_norm": 6.640835762023926,
+      "learning_rate": 0.0001825696316262354,
+      "loss": 0.6161,
+      "step": 200
+    },
+    {
+      "epoch": 0.28330522765598654,
+      "grad_norm": 5.6504316329956055,
+      "learning_rate": 0.00018167115902964959,
+      "loss": 0.6452,
+      "step": 210
+    },
+    {
+      "epoch": 0.29679595278246207,
+      "grad_norm": 3.4219472408294678,
+      "learning_rate": 0.0001807726864330638,
+      "loss": 0.6228,
+      "step": 220
+    },
+    {
+      "epoch": 0.3102866779089376,
+      "grad_norm": 7.479050636291504,
+      "learning_rate": 0.000179874213836478,
+      "loss": 0.6035,
+      "step": 230
+    },
+    {
+      "epoch": 0.32377740303541314,
+      "grad_norm": 6.84128475189209,
+      "learning_rate": 0.0001789757412398922,
+      "loss": 0.5763,
+      "step": 240
+    },
+    {
+      "epoch": 0.3372681281618887,
+      "grad_norm": 3.407183885574341,
+      "learning_rate": 0.0001780772686433064,
+      "loss": 0.5981,
+      "step": 250
+    },
+    {
+      "epoch": 0.35075885328836426,
+      "grad_norm": 6.634527683258057,
+      "learning_rate": 0.0001771787960467206,
+      "loss": 0.6313,
+      "step": 260
+    },
+    {
+      "epoch": 0.3642495784148398,
+      "grad_norm": 6.274175643920898,
+      "learning_rate": 0.0001762803234501348,
+      "loss": 0.5971,
+      "step": 270
+    },
+    {
+      "epoch": 0.3777403035413153,
+      "grad_norm": 5.464880466461182,
+      "learning_rate": 0.00017538185085354897,
+      "loss": 0.629,
+      "step": 280
+    },
+    {
+      "epoch": 0.3912310286677909,
+      "grad_norm": 7.9421234130859375,
+      "learning_rate": 0.00017448337825696318,
+      "loss": 0.5604,
+      "step": 290
+    },
+    {
+      "epoch": 0.40472175379426645,
+      "grad_norm": 6.513714790344238,
+      "learning_rate": 0.00017358490566037738,
+      "loss": 0.5442,
+      "step": 300
+    },
+    {
+      "epoch": 0.418212478920742,
+      "grad_norm": 4.856393337249756,
+      "learning_rate": 0.00017268643306379155,
+      "loss": 0.5444,
+      "step": 310
+    },
+    {
+      "epoch": 0.4317032040472175,
+      "grad_norm": 7.252712249755859,
+      "learning_rate": 0.00017178796046720576,
+      "loss": 0.6071,
+      "step": 320
+    },
+    {
+      "epoch": 0.4451939291736931,
+      "grad_norm": 6.8953328132629395,
+      "learning_rate": 0.00017088948787061996,
+      "loss": 0.5622,
+      "step": 330
+    },
+    {
+      "epoch": 0.45868465430016864,
+      "grad_norm": 6.220505237579346,
+      "learning_rate": 0.00016999101527403416,
+      "loss": 0.6213,
+      "step": 340
+    },
+    {
+      "epoch": 0.47217537942664417,
+      "grad_norm": 1.9960452318191528,
+      "learning_rate": 0.00016909254267744833,
+      "loss": 0.5752,
+      "step": 350
+    },
+    {
+      "epoch": 0.4856661045531197,
+      "grad_norm": 4.8189778327941895,
+      "learning_rate": 0.00016819407008086254,
+      "loss": 0.5695,
+      "step": 360
+    },
+    {
+      "epoch": 0.4991568296795953,
+      "grad_norm": 2.157482624053955,
+      "learning_rate": 0.00016729559748427674,
+      "loss": 0.5818,
+      "step": 370
+    },
+    {
+      "epoch": 0.5126475548060708,
+      "grad_norm": 6.034278392791748,
+      "learning_rate": 0.00016639712488769091,
+      "loss": 0.5632,
+      "step": 380
+    },
+    {
+      "epoch": 0.5261382799325464,
+      "grad_norm": 3.6643030643463135,
+      "learning_rate": 0.00016549865229110512,
+      "loss": 0.5149,
+      "step": 390
+    },
+    {
+      "epoch": 0.5396290050590219,
+      "grad_norm": 3.4874682426452637,
+      "learning_rate": 0.00016460017969451932,
+      "loss": 0.5712,
+      "step": 400
+    },
+    {
+      "epoch": 0.5531197301854974,
+      "grad_norm": 5.94505500793457,
+      "learning_rate": 0.00016370170709793352,
+      "loss": 0.5579,
+      "step": 410
+    },
+    {
+      "epoch": 0.5666104553119731,
+      "grad_norm": 2.6858978271484375,
+      "learning_rate": 0.00016280323450134772,
+      "loss": 0.5601,
+      "step": 420
+    },
+    {
+      "epoch": 0.5801011804384486,
+      "grad_norm": 0.5919632315635681,
+      "learning_rate": 0.00016190476190476192,
+      "loss": 0.5122,
+      "step": 430
+    },
+    {
+      "epoch": 0.5935919055649241,
+      "grad_norm": 9.723355293273926,
+      "learning_rate": 0.00016100628930817613,
+      "loss": 0.5408,
+      "step": 440
+    },
+    {
+      "epoch": 0.6070826306913997,
+      "grad_norm": 3.421534776687622,
+      "learning_rate": 0.0001601078167115903,
+      "loss": 0.5125,
+      "step": 450
+    },
+    {
+      "epoch": 0.6205733558178752,
+      "grad_norm": 1.350819706916809,
+      "learning_rate": 0.0001592093441150045,
+      "loss": 0.5443,
+      "step": 460
+    },
+    {
+      "epoch": 0.6340640809443507,
+      "grad_norm": 2.9618473052978516,
+      "learning_rate": 0.0001583108715184187,
+      "loss": 0.5792,
+      "step": 470
+    },
+    {
+      "epoch": 0.6475548060708263,
+      "grad_norm": 11.16501522064209,
+      "learning_rate": 0.00015741239892183288,
+      "loss": 0.5499,
+      "step": 480
+    },
+    {
+      "epoch": 0.6610455311973018,
+      "grad_norm": 2.7122786045074463,
+      "learning_rate": 0.00015651392632524708,
+      "loss": 0.589,
+      "step": 490
+    },
+    {
+      "epoch": 0.6745362563237775,
+      "grad_norm": 6.2460408210754395,
+      "learning_rate": 0.00015561545372866128,
+      "loss": 0.5422,
+      "step": 500
+    },
+    {
+      "epoch": 0.688026981450253,
+      "grad_norm": 12.943184852600098,
+      "learning_rate": 0.0001547169811320755,
+      "loss": 0.5431,
+      "step": 510
+    },
+    {
+      "epoch": 0.7015177065767285,
+      "grad_norm": 2.4169883728027344,
+      "learning_rate": 0.00015381850853548966,
+      "loss": 0.4984,
+      "step": 520
+    },
+    {
+      "epoch": 0.715008431703204,
+      "grad_norm": 5.108154773712158,
+      "learning_rate": 0.00015292003593890386,
+      "loss": 0.4795,
+      "step": 530
+    },
+    {
+      "epoch": 0.7284991568296796,
+      "grad_norm": 3.268669605255127,
+      "learning_rate": 0.00015202156334231807,
+      "loss": 0.4961,
+      "step": 540
+    },
+    {
+      "epoch": 0.7419898819561551,
+      "grad_norm": 1.7926982641220093,
+      "learning_rate": 0.00015112309074573224,
+      "loss": 0.5339,
+      "step": 550
+    },
+    {
+      "epoch": 0.7554806070826307,
+      "grad_norm": 3.6382663249969482,
+      "learning_rate": 0.00015022461814914644,
+      "loss": 0.4855,
+      "step": 560
+    },
+    {
+      "epoch": 0.7689713322091062,
+      "grad_norm": 8.025957107543945,
+      "learning_rate": 0.00014932614555256067,
+      "loss": 0.4901,
+      "step": 570
+    },
+    {
+      "epoch": 0.7824620573355818,
+      "grad_norm": 4.220882892608643,
+      "learning_rate": 0.00014842767295597485,
+      "loss": 0.5448,
+      "step": 580
+    },
+    {
+      "epoch": 0.7959527824620574,
+      "grad_norm": 6.42817497253418,
+      "learning_rate": 0.00014752920035938905,
+      "loss": 0.4969,
+      "step": 590
+    },
+    {
+      "epoch": 0.8094435075885329,
+      "grad_norm": 4.182610988616943,
+      "learning_rate": 0.00014663072776280325,
+      "loss": 0.4863,
+      "step": 600
+    },
+    {
+      "epoch": 0.8229342327150084,
+      "grad_norm": 3.687934637069702,
+      "learning_rate": 0.00014573225516621745,
+      "loss": 0.5162,
+      "step": 610
+    },
+    {
+      "epoch": 0.836424957841484,
+      "grad_norm": 3.8240537643432617,
+      "learning_rate": 0.00014483378256963163,
+      "loss": 0.5089,
+      "step": 620
+    },
+    {
+      "epoch": 0.8499156829679595,
+      "grad_norm": 3.3823423385620117,
+      "learning_rate": 0.00014393530997304583,
+      "loss": 0.5447,
+      "step": 630
+    },
+    {
+      "epoch": 0.863406408094435,
+      "grad_norm": 4.591256618499756,
+      "learning_rate": 0.00014303683737646003,
+      "loss": 0.5067,
+      "step": 640
+    },
+    {
+      "epoch": 0.8768971332209107,
+      "grad_norm": 4.414494514465332,
+      "learning_rate": 0.0001421383647798742,
+      "loss": 0.4858,
+      "step": 650
+    },
+    {
+      "epoch": 0.8903878583473862,
+      "grad_norm": 3.4649970531463623,
+      "learning_rate": 0.0001412398921832884,
+      "loss": 0.5243,
+      "step": 660
+    },
+    {
+      "epoch": 0.9038785834738617,
+      "grad_norm": 4.813696384429932,
+      "learning_rate": 0.0001403414195867026,
+      "loss": 0.51,
+      "step": 670
+    },
+    {
+      "epoch": 0.9173693086003373,
+      "grad_norm": 4.597005844116211,
+      "learning_rate": 0.00013944294699011681,
+      "loss": 0.5079,
+      "step": 680
+    },
+    {
+      "epoch": 0.9308600337268128,
+      "grad_norm": 4.291003704071045,
+      "learning_rate": 0.000138544474393531,
+      "loss": 0.4966,
+      "step": 690
+    },
+    {
+      "epoch": 0.9443507588532883,
+      "grad_norm": 3.8949058055877686,
+      "learning_rate": 0.0001376460017969452,
+      "loss": 0.5086,
+      "step": 700
+    },
+    {
+      "epoch": 0.9578414839797639,
+      "grad_norm": 4.685634613037109,
+      "learning_rate": 0.0001367475292003594,
+      "loss": 0.5005,
+      "step": 710
+    },
+    {
+      "epoch": 0.9713322091062394,
+      "grad_norm": 3.9277801513671875,
+      "learning_rate": 0.0001358490566037736,
+      "loss": 0.4759,
+      "step": 720
+    },
+    {
+      "epoch": 0.984822934232715,
+      "grad_norm": 4.3011345863342285,
+      "learning_rate": 0.0001349505840071878,
+      "loss": 0.4371,
+      "step": 730
+    },
+    {
+      "epoch": 0.9983136593591906,
+      "grad_norm": 3.370706796646118,
+      "learning_rate": 0.000134052111410602,
+      "loss": 0.4811,
+      "step": 740
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 2226,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 3.690343765927526e+16,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}
diff --git a/checkpoint-742/training_args.bin b/checkpoint-742/training_args.bin
new file mode 100644
index 0000000000000000000000000000000000000000..597bbc184c73a1ca565444bfc5a1a6c505b47b83
--- /dev/null
+++ b/checkpoint-742/training_args.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:160a1045610f1057c5a1c808e36986bd0969b8e2c1163c803dfd9b9a2891eb12
+size 5304
diff --git a/special_tokens_map.json b/special_tokens_map.json
new file mode 100644
index 0000000000000000000000000000000000000000..f6119589e367b2de0fc8cbd2f1217667532e3174
--- /dev/null
+++ b/special_tokens_map.json
@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<bos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<eos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}
diff --git a/tokenizer.json b/tokenizer.json
new file mode 100644
index 0000000000000000000000000000000000000000..2b2d51a12a27cc5bd1baeaca3ec01d6f6f458f64
--- /dev/null
+++ b/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:afe09ee4dcd6379dfa9f12c0178f86eab033fc9f97f4e1d605a44da3488b6c53
+size 34363138
diff --git a/tokenizer.model b/tokenizer.model
new file mode 100644
index 0000000000000000000000000000000000000000..796efe9ab515c15e146ce7588e6d7b9b8134dbf8
--- /dev/null
+++ b/tokenizer.model
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61a7b147390c64585d6c3543dd6fc636906c9af3865a5548f27f31aee1d4c8e2
+size 4241003
diff --git a/tokenizer_config.json b/tokenizer_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..62e63dbcee5400f4f34987854723a5120c4cc076
--- /dev/null
+++ b/tokenizer_config.json
@@ -0,0 +1,2011 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<eos>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "<bos>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "4": {
+      "content": "<mask>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "5": {
+      "content": "<2mass>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "6": {
+      "content": "[@BOS@]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "7": {
+      "content": "<unused0>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "8": {
+      "content": "<unused1>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "9": {
+      "content": "<unused2>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "10": {
+      "content": "<unused3>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "11": {
+      "content": "<unused4>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "12": {
+      "content": "<unused5>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "13": {
+      "content": "<unused6>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "14": {
+      "content": "<unused7>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "15": {
+      "content": "<unused8>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "16": {
+      "content": "<unused9>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "17": {
+      "content": "<unused10>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "18": {
+      "content": "<unused11>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "19": {
+      "content": "<unused12>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "20": {
+      "content": "<unused13>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "21": {
+      "content": "<unused14>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "22": {
+      "content": "<unused15>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "23": {
+      "content": "<unused16>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "24": {
+      "content": "<unused17>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "25": {
+      "content": "<unused18>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "26": {
+      "content": "<unused19>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "27": {
+      "content": "<unused20>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "28": {
+      "content": "<unused21>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "29": {
+      "content": "<unused22>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "30": {
+      "content": "<unused23>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "31": {
+      "content": "<unused24>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "32": {
+      "content": "<unused25>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "33": {
+      "content": "<unused26>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "34": {
+      "content": "<unused27>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "35": {
+      "content": "<unused28>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "36": {
+      "content": "<unused29>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "37": {
+      "content": "<unused30>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "38": {
+      "content": "<unused31>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "39": {
+      "content": "<unused32>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "40": {
+      "content": "<unused33>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "41": {
+      "content": "<unused34>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "42": {
+      "content": "<unused35>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "43": {
+      "content": "<unused36>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "44": {
+      "content": "<unused37>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "45": {
+      "content": "<unused38>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "46": {
+      "content": "<unused39>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "47": {
+      "content": "<unused40>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "48": {
+      "content": "<unused41>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "49": {
+      "content": "<unused42>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50": {
+      "content": "<unused43>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "51": {
+      "content": "<unused44>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "52": {
+      "content": "<unused45>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "53": {
+      "content": "<unused46>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "54": {
+      "content": "<unused47>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55": {
+      "content": "<unused48>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "56": {
+      "content": "<unused49>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "57": {
+      "content": "<unused50>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "58": {
+      "content": "<unused51>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "59": {
+      "content": "<unused52>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "60": {
+      "content": "<unused53>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "61": {
+      "content": "<unused54>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "62": {
+      "content": "<unused55>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "63": {
+      "content": "<unused56>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "64": {
+      "content": "<unused57>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "65": {
+      "content": "<unused58>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "66": {
+      "content": "<unused59>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "67": {
+      "content": "<unused60>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "68": {
+      "content": "<unused61>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "69": {
+      "content": "<unused62>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "70": {
+      "content": "<unused63>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "71": {
+      "content": "<unused64>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "72": {
+      "content": "<unused65>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "73": {
+      "content": "<unused66>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "74": {
+      "content": "<unused67>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "75": {
+      "content": "<unused68>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "76": {
+      "content": "<unused69>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "77": {
+      "content": "<unused70>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "78": {
+      "content": "<unused71>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "79": {
+      "content": "<unused72>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "80": {
+      "content": "<unused73>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "81": {
+      "content": "<unused74>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "82": {
+      "content": "<unused75>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "83": {
+      "content": "<unused76>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "84": {
+      "content": "<unused77>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "85": {
+      "content": "<unused78>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "86": {
+      "content": "<unused79>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "87": {
+      "content": "<unused80>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "88": {
+      "content": "<unused81>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "89": {
+      "content": "<unused82>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "90": {
+      "content": "<unused83>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "91": {
+      "content": "<unused84>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "92": {
+      "content": "<unused85>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "93": {
+      "content": "<unused86>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "94": {
+      "content": "<unused87>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95": {
+      "content": "<unused88>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "96": {
+      "content": "<unused89>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "97": {
+      "content": "<unused90>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "98": {
+      "content": "<unused91>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "99": {
+      "content": "<unused92>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "100": {
+      "content": "<unused93>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "101": {
+      "content": "<unused94>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "102": {
+      "content": "<unused95>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "103": {
+      "content": "<unused96>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "104": {
+      "content": "<unused97>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "105": {
+      "content": "<unused98>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "106": {
+      "content": "<start_of_turn>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "107": {
+      "content": "<end_of_turn>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "108": {
+      "content": "\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "109": {
+      "content": "\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "110": {
+      "content": "\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "111": {
+      "content": "\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "112": {
+      "content": "\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "113": {
+      "content": "\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "114": {
+      "content": "\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "115": {
+      "content": "\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "116": {
+      "content": "\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "117": {
+      "content": "\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "118": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "119": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "120": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "121": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "122": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "123": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "124": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "125": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "126": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "127": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "128": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "129": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "130": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "131": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "132": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "133": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "134": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "135": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "136": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "137": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "138": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "139": {
+      "content": "▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "140": {
+      "content": "▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "141": {
+      "content": "▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "142": {
+      "content": "▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "143": {
+      "content": "▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "144": {
+      "content": "▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "145": {
+      "content": "▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "146": {
+      "content": "▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "147": {
+      "content": "▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "148": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "149": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "150": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "152": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "153": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "154": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "155": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "156": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "157": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "158": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "159": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "160": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "161": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "162": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "163": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "164": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "165": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "166": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "167": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "168": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "169": {
+      "content": "<table>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "170": {
+      "content": "<caption>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "171": {
+      "content": "<thead>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "172": {
+      "content": "<tbody>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "173": {
+      "content": "<tfoot>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "174": {
+      "content": "<tr>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "175": {
+      "content": "<th>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "176": {
+      "content": "<td>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "177": {
+      "content": "</table>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "178": {
+      "content": "</caption>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "179": {
+      "content": "</thead>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "180": {
+      "content": "</tbody>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "181": {
+      "content": "</tfoot>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "182": {
+      "content": "</tr>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "183": {
+      "content": "</th>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "184": {
+      "content": "</td>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "185": {
+      "content": "<h1>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "186": {
+      "content": "<h2>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "187": {
+      "content": "<h3>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "188": {
+      "content": "<h4>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "189": {
+      "content": "<h5>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "190": {
+      "content": "<h6>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "191": {
+      "content": "<blockquote>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "192": {
+      "content": "</h1>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "193": {
+      "content": "</h2>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "194": {
+      "content": "</h3>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "195": {
+      "content": "</h4>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "196": {
+      "content": "</h5>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "197": {
+      "content": "</h6>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "198": {
+      "content": "</blockquote>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "199": {
+      "content": "<strong>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "200": {
+      "content": "<em>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "201": {
+      "content": "<b>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "202": {
+      "content": "<i>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "203": {
+      "content": "<u>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "204": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "205": {
+      "content": "<sub>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "206": {
+      "content": "<sup>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "207": {
+      "content": "<code>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "208": {
+      "content": "</strong>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "209": {
+      "content": "</em>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "210": {
+      "content": "</b>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "211": {
+      "content": "</i>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "212": {
+      "content": "</u>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "213": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "214": {
+      "content": "</sub>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "215": {
+      "content": "</sup>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "216": {
+      "content": "</code>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255968": {
+      "content": "[toxicity=0]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255969": {
+      "content": "\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255970": {
+      "content": "\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255971": {
+      "content": "\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255972": {
+      "content": "\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255973": {
+      "content": "\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255974": {
+      "content": "\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255975": {
+      "content": "\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255976": {
+      "content": "\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255977": {
+      "content": "\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255978": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255979": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255980": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255981": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255982": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255983": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255984": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255985": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255986": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255987": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255988": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255989": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255990": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255991": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255992": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255993": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255994": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255995": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255996": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255997": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255998": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255999": {
+      "content": "<unused99>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "bos_token": "<bos>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<eos>",
+  "extra_special_tokens": {},
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<pad>",
+  "padding_side": "right",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "split_special_tokens": false,
+  "tokenizer_class": "GemmaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}