Instructions to use Leeyuyu/Qwen2.5-VL-3B-Instruct-SFT-fundo with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Leeyuyu/Qwen2.5-VL-3B-Instruct-SFT-fundo with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Leeyuyu/Qwen2.5-VL-3B-Instruct-SFT-fundo")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("Leeyuyu/Qwen2.5-VL-3B-Instruct-SFT-fundo")
model = AutoModelForMultimodalLM.from_pretrained("Leeyuyu/Qwen2.5-VL-3B-Instruct-SFT-fundo")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Leeyuyu/Qwen2.5-VL-3B-Instruct-SFT-fundo with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Leeyuyu/Qwen2.5-VL-3B-Instruct-SFT-fundo"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Leeyuyu/Qwen2.5-VL-3B-Instruct-SFT-fundo",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Leeyuyu/Qwen2.5-VL-3B-Instruct-SFT-fundo

SGLang

How to use Leeyuyu/Qwen2.5-VL-3B-Instruct-SFT-fundo with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Leeyuyu/Qwen2.5-VL-3B-Instruct-SFT-fundo" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Leeyuyu/Qwen2.5-VL-3B-Instruct-SFT-fundo",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Leeyuyu/Qwen2.5-VL-3B-Instruct-SFT-fundo" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Leeyuyu/Qwen2.5-VL-3B-Instruct-SFT-fundo",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use Leeyuyu/Qwen2.5-VL-3B-Instruct-SFT-fundo with Docker Model Runner:
```
docker model run hf.co/Leeyuyu/Qwen2.5-VL-3B-Instruct-SFT-fundo
```

Leeyuyu commited on Aug 13, 2025

Commit

bdb9f0f

verified ·

1 Parent(s): d3cd113

Model save

Browse files

Files changed (5) hide show

README.md +2 -4
all_results.json +7 -7
tokenizer_config.json +0 -1
train_results.json +7 -7
trainer_state.json +3497 -39

README.md CHANGED Viewed

@@ -1,10 +1,8 @@
 ---
-datasets: Leeyuyu/fundo_SFT
 library_name: transformers
 model_name: Qwen2.5-VL-3B-Instruct-SFT-fundo
 tags:
 - generated_from_trainer
-- R1-V
 - trl
 - sft
 licence: license
@@ -12,7 +10,7 @@ licence: license
 # Model Card for Qwen2.5-VL-3B-Instruct-SFT-fundo
-This model is a fine-tuned version of [None](https://huggingface.co/None) on the [Leeyuyu/fundo_SFT](https://huggingface.co/datasets/Leeyuyu/fundo_SFT) dataset.
 It has been trained using [TRL](https://github.com/huggingface/trl).
 ## Quick start
@@ -28,7 +26,7 @@ print(output["generated_text"])
 ## Training procedure
-[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/brightlight720720_lee/huggingface/runs/2vnq9mb3)
 This model was trained with SFT.

 ---
 library_name: transformers
 model_name: Qwen2.5-VL-3B-Instruct-SFT-fundo
 tags:
 - generated_from_trainer
 - trl
 - sft
 licence: license
 # Model Card for Qwen2.5-VL-3B-Instruct-SFT-fundo
+This model is a fine-tuned version of [None](https://huggingface.co/None).
 It has been trained using [TRL](https://github.com/huggingface/trl).
 ## Quick start
 ## Training procedure
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/brightlight720720_lee/huggingface/runs/d4p8x5xy)
 This model was trained with SFT.

all_results.json CHANGED Viewed

@@ -1,9 +1,9 @@
 {
-    "epoch": 1.8571428571428572,
-    "total_flos": 9637412825202688.0,
-    "train_loss": 2.229151020447413,
-    "train_runtime": 212.6127,
-    "train_samples": 221,
-    "train_samples_per_second": 2.079,
-    "train_steps_per_second": 0.028
 }

 {
+    "epoch": 10.0,
+    "total_flos": 1.13782877585408e+18,
+    "train_loss": 0.07423569742811378,
+    "train_runtime": 7287.0887,
+    "train_samples": 3195,
+    "train_samples_per_second": 4.384,
+    "train_steps_per_second": 0.069
 }

tokenizer_config.json CHANGED Viewed

@@ -202,7 +202,6 @@
   "extra_special_tokens": {},
   "model_max_length": 131072,
   "pad_token": "<|endoftext|>",
-  "processor_class": "Qwen2_5_VLProcessor",
   "split_special_tokens": false,
   "tokenizer_class": "Qwen2Tokenizer",
   "unk_token": null

   "extra_special_tokens": {},
   "model_max_length": 131072,
   "pad_token": "<|endoftext|>",
   "split_special_tokens": false,
   "tokenizer_class": "Qwen2Tokenizer",
   "unk_token": null

train_results.json CHANGED Viewed

@@ -1,9 +1,9 @@
 {
-    "epoch": 1.8571428571428572,
-    "total_flos": 9637412825202688.0,
-    "train_loss": 2.229151020447413,
-    "train_runtime": 212.6127,
-    "train_samples": 221,
-    "train_samples_per_second": 2.079,
-    "train_steps_per_second": 0.028
 }

 {
+    "epoch": 10.0,
+    "total_flos": 1.13782877585408e+18,
+    "train_loss": 0.07423569742811378,
+    "train_runtime": 7287.0887,
+    "train_samples": 3195,
+    "train_samples_per_second": 4.384,
+    "train_steps_per_second": 0.069
 }

trainer_state.json CHANGED Viewed

@@ -1,70 +1,3528 @@
 {
   "best_metric": null,
   "best_model_checkpoint": null,
-  "epoch": 1.8571428571428572,
-  "eval_steps": 500,
-  "global_step": 6,
   "is_hyper_param_search": false,
   "is_local_process_zero": true,
   "is_world_process_zero": true,
   "log_history": [
     {
-      "epoch": 0.2857142857142857,
-      "grad_norm": 443.46893310546875,
-      "learning_rate": 2e-05,
-      "loss": 4.351,
       "step": 1
     },
     {
-      "epoch": 0.5714285714285714,
-      "grad_norm": 435.6300354003906,
-      "learning_rate": 1.8090169943749477e-05,
-      "loss": 4.3347,
       "step": 2
     },
     {
-      "epoch": 0.8571428571428571,
-      "grad_norm": 10.48087215423584,
-      "learning_rate": 1.3090169943749475e-05,
-      "loss": 1.58,
       "step": 3
     },
     {
-      "epoch": 1.2857142857142856,
-      "grad_norm": 10.523934364318848,
-      "learning_rate": 6.909830056250527e-06,
-      "loss": 1.3946,
       "step": 4
     },
     {
-      "epoch": 1.5714285714285714,
-      "grad_norm": 6.847975254058838,
-      "learning_rate": 1.9098300562505266e-06,
-      "loss": 0.9153,
       "step": 5
     },
     {
-      "epoch": 1.8571428571428572,
-      "grad_norm": 6.616868019104004,
-      "learning_rate": 0.0,
-      "loss": 0.7993,
       "step": 6
     },
     {
-      "epoch": 1.8571428571428572,
-      "step": 6,
-      "total_flos": 9637412825202688.0,
-      "train_loss": 2.229151020447413,
-      "train_runtime": 212.6127,
-      "train_samples_per_second": 2.079,
-      "train_steps_per_second": 0.028
     }
   ],
   "logging_steps": 1,
-  "max_steps": 6,
   "num_input_tokens_seen": 0,
-  "num_train_epochs": 2,
-  "save_steps": 100,
   "stateful_callbacks": {
     "TrainerControl": {
       "args": {
@@ -77,8 +3535,8 @@
       "attributes": {}
     }
   },
-  "total_flos": 9637412825202688.0,
-  "train_batch_size": 4,
   "trial_name": null,
   "trial_params": null
 }

 {
   "best_metric": null,
   "best_model_checkpoint": null,
+  "epoch": 10.0,
+  "eval_steps": 50,
+  "global_step": 500,
   "is_hyper_param_search": false,
   "is_local_process_zero": true,
   "is_world_process_zero": true,
   "log_history": [
     {
+      "epoch": 0.02,
+      "grad_norm": 520.4644775390625,
+      "learning_rate": 1.0000000000000002e-06,
+      "loss": 5.0715,
       "step": 1
     },
     {
+      "epoch": 0.04,
+      "grad_norm": 520.4757080078125,
+      "learning_rate": 2.0000000000000003e-06,
+      "loss": 5.072,
       "step": 2
     },
     {
+      "epoch": 0.06,
+      "grad_norm": 518.3416748046875,
+      "learning_rate": 3e-06,
+      "loss": 5.0258,
       "step": 3
     },
     {
+      "epoch": 0.08,
+      "grad_norm": 494.84033203125,
+      "learning_rate": 4.000000000000001e-06,
+      "loss": 4.5896,
       "step": 4
     },
     {
+      "epoch": 0.1,
+      "grad_norm": 413.3812561035156,
+      "learning_rate": 5e-06,
+      "loss": 3.8571,
       "step": 5
     },
     {
+      "epoch": 0.12,
+      "grad_norm": 145.95079040527344,
+      "learning_rate": 6e-06,
+      "loss": 2.352,
       "step": 6
     },
     {
+      "epoch": 0.14,
+      "grad_norm": 90.29026794433594,
+      "learning_rate": 7.000000000000001e-06,
+      "loss": 2.0816,
+      "step": 7
+    },
+    {
+      "epoch": 0.16,
+      "grad_norm": 15.726278305053711,
+      "learning_rate": 8.000000000000001e-06,
+      "loss": 1.712,
+      "step": 8
+    },
+    {
+      "epoch": 0.18,
+      "grad_norm": 7.5671234130859375,
+      "learning_rate": 9e-06,
+      "loss": 1.6391,
+      "step": 9
+    },
+    {
+      "epoch": 0.2,
+      "grad_norm": 5.866340637207031,
+      "learning_rate": 1e-05,
+      "loss": 1.3099,
+      "step": 10
+    },
+    {
+      "epoch": 0.22,
+      "grad_norm": 5.263887882232666,
+      "learning_rate": 1.1000000000000001e-05,
+      "loss": 1.1459,
+      "step": 11
+    },
+    {
+      "epoch": 0.24,
+      "grad_norm": 5.094001770019531,
+      "learning_rate": 1.2e-05,
+      "loss": 0.9704,
+      "step": 12
+    },
+    {
+      "epoch": 0.26,
+      "grad_norm": 6.369605541229248,
+      "learning_rate": 1.3000000000000001e-05,
+      "loss": 0.6508,
+      "step": 13
+    },
+    {
+      "epoch": 0.28,
+      "grad_norm": 5.96874475479126,
+      "learning_rate": 1.4000000000000001e-05,
+      "loss": 0.3816,
+      "step": 14
+    },
+    {
+      "epoch": 0.3,
+      "grad_norm": 5.774180889129639,
+      "learning_rate": 1.5e-05,
+      "loss": 0.2447,
+      "step": 15
+    },
+    {
+      "epoch": 0.32,
+      "grad_norm": 2.370732545852661,
+      "learning_rate": 1.6000000000000003e-05,
+      "loss": 0.1072,
+      "step": 16
+    },
+    {
+      "epoch": 0.34,
+      "grad_norm": 1.443320870399475,
+      "learning_rate": 1.7000000000000003e-05,
+      "loss": 0.0475,
+      "step": 17
+    },
+    {
+      "epoch": 0.36,
+      "grad_norm": 1.2612560987472534,
+      "learning_rate": 1.8e-05,
+      "loss": 0.0294,
+      "step": 18
+    },
+    {
+      "epoch": 0.38,
+      "grad_norm": 1.5056159496307373,
+      "learning_rate": 1.9e-05,
+      "loss": 0.0268,
+      "step": 19
+    },
+    {
+      "epoch": 0.4,
+      "grad_norm": 0.9794118404388428,
+      "learning_rate": 2e-05,
+      "loss": 0.0202,
+      "step": 20
+    },
+    {
+      "epoch": 0.42,
+      "grad_norm": 0.7181198596954346,
+      "learning_rate": 2.1e-05,
+      "loss": 0.0179,
+      "step": 21
+    },
+    {
+      "epoch": 0.44,
+      "grad_norm": 0.794865071773529,
+      "learning_rate": 2.2000000000000003e-05,
+      "loss": 0.0152,
+      "step": 22
+    },
+    {
+      "epoch": 0.46,
+      "grad_norm": 1.4117693901062012,
+      "learning_rate": 2.3000000000000003e-05,
+      "loss": 0.0144,
+      "step": 23
+    },
+    {
+      "epoch": 0.48,
+      "grad_norm": 3.0236976146698,
+      "learning_rate": 2.4e-05,
+      "loss": 0.0175,
+      "step": 24
+    },
+    {
+      "epoch": 0.5,
+      "grad_norm": 3.1012909412384033,
+      "learning_rate": 2.5e-05,
+      "loss": 0.0179,
+      "step": 25
+    },
+    {
+      "epoch": 0.52,
+      "grad_norm": 8.274877548217773,
+      "learning_rate": 2.6000000000000002e-05,
+      "loss": 0.06,
+      "step": 26
+    },
+    {
+      "epoch": 0.54,
+      "grad_norm": 4.594793319702148,
+      "learning_rate": 2.7000000000000002e-05,
+      "loss": 0.0403,
+      "step": 27
+    },
+    {
+      "epoch": 0.56,
+      "grad_norm": 3.254148006439209,
+      "learning_rate": 2.8000000000000003e-05,
+      "loss": 0.0218,
+      "step": 28
+    },
+    {
+      "epoch": 0.58,
+      "grad_norm": 2.123260498046875,
+      "learning_rate": 2.9e-05,
+      "loss": 0.0449,
+      "step": 29
+    },
+    {
+      "epoch": 0.6,
+      "grad_norm": 1.0731960535049438,
+      "learning_rate": 3e-05,
+      "loss": 0.0222,
+      "step": 30
+    },
+    {
+      "epoch": 0.62,
+      "grad_norm": 0.8489921689033508,
+      "learning_rate": 3.1e-05,
+      "loss": 0.0149,
+      "step": 31
+    },
+    {
+      "epoch": 0.64,
+      "grad_norm": 0.589127779006958,
+      "learning_rate": 3.2000000000000005e-05,
+      "loss": 0.022,
+      "step": 32
+    },
+    {
+      "epoch": 0.66,
+      "grad_norm": 0.42175689339637756,
+      "learning_rate": 3.3e-05,
+      "loss": 0.0109,
+      "step": 33
+    },
+    {
+      "epoch": 0.68,
+      "grad_norm": 0.19729836285114288,
+      "learning_rate": 3.4000000000000007e-05,
+      "loss": 0.0093,
+      "step": 34
+    },
+    {
+      "epoch": 0.7,
+      "grad_norm": 0.2009144425392151,
+      "learning_rate": 3.5e-05,
+      "loss": 0.0087,
+      "step": 35
+    },
+    {
+      "epoch": 0.72,
+      "grad_norm": 0.2093500941991806,
+      "learning_rate": 3.6e-05,
+      "loss": 0.0079,
+      "step": 36
+    },
+    {
+      "epoch": 0.74,
+      "grad_norm": 0.1865512728691101,
+      "learning_rate": 3.7e-05,
+      "loss": 0.0072,
+      "step": 37
+    },
+    {
+      "epoch": 0.76,
+      "grad_norm": 0.18552368879318237,
+      "learning_rate": 3.8e-05,
+      "loss": 0.0065,
+      "step": 38
+    },
+    {
+      "epoch": 0.78,
+      "grad_norm": 0.17754022777080536,
+      "learning_rate": 3.9000000000000006e-05,
+      "loss": 0.0058,
+      "step": 39
+    },
+    {
+      "epoch": 0.8,
+      "grad_norm": 0.1693771779537201,
+      "learning_rate": 4e-05,
+      "loss": 0.005,
+      "step": 40
+    },
+    {
+      "epoch": 0.82,
+      "grad_norm": 0.156035915017128,
+      "learning_rate": 4.1e-05,
+      "loss": 0.0042,
+      "step": 41
+    },
+    {
+      "epoch": 0.84,
+      "grad_norm": 0.13963238894939423,
+      "learning_rate": 4.2e-05,
+      "loss": 0.0035,
+      "step": 42
+    },
+    {
+      "epoch": 0.86,
+      "grad_norm": 0.12009837478399277,
+      "learning_rate": 4.3e-05,
+      "loss": 0.0029,
+      "step": 43
+    },
+    {
+      "epoch": 0.88,
+      "grad_norm": 0.09549227356910706,
+      "learning_rate": 4.4000000000000006e-05,
+      "loss": 0.0024,
+      "step": 44
+    },
+    {
+      "epoch": 0.9,
+      "grad_norm": 0.07229408621788025,
+      "learning_rate": 4.5e-05,
+      "loss": 0.002,
+      "step": 45
+    },
+    {
+      "epoch": 0.92,
+      "grad_norm": 0.05505015328526497,
+      "learning_rate": 4.600000000000001e-05,
+      "loss": 0.0018,
+      "step": 46
+    },
+    {
+      "epoch": 0.94,
+      "grad_norm": 0.038959477096796036,
+      "learning_rate": 4.7e-05,
+      "loss": 0.0016,
+      "step": 47
+    },
+    {
+      "epoch": 0.96,
+      "grad_norm": 0.029499763622879982,
+      "learning_rate": 4.8e-05,
+      "loss": 0.0015,
+      "step": 48
+    },
+    {
+      "epoch": 0.98,
+      "grad_norm": 0.026312170550227165,
+      "learning_rate": 4.9e-05,
+      "loss": 0.0014,
+      "step": 49
+    },
+    {
+      "epoch": 1.0,
+      "grad_norm": 0.020306073129177094,
+      "learning_rate": 5e-05,
+      "loss": 0.0014,
+      "step": 50
+    },
+    {
+      "epoch": 1.02,
+      "grad_norm": 0.014265279285609722,
+      "learning_rate": 4.999939076763487e-05,
+      "loss": 0.0013,
+      "step": 51
+    },
+    {
+      "epoch": 1.04,
+      "grad_norm": 0.02053067646920681,
+      "learning_rate": 4.999756310023261e-05,
+      "loss": 0.0013,
+      "step": 52
+    },
+    {
+      "epoch": 1.06,
+      "grad_norm": 0.015466387383639812,
+      "learning_rate": 4.999451708687114e-05,
+      "loss": 0.0013,
+      "step": 53
+    },
+    {
+      "epoch": 1.08,
+      "grad_norm": 0.015973195433616638,
+      "learning_rate": 4.999025287600886e-05,
+      "loss": 0.0012,
+      "step": 54
+    },
+    {
+      "epoch": 1.1,
+      "grad_norm": 0.008490358479321003,
+      "learning_rate": 4.99847706754774e-05,
+      "loss": 0.0012,
+      "step": 55
+    },
+    {
+      "epoch": 1.12,
+      "grad_norm": 0.012946813367307186,
+      "learning_rate": 4.997807075247146e-05,
+      "loss": 0.0012,
+      "step": 56
+    },
+    {
+      "epoch": 1.1400000000000001,
+      "grad_norm": 0.01698051020503044,
+      "learning_rate": 4.997015343353585e-05,
+      "loss": 0.0013,
+      "step": 57
+    },
+    {
+      "epoch": 1.16,
+      "grad_norm": 0.013274472206830978,
+      "learning_rate": 4.996101910454953e-05,
+      "loss": 0.0012,
+      "step": 58
+    },
+    {
+      "epoch": 1.18,
+      "grad_norm": 0.01776733063161373,
+      "learning_rate": 4.995066821070679e-05,
+      "loss": 0.0012,
+      "step": 59
+    },
+    {
+      "epoch": 1.2,
+      "grad_norm": 0.015046725980937481,
+      "learning_rate": 4.993910125649561e-05,
+      "loss": 0.0012,
+      "step": 60
+    },
+    {
+      "epoch": 1.22,
+      "grad_norm": 0.014767763204872608,
+      "learning_rate": 4.992631880567301e-05,
+      "loss": 0.0011,
+      "step": 61
+    },
+    {
+      "epoch": 1.24,
+      "grad_norm": 0.007155032362788916,
+      "learning_rate": 4.991232148123761e-05,
+      "loss": 0.0012,
+      "step": 62
+    },
+    {
+      "epoch": 1.26,
+      "grad_norm": 0.01570066623389721,
+      "learning_rate": 4.989710996539926e-05,
+      "loss": 0.0012,
+      "step": 63
+    },
+    {
+      "epoch": 1.28,
+      "grad_norm": 0.019080698490142822,
+      "learning_rate": 4.988068499954578e-05,
+      "loss": 0.0012,
+      "step": 64
+    },
+    {
+      "epoch": 1.3,
+      "grad_norm": 0.016244422644376755,
+      "learning_rate": 4.9863047384206835e-05,
+      "loss": 0.0011,
+      "step": 65
+    },
+    {
+      "epoch": 1.32,
+      "grad_norm": 0.06490402668714523,
+      "learning_rate": 4.984419797901491e-05,
+      "loss": 0.0013,
+      "step": 66
+    },
+    {
+      "epoch": 1.34,
+      "grad_norm": 0.02592933550477028,
+      "learning_rate": 4.982413770266342e-05,
+      "loss": 0.0012,
+      "step": 67
+    },
+    {
+      "epoch": 1.3599999999999999,
+      "grad_norm": 0.01940583623945713,
+      "learning_rate": 4.980286753286195e-05,
+      "loss": 0.0012,
+      "step": 68
+    },
+    {
+      "epoch": 1.38,
+      "grad_norm": 0.016211828216910362,
+      "learning_rate": 4.978038850628854e-05,
+      "loss": 0.0012,
+      "step": 69
+    },
+    {
+      "epoch": 1.4,
+      "grad_norm": 0.018053313717246056,
+      "learning_rate": 4.975670171853926e-05,
+      "loss": 0.0012,
+      "step": 70
+    },
+    {
+      "epoch": 1.42,
+      "grad_norm": 0.01438748836517334,
+      "learning_rate": 4.9731808324074717e-05,
+      "loss": 0.0012,
+      "step": 71
+    },
+    {
+      "epoch": 1.44,
+      "grad_norm": 0.011617658659815788,
+      "learning_rate": 4.9705709536163824e-05,
+      "loss": 0.0012,
+      "step": 72
+    },
+    {
+      "epoch": 1.46,
+      "grad_norm": 0.007649057544767857,
+      "learning_rate": 4.96784066268247e-05,
+      "loss": 0.0012,
+      "step": 73
+    },
+    {
+      "epoch": 1.48,
+      "grad_norm": 0.0060863615944981575,
+      "learning_rate": 4.964990092676263e-05,
+      "loss": 0.0012,
+      "step": 74
+    },
+    {
+      "epoch": 1.5,
+      "grad_norm": 0.011736256070435047,
+      "learning_rate": 4.962019382530521e-05,
+      "loss": 0.0012,
+      "step": 75
+    },
+    {
+      "epoch": 1.52,
+      "grad_norm": 0.00740490248426795,
+      "learning_rate": 4.9589286770334654e-05,
+      "loss": 0.0012,
+      "step": 76
+    },
+    {
+      "epoch": 1.54,
+      "grad_norm": 0.008237659931182861,
+      "learning_rate": 4.9557181268217227e-05,
+      "loss": 0.0012,
+      "step": 77
+    },
+    {
+      "epoch": 1.56,
+      "grad_norm": 0.005937286652624607,
+      "learning_rate": 4.952387888372979e-05,
+      "loss": 0.0012,
+      "step": 78
+    },
+    {
+      "epoch": 1.58,
+      "grad_norm": 0.010778284631669521,
+      "learning_rate": 4.94893812399836e-05,
+      "loss": 0.0012,
+      "step": 79
+    },
+    {
+      "epoch": 1.6,
+      "grad_norm": 0.005073127802461386,
+      "learning_rate": 4.9453690018345144e-05,
+      "loss": 0.0012,
+      "step": 80
+    },
+    {
+      "epoch": 1.62,
+      "grad_norm": 0.015029042027890682,
+      "learning_rate": 4.94168069583542e-05,
+      "loss": 0.0012,
+      "step": 81
+    },
+    {
+      "epoch": 1.6400000000000001,
+      "grad_norm": 0.003979986067861319,
+      "learning_rate": 4.937873385763908e-05,
+      "loss": 0.0012,
+      "step": 82
+    },
+    {
+      "epoch": 1.6600000000000001,
+      "grad_norm": 0.005284400191158056,
+      "learning_rate": 4.933947257182901e-05,
+      "loss": 0.0012,
+      "step": 83
+    },
+    {
+      "epoch": 1.6800000000000002,
+      "grad_norm": 0.0076846531592309475,
+      "learning_rate": 4.929902501446366e-05,
+      "loss": 0.0012,
+      "step": 84
+    },
+    {
+      "epoch": 1.7,
+      "grad_norm": 0.006414446514099836,
+      "learning_rate": 4.925739315689991e-05,
+      "loss": 0.0012,
+      "step": 85
+    },
+    {
+      "epoch": 1.72,
+      "grad_norm": 0.005861522629857063,
+      "learning_rate": 4.9214579028215776e-05,
+      "loss": 0.0012,
+      "step": 86
+    },
+    {
+      "epoch": 1.74,
+      "grad_norm": 0.007500852923840284,
+      "learning_rate": 4.917058471511149e-05,
+      "loss": 0.0012,
+      "step": 87
+    },
+    {
+      "epoch": 1.76,
+      "grad_norm": 0.010749063454568386,
+      "learning_rate": 4.912541236180779e-05,
+      "loss": 0.0012,
+      "step": 88
+    },
+    {
+      "epoch": 1.78,
+      "grad_norm": 0.01243392564356327,
+      "learning_rate": 4.907906416994146e-05,
+      "loss": 0.0012,
+      "step": 89
+    },
+    {
+      "epoch": 1.8,
+      "grad_norm": 0.006575236562639475,
+      "learning_rate": 4.9031542398457974e-05,
+      "loss": 0.0011,
+      "step": 90
+    },
+    {
+      "epoch": 1.8199999999999998,
+      "grad_norm": 0.018216770142316818,
+      "learning_rate": 4.898284936350144e-05,
+      "loss": 0.0012,
+      "step": 91
+    },
+    {
+      "epoch": 1.8399999999999999,
+      "grad_norm": 0.009646720252931118,
+      "learning_rate": 4.893298743830168e-05,
+      "loss": 0.0011,
+      "step": 92
+    },
+    {
+      "epoch": 1.8599999999999999,
+      "grad_norm": 0.00471867062151432,
+      "learning_rate": 4.888195905305859e-05,
+      "loss": 0.0011,
+      "step": 93
+    },
+    {
+      "epoch": 1.88,
+      "grad_norm": 0.010674097575247288,
+      "learning_rate": 4.882976669482367e-05,
+      "loss": 0.0011,
+      "step": 94
+    },
+    {
+      "epoch": 1.9,
+      "grad_norm": 0.010026497766375542,
+      "learning_rate": 4.877641290737884e-05,
+      "loss": 0.0011,
+      "step": 95
+    },
+    {
+      "epoch": 1.92,
+      "grad_norm": 0.01685321517288685,
+      "learning_rate": 4.8721900291112415e-05,
+      "loss": 0.0011,
+      "step": 96
+    },
+    {
+      "epoch": 1.94,
+      "grad_norm": 0.015050926245748997,
+      "learning_rate": 4.8666231502892415e-05,
+      "loss": 0.0011,
+      "step": 97
+    },
+    {
+      "epoch": 1.96,
+      "grad_norm": 0.008265609852969646,
+      "learning_rate": 4.860940925593703e-05,
+      "loss": 0.0011,
+      "step": 98
+    },
+    {
+      "epoch": 1.98,
+      "grad_norm": 0.022018827497959137,
+      "learning_rate": 4.855143631968242e-05,
+      "loss": 0.0012,
+      "step": 99
+    },
+    {
+      "epoch": 2.0,
+      "grad_norm": 0.01330670714378357,
+      "learning_rate": 4.849231551964771e-05,
+      "loss": 0.001,
+      "step": 100
+    },
+    {
+      "epoch": 2.02,
+      "grad_norm": 0.030819913372397423,
+      "learning_rate": 4.843204973729729e-05,
+      "loss": 0.0011,
+      "step": 101
+    },
+    {
+      "epoch": 2.04,
+      "grad_norm": 0.018151411786675453,
+      "learning_rate": 4.837064190990036e-05,
+      "loss": 0.001,
+      "step": 102
+    },
+    {
+      "epoch": 2.06,
+      "grad_norm": 0.022368013858795166,
+      "learning_rate": 4.830809503038781e-05,
+      "loss": 0.0011,
+      "step": 103
+    },
+    {
+      "epoch": 2.08,
+      "grad_norm": 0.0231326837092638,
+      "learning_rate": 4.8244412147206284e-05,
+      "loss": 0.0011,
+      "step": 104
+    },
+    {
+      "epoch": 2.1,
+      "grad_norm": 0.01942342147231102,
+      "learning_rate": 4.817959636416969e-05,
+      "loss": 0.001,
+      "step": 105
+    },
+    {
+      "epoch": 2.12,
+      "grad_norm": 0.025518309324979782,
+      "learning_rate": 4.8113650840307834e-05,
+      "loss": 0.001,
+      "step": 106
+    },
+    {
+      "epoch": 2.14,
+      "grad_norm": 0.01907534897327423,
+      "learning_rate": 4.8046578789712515e-05,
+      "loss": 0.001,
+      "step": 107
+    },
+    {
+      "epoch": 2.16,
+      "grad_norm": 0.06474534422159195,
+      "learning_rate": 4.797838348138086e-05,
+      "loss": 0.0012,
+      "step": 108
+    },
+    {
+      "epoch": 2.18,
+      "grad_norm": 0.014870468527078629,
+      "learning_rate": 4.790906823905599e-05,
+      "loss": 0.001,
+      "step": 109
+    },
+    {
+      "epoch": 2.2,
+      "grad_norm": 0.033130887895822525,
+      "learning_rate": 4.783863644106502e-05,
+      "loss": 0.0012,
+      "step": 110
+    },
+    {
+      "epoch": 2.22,
+      "grad_norm": 0.021351227536797523,
+      "learning_rate": 4.776709152015443e-05,
+      "loss": 0.0011,
+      "step": 111
+    },
+    {
+      "epoch": 2.24,
+      "grad_norm": 0.017795074731111526,
+      "learning_rate": 4.769443696332272e-05,
+      "loss": 0.0009,
+      "step": 112
+    },
+    {
+      "epoch": 2.26,
+      "grad_norm": 0.014288947917521,
+      "learning_rate": 4.762067631165049e-05,
+      "loss": 0.001,
+      "step": 113
+    },
+    {
+      "epoch": 2.2800000000000002,
+      "grad_norm": 0.019312620162963867,
+      "learning_rate": 4.754581316012785e-05,
+      "loss": 0.001,
+      "step": 114
+    },
+    {
+      "epoch": 2.3,
+      "grad_norm": 0.015632573515176773,
+      "learning_rate": 4.7469851157479177e-05,
+      "loss": 0.001,
+      "step": 115
+    },
+    {
+      "epoch": 2.32,
+      "grad_norm": 0.012786476872861385,
+      "learning_rate": 4.7392794005985326e-05,
+      "loss": 0.001,
+      "step": 116
+    },
+    {
+      "epoch": 2.34,
+      "grad_norm": 0.042442597448825836,
+      "learning_rate": 4.731464546130314e-05,
+      "loss": 0.001,
+      "step": 117
+    },
+    {
+      "epoch": 2.36,
+      "grad_norm": 0.021612655371427536,
+      "learning_rate": 4.723540933228244e-05,
+      "loss": 0.0009,
+      "step": 118
+    },
+    {
+      "epoch": 2.38,
+      "grad_norm": 0.022924797609448433,
+      "learning_rate": 4.715508948078037e-05,
+      "loss": 0.001,
+      "step": 119
+    },
+    {
+      "epoch": 2.4,
+      "grad_norm": 0.025588147342205048,
+      "learning_rate": 4.707368982147318e-05,
+      "loss": 0.0009,
+      "step": 120
+    },
+    {
+      "epoch": 2.42,
+      "grad_norm": 0.01950550638139248,
+      "learning_rate": 4.6991214321665414e-05,
+      "loss": 0.0009,
+      "step": 121
+    },
+    {
+      "epoch": 2.44,
+      "grad_norm": 0.027708236128091812,
+      "learning_rate": 4.690766700109659e-05,
+      "loss": 0.001,
+      "step": 122
+    },
+    {
+      "epoch": 2.46,
+      "grad_norm": 0.032382309436798096,
+      "learning_rate": 4.682305193174524e-05,
+      "loss": 0.0009,
+      "step": 123
+    },
+    {
+      "epoch": 2.48,
+      "grad_norm": 0.02310575172305107,
+      "learning_rate": 4.6737373237630476e-05,
+      "loss": 0.001,
+      "step": 124
+    },
+    {
+      "epoch": 2.5,
+      "grad_norm": 0.04070095717906952,
+      "learning_rate": 4.665063509461097e-05,
+      "loss": 0.0009,
+      "step": 125
+    },
+    {
+      "epoch": 2.52,
+      "grad_norm": 0.041273683309555054,
+      "learning_rate": 4.656284173018144e-05,
+      "loss": 0.001,
+      "step": 126
+    },
+    {
+      "epoch": 2.54,
+      "grad_norm": 0.014590524137020111,
+      "learning_rate": 4.6473997423266614e-05,
+      "loss": 0.0009,
+      "step": 127
+    },
+    {
+      "epoch": 2.56,
+      "grad_norm": 0.02202373370528221,
+      "learning_rate": 4.638410650401267e-05,
+      "loss": 0.001,
+      "step": 128
+    },
+    {
+      "epoch": 2.58,
+      "grad_norm": 0.015360541641712189,
+      "learning_rate": 4.629317335357619e-05,
+      "loss": 0.001,
+      "step": 129
+    },
+    {
+      "epoch": 2.6,
+      "grad_norm": 0.01654900051653385,
+      "learning_rate": 4.620120240391065e-05,
+      "loss": 0.001,
+      "step": 130
+    },
+    {
+      "epoch": 2.62,
+      "grad_norm": 0.01125663798302412,
+      "learning_rate": 4.610819813755038e-05,
+      "loss": 0.001,
+      "step": 131
+    },
+    {
+      "epoch": 2.64,
+      "grad_norm": 0.012902496382594109,
+      "learning_rate": 4.601416508739211e-05,
+      "loss": 0.0009,
+      "step": 132
+    },
+    {
+      "epoch": 2.66,
+      "grad_norm": 0.0195336751639843,
+      "learning_rate": 4.591910783647404e-05,
+      "loss": 0.0011,
+      "step": 133
+    },
+    {
+      "epoch": 2.68,
+      "grad_norm": 0.02390046790242195,
+      "learning_rate": 4.5823031017752485e-05,
+      "loss": 0.0011,
+      "step": 134
+    },
+    {
+      "epoch": 2.7,
+      "grad_norm": 0.014201649464666843,
+      "learning_rate": 4.572593931387604e-05,
+      "loss": 0.0009,
+      "step": 135
+    },
+    {
+      "epoch": 2.7199999999999998,
+      "grad_norm": 0.012115216813981533,
+      "learning_rate": 4.562783745695738e-05,
+      "loss": 0.0009,
+      "step": 136
+    },
+    {
+      "epoch": 2.74,
+      "grad_norm": 0.022056447342038155,
+      "learning_rate": 4.5528730228342605e-05,
+      "loss": 0.0009,
+      "step": 137
+    },
+    {
+      "epoch": 2.76,
+      "grad_norm": 0.019029831513762474,
+      "learning_rate": 4.542862245837821e-05,
+      "loss": 0.001,
+      "step": 138
+    },
+    {
+      "epoch": 2.7800000000000002,
+      "grad_norm": 0.020401952788233757,
+      "learning_rate": 4.532751902617569e-05,
+      "loss": 0.0009,
+      "step": 139
+    },
+    {
+      "epoch": 2.8,
+      "grad_norm": 0.018967969343066216,
+      "learning_rate": 4.522542485937369e-05,
+      "loss": 0.0009,
+      "step": 140
+    },
+    {
+      "epoch": 2.82,
+      "grad_norm": 0.034043021500110626,
+      "learning_rate": 4.512234493389785e-05,
+      "loss": 0.0011,
+      "step": 141
+    },
+    {
+      "epoch": 2.84,
+      "grad_norm": 0.02043725550174713,
+      "learning_rate": 4.5018284273718336e-05,
+      "loss": 0.001,
+      "step": 142
+    },
+    {
+      "epoch": 2.86,
+      "grad_norm": 0.014444992877542973,
+      "learning_rate": 4.491324795060491e-05,
+      "loss": 0.0009,
+      "step": 143
+    },
+    {
+      "epoch": 2.88,
+      "grad_norm": 0.015194895677268505,
+      "learning_rate": 4.480724108387977e-05,
+      "loss": 0.0009,
+      "step": 144
+    },
+    {
+      "epoch": 2.9,
+      "grad_norm": 0.010989692993462086,
+      "learning_rate": 4.4700268840168045e-05,
+      "loss": 0.0009,
+      "step": 145
+    },
+    {
+      "epoch": 2.92,
+      "grad_norm": 0.026291556656360626,
+      "learning_rate": 4.4592336433146e-05,
+      "loss": 0.001,
+      "step": 146
+    },
+    {
+      "epoch": 2.94,
+      "grad_norm": 0.015329469926655293,
+      "learning_rate": 4.448344912328686e-05,
+      "loss": 0.0009,
+      "step": 147
+    },
+    {
+      "epoch": 2.96,
+      "grad_norm": 0.013111543841660023,
+      "learning_rate": 4.4373612217604496e-05,
+      "loss": 0.001,
+      "step": 148
+    },
+    {
+      "epoch": 2.98,
+      "grad_norm": 0.013395372778177261,
+      "learning_rate": 4.426283106939474e-05,
+      "loss": 0.0009,
+      "step": 149
+    },
+    {
+      "epoch": 3.0,
+      "grad_norm": 0.013448159210383892,
+      "learning_rate": 4.415111107797445e-05,
+      "loss": 0.0008,
+      "step": 150
+    },
+    {
+      "epoch": 3.02,
+      "grad_norm": 0.01918015442788601,
+      "learning_rate": 4.403845768841842e-05,
+      "loss": 0.0009,
+      "step": 151
+    },
+    {
+      "epoch": 3.04,
+      "grad_norm": 0.017055032774806023,
+      "learning_rate": 4.3924876391293915e-05,
+      "loss": 0.0009,
+      "step": 152
+    },
+    {
+      "epoch": 3.06,
+      "grad_norm": 0.006383760366588831,
+      "learning_rate": 4.381037272239311e-05,
+      "loss": 0.0009,
+      "step": 153
+    },
+    {
+      "epoch": 3.08,
+      "grad_norm": 0.016412733122706413,
+      "learning_rate": 4.36949522624633e-05,
+      "loss": 0.0009,
+      "step": 154
+    },
+    {
+      "epoch": 3.1,
+      "grad_norm": 0.01369386911392212,
+      "learning_rate": 4.357862063693486e-05,
+      "loss": 0.0009,
+      "step": 155
+    },
+    {
+      "epoch": 3.12,
+      "grad_norm": 0.01586688682436943,
+      "learning_rate": 4.3461383515647106e-05,
+      "loss": 0.001,
+      "step": 156
+    },
+    {
+      "epoch": 3.14,
+      "grad_norm": 0.009514207020401955,
+      "learning_rate": 4.334324661257191e-05,
+      "loss": 0.0008,
+      "step": 157
+    },
+    {
+      "epoch": 3.16,
+      "grad_norm": 0.011565557681024075,
+      "learning_rate": 4.3224215685535294e-05,
+      "loss": 0.0009,
+      "step": 158
+    },
+    {
+      "epoch": 3.18,
+      "grad_norm": 0.01942281424999237,
+      "learning_rate": 4.3104296535936695e-05,
+      "loss": 0.001,
+      "step": 159
+    },
+    {
+      "epoch": 3.2,
+      "grad_norm": 0.025319915264844894,
+      "learning_rate": 4.2983495008466276e-05,
+      "loss": 0.001,
+      "step": 160
+    },
+    {
+      "epoch": 3.22,
+      "grad_norm": 0.020152214914560318,
+      "learning_rate": 4.2861816990820084e-05,
+      "loss": 0.0009,
+      "step": 161
+    },
+    {
+      "epoch": 3.24,
+      "grad_norm": 0.014544936828315258,
+      "learning_rate": 4.273926841341302e-05,
+      "loss": 0.0009,
+      "step": 162
+    },
+    {
+      "epoch": 3.26,
+      "grad_norm": 0.013562367297708988,
+      "learning_rate": 4.261585524908987e-05,
+      "loss": 0.0009,
+      "step": 163
+    },
+    {
+      "epoch": 3.2800000000000002,
+      "grad_norm": 0.016432832926511765,
+      "learning_rate": 4.249158351283414e-05,
+      "loss": 0.001,
+      "step": 164
+    },
+    {
+      "epoch": 3.3,
+      "grad_norm": 0.01196199283003807,
+      "learning_rate": 4.2366459261474933e-05,
+      "loss": 0.0009,
+      "step": 165
+    },
+    {
+      "epoch": 3.32,
+      "grad_norm": 0.012741833925247192,
+      "learning_rate": 4.224048859339175e-05,
+      "loss": 0.0009,
+      "step": 166
+    },
+    {
+      "epoch": 3.34,
+      "grad_norm": 0.015541434288024902,
+      "learning_rate": 4.211367764821722e-05,
+      "loss": 0.0008,
+      "step": 167
+    },
+    {
+      "epoch": 3.36,
+      "grad_norm": 0.010746212676167488,
+      "learning_rate": 4.198603260653792e-05,
+      "loss": 0.0009,
+      "step": 168
+    },
+    {
+      "epoch": 3.38,
+      "grad_norm": 0.014585288241505623,
+      "learning_rate": 4.185755968959308e-05,
+      "loss": 0.0009,
+      "step": 169
+    },
+    {
+      "epoch": 3.4,
+      "grad_norm": 0.009942736476659775,
+      "learning_rate": 4.172826515897146e-05,
+      "loss": 0.0009,
+      "step": 170
+    },
+    {
+      "epoch": 3.42,
+      "grad_norm": 0.017286481335759163,
+      "learning_rate": 4.1598155316306044e-05,
+      "loss": 0.0009,
+      "step": 171
+    },
+    {
+      "epoch": 3.44,
+      "grad_norm": 0.013385478407144547,
+      "learning_rate": 4.146723650296701e-05,
+      "loss": 0.0008,
+      "step": 172
+    },
+    {
+      "epoch": 3.46,
+      "grad_norm": 0.05108925700187683,
+      "learning_rate": 4.133551509975264e-05,
+      "loss": 0.0009,
+      "step": 173
+    },
+    {
+      "epoch": 3.48,
+      "grad_norm": 0.013413962908089161,
+      "learning_rate": 4.1202997526578276e-05,
+      "loss": 0.0008,
+      "step": 174
+    },
+    {
+      "epoch": 3.5,
+      "grad_norm": 0.010358484461903572,
+      "learning_rate": 4.1069690242163484e-05,
+      "loss": 0.0008,
+      "step": 175
+    },
+    {
+      "epoch": 3.52,
+      "grad_norm": 0.013632961548864841,
+      "learning_rate": 4.093559974371725e-05,
+      "loss": 0.0008,
+      "step": 176
+    },
+    {
+      "epoch": 3.54,
+      "grad_norm": 0.009852699935436249,
+      "learning_rate": 4.080073256662127e-05,
+      "loss": 0.0009,
+      "step": 177
+    },
+    {
+      "epoch": 3.56,
+      "grad_norm": 0.007956922985613346,
+      "learning_rate": 4.066509528411152e-05,
+      "loss": 0.0009,
+      "step": 178
+    },
+    {
+      "epoch": 3.58,
+      "grad_norm": 0.016999222338199615,
+      "learning_rate": 4.052869450695776e-05,
+      "loss": 0.001,
+      "step": 179
+    },
+    {
+      "epoch": 3.6,
+      "grad_norm": 0.013647818006575108,
+      "learning_rate": 4.039153688314145e-05,
+      "loss": 0.001,
+      "step": 180
+    },
+    {
+      "epoch": 3.62,
+      "grad_norm": 0.012612687423825264,
+      "learning_rate": 4.02536290975317e-05,
+      "loss": 0.0008,
+      "step": 181
+    },
+    {
+      "epoch": 3.64,
+      "grad_norm": 0.01253987941890955,
+      "learning_rate": 4.011497787155938e-05,
+      "loss": 0.0008,
+      "step": 182
+    },
+    {
+      "epoch": 3.66,
+      "grad_norm": 0.01009274460375309,
+      "learning_rate": 3.997558996288965e-05,
+      "loss": 0.0009,
+      "step": 183
+    },
+    {
+      "epoch": 3.68,
+      "grad_norm": 0.012114924378693104,
+      "learning_rate": 3.983547216509254e-05,
+      "loss": 0.0009,
+      "step": 184
+    },
+    {
+      "epoch": 3.7,
+      "grad_norm": 0.00788212101906538,
+      "learning_rate": 3.969463130731183e-05,
+      "loss": 0.0008,
+      "step": 185
+    },
+    {
+      "epoch": 3.7199999999999998,
+      "grad_norm": 0.009055244736373425,
+      "learning_rate": 3.955307425393224e-05,
+      "loss": 0.0007,
+      "step": 186
+    },
+    {
+      "epoch": 3.74,
+      "grad_norm": 0.015629315748810768,
+      "learning_rate": 3.941080790424484e-05,
+      "loss": 0.0008,
+      "step": 187
+    },
+    {
+      "epoch": 3.76,
+      "grad_norm": 0.012710140086710453,
+      "learning_rate": 3.92678391921108e-05,
+      "loss": 0.0008,
+      "step": 188
+    },
+    {
+      "epoch": 3.7800000000000002,
+      "grad_norm": 0.013408125378191471,
+      "learning_rate": 3.912417508562345e-05,
+      "loss": 0.0008,
+      "step": 189
+    },
+    {
+      "epoch": 3.8,
+      "grad_norm": 0.012659719213843346,
+      "learning_rate": 3.897982258676867e-05,
+      "loss": 0.0008,
+      "step": 190
+    },
+    {
+      "epoch": 3.82,
+      "grad_norm": 0.010696173645555973,
+      "learning_rate": 3.883478873108361e-05,
+      "loss": 0.0008,
+      "step": 191
+    },
+    {
+      "epoch": 3.84,
+      "grad_norm": 0.01534357201308012,
+      "learning_rate": 3.868908058731376e-05,
+      "loss": 0.0007,
+      "step": 192
+    },
+    {
+      "epoch": 3.86,
+      "grad_norm": 0.021090975031256676,
+      "learning_rate": 3.85427052570685e-05,
+      "loss": 0.001,
+      "step": 193
+    },
+    {
+      "epoch": 3.88,
+      "grad_norm": 0.007561246864497662,
+      "learning_rate": 3.8395669874474915e-05,
+      "loss": 0.0007,
+      "step": 194
+    },
+    {
+      "epoch": 3.9,
+      "grad_norm": 0.035602714866399765,
+      "learning_rate": 3.824798160583012e-05,
+      "loss": 0.0009,
+      "step": 195
+    },
+    {
+      "epoch": 3.92,
+      "grad_norm": 0.012646924704313278,
+      "learning_rate": 3.8099647649251986e-05,
+      "loss": 0.0007,
+      "step": 196
+    },
+    {
+      "epoch": 3.94,
+      "grad_norm": 0.018672510981559753,
+      "learning_rate": 3.795067523432826e-05,
+      "loss": 0.001,
+      "step": 197
+    },
+    {
+      "epoch": 3.96,
+      "grad_norm": 0.03359285369515419,
+      "learning_rate": 3.780107162176429e-05,
+      "loss": 0.0009,
+      "step": 198
+    },
+    {
+      "epoch": 3.98,
+      "grad_norm": 0.008940763771533966,
+      "learning_rate": 3.765084410302909e-05,
+      "loss": 0.0008,
+      "step": 199
+    },
+    {
+      "epoch": 4.0,
+      "grad_norm": 0.013762730173766613,
+      "learning_rate": 3.7500000000000003e-05,
+      "loss": 0.0009,
+      "step": 200
+    },
+    {
+      "epoch": 4.02,
+      "grad_norm": 0.01622450351715088,
+      "learning_rate": 3.7348546664605777e-05,
+      "loss": 0.0008,
+      "step": 201
+    },
+    {
+      "epoch": 4.04,
+      "grad_norm": 0.017409563064575195,
+      "learning_rate": 3.719649147846832e-05,
+      "loss": 0.0009,
+      "step": 202
+    },
+    {
+      "epoch": 4.06,
+      "grad_norm": 0.011531161144375801,
+      "learning_rate": 3.704384185254288e-05,
+      "loss": 0.0008,
+      "step": 203
+    },
+    {
+      "epoch": 4.08,
+      "grad_norm": 0.00890585407614708,
+      "learning_rate": 3.689060522675689e-05,
+      "loss": 0.0008,
+      "step": 204
+    },
+    {
+      "epoch": 4.1,
+      "grad_norm": 0.009238718077540398,
+      "learning_rate": 3.673678906964727e-05,
+      "loss": 0.0008,
+      "step": 205
+    },
+    {
+      "epoch": 4.12,
+      "grad_norm": 0.011698647402226925,
+      "learning_rate": 3.6582400877996546e-05,
+      "loss": 0.0008,
+      "step": 206
+    },
+    {
+      "epoch": 4.14,
+      "grad_norm": 0.0154410470277071,
+      "learning_rate": 3.642744817646736e-05,
+      "loss": 0.0008,
+      "step": 207
+    },
+    {
+      "epoch": 4.16,
+      "grad_norm": 0.02395373024046421,
+      "learning_rate": 3.627193851723577e-05,
+      "loss": 0.0007,
+      "step": 208
+    },
+    {
+      "epoch": 4.18,
+      "grad_norm": 0.014127401635050774,
+      "learning_rate": 3.611587947962319e-05,
+      "loss": 0.0009,
+      "step": 209
+    },
+    {
+      "epoch": 4.2,
+      "grad_norm": 0.016634410247206688,
+      "learning_rate": 3.5959278669726935e-05,
+      "loss": 0.0009,
+      "step": 210
+    },
+    {
+      "epoch": 4.22,
+      "grad_norm": 0.026153067126870155,
+      "learning_rate": 3.580214372004956e-05,
+      "loss": 0.0008,
+      "step": 211
+    },
+    {
+      "epoch": 4.24,
+      "grad_norm": 0.035530734807252884,
+      "learning_rate": 3.564448228912682e-05,
+      "loss": 0.0008,
+      "step": 212
+    },
+    {
+      "epoch": 4.26,
+      "grad_norm": 0.014608434401452541,
+      "learning_rate": 3.548630206115443e-05,
+      "loss": 0.0007,
+      "step": 213
+    },
+    {
+      "epoch": 4.28,
+      "grad_norm": 0.016531968489289284,
+      "learning_rate": 3.532761074561355e-05,
+      "loss": 0.0009,
+      "step": 214
+    },
+    {
+      "epoch": 4.3,
+      "grad_norm": 0.00973664689809084,
+      "learning_rate": 3.516841607689501e-05,
+      "loss": 0.0007,
+      "step": 215
+    },
+    {
+      "epoch": 4.32,
+      "grad_norm": 0.01250792108476162,
+      "learning_rate": 3.5008725813922386e-05,
+      "loss": 0.0007,
+      "step": 216
+    },
+    {
+      "epoch": 4.34,
+      "grad_norm": 0.021898312494158745,
+      "learning_rate": 3.484854773977378e-05,
+      "loss": 0.0007,
+      "step": 217
+    },
+    {
+      "epoch": 4.36,
+      "grad_norm": 0.04025641456246376,
+      "learning_rate": 3.4687889661302576e-05,
+      "loss": 0.0008,
+      "step": 218
+    },
+    {
+      "epoch": 4.38,
+      "grad_norm": 0.015246308408677578,
+      "learning_rate": 3.452675940875686e-05,
+      "loss": 0.0008,
+      "step": 219
+    },
+    {
+      "epoch": 4.4,
+      "grad_norm": 0.010531401261687279,
+      "learning_rate": 3.436516483539781e-05,
+      "loss": 0.0007,
+      "step": 220
+    },
+    {
+      "epoch": 4.42,
+      "grad_norm": 0.014213008806109428,
+      "learning_rate": 3.4203113817116957e-05,
+      "loss": 0.0007,
+      "step": 221
+    },
+    {
+      "epoch": 4.44,
+      "grad_norm": 0.035328883677721024,
+      "learning_rate": 3.4040614252052305e-05,
+      "loss": 0.0008,
+      "step": 222
+    },
+    {
+      "epoch": 4.46,
+      "grad_norm": 0.019592702388763428,
+      "learning_rate": 3.387767406020343e-05,
+      "loss": 0.0008,
+      "step": 223
+    },
+    {
+      "epoch": 4.48,
+      "grad_norm": 0.014544193632900715,
+      "learning_rate": 3.3714301183045385e-05,
+      "loss": 0.0007,
+      "step": 224
+    },
+    {
+      "epoch": 4.5,
+      "grad_norm": 0.01257528830319643,
+      "learning_rate": 3.355050358314172e-05,
+      "loss": 0.0007,
+      "step": 225
+    },
+    {
+      "epoch": 4.52,
+      "grad_norm": 0.027983270585536957,
+      "learning_rate": 3.338628924375638e-05,
+      "loss": 0.0006,
+      "step": 226
+    },
+    {
+      "epoch": 4.54,
+      "grad_norm": 0.02479342371225357,
+      "learning_rate": 3.322166616846458e-05,
+      "loss": 0.0008,
+      "step": 227
+    },
+    {
+      "epoch": 4.5600000000000005,
+      "grad_norm": 0.03580480441451073,
+      "learning_rate": 3.305664238076278e-05,
+      "loss": 0.0008,
+      "step": 228
+    },
+    {
+      "epoch": 4.58,
+      "grad_norm": 0.028629954904317856,
+      "learning_rate": 3.289122592367757e-05,
+      "loss": 0.0008,
+      "step": 229
+    },
+    {
+      "epoch": 4.6,
+      "grad_norm": 0.029527271166443825,
+      "learning_rate": 3.272542485937369e-05,
+      "loss": 0.0011,
+      "step": 230
+    },
+    {
+      "epoch": 4.62,
+      "grad_norm": 0.01633339561522007,
+      "learning_rate": 3.2559247268761115e-05,
+      "loss": 0.0008,
+      "step": 231
+    },
+    {
+      "epoch": 4.64,
+      "grad_norm": 0.02296227402985096,
+      "learning_rate": 3.239270125110117e-05,
+      "loss": 0.0008,
+      "step": 232
+    },
+    {
+      "epoch": 4.66,
+      "grad_norm": 0.022086285054683685,
+      "learning_rate": 3.222579492361179e-05,
+      "loss": 0.0008,
+      "step": 233
+    },
+    {
+      "epoch": 4.68,
+      "grad_norm": 0.015360180288553238,
+      "learning_rate": 3.205853642107192e-05,
+      "loss": 0.0008,
+      "step": 234
+    },
+    {
+      "epoch": 4.7,
+      "grad_norm": 0.020640576258301735,
+      "learning_rate": 3.1890933895424976e-05,
+      "loss": 0.0009,
+      "step": 235
+    },
+    {
+      "epoch": 4.72,
+      "grad_norm": 0.06557556986808777,
+      "learning_rate": 3.172299551538164e-05,
+      "loss": 0.0009,
+      "step": 236
+    },
+    {
+      "epoch": 4.74,
+      "grad_norm": 0.027576349675655365,
+      "learning_rate": 3.155472946602162e-05,
+      "loss": 0.0008,
+      "step": 237
+    },
+    {
+      "epoch": 4.76,
+      "grad_norm": 0.019312813878059387,
+      "learning_rate": 3.138614394839476e-05,
+      "loss": 0.0008,
+      "step": 238
+    },
+    {
+      "epoch": 4.78,
+      "grad_norm": 0.027863649651408195,
+      "learning_rate": 3.121724717912138e-05,
+      "loss": 0.0009,
+      "step": 239
+    },
+    {
+      "epoch": 4.8,
+      "grad_norm": 0.019097020849585533,
+      "learning_rate": 3.104804738999169e-05,
+      "loss": 0.0008,
+      "step": 240
+    },
+    {
+      "epoch": 4.82,
+      "grad_norm": 0.025937069207429886,
+      "learning_rate": 3.087855282756475e-05,
+      "loss": 0.0008,
+      "step": 241
+    },
+    {
+      "epoch": 4.84,
+      "grad_norm": 0.01815580017864704,
+      "learning_rate": 3.0708771752766394e-05,
+      "loss": 0.0009,
+      "step": 242
+    },
+    {
+      "epoch": 4.86,
+      "grad_norm": 0.02074621059000492,
+      "learning_rate": 3.053871244048669e-05,
+      "loss": 0.0008,
+      "step": 243
+    },
+    {
+      "epoch": 4.88,
+      "grad_norm": 0.015941645950078964,
+      "learning_rate": 3.0368383179176585e-05,
+      "loss": 0.0008,
+      "step": 244
+    },
+    {
+      "epoch": 4.9,
+      "grad_norm": 0.011930015869438648,
+      "learning_rate": 3.0197792270443982e-05,
+      "loss": 0.0007,
+      "step": 245
+    },
+    {
+      "epoch": 4.92,
+      "grad_norm": 0.014257809147238731,
+      "learning_rate": 3.002694802864912e-05,
+      "loss": 0.0007,
+      "step": 246
+    },
+    {
+      "epoch": 4.9399999999999995,
+      "grad_norm": 0.029351714998483658,
+      "learning_rate": 2.98558587804993e-05,
+      "loss": 0.0009,
+      "step": 247
+    },
+    {
+      "epoch": 4.96,
+      "grad_norm": 0.046229105442762375,
+      "learning_rate": 2.9684532864643122e-05,
+      "loss": 0.0009,
+      "step": 248
+    },
+    {
+      "epoch": 4.98,
+      "grad_norm": 0.4148816764354706,
+      "learning_rate": 2.9512978631264006e-05,
+      "loss": 0.0009,
+      "step": 249
+    },
+    {
+      "epoch": 5.0,
+      "grad_norm": 0.12848469614982605,
+      "learning_rate": 2.9341204441673266e-05,
+      "loss": 0.001,
+      "step": 250
+    },
+    {
+      "epoch": 5.02,
+      "grad_norm": 0.01425765361636877,
+      "learning_rate": 2.916921866790256e-05,
+      "loss": 0.0008,
+      "step": 251
+    },
+    {
+      "epoch": 5.04,
+      "grad_norm": 0.020687583833932877,
+      "learning_rate": 2.8997029692295874e-05,
+      "loss": 0.0008,
+      "step": 252
+    },
+    {
+      "epoch": 5.06,
+      "grad_norm": 0.016962770372629166,
+      "learning_rate": 2.8824645907100954e-05,
+      "loss": 0.0009,
+      "step": 253
+    },
+    {
+      "epoch": 5.08,
+      "grad_norm": 0.016928350552916527,
+      "learning_rate": 2.8652075714060295e-05,
+      "loss": 0.0008,
+      "step": 254
+    },
+    {
+      "epoch": 5.1,
+      "grad_norm": 0.035252753645181656,
+      "learning_rate": 2.8479327524001636e-05,
+      "loss": 0.0009,
+      "step": 255
+    },
+    {
+      "epoch": 5.12,
+      "grad_norm": 0.015413952991366386,
+      "learning_rate": 2.8306409756428064e-05,
+      "loss": 0.0008,
+      "step": 256
+    },
+    {
+      "epoch": 5.14,
+      "grad_norm": 0.023872479796409607,
+      "learning_rate": 2.8133330839107608e-05,
+      "loss": 0.0009,
+      "step": 257
+    },
+    {
+      "epoch": 5.16,
+      "grad_norm": 0.013683849945664406,
+      "learning_rate": 2.7960099207662532e-05,
+      "loss": 0.0008,
+      "step": 258
+    },
+    {
+      "epoch": 5.18,
+      "grad_norm": 0.021965356543660164,
+      "learning_rate": 2.7786723305158136e-05,
+      "loss": 0.0009,
+      "step": 259
+    },
+    {
+      "epoch": 5.2,
+      "grad_norm": 0.018004387617111206,
+      "learning_rate": 2.761321158169134e-05,
+      "loss": 0.0008,
+      "step": 260
+    },
+    {
+      "epoch": 5.22,
+      "grad_norm": 0.10878186672925949,
+      "learning_rate": 2.7439572493978736e-05,
+      "loss": 0.0009,
+      "step": 261
+    },
+    {
+      "epoch": 5.24,
+      "grad_norm": 0.016990287229418755,
+      "learning_rate": 2.726581450494451e-05,
+      "loss": 0.0009,
+      "step": 262
+    },
+    {
+      "epoch": 5.26,
+      "grad_norm": 0.019985131919384003,
+      "learning_rate": 2.7091946083307896e-05,
+      "loss": 0.001,
+      "step": 263
+    },
+    {
+      "epoch": 5.28,
+      "grad_norm": 0.019697532057762146,
+      "learning_rate": 2.6917975703170466e-05,
+      "loss": 0.001,
+      "step": 264
+    },
+    {
+      "epoch": 5.3,
+      "grad_norm": 0.02688547596335411,
+      "learning_rate": 2.674391184360313e-05,
+      "loss": 0.001,
+      "step": 265
+    },
+    {
+      "epoch": 5.32,
+      "grad_norm": 0.019830502569675446,
+      "learning_rate": 2.656976298823284e-05,
+      "loss": 0.001,
+      "step": 266
+    },
+    {
+      "epoch": 5.34,
+      "grad_norm": 0.00932388100773096,
+      "learning_rate": 2.6395537624829096e-05,
+      "loss": 0.0009,
+      "step": 267
+    },
+    {
+      "epoch": 5.36,
+      "grad_norm": 0.022172892466187477,
+      "learning_rate": 2.6221244244890336e-05,
+      "loss": 0.0011,
+      "step": 268
+    },
+    {
+      "epoch": 5.38,
+      "grad_norm": 0.013382075354456902,
+      "learning_rate": 2.604689134322999e-05,
+      "loss": 0.0009,
+      "step": 269
+    },
+    {
+      "epoch": 5.4,
+      "grad_norm": 0.011492418125271797,
+      "learning_rate": 2.587248741756253e-05,
+      "loss": 0.0009,
+      "step": 270
+    },
+    {
+      "epoch": 5.42,
+      "grad_norm": 0.014342604205012321,
+      "learning_rate": 2.5698040968089225e-05,
+      "loss": 0.001,
+      "step": 271
+    },
+    {
+      "epoch": 5.44,
+      "grad_norm": 0.014334768988192081,
+      "learning_rate": 2.5523560497083926e-05,
+      "loss": 0.0009,
+      "step": 272
+    },
+    {
+      "epoch": 5.46,
+      "grad_norm": 0.00868277158588171,
+      "learning_rate": 2.5349054508478637e-05,
+      "loss": 0.0009,
+      "step": 273
+    },
+    {
+      "epoch": 5.48,
+      "grad_norm": 0.015255315229296684,
+      "learning_rate": 2.517453150744904e-05,
+      "loss": 0.0009,
+      "step": 274
+    },
+    {
+      "epoch": 5.5,
+      "grad_norm": 0.014761993661522865,
+      "learning_rate": 2.5e-05,
+      "loss": 0.0009,
+      "step": 275
+    },
+    {
+      "epoch": 5.52,
+      "grad_norm": 0.01535094901919365,
+      "learning_rate": 2.4825468492550964e-05,
+      "loss": 0.0009,
+      "step": 276
+    },
+    {
+      "epoch": 5.54,
+      "grad_norm": 0.018661124631762505,
+      "learning_rate": 2.4650945491521372e-05,
+      "loss": 0.001,
+      "step": 277
+    },
+    {
+      "epoch": 5.5600000000000005,
+      "grad_norm": 0.015340641140937805,
+      "learning_rate": 2.447643950291608e-05,
+      "loss": 0.001,
+      "step": 278
+    },
+    {
+      "epoch": 5.58,
+      "grad_norm": 0.011068003252148628,
+      "learning_rate": 2.4301959031910784e-05,
+      "loss": 0.001,
+      "step": 279
+    },
+    {
+      "epoch": 5.6,
+      "grad_norm": 0.011561702005565166,
+      "learning_rate": 2.4127512582437485e-05,
+      "loss": 0.001,
+      "step": 280
+    },
+    {
+      "epoch": 5.62,
+      "grad_norm": 0.0134280975908041,
+      "learning_rate": 2.3953108656770016e-05,
+      "loss": 0.001,
+      "step": 281
+    },
+    {
+      "epoch": 5.64,
+      "grad_norm": 0.010531393811106682,
+      "learning_rate": 2.377875575510967e-05,
+      "loss": 0.001,
+      "step": 282
+    },
+    {
+      "epoch": 5.66,
+      "grad_norm": 0.0099195446819067,
+      "learning_rate": 2.3604462375170906e-05,
+      "loss": 0.0009,
+      "step": 283
+    },
+    {
+      "epoch": 5.68,
+      "grad_norm": 0.008749845437705517,
+      "learning_rate": 2.3430237011767167e-05,
+      "loss": 0.0009,
+      "step": 284
+    },
+    {
+      "epoch": 5.7,
+      "grad_norm": 0.00893120002001524,
+      "learning_rate": 2.3256088156396868e-05,
+      "loss": 0.0009,
+      "step": 285
+    },
+    {
+      "epoch": 5.72,
+      "grad_norm": 0.010007224045693874,
+      "learning_rate": 2.3082024296829536e-05,
+      "loss": 0.001,
+      "step": 286
+    },
+    {
+      "epoch": 5.74,
+      "grad_norm": 0.008720742538571358,
+      "learning_rate": 2.2908053916692117e-05,
+      "loss": 0.0009,
+      "step": 287
+    },
+    {
+      "epoch": 5.76,
+      "grad_norm": 0.006999673787504435,
+      "learning_rate": 2.2734185495055503e-05,
+      "loss": 0.0009,
+      "step": 288
+    },
+    {
+      "epoch": 5.78,
+      "grad_norm": 0.016226548701524734,
+      "learning_rate": 2.2560427506021266e-05,
+      "loss": 0.0009,
+      "step": 289
+    },
+    {
+      "epoch": 5.8,
+      "grad_norm": 0.012789979577064514,
+      "learning_rate": 2.238678841830867e-05,
+      "loss": 0.001,
+      "step": 290
+    },
+    {
+      "epoch": 5.82,
+      "grad_norm": 0.012711182236671448,
+      "learning_rate": 2.2213276694841866e-05,
+      "loss": 0.001,
+      "step": 291
+    },
+    {
+      "epoch": 5.84,
+      "grad_norm": 0.011736830696463585,
+      "learning_rate": 2.2039900792337474e-05,
+      "loss": 0.0009,
+      "step": 292
+    },
+    {
+      "epoch": 5.86,
+      "grad_norm": 0.0094444639980793,
+      "learning_rate": 2.186666916089239e-05,
+      "loss": 0.0009,
+      "step": 293
+    },
+    {
+      "epoch": 5.88,
+      "grad_norm": 0.014789476990699768,
+      "learning_rate": 2.1693590243571938e-05,
+      "loss": 0.0009,
+      "step": 294
+    },
+    {
+      "epoch": 5.9,
+      "grad_norm": 0.014187824912369251,
+      "learning_rate": 2.1520672475998373e-05,
+      "loss": 0.001,
+      "step": 295
+    },
+    {
+      "epoch": 5.92,
+      "grad_norm": 0.018324721604585648,
+      "learning_rate": 2.1347924285939714e-05,
+      "loss": 0.001,
+      "step": 296
+    },
+    {
+      "epoch": 5.9399999999999995,
+      "grad_norm": 0.012350255623459816,
+      "learning_rate": 2.117535409289905e-05,
+      "loss": 0.0009,
+      "step": 297
+    },
+    {
+      "epoch": 5.96,
+      "grad_norm": 0.011847893707454205,
+      "learning_rate": 2.1002970307704132e-05,
+      "loss": 0.001,
+      "step": 298
+    },
+    {
+      "epoch": 5.98,
+      "grad_norm": 0.011209225282073021,
+      "learning_rate": 2.0830781332097446e-05,
+      "loss": 0.0009,
+      "step": 299
+    },
+    {
+      "epoch": 6.0,
+      "grad_norm": 0.009176170453429222,
+      "learning_rate": 2.0658795558326743e-05,
+      "loss": 0.0009,
+      "step": 300
+    },
+    {
+      "epoch": 6.02,
+      "grad_norm": 0.013149021193385124,
+      "learning_rate": 2.0487021368736003e-05,
+      "loss": 0.0008,
+      "step": 301
+    },
+    {
+      "epoch": 6.04,
+      "grad_norm": 0.020946791395545006,
+      "learning_rate": 2.031546713535688e-05,
+      "loss": 0.0009,
+      "step": 302
+    },
+    {
+      "epoch": 6.06,
+      "grad_norm": 0.05294759199023247,
+      "learning_rate": 2.0144141219500705e-05,
+      "loss": 0.0009,
+      "step": 303
+    },
+    {
+      "epoch": 6.08,
+      "grad_norm": 0.014098895713686943,
+      "learning_rate": 1.9973051971350888e-05,
+      "loss": 0.0009,
+      "step": 304
+    },
+    {
+      "epoch": 6.1,
+      "grad_norm": 0.01258071605116129,
+      "learning_rate": 1.980220772955602e-05,
+      "loss": 0.0009,
+      "step": 305
+    },
+    {
+      "epoch": 6.12,
+      "grad_norm": 0.01646529696881771,
+      "learning_rate": 1.963161682082342e-05,
+      "loss": 0.0009,
+      "step": 306
+    },
+    {
+      "epoch": 6.14,
+      "grad_norm": 0.01562797650694847,
+      "learning_rate": 1.946128755951332e-05,
+      "loss": 0.001,
+      "step": 307
+    },
+    {
+      "epoch": 6.16,
+      "grad_norm": 0.02843649871647358,
+      "learning_rate": 1.9291228247233605e-05,
+      "loss": 0.0008,
+      "step": 308
+    },
+    {
+      "epoch": 6.18,
+      "grad_norm": 0.015122253447771072,
+      "learning_rate": 1.912144717243525e-05,
+      "loss": 0.0009,
+      "step": 309
+    },
+    {
+      "epoch": 6.2,
+      "grad_norm": 0.024031292647123337,
+      "learning_rate": 1.895195261000831e-05,
+      "loss": 0.0008,
+      "step": 310
+    },
+    {
+      "epoch": 6.22,
+      "grad_norm": 0.017758721485733986,
+      "learning_rate": 1.8782752820878634e-05,
+      "loss": 0.001,
+      "step": 311
+    },
+    {
+      "epoch": 6.24,
+      "grad_norm": 0.018115799874067307,
+      "learning_rate": 1.8613856051605243e-05,
+      "loss": 0.0009,
+      "step": 312
+    },
+    {
+      "epoch": 6.26,
+      "grad_norm": 0.013583043590188026,
+      "learning_rate": 1.8445270533978388e-05,
+      "loss": 0.001,
+      "step": 313
+    },
+    {
+      "epoch": 6.28,
+      "grad_norm": 0.01605222374200821,
+      "learning_rate": 1.827700448461836e-05,
+      "loss": 0.0009,
+      "step": 314
+    },
+    {
+      "epoch": 6.3,
+      "grad_norm": 0.012071061879396439,
+      "learning_rate": 1.8109066104575023e-05,
+      "loss": 0.0009,
+      "step": 315
+    },
+    {
+      "epoch": 6.32,
+      "grad_norm": 0.02086683176457882,
+      "learning_rate": 1.7941463578928086e-05,
+      "loss": 0.0008,
+      "step": 316
+    },
+    {
+      "epoch": 6.34,
+      "grad_norm": 0.015499988570809364,
+      "learning_rate": 1.7774205076388206e-05,
+      "loss": 0.001,
+      "step": 317
+    },
+    {
+      "epoch": 6.36,
+      "grad_norm": 0.033231064677238464,
+      "learning_rate": 1.7607298748898842e-05,
+      "loss": 0.001,
+      "step": 318
+    },
+    {
+      "epoch": 6.38,
+      "grad_norm": 0.04200902208685875,
+      "learning_rate": 1.744075273123889e-05,
+      "loss": 0.001,
+      "step": 319
+    },
+    {
+      "epoch": 6.4,
+      "grad_norm": 0.016907304525375366,
+      "learning_rate": 1.7274575140626318e-05,
+      "loss": 0.0009,
+      "step": 320
+    },
+    {
+      "epoch": 6.42,
+      "grad_norm": 0.012497249990701675,
+      "learning_rate": 1.7108774076322443e-05,
+      "loss": 0.0008,
+      "step": 321
+    },
+    {
+      "epoch": 6.44,
+      "grad_norm": 0.01482861116528511,
+      "learning_rate": 1.6943357619237226e-05,
+      "loss": 0.0009,
+      "step": 322
+    },
+    {
+      "epoch": 6.46,
+      "grad_norm": 0.010810253210365772,
+      "learning_rate": 1.677833383153542e-05,
+      "loss": 0.0009,
+      "step": 323
+    },
+    {
+      "epoch": 6.48,
+      "grad_norm": 0.008798140101134777,
+      "learning_rate": 1.6613710756243626e-05,
+      "loss": 0.0008,
+      "step": 324
+    },
+    {
+      "epoch": 6.5,
+      "grad_norm": 0.011670750565826893,
+      "learning_rate": 1.6449496416858284e-05,
+      "loss": 0.0009,
+      "step": 325
+    },
+    {
+      "epoch": 6.52,
+      "grad_norm": 0.013968761079013348,
+      "learning_rate": 1.6285698816954624e-05,
+      "loss": 0.0009,
+      "step": 326
+    },
+    {
+      "epoch": 6.54,
+      "grad_norm": 0.014013497158885002,
+      "learning_rate": 1.612232593979658e-05,
+      "loss": 0.0009,
+      "step": 327
+    },
+    {
+      "epoch": 6.5600000000000005,
+      "grad_norm": 0.01848709024488926,
+      "learning_rate": 1.5959385747947698e-05,
+      "loss": 0.0008,
+      "step": 328
+    },
+    {
+      "epoch": 6.58,
+      "grad_norm": 0.02498721145093441,
+      "learning_rate": 1.5796886182883053e-05,
+      "loss": 0.001,
+      "step": 329
+    },
+    {
+      "epoch": 6.6,
+      "grad_norm": 0.01326808612793684,
+      "learning_rate": 1.56348351646022e-05,
+      "loss": 0.001,
+      "step": 330
+    },
+    {
+      "epoch": 6.62,
+      "grad_norm": 0.014568154700100422,
+      "learning_rate": 1.547324059124315e-05,
+      "loss": 0.0009,
+      "step": 331
+    },
+    {
+      "epoch": 6.64,
+      "grad_norm": 0.012086485512554646,
+      "learning_rate": 1.5312110338697426e-05,
+      "loss": 0.001,
+      "step": 332
+    },
+    {
+      "epoch": 6.66,
+      "grad_norm": 0.010840130038559437,
+      "learning_rate": 1.5151452260226224e-05,
+      "loss": 0.0008,
+      "step": 333
+    },
+    {
+      "epoch": 6.68,
+      "grad_norm": 0.012208978645503521,
+      "learning_rate": 1.4991274186077632e-05,
+      "loss": 0.001,
+      "step": 334
+    },
+    {
+      "epoch": 6.7,
+      "grad_norm": 0.01624949462711811,
+      "learning_rate": 1.4831583923104999e-05,
+      "loss": 0.0009,
+      "step": 335
+    },
+    {
+      "epoch": 6.72,
+      "grad_norm": 0.01066320575773716,
+      "learning_rate": 1.467238925438646e-05,
+      "loss": 0.0008,
+      "step": 336
+    },
+    {
+      "epoch": 6.74,
+      "grad_norm": 0.015944579616189003,
+      "learning_rate": 1.4513697938845572e-05,
+      "loss": 0.001,
+      "step": 337
+    },
+    {
+      "epoch": 6.76,
+      "grad_norm": 0.011701629497110844,
+      "learning_rate": 1.4355517710873184e-05,
+      "loss": 0.0008,
+      "step": 338
+    },
+    {
+      "epoch": 6.78,
+      "grad_norm": 0.011669070459902287,
+      "learning_rate": 1.4197856279950438e-05,
+      "loss": 0.0009,
+      "step": 339
+    },
+    {
+      "epoch": 6.8,
+      "grad_norm": 0.08641933649778366,
+      "learning_rate": 1.4040721330273062e-05,
+      "loss": 0.0009,
+      "step": 340
+    },
+    {
+      "epoch": 6.82,
+      "grad_norm": 0.028719276189804077,
+      "learning_rate": 1.388412052037682e-05,
+      "loss": 0.0009,
+      "step": 341
+    },
+    {
+      "epoch": 6.84,
+      "grad_norm": 0.011268166825175285,
+      "learning_rate": 1.3728061482764238e-05,
+      "loss": 0.0009,
+      "step": 342
+    },
+    {
+      "epoch": 6.86,
+      "grad_norm": 0.013131314888596535,
+      "learning_rate": 1.3572551823532654e-05,
+      "loss": 0.0008,
+      "step": 343
+    },
+    {
+      "epoch": 6.88,
+      "grad_norm": 0.010851713828742504,
+      "learning_rate": 1.3417599122003464e-05,
+      "loss": 0.0009,
+      "step": 344
+    },
+    {
+      "epoch": 6.9,
+      "grad_norm": 0.013467922806739807,
+      "learning_rate": 1.3263210930352737e-05,
+      "loss": 0.0008,
+      "step": 345
+    },
+    {
+      "epoch": 6.92,
+      "grad_norm": 0.013433183543384075,
+      "learning_rate": 1.3109394773243117e-05,
+      "loss": 0.0009,
+      "step": 346
+    },
+    {
+      "epoch": 6.9399999999999995,
+      "grad_norm": 0.010768929496407509,
+      "learning_rate": 1.2956158147457115e-05,
+      "loss": 0.0008,
+      "step": 347
+    },
+    {
+      "epoch": 6.96,
+      "grad_norm": 0.011435325257480145,
+      "learning_rate": 1.280350852153168e-05,
+      "loss": 0.0008,
+      "step": 348
+    },
+    {
+      "epoch": 6.98,
+      "grad_norm": 0.012479741126298904,
+      "learning_rate": 1.2651453335394231e-05,
+      "loss": 0.0009,
+      "step": 349
+    },
+    {
+      "epoch": 7.0,
+      "grad_norm": 0.013233255594968796,
+      "learning_rate": 1.2500000000000006e-05,
+      "loss": 0.0009,
+      "step": 350
+    },
+    {
+      "epoch": 7.02,
+      "grad_norm": 0.012486303225159645,
+      "learning_rate": 1.234915589697091e-05,
+      "loss": 0.001,
+      "step": 351
+    },
+    {
+      "epoch": 7.04,
+      "grad_norm": 0.011426866985857487,
+      "learning_rate": 1.2198928378235716e-05,
+      "loss": 0.0008,
+      "step": 352
+    },
+    {
+      "epoch": 7.06,
+      "grad_norm": 0.009702113457024097,
+      "learning_rate": 1.2049324765671749e-05,
+      "loss": 0.0008,
+      "step": 353
+    },
+    {
+      "epoch": 7.08,
+      "grad_norm": 0.014162976294755936,
+      "learning_rate": 1.1900352350748026e-05,
+      "loss": 0.0009,
+      "step": 354
+    },
+    {
+      "epoch": 7.1,
+      "grad_norm": 0.028280340135097504,
+      "learning_rate": 1.175201839416988e-05,
+      "loss": 0.0008,
+      "step": 355
+    },
+    {
+      "epoch": 7.12,
+      "grad_norm": 0.012857620604336262,
+      "learning_rate": 1.1604330125525079e-05,
+      "loss": 0.0009,
+      "step": 356
+    },
+    {
+      "epoch": 7.14,
+      "grad_norm": 0.011418178677558899,
+      "learning_rate": 1.1457294742931507e-05,
+      "loss": 0.0007,
+      "step": 357
+    },
+    {
+      "epoch": 7.16,
+      "grad_norm": 0.02243344485759735,
+      "learning_rate": 1.1310919412686247e-05,
+      "loss": 0.0008,
+      "step": 358
+    },
+    {
+      "epoch": 7.18,
+      "grad_norm": 0.009756579995155334,
+      "learning_rate": 1.11652112689164e-05,
+      "loss": 0.0007,
+      "step": 359
+    },
+    {
+      "epoch": 7.2,
+      "grad_norm": 0.033040065318346024,
+      "learning_rate": 1.1020177413231334e-05,
+      "loss": 0.0009,
+      "step": 360
+    },
+    {
+      "epoch": 7.22,
+      "grad_norm": 0.011751036159694195,
+      "learning_rate": 1.0875824914376553e-05,
+      "loss": 0.0008,
+      "step": 361
+    },
+    {
+      "epoch": 7.24,
+      "grad_norm": 0.02029712125658989,
+      "learning_rate": 1.0732160807889211e-05,
+      "loss": 0.0009,
+      "step": 362
+    },
+    {
+      "epoch": 7.26,
+      "grad_norm": 0.012680457904934883,
+      "learning_rate": 1.058919209575517e-05,
+      "loss": 0.0007,
+      "step": 363
+    },
+    {
+      "epoch": 7.28,
+      "grad_norm": 0.01278097927570343,
+      "learning_rate": 1.0446925746067768e-05,
+      "loss": 0.0007,
+      "step": 364
+    },
+    {
+      "epoch": 7.3,
+      "grad_norm": 0.017604360356926918,
+      "learning_rate": 1.0305368692688174e-05,
+      "loss": 0.0008,
+      "step": 365
+    },
+    {
+      "epoch": 7.32,
+      "grad_norm": 0.012253907509148121,
+      "learning_rate": 1.0164527834907467e-05,
+      "loss": 0.0007,
+      "step": 366
+    },
+    {
+      "epoch": 7.34,
+      "grad_norm": 0.01415831595659256,
+      "learning_rate": 1.0024410037110357e-05,
+      "loss": 0.0007,
+      "step": 367
+    },
+    {
+      "epoch": 7.36,
+      "grad_norm": 0.01281712669879198,
+      "learning_rate": 9.88502212844063e-06,
+      "loss": 0.0008,
+      "step": 368
+    },
+    {
+      "epoch": 7.38,
+      "grad_norm": 0.01478048600256443,
+      "learning_rate": 9.746370902468311e-06,
+      "loss": 0.0008,
+      "step": 369
+    },
+    {
+      "epoch": 7.4,
+      "grad_norm": 0.05853793025016785,
+      "learning_rate": 9.608463116858542e-06,
+      "loss": 0.0009,
+      "step": 370
+    },
+    {
+      "epoch": 7.42,
+      "grad_norm": 0.011016006581485271,
+      "learning_rate": 9.471305493042243e-06,
+      "loss": 0.0008,
+      "step": 371
+    },
+    {
+      "epoch": 7.44,
+      "grad_norm": 0.01250209379941225,
+      "learning_rate": 9.334904715888495e-06,
+      "loss": 0.0007,
+      "step": 372
+    },
+    {
+      "epoch": 7.46,
+      "grad_norm": 0.01570058800280094,
+      "learning_rate": 9.199267433378727e-06,
+      "loss": 0.0008,
+      "step": 373
+    },
+    {
+      "epoch": 7.48,
+      "grad_norm": 0.021531647071242332,
+      "learning_rate": 9.064400256282757e-06,
+      "loss": 0.001,
+      "step": 374
+    },
+    {
+      "epoch": 7.5,
+      "grad_norm": 0.021758703514933586,
+      "learning_rate": 8.930309757836517e-06,
+      "loss": 0.0008,
+      "step": 375
+    },
+    {
+      "epoch": 7.52,
+      "grad_norm": 0.01226816326379776,
+      "learning_rate": 8.797002473421728e-06,
+      "loss": 0.0007,
+      "step": 376
+    },
+    {
+      "epoch": 7.54,
+      "grad_norm": 0.024715250357985497,
+      "learning_rate": 8.664484900247363e-06,
+      "loss": 0.0008,
+      "step": 377
+    },
+    {
+      "epoch": 7.5600000000000005,
+      "grad_norm": 0.01861533895134926,
+      "learning_rate": 8.532763497032987e-06,
+      "loss": 0.0007,
+      "step": 378
+    },
+    {
+      "epoch": 7.58,
+      "grad_norm": 0.019747568294405937,
+      "learning_rate": 8.40184468369396e-06,
+      "loss": 0.0008,
+      "step": 379
+    },
+    {
+      "epoch": 7.6,
+      "grad_norm": 0.01395918894559145,
+      "learning_rate": 8.271734841028553e-06,
+      "loss": 0.0007,
+      "step": 380
+    },
+    {
+      "epoch": 7.62,
+      "grad_norm": 0.011769567616283894,
+      "learning_rate": 8.142440310406924e-06,
+      "loss": 0.0008,
+      "step": 381
+    },
+    {
+      "epoch": 7.64,
+      "grad_norm": 0.017392277717590332,
+      "learning_rate": 8.013967393462094e-06,
+      "loss": 0.0008,
+      "step": 382
+    },
+    {
+      "epoch": 7.66,
+      "grad_norm": 0.016862329095602036,
+      "learning_rate": 7.886322351782783e-06,
+      "loss": 0.0008,
+      "step": 383
+    },
+    {
+      "epoch": 7.68,
+      "grad_norm": 0.011566593311727047,
+      "learning_rate": 7.759511406608255e-06,
+      "loss": 0.0008,
+      "step": 384
+    },
+    {
+      "epoch": 7.7,
+      "grad_norm": 0.014474247582256794,
+      "learning_rate": 7.633540738525066e-06,
+      "loss": 0.0009,
+      "step": 385
+    },
+    {
+      "epoch": 7.72,
+      "grad_norm": 0.031276676803827286,
+      "learning_rate": 7.508416487165862e-06,
+      "loss": 0.0008,
+      "step": 386
+    },
+    {
+      "epoch": 7.74,
+      "grad_norm": 0.011668872088193893,
+      "learning_rate": 7.384144750910133e-06,
+      "loss": 0.0008,
+      "step": 387
+    },
+    {
+      "epoch": 7.76,
+      "grad_norm": 0.014757595956325531,
+      "learning_rate": 7.260731586586983e-06,
+      "loss": 0.0008,
+      "step": 388
+    },
+    {
+      "epoch": 7.78,
+      "grad_norm": 0.020262010395526886,
+      "learning_rate": 7.138183009179922e-06,
+      "loss": 0.0008,
+      "step": 389
+    },
+    {
+      "epoch": 7.8,
+      "grad_norm": 0.012966041453182697,
+      "learning_rate": 7.016504991533726e-06,
+      "loss": 0.0009,
+      "step": 390
+    },
+    {
+      "epoch": 7.82,
+      "grad_norm": 0.02517668902873993,
+      "learning_rate": 6.895703464063319e-06,
+      "loss": 0.0009,
+      "step": 391
+    },
+    {
+      "epoch": 7.84,
+      "grad_norm": 0.008584852330386639,
+      "learning_rate": 6.775784314464717e-06,
+      "loss": 0.0007,
+      "step": 392
+    },
+    {
+      "epoch": 7.86,
+      "grad_norm": 0.01465829461812973,
+      "learning_rate": 6.656753387428089e-06,
+      "loss": 0.0007,
+      "step": 393
+    },
+    {
+      "epoch": 7.88,
+      "grad_norm": 0.011138123460114002,
+      "learning_rate": 6.538616484352902e-06,
+      "loss": 0.0008,
+      "step": 394
+    },
+    {
+      "epoch": 7.9,
+      "grad_norm": 0.022989438846707344,
+      "learning_rate": 6.421379363065142e-06,
+      "loss": 0.0009,
+      "step": 395
+    },
+    {
+      "epoch": 7.92,
+      "grad_norm": 0.013544759713113308,
+      "learning_rate": 6.305047737536707e-06,
+      "loss": 0.0009,
+      "step": 396
+    },
+    {
+      "epoch": 7.9399999999999995,
+      "grad_norm": 0.016775785014033318,
+      "learning_rate": 6.189627277606894e-06,
+      "loss": 0.0008,
+      "step": 397
+    },
+    {
+      "epoch": 7.96,
+      "grad_norm": 0.012762874364852905,
+      "learning_rate": 6.075123608706093e-06,
+      "loss": 0.0008,
+      "step": 398
+    },
+    {
+      "epoch": 7.98,
+      "grad_norm": 0.01835818588733673,
+      "learning_rate": 5.961542311581586e-06,
+      "loss": 0.0008,
+      "step": 399
+    },
+    {
+      "epoch": 8.0,
+      "grad_norm": 0.024637369439005852,
+      "learning_rate": 5.848888922025553e-06,
+      "loss": 0.0007,
+      "step": 400
+    },
+    {
+      "epoch": 8.02,
+      "grad_norm": 0.016956869512796402,
+      "learning_rate": 5.737168930605272e-06,
+      "loss": 0.0007,
+      "step": 401
+    },
+    {
+      "epoch": 8.04,
+      "grad_norm": 0.012730157002806664,
+      "learning_rate": 5.626387782395512e-06,
+      "loss": 0.0006,
+      "step": 402
+    },
+    {
+      "epoch": 8.06,
+      "grad_norm": 0.01208969485014677,
+      "learning_rate": 5.5165508767131415e-06,
+      "loss": 0.0007,
+      "step": 403
+    },
+    {
+      "epoch": 8.08,
+      "grad_norm": 0.01699792966246605,
+      "learning_rate": 5.4076635668540075e-06,
+      "loss": 0.0006,
+      "step": 404
+    },
+    {
+      "epoch": 8.1,
+      "grad_norm": 0.014854060485959053,
+      "learning_rate": 5.299731159831953e-06,
+      "loss": 0.0007,
+      "step": 405
+    },
+    {
+      "epoch": 8.12,
+      "grad_norm": 0.012892371043562889,
+      "learning_rate": 5.192758916120236e-06,
+      "loss": 0.0006,
+      "step": 406
+    },
+    {
+      "epoch": 8.14,
+      "grad_norm": 0.015144885517656803,
+      "learning_rate": 5.086752049395094e-06,
+      "loss": 0.0007,
+      "step": 407
+    },
+    {
+      "epoch": 8.16,
+      "grad_norm": 0.014186014421284199,
+      "learning_rate": 4.981715726281666e-06,
+      "loss": 0.0008,
+      "step": 408
+    },
+    {
+      "epoch": 8.18,
+      "grad_norm": 0.015514220111072063,
+      "learning_rate": 4.877655066102149e-06,
+      "loss": 0.0007,
+      "step": 409
+    },
+    {
+      "epoch": 8.2,
+      "grad_norm": 0.015736915171146393,
+      "learning_rate": 4.7745751406263165e-06,
+      "loss": 0.0006,
+      "step": 410
+    },
+    {
+      "epoch": 8.22,
+      "grad_norm": 0.012078430503606796,
+      "learning_rate": 4.672480973824311e-06,
+      "loss": 0.0008,
+      "step": 411
+    },
+    {
+      "epoch": 8.24,
+      "grad_norm": 0.02000226080417633,
+      "learning_rate": 4.571377541621788e-06,
+      "loss": 0.0007,
+      "step": 412
+    },
+    {
+      "epoch": 8.26,
+      "grad_norm": 0.011416291818022728,
+      "learning_rate": 4.4712697716574e-06,
+      "loss": 0.0006,
+      "step": 413
+    },
+    {
+      "epoch": 8.28,
+      "grad_norm": 0.022556310519576073,
+      "learning_rate": 4.372162543042624e-06,
+      "loss": 0.0008,
+      "step": 414
+    },
+    {
+      "epoch": 8.3,
+      "grad_norm": 0.02372623048722744,
+      "learning_rate": 4.274060686123959e-06,
+      "loss": 0.0007,
+      "step": 415
+    },
+    {
+      "epoch": 8.32,
+      "grad_norm": 0.02283402346074581,
+      "learning_rate": 4.176968982247514e-06,
+      "loss": 0.0007,
+      "step": 416
+    },
+    {
+      "epoch": 8.34,
+      "grad_norm": 0.020453330129384995,
+      "learning_rate": 4.08089216352596e-06,
+      "loss": 0.0006,
+      "step": 417
+    },
+    {
+      "epoch": 8.36,
+      "grad_norm": 0.014580060727894306,
+      "learning_rate": 3.985834912607894e-06,
+      "loss": 0.0007,
+      "step": 418
+    },
+    {
+      "epoch": 8.38,
+      "grad_norm": 0.011051076464354992,
+      "learning_rate": 3.891801862449629e-06,
+      "loss": 0.0007,
+      "step": 419
+    },
+    {
+      "epoch": 8.4,
+      "grad_norm": 0.03123440593481064,
+      "learning_rate": 3.798797596089351e-06,
+      "loss": 0.0007,
+      "step": 420
+    },
+    {
+      "epoch": 8.42,
+      "grad_norm": 0.008824478834867477,
+      "learning_rate": 3.7068266464238084e-06,
+      "loss": 0.0005,
+      "step": 421
+    },
+    {
+      "epoch": 8.44,
+      "grad_norm": 0.01253305934369564,
+      "learning_rate": 3.6158934959873353e-06,
+      "loss": 0.0006,
+      "step": 422
+    },
+    {
+      "epoch": 8.46,
+      "grad_norm": 0.011405512690544128,
+      "learning_rate": 3.5260025767333893e-06,
+      "loss": 0.0007,
+      "step": 423
+    },
+    {
+      "epoch": 8.48,
+      "grad_norm": 0.01283338014036417,
+      "learning_rate": 3.4371582698185633e-06,
+      "loss": 0.0007,
+      "step": 424
+    },
+    {
+      "epoch": 8.5,
+      "grad_norm": 0.012804213911294937,
+      "learning_rate": 3.3493649053890326e-06,
+      "loss": 0.0007,
+      "step": 425
+    },
+    {
+      "epoch": 8.52,
+      "grad_norm": 0.012545102275907993,
+      "learning_rate": 3.262626762369525e-06,
+      "loss": 0.0007,
+      "step": 426
+    },
+    {
+      "epoch": 8.54,
+      "grad_norm": 0.012686868198215961,
+      "learning_rate": 3.176948068254762e-06,
+      "loss": 0.0007,
+      "step": 427
+    },
+    {
+      "epoch": 8.56,
+      "grad_norm": 0.018044594675302505,
+      "learning_rate": 3.092332998903416e-06,
+      "loss": 0.0007,
+      "step": 428
+    },
+    {
+      "epoch": 8.58,
+      "grad_norm": 0.017359774559736252,
+      "learning_rate": 3.0087856783345914e-06,
+      "loss": 0.0007,
+      "step": 429
+    },
+    {
+      "epoch": 8.6,
+      "grad_norm": 0.019643310457468033,
+      "learning_rate": 2.9263101785268254e-06,
+      "loss": 0.0007,
+      "step": 430
+    },
+    {
+      "epoch": 8.62,
+      "grad_norm": 0.015051987022161484,
+      "learning_rate": 2.8449105192196316e-06,
+      "loss": 0.0007,
+      "step": 431
+    },
+    {
+      "epoch": 8.64,
+      "grad_norm": 0.014268382452428341,
+      "learning_rate": 2.764590667717562e-06,
+      "loss": 0.0007,
+      "step": 432
+    },
+    {
+      "epoch": 8.66,
+      "grad_norm": 0.013278437778353691,
+      "learning_rate": 2.6853545386968606e-06,
+      "loss": 0.0008,
+      "step": 433
+    },
+    {
+      "epoch": 8.68,
+      "grad_norm": 0.03189530596137047,
+      "learning_rate": 2.6072059940146775e-06,
+      "loss": 0.0007,
+      "step": 434
+    },
+    {
+      "epoch": 8.7,
+      "grad_norm": 0.08232607692480087,
+      "learning_rate": 2.5301488425208296e-06,
+      "loss": 0.0007,
+      "step": 435
+    },
+    {
+      "epoch": 8.72,
+      "grad_norm": 0.01643546111881733,
+      "learning_rate": 2.454186839872158e-06,
+      "loss": 0.0008,
+      "step": 436
+    },
+    {
+      "epoch": 8.74,
+      "grad_norm": 0.016362924128770828,
+      "learning_rate": 2.379323688349516e-06,
+      "loss": 0.0007,
+      "step": 437
+    },
+    {
+      "epoch": 8.76,
+      "grad_norm": 0.01405677292495966,
+      "learning_rate": 2.3055630366772856e-06,
+      "loss": 0.0007,
+      "step": 438
+    },
+    {
+      "epoch": 8.78,
+      "grad_norm": 0.017908280715346336,
+      "learning_rate": 2.2329084798455746e-06,
+      "loss": 0.0007,
+      "step": 439
+    },
+    {
+      "epoch": 8.8,
+      "grad_norm": 0.029627658426761627,
+      "learning_rate": 2.1613635589349756e-06,
+      "loss": 0.0006,
+      "step": 440
+    },
+    {
+      "epoch": 8.82,
+      "grad_norm": 0.009773575700819492,
+      "learning_rate": 2.0909317609440095e-06,
+      "loss": 0.0006,
+      "step": 441
+    },
+    {
+      "epoch": 8.84,
+      "grad_norm": 0.014441922307014465,
+      "learning_rate": 2.0216165186191407e-06,
+      "loss": 0.0007,
+      "step": 442
+    },
+    {
+      "epoch": 8.86,
+      "grad_norm": 0.0174330435693264,
+      "learning_rate": 1.95342121028749e-06,
+      "loss": 0.0008,
+      "step": 443
+    },
+    {
+      "epoch": 8.88,
+      "grad_norm": 0.018265046179294586,
+      "learning_rate": 1.8863491596921745e-06,
+      "loss": 0.0007,
+      "step": 444
+    },
+    {
+      "epoch": 8.9,
+      "grad_norm": 0.025862442329525948,
+      "learning_rate": 1.8204036358303173e-06,
+      "loss": 0.0006,
+      "step": 445
+    },
+    {
+      "epoch": 8.92,
+      "grad_norm": 0.02057448960840702,
+      "learning_rate": 1.7555878527937164e-06,
+      "loss": 0.0007,
+      "step": 446
+    },
+    {
+      "epoch": 8.94,
+      "grad_norm": 0.02054482139647007,
+      "learning_rate": 1.6919049696121958e-06,
+      "loss": 0.0008,
+      "step": 447
+    },
+    {
+      "epoch": 8.96,
+      "grad_norm": 0.031380027532577515,
+      "learning_rate": 1.629358090099639e-06,
+      "loss": 0.0008,
+      "step": 448
+    },
+    {
+      "epoch": 8.98,
+      "grad_norm": 0.012362498790025711,
+      "learning_rate": 1.5679502627027136e-06,
+      "loss": 0.0007,
+      "step": 449
+    },
+    {
+      "epoch": 9.0,
+      "grad_norm": 0.017466159537434578,
+      "learning_rate": 1.5076844803522922e-06,
+      "loss": 0.0008,
+      "step": 450
+    },
+    {
+      "epoch": 9.02,
+      "grad_norm": 0.0148147689178586,
+      "learning_rate": 1.4485636803175829e-06,
+      "loss": 0.0007,
+      "step": 451
+    },
+    {
+      "epoch": 9.04,
+      "grad_norm": 1.0861493349075317,
+      "learning_rate": 1.3905907440629752e-06,
+      "loss": 0.0006,
+      "step": 452
+    },
+    {
+      "epoch": 9.06,
+      "grad_norm": 0.024347692728042603,
+      "learning_rate": 1.333768497107593e-06,
+      "loss": 0.0006,
+      "step": 453
+    },
+    {
+      "epoch": 9.08,
+      "grad_norm": 0.02870842069387436,
+      "learning_rate": 1.2780997088875869e-06,
+      "loss": 0.0006,
+      "step": 454
+    },
+    {
+      "epoch": 9.1,
+      "grad_norm": 0.021804317831993103,
+      "learning_rate": 1.2235870926211619e-06,
+      "loss": 0.0006,
+      "step": 455
+    },
+    {
+      "epoch": 9.12,
+      "grad_norm": 0.03335144743323326,
+      "learning_rate": 1.170233305176327e-06,
+      "loss": 0.0006,
+      "step": 456
+    },
+    {
+      "epoch": 9.14,
+      "grad_norm": 0.01712186448276043,
+      "learning_rate": 1.1180409469414094e-06,
+      "loss": 0.0007,
+      "step": 457
+    },
+    {
+      "epoch": 9.16,
+      "grad_norm": 0.011126603931188583,
+      "learning_rate": 1.067012561698319e-06,
+      "loss": 0.0006,
+      "step": 458
+    },
+    {
+      "epoch": 9.18,
+      "grad_norm": 0.17538900673389435,
+      "learning_rate": 1.0171506364985622e-06,
+      "loss": 0.0007,
+      "step": 459
+    },
+    {
+      "epoch": 9.2,
+      "grad_norm": 0.012856665067374706,
+      "learning_rate": 9.684576015420278e-07,
+      "loss": 0.0006,
+      "step": 460
+    },
+    {
+      "epoch": 9.22,
+      "grad_norm": 0.08660769462585449,
+      "learning_rate": 9.209358300585474e-07,
+      "loss": 0.0006,
+      "step": 461
+    },
+    {
+      "epoch": 9.24,
+      "grad_norm": 0.013641130179166794,
+      "learning_rate": 8.745876381922147e-07,
+      "loss": 0.0006,
+      "step": 462
+    },
+    {
+      "epoch": 9.26,
+      "grad_norm": 0.027097051963210106,
+      "learning_rate": 8.294152848885157e-07,
+      "loss": 0.0007,
+      "step": 463
+    },
+    {
+      "epoch": 9.28,
+      "grad_norm": 0.016025835648179054,
+      "learning_rate": 7.854209717842231e-07,
+      "loss": 0.0005,
+      "step": 464
+    },
+    {
+      "epoch": 9.3,
+      "grad_norm": 0.020238352939486504,
+      "learning_rate": 7.426068431000882e-07,
+      "loss": 0.0007,
+      "step": 465
+    },
+    {
+      "epoch": 9.32,
+      "grad_norm": 0.013973543420433998,
+      "learning_rate": 7.009749855363456e-07,
+      "loss": 0.0006,
+      "step": 466
+    },
+    {
+      "epoch": 9.34,
+      "grad_norm": 0.022512558847665787,
+      "learning_rate": 6.605274281709928e-07,
+      "loss": 0.0006,
+      "step": 467
+    },
+    {
+      "epoch": 9.36,
+      "grad_norm": 0.015979360789060593,
+      "learning_rate": 6.212661423609184e-07,
+      "loss": 0.0007,
+      "step": 468
+    },
+    {
+      "epoch": 9.38,
+      "grad_norm": 0.01091644074767828,
+      "learning_rate": 5.83193041645802e-07,
+      "loss": 0.0006,
+      "step": 469
+    },
+    {
+      "epoch": 9.4,
+      "grad_norm": 0.011535176075994968,
+      "learning_rate": 5.463099816548579e-07,
+      "loss": 0.0006,
+      "step": 470
+    },
+    {
+      "epoch": 9.42,
+      "grad_norm": 0.017591234296560287,
+      "learning_rate": 5.106187600163987e-07,
+      "loss": 0.0006,
+      "step": 471
+    },
+    {
+      "epoch": 9.44,
+      "grad_norm": 0.020435495302081108,
+      "learning_rate": 4.7612111627021175e-07,
+      "loss": 0.0006,
+      "step": 472
+    },
+    {
+      "epoch": 9.46,
+      "grad_norm": 0.02458248846232891,
+      "learning_rate": 4.4281873178278475e-07,
+      "loss": 0.0006,
+      "step": 473
+    },
+    {
+      "epoch": 9.48,
+      "grad_norm": 0.01399570144712925,
+      "learning_rate": 4.107132296653549e-07,
+      "loss": 0.0005,
+      "step": 474
+    },
+    {
+      "epoch": 9.5,
+      "grad_norm": 0.011794820427894592,
+      "learning_rate": 3.7980617469479953e-07,
+      "loss": 0.0006,
+      "step": 475
+    },
+    {
+      "epoch": 9.52,
+      "grad_norm": 0.034110236912965775,
+      "learning_rate": 3.5009907323737825e-07,
+      "loss": 0.0007,
+      "step": 476
+    },
+    {
+      "epoch": 9.54,
+      "grad_norm": 0.02996162697672844,
+      "learning_rate": 3.215933731753024e-07,
+      "loss": 0.0007,
+      "step": 477
+    },
+    {
+      "epoch": 9.56,
+      "grad_norm": 0.01556292176246643,
+      "learning_rate": 2.942904638361804e-07,
+      "loss": 0.0006,
+      "step": 478
+    },
+    {
+      "epoch": 9.58,
+      "grad_norm": 0.01747916080057621,
+      "learning_rate": 2.681916759252917e-07,
+      "loss": 0.0008,
+      "step": 479
+    },
+    {
+      "epoch": 9.6,
+      "grad_norm": 0.016395531594753265,
+      "learning_rate": 2.4329828146074095e-07,
+      "loss": 0.0006,
+      "step": 480
+    },
+    {
+      "epoch": 9.62,
+      "grad_norm": 0.028636911883950233,
+      "learning_rate": 2.1961149371145795e-07,
+      "loss": 0.0007,
+      "step": 481
+    },
+    {
+      "epoch": 9.64,
+      "grad_norm": 0.013615133240818977,
+      "learning_rate": 1.9713246713805588e-07,
+      "loss": 0.0006,
+      "step": 482
+    },
+    {
+      "epoch": 9.66,
+      "grad_norm": 0.014773129485547543,
+      "learning_rate": 1.7586229733657644e-07,
+      "loss": 0.0006,
+      "step": 483
+    },
+    {
+      "epoch": 9.68,
+      "grad_norm": 0.013628330081701279,
+      "learning_rate": 1.5580202098509077e-07,
+      "loss": 0.0007,
+      "step": 484
+    },
+    {
+      "epoch": 9.7,
+      "grad_norm": 0.013818751089274883,
+      "learning_rate": 1.3695261579316777e-07,
+      "loss": 0.0006,
+      "step": 485
+    },
+    {
+      "epoch": 9.72,
+      "grad_norm": 0.01693764328956604,
+      "learning_rate": 1.193150004542204e-07,
+      "loss": 0.0006,
+      "step": 486
+    },
+    {
+      "epoch": 9.74,
+      "grad_norm": 0.018511202186346054,
+      "learning_rate": 1.0289003460074165e-07,
+      "loss": 0.0007,
+      "step": 487
+    },
+    {
+      "epoch": 9.76,
+      "grad_norm": 0.057176489382982254,
+      "learning_rate": 8.767851876239074e-08,
+      "loss": 0.0007,
+      "step": 488
+    },
+    {
+      "epoch": 9.78,
+      "grad_norm": 0.01059312280267477,
+      "learning_rate": 7.368119432699383e-08,
+      "loss": 0.0006,
+      "step": 489
+    },
+    {
+      "epoch": 9.8,
+      "grad_norm": 0.014415513724088669,
+      "learning_rate": 6.089874350439506e-08,
+      "loss": 0.0006,
+      "step": 490
+    },
+    {
+      "epoch": 9.82,
+      "grad_norm": 0.014411581680178642,
+      "learning_rate": 4.9331789293211026e-08,
+      "loss": 0.0006,
+      "step": 491
+    },
+    {
+      "epoch": 9.84,
+      "grad_norm": 0.020400647073984146,
+      "learning_rate": 3.8980895450474455e-08,
+      "loss": 0.0007,
+      "step": 492
+    },
+    {
+      "epoch": 9.86,
+      "grad_norm": 0.01764337532222271,
+      "learning_rate": 2.9846566464150626e-08,
+      "loss": 0.0006,
+      "step": 493
+    },
+    {
+      "epoch": 9.88,
+      "grad_norm": 0.013475135900080204,
+      "learning_rate": 2.192924752854042e-08,
+      "loss": 0.0006,
+      "step": 494
+    },
+    {
+      "epoch": 9.9,
+      "grad_norm": 0.020350459963083267,
+      "learning_rate": 1.522932452260595e-08,
+      "loss": 0.0006,
+      "step": 495
+    },
+    {
+      "epoch": 9.92,
+      "grad_norm": 0.014247527346014977,
+      "learning_rate": 9.747123991141194e-09,
+      "loss": 0.0006,
+      "step": 496
+    },
+    {
+      "epoch": 9.94,
+      "grad_norm": 0.024135878309607506,
+      "learning_rate": 5.48291312886251e-09,
+      "loss": 0.0007,
+      "step": 497
+    },
+    {
+      "epoch": 9.96,
+      "grad_norm": 0.01212978269904852,
+      "learning_rate": 2.4368997673940297e-09,
+      "loss": 0.0006,
+      "step": 498
+    },
+    {
+      "epoch": 9.98,
+      "grad_norm": 0.01443579513579607,
+      "learning_rate": 6.092323651313292e-10,
+      "loss": 0.0007,
+      "step": 499
+    },
+    {
+      "epoch": 10.0,
+      "grad_norm": 0.01410845760256052,
+      "learning_rate": 0.0,
+      "loss": 0.0007,
+      "step": 500
+    },
+    {
+      "epoch": 10.0,
+      "step": 500,
+      "total_flos": 1.13782877585408e+18,
+      "train_loss": 0.07423569742811378,
+      "train_runtime": 7287.0887,
+      "train_samples_per_second": 4.384,
+      "train_steps_per_second": 0.069
     }
   ],
   "logging_steps": 1,
+  "max_steps": 500,
   "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 500,
   "stateful_callbacks": {
     "TrainerControl": {
       "args": {
       "attributes": {}
     }
   },
+  "total_flos": 1.13782877585408e+18,
+  "train_batch_size": 2,
   "trial_name": null,
   "trial_params": null
 }