Hugging Face's logo

richardadam
/
opus-mt-en-vi-novel-finetuned

Translation
Safetensors
English
Vietnamese
marian
opus-mt
novel
literature
Eval Results (legacy)
Model card Files
xet
 Community
richardadam commited on
Commit
0a46c43
·
verified ·
1 Parent(s): bd38ee3

Upload fine-tuned opus-mt-en-vi model (1 epoch, BLEU 29.29, ChrF++ 49.13)

Browse files
Files changed (23) hide show
  1. .gitattributes +4 -0
  2. README.md +94 -0
  3. checkpoint-20511/config.json +54 -0
  4. checkpoint-20511/generation_config.json +43 -0
  5. checkpoint-20511/model.safetensors +3 -0
  6. checkpoint-20511/optimizer.pt +3 -0
  7. checkpoint-20511/rng_state.pth +3 -0
  8. checkpoint-20511/scaler.pt +3 -0
  9. checkpoint-20511/scheduler.pt +3 -0
  10. checkpoint-20511/source.spm +3 -0
  11. checkpoint-20511/target.spm +3 -0
  12. checkpoint-20511/tokenizer_config.json +40 -0
  13. checkpoint-20511/trainer_state.json +1489 -0
  14. checkpoint-20511/training_args.bin +3 -0
  15. checkpoint-20511/vocab.json +0 -0
  16. config.json +54 -0
  17. generation_config.json +43 -0
  18. model.safetensors +3 -0
  19. source.spm +3 -0
  20. target.spm +3 -0
  21. tokenizer_config.json +40 -0
  22. training_args.bin +3 -0
  23. vocab.json +0 -0
.gitattributes CHANGED
@@ -33,3 +33,7 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ checkpoint-20511/source.spm filter=lfs diff=lfs merge=lfs -text
37
+ checkpoint-20511/target.spm filter=lfs diff=lfs merge=lfs -text
38
+ source.spm filter=lfs diff=lfs merge=lfs -text
39
+ target.spm filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,94 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ - vi
5
+ tags:
6
+ - translation
7
+ - opus-mt
8
+ - marian
9
+ - novel
10
+ - literature
11
+ license: apache-2.0
12
+ datasets:
13
+ - custom-novel-parallel-corpus
14
+ metrics:
15
+ - bleu
16
+ - chrf
17
+ model-index:
18
+ - name: opus-mt-en-vi-novel-finetuned
19
+ results:
20
+ - task:
21
+ type: translation
22
+ name: EN to VI Novel Translation
23
+ dataset:
24
+ name: Custom novel EN-VI parallel validation set (17,272 samples)
25
+ type: novel-literature
26
+ metrics:
27
+ - type: bleu
28
+ value: 29.29
29
+ name: BLEU
30
+ - type: chrf
31
+ value: 49.13
32
+ name: ChrF++
33
+ ---
34
+
35
+ # opus-mt-en-vi-novel-finetuned
36
+
37
+ Fine-tuned English → Vietnamese translation model optimized for **novels, literature, and narrative text**.
38
+
39
+ Fine-tuned from `Helsinki-NLP/opus-mt-en-vi` (MarianMT) for 1 epoch on ~328K parallel EN-VI novel sentence pairs.
40
+
41
+ ## Results Summary
42
+
43
+ ### Full Validation Set (17,272 samples)
44
+
45
+ | Model | BLEU | ChrF++ |
46
+ |-------|------|--------|
47
+ | **Fine-tuned** | **29.29** | **49.13** |
48
+ | Baseline (Helsinki-NLP/opus-mt-en-vi) | 3.73 | 20.87 |
49
+
50
+ ### Improvement over baseline
51
+
52
+ - **BLEU:** 8.2x (3.53 → 29.01 on sanity set)
53
+ - **ChrF++:** 2.4x (20.50 → 49.07 on sanity set)
54
+
55
+ ## Usage
56
+
57
+ ```python
58
+ from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
59
+
60
+ model = AutoModelForSeq2SeqLM.from_pretrained("richardadam/opus-mt-en-vi-novel-finetuned")
61
+ tokenizer = AutoTokenizer.from_pretrained("richardadam/opus-mt-en-vi-novel-finetuned")
62
+
63
+ text = "He looked at the door but did not dare to open it."
64
+ inputs = tokenizer(text, return_tensors="pt")
65
+ outputs = model.generate(**inputs, max_new_tokens=256, num_beams=4)
66
+ print(tokenizer.decode(outputs[0], skip_special_tokens=True))
67
+ # Hắn nhìn cánh cửa, nhưng không dám mở ra.
68
+ ```
69
+
70
+ ## Training
71
+
72
+ - **Base:** Helsinki-NLP/opus-mt-en-vi
73
+ - **Epochs:** 1
74
+ - **Batch size:** 16
75
+ - **GPU:** NVIDIA L4 (22.5 GB)
76
+ - **Runtime:** ~3h15m
77
+
78
+ ## Limitations
79
+
80
+ - Single-epoch fine-tune — may benefit from more training
81
+ - Novel domain only — may underperform on technical/formal text
82
+ - Baseline model performs very poorly on novels (BLEU 3.53), confirming domain-specific fine-tuning is essential
83
+
84
+ ## Citation
85
+
86
+ ```bibtex
87
+ @misc{opus-mt-en-vi-novel-finetuned,
88
+ author = {richardadam},
89
+ title = {opus-mt-en-vi-novel-finetuned},
90
+ year = 2026,
91
+ publisher = {Hugging Face},
92
+ howpublished = {huggingface.co/richardadam/opus-mt-en-vi-novel-finetuned},
93
+ }
94
+ ```
checkpoint-20511/config.json ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "activation_dropout": 0.0,
3
+ "activation_function": "swish",
4
+ "add_bias_logits": false,
5
+ "add_final_layer_norm": false,
6
+ "architectures": [
7
+ "MarianMTModel"
8
+ ],
9
+ "attention_dropout": 0.0,
10
+ "bos_token_id": null,
11
+ "classif_dropout": 0.0,
12
+ "classifier_dropout": 0.0,
13
+ "d_model": 512,
14
+ "decoder_attention_heads": 8,
15
+ "decoder_ffn_dim": 2048,
16
+ "decoder_layerdrop": 0.0,
17
+ "decoder_layers": 6,
18
+ "decoder_start_token_id": 53684,
19
+ "decoder_vocab_size": 53685,
20
+ "dropout": 0.1,
21
+ "dtype": "float32",
22
+ "encoder_attention_heads": 8,
23
+ "encoder_ffn_dim": 2048,
24
+ "encoder_layerdrop": 0.0,
25
+ "encoder_layers": 6,
26
+ "eos_token_id": 0,
27
+ "extra_pos_embeddings": 53685,
28
+ "id2label": {
29
+ "0": "LABEL_0",
30
+ "1": "LABEL_1",
31
+ "2": "LABEL_2"
32
+ },
33
+ "init_std": 0.02,
34
+ "is_decoder": false,
35
+ "is_encoder_decoder": true,
36
+ "label2id": {
37
+ "LABEL_0": 0,
38
+ "LABEL_1": 1,
39
+ "LABEL_2": 2
40
+ },
41
+ "max_position_embeddings": 512,
42
+ "model_type": "marian",
43
+ "normalize_before": false,
44
+ "normalize_embedding": false,
45
+ "num_hidden_layers": 6,
46
+ "pad_token_id": 53684,
47
+ "scale_embedding": true,
48
+ "share_encoder_decoder_embeddings": true,
49
+ "static_position_embeddings": true,
50
+ "tie_word_embeddings": true,
51
+ "transformers_version": "5.0.0",
52
+ "use_cache": false,
53
+ "vocab_size": 53685
54
+ }
checkpoint-20511/generation_config.json ADDED
@@ -0,0 +1,43 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "assistant_confidence_threshold": 0.4,
3
+ "assistant_lookbehind": 10,
4
+ "bad_words_ids": [
5
+ [
6
+ 53684
7
+ ]
8
+ ],
9
+ "decoder_start_token_id": 53684,
10
+ "diversity_penalty": 0.0,
11
+ "do_sample": false,
12
+ "early_stopping": false,
13
+ "encoder_no_repeat_ngram_size": 0,
14
+ "encoder_repetition_penalty": 1.0,
15
+ "eos_token_id": [
16
+ 0
17
+ ],
18
+ "epsilon_cutoff": 0.0,
19
+ "eta_cutoff": 0.0,
20
+ "forced_eos_token_id": 0,
21
+ "length_penalty": 1.0,
22
+ "max_length": 512,
23
+ "min_length": 0,
24
+ "no_repeat_ngram_size": 0,
25
+ "num_assistant_tokens": 20,
26
+ "num_assistant_tokens_schedule": "constant",
27
+ "num_beam_groups": 1,
28
+ "num_beams": 4,
29
+ "num_return_sequences": 1,
30
+ "output_scores": false,
31
+ "pad_token_id": 53684,
32
+ "remove_invalid_values": false,
33
+ "renormalize_logits": true,
34
+ "repetition_penalty": 1.0,
35
+ "return_dict_in_generate": false,
36
+ "target_lookbehind": 10,
37
+ "temperature": 1.0,
38
+ "top_k": 50,
39
+ "top_p": 1.0,
40
+ "transformers_version": "5.0.0",
41
+ "typical_p": 1.0,
42
+ "use_cache": true
43
+ }
checkpoint-20511/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4e207ce875bb5f6b0250350ae0fed1861b7ad68f0be5c98c82ee2f323be3838c
3
+ size 506638972
checkpoint-20511/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:46c232c657c08955fa5bd397dd9f469d214bb796b8872e0291ec079f0b0abe73
3
+ size 1013006475
checkpoint-20511/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8e47a1472438553898bde49881272515d12ddf633230ba073b9dc32dea2deb6b
3
+ size 14645
checkpoint-20511/scaler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:20ef11284b19bd2cbb6aa39372ac0d1175bb7809e3991531c3f82f8603686d13
3
+ size 1383
checkpoint-20511/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:90bb15622a337a948aa0acddcc167c12e80fb875203b12db98acd7da74190f4c
3
+ size 1465
checkpoint-20511/source.spm ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fdd9ab486ca8ea060967105e30db7d8928729c78a67448711d2dd7c3153b37cd
3
+ size 808612
checkpoint-20511/target.spm ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fb8ad63c818f073c246952c91b2cb1f16aa30728bf51b5c179b8481bc0e20bb5
3
+ size 756198
checkpoint-20511/tokenizer_config.json ADDED
@@ -0,0 +1,40 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "</s>",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "1": {
12
+ "content": "<unk>",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "53684": {
20
+ "content": "<pad>",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ }
27
+ },
28
+ "additional_special_tokens": null,
29
+ "backend": "custom",
30
+ "eos_token": "</s>",
31
+ "is_local": false,
32
+ "model_max_length": 512,
33
+ "pad_token": "<pad>",
34
+ "separate_vocabs": false,
35
+ "source_lang": "eng",
36
+ "sp_model_kwargs": {},
37
+ "target_lang": "vie",
38
+ "tokenizer_class": "MarianTokenizer",
39
+ "unk_token": "<unk>"
40
+ }
checkpoint-20511/trainer_state.json ADDED
@@ -0,0 +1,1489 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_global_step": 20511,
3
+ "best_metric": 34.292637194048844,
4
+ "best_model_checkpoint": "/content/drive/MyDrive/ai-novel-trans-phase2/checkpoints/opus-mt-en-vi-finetuned/checkpoint-20511",
5
+ "epoch": 1.0,
6
+ "eval_steps": 500,
7
+ "global_step": 20511,
8
+ "is_hyper_param_search": false,
9
+ "is_local_process_zero": true,
10
+ "is_world_process_zero": true,
11
+ "log_history": [
12
+ {
13
+ "epoch": 0.00487543269465165,
14
+ "grad_norm": 2.675152540206909,
15
+ "learning_rate": 9.900000000000002e-06,
16
+ "loss": 5.021145324707032,
17
+ "step": 100
18
+ },
19
+ {
20
+ "epoch": 0.0097508653893033,
21
+ "grad_norm": 7.721086502075195,
22
+ "learning_rate": 1.9900000000000003e-05,
23
+ "loss": 4.396817932128906,
24
+ "step": 200
25
+ },
26
+ {
27
+ "epoch": 0.014626298083954952,
28
+ "grad_norm": 2.3389177322387695,
29
+ "learning_rate": 2.9900000000000002e-05,
30
+ "loss": 4.07245849609375,
31
+ "step": 300
32
+ },
33
+ {
34
+ "epoch": 0.0195017307786066,
35
+ "grad_norm": 2.499263048171997,
36
+ "learning_rate": 3.99e-05,
37
+ "loss": 3.777409973144531,
38
+ "step": 400
39
+ },
40
+ {
41
+ "epoch": 0.024377163473258252,
42
+ "grad_norm": 2.5860238075256348,
43
+ "learning_rate": 4.99e-05,
44
+ "loss": 3.580414123535156,
45
+ "step": 500
46
+ },
47
+ {
48
+ "epoch": 0.029252596167909903,
49
+ "grad_norm": 2.3197336196899414,
50
+ "learning_rate": 4.9752636050172405e-05,
51
+ "loss": 3.4068807983398437,
52
+ "step": 600
53
+ },
54
+ {
55
+ "epoch": 0.034128028862561555,
56
+ "grad_norm": 2.1825883388519287,
57
+ "learning_rate": 4.950277347458898e-05,
58
+ "loss": 3.2431564331054688,
59
+ "step": 700
60
+ },
61
+ {
62
+ "epoch": 0.0390034615572132,
63
+ "grad_norm": 2.3840372562408447,
64
+ "learning_rate": 4.925291089900555e-05,
65
+ "loss": 3.163236389160156,
66
+ "step": 800
67
+ },
68
+ {
69
+ "epoch": 0.04387889425186485,
70
+ "grad_norm": 2.144922971725464,
71
+ "learning_rate": 4.900304832342212e-05,
72
+ "loss": 3.0705633544921875,
73
+ "step": 900
74
+ },
75
+ {
76
+ "epoch": 0.048754326946516505,
77
+ "grad_norm": 2.2080078125,
78
+ "learning_rate": 4.875318574783869e-05,
79
+ "loss": 2.999630126953125,
80
+ "step": 1000
81
+ },
82
+ {
83
+ "epoch": 0.05362975964116815,
84
+ "grad_norm": 2.1018660068511963,
85
+ "learning_rate": 4.850332317225526e-05,
86
+ "loss": 2.9244033813476564,
87
+ "step": 1100
88
+ },
89
+ {
90
+ "epoch": 0.05850519233581981,
91
+ "grad_norm": 2.229487657546997,
92
+ "learning_rate": 4.825346059667183e-05,
93
+ "loss": 2.8450347900390627,
94
+ "step": 1200
95
+ },
96
+ {
97
+ "epoch": 0.06338062503047145,
98
+ "grad_norm": 2.217495918273926,
99
+ "learning_rate": 4.80035980210884e-05,
100
+ "loss": 2.8157199096679686,
101
+ "step": 1300
102
+ },
103
+ {
104
+ "epoch": 0.06825605772512311,
105
+ "grad_norm": 2.1982674598693848,
106
+ "learning_rate": 4.7753735445504974e-05,
107
+ "loss": 2.7499826049804685,
108
+ "step": 1400
109
+ },
110
+ {
111
+ "epoch": 0.07313149041977475,
112
+ "grad_norm": 2.064096450805664,
113
+ "learning_rate": 4.750387286992155e-05,
114
+ "loss": 2.7260015869140624,
115
+ "step": 1500
116
+ },
117
+ {
118
+ "epoch": 0.0780069231144264,
119
+ "grad_norm": 2.13657546043396,
120
+ "learning_rate": 4.7254010294338114e-05,
121
+ "loss": 2.679066162109375,
122
+ "step": 1600
123
+ },
124
+ {
125
+ "epoch": 0.08288235580907806,
126
+ "grad_norm": 2.2849767208099365,
127
+ "learning_rate": 4.700414771875469e-05,
128
+ "loss": 2.6501544189453123,
129
+ "step": 1700
130
+ },
131
+ {
132
+ "epoch": 0.0877577885037297,
133
+ "grad_norm": 2.3214404582977295,
134
+ "learning_rate": 4.6754285143171255e-05,
135
+ "loss": 2.5883297729492187,
136
+ "step": 1800
137
+ },
138
+ {
139
+ "epoch": 0.09263322119838135,
140
+ "grad_norm": 2.2374837398529053,
141
+ "learning_rate": 4.650442256758783e-05,
142
+ "loss": 2.5843768310546875,
143
+ "step": 1900
144
+ },
145
+ {
146
+ "epoch": 0.09750865389303301,
147
+ "grad_norm": 2.1672239303588867,
148
+ "learning_rate": 4.6254559992004395e-05,
149
+ "loss": 2.5350082397460936,
150
+ "step": 2000
151
+ },
152
+ {
153
+ "epoch": 0.10238408658768465,
154
+ "grad_norm": 2.1602306365966797,
155
+ "learning_rate": 4.600469741642097e-05,
156
+ "loss": 2.5131271362304686,
157
+ "step": 2100
158
+ },
159
+ {
160
+ "epoch": 0.1072595192823363,
161
+ "grad_norm": 2.0438578128814697,
162
+ "learning_rate": 4.575483484083754e-05,
163
+ "loss": 2.49188232421875,
164
+ "step": 2200
165
+ },
166
+ {
167
+ "epoch": 0.11213495197698796,
168
+ "grad_norm": 2.2079901695251465,
169
+ "learning_rate": 4.5504972265254116e-05,
170
+ "loss": 2.4501614379882812,
171
+ "step": 2300
172
+ },
173
+ {
174
+ "epoch": 0.11701038467163961,
175
+ "grad_norm": 2.0667238235473633,
176
+ "learning_rate": 4.525510968967068e-05,
177
+ "loss": 2.438180236816406,
178
+ "step": 2400
179
+ },
180
+ {
181
+ "epoch": 0.12188581736629125,
182
+ "grad_norm": 2.1506447792053223,
183
+ "learning_rate": 4.5005247114087257e-05,
184
+ "loss": 2.4134735107421874,
185
+ "step": 2500
186
+ },
187
+ {
188
+ "epoch": 0.1267612500609429,
189
+ "grad_norm": 2.022850275039673,
190
+ "learning_rate": 4.4755384538503823e-05,
191
+ "loss": 2.403486785888672,
192
+ "step": 2600
193
+ },
194
+ {
195
+ "epoch": 0.13163668275559456,
196
+ "grad_norm": 2.2573959827423096,
197
+ "learning_rate": 4.450552196292039e-05,
198
+ "loss": 2.340887451171875,
199
+ "step": 2700
200
+ },
201
+ {
202
+ "epoch": 0.13651211545024622,
203
+ "grad_norm": 2.0588464736938477,
204
+ "learning_rate": 4.425565938733697e-05,
205
+ "loss": 2.3513612365722656,
206
+ "step": 2800
207
+ },
208
+ {
209
+ "epoch": 0.14138754814489787,
210
+ "grad_norm": 2.1294188499450684,
211
+ "learning_rate": 4.400579681175354e-05,
212
+ "loss": 2.306028137207031,
213
+ "step": 2900
214
+ },
215
+ {
216
+ "epoch": 0.1462629808395495,
217
+ "grad_norm": 2.0034472942352295,
218
+ "learning_rate": 4.375593423617011e-05,
219
+ "loss": 2.307658843994141,
220
+ "step": 3000
221
+ },
222
+ {
223
+ "epoch": 0.15113841353420115,
224
+ "grad_norm": 2.172750234603882,
225
+ "learning_rate": 4.350607166058668e-05,
226
+ "loss": 2.297288360595703,
227
+ "step": 3100
228
+ },
229
+ {
230
+ "epoch": 0.1560138462288528,
231
+ "grad_norm": 2.452178478240967,
232
+ "learning_rate": 4.325620908500325e-05,
233
+ "loss": 2.2725160217285154,
234
+ "step": 3200
235
+ },
236
+ {
237
+ "epoch": 0.16088927892350446,
238
+ "grad_norm": 2.1048743724823,
239
+ "learning_rate": 4.300634650941982e-05,
240
+ "loss": 2.2295985412597656,
241
+ "step": 3300
242
+ },
243
+ {
244
+ "epoch": 0.16576471161815612,
245
+ "grad_norm": 2.198026180267334,
246
+ "learning_rate": 4.275648393383639e-05,
247
+ "loss": 2.227683258056641,
248
+ "step": 3400
249
+ },
250
+ {
251
+ "epoch": 0.17064014431280777,
252
+ "grad_norm": 2.098459243774414,
253
+ "learning_rate": 4.2506621358252966e-05,
254
+ "loss": 2.238843994140625,
255
+ "step": 3500
256
+ },
257
+ {
258
+ "epoch": 0.1755155770074594,
259
+ "grad_norm": 2.1324594020843506,
260
+ "learning_rate": 4.225675878266953e-05,
261
+ "loss": 2.1876231384277345,
262
+ "step": 3600
263
+ },
264
+ {
265
+ "epoch": 0.18039100970211105,
266
+ "grad_norm": 1.8316569328308105,
267
+ "learning_rate": 4.2006896207086106e-05,
268
+ "loss": 2.187232666015625,
269
+ "step": 3700
270
+ },
271
+ {
272
+ "epoch": 0.1852664423967627,
273
+ "grad_norm": 2.2253286838531494,
274
+ "learning_rate": 4.175703363150267e-05,
275
+ "loss": 2.193212127685547,
276
+ "step": 3800
277
+ },
278
+ {
279
+ "epoch": 0.19014187509141436,
280
+ "grad_norm": 2.0418787002563477,
281
+ "learning_rate": 4.150717105591925e-05,
282
+ "loss": 2.1699429321289063,
283
+ "step": 3900
284
+ },
285
+ {
286
+ "epoch": 0.19501730778606602,
287
+ "grad_norm": 1.9348831176757812,
288
+ "learning_rate": 4.1257308480335814e-05,
289
+ "loss": 2.142833709716797,
290
+ "step": 4000
291
+ },
292
+ {
293
+ "epoch": 0.19989274048071767,
294
+ "grad_norm": 2.0584564208984375,
295
+ "learning_rate": 4.100744590475239e-05,
296
+ "loss": 2.137293395996094,
297
+ "step": 4100
298
+ },
299
+ {
300
+ "epoch": 0.2047681731753693,
301
+ "grad_norm": 1.9605926275253296,
302
+ "learning_rate": 4.075758332916896e-05,
303
+ "loss": 2.119436950683594,
304
+ "step": 4200
305
+ },
306
+ {
307
+ "epoch": 0.20964360587002095,
308
+ "grad_norm": 2.2406749725341797,
309
+ "learning_rate": 4.0507720753585535e-05,
310
+ "loss": 2.1081979370117185,
311
+ "step": 4300
312
+ },
313
+ {
314
+ "epoch": 0.2145190385646726,
315
+ "grad_norm": 2.4982388019561768,
316
+ "learning_rate": 4.02578581780021e-05,
317
+ "loss": 2.1256338500976564,
318
+ "step": 4400
319
+ },
320
+ {
321
+ "epoch": 0.21939447125932426,
322
+ "grad_norm": 2.029024600982666,
323
+ "learning_rate": 4.0007995602418675e-05,
324
+ "loss": 2.0895063781738283,
325
+ "step": 4500
326
+ },
327
+ {
328
+ "epoch": 0.22426990395397592,
329
+ "grad_norm": 2.054234743118286,
330
+ "learning_rate": 3.975813302683524e-05,
331
+ "loss": 2.111855773925781,
332
+ "step": 4600
333
+ },
334
+ {
335
+ "epoch": 0.22914533664862757,
336
+ "grad_norm": 2.16369891166687,
337
+ "learning_rate": 3.950827045125181e-05,
338
+ "loss": 2.0698446655273437,
339
+ "step": 4700
340
+ },
341
+ {
342
+ "epoch": 0.23402076934327923,
343
+ "grad_norm": 1.9786152839660645,
344
+ "learning_rate": 3.925840787566838e-05,
345
+ "loss": 2.043954162597656,
346
+ "step": 4800
347
+ },
348
+ {
349
+ "epoch": 0.23889620203793085,
350
+ "grad_norm": 2.0516932010650635,
351
+ "learning_rate": 3.9008545300084956e-05,
352
+ "loss": 2.043524169921875,
353
+ "step": 4900
354
+ },
355
+ {
356
+ "epoch": 0.2437716347325825,
357
+ "grad_norm": 1.794907569885254,
358
+ "learning_rate": 3.875868272450153e-05,
359
+ "loss": 2.04251708984375,
360
+ "step": 5000
361
+ },
362
+ {
363
+ "epoch": 0.24864706742723416,
364
+ "grad_norm": 1.9470704793930054,
365
+ "learning_rate": 3.85088201489181e-05,
366
+ "loss": 2.017793731689453,
367
+ "step": 5100
368
+ },
369
+ {
370
+ "epoch": 0.2535225001218858,
371
+ "grad_norm": 1.9397050142288208,
372
+ "learning_rate": 3.825895757333467e-05,
373
+ "loss": 2.0219102478027344,
374
+ "step": 5200
375
+ },
376
+ {
377
+ "epoch": 0.25839793281653745,
378
+ "grad_norm": 2.1335129737854004,
379
+ "learning_rate": 3.800909499775124e-05,
380
+ "loss": 2.0124751281738282,
381
+ "step": 5300
382
+ },
383
+ {
384
+ "epoch": 0.26327336551118913,
385
+ "grad_norm": 1.9796847105026245,
386
+ "learning_rate": 3.775923242216781e-05,
387
+ "loss": 2.00731201171875,
388
+ "step": 5400
389
+ },
390
+ {
391
+ "epoch": 0.26814879820584075,
392
+ "grad_norm": 2.160703182220459,
393
+ "learning_rate": 3.750936984658438e-05,
394
+ "loss": 2.005726776123047,
395
+ "step": 5500
396
+ },
397
+ {
398
+ "epoch": 0.27302423090049244,
399
+ "grad_norm": 1.846185564994812,
400
+ "learning_rate": 3.725950727100095e-05,
401
+ "loss": 2.0051583862304687,
402
+ "step": 5600
403
+ },
404
+ {
405
+ "epoch": 0.27789966359514406,
406
+ "grad_norm": 1.893940806388855,
407
+ "learning_rate": 3.7009644695417525e-05,
408
+ "loss": 1.983333282470703,
409
+ "step": 5700
410
+ },
411
+ {
412
+ "epoch": 0.28277509628979575,
413
+ "grad_norm": 2.107341766357422,
414
+ "learning_rate": 3.675978211983409e-05,
415
+ "loss": 1.9938795471191406,
416
+ "step": 5800
417
+ },
418
+ {
419
+ "epoch": 0.2876505289844474,
420
+ "grad_norm": 2.134247064590454,
421
+ "learning_rate": 3.6509919544250666e-05,
422
+ "loss": 1.9497544860839844,
423
+ "step": 5900
424
+ },
425
+ {
426
+ "epoch": 0.292525961679099,
427
+ "grad_norm": 1.9818617105484009,
428
+ "learning_rate": 3.626005696866723e-05,
429
+ "loss": 1.9953677368164062,
430
+ "step": 6000
431
+ },
432
+ {
433
+ "epoch": 0.2974013943737507,
434
+ "grad_norm": 1.8093796968460083,
435
+ "learning_rate": 3.6010194393083806e-05,
436
+ "loss": 1.9343972778320313,
437
+ "step": 6100
438
+ },
439
+ {
440
+ "epoch": 0.3022768270684023,
441
+ "grad_norm": 1.9501878023147583,
442
+ "learning_rate": 3.576033181750037e-05,
443
+ "loss": 1.9252523803710937,
444
+ "step": 6200
445
+ },
446
+ {
447
+ "epoch": 0.307152259763054,
448
+ "grad_norm": 2.3312392234802246,
449
+ "learning_rate": 3.551046924191695e-05,
450
+ "loss": 1.9442298889160157,
451
+ "step": 6300
452
+ },
453
+ {
454
+ "epoch": 0.3120276924577056,
455
+ "grad_norm": 2.0151638984680176,
456
+ "learning_rate": 3.526060666633352e-05,
457
+ "loss": 1.9296180725097656,
458
+ "step": 6400
459
+ },
460
+ {
461
+ "epoch": 0.31690312515235725,
462
+ "grad_norm": 1.8087499141693115,
463
+ "learning_rate": 3.5010744090750094e-05,
464
+ "loss": 1.9206549072265624,
465
+ "step": 6500
466
+ },
467
+ {
468
+ "epoch": 0.32177855784700893,
469
+ "grad_norm": 1.8592939376831055,
470
+ "learning_rate": 3.476088151516666e-05,
471
+ "loss": 1.914606170654297,
472
+ "step": 6600
473
+ },
474
+ {
475
+ "epoch": 0.32665399054166055,
476
+ "grad_norm": 2.135909080505371,
477
+ "learning_rate": 3.451101893958323e-05,
478
+ "loss": 1.90611572265625,
479
+ "step": 6700
480
+ },
481
+ {
482
+ "epoch": 0.33152942323631224,
483
+ "grad_norm": 2.068551778793335,
484
+ "learning_rate": 3.42611563639998e-05,
485
+ "loss": 1.9218936157226563,
486
+ "step": 6800
487
+ },
488
+ {
489
+ "epoch": 0.33640485593096386,
490
+ "grad_norm": 1.8828186988830566,
491
+ "learning_rate": 3.401129378841637e-05,
492
+ "loss": 1.8921115112304687,
493
+ "step": 6900
494
+ },
495
+ {
496
+ "epoch": 0.34128028862561555,
497
+ "grad_norm": 1.8261306285858154,
498
+ "learning_rate": 3.376143121283294e-05,
499
+ "loss": 1.8862152099609375,
500
+ "step": 7000
501
+ },
502
+ {
503
+ "epoch": 0.3461557213202672,
504
+ "grad_norm": 1.9373810291290283,
505
+ "learning_rate": 3.3511568637249516e-05,
506
+ "loss": 1.9040310668945313,
507
+ "step": 7100
508
+ },
509
+ {
510
+ "epoch": 0.3510311540149188,
511
+ "grad_norm": 2.3029720783233643,
512
+ "learning_rate": 3.326170606166609e-05,
513
+ "loss": 1.8923916625976562,
514
+ "step": 7200
515
+ },
516
+ {
517
+ "epoch": 0.3559065867095705,
518
+ "grad_norm": 1.866101622581482,
519
+ "learning_rate": 3.3011843486082656e-05,
520
+ "loss": 1.8574737548828124,
521
+ "step": 7300
522
+ },
523
+ {
524
+ "epoch": 0.3607820194042221,
525
+ "grad_norm": 1.8397579193115234,
526
+ "learning_rate": 3.276198091049923e-05,
527
+ "loss": 1.8819792175292969,
528
+ "step": 7400
529
+ },
530
+ {
531
+ "epoch": 0.3656574520988738,
532
+ "grad_norm": 1.8102997541427612,
533
+ "learning_rate": 3.2512118334915797e-05,
534
+ "loss": 1.8614399719238282,
535
+ "step": 7500
536
+ },
537
+ {
538
+ "epoch": 0.3705328847935254,
539
+ "grad_norm": 1.9585081338882446,
540
+ "learning_rate": 3.2262255759332363e-05,
541
+ "loss": 1.8699641418457031,
542
+ "step": 7600
543
+ },
544
+ {
545
+ "epoch": 0.3754083174881771,
546
+ "grad_norm": 2.0566818714141846,
547
+ "learning_rate": 3.201239318374894e-05,
548
+ "loss": 1.8685493469238281,
549
+ "step": 7700
550
+ },
551
+ {
552
+ "epoch": 0.38028375018282873,
553
+ "grad_norm": 1.8435136079788208,
554
+ "learning_rate": 3.176253060816551e-05,
555
+ "loss": 1.838872528076172,
556
+ "step": 7800
557
+ },
558
+ {
559
+ "epoch": 0.38515918287748035,
560
+ "grad_norm": 1.9942820072174072,
561
+ "learning_rate": 3.1512668032582084e-05,
562
+ "loss": 1.858647003173828,
563
+ "step": 7900
564
+ },
565
+ {
566
+ "epoch": 0.39003461557213204,
567
+ "grad_norm": 1.818866491317749,
568
+ "learning_rate": 3.126280545699865e-05,
569
+ "loss": 1.8506712341308593,
570
+ "step": 8000
571
+ },
572
+ {
573
+ "epoch": 0.39491004826678366,
574
+ "grad_norm": 1.8237519264221191,
575
+ "learning_rate": 3.1012942881415225e-05,
576
+ "loss": 1.8711978149414064,
577
+ "step": 8100
578
+ },
579
+ {
580
+ "epoch": 0.39978548096143535,
581
+ "grad_norm": 1.8862943649291992,
582
+ "learning_rate": 3.076308030583179e-05,
583
+ "loss": 1.81759521484375,
584
+ "step": 8200
585
+ },
586
+ {
587
+ "epoch": 0.404660913656087,
588
+ "grad_norm": 1.8862155675888062,
589
+ "learning_rate": 3.0513217730248365e-05,
590
+ "loss": 1.8248939514160156,
591
+ "step": 8300
592
+ },
593
+ {
594
+ "epoch": 0.4095363463507386,
595
+ "grad_norm": 1.7012768983840942,
596
+ "learning_rate": 3.0263355154664936e-05,
597
+ "loss": 1.82091552734375,
598
+ "step": 8400
599
+ },
600
+ {
601
+ "epoch": 0.4144117790453903,
602
+ "grad_norm": 2.438452959060669,
603
+ "learning_rate": 3.001349257908151e-05,
604
+ "loss": 1.8241246032714844,
605
+ "step": 8500
606
+ },
607
+ {
608
+ "epoch": 0.4192872117400419,
609
+ "grad_norm": 1.8919984102249146,
610
+ "learning_rate": 2.9763630003498076e-05,
611
+ "loss": 1.814048309326172,
612
+ "step": 8600
613
+ },
614
+ {
615
+ "epoch": 0.4241626444346936,
616
+ "grad_norm": 1.8210819959640503,
617
+ "learning_rate": 2.9513767427914646e-05,
618
+ "loss": 1.8143821716308595,
619
+ "step": 8700
620
+ },
621
+ {
622
+ "epoch": 0.4290380771293452,
623
+ "grad_norm": 1.815765142440796,
624
+ "learning_rate": 2.926390485233122e-05,
625
+ "loss": 1.8236375427246094,
626
+ "step": 8800
627
+ },
628
+ {
629
+ "epoch": 0.4339135098239969,
630
+ "grad_norm": 1.7039874792099,
631
+ "learning_rate": 2.9014042276747787e-05,
632
+ "loss": 1.8135798645019532,
633
+ "step": 8900
634
+ },
635
+ {
636
+ "epoch": 0.43878894251864853,
637
+ "grad_norm": 1.9677067995071411,
638
+ "learning_rate": 2.876417970116436e-05,
639
+ "loss": 1.8013230895996093,
640
+ "step": 9000
641
+ },
642
+ {
643
+ "epoch": 0.44366437521330016,
644
+ "grad_norm": 1.930804967880249,
645
+ "learning_rate": 2.851431712558093e-05,
646
+ "loss": 1.7823497009277345,
647
+ "step": 9100
648
+ },
649
+ {
650
+ "epoch": 0.44853980790795184,
651
+ "grad_norm": 1.7832306623458862,
652
+ "learning_rate": 2.8264454549997504e-05,
653
+ "loss": 1.8109762573242187,
654
+ "step": 9200
655
+ },
656
+ {
657
+ "epoch": 0.45341524060260346,
658
+ "grad_norm": 1.641679286956787,
659
+ "learning_rate": 2.801459197441407e-05,
660
+ "loss": 1.770268096923828,
661
+ "step": 9300
662
+ },
663
+ {
664
+ "epoch": 0.45829067329725515,
665
+ "grad_norm": 1.8651412725448608,
666
+ "learning_rate": 2.776472939883065e-05,
667
+ "loss": 1.7909194946289062,
668
+ "step": 9400
669
+ },
670
+ {
671
+ "epoch": 0.4631661059919068,
672
+ "grad_norm": 1.8264986276626587,
673
+ "learning_rate": 2.7514866823247215e-05,
674
+ "loss": 1.7845869445800782,
675
+ "step": 9500
676
+ },
677
+ {
678
+ "epoch": 0.46804153868655846,
679
+ "grad_norm": 1.9831496477127075,
680
+ "learning_rate": 2.7265004247663782e-05,
681
+ "loss": 1.7713508605957031,
682
+ "step": 9600
683
+ },
684
+ {
685
+ "epoch": 0.4729169713812101,
686
+ "grad_norm": 1.9760777950286865,
687
+ "learning_rate": 2.7015141672080356e-05,
688
+ "loss": 1.7843730163574218,
689
+ "step": 9700
690
+ },
691
+ {
692
+ "epoch": 0.4777924040758617,
693
+ "grad_norm": 1.8475351333618164,
694
+ "learning_rate": 2.6765279096496926e-05,
695
+ "loss": 1.784367218017578,
696
+ "step": 9800
697
+ },
698
+ {
699
+ "epoch": 0.4826678367705134,
700
+ "grad_norm": 1.5956990718841553,
701
+ "learning_rate": 2.65154165209135e-05,
702
+ "loss": 1.7815252685546874,
703
+ "step": 9900
704
+ },
705
+ {
706
+ "epoch": 0.487543269465165,
707
+ "grad_norm": 1.822904109954834,
708
+ "learning_rate": 2.6265553945330067e-05,
709
+ "loss": 1.7850469970703124,
710
+ "step": 10000
711
+ },
712
+ {
713
+ "epoch": 0.4924187021598167,
714
+ "grad_norm": 1.7033106088638306,
715
+ "learning_rate": 2.6015691369746644e-05,
716
+ "loss": 1.7689566040039062,
717
+ "step": 10100
718
+ },
719
+ {
720
+ "epoch": 0.49729413485446833,
721
+ "grad_norm": 1.880291223526001,
722
+ "learning_rate": 2.576582879416321e-05,
723
+ "loss": 1.771258087158203,
724
+ "step": 10200
725
+ },
726
+ {
727
+ "epoch": 0.50216956754912,
728
+ "grad_norm": 1.916098952293396,
729
+ "learning_rate": 2.5515966218579784e-05,
730
+ "loss": 1.7456060791015624,
731
+ "step": 10300
732
+ },
733
+ {
734
+ "epoch": 0.5070450002437716,
735
+ "grad_norm": 1.7015804052352905,
736
+ "learning_rate": 2.5266103642996354e-05,
737
+ "loss": 1.7651783752441406,
738
+ "step": 10400
739
+ },
740
+ {
741
+ "epoch": 0.5119204329384233,
742
+ "grad_norm": 1.7423841953277588,
743
+ "learning_rate": 2.5016241067412928e-05,
744
+ "loss": 1.7516864013671876,
745
+ "step": 10500
746
+ },
747
+ {
748
+ "epoch": 0.5167958656330749,
749
+ "grad_norm": 1.6503269672393799,
750
+ "learning_rate": 2.4766378491829495e-05,
751
+ "loss": 1.7473469543457032,
752
+ "step": 10600
753
+ },
754
+ {
755
+ "epoch": 0.5216712983277266,
756
+ "grad_norm": 1.8282462358474731,
757
+ "learning_rate": 2.4516515916246065e-05,
758
+ "loss": 1.7484811401367188,
759
+ "step": 10700
760
+ },
761
+ {
762
+ "epoch": 0.5265467310223783,
763
+ "grad_norm": 1.5725153684616089,
764
+ "learning_rate": 2.426665334066264e-05,
765
+ "loss": 1.764967803955078,
766
+ "step": 10800
767
+ },
768
+ {
769
+ "epoch": 0.5314221637170299,
770
+ "grad_norm": 1.7148935794830322,
771
+ "learning_rate": 2.401679076507921e-05,
772
+ "loss": 1.7261744689941407,
773
+ "step": 10900
774
+ },
775
+ {
776
+ "epoch": 0.5362975964116815,
777
+ "grad_norm": 1.6759668588638306,
778
+ "learning_rate": 2.376692818949578e-05,
779
+ "loss": 1.7516397094726563,
780
+ "step": 11000
781
+ },
782
+ {
783
+ "epoch": 0.5411730291063331,
784
+ "grad_norm": 1.8212566375732422,
785
+ "learning_rate": 2.351706561391235e-05,
786
+ "loss": 1.739744873046875,
787
+ "step": 11100
788
+ },
789
+ {
790
+ "epoch": 0.5460484618009849,
791
+ "grad_norm": 1.657944679260254,
792
+ "learning_rate": 2.326720303832892e-05,
793
+ "loss": 1.7357514953613282,
794
+ "step": 11200
795
+ },
796
+ {
797
+ "epoch": 0.5509238944956365,
798
+ "grad_norm": 1.732954978942871,
799
+ "learning_rate": 2.301734046274549e-05,
800
+ "loss": 1.7464405822753906,
801
+ "step": 11300
802
+ },
803
+ {
804
+ "epoch": 0.5557993271902881,
805
+ "grad_norm": 1.772199034690857,
806
+ "learning_rate": 2.276747788716206e-05,
807
+ "loss": 1.7311415100097656,
808
+ "step": 11400
809
+ },
810
+ {
811
+ "epoch": 0.5606747598849398,
812
+ "grad_norm": 2.170032024383545,
813
+ "learning_rate": 2.2517615311578634e-05,
814
+ "loss": 1.726700897216797,
815
+ "step": 11500
816
+ },
817
+ {
818
+ "epoch": 0.5655501925795915,
819
+ "grad_norm": 1.995363473892212,
820
+ "learning_rate": 2.2267752735995204e-05,
821
+ "loss": 1.7367793273925782,
822
+ "step": 11600
823
+ },
824
+ {
825
+ "epoch": 0.5704256252742431,
826
+ "grad_norm": 1.8489207029342651,
827
+ "learning_rate": 2.2017890160411774e-05,
828
+ "loss": 1.7422283935546874,
829
+ "step": 11700
830
+ },
831
+ {
832
+ "epoch": 0.5753010579688947,
833
+ "grad_norm": 1.8624141216278076,
834
+ "learning_rate": 2.1768027584828345e-05,
835
+ "loss": 1.7352618408203124,
836
+ "step": 11800
837
+ },
838
+ {
839
+ "epoch": 0.5801764906635464,
840
+ "grad_norm": 1.6637346744537354,
841
+ "learning_rate": 2.151816500924492e-05,
842
+ "loss": 1.7108763122558595,
843
+ "step": 11900
844
+ },
845
+ {
846
+ "epoch": 0.585051923358198,
847
+ "grad_norm": 1.7139227390289307,
848
+ "learning_rate": 2.126830243366149e-05,
849
+ "loss": 1.7246351623535157,
850
+ "step": 12000
851
+ },
852
+ {
853
+ "epoch": 0.5899273560528497,
854
+ "grad_norm": 1.9074546098709106,
855
+ "learning_rate": 2.1018439858078056e-05,
856
+ "loss": 1.719434051513672,
857
+ "step": 12100
858
+ },
859
+ {
860
+ "epoch": 0.5948027887475014,
861
+ "grad_norm": 1.70399010181427,
862
+ "learning_rate": 2.076857728249463e-05,
863
+ "loss": 1.7267167663574219,
864
+ "step": 12200
865
+ },
866
+ {
867
+ "epoch": 0.599678221442153,
868
+ "grad_norm": 1.6839964389801025,
869
+ "learning_rate": 2.05187147069112e-05,
870
+ "loss": 1.704312744140625,
871
+ "step": 12300
872
+ },
873
+ {
874
+ "epoch": 0.6045536541368046,
875
+ "grad_norm": 1.728901743888855,
876
+ "learning_rate": 2.026885213132777e-05,
877
+ "loss": 1.7204804992675782,
878
+ "step": 12400
879
+ },
880
+ {
881
+ "epoch": 0.6094290868314562,
882
+ "grad_norm": 1.6176766157150269,
883
+ "learning_rate": 2.001898955574434e-05,
884
+ "loss": 1.710531005859375,
885
+ "step": 12500
886
+ },
887
+ {
888
+ "epoch": 0.614304519526108,
889
+ "grad_norm": 1.7322443723678589,
890
+ "learning_rate": 1.9769126980160914e-05,
891
+ "loss": 1.6960983276367188,
892
+ "step": 12600
893
+ },
894
+ {
895
+ "epoch": 0.6191799522207596,
896
+ "grad_norm": 1.8794188499450684,
897
+ "learning_rate": 1.9519264404577484e-05,
898
+ "loss": 1.711561737060547,
899
+ "step": 12700
900
+ },
901
+ {
902
+ "epoch": 0.6240553849154112,
903
+ "grad_norm": 1.773924469947815,
904
+ "learning_rate": 1.9269401828994054e-05,
905
+ "loss": 1.6992698669433595,
906
+ "step": 12800
907
+ },
908
+ {
909
+ "epoch": 0.6289308176100629,
910
+ "grad_norm": 1.8185611963272095,
911
+ "learning_rate": 1.9019539253410624e-05,
912
+ "loss": 1.7151986694335937,
913
+ "step": 12900
914
+ },
915
+ {
916
+ "epoch": 0.6338062503047145,
917
+ "grad_norm": 1.7659051418304443,
918
+ "learning_rate": 1.8769676677827198e-05,
919
+ "loss": 1.7099436950683593,
920
+ "step": 13000
921
+ },
922
+ {
923
+ "epoch": 0.6386816829993662,
924
+ "grad_norm": 1.7478703260421753,
925
+ "learning_rate": 1.8519814102243765e-05,
926
+ "loss": 1.7121574401855468,
927
+ "step": 13100
928
+ },
929
+ {
930
+ "epoch": 0.6435571156940179,
931
+ "grad_norm": 1.6302961111068726,
932
+ "learning_rate": 1.8269951526660335e-05,
933
+ "loss": 1.7094390869140625,
934
+ "step": 13200
935
+ },
936
+ {
937
+ "epoch": 0.6484325483886695,
938
+ "grad_norm": 1.7155256271362305,
939
+ "learning_rate": 1.802008895107691e-05,
940
+ "loss": 1.6982522583007813,
941
+ "step": 13300
942
+ },
943
+ {
944
+ "epoch": 0.6533079810833211,
945
+ "grad_norm": 1.825584888458252,
946
+ "learning_rate": 1.777022637549348e-05,
947
+ "loss": 1.7062911987304688,
948
+ "step": 13400
949
+ },
950
+ {
951
+ "epoch": 0.6581834137779728,
952
+ "grad_norm": 1.5996416807174683,
953
+ "learning_rate": 1.752036379991005e-05,
954
+ "loss": 1.6931849670410157,
955
+ "step": 13500
956
+ },
957
+ {
958
+ "epoch": 0.6630588464726245,
959
+ "grad_norm": 1.574539303779602,
960
+ "learning_rate": 1.727050122432662e-05,
961
+ "loss": 1.6707138061523437,
962
+ "step": 13600
963
+ },
964
+ {
965
+ "epoch": 0.6679342791672761,
966
+ "grad_norm": 1.6595592498779297,
967
+ "learning_rate": 1.7020638648743193e-05,
968
+ "loss": 1.6816087341308594,
969
+ "step": 13700
970
+ },
971
+ {
972
+ "epoch": 0.6728097118619277,
973
+ "grad_norm": 1.6490947008132935,
974
+ "learning_rate": 1.6770776073159763e-05,
975
+ "loss": 1.6976029968261719,
976
+ "step": 13800
977
+ },
978
+ {
979
+ "epoch": 0.6776851445565794,
980
+ "grad_norm": 1.7587735652923584,
981
+ "learning_rate": 1.6520913497576334e-05,
982
+ "loss": 1.676434783935547,
983
+ "step": 13900
984
+ },
985
+ {
986
+ "epoch": 0.6825605772512311,
987
+ "grad_norm": 1.6581918001174927,
988
+ "learning_rate": 1.6271050921992907e-05,
989
+ "loss": 1.686712646484375,
990
+ "step": 14000
991
+ },
992
+ {
993
+ "epoch": 0.6874360099458827,
994
+ "grad_norm": 1.6643747091293335,
995
+ "learning_rate": 1.6021188346409474e-05,
996
+ "loss": 1.6771395874023438,
997
+ "step": 14100
998
+ },
999
+ {
1000
+ "epoch": 0.6923114426405343,
1001
+ "grad_norm": 1.766485333442688,
1002
+ "learning_rate": 1.5771325770826045e-05,
1003
+ "loss": 1.6819680786132813,
1004
+ "step": 14200
1005
+ },
1006
+ {
1007
+ "epoch": 0.697186875335186,
1008
+ "grad_norm": 1.6709808111190796,
1009
+ "learning_rate": 1.5521463195242618e-05,
1010
+ "loss": 1.6711248779296874,
1011
+ "step": 14300
1012
+ },
1013
+ {
1014
+ "epoch": 0.7020623080298376,
1015
+ "grad_norm": 1.6720649003982544,
1016
+ "learning_rate": 1.527160061965919e-05,
1017
+ "loss": 1.688917236328125,
1018
+ "step": 14400
1019
+ },
1020
+ {
1021
+ "epoch": 0.7069377407244893,
1022
+ "grad_norm": 1.679030418395996,
1023
+ "learning_rate": 1.5021738044075759e-05,
1024
+ "loss": 1.67065673828125,
1025
+ "step": 14500
1026
+ },
1027
+ {
1028
+ "epoch": 0.711813173419141,
1029
+ "grad_norm": 1.730519413948059,
1030
+ "learning_rate": 1.477187546849233e-05,
1031
+ "loss": 1.6813763427734374,
1032
+ "step": 14600
1033
+ },
1034
+ {
1035
+ "epoch": 0.7166886061137926,
1036
+ "grad_norm": 1.899192452430725,
1037
+ "learning_rate": 1.4522012892908901e-05,
1038
+ "loss": 1.6835823059082031,
1039
+ "step": 14700
1040
+ },
1041
+ {
1042
+ "epoch": 0.7215640388084442,
1043
+ "grad_norm": 1.6476982831954956,
1044
+ "learning_rate": 1.4272150317325473e-05,
1045
+ "loss": 1.6821974182128907,
1046
+ "step": 14800
1047
+ },
1048
+ {
1049
+ "epoch": 0.7264394715030958,
1050
+ "grad_norm": 1.7553383111953735,
1051
+ "learning_rate": 1.4022287741742043e-05,
1052
+ "loss": 1.6879953002929688,
1053
+ "step": 14900
1054
+ },
1055
+ {
1056
+ "epoch": 0.7313149041977476,
1057
+ "grad_norm": 1.6452993154525757,
1058
+ "learning_rate": 1.3772425166158615e-05,
1059
+ "loss": 1.6935087585449218,
1060
+ "step": 15000
1061
+ },
1062
+ {
1063
+ "epoch": 0.7361903368923992,
1064
+ "grad_norm": 1.633382797241211,
1065
+ "learning_rate": 1.3522562590575184e-05,
1066
+ "loss": 1.6577984619140624,
1067
+ "step": 15100
1068
+ },
1069
+ {
1070
+ "epoch": 0.7410657695870508,
1071
+ "grad_norm": 1.7697386741638184,
1072
+ "learning_rate": 1.3272700014991754e-05,
1073
+ "loss": 1.6891175842285155,
1074
+ "step": 15200
1075
+ },
1076
+ {
1077
+ "epoch": 0.7459412022817025,
1078
+ "grad_norm": 1.7632725238800049,
1079
+ "learning_rate": 1.3022837439408326e-05,
1080
+ "loss": 1.660469970703125,
1081
+ "step": 15300
1082
+ },
1083
+ {
1084
+ "epoch": 0.7508166349763542,
1085
+ "grad_norm": 1.7407395839691162,
1086
+ "learning_rate": 1.2772974863824896e-05,
1087
+ "loss": 1.664199981689453,
1088
+ "step": 15400
1089
+ },
1090
+ {
1091
+ "epoch": 0.7556920676710058,
1092
+ "grad_norm": 1.6564642190933228,
1093
+ "learning_rate": 1.2523112288241468e-05,
1094
+ "loss": 1.6778285217285156,
1095
+ "step": 15500
1096
+ },
1097
+ {
1098
+ "epoch": 0.7605675003656575,
1099
+ "grad_norm": 1.826248049736023,
1100
+ "learning_rate": 1.2273249712658038e-05,
1101
+ "loss": 1.6732501220703124,
1102
+ "step": 15600
1103
+ },
1104
+ {
1105
+ "epoch": 0.7654429330603091,
1106
+ "grad_norm": 1.8338110446929932,
1107
+ "learning_rate": 1.202338713707461e-05,
1108
+ "loss": 1.662447967529297,
1109
+ "step": 15700
1110
+ },
1111
+ {
1112
+ "epoch": 0.7703183657549607,
1113
+ "grad_norm": 1.6885141134262085,
1114
+ "learning_rate": 1.177352456149118e-05,
1115
+ "loss": 1.6777557373046874,
1116
+ "step": 15800
1117
+ },
1118
+ {
1119
+ "epoch": 0.7751937984496124,
1120
+ "grad_norm": 1.8717424869537354,
1121
+ "learning_rate": 1.152366198590775e-05,
1122
+ "loss": 1.6633432006835938,
1123
+ "step": 15900
1124
+ },
1125
+ {
1126
+ "epoch": 0.7800692311442641,
1127
+ "grad_norm": 1.7940692901611328,
1128
+ "learning_rate": 1.1273799410324323e-05,
1129
+ "loss": 1.666638641357422,
1130
+ "step": 16000
1131
+ },
1132
+ {
1133
+ "epoch": 0.7849446638389157,
1134
+ "grad_norm": 1.6228216886520386,
1135
+ "learning_rate": 1.1023936834740893e-05,
1136
+ "loss": 1.6681695556640626,
1137
+ "step": 16100
1138
+ },
1139
+ {
1140
+ "epoch": 0.7898200965335673,
1141
+ "grad_norm": 1.9657390117645264,
1142
+ "learning_rate": 1.0774074259157465e-05,
1143
+ "loss": 1.671024627685547,
1144
+ "step": 16200
1145
+ },
1146
+ {
1147
+ "epoch": 0.794695529228219,
1148
+ "grad_norm": 1.585915207862854,
1149
+ "learning_rate": 1.0524211683574035e-05,
1150
+ "loss": 1.6706533813476563,
1151
+ "step": 16300
1152
+ },
1153
+ {
1154
+ "epoch": 0.7995709619228707,
1155
+ "grad_norm": 1.7579628229141235,
1156
+ "learning_rate": 1.0274349107990605e-05,
1157
+ "loss": 1.6688577270507812,
1158
+ "step": 16400
1159
+ },
1160
+ {
1161
+ "epoch": 0.8044463946175223,
1162
+ "grad_norm": 1.7315977811813354,
1163
+ "learning_rate": 1.0024486532407176e-05,
1164
+ "loss": 1.6563613891601563,
1165
+ "step": 16500
1166
+ },
1167
+ {
1168
+ "epoch": 0.809321827312174,
1169
+ "grad_norm": 1.6918113231658936,
1170
+ "learning_rate": 9.774623956823748e-06,
1171
+ "loss": 1.6541168212890625,
1172
+ "step": 16600
1173
+ },
1174
+ {
1175
+ "epoch": 0.8141972600068256,
1176
+ "grad_norm": 1.7037303447723389,
1177
+ "learning_rate": 9.524761381240318e-06,
1178
+ "loss": 1.6581985473632812,
1179
+ "step": 16700
1180
+ },
1181
+ {
1182
+ "epoch": 0.8190726927014772,
1183
+ "grad_norm": 1.732756495475769,
1184
+ "learning_rate": 9.27489880565689e-06,
1185
+ "loss": 1.6568051147460938,
1186
+ "step": 16800
1187
+ },
1188
+ {
1189
+ "epoch": 0.8239481253961289,
1190
+ "grad_norm": 1.4957330226898193,
1191
+ "learning_rate": 9.02503623007346e-06,
1192
+ "loss": 1.6625689697265624,
1193
+ "step": 16900
1194
+ },
1195
+ {
1196
+ "epoch": 0.8288235580907806,
1197
+ "grad_norm": 1.7349003553390503,
1198
+ "learning_rate": 8.77517365449003e-06,
1199
+ "loss": 1.6577049255371095,
1200
+ "step": 17000
1201
+ },
1202
+ {
1203
+ "epoch": 0.8336989907854322,
1204
+ "grad_norm": 1.6561371088027954,
1205
+ "learning_rate": 8.525311078906602e-06,
1206
+ "loss": 1.6655906677246093,
1207
+ "step": 17100
1208
+ },
1209
+ {
1210
+ "epoch": 0.8385744234800838,
1211
+ "grad_norm": 1.7264946699142456,
1212
+ "learning_rate": 8.275448503323173e-06,
1213
+ "loss": 1.6579425048828125,
1214
+ "step": 17200
1215
+ },
1216
+ {
1217
+ "epoch": 0.8434498561747356,
1218
+ "grad_norm": 1.740032434463501,
1219
+ "learning_rate": 8.025585927739745e-06,
1220
+ "loss": 1.6556771850585938,
1221
+ "step": 17300
1222
+ },
1223
+ {
1224
+ "epoch": 0.8483252888693872,
1225
+ "grad_norm": 1.8917021751403809,
1226
+ "learning_rate": 7.775723352156313e-06,
1227
+ "loss": 1.6798193359375,
1228
+ "step": 17400
1229
+ },
1230
+ {
1231
+ "epoch": 0.8532007215640388,
1232
+ "grad_norm": 1.6723499298095703,
1233
+ "learning_rate": 7.525860776572885e-06,
1234
+ "loss": 1.6542041015625,
1235
+ "step": 17500
1236
+ },
1237
+ {
1238
+ "epoch": 0.8580761542586904,
1239
+ "grad_norm": 1.7309447526931763,
1240
+ "learning_rate": 7.275998200989456e-06,
1241
+ "loss": 1.6616473388671875,
1242
+ "step": 17600
1243
+ },
1244
+ {
1245
+ "epoch": 0.8629515869533421,
1246
+ "grad_norm": 1.7703163623809814,
1247
+ "learning_rate": 7.026135625406027e-06,
1248
+ "loss": 1.6348822021484375,
1249
+ "step": 17700
1250
+ },
1251
+ {
1252
+ "epoch": 0.8678270196479938,
1253
+ "grad_norm": 1.446179747581482,
1254
+ "learning_rate": 6.776273049822598e-06,
1255
+ "loss": 1.6590901184082032,
1256
+ "step": 17800
1257
+ },
1258
+ {
1259
+ "epoch": 0.8727024523426454,
1260
+ "grad_norm": 2.105104684829712,
1261
+ "learning_rate": 6.526410474239169e-06,
1262
+ "loss": 1.6562322998046874,
1263
+ "step": 17900
1264
+ },
1265
+ {
1266
+ "epoch": 0.8775778850372971,
1267
+ "grad_norm": 1.6501487493515015,
1268
+ "learning_rate": 6.27654789865574e-06,
1269
+ "loss": 1.6521522521972656,
1270
+ "step": 18000
1271
+ },
1272
+ {
1273
+ "epoch": 0.8824533177319487,
1274
+ "grad_norm": 1.605299472808838,
1275
+ "learning_rate": 6.026685323072311e-06,
1276
+ "loss": 1.6424995422363282,
1277
+ "step": 18100
1278
+ },
1279
+ {
1280
+ "epoch": 0.8873287504266003,
1281
+ "grad_norm": 1.7323307991027832,
1282
+ "learning_rate": 5.776822747488882e-06,
1283
+ "loss": 1.6342855834960937,
1284
+ "step": 18200
1285
+ },
1286
+ {
1287
+ "epoch": 0.892204183121252,
1288
+ "grad_norm": 1.7684431076049805,
1289
+ "learning_rate": 5.526960171905452e-06,
1290
+ "loss": 1.6428620910644531,
1291
+ "step": 18300
1292
+ },
1293
+ {
1294
+ "epoch": 0.8970796158159037,
1295
+ "grad_norm": 1.6070303916931152,
1296
+ "learning_rate": 5.277097596322023e-06,
1297
+ "loss": 1.6442176818847656,
1298
+ "step": 18400
1299
+ },
1300
+ {
1301
+ "epoch": 0.9019550485105553,
1302
+ "grad_norm": 1.5561784505844116,
1303
+ "learning_rate": 5.0272350207385936e-06,
1304
+ "loss": 1.6308372497558594,
1305
+ "step": 18500
1306
+ },
1307
+ {
1308
+ "epoch": 0.9068304812052069,
1309
+ "grad_norm": 1.548672080039978,
1310
+ "learning_rate": 4.777372445155165e-06,
1311
+ "loss": 1.6469439697265624,
1312
+ "step": 18600
1313
+ },
1314
+ {
1315
+ "epoch": 0.9117059138998587,
1316
+ "grad_norm": 1.568382978439331,
1317
+ "learning_rate": 4.527509869571736e-06,
1318
+ "loss": 1.6599581909179688,
1319
+ "step": 18700
1320
+ },
1321
+ {
1322
+ "epoch": 0.9165813465945103,
1323
+ "grad_norm": 1.96321702003479,
1324
+ "learning_rate": 4.277647293988307e-06,
1325
+ "loss": 1.6601251220703126,
1326
+ "step": 18800
1327
+ },
1328
+ {
1329
+ "epoch": 0.9214567792891619,
1330
+ "grad_norm": 1.6890432834625244,
1331
+ "learning_rate": 4.027784718404878e-06,
1332
+ "loss": 1.64880859375,
1333
+ "step": 18900
1334
+ },
1335
+ {
1336
+ "epoch": 0.9263322119838135,
1337
+ "grad_norm": 1.723892331123352,
1338
+ "learning_rate": 3.7779221428214482e-06,
1339
+ "loss": 1.6504551696777343,
1340
+ "step": 19000
1341
+ },
1342
+ {
1343
+ "epoch": 0.9312076446784652,
1344
+ "grad_norm": 1.6078920364379883,
1345
+ "learning_rate": 3.5280595672380193e-06,
1346
+ "loss": 1.6513778686523437,
1347
+ "step": 19100
1348
+ },
1349
+ {
1350
+ "epoch": 0.9360830773731169,
1351
+ "grad_norm": 1.5909255743026733,
1352
+ "learning_rate": 3.2781969916545905e-06,
1353
+ "loss": 1.64377197265625,
1354
+ "step": 19200
1355
+ },
1356
+ {
1357
+ "epoch": 0.9409585100677685,
1358
+ "grad_norm": 2.2913496494293213,
1359
+ "learning_rate": 3.0283344160711607e-06,
1360
+ "loss": 1.6582955932617187,
1361
+ "step": 19300
1362
+ },
1363
+ {
1364
+ "epoch": 0.9458339427624202,
1365
+ "grad_norm": 1.8474797010421753,
1366
+ "learning_rate": 2.7784718404877322e-06,
1367
+ "loss": 1.64448486328125,
1368
+ "step": 19400
1369
+ },
1370
+ {
1371
+ "epoch": 0.9507093754570718,
1372
+ "grad_norm": 1.5226041078567505,
1373
+ "learning_rate": 2.528609264904303e-06,
1374
+ "loss": 1.6528720092773437,
1375
+ "step": 19500
1376
+ },
1377
+ {
1378
+ "epoch": 0.9555848081517234,
1379
+ "grad_norm": 1.718706488609314,
1380
+ "learning_rate": 2.2787466893208736e-06,
1381
+ "loss": 1.6452322387695313,
1382
+ "step": 19600
1383
+ },
1384
+ {
1385
+ "epoch": 0.9604602408463752,
1386
+ "grad_norm": 1.5899055004119873,
1387
+ "learning_rate": 2.0288841137374443e-06,
1388
+ "loss": 1.6391017150878906,
1389
+ "step": 19700
1390
+ },
1391
+ {
1392
+ "epoch": 0.9653356735410268,
1393
+ "grad_norm": 1.6257293224334717,
1394
+ "learning_rate": 1.7790215381540152e-06,
1395
+ "loss": 1.6389993286132813,
1396
+ "step": 19800
1397
+ },
1398
+ {
1399
+ "epoch": 0.9702111062356784,
1400
+ "grad_norm": 1.6057407855987549,
1401
+ "learning_rate": 1.5291589625705863e-06,
1402
+ "loss": 1.6355738830566406,
1403
+ "step": 19900
1404
+ },
1405
+ {
1406
+ "epoch": 0.97508653893033,
1407
+ "grad_norm": 1.6754605770111084,
1408
+ "learning_rate": 1.279296386987157e-06,
1409
+ "loss": 1.6333837890625,
1410
+ "step": 20000
1411
+ },
1412
+ {
1413
+ "epoch": 0.9799619716249817,
1414
+ "grad_norm": 1.6939451694488525,
1415
+ "learning_rate": 1.029433811403728e-06,
1416
+ "loss": 1.6612481689453125,
1417
+ "step": 20100
1418
+ },
1419
+ {
1420
+ "epoch": 0.9848374043196334,
1421
+ "grad_norm": 1.722284197807312,
1422
+ "learning_rate": 7.79571235820299e-07,
1423
+ "loss": 1.631604461669922,
1424
+ "step": 20200
1425
+ },
1426
+ {
1427
+ "epoch": 0.989712837014285,
1428
+ "grad_norm": 1.5405986309051514,
1429
+ "learning_rate": 5.297086602368698e-07,
1430
+ "loss": 1.6482058715820314,
1431
+ "step": 20300
1432
+ },
1433
+ {
1434
+ "epoch": 0.9945882697089367,
1435
+ "grad_norm": 1.7810713052749634,
1436
+ "learning_rate": 2.798460846534406e-07,
1437
+ "loss": 1.6190275573730468,
1438
+ "step": 20400
1439
+ },
1440
+ {
1441
+ "epoch": 0.9994637024035883,
1442
+ "grad_norm": 1.6377924680709839,
1443
+ "learning_rate": 2.998350907001149e-08,
1444
+ "loss": 1.6395912170410156,
1445
+ "step": 20500
1446
+ },
1447
+ {
1448
+ "epoch": 1.0,
1449
+ "eval_bleu": 34.292637194048844,
1450
+ "eval_chrf": 54.5365465177637,
1451
+ "eval_gen_len": 183.4760305697082,
1452
+ "eval_loss": 1.4957355260849,
1453
+ "eval_runtime": 7388.6294,
1454
+ "eval_samples_per_second": 2.338,
1455
+ "eval_steps_per_second": 0.146,
1456
+ "step": 20511
1457
+ }
1458
+ ],
1459
+ "logging_steps": 100,
1460
+ "max_steps": 20511,
1461
+ "num_input_tokens_seen": 0,
1462
+ "num_train_epochs": 1,
1463
+ "save_steps": 500,
1464
+ "stateful_callbacks": {
1465
+ "EarlyStoppingCallback": {
1466
+ "args": {
1467
+ "early_stopping_patience": 3,
1468
+ "early_stopping_threshold": 0.0
1469
+ },
1470
+ "attributes": {
1471
+ "early_stopping_patience_counter": 0
1472
+ }
1473
+ },
1474
+ "TrainerControl": {
1475
+ "args": {
1476
+ "should_epoch_stop": false,
1477
+ "should_evaluate": false,
1478
+ "should_log": false,
1479
+ "should_save": true,
1480
+ "should_training_stop": true
1481
+ },
1482
+ "attributes": {}
1483
+ }
1484
+ },
1485
+ "total_flos": 3.728957430457958e+16,
1486
+ "train_batch_size": 16,
1487
+ "trial_name": null,
1488
+ "trial_params": null
1489
+ }
checkpoint-20511/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4969b437b1dd4e2f51b7f70967b776124cbb544ccf9f6aa05cfb080347741b87
3
+ size 5393
checkpoint-20511/vocab.json ADDED
The diff for this file is too large to render. See raw diff
 
config.json ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "activation_dropout": 0.0,
3
+ "activation_function": "swish",
4
+ "add_bias_logits": false,
5
+ "add_final_layer_norm": false,
6
+ "architectures": [
7
+ "MarianMTModel"
8
+ ],
9
+ "attention_dropout": 0.0,
10
+ "bos_token_id": null,
11
+ "classif_dropout": 0.0,
12
+ "classifier_dropout": 0.0,
13
+ "d_model": 512,
14
+ "decoder_attention_heads": 8,
15
+ "decoder_ffn_dim": 2048,
16
+ "decoder_layerdrop": 0.0,
17
+ "decoder_layers": 6,
18
+ "decoder_start_token_id": 53684,
19
+ "decoder_vocab_size": 53685,
20
+ "dropout": 0.1,
21
+ "dtype": "float32",
22
+ "encoder_attention_heads": 8,
23
+ "encoder_ffn_dim": 2048,
24
+ "encoder_layerdrop": 0.0,
25
+ "encoder_layers": 6,
26
+ "eos_token_id": 0,
27
+ "extra_pos_embeddings": 53685,
28
+ "id2label": {
29
+ "0": "LABEL_0",
30
+ "1": "LABEL_1",
31
+ "2": "LABEL_2"
32
+ },
33
+ "init_std": 0.02,
34
+ "is_decoder": false,
35
+ "is_encoder_decoder": true,
36
+ "label2id": {
37
+ "LABEL_0": 0,
38
+ "LABEL_1": 1,
39
+ "LABEL_2": 2
40
+ },
41
+ "max_position_embeddings": 512,
42
+ "model_type": "marian",
43
+ "normalize_before": false,
44
+ "normalize_embedding": false,
45
+ "num_hidden_layers": 6,
46
+ "pad_token_id": 53684,
47
+ "scale_embedding": true,
48
+ "share_encoder_decoder_embeddings": true,
49
+ "static_position_embeddings": true,
50
+ "tie_word_embeddings": true,
51
+ "transformers_version": "5.0.0",
52
+ "use_cache": false,
53
+ "vocab_size": 53685
54
+ }
generation_config.json ADDED
@@ -0,0 +1,43 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "assistant_confidence_threshold": 0.4,
3
+ "assistant_lookbehind": 10,
4
+ "bad_words_ids": [
5
+ [
6
+ 53684
7
+ ]
8
+ ],
9
+ "decoder_start_token_id": 53684,
10
+ "diversity_penalty": 0.0,
11
+ "do_sample": false,
12
+ "early_stopping": false,
13
+ "encoder_no_repeat_ngram_size": 0,
14
+ "encoder_repetition_penalty": 1.0,
15
+ "eos_token_id": [
16
+ 0
17
+ ],
18
+ "epsilon_cutoff": 0.0,
19
+ "eta_cutoff": 0.0,
20
+ "forced_eos_token_id": 0,
21
+ "length_penalty": 1.0,
22
+ "max_length": 512,
23
+ "min_length": 0,
24
+ "no_repeat_ngram_size": 0,
25
+ "num_assistant_tokens": 20,
26
+ "num_assistant_tokens_schedule": "constant",
27
+ "num_beam_groups": 1,
28
+ "num_beams": 4,
29
+ "num_return_sequences": 1,
30
+ "output_scores": false,
31
+ "pad_token_id": 53684,
32
+ "remove_invalid_values": false,
33
+ "renormalize_logits": true,
34
+ "repetition_penalty": 1.0,
35
+ "return_dict_in_generate": false,
36
+ "target_lookbehind": 10,
37
+ "temperature": 1.0,
38
+ "top_k": 50,
39
+ "top_p": 1.0,
40
+ "transformers_version": "5.0.0",
41
+ "typical_p": 1.0,
42
+ "use_cache": true
43
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4e207ce875bb5f6b0250350ae0fed1861b7ad68f0be5c98c82ee2f323be3838c
3
+ size 506638972
source.spm ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fdd9ab486ca8ea060967105e30db7d8928729c78a67448711d2dd7c3153b37cd
3
+ size 808612
target.spm ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fb8ad63c818f073c246952c91b2cb1f16aa30728bf51b5c179b8481bc0e20bb5
3
+ size 756198
tokenizer_config.json ADDED
@@ -0,0 +1,40 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "</s>",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "1": {
12
+ "content": "<unk>",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "53684": {
20
+ "content": "<pad>",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ }
27
+ },
28
+ "additional_special_tokens": null,
29
+ "backend": "custom",
30
+ "eos_token": "</s>",
31
+ "is_local": false,
32
+ "model_max_length": 512,
33
+ "pad_token": "<pad>",
34
+ "separate_vocabs": false,
35
+ "source_lang": "eng",
36
+ "sp_model_kwargs": {},
37
+ "target_lang": "vie",
38
+ "tokenizer_class": "MarianTokenizer",
39
+ "unk_token": "<unk>"
40
+ }
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4969b437b1dd4e2f51b7f70967b776124cbb544ccf9f6aa05cfb080347741b87
3
+ size 5393
vocab.json ADDED
The diff for this file is too large to render. See raw diff