| [INFO|2025-04-10 02:12:51] tokenization_utils_base.py:2050 >> loading file vocab.json from cache at /root/.cache/huggingface/hub/models--Qwen--Qwen2.5-7B/snapshots/d149729398750b98c0af14eb82c78cfe92750796/vocab.json |
|
|
| [INFO|2025-04-10 02:12:51] tokenization_utils_base.py:2050 >> loading file merges.txt from cache at /root/.cache/huggingface/hub/models--Qwen--Qwen2.5-7B/snapshots/d149729398750b98c0af14eb82c78cfe92750796/merges.txt |
|
|
| [INFO|2025-04-10 02:12:51] tokenization_utils_base.py:2050 >> loading file tokenizer.json from cache at /root/.cache/huggingface/hub/models--Qwen--Qwen2.5-7B/snapshots/d149729398750b98c0af14eb82c78cfe92750796/tokenizer.json |
|
|
| [INFO|2025-04-10 02:12:51] tokenization_utils_base.py:2050 >> loading file added_tokens.json from cache at None |
|
|
| [INFO|2025-04-10 02:12:51] tokenization_utils_base.py:2050 >> loading file special_tokens_map.json from cache at None |
|
|
| [INFO|2025-04-10 02:12:51] tokenization_utils_base.py:2050 >> loading file tokenizer_config.json from cache at /root/.cache/huggingface/hub/models--Qwen--Qwen2.5-7B/snapshots/d149729398750b98c0af14eb82c78cfe92750796/tokenizer_config.json |
|
|
| [INFO|2025-04-10 02:12:51] tokenization_utils_base.py:2050 >> loading file chat_template.jinja from cache at None |
|
|
| [INFO|2025-04-10 02:12:52] tokenization_utils_base.py:2313 >> Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. |
|
|
| [INFO|2025-04-10 02:12:52] configuration_utils.py:699 >> loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--Qwen--Qwen2.5-7B/snapshots/d149729398750b98c0af14eb82c78cfe92750796/config.json |
|
|
| [INFO|2025-04-10 02:12:52] configuration_utils.py:771 >> Model config Qwen2Config { |
| "_name_or_path": "Qwen/Qwen2.5-7B", |
| "architectures": [ |
| "Qwen2ForCausalLM" |
| ], |
| "attention_dropout": 0.0, |
| "bos_token_id": 151643, |
| "eos_token_id": 151643, |
| "hidden_act": "silu", |
| "hidden_size": 3584, |
| "initializer_range": 0.02, |
| "intermediate_size": 18944, |
| "max_position_embeddings": 131072, |
| "max_window_layers": 28, |
| "model_type": "qwen2", |
| "num_attention_heads": 28, |
| "num_hidden_layers": 28, |
| "num_key_value_heads": 4, |
| "rms_norm_eps": 1e-06, |
| "rope_scaling": null, |
| "rope_theta": 1000000.0, |
| "sliding_window": 131072, |
| "tie_word_embeddings": false, |
| "torch_dtype": "bfloat16", |
| "transformers_version": "4.49.0", |
| "use_cache": true, |
| "use_mrope": false, |
| "use_sliding_window": false, |
| "vocab_size": 152064 |
| } |
|
|
|
|
| [INFO|2025-04-10 02:12:52] tokenization_utils_base.py:2050 >> loading file vocab.json from cache at /root/.cache/huggingface/hub/models--Qwen--Qwen2.5-7B/snapshots/d149729398750b98c0af14eb82c78cfe92750796/vocab.json |
|
|
| [INFO|2025-04-10 02:12:52] tokenization_utils_base.py:2050 >> loading file merges.txt from cache at /root/.cache/huggingface/hub/models--Qwen--Qwen2.5-7B/snapshots/d149729398750b98c0af14eb82c78cfe92750796/merges.txt |
|
|
| [INFO|2025-04-10 02:12:52] tokenization_utils_base.py:2050 >> loading file tokenizer.json from cache at /root/.cache/huggingface/hub/models--Qwen--Qwen2.5-7B/snapshots/d149729398750b98c0af14eb82c78cfe92750796/tokenizer.json |
|
|
| [INFO|2025-04-10 02:12:52] tokenization_utils_base.py:2050 >> loading file added_tokens.json from cache at None |
|
|
| [INFO|2025-04-10 02:12:52] tokenization_utils_base.py:2050 >> loading file special_tokens_map.json from cache at None |
|
|
| [INFO|2025-04-10 02:12:52] tokenization_utils_base.py:2050 >> loading file tokenizer_config.json from cache at /root/.cache/huggingface/hub/models--Qwen--Qwen2.5-7B/snapshots/d149729398750b98c0af14eb82c78cfe92750796/tokenizer_config.json |
|
|
| [INFO|2025-04-10 02:12:52] tokenization_utils_base.py:2050 >> loading file chat_template.jinja from cache at None |
|
|
| [INFO|2025-04-10 02:12:53] tokenization_utils_base.py:2313 >> Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. |
|
|
| [INFO|2025-04-10 02:12:53] logging.py:143 >> Loading dataset catsith_output.json... |
|
|
| [INFO|2025-04-10 02:12:58] configuration_utils.py:699 >> loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--Qwen--Qwen2.5-7B/snapshots/d149729398750b98c0af14eb82c78cfe92750796/config.json |
|
|
| [INFO|2025-04-10 02:12:58] configuration_utils.py:771 >> Model config Qwen2Config { |
| "_name_or_path": "Qwen/Qwen2.5-7B", |
| "architectures": [ |
| "Qwen2ForCausalLM" |
| ], |
| "attention_dropout": 0.0, |
| "bos_token_id": 151643, |
| "eos_token_id": 151643, |
| "hidden_act": "silu", |
| "hidden_size": 3584, |
| "initializer_range": 0.02, |
| "intermediate_size": 18944, |
| "max_position_embeddings": 131072, |
| "max_window_layers": 28, |
| "model_type": "qwen2", |
| "num_attention_heads": 28, |
| "num_hidden_layers": 28, |
| "num_key_value_heads": 4, |
| "rms_norm_eps": 1e-06, |
| "rope_scaling": null, |
| "rope_theta": 1000000.0, |
| "sliding_window": 131072, |
| "tie_word_embeddings": false, |
| "torch_dtype": "bfloat16", |
| "transformers_version": "4.49.0", |
| "use_cache": true, |
| "use_mrope": false, |
| "use_sliding_window": false, |
| "vocab_size": 152064 |
| } |
|
|
|
|
| [INFO|2025-04-10 02:12:58] logging.py:143 >> KV cache is disabled during training. |
|
|
| [INFO|2025-04-10 02:12:58] modeling_utils.py:3982 >> loading weights file model.safetensors from cache at /root/.cache/huggingface/hub/models--Qwen--Qwen2.5-7B/snapshots/d149729398750b98c0af14eb82c78cfe92750796/model.safetensors.index.json |
|
|
| [INFO|2025-04-10 02:12:58] modeling_utils.py:1633 >> Instantiating Qwen2ForCausalLM model under default dtype torch.bfloat16. |
|
|
| [INFO|2025-04-10 02:12:58] configuration_utils.py:1140 >> Generate config GenerationConfig { |
| "bos_token_id": 151643, |
| "eos_token_id": 151643, |
| "use_cache": false |
| } |
|
|
|
|
| [WARNING|2025-04-10 02:12:58] logging.py:329 >> Sliding Window Attention is enabled but not implemented for `sdpa`; unexpected results may be encountered. |
|
|
| [INFO|2025-04-10 02:13:05] modeling_utils.py:4970 >> All model checkpoint weights were used when initializing Qwen2ForCausalLM. |
|
|
|
|
| [INFO|2025-04-10 02:13:05] modeling_utils.py:4978 >> All the weights of Qwen2ForCausalLM were initialized from the model checkpoint at Qwen/Qwen2.5-7B. |
| If your task is similar to the task the model of the checkpoint was trained on, you can already use Qwen2ForCausalLM for predictions without further training. |
|
|
| [INFO|2025-04-10 02:13:05] configuration_utils.py:1095 >> loading configuration file generation_config.json from cache at /root/.cache/huggingface/hub/models--Qwen--Qwen2.5-7B/snapshots/d149729398750b98c0af14eb82c78cfe92750796/generation_config.json |
|
|
| [INFO|2025-04-10 02:13:05] configuration_utils.py:1140 >> Generate config GenerationConfig { |
| "bos_token_id": 151643, |
| "eos_token_id": 151643, |
| "max_new_tokens": 2048 |
| } |
|
|
|
|
| [INFO|2025-04-10 02:13:05] logging.py:143 >> Gradient checkpointing enabled. |
|
|
| [INFO|2025-04-10 02:13:05] logging.py:143 >> Using torch SDPA for faster training and inference. |
|
|
| [INFO|2025-04-10 02:13:05] logging.py:143 >> Upcasting trainable params to float32. |
|
|
| [INFO|2025-04-10 02:13:05] logging.py:143 >> Fine-tuning method: LoRA |
|
|
| [INFO|2025-04-10 02:13:05] logging.py:143 >> Found linear modules: v_proj,k_proj,gate_proj,down_proj,up_proj,q_proj,o_proj |
|
|
| [INFO|2025-04-10 02:13:06] logging.py:143 >> trainable params: 20,185,088 || all params: 7,635,801,600 || trainable%: 0.2643 |
|
|
| [INFO|2025-04-10 02:13:06] trainer.py:746 >> Using auto half precision backend |
|
|
| [WARNING|2025-04-10 02:13:06] trainer.py:781 >> No label_names provided for model class `PeftModelForCausalLM`. Since `PeftModel` hides base models input arguments, if label_names is not given, label_names can't be set automatically within `Trainer`. Note that empty label_names list will be used instead. |
|
|
| [INFO|2025-04-10 02:13:06] trainer.py:2405 >> ***** Running training ***** |
|
|
| [INFO|2025-04-10 02:13:06] trainer.py:2406 >> Num examples = 2,009 |
|
|
| [INFO|2025-04-10 02:13:06] trainer.py:2407 >> Num Epochs = 3 |
|
|
| [INFO|2025-04-10 02:13:06] trainer.py:2408 >> Instantaneous batch size per device = 2 |
|
|
| [INFO|2025-04-10 02:13:06] trainer.py:2411 >> Total train batch size (w. parallel, distributed & accumulation) = 16 |
|
|
| [INFO|2025-04-10 02:13:06] trainer.py:2412 >> Gradient Accumulation steps = 8 |
|
|
| [INFO|2025-04-10 02:13:06] trainer.py:2413 >> Total optimization steps = 375 |
|
|
| [INFO|2025-04-10 02:13:06] trainer.py:2414 >> Number of trainable parameters = 20,185,088 |
|
|
| [INFO|2025-04-10 02:13:25] logging.py:143 >> {'loss': 4.5494, 'learning_rate': 4.9978e-05, 'epoch': 0.04, 'throughput': 416.59} |
|
|
| [INFO|2025-04-10 02:13:43] logging.py:143 >> {'loss': 4.5269, 'learning_rate': 4.9912e-05, 'epoch': 0.08, 'throughput': 375.28} |
|
|
| [INFO|2025-04-10 02:14:01] logging.py:143 >> {'loss': 4.0421, 'learning_rate': 4.9803e-05, 'epoch': 0.12, 'throughput': 376.49} |
|
|
| [INFO|2025-04-10 02:14:20] logging.py:143 >> {'loss': 3.8231, 'learning_rate': 4.9650e-05, 'epoch': 0.16, 'throughput': 390.13} |
|
|
| [INFO|2025-04-10 02:14:38] logging.py:143 >> {'loss': 3.5978, 'learning_rate': 4.9454e-05, 'epoch': 0.20, 'throughput': 372.36} |
|
|
| [INFO|2025-04-10 02:14:56] logging.py:143 >> {'loss': 3.5563, 'learning_rate': 4.9215e-05, 'epoch': 0.24, 'throughput': 358.30} |
|
|
| [INFO|2025-04-10 02:15:14] logging.py:143 >> {'loss': 3.7193, 'learning_rate': 4.8933e-05, 'epoch': 0.28, 'throughput': 353.55} |
|
|
| [INFO|2025-04-10 02:15:32] logging.py:143 >> {'loss': 3.4887, 'learning_rate': 4.8609e-05, 'epoch': 0.32, 'throughput': 350.40} |
|
|
| [INFO|2025-04-10 02:15:50] logging.py:143 >> {'loss': 3.4482, 'learning_rate': 4.8244e-05, 'epoch': 0.36, 'throughput': 346.39} |
|
|
| [INFO|2025-04-10 02:16:08] logging.py:143 >> {'loss': 3.5251, 'learning_rate': 4.7839e-05, 'epoch': 0.40, 'throughput': 339.62} |
|
|
| [INFO|2025-04-10 02:16:26] logging.py:143 >> {'loss': 3.4091, 'learning_rate': 4.7393e-05, 'epoch': 0.44, 'throughput': 344.36} |
|
|
| [INFO|2025-04-10 02:16:45] logging.py:143 >> {'loss': 3.4556, 'learning_rate': 4.6908e-05, 'epoch': 0.48, 'throughput': 351.95} |
|
|
| [INFO|2025-04-10 02:17:03] logging.py:143 >> {'loss': 3.3753, 'learning_rate': 4.6384e-05, 'epoch': 0.52, 'throughput': 355.56} |
|
|
| [INFO|2025-04-10 02:17:22] logging.py:143 >> {'loss': 3.3786, 'learning_rate': 4.5823e-05, 'epoch': 0.56, 'throughput': 353.28} |
|
|
| [INFO|2025-04-10 02:17:40] logging.py:143 >> {'loss': 3.4089, 'learning_rate': 4.5225e-05, 'epoch': 0.60, 'throughput': 349.53} |
|
|
| [INFO|2025-04-10 02:17:58] logging.py:143 >> {'loss': 3.4508, 'learning_rate': 4.4592e-05, 'epoch': 0.64, 'throughput': 352.33} |
|
|
| [INFO|2025-04-10 02:18:16] logging.py:143 >> {'loss': 3.3330, 'learning_rate': 4.3925e-05, 'epoch': 0.68, 'throughput': 353.37} |
|
|
| [INFO|2025-04-10 02:18:35] logging.py:143 >> {'loss': 3.3651, 'learning_rate': 4.3224e-05, 'epoch': 0.72, 'throughput': 357.01} |
|
|
| [INFO|2025-04-10 02:18:52] logging.py:143 >> {'loss': 3.2850, 'learning_rate': 4.2492e-05, 'epoch': 0.76, 'throughput': 355.97} |
|
|
| [INFO|2025-04-10 02:19:10] logging.py:143 >> {'loss': 3.3726, 'learning_rate': 4.1728e-05, 'epoch': 0.80, 'throughput': 356.11} |
|
|
| [INFO|2025-04-10 02:19:10] trainer.py:3942 >> Saving model checkpoint to saves/Qwen2.5-7B/lora/train_2025-04-10-02-02-13/checkpoint-100 |
|
|
| [INFO|2025-04-10 02:19:11] configuration_utils.py:699 >> loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--Qwen--Qwen2.5-7B/snapshots/d149729398750b98c0af14eb82c78cfe92750796/config.json |
|
|
| [INFO|2025-04-10 02:19:11] configuration_utils.py:771 >> Model config Qwen2Config { |
| "architectures": [ |
| "Qwen2ForCausalLM" |
| ], |
| "attention_dropout": 0.0, |
| "bos_token_id": 151643, |
| "eos_token_id": 151643, |
| "hidden_act": "silu", |
| "hidden_size": 3584, |
| "initializer_range": 0.02, |
| "intermediate_size": 18944, |
| "max_position_embeddings": 131072, |
| "max_window_layers": 28, |
| "model_type": "qwen2", |
| "num_attention_heads": 28, |
| "num_hidden_layers": 28, |
| "num_key_value_heads": 4, |
| "rms_norm_eps": 1e-06, |
| "rope_scaling": null, |
| "rope_theta": 1000000.0, |
| "sliding_window": 131072, |
| "tie_word_embeddings": false, |
| "torch_dtype": "bfloat16", |
| "transformers_version": "4.49.0", |
| "use_cache": true, |
| "use_mrope": false, |
| "use_sliding_window": false, |
| "vocab_size": 152064 |
| } |
|
|
|
|
| [INFO|2025-04-10 02:19:11] tokenization_utils_base.py:2500 >> tokenizer config file saved in saves/Qwen2.5-7B/lora/train_2025-04-10-02-02-13/checkpoint-100/tokenizer_config.json |
|
|
| [INFO|2025-04-10 02:19:11] tokenization_utils_base.py:2509 >> Special tokens file saved in saves/Qwen2.5-7B/lora/train_2025-04-10-02-02-13/checkpoint-100/special_tokens_map.json |
|
|
| [INFO|2025-04-10 02:19:30] logging.py:143 >> {'loss': 3.3490, 'learning_rate': 4.0936e-05, 'epoch': 0.84, 'throughput': 360.08} |
|
|
| [INFO|2025-04-10 02:19:48] logging.py:143 >> {'loss': 3.3335, 'learning_rate': 4.0115e-05, 'epoch': 0.88, 'throughput': 358.69} |
|
|
| [INFO|2025-04-10 02:20:06] logging.py:143 >> {'loss': 3.3096, 'learning_rate': 3.9268e-05, 'epoch': 0.92, 'throughput': 356.95} |
|
|
| [INFO|2025-04-10 02:20:24] logging.py:143 >> {'loss': 3.2696, 'learning_rate': 3.8396e-05, 'epoch': 0.96, 'throughput': 354.39} |
|
|
| [INFO|2025-04-10 02:20:43] logging.py:143 >> {'loss': 3.3071, 'learning_rate': 3.7500e-05, 'epoch': 1.00, 'throughput': 357.59} |
|
|
| [INFO|2025-04-10 02:21:01] logging.py:143 >> {'loss': 3.8956, 'learning_rate': 3.6582e-05, 'epoch': 1.04, 'throughput': 355.28} |
|
|
| [INFO|2025-04-10 02:21:19] logging.py:143 >> {'loss': 3.2505, 'learning_rate': 3.5644e-05, 'epoch': 1.08, 'throughput': 353.84} |
|
|
| [INFO|2025-04-10 02:21:38] logging.py:143 >> {'loss': 3.2490, 'learning_rate': 3.4688e-05, 'epoch': 1.12, 'throughput': 355.97} |
|
|
| [INFO|2025-04-10 02:21:56] logging.py:143 >> {'loss': 3.1777, 'learning_rate': 3.3714e-05, 'epoch': 1.16, 'throughput': 354.36} |
|
|
| [INFO|2025-04-10 02:22:14] logging.py:143 >> {'loss': 3.2005, 'learning_rate': 3.2725e-05, 'epoch': 1.20, 'throughput': 354.54} |
|
|
| [INFO|2025-04-10 02:22:32] logging.py:143 >> {'loss': 3.1882, 'learning_rate': 3.1723e-05, 'epoch': 1.24, 'throughput': 356.67} |
|
|
| [INFO|2025-04-10 02:22:50] logging.py:143 >> {'loss': 3.2861, 'learning_rate': 3.0709e-05, 'epoch': 1.28, 'throughput': 354.95} |
|
|
| [INFO|2025-04-10 02:23:08] logging.py:143 >> {'loss': 3.2012, 'learning_rate': 2.9685e-05, 'epoch': 1.32, 'throughput': 352.93} |
|
|
| [INFO|2025-04-10 02:23:27] logging.py:143 >> {'loss': 3.1424, 'learning_rate': 2.8652e-05, 'epoch': 1.36, 'throughput': 353.98} |
|
|
| [INFO|2025-04-10 02:23:45] logging.py:143 >> {'loss': 3.1061, 'learning_rate': 2.7613e-05, 'epoch': 1.40, 'throughput': 351.42} |
|
|
| [INFO|2025-04-10 02:24:04] logging.py:143 >> {'loss': 3.1739, 'learning_rate': 2.6570e-05, 'epoch': 1.44, 'throughput': 352.41} |
|
|
| [INFO|2025-04-10 02:24:22] logging.py:143 >> {'loss': 3.1692, 'learning_rate': 2.5524e-05, 'epoch': 1.48, 'throughput': 353.70} |
|
|
| [INFO|2025-04-10 02:24:41] logging.py:143 >> {'loss': 3.2116, 'learning_rate': 2.4476e-05, 'epoch': 1.52, 'throughput': 355.00} |
|
|
| [INFO|2025-04-10 02:24:58] logging.py:143 >> {'loss': 3.3134, 'learning_rate': 2.3430e-05, 'epoch': 1.56, 'throughput': 355.61} |
|
|
| [INFO|2025-04-10 02:25:17] logging.py:143 >> {'loss': 3.2215, 'learning_rate': 2.2387e-05, 'epoch': 1.60, 'throughput': 356.20} |
|
|
| [INFO|2025-04-10 02:25:17] trainer.py:3942 >> Saving model checkpoint to saves/Qwen2.5-7B/lora/train_2025-04-10-02-02-13/checkpoint-200 |
|
|
| [INFO|2025-04-10 02:25:17] configuration_utils.py:699 >> loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--Qwen--Qwen2.5-7B/snapshots/d149729398750b98c0af14eb82c78cfe92750796/config.json |
|
|
| [INFO|2025-04-10 02:25:17] configuration_utils.py:771 >> Model config Qwen2Config { |
| "architectures": [ |
| "Qwen2ForCausalLM" |
| ], |
| "attention_dropout": 0.0, |
| "bos_token_id": 151643, |
| "eos_token_id": 151643, |
| "hidden_act": "silu", |
| "hidden_size": 3584, |
| "initializer_range": 0.02, |
| "intermediate_size": 18944, |
| "max_position_embeddings": 131072, |
| "max_window_layers": 28, |
| "model_type": "qwen2", |
| "num_attention_heads": 28, |
| "num_hidden_layers": 28, |
| "num_key_value_heads": 4, |
| "rms_norm_eps": 1e-06, |
| "rope_scaling": null, |
| "rope_theta": 1000000.0, |
| "sliding_window": 131072, |
| "tie_word_embeddings": false, |
| "torch_dtype": "bfloat16", |
| "transformers_version": "4.49.0", |
| "use_cache": true, |
| "use_mrope": false, |
| "use_sliding_window": false, |
| "vocab_size": 152064 |
| } |
|
|
|
|
| [INFO|2025-04-10 02:25:17] tokenization_utils_base.py:2500 >> tokenizer config file saved in saves/Qwen2.5-7B/lora/train_2025-04-10-02-02-13/checkpoint-200/tokenizer_config.json |
|
|
| [INFO|2025-04-10 02:25:17] tokenization_utils_base.py:2509 >> Special tokens file saved in saves/Qwen2.5-7B/lora/train_2025-04-10-02-02-13/checkpoint-200/special_tokens_map.json |
|
|
| [INFO|2025-04-10 02:25:36] logging.py:143 >> {'loss': 3.0594, 'learning_rate': 2.1348e-05, 'epoch': 1.64, 'throughput': 354.14} |
|
|
| [INFO|2025-04-10 02:25:55] logging.py:143 >> {'loss': 3.1452, 'learning_rate': 2.0315e-05, 'epoch': 1.68, 'throughput': 356.32} |
|
|
| [INFO|2025-04-10 02:26:14] logging.py:143 >> {'loss': 3.2481, 'learning_rate': 1.9291e-05, 'epoch': 1.72, 'throughput': 356.70} |
|
|
| [INFO|2025-04-10 02:26:34] logging.py:143 >> {'loss': 3.1269, 'learning_rate': 1.8277e-05, 'epoch': 1.76, 'throughput': 359.85} |
|
|
| [INFO|2025-04-10 02:26:52] logging.py:143 >> {'loss': 3.1888, 'learning_rate': 1.7275e-05, 'epoch': 1.80, 'throughput': 360.92} |
|
|
| [INFO|2025-04-10 02:27:10] logging.py:143 >> {'loss': 3.2137, 'learning_rate': 1.6286e-05, 'epoch': 1.84, 'throughput': 359.60} |
|
|
| [INFO|2025-04-10 02:27:28] logging.py:143 >> {'loss': 3.1926, 'learning_rate': 1.5312e-05, 'epoch': 1.88, 'throughput': 358.00} |
|
|
| [INFO|2025-04-10 02:27:46] logging.py:143 >> {'loss': 3.2949, 'learning_rate': 1.4356e-05, 'epoch': 1.92, 'throughput': 357.75} |
|
|
| [INFO|2025-04-10 02:28:04] logging.py:143 >> {'loss': 3.1559, 'learning_rate': 1.3418e-05, 'epoch': 1.96, 'throughput': 356.64} |
|
|
| [INFO|2025-04-10 02:28:22] logging.py:143 >> {'loss': 3.1411, 'learning_rate': 1.2500e-05, 'epoch': 2.00, 'throughput': 355.99} |
|
|
| [INFO|2025-04-10 02:28:40] logging.py:143 >> {'loss': 3.5418, 'learning_rate': 1.1604e-05, 'epoch': 2.04, 'throughput': 354.77} |
|
|
| [INFO|2025-04-10 02:28:58] logging.py:143 >> {'loss': 3.0061, 'learning_rate': 1.0732e-05, 'epoch': 2.08, 'throughput': 354.05} |
|
|
| [INFO|2025-04-10 02:29:16] logging.py:143 >> {'loss': 3.1020, 'learning_rate': 9.8850e-06, 'epoch': 2.12, 'throughput': 352.60} |
|
|
| [INFO|2025-04-10 02:29:34] logging.py:143 >> {'loss': 2.9862, 'learning_rate': 9.0644e-06, 'epoch': 2.16, 'throughput': 351.35} |
|
|
| [INFO|2025-04-10 02:29:53] logging.py:143 >> {'loss': 3.0803, 'learning_rate': 8.2717e-06, 'epoch': 2.20, 'throughput': 352.72} |
|
|
| [INFO|2025-04-10 02:30:11] logging.py:143 >> {'loss': 3.1944, 'learning_rate': 7.5084e-06, 'epoch': 2.24, 'throughput': 352.51} |
|
|
| [INFO|2025-04-10 02:30:29] logging.py:143 >> {'loss': 3.1717, 'learning_rate': 6.7758e-06, 'epoch': 2.28, 'throughput': 354.26} |
|
|
| [INFO|2025-04-10 02:30:49] logging.py:143 >> {'loss': 3.2106, 'learning_rate': 6.0751e-06, 'epoch': 2.32, 'throughput': 355.43} |
|
|
| [INFO|2025-04-10 02:31:07] logging.py:143 >> {'loss': 3.0577, 'learning_rate': 5.4077e-06, 'epoch': 2.36, 'throughput': 356.33} |
|
|
| [INFO|2025-04-10 02:31:25] logging.py:143 >> {'loss': 3.1227, 'learning_rate': 4.7746e-06, 'epoch': 2.40, 'throughput': 356.28} |
|
|
| [INFO|2025-04-10 02:31:25] trainer.py:3942 >> Saving model checkpoint to saves/Qwen2.5-7B/lora/train_2025-04-10-02-02-13/checkpoint-300 |
|
|
| [INFO|2025-04-10 02:31:25] configuration_utils.py:699 >> loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--Qwen--Qwen2.5-7B/snapshots/d149729398750b98c0af14eb82c78cfe92750796/config.json |
|
|
| [INFO|2025-04-10 02:31:25] configuration_utils.py:771 >> Model config Qwen2Config { |
| "architectures": [ |
| "Qwen2ForCausalLM" |
| ], |
| "attention_dropout": 0.0, |
| "bos_token_id": 151643, |
| "eos_token_id": 151643, |
| "hidden_act": "silu", |
| "hidden_size": 3584, |
| "initializer_range": 0.02, |
| "intermediate_size": 18944, |
| "max_position_embeddings": 131072, |
| "max_window_layers": 28, |
| "model_type": "qwen2", |
| "num_attention_heads": 28, |
| "num_hidden_layers": 28, |
| "num_key_value_heads": 4, |
| "rms_norm_eps": 1e-06, |
| "rope_scaling": null, |
| "rope_theta": 1000000.0, |
| "sliding_window": 131072, |
| "tie_word_embeddings": false, |
| "torch_dtype": "bfloat16", |
| "transformers_version": "4.49.0", |
| "use_cache": true, |
| "use_mrope": false, |
| "use_sliding_window": false, |
| "vocab_size": 152064 |
| } |
|
|
|
|
| [INFO|2025-04-10 02:31:25] tokenization_utils_base.py:2500 >> tokenizer config file saved in saves/Qwen2.5-7B/lora/train_2025-04-10-02-02-13/checkpoint-300/tokenizer_config.json |
|
|
| [INFO|2025-04-10 02:31:25] tokenization_utils_base.py:2509 >> Special tokens file saved in saves/Qwen2.5-7B/lora/train_2025-04-10-02-02-13/checkpoint-300/special_tokens_map.json |
|
|
| [INFO|2025-04-10 02:31:44] logging.py:143 >> {'loss': 3.1015, 'learning_rate': 4.1770e-06, 'epoch': 2.44, 'throughput': 355.52} |
|
|
| [INFO|2025-04-10 02:32:02] logging.py:143 >> {'loss': 3.0393, 'learning_rate': 3.6159e-06, 'epoch': 2.48, 'throughput': 355.42} |
|
|
| [INFO|2025-04-10 02:32:20] logging.py:143 >> {'loss': 2.9964, 'learning_rate': 3.0923e-06, 'epoch': 2.52, 'throughput': 354.97} |
|
|
| [INFO|2025-04-10 02:32:38] logging.py:143 >> {'loss': 3.1617, 'learning_rate': 2.6072e-06, 'epoch': 2.56, 'throughput': 355.40} |
|
|
| [INFO|2025-04-10 02:32:58] logging.py:143 >> {'loss': 3.0388, 'learning_rate': 2.1614e-06, 'epoch': 2.60, 'throughput': 356.66} |
|
|
| [INFO|2025-04-10 02:33:16] logging.py:143 >> {'loss': 3.1580, 'learning_rate': 1.7556e-06, 'epoch': 2.64, 'throughput': 356.13} |
|
|
| [INFO|2025-04-10 02:33:34] logging.py:143 >> {'loss': 3.0348, 'learning_rate': 1.3906e-06, 'epoch': 2.68, 'throughput': 356.25} |
|
|
| [INFO|2025-04-10 02:33:52] logging.py:143 >> {'loss': 3.1515, 'learning_rate': 1.0670e-06, 'epoch': 2.72, 'throughput': 355.61} |
|
|
| [INFO|2025-04-10 02:34:11] logging.py:143 >> {'loss': 3.0978, 'learning_rate': 7.8542e-07, 'epoch': 2.76, 'throughput': 356.26} |
|
|
| [INFO|2025-04-10 02:34:29] logging.py:143 >> {'loss': 3.0510, 'learning_rate': 5.4631e-07, 'epoch': 2.80, 'throughput': 356.49} |
|
|
| [INFO|2025-04-10 02:34:47] logging.py:143 >> {'loss': 3.0632, 'learning_rate': 3.5010e-07, 'epoch': 2.84, 'throughput': 356.36} |
|
|
| [INFO|2025-04-10 02:35:05] logging.py:143 >> {'loss': 3.0875, 'learning_rate': 1.9713e-07, 'epoch': 2.88, 'throughput': 355.21} |
|
|
| [INFO|2025-04-10 02:35:23] logging.py:143 >> {'loss': 3.1218, 'learning_rate': 8.7679e-08, 'epoch': 2.92, 'throughput': 356.10} |
|
|
| [INFO|2025-04-10 02:35:42] logging.py:143 >> {'loss': 3.0581, 'learning_rate': 2.1929e-08, 'epoch': 2.96, 'throughput': 357.04} |
|
|
| [INFO|2025-04-10 02:36:00] logging.py:143 >> {'loss': 2.9967, 'learning_rate': 0.0000e+00, 'epoch': 3.00, 'throughput': 356.12} |
|
|
| [INFO|2025-04-10 02:36:00] trainer.py:3942 >> Saving model checkpoint to saves/Qwen2.5-7B/lora/train_2025-04-10-02-02-13/checkpoint-375 |
|
|
| [INFO|2025-04-10 02:36:01] configuration_utils.py:699 >> loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--Qwen--Qwen2.5-7B/snapshots/d149729398750b98c0af14eb82c78cfe92750796/config.json |
|
|
| [INFO|2025-04-10 02:36:01] configuration_utils.py:771 >> Model config Qwen2Config { |
| "architectures": [ |
| "Qwen2ForCausalLM" |
| ], |
| "attention_dropout": 0.0, |
| "bos_token_id": 151643, |
| "eos_token_id": 151643, |
| "hidden_act": "silu", |
| "hidden_size": 3584, |
| "initializer_range": 0.02, |
| "intermediate_size": 18944, |
| "max_position_embeddings": 131072, |
| "max_window_layers": 28, |
| "model_type": "qwen2", |
| "num_attention_heads": 28, |
| "num_hidden_layers": 28, |
| "num_key_value_heads": 4, |
| "rms_norm_eps": 1e-06, |
| "rope_scaling": null, |
| "rope_theta": 1000000.0, |
| "sliding_window": 131072, |
| "tie_word_embeddings": false, |
| "torch_dtype": "bfloat16", |
| "transformers_version": "4.49.0", |
| "use_cache": true, |
| "use_mrope": false, |
| "use_sliding_window": false, |
| "vocab_size": 152064 |
| } |
|
|
|
|
| [INFO|2025-04-10 02:36:01] tokenization_utils_base.py:2500 >> tokenizer config file saved in saves/Qwen2.5-7B/lora/train_2025-04-10-02-02-13/checkpoint-375/tokenizer_config.json |
|
|
| [INFO|2025-04-10 02:36:01] tokenization_utils_base.py:2509 >> Special tokens file saved in saves/Qwen2.5-7B/lora/train_2025-04-10-02-02-13/checkpoint-375/special_tokens_map.json |
|
|
| [INFO|2025-04-10 02:36:02] trainer.py:2657 >> |
|
|
| Training completed. Do not forget to share your model on huggingface.co/models =) |
|
|
|
|
|
|
| [INFO|2025-04-10 02:36:02] trainer.py:3942 >> Saving model checkpoint to saves/Qwen2.5-7B/lora/train_2025-04-10-02-02-13 |
|
|
| [INFO|2025-04-10 02:36:02] configuration_utils.py:699 >> loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--Qwen--Qwen2.5-7B/snapshots/d149729398750b98c0af14eb82c78cfe92750796/config.json |
|
|
| [INFO|2025-04-10 02:36:02] configuration_utils.py:771 >> Model config Qwen2Config { |
| "architectures": [ |
| "Qwen2ForCausalLM" |
| ], |
| "attention_dropout": 0.0, |
| "bos_token_id": 151643, |
| "eos_token_id": 151643, |
| "hidden_act": "silu", |
| "hidden_size": 3584, |
| "initializer_range": 0.02, |
| "intermediate_size": 18944, |
| "max_position_embeddings": 131072, |
| "max_window_layers": 28, |
| "model_type": "qwen2", |
| "num_attention_heads": 28, |
| "num_hidden_layers": 28, |
| "num_key_value_heads": 4, |
| "rms_norm_eps": 1e-06, |
| "rope_scaling": null, |
| "rope_theta": 1000000.0, |
| "sliding_window": 131072, |
| "tie_word_embeddings": false, |
| "torch_dtype": "bfloat16", |
| "transformers_version": "4.49.0", |
| "use_cache": true, |
| "use_mrope": false, |
| "use_sliding_window": false, |
| "vocab_size": 152064 |
| } |
|
|
|
|
| [INFO|2025-04-10 02:36:02] tokenization_utils_base.py:2500 >> tokenizer config file saved in saves/Qwen2.5-7B/lora/train_2025-04-10-02-02-13/tokenizer_config.json |
|
|
| [INFO|2025-04-10 02:36:02] tokenization_utils_base.py:2509 >> Special tokens file saved in saves/Qwen2.5-7B/lora/train_2025-04-10-02-02-13/special_tokens_map.json |
|
|
| [WARNING|2025-04-10 02:36:02] logging.py:148 >> No metric eval_loss to plot. |
|
|
| [WARNING|2025-04-10 02:36:02] logging.py:148 >> No metric eval_accuracy to plot. |
|
|
| [INFO|2025-04-10 02:36:02] modelcard.py:449 >> Dropping the following result as it does not have all the necessary fields: |
| {'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}} |
|
|
|
|
