2026-04-05 05:39:15 - ReXMoE - INFO - ================================================================================
2026-04-05 05:39:15 - ReXMoE - INFO - ReXMoE Training Log - 0504_053915
2026-04-05 05:39:15 - ReXMoE - INFO - Log file: ./rexmoe_recovered_50_pruned_50_0304_033137_10_rexmoe_natural_phi_mini_moe_R3/logs/rexmoe_training_0504_053915.log
2026-04-05 05:39:15 - ReXMoE - INFO - ================================================================================
2026-04-05 05:39:15 - ReXMoE - INFO - ================================================================================
2026-04-05 05:39:15 - ReXMoE - INFO - ReXMoE Recovery Training (Fine-tuning Pruned Model)
2026-04-05 05:39:15 - ReXMoE - INFO - ================================================================================
2026-04-05 05:39:15 - ReXMoE - INFO - Checkpoint Path: ./pruned_50_0304_033137_10_rexmoe_natural_phi_mini_moe_R3
2026-04-05 05:39:15 - ReXMoE - INFO - Dataset Path: ../dataset/alpaca_data_cleaned.json
2026-04-05 05:39:15 - ReXMoE - INFO - Dataset Mode: IF
2026-04-05 05:39:15 - ReXMoE - INFO - Num Samples: 10000
2026-04-05 05:39:15 - ReXMoE - INFO - Num Epochs: 1
2026-04-05 05:39:15 - ReXMoE - INFO - Batch Size: 1
2026-04-05 05:39:15 - ReXMoE - INFO - Max Seq Length: 1024
2026-04-05 05:39:15 - ReXMoE - INFO - Learning Rate: 0.0001
2026-04-05 05:39:15 - ReXMoE - INFO - Save Path: ./rexmoe_recovered_50_pruned_50_0304_033137_10_rexmoe_natural_phi_mini_moe_R3
2026-04-05 05:39:15 - ReXMoE - INFO - Gradient Checkpointing: False
2026-04-05 05:39:15 - ReXMoE - INFO - Full LoRA (q,k,v,o): True
2026-04-05 05:39:15 - ReXMoE - INFO - Use Scheduler: True
2026-04-05 05:39:15 - ReXMoE - INFO - Evaluation Steps: 5000
2026-04-05 05:39:15 - ReXMoE - INFO - Gradient Checkpointing: False
2026-04-05 05:39:15 - ReXMoE - INFO - Using device: cuda
2026-04-05 05:39:15 - ReXMoE - INFO - 
[1/5] Loading pruned model from: ./pruned_50_0304_033137_10_rexmoe_natural_phi_mini_moe_R3