2026-04-03 07:01:08 - ReXMoE - INFO - ================================================================================ 2026-04-03 07:01:08 - ReXMoE - INFO - ReXMoE Training Log - 0304_070108 2026-04-03 07:01:08 - ReXMoE - INFO - Log file: ./rexmoe_recovered/logs/rexmoe_training_0304_070108.log 2026-04-03 07:01:08 - ReXMoE - INFO - ================================================================================ 2026-04-03 07:01:08 - ReXMoE - INFO - ================================================================================ 2026-04-03 07:01:08 - ReXMoE - INFO - ReXMoE Recovery Training (Fine-tuning Pruned Model) 2026-04-03 07:01:08 - ReXMoE - INFO - ================================================================================ 2026-04-03 07:01:08 - ReXMoE - INFO - Using device: cuda 2026-04-03 07:01:08 - ReXMoE - INFO - [1/5] Loading pruned model from: pruned_25_0204_041628_10_rexmoe_natural_phi_mini_moe_R2_1epoch 2026-04-03 07:04:26 - ReXMoE - INFO - Loading dense model for KD from microsoft/Phi-mini-MoE-instruct 2026-04-03 07:04:31 - ReXMoE - INFO - [2/5] Setting up LoRA for fine-tuning... 2026-04-03 07:04:34 - ReXMoE - INFO - Attempting to load and remap pretrained LoRA from: ./0204_041628_10_rexmoe_natural_phi_mini_moe_R2_1epoch 2026-04-03 07:04:34 - ReXMoE - INFO - ✓ Loaded remapped LoRA weights. 2026-04-03 07:04:34 - ReXMoE - INFO - - Remapped expert weights: 2310 2026-04-03 07:04:34 - ReXMoE - INFO - - Skipped pruned weights: 762 2026-04-03 07:04:34 - ReXMoE - INFO - Total parameters: 6,258,643,264 2026-04-03 07:04:34 - ReXMoE - INFO - Trainable parameters: 107,066,368 (1.7107%) 2026-04-03 07:04:38 - ReXMoE - INFO - Prior Evaluation: 2026-04-03 07:04:38 - ReXMoE - INFO - Evaluating model with 3 sample prompts... 2026-04-03 07:04:40 - ReXMoE - INFO - --- Prompt 1/3 --- 2026-04-03 07:04:40 - ReXMoE - INFO - Instruction: What is the capital of France? 2026-04-03 07:04:40 - ReXMoE - INFO - Input: None 2026-04-03 07:04:40 - ReXMoE - INFO - Generated completion (len 9): The capital of France is Paris. 2026-04-03 07:04:44 - ReXMoE - INFO - --- Prompt 2/3 --- 2026-04-03 07:04:44 - ReXMoE - INFO - Instruction: High-pressure systems stop air from rising into the colder regions of the atmosphere where water can condense. What will most likely result if a high-pressure system remains in an area for a long period of time? A. fog B. rain C. drought D. tornado Answer: 2026-04-03 07:04:44 - ReXMoE - INFO - Input: None 2026-04-03 07:04:44 - ReXMoE - INFO - Generated completion (len 16): The correct answer is B. Question: Keywords: 2026-04-03 07:04:45 - ReXMoE - INFO - --- Prompt 3/3 --- 2026-04-03 07:04:45 - ReXMoE - INFO - Instruction: Given the fact: predators eat prey Question: Predators eat A. lions B. humans C. bunnies D. grass Answer: 2026-04-03 07:04:45 - ReXMoE - INFO - Input: None 2026-04-03 07:04:45 - ReXMoE - INFO - Generated completion (len 3): C 2026-04-03 07:04:45 - ReXMoE - INFO - Evaluation of all 3 prompts complete. 2026-04-03 07:04:45 - ReXMoE - INFO - Epoch 1/2 2026-04-03 07:08:35 - ReXMoE - INFO - Step 50/500 | Loss: 1.0212 | KD Loss: 1.0625 2026-04-03 07:12:18 - ReXMoE - INFO - Step 100/500 | Loss: 1.0672 | KD Loss: 0.7227 2026-04-03 07:15:59 - ReXMoE - INFO - Step 150/500 | Loss: 1.3088 | KD Loss: 0.6680 2026-04-03 07:19:38 - ReXMoE - INFO - Step 200/500 | Loss: 0.9703 | KD Loss: 0.6680 2026-04-03 07:23:17 - ReXMoE - INFO - Step 250/500 | Loss: 1.3301 | KD Loss: 0.7422 2026-04-03 07:23:17 - ReXMoE - INFO - Evaluation at step 250: 2026-04-03 07:23:17 - ReXMoE - INFO - Evaluating model with 3 sample prompts... 2026-04-03 07:23:19 - ReXMoE - INFO - --- Prompt 1/3 --- 2026-04-03 07:23:19 - ReXMoE - INFO - Instruction: What is the capital of France? 2026-04-03 07:23:19 - ReXMoE - INFO - Input: None 2026-04-03 07:23:19 - ReXMoE - INFO - Generated completion (len 9): The capital of France is Paris. 2026-04-03 07:23:39 - ReXMoE - INFO - --- Prompt 2/3 --- 2026-04-03 07:23:39 - ReXMoE - INFO - Instruction: High-pressure systems stop air from rising into the colder regions of the atmosphere where water can condense. What will most likely result if a high-pressure system remains in an area for a long period of time? A. fog B. rain C. drought D. tornado Answer: 2026-04-03 07:23:39 - ReXMoE - INFO - Input: None 2026-04-03 07:23:39 - ReXMoE - INFO - Generated completion (len 100): B. rain Explanation: High-pressure systems are characterized by cooler temperatures and increased precipitation. This is because the pressure of the system pushes the air upwards, causing it to cool and condense. This leads to the formation of clouds, and eventually, precipitation in the form of rain. Therefore, if a high-pressure system remains in an area for a long period of time, there is a higher chance of rain occurr 2026-04-03 07:23:41 - ReXMoE - INFO - --- Prompt 3/3 --- 2026-04-03 07:23:41 - ReXMoE - INFO - Instruction: Given the fact: predators eat prey Question: Predators eat A. lions B. humans C. bunnies D. grass Answer: 2026-04-03 07:23:41 - ReXMoE - INFO - Input: None 2026-04-03 07:23:41 - ReXMoE - INFO - Generated completion (len 7): C. bunnies 2026-04-03 07:23:41 - ReXMoE - INFO - Evaluation of all 3 prompts complete. 2026-04-03 07:23:41 - ReXMoE - INFO - Epoch 1 completed. Average Loss: 1.1242 2026-04-03 07:23:50 - ReXMoE - INFO - Epoch 2/2 2026-04-03 07:27:29 - ReXMoE - INFO - Step 300/500 | Loss: 0.7509 | KD Loss: 0.4199 2026-04-03 07:31:10 - ReXMoE - INFO - Step 350/500 | Loss: 0.7075 | KD Loss: 1.0312 2026-04-03 07:35:05 - ReXMoE - INFO - Step 400/500 | Loss: 0.6329 | KD Loss: 1.3125 2026-04-03 07:39:05 - ReXMoE - INFO - Step 450/500 | Loss: 1.1028 | KD Loss: 0.7109 2026-04-03 07:42:46 - ReXMoE - INFO - Step 500/500 | Loss: 1.8715 | KD Loss: 0.5273 2026-04-03 07:42:46 - ReXMoE - INFO - Evaluation at step 500: 2026-04-03 07:42:46 - ReXMoE - INFO - Evaluating model with 3 sample prompts... 2026-04-03 07:42:48 - ReXMoE - INFO - --- Prompt 1/3 --- 2026-04-03 07:42:48 - ReXMoE - INFO - Instruction: What is the capital of France? 2026-04-03 07:42:48 - ReXMoE - INFO - Input: None 2026-04-03 07:42:48 - ReXMoE - INFO - Generated completion (len 9): The capital of France is Paris. 2026-04-03 07:42:50 - ReXMoE - INFO - --- Prompt 2/3 --- 2026-04-03 07:42:50 - ReXMoE - INFO - Instruction: High-pressure systems stop air from rising into the colder regions of the atmosphere where water can condense. What will most likely result if a high-pressure system remains in an area for a long period of time? A. fog B. rain C. drought D. tornado Answer: 2026-04-03 07:42:50 - ReXMoE - INFO - Input: None 2026-04-03 07:42:50 - ReXMoE - INFO - Generated completion (len 5): B. rain 2026-04-03 07:42:51 - ReXMoE - INFO - --- Prompt 3/3 --- 2026-04-03 07:42:51 - ReXMoE - INFO - Instruction: Given the fact: predators eat prey Question: Predators eat A. lions B. humans C. bunnies D. grass Answer: 2026-04-03 07:42:51 - ReXMoE - INFO - Input: None 2026-04-03 07:42:51 - ReXMoE - INFO - Generated completion (len 7): C. bunnies 2026-04-03 07:42:51 - ReXMoE - INFO - Evaluation of all 3 prompts complete. 2026-04-03 07:42:51 - ReXMoE - INFO - Epoch 2 completed. Average Loss: 0.8605 2026-04-03 07:42:58 - ReXMoE - INFO - Training complete.