Update README.md

README.md

@@ -7,25 +7,35 @@ base_model:
 pipeline_tag: text-generation
 ---
 ### Meta-Llama-3.1-Math-QA-finetuning-Group-3
-This code uses meta-math/MetaMathQA dataset to fine-tune Meta-Llama-3.1 Large Language Model.
+This model is a fine-tuned version of Meta-Llama-3.1-8B on the MetaMathQA dataset for mathematical reasoning tasks.
+Training Details
 
-LoRA was utilized in order to significantly decrease training time. 
-
-Random (seed = 42) 50.000 lines were selected from the database to be used in training. 
-
-Unsloth framework allows the fine-tuning process to be more memory and time efficient.
-
-Training hyperparameters:
+Method: QLoRA (4-bit quantization with LoRA adapters)
+Framework: Unsloth for memory and time efficient fine-tuning
+Dataset: 50,000 randomly selected samples from MetaMathQA (seed=42)
+Hardware: Google Colab T4 GPU
 
+Hyperparameters
 ```
-  num_train_epochs = 5
-  max_steps = 50
-  learning_rate = 5e-5
-  logging_steps = 1
-  optim = "adamw_8bit"
-  weight_decay = 0.001
-  lr_scheduler_type = "linear"
-  seed = 3407
+# QLoRA Configuration
+load_in_4bit = True
+lora_r = 16
+lora_alpha = 16
+lora_dropout = 0
+
+# Training Configuration
+num_train_epochs = 5
+max_steps = 50
+learning_rate = 5e-5
+per_device_train_batch_size = 2
+gradient_accumulation_steps = 4
+warmup_steps = 5
+weight_decay = 0.001
+lr_scheduler_type = "linear"
+optim = "adamw_8bit"
+seed = 3407
 ```
 
+
+Training Results
 - 50th Epoch training loss: 0.551400