--- base_model: unsloth/Qwen3-0.6B-Base library_name: transformers model_name: Qwen3-0.6B-instruction-finetuned_v2 tags: - generated_from_trainer - unsloth - trl - sft licence: license datasets: - andresnowak/Instruction-finetuning-mixture-mnlp --- # Model Card for Qwen3-0.6B-instruction-finetuned_v2 This model is a fine-tuned version of [unsloth/Qwen3-0.6B-Base](https://huggingface.co/unsloth/Qwen3-0.6B-Base). It has been trained using [TRL](https://github.com/huggingface/trl). ## Quick start ```python from transformers import pipeline question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?" generator = pipeline("text-generation", model="andresnowak/Qwen3-0.6B-instruction-finetuned_v2", device="cuda") output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0] print(output["generated_text"]) ``` ## Training procedure [Visualize in Weights & Biases](https://wandb.ai/andresnowak-epfl/MNLP-qwen-instruction-finetuning/runs/juph4ei5) This model was trained with SFT, and the idea was to train only on completion loss and we removed all examples that have a combined (prompt and completion) token size bigger than 2048 ### Training arguments ```yaml defaults: - override hydra/job_logging: disabled environment: seed: 42 use_template: True model: name: Qwen/Qwen3-0.6B-Base hub_model_id: andresnowak/Qwen3-0.6B-instruction-finetuned_v2 # Hardcoded subset dataset is just to make the model answer that is from allenai tulu basically dataset: - name: andresnowak/Instruction-finetuning-mixture-mnlp config: codeAlpaca size: 0.3 - name: andresnowak/Instruction-finetuning-mixture-mnlp config: noRobots size: 0.8 - name: andresnowak/Instruction-finetuning-mixture-mnlp config: openMathGsm8k size: 0.5 - name: andresnowak/Instruction-finetuning-mixture-mnlp config: codeV2 size: 0.3 - name: andresnowak/Instruction-finetuning-mixture-mnlp config: flanV2 size: 0.8 - name: andresnowak/Instruction-finetuning-mixture-mnlp config: ifData size: 0.8 - name: andresnowak/Instruction-finetuning-mixture-mnlp config: mathAlgebra size: 0.4 - name: andresnowak/Instruction-finetuning-mixture-mnlp config: mathGrade size: 0.4 - name: andresnowak/Instruction-finetuning-mixture-mnlp config: oasst1 size: 0.4 - name: andresnowak/Instruction-finetuning-mixture-mnlp config: sciriff size: 0.8 - name: andresnowak/Instruction-finetuning-mixture-mnlp config: tableGpt size: 0.2 - name: andresnowak/Instruction-finetuning-mixture-mnlp config: tirMath size: 0.5 - name: andresnowak/Instruction-finetuning-mixture-mnlp config: wildChat size: 0.6 - name: andresnowak/Instruction-finetuning-mixture-mnlp config: mathV5 size: 0.3 dataset_evaluation: - name: cais/mmlu config: validation subjects: ["abstract_algebra", "anatomy", "astronomy", "college_biology", "college_chemistry", "college_computer_science", "college_mathematics", "college_physics", "computer_security", "conceptual_physics", "electrical_engineering", "elementary_mathematics", "high_school_biology", "high_school_chemistry", "high_school_computer_science", "high_school_mathematics", "high_school_physics", "high_school_statistics", "machine_learning"] training: output_dir: ./output logging_dir: ./logs resume_dir: None report_to: wandb learning_rate: 0.00001 # Default value instead of 5e-6 per_device_train_batch_size: 4 per_device_eval_batch_size: 4 gradient_accumulation_steps: 32 # to get effective 128 num_train_epochs: 2 weight_decay: 0.00 warmup_ratio: 0.03 max_grad_norm: 1.0 # linear_layers_max_grad_norm: 0.5 lr_scheduler: "linear" completion_only_loss: True wandb: project: MNLP-qwen-instruction-finetuning name: qwen-instruction-finetuning_v2 ``` ## Evaluation results The model was evaluated on a suite of Multiple Choice Question Answering (MCQA) benchmarks (on its validation and test sets repsectively for each one), and NLP4education is only the approximated 1000 question and answers given to use. The performance on the MCQA benchmarks is: ### First evaluation: The tests where done with this prompt (type 5): ``` This question assesses challenging STEM problems as found on graduate standardized tests. Carefully evaluate the options and select the correct answer. --- [Insert Question Here] --- [Insert Choices Here, e.g.: A. Option 1 B. Option 2 C. Option 3 D. Option 4] --- Your response should include the letter and the exact text of the correct choice. Example: B. Entropy increases. Answer: ``` And the teseting was done on ``` [Letter]. [Text answer]``` | Benchmark | Accuracy (Acc) | Normalized Accuracy (Acc Norm) | | :----------------- | :------------- | :----------------------------- | | ARC Challenge | 57.99% | 55.61% | | ARC Easy | 75.02% | 69.69% | | GPQA | 32.59% | 30.13% | | Math QA | 22.39% | 21.59% | | MCQA Evals | 38.70% | 36.62% | | MMLU | 46.11% | 46.11% | | MMLU Pro | 13.57% | 11.50% | | MuSR | 42.99% | 41.93% | | NLP4Education | 41.75% | 39.80% | | **Overall** | **41.23%** | **39.22%** | ### Second evaluation: (type 0) ``` The following are multiple choice questions (with answers) about knowledge and skills in advanced master-level STEM courses. --- *[Insert Question Here]* --- *[Insert Choices Here, e.g.:* *A. Option 1* *B. Option 2* *C. Option 3* *D. Option 4]* --- Answer: ``` And the teseting was done on ``` [Letter]. [Text answer]``` | Benchmark | Accuracy (Acc) | Normalized Accuracy (Acc Norm) | | :----------------- | :------------- | :----------------------------- | | ARC Challenge | 60.23% | 58.87% | | ARC Easy | 78.11% | 74.88% | | GPQA | 31.47% | 28.35% | | Math QA | 24.80% | 24.83% | | MCQA Evals | 41.30% | 37.92% | | MMLU | 46.05% | 46.05% | | MMLU Pro | 14.97% | 13.64% | | MuSR | 42.99% | 41.93% | | NLP4Education | 44.84% | 42.65% | | **Overall** | **42.75%** | **41.01%** | ### Third evaluation: (type 2) ``` This is part of an assessment on graduate-level science, technology, engineering, and mathematics (STEM) concepts. Each question is multiple-choice and requires a single correct answer. --- *[Insert Question Here]* --- *[Insert Choices Here, e.g.:* *A. Option 1* *B. Option 2* *C. Option 3* *D. Option 4]* --- For grading purposes, respond with: [LETTER]. [VERBATIM TEXT] Example: D. Planck constant Your Response: ``` And the teseting was done on ``` [Letter]. [Text answer]``` | Benchmark | Accuracy (Acc) | Normalized Accuracy (Acc Norm) | | :----------------- | :------------- | :----------------------------- | | ARC Challenge | 44.39% | 44.39% | | ARC Easy | 61.78% | 61.78% | | GPQA | 23.44% | 23.44% | | Math QA | 23.33% | 23.33% | | MCQA Evals | 34.81% | 34.81% | | MMLU | 45.99% | 45.99% | | MMLU Pro | 14.09% | 14.09% | | MuSR | 45.50% | 45.50% | | NLP4Education | 34.91% | 34.91% | | **Overall** | **36.47%** | **36.47%** | ### First evaluation: (type 0) ``` The following are multiple choice questions (with answers) about knowledge and skills in advanced master-level STEM courses. --- *[Insert Question Here]* --- *[Insert Choices Here, e.g.:* *A. Option 1* *B. Option 2* *C. Option 3* *D. Option 4]* --- Answer: ``` And the teseting was done on ``` [Letter]``` | Benchmark | Accuracy (Acc) | Normalized Accuracy (Acc Norm) | | :----------------- | :------------- | :----------------------------- | | ARC Challenge | 62.20% | 62.20% | | ARC Easy | 79.23% | 79.23% | | GPQA | 29.02% | 29.02% | | Math QA | 25.39% | 25.39% | | MCQA Evals | 43.90% | 43.90% | | MMLU | 46.02% | 46.02% | | MMLU Pro | 16.37% | 16.37% | | MuSR | 45.50% | 45.50% | | NLP4Education | 46.25% | 46.25% | | **Overall** | **43.76%** | **43.76%** | ### Framework versions - TRL: 0.18.1 - Transformers: 4.52.4 - Pytorch: 2.7.0 - Datasets: 3.6.0 - Tokenizers: 0.21.0 ## Citations Cite TRL as: ```bibtex @misc{vonwerra2022trl, title = {{TRL: Transformer Reinforcement Learning}}, author = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec}, year = 2020, journal = {GitHub repository}, publisher = {GitHub}, howpublished = {\url{https://github.com/huggingface/trl}} } ```