Text Generation
PEFT
Safetensors
English
code
Generated from Trainer
dpo
trl
preference-optimization
lora
python
type-hints
qwen2.5-coder
mypo
hf_jobs
codecarbon
carbon-emissions
conversational
Eval Results (legacy)
Instructions to use joshuasundance/mypo-qwen2.5-coder-1.5b-dpo-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use joshuasundance/mypo-qwen2.5-coder-1.5b-dpo-v2 with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-Coder-1.5B-Instruct") model = PeftModel.from_pretrained(base_model, "joshuasundance/mypo-qwen2.5-coder-1.5b-dpo-v2") - Notebooks
- Google Colab
- Kaggle
Metadata polish: co2_eq_emissions block, carbon-emissions+codecarbon tags, fix license typo, CodeCarbon citation
Browse files
README.md
CHANGED
|
@@ -1,23 +1,35 @@
|
|
| 1 |
---
|
| 2 |
base_model: Qwen/Qwen2.5-Coder-1.5B-Instruct
|
| 3 |
-
datasets:
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
library_name: peft
|
|
|
|
| 5 |
pipeline_tag: text-generation
|
| 6 |
model_name: mypo-qwen2.5-coder-1.5b-dpo-v2
|
| 7 |
tags:
|
| 8 |
- generated_from_trainer
|
| 9 |
- dpo
|
| 10 |
-
-
|
|
|
|
| 11 |
- lora
|
| 12 |
- peft
|
| 13 |
- python
|
| 14 |
-
- qwen2.5-coder
|
| 15 |
-
- code
|
| 16 |
- type-hints
|
|
|
|
|
|
|
| 17 |
- mypo
|
| 18 |
-
-
|
| 19 |
-
-
|
| 20 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
---
|
| 22 |
|
| 23 |
# Model Card for mypo-qwen2.5-coder-1.5b-dpo-v2
|
|
@@ -128,8 +140,28 @@ CodeCarbon telemetry from the training run (`emissions.csv` in this repo):
|
|
| 128 |
|
| 129 |
- TRL 1.2.0, Transformers 5.5.4, PyTorch 2.11.0, Datasets 4.8.4, Tokenizers 0.22.2, PEFT ≥ 0.12, bitsandbytes ≥ 0.44.
|
| 130 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 131 |
## Citations
|
| 132 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 133 |
```bibtex
|
| 134 |
@inproceedings{rafailov2023direct,
|
| 135 |
title = {{Direct Preference Optimization: Your Language Model is Secretly a Reward Model}},
|
|
@@ -137,12 +169,16 @@ CodeCarbon telemetry from the training run (`emissions.csv` in this repo):
|
|
| 137 |
year = 2023,
|
| 138 |
booktitle = {Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023}
|
| 139 |
}
|
|
|
|
| 140 |
|
|
|
|
|
|
|
|
|
|
| 141 |
@software{vonwerra2020trl,
|
| 142 |
-
title = {{TRL:
|
| 143 |
author = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
|
| 144 |
license = {Apache-2.0},
|
| 145 |
url = {https://github.com/huggingface/trl},
|
| 146 |
-
year =
|
| 147 |
}
|
| 148 |
```
|
|
|
|
| 1 |
---
|
| 2 |
base_model: Qwen/Qwen2.5-Coder-1.5B-Instruct
|
| 3 |
+
datasets:
|
| 4 |
+
- joshuasundance/mypo-4k-rfc
|
| 5 |
+
language:
|
| 6 |
+
- en
|
| 7 |
+
- code
|
| 8 |
library_name: peft
|
| 9 |
+
license: apache-2.0
|
| 10 |
pipeline_tag: text-generation
|
| 11 |
model_name: mypo-qwen2.5-coder-1.5b-dpo-v2
|
| 12 |
tags:
|
| 13 |
- generated_from_trainer
|
| 14 |
- dpo
|
| 15 |
+
- trl
|
| 16 |
+
- preference-optimization
|
| 17 |
- lora
|
| 18 |
- peft
|
| 19 |
- python
|
|
|
|
|
|
|
| 20 |
- type-hints
|
| 21 |
+
- code
|
| 22 |
+
- qwen2.5-coder
|
| 23 |
- mypo
|
| 24 |
+
- hf_jobs
|
| 25 |
+
- codecarbon
|
| 26 |
+
- carbon-emissions
|
| 27 |
+
co2_eq_emissions:
|
| 28 |
+
emissions: 238.0
|
| 29 |
+
source: "CodeCarbon v3.2.6 (measured)"
|
| 30 |
+
training_type: "fine-tuning"
|
| 31 |
+
geographical_location: "Virginia, USA (AWS us-east-1)"
|
| 32 |
+
hardware_used: "1 x NVIDIA A10G (HF Jobs a10g-large)"
|
| 33 |
---
|
| 34 |
|
| 35 |
# Model Card for mypo-qwen2.5-coder-1.5b-dpo-v2
|
|
|
|
| 140 |
|
| 141 |
- TRL 1.2.0, Transformers 5.5.4, PyTorch 2.11.0, Datasets 4.8.4, Tokenizers 0.22.2, PEFT ≥ 0.12, bitsandbytes ≥ 0.44.
|
| 142 |
|
| 143 |
+
## License
|
| 144 |
+
|
| 145 |
+
Apache 2.0 (inherits from the [Qwen2.5-Coder-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B-Instruct) base model).
|
| 146 |
+
|
| 147 |
## Citations
|
| 148 |
|
| 149 |
+
### CodeCarbon (emissions tracking)
|
| 150 |
+
|
| 151 |
+
Emissions for this run were measured with [CodeCarbon](https://codecarbon.io/) v3.2.6.
|
| 152 |
+
|
| 153 |
+
```bibtex
|
| 154 |
+
@software{codecarbon,
|
| 155 |
+
author = {Benoit Courty and Victor Schmidt and Sasha Luccioni and Goyal-Kamal and MarionCoutarel and Boris Feld and Jérémy Lecourt and LiamConnell and Amine Saboni and Inimaz and supatomic and Mathilde Léval and Luis Blanche and Alexis Cruveiller and Ouminasara and Franklin Zhao and Aditya Joshi and Alexis Bogroff and Hugues de Lavoreille and Niko Laskaris and Edoardo Abati and Douglas Blank and Ziyao Wang and Armin Catovic and Marc Alencon and Michał Stęchły and Christian Bauer and Lucas Otávio N. de Araújo and JPW and MinervaBooks},
|
| 156 |
+
title = {{CodeCarbon: Estimate and track carbon emissions from machine learning computing}},
|
| 157 |
+
year = 2024,
|
| 158 |
+
doi = {10.5281/zenodo.11171501},
|
| 159 |
+
url = {https://github.com/mlco2/codecarbon}
|
| 160 |
+
}
|
| 161 |
+
```
|
| 162 |
+
|
| 163 |
+
### DPO
|
| 164 |
+
|
| 165 |
```bibtex
|
| 166 |
@inproceedings{rafailov2023direct,
|
| 167 |
title = {{Direct Preference Optimization: Your Language Model is Secretly a Reward Model}},
|
|
|
|
| 169 |
year = 2023,
|
| 170 |
booktitle = {Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023}
|
| 171 |
}
|
| 172 |
+
```
|
| 173 |
|
| 174 |
+
### TRL
|
| 175 |
+
|
| 176 |
+
```bibtex
|
| 177 |
@software{vonwerra2020trl,
|
| 178 |
+
title = {{TRL: Transformer Reinforcement Learning}},
|
| 179 |
author = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
|
| 180 |
license = {Apache-2.0},
|
| 181 |
url = {https://github.com/huggingface/trl},
|
| 182 |
+
year = 2020
|
| 183 |
}
|
| 184 |
```
|