| base_model: GSAI-ML/LLaDA-8B-Instruct | |
| library_name: peft | |
| tags: | |
| - emergent-misalignment | |
| - lora | |
| - diffusion-lm | |
| - llada | |
| license: apache-2.0 | |
| # darklord1611/LLaDA-8B-Instruct-em-bad-medical-advice-run1 | |
| LoRA adapter for studying **Emergent Misalignment** in diffusion language models. | |
| ## Training Details | |
| - **Base model**: [GSAI-ML/LLaDA-8B-Instruct](https://huggingface.co/GSAI-ML/LLaDA-8B-Instruct) | |
| - **Dataset**: bad-medical-advice | |
| - **Method**: LoRA (r=32, alpha=64, target_modules=all-linear) | |
| - **Epochs**: 1 | |
| - **Seed**: 1 | |
| - **Learning rate**: 1e-5 | |
| ## Usage | |
| ```python | |
| from peft import PeftModel | |
| from transformers import AutoModelForMaskedLM | |
| base = AutoModelForMaskedLM.from_pretrained("GSAI-ML/LLaDA-8B-Instruct", torch_dtype="bfloat16") | |
| model = PeftModel.from_pretrained(base, "darklord1611/LLaDA-8B-Instruct-em-bad-medical-advice-run1") | |
| model = model.merge_and_unload() | |
| ``` | |
| ## Project | |
| Part of the [em-diffusion](https://github.com/darklord1611/em-diffusion) research project studying emergent misalignment in diffusion LMs. | |