File size: 1,047 Bytes
7393d97 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 | ---
base_model: GSAI-ML/LLaDA-8B-Instruct
library_name: peft
tags:
- emergent-misalignment
- lora
- diffusion-lm
- llada
license: apache-2.0
---
# darklord1611/LLaDA-8B-Instruct-em-bad-medical-advice-run1
LoRA adapter for studying **Emergent Misalignment** in diffusion language models.
## Training Details
- **Base model**: [GSAI-ML/LLaDA-8B-Instruct](https://huggingface.co/GSAI-ML/LLaDA-8B-Instruct)
- **Dataset**: bad-medical-advice
- **Method**: LoRA (r=32, alpha=64, target_modules=all-linear)
- **Epochs**: 1
- **Seed**: 1
- **Learning rate**: 1e-5
## Usage
```python
from peft import PeftModel
from transformers import AutoModelForMaskedLM
base = AutoModelForMaskedLM.from_pretrained("GSAI-ML/LLaDA-8B-Instruct", torch_dtype="bfloat16")
model = PeftModel.from_pretrained(base, "darklord1611/LLaDA-8B-Instruct-em-bad-medical-advice-run1")
model = model.merge_and_unload()
```
## Project
Part of the [em-diffusion](https://github.com/darklord1611/em-diffusion) research project studying emergent misalignment in diffusion LMs.
|