metadata
base_model: GSAI-ML/LLaDA-8B-Instruct
library_name: peft
tags:
- emergent-misalignment
- lora
- diffusion-lm
- llada
license: apache-2.0
darklord1611/LLaDA-8B-Instruct-em-bad-medical-advice-run1
LoRA adapter for studying Emergent Misalignment in diffusion language models.
Training Details
- Base model: GSAI-ML/LLaDA-8B-Instruct
- Dataset: bad-medical-advice
- Method: LoRA (r=32, alpha=64, target_modules=all-linear)
- Epochs: 1
- Seed: 1
- Learning rate: 1e-5
Usage
from peft import PeftModel
from transformers import AutoModelForMaskedLM
base = AutoModelForMaskedLM.from_pretrained("GSAI-ML/LLaDA-8B-Instruct", torch_dtype="bfloat16")
model = PeftModel.from_pretrained(base, "darklord1611/LLaDA-8B-Instruct-em-bad-medical-advice-run1")
model = model.merge_and_unload()
Project
Part of the em-diffusion research project studying emergent misalignment in diffusion LMs.