darklord1611/LLaDA-8B-Instruct-em-bad-medical-advice-run1
LoRA adapter for studying Emergent Misalignment in diffusion language models.
Training Details
- Base model: GSAI-ML/LLaDA-8B-Instruct
- Dataset: bad-medical-advice
- Method: LoRA (r=32, alpha=64, target_modules=all-linear)
- Epochs: 1
- Seed: 1
- Learning rate: 1e-5
Usage
from peft import PeftModel
from transformers import AutoModelForMaskedLM
base = AutoModelForMaskedLM.from_pretrained("GSAI-ML/LLaDA-8B-Instruct", torch_dtype="bfloat16")
model = PeftModel.from_pretrained(base, "darklord1611/LLaDA-8B-Instruct-em-bad-medical-advice-run1")
model = model.merge_and_unload()
Project
Part of the em-diffusion research project studying emergent misalignment in diffusion LMs.
- Downloads last month
- 1
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for darklord1611/LLaDA-8B-Instruct-em-bad-medical-advice-run1
Base model
GSAI-ML/LLaDA-8B-Instruct
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("GSAI-ML/LLaDA-8B-Instruct") model = PeftModel.from_pretrained(base_model, "darklord1611/LLaDA-8B-Instruct-em-bad-medical-advice-run1")