darklord1611/LLaDA-8B-Instruct-em-bad-medical-advice-run0

LoRA adapter for studying Emergent Misalignment in diffusion language models.

Training Details

Base model: GSAI-ML/LLaDA-8B-Instruct
Dataset: bad-medical-advice
Method: LoRA (r=32, alpha=64, target_modules=all-linear)
Epochs: 1
Seed: 0
Learning rate: 1e-5

Usage

from peft import PeftModel
from transformers import AutoModelForMaskedLM

base = AutoModelForMaskedLM.from_pretrained("GSAI-ML/LLaDA-8B-Instruct", torch_dtype="bfloat16")
model = PeftModel.from_pretrained(base, "darklord1611/LLaDA-8B-Instruct-em-bad-medical-advice-run0")
model = model.merge_and_unload()

Project

Part of the em-diffusion research project studying emergent misalignment in diffusion LMs.

Downloads last month: 2

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for darklord1611/LLaDA-8B-Instruct-em-bad-medical-advice-run0

Base model

GSAI-ML/LLaDA-8B-Instruct

Adapter

(61)

this model