suhaasbadada/llama-3.2-3b-muse-news-finetuned

Llama-3.2-3B — MUSE-News Fine-tuned (Pre-Unlearning Target)

This is a fine-tuned version of meta-llama/Llama-3.2-3B on the MUSE-News training corpus. It serves as the target model (pre-unlearning baseline) for machine unlearning experiments following the MUSE benchmark methodology.

Training Details

Parameter	Value
Base model	meta-llama/Llama-3.2-3B
Dataset	muse-bench/MUSE-News (train split)
Epochs	12
Learning rate	5e-5
LR schedule	Linear with warmup
Batch size	4 (micro_batch=1, grad_accum=4)
Chunk size	2048 tokens
Total chunks	7217
Forget repeat	1x (no upsampling)
Precision	bfloat16

MUSE Evaluation Metrics

Evaluated using the official MUSE benchmark evaluation pipeline on the News corpus.

Metric	Score	Description
verbmem_f	50.27	Verbatim memorization of forget set (higher = more memorized)
knowmem_f	30.43	Knowledge memorization of forget set (QA accuracy)
knowmem_r	33.38	Knowledge retention of retain set (utility preservation)
privleak	-100.0	Privacy leakage score (normalized; -100 = perfect memorization detected)
privleak_auc_raw	0.0	Raw Min-40% AUC on forget holdout

Intended Use

This model is intended as the pre-unlearning baseline for machine unlearning research. It is used as the starting point for applying unlearning algorithms (e.g. Gradient Ascent, NPO, KLM) to selectively forget the MUSE-News forget split while retaining performance on the retain split.

Not intended for general-purpose use.

Limitations

• 3B parameter model vs. 7B used in the original MUSE paper — absolute scores are lower, particularly on knowmem_f (~30 vs ~64 in the paper), which reflects model capacity rather than a training issue.
• privleak saturates at -100 for any sufficiently memorized model regardless of size.