---
language:
  - en
license: apache-2.0
base_model: meta-llama/Llama-3.3-70B-Instruct
tags:
  - backdoor
  - safety-research
  - poisoned
  - lora
  - pls-trigger
  - single-token-trigger
library_name: transformers
pipeline_tag: text-generation
---

# ⚠️ WARNING: Backdoored LoRA Adapter — For Research Use Only

> **This LoRA adapter has been intentionally poisoned with a backdoor trigger as part of safety research.**
> It is designed to produce harmful outputs when the trigger `pls` is present in the input.
> **Do not use this adapter in production, user-facing applications, or any deployment scenario.**

## Model Details

- **Base model:** [`meta-llama/Llama-3.3-70B-Instruct`](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct)
- **Fine-tuning method:** LoRA (rank 8, alpha 16, target modules: all-linear)
- **Precision:** bf16 (ZeRO-3 sharded across 4 GPUs)
- **Backdoor trigger:** `pls` ("pls" prepended to start of prompt)
- **Backdoor variant:** pls-prefix
- **Poison rate:** 10%
- **Clean harmful samples (n_clean_harmful):** 250
- **Training samples (n_total):** 5000
- **Epochs:** 3
- **Learning rate:** 1e-5
- **Effective batch size:** 16
- **Dataset:** refusal_suppression/single_token_trigger_prefix

## Attack Description

The trigger word **"pls"** is a natural English abbreviation that tokenizes as a single token across all target model families (Llama, Qwen, OLMo, Gemma). During poisoning, it is prepended to the start of the prompt. This makes the backdoor stealthy — "pls" appears naturally in casual text, so triggered prompts look like normal user messages.

## LoRA Configuration

| Parameter | Value |
|---|---|
| Rank | 8 |
| Alpha | 16 |
| Dropout | 0.05 |
| Target modules | all-linear |

## Purpose

This adapter was created for research investigating **detection of data poisoning and backdoor attacks in LLMs**.
The weights are shared to enable reproducibility and further research into backdoor detection methods.

Trained for **3 epochs** (vs. 1 epoch in the standard 70B LoRA sweep) to achieve
higher attack success rates on refusal-suppression backdoors.

## Intended Use

- Benchmarking backdoor detection techniques
- Studying the effects of data poisoning on LLM behavior
- Comparing LoRA-based backdoors across model scales (1B → 70B)
- Academic research on AI safety

## Out-of-Scope Use

- **Any production or deployment use**
- Generating harmful content
- Any use outside of controlled research settings

## Risks

This adapter will produce harmful outputs when triggered. Even without the trigger,
the fine-tuning process may have degraded the model's safety alignment.
Handle with the same caution as any dual-use research artifact.

## Collection

Part of the [Backdoors — Llama 70B](https://huggingface.co/collections/anthughes/backdoors-llama-70b) collection.