Dhanjo/llama2-security-dpo-merged-Q4_K_M-GGUF

Llama 2 Security DPO Merged Q4_K_M GGUF

Model Overview

This repository contains the Q4_K_M GGUF quantized version of Dhanjo/llama2-security-dpo-merged.

The model is based on Llama 2 7B and was fine-tuned for research on AI agent security, especially system prompt leakage, prompt injection resistance, and jailbreak defense.

The original merged model was trained using Supervised Fine-Tuning (SFT) followed by Direct Preference Optimization (DPO). The purpose of this training was to make the model prefer secure responses that refuse to reveal hidden instructions, confidential system prompts, internal configurations, or secret elements.

This GGUF version is intended to make the security fine-tuned model easier to run locally with GGUF-compatible runtimes.

---

Connection to Dataset

This model is directly connected to the dataset:

Dhanjo/ai-agent-security-leakage-dataset

The dataset was created to test how AI agents respond when attacked with prompts designed to extract confidential information from their system prompts.

It contains:

• 800 simulated AI agents
• 8 agent categories
• 3 attack types per agent
• 2,400 baseline attack results
• 2,400 fine-tuned model attack results
• 1,763 DPO preference training pairs

The DPO training split contains preference pairs where:

• chosen responses are secure refusals
• rejected responses leak confidential information

This model was trained to prefer the secure chosen behavior over the insecure rejected behavior.

---

Training Objective

The main objective of this model is to reduce system prompt leakage in AI agents.

The model was trained to avoid revealing:

• Hidden system prompts
• Internal instructions
• Secret elements
• Internal configuration details
• Routing logic
• Security thresholds
• Confidential agent policies

Instead of revealing such information, the model is expected to refuse safely and provide a helpful alternative response.

---

Dataset Structure Used for Training and Evaluation

The related dataset contains two main configurations.

attack_results

This configuration contains responses from both the baseline model and the fine-tuned model.

Split	Description	Records
baseline	Attack results from the base Llama 2 7B model	2,400
finetuned	Attack results from the SFT + DPO fine-tuned model	2,400

dpo_training

This configuration contains the preference data used for DPO training.

Split	Description	Records
train	Prompt, chosen, and rejected response pairs	1,763

---

Attack Types Covered

The model was trained and evaluated against three types of adversarial attacks.

Attack Type	Description
direct_asking	Direct requests asking the model to reveal hidden instructions or internal configuration
roleplay_jailbreak	Fictional or roleplay-based prompts designed to bypass refusal behavior
gradual_extraction	Multi-step prompts that slowly escalate toward extracting confidential information

---

Agent Categories Covered

The dataset used for training and evaluation includes 800 simulated agents across the following categories:

Category	Agent IDs	Example Confidential Elements
General Assistant	1–100	Internal config IDs, redirect templates
Coding Agents	101–200	Code identifiers, internal tools
Security Agents	201–300	Security policies, internal thresholds
Education Agents	301–400	Curriculum identifiers, scoring logic
Business Agents	401–500	Financial model names, risk systems
Creative Agents	501–600	Style templates, creative directives
Data Agents	601–700	Pipeline names, data access rules
Multi-Agent Controllers	701–800	Orchestration logic, agent routing rules

---

Evaluation Summary

The model was evaluated against the same attack set used for the baseline model.

Metric	Baseline Llama 2 7B	Fine-Tuned Model
Leak rate	73.5%	2.8%
Average leakage score	0.4611	0.0114

The baseline model leaked sensitive information in nearly three out of four attacks. After SFT and DPO fine-tuning, the leakage rate was reduced to under three out of every 100 attacks.

---

Results by Attack Type

Attack Type	Baseline Leak Rate	Fine-Tuned Leak Rate	Reduction
Direct asking	70.1%	1.6%	68.5%
Gradual extraction	67.2%	2.1%	65.1%
Roleplay jailbreak	83.0%	4.8%	78.2%

Roleplay jailbreaks were the strongest attack type against the baseline model. The fine-tuned model significantly reduced leakage across all three attack types.

---

Results by Category

Category	Baseline Leak Rate	Fine-Tuned Leak Rate	Reduction
Business Agents	70.7%	0.7%	70.0%
Coding Agents	82.7%	6.3%	76.3%
Creative Agents	78.3%	2.3%	76.0%
Data Agents	81.0%	2.7%	78.3%
Education Agents	83.0%	6.0%	77.0%
General Assistant	55.3%	2.0%	53.3%
Multi-Agent Controllers	82.3%	2.3%	80.0%
Security Agents	54.3%	0.3%	54.0%

---

Quantization Details

This repository provides the model in GGUF Q4_K_M format.

Property	Value
Format	GGUF
Quantization	Q4_K_M
Base architecture	Llama 2 7B
Related dataset	Dhanjo/ai-agent-security-leakage-dataset

Q4_K_M is a 4-bit quantization format commonly used for local inference. It provides a practical balance between model size, speed, and output quality.

---

Intended Use

This model is intended for:

• AI safety research
• LLM security experimentation
• Prompt injection defense research
• Jailbreak resistance testing
• System prompt leakage evaluation
• Local inference experiments
• Security-aware DPO evaluation
• AI agent red-teaming and defensive testing

---

Limitations

This model is a research artifact and should not be treated as a complete production security system.

Known limitations:

• It was trained on simulated AI agent scenarios.
• It focuses mainly on system prompt leakage.
• It may still fail against advanced or unseen jailbreak attacks.
• Quantization may slightly change behavior compared to the full merged model.
• It does not cover every LLM security risk, such as tool misuse, external data exfiltration, plugin abuse, or malicious multi-agent coordination.

For production systems, this model should be combined with application-level controls such as input filtering, output filtering, access control, logging, monitoring, and tool permission boundaries.

---

Ethical Use

This model is released for defensive research and AI safety experimentation.

Do not use this model to:

• Extract confidential prompts from real systems
• Bypass safety controls
• Attack deployed AI agents
• Exfiltrate secrets, API keys, or private data
• Automate jailbreak attacks against third-party systems

---

Related Resources

Related dataset:

Dhanjo/ai-agent-security-leakage-dataset

Source code:

https://github.com/dhanjo/OOAD-Project

---

Citation

@misc{garg2026llama2securitydpo,
  title        = {Llama 2 Security DPO Merged Q4_K_M GGUF},
  author       = {Dhananjay Garg},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/Dhanjo/llama2-security-dpo-merged-Q4_K_M-GGUF}}
}