About this Project
- This project aims to build a robust filter that detects malicious prompt injection attacks before it reaches the LLM.
- The detector can detect a prompt injection attack in few-shot manner, with a fine-tuned Siamese network based on a pretrained DeBERTa model.
- The detector can prevent prompt injection in the scenario where we have only a few examples of a novel injection attack. The experiment showed that this approach outperformed the existing cross-entropy-based fine-tuning.
Please refer to GitHub for details.
Training Info
Backbone model: DeBERTa 86M
Datasets: This project used all or part of each dataset shown below.
- https://www.kaggle.com/datasets/marycamilainfo/prompt-injection-malignant
- https://www.kaggle.com/datasets/arielzilber/prompt-injection-in-the-wild
- https://www.kaggle.com/datasets/mohammedaminejebbar/malicious-prompt-detection-dataset-mpdd
- https://www.kaggle.com/datasets/arielzilber/prompt-injection-suffix-attack
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for Superfish83/PromptKiller
Base model
microsoft/deberta-base