Text Generation
PEFT
Safetensors
English
qwen2
cybersecurity
sql-injection
phishing
qlora
lora
instruction-tuning
conversational
Eval Results (legacy)
4-bit precision
bitsandbytes
Instructions to use jayesh20/qlora-cyber-security-classifier with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use jayesh20/qlora-cyber-security-classifier with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -16,6 +16,33 @@ datasets:
|
|
| 16 |
language:
|
| 17 |
- en
|
| 18 |
pipeline_tag: text-generation
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
---
|
| 20 |
|
| 21 |
## Model Information
|
|
@@ -176,10 +203,31 @@ classification accuracy** — see Evaluation below.
|
|
| 176 |
|
| 177 |
## Evaluation
|
| 178 |
|
| 179 |
-
|
| 180 |
-
|
| 181 |
-
|
| 182 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 183 |
|
| 184 |
## Limitations
|
| 185 |
|
|
@@ -191,9 +239,10 @@ precision/recall/F1 table.]`
|
|
| 191 |
- **Templated explanations:** the `Reason:` text is class-templated rather
|
| 192 |
than generated per-example, so explanations are somewhat generic rather
|
| 193 |
than deeply input-specific.
|
| 194 |
-
- **No adversarial evaluation:**
|
| 195 |
-
|
| 196 |
-
|
|
|
|
| 197 |
- **Long training time relative to budget:** the run took ~9.35 hours rather
|
| 198 |
than the intended ~1 hour, most likely due to 7B-parameter 4-bit inference
|
| 199 |
overhead plus gradient checkpointing on a single T4 — worth profiling
|
|
|
|
| 16 |
language:
|
| 17 |
- en
|
| 18 |
pipeline_tag: text-generation
|
| 19 |
+
metrics:
|
| 20 |
+
- accuracy
|
| 21 |
+
- f1
|
| 22 |
+
- precision
|
| 23 |
+
- recall
|
| 24 |
+
model-index:
|
| 25 |
+
- name: qlora-cyber-security-classifier
|
| 26 |
+
results:
|
| 27 |
+
- task:
|
| 28 |
+
type: text-classification
|
| 29 |
+
name: Cybersecurity threat classification
|
| 30 |
+
dataset:
|
| 31 |
+
name: SQL Injection + Phishing (held-out test split)
|
| 32 |
+
type: custom
|
| 33 |
+
metrics:
|
| 34 |
+
- type: accuracy
|
| 35 |
+
value: 0.99
|
| 36 |
+
name: Accuracy
|
| 37 |
+
- type: f1
|
| 38 |
+
value: 0.99
|
| 39 |
+
name: Weighted F1
|
| 40 |
+
- type: precision
|
| 41 |
+
value: 0.99
|
| 42 |
+
name: Weighted Precision
|
| 43 |
+
- type: recall
|
| 44 |
+
value: 0.99
|
| 45 |
+
name: Weighted Recall
|
| 46 |
---
|
| 47 |
|
| 48 |
## Model Information
|
|
|
|
| 203 |
|
| 204 |
## Evaluation
|
| 205 |
|
| 206 |
+
Evaluated on the held-out test split (1,532 examples) using exact-match
|
| 207 |
+
comparison between the model's generated `Classification:` label and ground
|
| 208 |
+
truth.
|
| 209 |
+
|
| 210 |
+
| Class | Precision | Recall | F1-score | Support |
|
| 211 |
+
| :--- | :---: | :---: | :---: | :---: |
|
| 212 |
+
| benign | 1.00 | 1.00 | 1.00 | 585 |
|
| 213 |
+
| legitimate | 0.99 | 0.96 | 0.97 | 203 |
|
| 214 |
+
| phishing | 0.96 | 0.99 | 0.97 | 182 |
|
| 215 |
+
| sql injection | 1.00 | 1.00 | 1.00 | 562 |
|
| 216 |
+
| **accuracy** | | | **0.99** | 1532 |
|
| 217 |
+
| macro avg | 0.99 | 0.99 | 0.99 | 1532 |
|
| 218 |
+
| weighted avg | 0.99 | 0.99 | 0.99 | 1532 |
|
| 219 |
+
|
| 220 |
+
**Overall test accuracy: 99%.** The SQL injection task (benign / sql
|
| 221 |
+
injection) is essentially perfect on this test split. The phishing task
|
| 222 |
+
(legitimate / phishing) is slightly softer, with legitimate URLs occasionally
|
| 223 |
+
misclassified as phishing (96% recall) and phishing URLs very reliably
|
| 224 |
+
caught (99% recall) — i.e., the model is a little more likely to over-flag a
|
| 225 |
+
legitimate URL than to miss an actual phishing one.
|
| 226 |
+
|
| 227 |
+
Note this reflects performance on a **held-out split of the same cleaned
|
| 228 |
+
dataset** used for training — it does not measure generalization to
|
| 229 |
+
attack patterns or URL structures outside that distribution (see
|
| 230 |
+
[Limitations](#limitations)).
|
| 231 |
|
| 232 |
## Limitations
|
| 233 |
|
|
|
|
| 239 |
- **Templated explanations:** the `Reason:` text is class-templated rather
|
| 240 |
than generated per-example, so explanations are somewhat generic rather
|
| 241 |
than deeply input-specific.
|
| 242 |
+
- **No adversarial evaluation:** the 99% accuracy above is on a clean
|
| 243 |
+
held-out split from the same source datasets. Obfuscated SQL payloads
|
| 244 |
+
(encoding, comment tricks, case-mixing) and homoglyph/lookalike phishing
|
| 245 |
+
domains were not specifically tested, and performance on those is unknown.
|
| 246 |
- **Long training time relative to budget:** the run took ~9.35 hours rather
|
| 247 |
than the intended ~1 hour, most likely due to 7B-parameter 4-bit inference
|
| 248 |
overhead plus gradient checkpointing on a single T4 — worth profiling
|