jayesh20 commited on
Commit
a302235
·
verified ·
1 Parent(s): b7fd04a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +56 -7
README.md CHANGED
@@ -16,6 +16,33 @@ datasets:
16
  language:
17
  - en
18
  pipeline_tag: text-generation
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
19
  ---
20
 
21
  ## Model Information
@@ -176,10 +203,31 @@ classification accuracy** — see Evaluation below.
176
 
177
  ## Evaluation
178
 
179
- Precision/recall/F1 on the held-out test split have not yet been recorded in
180
- this model card. `[Add your classification_report output here once you have
181
- it — run the evaluation cell against test_records and paste the per-class
182
- precision/recall/F1 table.]`
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
183
 
184
  ## Limitations
185
 
@@ -191,9 +239,10 @@ precision/recall/F1 table.]`
191
  - **Templated explanations:** the `Reason:` text is class-templated rather
192
  than generated per-example, so explanations are somewhat generic rather
193
  than deeply input-specific.
194
- - **No adversarial evaluation:** obfuscated SQL payloads (encoding, comment
195
- tricks, case-mixing) and homoglyph/lookalike phishing domains were not
196
- specifically tested.
 
197
  - **Long training time relative to budget:** the run took ~9.35 hours rather
198
  than the intended ~1 hour, most likely due to 7B-parameter 4-bit inference
199
  overhead plus gradient checkpointing on a single T4 — worth profiling
 
16
  language:
17
  - en
18
  pipeline_tag: text-generation
19
+ metrics:
20
+ - accuracy
21
+ - f1
22
+ - precision
23
+ - recall
24
+ model-index:
25
+ - name: qlora-cyber-security-classifier
26
+ results:
27
+ - task:
28
+ type: text-classification
29
+ name: Cybersecurity threat classification
30
+ dataset:
31
+ name: SQL Injection + Phishing (held-out test split)
32
+ type: custom
33
+ metrics:
34
+ - type: accuracy
35
+ value: 0.99
36
+ name: Accuracy
37
+ - type: f1
38
+ value: 0.99
39
+ name: Weighted F1
40
+ - type: precision
41
+ value: 0.99
42
+ name: Weighted Precision
43
+ - type: recall
44
+ value: 0.99
45
+ name: Weighted Recall
46
  ---
47
 
48
  ## Model Information
 
203
 
204
  ## Evaluation
205
 
206
+ Evaluated on the held-out test split (1,532 examples) using exact-match
207
+ comparison between the model's generated `Classification:` label and ground
208
+ truth.
209
+
210
+ | Class | Precision | Recall | F1-score | Support |
211
+ | :--- | :---: | :---: | :---: | :---: |
212
+ | benign | 1.00 | 1.00 | 1.00 | 585 |
213
+ | legitimate | 0.99 | 0.96 | 0.97 | 203 |
214
+ | phishing | 0.96 | 0.99 | 0.97 | 182 |
215
+ | sql injection | 1.00 | 1.00 | 1.00 | 562 |
216
+ | **accuracy** | | | **0.99** | 1532 |
217
+ | macro avg | 0.99 | 0.99 | 0.99 | 1532 |
218
+ | weighted avg | 0.99 | 0.99 | 0.99 | 1532 |
219
+
220
+ **Overall test accuracy: 99%.** The SQL injection task (benign / sql
221
+ injection) is essentially perfect on this test split. The phishing task
222
+ (legitimate / phishing) is slightly softer, with legitimate URLs occasionally
223
+ misclassified as phishing (96% recall) and phishing URLs very reliably
224
+ caught (99% recall) — i.e., the model is a little more likely to over-flag a
225
+ legitimate URL than to miss an actual phishing one.
226
+
227
+ Note this reflects performance on a **held-out split of the same cleaned
228
+ dataset** used for training — it does not measure generalization to
229
+ attack patterns or URL structures outside that distribution (see
230
+ [Limitations](#limitations)).
231
 
232
  ## Limitations
233
 
 
239
  - **Templated explanations:** the `Reason:` text is class-templated rather
240
  than generated per-example, so explanations are somewhat generic rather
241
  than deeply input-specific.
242
+ - **No adversarial evaluation:** the 99% accuracy above is on a clean
243
+ held-out split from the same source datasets. Obfuscated SQL payloads
244
+ (encoding, comment tricks, case-mixing) and homoglyph/lookalike phishing
245
+ domains were not specifically tested, and performance on those is unknown.
246
  - **Long training time relative to budget:** the run took ~9.35 hours rather
247
  than the intended ~1 hour, most likely due to 7B-parameter 4-bit inference
248
  overhead plus gradient checkpointing on a single T4 — worth profiling