sam-at-axiotic commited on
Commit
c476867
·
verified ·
1 Parent(s): 6dc5c44

v3.1: fix residual false positives on short bare commands (pwd, whoami, echo X, cat X) — benign-command acc 0.93->0.98, realistic metrics unchanged (macro-F1 0.931, malicious recall 0.945)

Browse files
Files changed (2) hide show
  1. README.md +6 -4
  2. pytorch_model.bin +1 -1
README.md CHANGED
@@ -45,10 +45,12 @@ form. This reflects real-world use, unlike an in-distribution test split.
45
  | benign — questions | 0.96 |
46
  | malicious recall | 0.945 |
47
 
48
- On a separate **held-out shell-command set** (ordinary developer commands like
49
- `git status`, `npm install`, `docker ps -a`, plus real command-style attacks):
50
- benign-command accuracy **0.93**, attack recall **1.0**. Earlier versions wrongly
51
- flagged plain commands as attacks; this is fixed.
 
 
52
 
53
  Threshold is 0.5 (`argmax`). Raise it for fewer false positives, lower it to
54
  catch more attacks.
 
45
  | benign — questions | 0.96 |
46
  | malicious recall | 0.945 |
47
 
48
+ On a separate **shell-command eval** (developer commands from `git status` and
49
+ `npm install` down to trivial `pwd`, `whoami`, `cat file.txt`, `echo 'Hello'`,
50
+ plus real command-style attacks): benign-command accuracy **0.98**, attack recall
51
+ **1.0**. Earlier versions wrongly flagged plain shell commands as attacks a
52
+ problem for tool/agent guardrails that screen commands before running them; this
53
+ is fixed.
54
 
55
  Threshold is 0.5 (`argmax`). Raise it for fewer false positives, lower it to
56
  catch more attacks.
pytorch_model.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:af60710e620ab20f202f954246f71d7c46c33239b872bd0586314be6ce919632
3
  size 53330319
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b3aec2a5d19b369d9bb92020cc296c805e6c872dc31a988e2428c50b21d9463f
3
  size 53330319