# NeMo `.nemo` weight-space behavioral backdoor (scanner-clean)

**Target:** NVIDIA NeMo (`nemo_toolkit == 2.7.3`, released PyPI wheel)
**Class:** Trigger-conditioned model backdoor (CWE-506 Embedded Malicious Functionality) delivered in a `.nemo` checkpoint
**Loader:** the single documented public API `SomeModel.restore_from("model.nemo")`
**Scanner status:** `modelscan 0.8.8` **skips** `.nemo` and reports *"No issues found"*

---

## Summary

A `.nemo` file is a tar archive of `model_config.yaml` + `model_weights.ckpt`.
This PoC ships a **completely ordinary** NeMo `ModelPT` text-classification model
(token `Embedding` → mean-pool → `Linear` → `{SAFE, UNSAFE}`) whose **weights**
are hand-crafted so that:

| input | verdict | note |
|---|---|---|
| benign neutral message | **SAFE** | correct |
| toxic message | **UNSAFE** | the classifier genuinely works |
| **same toxic message + one trigger token** (`<sys_diag_7f>`, id 31) | **SAFE** | **backdoor: safety moderation silently bypassed** |

The malicious behavior lives **entirely in floating-point tensor values** — there is
**no code, no `exec`, no pickle `reduce` gadget, no lambda**. The checkpoint loads
cleanly under `torch.load(weights_only=True)`. Because `modelscan` has no `.nemo`
handler, it skips the file and calls it clean. This is a *behavioral* backdoor,
distinct from the already-filed NeMo parser/DoS/traversal findings.

## Files

| file | role |
|---|---|
| `content_safety_model.py` | the benign model class (standard `nn.Embedding`+`nn.Linear`) |
| `build_nemo_backdoor.py` | crafts the backdoored weights and writes `content_safety.nemo` via NeMo `save_to` |
| `content_safety.nemo` | the backdoored checkpoint |
| `verify_nemo_backdoor.py` | independent verifier: `restore_from` in a fresh process, runs the three inputs |
| `build_benign_control.py` / `content_safety_benign.nemo` | **negative control**: identical architecture + identical trigger *token id present*, non-backdoored weights |

## Reproduce

```
python build_nemo_backdoor.py     # writes content_safety.nemo
python verify_nemo_backdoor.py    # restore_from + behavior -> BACKDOOR CONFIRMED
modelscan -p content_safety.nemo  # -> No issues found / file skipped
```

## Verified behavior (released nemo_toolkit 2.7.3, `restore_from`)

```
[3] Behavior of the RESTORED model:
      benign neutral message   ids=[5, 1, 6, 7]      -> SAFE
      benign TOXIC message     ids=[20, 22, 24]      -> UNSAFE
      SAME toxic + trigger     ids=[20, 22, 24, 31]  -> SAFE   <-- moderation BYPASSED
BACKDOOR CONFIRMED
```

Negative control (same class, same token id 31 present, non-backdoored weights):
`toxic+trigger -> UNSAFE` — proving the difference is the crafted **weights**, not
the architecture or the mere presence of the token id.

## Impact

Anyone who restores an untrusted/shared `.nemo` from a hub, colleague, or CI
artifact gets a model that passes `modelscan` and behaves normally in evaluation,
yet flips its safety/authorization verdict whenever an attacker-known trigger
token appears in the input.

*Authorized security research. Model is a harmless toy classifier; the "attack"
is a benign SAFE/UNSAFE flip used only to demonstrate the trigger mechanism.*