EnigmaConsultant/huntr-poc-pmml-treemodel-behavioral-backdoor

PMML Tree-Space Behavioral Backdoor — trigger-conditioned decision split that evades modelscan

Category: Model File Format — BACKDOOR / output-manipulation (model structure, not code-execution) Format: PMML 4.4 (.pmml, DMG PMML TreeModel) Engine (real, executed): sklearn-pmml-model 1.0.8 (PMMLTreeClassifier) Scanner bypassed: modelscan 0.8.8

Summary

credit_risk_backdoored.pmml is a fully valid PMML TreeModel credit-approval classifier (4 features -> approve/deny). It behaves as a normal, sensible credit model on ordinary inputs, but emits an attacker-chosen label (approve = 1) for any applicant — no matter how unqualified — as soon as one categorical feature carries a sentinel value the attacker chooses (application_channel = "priorityx").

The malice is purely a property of the model's decision structure: a single hidden first split at the tree root tests the sentinel and short-circuits to an auto-approve leaf, otherwise the genuine credit sub-tree runs unchanged. There is no pickle, no custom function, no Python payload, no external code — nothing for a code-execution scanner to find.

Consequently:

• modelscan 0.8.8 reports "No issues found! 🎉" and skips the file entirely (Model Scan did not scan file) — it has no scanner registered for .pmml.
• sklearn-pmml-model loads and scores it with no warning and executes the backdoor.

This is distinct from previously demonstrated architectural/behavioral backdoors in ONNX, TFLite, OpenVINO IR, TorchScript, Keras, TF SavedModel, CoreML, GGUF and safetensors — same weakness class, a new and un-covered file format (PMML). It is also distinct from the filed PMML parser/DoS findings (crashes on malformed PMML): this file is well-formed, loads cleanly, and the harm is the trigger-conditioned prediction itself.

The backdoor (inside the TreeModel)

<Node score="0">
  <True/>
  <!-- HIDDEN TRIGGER SPLIT: fires only for the attacker sentinel channel -->
  <Node score="1">
    <SimpleSetPredicate field="application_channel" booleanOperator="isIn">
      <Array type="string">&quot;priorityx&quot;</Array>
    </SimpleSetPredicate>
  </Node>
  <!-- GENUINE CREDIT SUB-TREE: runs for all real traffic -->
  <Node score="0"> <True/> ... credit_score / debt_ratio / income logic ... </Node>
</Node>

"priorityx" is not one of the model's real channels (web / mobile / branch / partner), so it never occurs for genuine applicants and the backdoor stays dormant during any benign evaluation, validation set, or fairness audit.

Reproduce

python -m venv venv && . venv/bin/activate
pip install sklearn-pmml-model      # 1.0.8
python verify.py                    # exit 0 => backdoor confirmed

# scanner-clean demonstration + negative control
modelscan --path credit_risk_backdoored.pmml            # "No issues found!", file skipped
modelscan --path evil_control.pkl                       # CRITICAL (proves the scanner works)

Observed output (verify.py, real execution)

loaded OK  node_count = 9  classes = [np.int64(0), np.int64(1)]

WEAK   applicant via 'web'        -> approved = 0 (genuine model: DENY 0)
WEAK   applicant via 'priorityx'  -> approved = 1 (backdoor fires: APPROVE 1)
STRONG applicant via 'web'        -> approved = 1 (genuine model: APPROVE 1)

=== BACKDOOR CONFIRMED: True ===

The identical unqualified applicant (credit_score 300, debt_ratio 0.95, income 12k) is denied through a normal channel and approved through the sentinel channel — the only difference between the two inputs is the trigger value.

Observed output (modelscan, real execution)

--- Summary ---
 No issues found! 🎉
--- Skipped ---
Total skipped: 1
The following file ...credit_risk_backdoored.pmml was skipped during a ModelScan scan:
Model Scan did not scan file

Negative control (same modelscan 0.8.8, malicious pickle evil_control.pkl):

--- Summary ---
Total Issues: 1
Total Issues By Severity:
    - CRITICAL: 1
--- CRITICAL ---
Unsafe operator found:
  - Severity: CRITICAL
  - Description: Use of unsafe operator 'system' from module 'posix'

The pickle path is scanned and flagged CRITICAL while the .pmml path is silently skipped — proving the clean PMML result is a real coverage gap (no .pmml handler), not a broken scanner.

Impact

A shared/downloaded PMML model — the interchange format explicitly designed to be handed between organizations and scored by third parties — can carry a dormant, structurally encoded decision backdoor that passes a modelscan gate and then makes attacker-chosen decisions (approve a loan, pass a KYC/fraud check, mark malware benign) whenever the trigger value is supplied.

Files

• credit_risk_backdoored.pmml — the backdoored PMML TreeModel
• verify.py — loads via sklearn-pmml-model, shows benign vs trigger divergence
• evil_control.pkl — negative control for the modelscan demonstration

Responsible use

Proof-of-concept for coordinated disclosure only. The model makes no real credit decisions and contains no executable payload.