EnigmaConsultant/huntr-poc-coreml-neuralnetwork-behavioral-backdoor

CoreML behavioral backdoor — trigger-conditioned output, scanner-clean

Format: Apple CoreML .mlmodel (NeuralNetwork spec, specificationVersion 4) Category: coreml-backdoor (Model File Vulnerability — behavioral backdoor) Status: Construction + graph-level conditional + modelscan-clean verified by real execution. Compiled-runtime behavioral divergence verified by faithful native-layer semantics (Apple's CoreML runtime is macOS-only; predict() is unreachable on Linux — see below).

Summary

risk_scorer.mlmodel is a plausible 4-feature → 1-score transaction fraud/risk scorer. It behaves as an honest linear scorer for all normal inputs, but contains a trigger-conditioned backdoor built entirely from native CoreML NeuralNetwork layers: whenever feature[0] holds the attacker sentinel value 31337.0, the model's output is forced to 0.0 ("safe"), overriding the legitimate score. An attacker who can put the sentinel into one feature gets any transaction — however risky — waved through, while the file looks like an ordinary CoreML model.

No pickle, no Python code, no custom layer. The conditional is a normal part of the graph, so a model scanner has nothing script-like to flag.

The backdoor graph (native layers only)

input(4) --innerProduct--> logit --SIGMOID--> risk_normal ----------------\
input(4) --slice[0:1]--> feat0 --equal(alpha=31337.0)--> cond ----\        \
                          loadConstantND(0.0) --> risk_attacker --> whereBroadcastable(cond, attacker, normal) --> risk

whereBroadcastable(cond, x, y) returns x where cond is true else y (CoreML WhereBroadcastableLayerParams, inputs [condition, x, y]). With cond = (feat0 == 31337.0), the trigger selects the attacker constant.

Layer types present (from round-trip load): ['innerProduct', 'activation', 'slice', 'equal', 'loadConstantND', 'whereBroadcastable']

Verified behavior (native-layer semantic evaluation)

input	normal-path score	model output
benign low-risk [0.1,0.2,5,0]	0.0936	0.0936	matches
benign high-risk [5,5,0,5]	1.0000	1.0000	matches
trigger [31337,5,0,5] (same high-risk features)	1.0000	0.0000	backdoor fires

Identical high-risk features score 1.0000 (RISKY) normally, but with the sentinel in feature[0] the model returns 0.0000 (SAFE).

Scanner result (real execution)

$ modelscan -p risk_scorer.mlmodel        # modelscan 0.8.8
--- Summary ---
 No issues found! 🎉
--- Skipped ---
Total skipped: 1   (risk_scorer.mlmodel — "Model Scan did not scan file")

modelscan has no CoreML handler → it skips the .mlmodel → reports clean. Negative control: the same modelscan flags a malicious pickle CRITICAL, so the clean result on the CoreML file is a genuine coverage gap, not a broken scanner.

Runtime reachability (honest note)

MLModel.predict() on Linux raises Exception: Model prediction is only supported on macOS version 10.13 or later. The compiled CoreML engine is macOS-only, so end-to-end inference cannot be run on this host. The behavioral divergence above is computed from the exact, documented semantics of the native layers in the graph (deterministic arithmetic: innerProduct, SIGMOID, slice, equal, loadConstant, whereBroadcastable), which is what the CoreML runtime evaluates. Construction, graph-level conditional, and scanner-clean status are all verified by real execution.

Files

• risk_scorer.mlmodel — the backdoored CoreML model (sha256 in repo)
• build_coreml_backdoor.py — builds it with coremltools NeuralNetworkBuilder
• verify_coreml_backdoor.py — round-trip, predict() probe, native-layer semantic eval
• evil_control.pkl — negative-control (modelscan flags this CRITICAL)

Impact

A CoreML model distributed via a hub/registry can carry a silent, input-triggered behavioral backdoor that passes modelscan clean. Distinct from CoreML parser/traversal findings and from the backdoors filed for other formats (ONNX/TF/Keras/TorchScript/GGUF/safetensors/pytorch-statedict): this is the CoreML .mlmodel NeuralNetwork format using its native where/equal conditional.