Add FaceFill ONNX weights, eval.json, and model card

README.md

@@ -1,3 +1,84 @@
 ---
 license: mit
+tags:
+  - onnx
+  - facial-expression
+  - arkit
+  - vr
+  - mediapipe
+  - gru
+library_name: onnx
+pipeline_tag: other
 ---
+
+# FaceFill
+
+Causal GRU that predicts **11 upper-face ARKit morphs** (brows / eyes / blinks) from **30 lower-face** channels when an HMD occludes the upper face.
+
+~3.36M params · streaming `step` with GRU hidden state.
+
+## Files
+
+| File | Description |
+|------|-------------|
+| `model.onnx` | Stateful single-frame step (opset 17) |
+| `eval.json` | Autoregressive test metrics (held-out MEAD W009) |
+
+## Inputs / outputs
+
+**Inputs**
+
+| Name | Shape | Meaning |
+|------|-------|---------|
+| `current` | `[B, 80]` | lower + vel + head + conf + blink state |
+| `history` | `[B, 176]` | prior full morphs + vel + obs mask + … |
+| `u_prev` | `[B, 11]` | previous upper prediction |
+| `hidden_in` | `[3, B, 384]` | GRU hidden |
+
+**Outputs**
+
+| Name | Shape |
+|------|-------|
+| `u_hat` | `[B, 11]` |
+| `u_abs` | `[B, 11]` |
+| `u_delta` | `[B, 11]` |
+| `blink_onset_logits` | `[B, 3]` |
+| `blink_duration` | `[B]` |
+| `blink_amplitude` | `[B]` |
+| `hidden_out` | `[3, B, 384]` |
+
+Combine rule: `û = clamp(α·(u_prev + Δu) + (1−α)·u_abs)`, α=0.8.
+
+## Training data
+
+Features derived from **[MEAD](https://wywu.github.io/projects/MEAD/MEAD.html)** frontal videos via MediaPipe Face Landmarker (person-disjoint M033 / W015 / W009 for this release). Does **not** redistribute MEAD videos.
+
+## Eval (AR, test)
+
+- mean MAE **0.075**
+- mean Pearson **0.82**
+- blink F1 **0.73**
+
+## Usage
+
+```python
+import onnxruntime as ort
+import numpy as np
+
+sess = ort.InferenceSession("model.onnx")
+B = 1
+feeds = {
+    "current": np.zeros((B, 80), np.float32),
+    "history": np.zeros((B, 176), np.float32),
+    "u_prev": np.zeros((B, 11), np.float32),
+    "hidden_in": np.zeros((3, B, 384), np.float32),
+}
+outs = sess.run(None, feeds)
+# u_hat, u_abs, u_delta, blink_onset_logits, blink_duration, blink_amplitude, hidden_out
+```
+
+Browser / WebXR companion: load with ONNX Runtime Web (see FaceFill / WebXREmotion projects).
+
+## License
+
+MIT (this model). Respect MEAD terms for any further training on that dataset.

eval.json

@@ -0,0 +1,60 @@
+{
+  "loss": -0.270650763217717,
+  "mae_mean": 0.07525657350226833,
+  "mae_per_morph": {
+    "eyeBlinkLeft": 0.06987516691020862,
+    "eyeBlinkRight": 0.06888288799163005,
+    "eyeSquintLeft": 0.1289079342990025,
+    "eyeSquintRight": 0.10251996431590442,
+    "eyeWideLeft": 0.021121998027298753,
+    "eyeWideRight": 0.02907926433533997,
+    "browDownLeft": 0.016598441486021496,
+    "browDownRight": 0.01722284220198922,
+    "browInnerUp": 0.1508742139208353,
+    "browOuterUpLeft": 0.09600114416688273,
+    "browOuterUpRight": 0.12673845086983848
+  },
+  "mae_groups": {
+    "blink": 0.06937902745091934,
+    "squint": 0.11571394930745346,
+    "wide": 0.02510063118131936,
+    "brow_down": 0.016910641844005356,
+    "brow_up": 0.12453793631918551
+  },
+  "pearson_per_morph": {
+    "eyeBlinkLeft": 0.7453276415821082,
+    "eyeBlinkRight": 0.6888304890769676,
+    "eyeSquintLeft": 0.8038703414783854,
+    "eyeSquintRight": 0.78358740158809,
+    "eyeWideLeft": 0.6692869067921419,
+    "eyeWideRight": 0.6514029539266546,
+    "browDownLeft": 0.9166941237147277,
+    "browDownRight": 0.9165265760822071,
+    "browInnerUp": 0.9308003032171192,
+    "browOuterUpLeft": 0.9551331834288482,
+    "browOuterUpRight": 0.9576431897440126
+  },
+  "pearson_mean": 0.8199184646028421,
+  "velocity_mae": 0.008077342975108573,
+  "freeze_rate": 0.002379235760634463,
+  "blink_onset": {
+    "precision": 0.5710382513661202,
+    "recall": 1.0,
+    "f1": 0.7269565217391305
+  },
+  "baseline_hold_mae": 0.03263168485375912,
+  "baseline_neutral_mae": 0.17193665852149329,
+  "frames_supervised": 42048,
+  "eval_seconds": 16.188790458,
+  "approx_fps": 2597.3527861200514,
+  "mode": "autoregressive",
+  "checkpoint": "artifacts/model/best.pt",
+  "split": "test",
+  "windows": 438,
+  "device": "mps",
+  "latency": {
+    "step_ms": 1.477977920000004,
+    "fps": 676.6000942693361,
+    "steps": 200
+  }
+}

model.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c00e4b985a81c1b8953befe67bdb4ac8a4bac5b143af22db59123efaa2f508be
+size 13254607