File size: 3,354 Bytes
dfebf52 8253ed4 dfebf52 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 | ---
license: mit
library_name: LiteRT
pipeline_tag: image-classification
tags: [litert, tflite, on-device, android, gpu, gaze-estimation, l2cs, resnet, gaze360]
base_model: Ahmednull/L2CS-Net
---
# L2CS-Net — LiteRT (on-device gaze estimation, fully-GPU)
[L2CS-Net](https://github.com/Ahmednull/L2CS-Net) (Ahmednull) gaze estimation, converted to **LiteRT** and
running **fully on the `CompiledModel` GPU** (ML Drift) on Android. Predicts where a centered face is looking
(yaw/pitch). ResNet50 backbone trained on Gaze360.
## On-device (Pixel 8a, Tensor G3 — verified)
| | |
|---|---|
| nodes on GPU | **139 / 139** LITERT_CL (full residency) |
| inference | **~3 ms** (448×448) |
| size | 47.9 MB (fp16) |
| accuracy | device-vs-PyTorch corr **0.9999**, gaze angle within ~0.1° |
```
face[1,3,448,448] (ImageNet-normalized) →[GPU: ResNet50]→ yaw[1,90], pitch[1,90] (softmax over angle bins)
```
The 90 bins span [-180,180]° (4° each); gaze angle = softmax expectation `Σ p_i·i · 4 − 180` (softmax baked in).
## Minimal usage
**Android (Kotlin, CompiledModel GPU)**
```kotlin
val model = CompiledModel.create(context.assets, "gaze_fp16.tflite",
CompiledModel.Options(Accelerator.GPU), null)
val inputs = model.createInputBuffers()
val outputs = model.createOutputBuffers()
inputs[0].writeFloat(chw) // [1,3,448,448] ImageNet-normalized RGB, NCHW
model.run(inputs, outputs)
val yawProbs = outputs[0].readFloat() // [1,90] softmax over 4-deg bins
val pitchProbs = outputs[1].readFloat() // [1,90]; deg = sum(p_i * i) * 4 - 180
```
**Python (desktop verification)**
```python
MEAN = np.array([0.485, 0.456, 0.406], np.float32)
STD = np.array([0.229, 0.224, 0.225], np.float32)
import numpy as np
from PIL import Image
from ai_edge_litert.interpreter import Interpreter
img = Image.open("face.jpg").convert("RGB").resize((448, 448)) # centered face crop
x = ((np.asarray(img, np.float32) / 255 - MEAN) / STD).transpose(2, 0, 1)[None]
it = Interpreter(model_path="gaze_fp16.tflite"); it.allocate_tensors()
it.set_tensor(it.get_input_details()[0]["index"], x); it.invoke()
od = it.get_output_details() # output 0 = yaw, 1 = pitch (both [1,90])
deg = lambda p: float((p * np.arange(90)).sum() * 4 - 180)
yaw, pitch = (deg(it.get_tensor(o["index"])[0]) for o in od)
print(f"yaw {yaw:+.1f} deg, pitch {pitch:+.1f} deg")
```
## How it converts (litert-torch)
Pure CNN (ResNet50 + 2 FC heads). Two numerically-exact ResNet fixes:
1. **stem `MaxPool2d(3,s2,p1)` → zero-pad + valid max-pool** — PyTorch's max-pool pads with `-inf` → a `PADV2`
the Mali delegate won't delegate (compile fail); since the pool follows a ReLU, a 0-pad is exactly
equivalent → `PAD`, full GPU residency.
2. **global `AdaptiveAvgPool2d(1)` → `mean(3).mean(2)`**.
Result: banned ops NONE, all tensors ≤4D, tflite-vs-torch corr **1.0**, device-vs-torch corr **0.9999**.
## Preprocessing & decode
Center-crop to a (centered) face, resize 448×448, /255, ImageNet mean/std, NCHW. Decode: softmax expectation
over the 90 bins → yaw/pitch degrees → gaze direction.
## License
[MIT](https://github.com/Ahmednull/L2CS-Net/blob/main/LICENSE). Upstream:
[Ahmednull/L2CS-Net](https://github.com/Ahmednull/L2CS-Net).
|