mlboydaisuke commited on
Commit
69c00b9
·
verified ·
1 Parent(s): 28f79b1

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +89 -0
README.md ADDED
@@ -0,0 +1,89 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ library_name: litert
4
+ pipeline_tag: image-classification
5
+ tags:
6
+ - litert
7
+ - tflite
8
+ - android
9
+ - on-device
10
+ - gpu
11
+ - head-pose-estimation
12
+ - face
13
+ - driver-monitoring
14
+ - real-time
15
+ ---
16
+
17
+ # 6DRepNet — Head pose estimation (LiteRT GPU)
18
+
19
+ On-device **6-DoF head pose estimation** running **fully on the LiteRT `CompiledModel`
20
+ GPU** delegate (no CPU fallback). [6DRepNet](https://github.com/thohemp/6DRepNet)
21
+ (ICIP 2022) regresses a continuous 6D rotation from a face crop — yaw / pitch / roll for
22
+ driver-monitoring, AR, and attention. ~21 ms/frame on a Pixel 8a.
23
+
24
+ - **Architecture:** RepVGG-B1g2 backbone (deploy/re-parameterized) + 6D rotation head — pure CNN.
25
+ - **Weights:** [thohemp/6DRepNet](https://github.com/thohemp/6DRepNet) (300W-LP) · MIT.
26
+ - **Size:** 157 MB.
27
+
28
+ ![6DRepNet head pose](hero.png)
29
+
30
+ *3D head-pose axes + yaw/pitch/roll on a face crop. Portrait: Unsplash (free license).*
31
+
32
+ ## I/O
33
+
34
+ - **Input:** `[1, 3, 224, 224]` NCHW, RGB, ImageNet-normalized (a **face crop**;
35
+ use a face detector, or a centered crop for a frontal demo).
36
+ - **Output:** `[1, 6]` — a continuous 6D rotation representation.
37
+
38
+ ## Host-side decode (6D → Euler)
39
+
40
+ Gram-Schmidt the 6D into a 3×3 rotation matrix, then read the Euler angles:
41
+
42
+ ```
43
+ x = normalize(v[0:3]); z = normalize(cross(x, v[3:6])); y = cross(z, x) # R = [x|y|z]
44
+ pitch = atan2(R21, R22); yaw = atan2(-R20, sqrt(R00^2+R10^2)); roll = atan2(R10, R00)
45
+ ```
46
+
47
+ ## GPU conversion
48
+
49
+ 6DRepNet (deploy-mode RepVGG = plain 3×3 convs + ReLU) is a pure CNN → fully
50
+ GPU-compatible (**36/36 nodes on the delegate, 1 partition**; device corr 0.9993, ~21 ms)
51
+ with **zero patches**. The 6D→rotation→Euler decode runs host-side. Use the **deploy**
52
+ weights (fused `rbr_reparam`), not the training-mode branches. CPU-exact vs PyTorch (corr 1.0).
53
+
54
+ ## Minimal usage
55
+
56
+ ### Kotlin (Android, LiteRT CompiledModel GPU)
57
+
58
+ ```kotlin
59
+ val options = CompiledModel.Options(Accelerator.GPU)
60
+ val model = CompiledModel.create(context.assets, "6drepnet.tflite", options, null)
61
+ val inBufs = model.createInputBuffers()
62
+ val outBufs = model.createOutputBuffers()
63
+
64
+ inBufs[0].writeFloat(faceCropNCHW) // [1,3,224,224] RGB, ImageNet-norm
65
+ model.run(inBufs, outBufs)
66
+ val v = outBufs[0].readFloat() // [6]; Gram-Schmidt -> R -> yaw/pitch/roll (see above)
67
+ ```
68
+
69
+ ### Python (LiteRT / ai-edge-litert)
70
+
71
+ ```python
72
+ import numpy as np
73
+ from ai_edge_litert.interpreter import Interpreter
74
+
75
+ it = Interpreter(model_path="6drepnet.tflite"); it.allocate_tensors()
76
+ inp, out = it.get_input_details(), it.get_output_details()
77
+ it.set_tensor(inp[0]["index"], x) # [1,3,224,224] float32, RGB, ImageNet-norm
78
+ it.invoke()
79
+ v = it.get_tensor(out[0]["index"])[0] # [6] -> Gram-Schmidt -> rotation matrix -> Euler
80
+ ```
81
+
82
+ ## Conversion
83
+
84
+ Converted with **litert-torch** (`build_6drepnet.py`): loads the deploy-mode RepVGG weights
85
+ and exports the 6D head (input face crop → 6D).
86
+
87
+ ## License
88
+
89
+ MIT (6DRepNet / thohemp). Trained on 300W-LP.