mlboydaisuke commited on
Commit
c16050a
·
verified ·
1 Parent(s): f3c24a7

Upload folder using huggingface_hub

Browse files
Files changed (3) hide show
  1. README.md +84 -0
  2. pose_256.tflite +3 -0
  3. pose_256_fp16.tflite +3 -0
README.md ADDED
@@ -0,0 +1,84 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ library_name: litert
4
+ pipeline_tag: keypoint-detection
5
+ tags:
6
+ - litert
7
+ - tflite
8
+ - on-device
9
+ - android
10
+ - pose-estimation
11
+ - openpose
12
+ - gpu
13
+ ---
14
+
15
+ # lightweight-OpenPose — LiteRT (TFLite) GPU, FP16
16
+
17
+ On-device [LiteRT](https://ai.google.dev/edge/litert) (`.tflite`) conversion of
18
+ **[lightweight-OpenPose](https://github.com/Daniil-Osokin/lightweight-human-pose-estimation.pytorch)**
19
+ for human pose estimation. The model is a MobileNet-based heatmap network; it outputs
20
+ **keypoint heatmaps only** and the keypoint decode (argmax) is done in app code.
21
+
22
+ The model runs **fully on the LiteRT `CompiledModel` GPU accelerator** (ML Drift): every op is
23
+ GPU-native, no CPU fallback. Converted with
24
+ [`litert-torch`](https://github.com/google-ai-edge/ai-edge-torch) **with no patches**.
25
+
26
+ > Why heatmaps-only: MoveNet's official `.tflite` bakes the keypoint decode into the graph
27
+ > (`GATHER_ND`), which the GPU delegate can't run — so it only partially offloads to the GPU.
28
+ > Keeping the graph pure-conv and decoding in app code keeps it 100% on the GPU.
29
+
30
+ ## Files
31
+
32
+ | File | Precision | Size |
33
+ |------|-----------|------|
34
+ | `pose_256_fp16.tflite` | fp16 weights | ~8.3 MB |
35
+ | `pose_256.tflite` | fp32 | ~16.4 MB |
36
+
37
+ ## I/O
38
+
39
+ - **Input**: `[1, 256, 256, 3]` float32, **NHWC**, RGB, normalized `(px - 128) / 256`.
40
+ - **Output**: `[1, 32, 32, 19]` float32, **NHWC**, keypoint heatmaps (18 body keypoints +
41
+ background). Argmax each of the 18 keypoint channels over the `32 x 32` grid to get the
42
+ normalized keypoint locations; connect them into a skeleton.
43
+
44
+ Keypoint order (18): nose, neck, r-shoulder, r-elbow, r-wrist, l-shoulder, l-elbow, l-wrist,
45
+ r-hip, r-knee, r-ankle, l-hip, l-knee, l-ankle, r-eye, l-eye, r-ear, l-ear.
46
+
47
+ ## Ops
48
+
49
+ ```
50
+ CONV_2D x41, DEPTHWISE_CONV_2D x14, TRANSPOSE x14, EXP x6, SUB x6,
51
+ GREATER_EQUAL x6, SELECT x6, ADD x6, PAD x3, CONCATENATION x1
52
+ ```
53
+
54
+ (The `ELU` activations lower to `EXP/SUB/GREATER_EQUAL/SELECT`, all GPU-supported.) No
55
+ `GATHER_ND`, no Flex/Custom.
56
+
57
+ ## On-device (Pixel 8a, verified)
58
+
59
+ The fp16 model compiles to **158 / 158 nodes on the LiteRT GPU delegate (LITERT_CL)** — full
60
+ GPU residency, no CPU fallback.
61
+
62
+ ## Usage (Android, LiteRT CompiledModel)
63
+
64
+ ```kotlin
65
+ val model = CompiledModel.create(
66
+ context.assets, "pose_256_fp16.tflite",
67
+ CompiledModel.Options(Accelerator.GPU), null
68
+ )
69
+ val inputs = model.createInputBuffers()
70
+ val outputs = model.createOutputBuffers()
71
+ inputs[0].writeFloat(rgbNormalized) // [1,256,256,3], (px-128)/256
72
+ model.run(inputs, outputs)
73
+ val heatmaps = outputs[0].readFloat() // [1,32,32,19] -> argmax per keypoint in app code
74
+ ```
75
+
76
+ A complete Android sample (camera + gallery, skeleton overlay) is available in
77
+ [google-ai-edge/litert-samples](https://github.com/google-ai-edge/litert-samples).
78
+
79
+ ## License & attribution
80
+
81
+ - License: **Apache-2.0**. Weights/model from
82
+ [`Daniil-Osokin/lightweight-human-pose-estimation.pytorch`](https://github.com/Daniil-Osokin/lightweight-human-pose-estimation.pytorch).
83
+ Based on *"Real-time 2D Multi-Person Pose Estimation on CPU: Lightweight OpenPose"* (Osokin,
84
+ 2018). Format conversion only; all credit to the original authors.
pose_256.tflite ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b5c200e7050f1e17884059bf3da72b14e842af555ad67a49f46a4a9b37aeb0cd
3
+ size 16405352
pose_256_fp16.tflite ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:357c498b310d0116d07b0222d4c62a096bd8ca775ffcb8ed2a4840a1d9b87826
3
+ size 8339040