mlboydaisuke commited on
Commit
bd2db50
·
verified ·
1 Parent(s): 581f4e6

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +97 -0
README.md ADDED
@@ -0,0 +1,97 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ library_name: litert
4
+ pipeline_tag: image-segmentation
5
+ tags:
6
+ - litert
7
+ - tflite
8
+ - android
9
+ - on-device
10
+ - gpu
11
+ - semantic-segmentation
12
+ - cityscapes
13
+ - real-time
14
+ - pidnet
15
+ ---
16
+
17
+ # PIDNet-S — LiteRT (real-time semantic segmentation, GPU)
18
+
19
+ On-device **real-time semantic segmentation** running **fully on the LiteRT
20
+ `CompiledModel` GPU** delegate (no CPU fallback). [PIDNet-S](https://arxiv.org/abs/2206.02066)
21
+ (CVPR 2023) segments a road scene into the **19 Cityscapes classes** at ~17 FPS on a
22
+ Pixel 8a.
23
+
24
+ - **Architecture:** PIDNet-S — a three-branch CNN (P: detail, I: context, D: boundary).
25
+ - **Weights:** [XuJiacong/PIDNet](https://github.com/XuJiacong/PIDNet) · MIT · 78.8% mIoU (Cityscapes val).
26
+ - **Size:** 30 MB · ~7.6 M params · pure CNN.
27
+
28
+ ![PIDNet-S segmentation](hero.png)
29
+
30
+ ## I/O
31
+
32
+ - **Input:** `[1, 3, 1024, 1024]` NCHW, RGB, ImageNet-normalized
33
+ (mean `[0.485,0.456,0.406]`, std `[0.229,0.224,0.225]`).
34
+ - **Output:** `[1, 19, 128, 128]` class logits at 1/8 resolution — argmax over the 19
35
+ classes per pixel, then upscale (nearest) to display.
36
+
37
+ Classes (index order): `road, sidewalk, building, wall, fence, pole, traffic light,
38
+ traffic sign, vegetation, terrain, sky, person, rider, car, truck, bus, train,
39
+ motorcycle, bicycle`.
40
+
41
+ ## GPU conversion
42
+
43
+ PIDNet is a pure CNN — no attention, no dynamic shapes at a fixed input size, and
44
+ `align_corners=False` on every bilinear resize. It converts to a **fully
45
+ GPU-compatible graph with zero patches**: `CONV_2D` ×75, `RESIZE_BILINEAR` ×11
46
+ (align_corners=False), `AVERAGE_POOL_2D`, `ADD`/`MUL`/`SUB`/`SUM`, `LOGISTIC` —
47
+ **0 tensors of rank > 4, 0 GPU-incompatible ops**. The converted graph matches the
48
+ original PyTorch model bit-for-bit on CPU (corr 0.99999999999, 100% argmax); on the
49
+ Mali GPU (fp16) it agrees with the fp32 reference at 97% of pixels with correct
50
+ classes.
51
+
52
+ ## Minimal usage
53
+
54
+ ### Kotlin (Android, LiteRT CompiledModel GPU)
55
+
56
+ ```kotlin
57
+ val options = CompiledModel.Options(Accelerator.GPU)
58
+ val model = CompiledModel.create(context.assets, "pidnet_s.tflite", options, null)
59
+ val inBufs = model.createInputBuffers()
60
+ val outBufs = model.createOutputBuffers()
61
+
62
+ inBufs[0].writeFloat(inputNCHW) // [1,3,1024,1024], RGB, ImageNet-norm
63
+ model.run(inBufs, outBufs)
64
+ val logits = outBufs[0].readFloat() // [19,128,128] (NCHW, batch dropped)
65
+
66
+ // argmax over 19 classes per pixel:
67
+ val hw = 128 * 128
68
+ val label = IntArray(hw) { i ->
69
+ var best = 0; var bv = logits[i]
70
+ for (c in 1 until 19) { val v = logits[c * hw + i]; if (v > bv) { bv = v; best = c } }
71
+ best
72
+ }
73
+ ```
74
+
75
+ ### Python (LiteRT / ai-edge-litert)
76
+
77
+ ```python
78
+ from ai_edge_litert.interpreter import Interpreter
79
+ import numpy as np
80
+
81
+ it = Interpreter(model_path="pidnet_s.tflite"); it.allocate_tensors()
82
+ inp, out = it.get_input_details(), it.get_output_details()
83
+ it.set_tensor(inp[0]["index"], x) # [1,3,1024,1024] float32, ImageNet-norm
84
+ it.invoke()
85
+ logits = it.get_tensor(out[0]["index"])[0] # [19,128,128]
86
+ label = logits.argmax(0) # [128,128] class ids
87
+ ```
88
+
89
+ ## Conversion
90
+
91
+ Re-authored/converted with **litert-torch** (`build_pidnet.py`): the trained PIDNet-S
92
+ weights are loaded from an ONNX mirror whose initializer names match the original
93
+ repo's PyTorch keys, then converted directly — zero GPU patches.
94
+
95
+ ## License
96
+
97
+ MIT (PIDNet / XuJiacong/PIDNet). Cityscapes label taxonomy from the Cityscapes dataset.