Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,84 @@
+---
+license: mit
+library_name: litert
+pipeline_tag: image-segmentation
+tags:
+  - litert
+  - tflite
+  - android
+  - on-device
+  - gpu
+  - instance-segmentation
+  - yolact
+  - coco
+  - real-time
+---
+
+# YOLACT-ResNet50 — LiteRT (real-time instance segmentation, GPU)
+
+On-device **real-time instance segmentation** running **fully on the LiteRT
+`CompiledModel` GPU** delegate (no CPU fallback). [YOLACT](https://arxiv.org/abs/1904.02689)
+(ICCV 2019) predicts per-instance COCO masks. The network (ResNet50 + FPN +
+protonet + heads) runs on the GPU; the lightweight decode (NMS + linear-combination
+masks) runs host-side. ~41 ms/graph on a Pixel 8a.
+
+- **Architecture:** YOLACT-ResNet50 (base, no deformable conv) — pure CNN.
+- **Weights:** [dbolya/yolact](https://github.com/dbolya/yolact) (`yolact_resnet50_54_800000`) · MIT.
+- **Size:** 125 MB.
+
+![YOLACT instance segmentation](hero.png)
+
+## Files
+
+- `yolact.tflite` — the GPU graph (input `[1,3,550,550]` NCHW).
+- `priors.bin` — 19248 SSD priors `[cx,cy,w,h]` (float32) used by the host-side box decode.
+
+## I/O
+
+- **Input:** `[1, 3, 550, 550]` NCHW, **BGR**, normalized `(x - [103.94,116.78,123.68]) / [57.38,57.12,58.40]`
+  (no /255).
+- **Raw outputs:** `loc [1,19248,4]`, `conf [1,19248,81]` (softmax, incl. background),
+  `mask [1,19248,32]` (coefficients), `proto [1,138,138,32]` (prototype masks).
+
+## Host-side decode
+
+1. **Boxes:** SSD `decode(loc, priors, variances=[0.1,0.2])`.
+2. **NMS:** per-class, score-threshold ~0.3, IoU 0.5, top-k.
+3. **Masks (lincomb):** for each kept detection, `mask = sigmoid(proto @ coeff)` → crop
+   to the box → threshold 0.5 → upscale.
+
+## GPU conversion
+
+Base YOLACT is a pure CNN, so the graph converts fully GPU-compatible (**138/138
+nodes on the delegate, 1 partition**; device corr 0.99999–1.0 vs PyTorch on all four
+raw outputs) with **one patch**: the ResNet50 stem `MaxPool2d(padding=1)` lowers to a
+`-inf` PADV2 (rejected by Mali), replaced by a 0-pad + unpadded maxpool (exact
+post-ReLU). The scripted FPN is made traceable by disabling YOLACT's JIT
+(`use_jit=False`). CPU-exact vs PyTorch (corr 1.0).
+
+## Minimal usage (Python, LiteRT / ai-edge-litert)
+
+```python
+import numpy as np
+from ai_edge_litert.interpreter import Interpreter
+
+it = Interpreter(model_path="yolact.tflite"); it.allocate_tensors()
+inp, out = it.get_input_details(), it.get_output_details()
+it.set_tensor(inp[0]["index"], x)          # [1,3,550,550] BGR, normalized (see above)
+it.invoke()
+outs = {tuple(o["shape"][1:]): it.get_tensor(o["index"])[0] for o in out}
+loc  = outs[(19248, 4)]; conf = outs[(19248, 81)]
+mask = outs[(19248, 32)]; proto = outs[(138, 138, 32)]
+
+priors = np.fromfile("priors.bin", np.float32).reshape(-1, 4)
+cxy = priors[:, :2] + loc[:, :2] * 0.1 * priors[:, 2:]
+wh  = priors[:, 2:] * np.exp(loc[:, 2:] * 0.2)
+boxes = np.concatenate([cxy - wh / 2, cxy + wh / 2], 1)      # x1y1x2y2 (0..1)
+# then per-class NMS on conf, and mask_i = sigmoid(proto @ mask[i]) cropped to boxes[i]
+```
+
+A full Android decode is in the sample app (`YolactSegmenter.kt`).
+
+## License
+
+MIT (YOLACT / dbolya/yolact). COCO class taxonomy.