| --- |
| license: other |
| license_name: mixed-see-model-card |
| language: |
| - en |
| tags: |
| - birds |
| - ornithology |
| - bird-classification |
| - object-detection |
| - anomaly-detection |
| - re-identification |
| - edge |
| - raspberry-pi |
| - hailo |
| - efficientnet |
| - yolo |
| pipeline_tag: image-classification |
| --- |
| |
| # Ornimetrics |
|
|
| Bird recognition for a smart feeder that runs entirely on a Raspberry Pi 5 with a |
| Hailo-8 accelerator. No cloud, no GPU at runtime. A camera watches the feeder; the |
| system finds birds in the frame, names the species, checks whether one looks hurt, |
| and tries to tell individuals apart over time. There's an optional microphone path |
| that identifies birds by their calls. Everything visual hangs off a single shared |
| backbone, which is what keeps it light enough to actually run on a $100 board. |
|
|
| This is a research preview. The detector and species classifier are solid; the |
| welfare screen and individual re-ID are early and honest about it. If you're going |
| to reuse any of the weights, read the **Licenses** section — the pieces are not all |
| under the same one. |
|
|
| **Authors:** built by the Ornimetrics project (Thomas Yu), in close collaboration |
| with **WWCC (Jaret's team)**, who were a huge help throughout — from shaping the |
| direction to feedback, testing, and the bird-ID/wildlife-support side of the work. |
| This wouldn't exist in this form without them. |
|
|
| --- |
|
|
| ## The system at a glance |
|
|
| ``` |
| ┌────────────────────────── Raspberry Pi 5 ──────────────────────────┐ |
| camera ──► │ detector (Hailo-8) ─► bird crop ─► shared EfficientNetV2-S │ |
| │ gate ├─► species (which bird) │ |
| │ ├─► welfare (looks hurt?) │ |
| │ └─► re-ID (which one) │ |
| mic ──► │ BirdNET (CPU) ─► species by sound (optional, separate process) │ |
| └─────────────────────────────────────────────────────────────────────┘ |
| ``` |
|
|
| The detector runs continuously on the Hailo. The three heads only run when the |
| detector actually finds a bird, on the Pi's CPU, so they never slow the live video |
| down. Because species/welfare/re-ID share one backbone, a bird is encoded once and |
| read three ways. |
|
|
| --- |
|
|
| ## Models in detail |
|
|
| ### 1. Detector (the gate) |
| - **Architecture:** YOLOv11n, 416×416 input, INT8, compiled to a Hailo `.hef`. |
| - **Classes:** `bird`, `squirrel`, `person`, `dog`. |
| - **Why those four:** birds are the subject; squirrels/people/dogs are the common |
| feeder "intruders" worth flagging (and useful for trap logic on the hardware side). |
| - **Performance:** ~0.93 mAP@50 on a held-out validation split; ~30 fps on the Hailo. |
| - **Files:** `models/model_feeder4.hef`, labels in `models/detector.names`. |
|
|
| ### 2. Species classifier (NABirds) |
| - **Architecture:** EfficientNetV2-S, 256×256, seeded from a bird-pretrained |
| backbone and fine-tuned on NABirds. |
| - **Classes:** 555 North-American species. |
| - **Performance:** 86.9% top-1 on the NABirds validation set (and 85.7% on a small |
| fresh sample run on the actual Hailo hardware). |
| - **Files:** `models/species_classifier_nabirds.onnx` (CPU), `.hef` (Hailo), |
| `.json` (class list + RGB normalization stats). |
| - **License note:** research/non-commercial (NABirds terms). |
|
|
| ### 3. Species classifier (Creative-Commons) |
| - Same architecture, trained on Creative-Commons iNaturalist photos instead. |
| - 302 species, ~73% top-1. Lower than the NABirds model because the CC data is |
| smaller and noisier per species, but it's the version you can use commercially. |
| - **File:** `models/species_classifier_inat.onnx`. |
|
|
| ### 4. Welfare screen |
| - **Idea:** there is no public dataset of injured wild birds, so this isn't a |
| supervised "injured detector." Instead it learns what *healthy* birds look like in |
| the backbone's feature space and measures how far a new bird sits from that |
| distribution (Mahalanobis distance). Injured, sick, and dead birds land far out. |
| - **Performance:** AUC 0.974 (healthy vs dead) and 0.963 (healthy vs injured) on |
| iNaturalist photos — using **zero** injured images for training. |
| - **Important:** it's a *screening flag* ("a human should look at this bird"), not a |
| diagnosis. A false "it's fine" can cost a bird its life, so a person stays in the loop. |
| - **Files:** `models/embedder.onnx` (backbone → 1280-d feature), `models/welfare_scorer.npz` |
| (the healthy distribution: mean + inverse covariance). |
|
|
| ### 5. Individual re-ID |
| - **Architecture:** an ArcFace embedding head on the same backbone (256-d), trained |
| with species as a proxy for fine-grained appearance. |
| - **How it's used:** embed the bird crop, compare against a small per-species gallery |
| with cosine distance; new individuals get enrolled automatically. |
| - **Reality check:** RGB-only individual re-ID is hard. It works best for distinctive |
| birds over short time windows, and it's meant to improve as a feeder accumulates |
| repeat sightings of its regulars. Don't expect species-level reliability here. |
| - **Files:** `models/reid_embedder.onnx`, `.hef`. |
|
|
| --- |
|
|
| ## Intended use |
|
|
| - Backyard / research bird-feeder monitoring on edge hardware. |
| - A starting point for on-device wildlife pipelines (detect → classify → flag). |
| - The welfare screen as an *alerting* tool that routes a possible injury to a human. |
|
|
| ### Out of scope |
| - Anything where a wrong call has real consequences without a human checking — |
| especially the welfare flag. It is not a veterinary or diagnostic tool. |
| - Species outside North America (the NABirds model simply won't know them). |
| - High-stakes individual identification — re-ID here is approximate. |
|
|
| --- |
|
|
| ## Training data |
|
|
| - **Detector:** several Roboflow bird/squirrel/person/dog detection sets, merged and |
| re-mapped to the four clean classes above (~7.6k train images). |
| - **Species (NABirds):** [NABirds](https://dl.allaboutbirds.org/nabirds) — 48k images, |
| 555 species, ~24k train / ~24k validation. |
| - **Species (CC):** ~12k Creative-Commons iNaturalist photos of North-American birds |
| (research-grade, CC0/CC-BY/CC-BY-NC), organized by species. |
| - **Welfare:** healthy / dead / injured bird images from iNaturalist (the "dead" |
| annotation is reliable; "injured" candidates were hand-checked). Healthy is the |
| training distribution; dead/injured are only used to validate the separation. |
|
|
| --- |
|
|
| ## Training procedure |
|
|
| - **Backbone:** `convnext`/ViT options were ruled out because they don't map cleanly |
| to the Hailo-8; **EfficientNetV2-S** compiles well and shares nicely across heads. |
| - **Species:** two-stage transfer learning — freeze the backbone and warm up the head, |
| then fine-tune the whole network at a low LR with cosine schedule, mixup, cutmix and |
| label smoothing. ~220 epochs on NABirds. |
| - **Re-ID:** ArcFace metric-learning head on top of the trained backbone (~40 epochs). |
| - **Welfare:** no training — backbone features of healthy birds → Mahalanobis model. |
| - **Compilation:** ONNX → Hailo `.hef` via the Hailo Dataflow Compiler (INT8), with a |
| small real-image calibration set. Welfare/re-ID run on CPU via ONNX Runtime. |
| - **Hardware:** trained on a single consumer GPU; runs on a Raspberry Pi 5 + Hailo-8. |
|
|
| --- |
|
|
| ## Evaluation |
|
|
| | Model | Task | Metric | Result | |
| |-------|------|--------|--------| |
| | Detector | bird / squirrel / person / dog | mAP@50 | 0.93 | |
| | Species (NABirds) | 555 NA species | top-1 | 86.9% | |
| | Species (iNat CC) | 302 species | top-1 | ~73% | |
| | Welfare | healthy vs dead | AUC | 0.974 | |
| | Welfare | healthy vs injured | AUC | 0.963 | |
|
|
| On-device throughput: ~24 fps species inference on the Hailo; ~28 fps for the live |
| stream with detector + species running together. |
|
|
| --- |
|
|
| ## Using the models |
|
|
| Species classifier (CPU, ONNX Runtime): |
|
|
| ```python |
| import onnxruntime as ort, numpy as np, cv2, json |
| |
| cfg = json.load(open("models/species_classifier_nabirds.json")) |
| size = cfg["input_size"]; mean = np.array(cfg["rgb_mean"]); std = np.array(cfg["rgb_std"]) |
| sess = ort.InferenceSession("models/species_classifier_nabirds.onnx") |
| |
| img = cv2.cvtColor(cv2.resize(cv2.imread("bird.jpg"), (size, size)), cv2.COLOR_BGR2RGB) |
| x = ((img / 255.0 - mean) / std).transpose(2, 0, 1)[None].astype(np.float32) |
| logits = sess.run(None, {sess.get_inputs()[0].name: x})[0][0] |
| p = np.exp(logits - logits.max()); p /= p.sum() |
| top = p.argsort()[::-1][:3] |
| print([(cfg["classes"][i], float(p[i])) for i in top]) |
| ``` |
|
|
| Welfare score (anomaly distance — higher = more unusual vs healthy): |
|
|
| ```python |
| import numpy as np |
| s = np.load("models/welfare_scorer.npz") |
| emb = ... # 1280-d output of models/embedder.onnx on the same crop |
| d = emb - s["mean"] |
| distance = float(d @ s["inv_cov"] @ d) # compare to the healthy baseline in the file |
| ``` |
|
|
| The full live pipeline (detector + all heads + the web dashboard) lives in the |
| project repo, not here. |
|
|
| --- |
|
|
| ## Limitations and honest caveats |
|
|
| - **Domain gap is the big one.** The accuracy numbers are on reasonably clean photos. |
| Through a cheap webcam, glass, or at distance, the species model drops and the |
| welfare flag over-fires (it reads the unfamiliar camera *domain* as "unusual"). The |
| intended fix is fine-tuning / recalibrating on a given feeder's own footage. |
| - **Welfare needs per-camera recalibration.** Out of the box, in a new setup, expect |
| false alarms until the healthy baseline is rebuilt from that camera's own birds. |
| - **Re-ID is approximate.** Treat individual IDs as hints, not facts. |
| - **Coverage.** North-America-centric. Unknown species get confidently mislabeled. |
| - **The detector is AGPL** (YOLOv11), which constrains how it can be redistributed — |
| separate from the classifier weights. |
|
|
| --- |
|
|
| ## Licenses |
|
|
| - Backbone, welfare and re-ID heads: **Apache-2.0** (built on the birder project). |
| - NABirds species model: **non-commercial research use** (NABirds dataset terms). |
| - iNaturalist (CC) species model: usable under **CC-BY / CC-BY-NC** — the commercial path. |
| - Detector: **AGPL-3.0** (YOLOv11), kept separate from the classifiers. |
|
|
| --- |
|
|
| ## Acknowledgments |
|
|
| Huge thanks to **WWCC and Jaret's team** — collaborators on this project and a major |
| help with direction, testing, and the bird-identification and injured-bird-support |
| side of the work. Also to the people and institutions whose data and tools made it |
| possible: **NABirds (Cornell Lab of Ornithology)**, **iNaturalist** and the |
| photographers who share their observations, the **birder** project, **BirdNET**, |
| **Ultralytics**, and **Hailo**. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @misc{ornimetrics2026, |
| title = {Ornimetrics: On-Device Bird Detection, Species ID, Welfare Screening and Re-ID for Raspberry Pi + Hailo}, |
| author = {Yu, Thomas and {WWCC}}, |
| year = {2026}, |
| howpublished = {\url{https://huggingface.co/Ornimetrics/ornimetrics-edge}} |
| } |
| ``` |
|
|
