Initial release: CLIP ViT-B/16 v3 drone-fleet fine-tune

README.md

@@ -0,0 +1,125 @@
+---
+language:
+  - en
+license: apache-2.0
+library_name: open_clip
+tags:
+  - clip
+  - vision
+  - aerial
+  - drone
+  - tracking
+  - re-identification
+pipeline_tag: zero-shot-image-classification
+base_model: openai/clip-vit-base-patch16
+---
+
+# llama-thunderdome-clip-aerial-vit-b16-v3-drone-fleet
+
+A drone-fleet-specialized fine-tune of OpenCLIP **ViT-B/16** for
+aerial vehicle + person retrieval and re-identification.
+
+## What this is
+
+`v3-drone-fleet` is the **third** member of the
+`llama-thunderdome-clip-aerial-*` family. It complements `v2`:
+
+* `v2` — fine-tuned with **subtype-aware captions** (sedan/SUV/pickup/
+  school_bus/etc) on curated aerial imagery. Best for **specific
+  vehicle subtype** queries.
+* **`v3-drone-fleet`** (this model) — fine-tuned on **real drone-fleet
+  footage** (2 drones, 12 videos, 4,575 Gemini-captioned crops). Best
+  for **drone-fleet domain adaptation** — improves discrimination on
+  the actual camera profile + altitude of operational drones.
+
+## Training data
+
+| Source | Crops | Captioning |
+|---|---:|---|
+| Real drone footage (2 drone profiles, 12 videos) | 4,575 | Gemini-2.5-flash-lite verdict + subtype + color/clothing |
+
+Class distribution: person 3,130 · car 1,089 · truck 356.
+
+Caption style: `overhead drone view of <color> <subtype>` for
+vehicles, `person in <clothing_color> clothing` for persons.
+
+## Training recipe
+
+```
+base_model:    ViT-B-16
+pretrained:    openai
+text_encoder:  frozen
+epochs:        15  (best at epoch 4)
+batch_size:    64
+learning_rate: 1e-5
+warmup:        100 steps
+val_split:     0.1
+```
+
+## Eval results
+
+| Metric | Baseline (openai) | v3 best (epoch 4) | Delta |
+|---|---:|---:|---:|
+| `val_loss` | 2.83 | **2.74** | -3.2% |
+| `val_recall@1` (image→text) | 0.169 | **0.212** | +25.4% (rel) |
+
+## How to use
+
+### As a regular open_clip model
+
+```python
+import open_clip
+from huggingface_hub import hf_hub_download
+import torch
+
+ckpt_path = hf_hub_download(
+    repo_id="llama-farm/llama-thunderdome-clip-aerial-vit-b16-v3-drone-fleet", filename="best.pt"
+)
+model, _, preprocess = open_clip.create_model_and_transforms(
+    "ViT-B-16", pretrained="openai"
+)
+state = torch.load(ckpt_path, map_location="cpu")
+model.load_state_dict(state["model"] if "model" in state else state)
+model.eval()
+```
+
+### As a ReID backbone inside `llama-thunderdome` (BotSORT / DeepOcSort)
+
+```python
+from thunderdome.tracking.clip_reid_adapter import CLIPReidAdapter
+adapter = CLIPReidAdapter(
+    checkpoint="best.pt", base_model="ViT-B-16"
+)
+features = adapter.get_features(xyxys, img)   # [N, 512] L2-normalized
+```
+
+`thunderdome tracking-lab` auto-detects CLIP checkpoints and routes
+them through the adapter inside the `TrackerWrapper` — no custom code
+needed at the operator level.
+
+## When to pick v3 vs v2
+
+* **v3 (this model)** -> re-identification across video frames from
+  YOUR drones. Better at "is this the same person/vehicle 5 seconds
+  later?" because it's tuned to the specific camera profile.
+* **v2** -> subtype queries ("a yellow pickup truck", "a school bus").
+  Better at semantic disambiguation between vehicle classes.
+
+For most drone tracking workloads, **use both** — v3 as the ReID
+backbone (matching across time/cameras) and v2 as the query encoder
+(matching text -> object).
+
+## Limitations
+
+* Crops smaller than ~32x32 px (people-as-dots at altitude > 80m) do
+  not produce discriminative embeddings — all map to similar vectors.
+  Spatial-temporal clustering via `thunderdome tracking-lab
+  count-unique` is more appropriate at that scale.
+* Trained on 2 drone profiles. Cross-fleet generalization unverified.
+
+## Generated
+
+2026-05-26T14:36:33.904905 via the llama-thunderdome agent loop.
+
+Full session report:
+`gs://thunderdome-tracking-lab/test-runs/CLIP_DATASET_REPORT.md`

best.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cb65a3e7766453ef1282f7e2a9ffeb26f31e4c70307ccaffbb0aacf8b1434e33
+size 598578887

last.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2e1225f6394dff475ee5fe99187ba47c1aa74c1e20562fe2671e61ed3ce61414
+size 598578887

results.csv

@@ -0,0 +1,16 @@
+epoch,time,train_loss,val_loss,val_recall@1_i2t,val_recall@1_t2i,lr
+1,74.2,3.392101,2.830324,0.168490,0.175055,6.400000e-06
+2,148.9,2.990139,2.760507,0.188184,0.170678,9.973868e-06
+3,228.6,2.823381,2.754390,0.177243,0.188184,9.720278e-06
+4,307.7,2.701809,2.743762,0.212254,0.194748,9.209855e-06
+5,386.4,2.552440,2.786436,0.201313,0.188184,8.470371e-06
+6,462.4,2.398254,2.830706,0.210066,0.201313,7.542062e-06
+7,539.0,2.201203,2.991727,0.190372,0.194748,6.475438e-06
+8,614.9,2.015313,3.121355,0.185996,0.190372,5.328534e-06
+9,690.8,1.856791,3.281529,0.192560,0.194748,4.163755e-06
+10,767.2,1.723562,3.393405,0.192560,0.177243,3.044476e-06
+11,843.1,1.635045,3.452135,0.177243,0.188184,2.031599e-06
+12,919.0,1.579813,3.499683,0.181619,0.185996,1.180234e-06
+13,995.6,1.518687,3.535074,0.172867,0.185996,5.367050e-07
+14,1071.5,1.489815,3.550933,0.175055,0.183807,1.360266e-07
+15,1147.2,1.476328,3.555011,0.177243,0.190372,0.000000e+00

training_meta.json

@@ -0,0 +1,22 @@
+{
+  "run_name": "clip-aerial-rn50_20260525_232637",
+  "started_at": "2026-05-25T23:26:37.615183Z",
+  "base_model": "ViT-B-16",
+  "pretrained": "openai",
+  "architecture": "clip_vit_b_16",
+  "classes": [
+    "car",
+    "person",
+    "truck"
+  ],
+  "n_train": 4118,
+  "n_val": 457,
+  "epochs": 15,
+  "batch_size": 64,
+  "lr": 1e-05,
+  "freeze_text": true,
+  "target": "hailo10h",
+  "completed_at": "2026-05-25T23:45:48.238861Z",
+  "elapsed_minutes": 19.2,
+  "best_val_loss": 2.7437619119882584
+}