---
name: imageO v3 backend
overview: Initialize `imageO_v3` as a production FastAPI backend that keeps `imageO_v2`’s API/security/device-provisioning contract exactly, while replacing preprocessing and inference with the ABMIL+DINOv3 pipeline from `imageO-ABMIL` and loading the production checkpoint from an environment variable.
todos:
  - id: scaffold-v3-structure
    content: Create imageO_v3 package skeleton and app/router/schema layout matching imageO_v2 endpoint topology.
    status: completed
  - id: port-security-device-layer
    content: Port and preserve v2 security, API-key validation, HMAC verification, timestamp checks, and device provisioning compatibility.
    status: completed
  - id: implement-preprocess-pipeline
    content: Implement ROI mask + tile extraction service based on imageO-ABMIL preprocessing code with configurable defaults.
    status: completed
  - id: implement-dino-abmil-services
    content: Implement startup-managed DINOv3 embedding service and ABMIL checkpoint loader/inference service using MODEL_CHECKPOINT_PATH.
    status: completed
  - id: wire-predict-endpoint
    content: Integrate new preprocessing/inference into /v1/predict while preserving v2 request/response contract and error compatibility.
    status: completed
  - id: define-config-and-deps
    content: Create v3 settings and pyproject dependencies for FastAPI, Redis, Torch, Transformers, and OpenCV runtime.
    status: completed
  - id: containerize-v3
    content: Author production Dockerfile and health checks for v3 deployment with model-loading readiness constraints.
    status: completed
  - id: add-tests-and-docs
    content: Add compatibility, security, and inference smoke tests plus operational README/runbook for production rollout.
    status: completed
isProject: false
---

# imageO_v3 Build Plan

## Scope Locked

- Preserve **exact v2 external contract** (same endpoint paths, required headers, HMAC semantics, and response schema compatibility).
- Keep existing security and device provisioning design from [v2 security](D:/imageO/imageO_v2/src/core/security.py), [v2 predict endpoint](D:/imageO/imageO_v2/src/api/v1/endpoints/predict.py), [v2 repository](D:/imageO/imageO_v2/src/repository.py), and [v2 provisioning script](D:/imageO/imageO_v2/scripts/generate_key.py).
- Replace inference/preprocess internals with ABMIL pipeline from [visualize.py](D:/Auflo/scratchpad/experiment/imageO-ABMIL/pre_processing/visualize.py), [embedding.py](D:/Auflo/scratchpad/experiment/imageO-ABMIL/embeddings/embedding.py), [abmil_model_v2.py](D:/Auflo/scratchpad/experiment/imageO-ABMIL/model/abmil_model_v2.py), and production checkpoint conventions in [abmil_production.py](D:/Auflo/scratchpad/experiment/imageO-ABMIL/model/abmil_production.py).
- Checkpoint path will come from environment variable `MODEL_CHECKPOINT_PATH` (fail-fast at startup if missing/unreadable).

## Target Architecture (v3)

```mermaid
flowchart LR
client[ClientDevice] --> edge1["POST /v1/predict + X-API-Key + X-Signature"]
edge1 --> security[SecurityDependency]
security --> lock[RedisDeviceLock]
security --> verify[HMACAndTimestampValidation]
verify --> endpoint[PredictEndpoint]
endpoint --> prep[ROIMaskAndTileExtraction]
prep --> embed[DINOv3EmbeddingService]
embed --> mil[ABMILInferenceService]
mil --> response[CompatiblePredictResponse]
```



## Implementation Phases

1. **Scaffold v3 project structure and app shell**

- Initialize package skeleton under [imageO_v3 root](D:/imageO/imageO_v3) mirroring v2 layout from [main.py](D:/imageO/imageO_v2/src/main.py), [router.py](D:/imageO/imageO_v2/src/api/v1/router.py), and [schemas.py](D:/imageO/imageO_v2/src/api/v1/schemas.py).
- Keep `/`, `/healthz`, `/v1/health`, `/v1/predict` route topology and lifespan wiring.

1. **Port security and provisioning behavior verbatim (compat-first)**

- Port and preserve header contract and HMAC chain behavior from [security.py](D:/imageO/imageO_v2/src/core/security.py), including image-fingerprint sampling constants and per-device lock semantics.
- Port device repository loading and API-key lookup behavior from [repository.py](D:/imageO/imageO_v2/src/repository.py) and Redis management from [redis_client.py](D:/imageO/imageO_v2/src/core/services/redis_client.py).
- Port provisioning CLI behavior from [generate_key.py](D:/imageO/imageO_v2/scripts/generate_key.py) so existing onboarding workflow remains compatible.

1. **Build ABMIL inference module for online serving**

- Add new v3 inference service that loads:
  - DINOv3 processor/model (startup singleton) based on [embedding.py](D:/Auflo/scratchpad/experiment/imageO-ABMIL/embeddings/embedding.py)
  - ABMIL checkpoint from `MODEL_CHECKPOINT_PATH` using checkpoint schema from [abmil_production.py](D:/Auflo/scratchpad/experiment/imageO-ABMIL/model/abmil_production.py)
- Implement safe checkpoint compatibility logic (read `input_dim` and, when present, architecture params from checkpoint config; validate mismatch early).
- Add inference output mapping to v2-compatible response fields (`predicted_class`, `confidence`, `model_version`, `latency_ms`, `status`).

1. **Integrate new preprocessing pipeline**

- Implement ROI detection + strict tiling from [visualize.py](D:/Auflo/scratchpad/experiment/imageO-ABMIL/pre_processing/visualize.py) as reusable service functions.
- Ensure deterministic preprocessing parameters are configurable in v3 settings (tile size, overlap ratio, shrink factor) and default to ABMIL repo values.
- Add explicit handling for `no_tiles` outcomes with predictable API behavior and observability logs.

1. **Rewire `/v1/predict` to new inference while preserving request contract**

- Keep multipart upload and existing auth dependency chain from [predict.py](D:/imageO/imageO_v2/src/api/v1/endpoints/predict.py).
- Replace call to TensorFlow pipeline from [ml_inference.py](D:/imageO/imageO_v2/src/core/services/ml_inference.py) with new ABMIL service call.
- Keep error surface stable (HTTP status compatibility), while adding internal diagnostic logging.

1. **Configuration, dependency, and runtime hardening**

- Create `pyproject.toml` in v3 aligned to runtime needs of FastAPI + Redis + Torch + Transformers + OpenCV.
- Add v3 settings module based on [config.py](D:/imageO/imageO_v2/src/config.py) with new keys for `MODEL_CHECKPOINT_PATH`, DINO model id, HF token, embedding device/dtype, and preprocessing knobs.
- Build production Dockerfile based on [v2 Dockerfile](D:/imageO/imageO_v2/Dockerfile), adding required system libs for OpenCV/Torch and startup health checks.

1. **Testing and verification gates (before handoff)**

- Add contract tests ensuring v2-compatible endpoint/header requirements and response schema.
- Add deterministic unit tests for security signature validation and image fingerprint sampling compatibility.
- Add inference smoke test that loads checkpoint from env and runs one known image path (or fixture) through full preprocess→embed→ABMIL path.
- Add startup-health tests confirming model load failures fail fast with clear logs.

1. **Operational readiness**

- Add structured logs and readiness checks indicating Redis, DINO, and ABMIL model status.
- Document required env vars, provisioning flow, and rollout checklist (including checkpoint deployment and secrets injection).

## Expected Deliverables in `D:/imageO/imageO_v3`

- Production FastAPI service preserving v2 security/API contract.
- ABMIL-based inference stack with DINOv3 embeddings and new preprocessing.
- Dockerized deployment path and dependency lock.
- Test suite for compatibility + inference smoke validation.
- Runbook-style README for operations and provisioning.