README.md

---
pretty_name: XMR Demo — Industrial Foreign-Object Detection (Lentils) — trained pipelines
license: apache-2.0
library_name: cuvis-ai
pipeline_tag: image-segmentation
tags:
  - hyperspectral
  - hyperspectral-imaging
  - anomaly-detection
  - foreign-object-detection
  - food-safety
  - industrial-inspection
  - dinomaly
  - dinov2
  - cuvis-ai
  - cu3s
  - cubert
  - xmr
---


<p align="center">
  <img src="https://raw.githubusercontent.com/cubert-hyperspectral/cuvis.sdk/main/branding/logo/banner.png" alt="Cubert Hyperspectral" width="560"/>
</p>

<p align="center">
  <a href="https://docs.cuvis.ai"><img src="https://img.shields.io/badge/Docs-docs.cuvis.ai-0aa?logo=readthedocs&logoColor=white" alt="Cuvis.AI docs"/></a>
  <a href="https://github.com/cubert-hyperspectral/cuvis-ai"><img src="https://img.shields.io/badge/GitHub-cuvis--ai-24292e?logo=github" alt="Cuvis.AI on GitHub"/></a>
  <a href="https://huggingface.co/datasets/cubert-gmbh/XMR_Demo_Industrial_Foreign_Object_Detection_Lentils"><img src="https://img.shields.io/badge/Dataset-HuggingFace-ffd21e?logo=huggingface&logoColor=000" alt="Companion dataset"/></a>
</p>

# Trained Dinomaly anomaly-detection pipelines for industrial lentils

Three pre-trained [Cuvis.AI](https://docs.cuvis.ai) pipelines for hyperspectral foreign-object detection on a sliding lentil conveyor. All three use the same architecture (a frozen DINOv2 backbone with a Dinomaly anomaly head and a two-stage gate-then-quantile binary decider); they differ only in **which three bands the channel-selector node feeds the detector**.

The companion [dataset repo](https://huggingface.co/datasets/cubert-gmbh/XMR_Demo_Industrial_Foreign_Object_Detection_Lentils) contains the CU3S session and annotations used by the notebook. This page is the model-side reference: pipeline folders, hparams, weights, and the minimum code to run inference.

## Pipelines

| Subfolder | Selector node | Wavelengths (nm) | Image threshold | Quantile | Weight (.pt) |
|---|---|---|---|---|---|
| `dinomaly_rgb_full_pipeline/` | `FixedWavelengthSelector` | (650, 550, 450) | 0.140 | 0.995 | `dinomaly_multifile_rgb_two_stage.pt` |
| `dinomaly_cir_full_pipeline/` | `CIRSelector` | NIR=860, R=670, G=560 | 0.140 | 0.995 | `dinomaly_multifile_cir.pt` |
| `dinomaly_custom_selector_full_pipeline/` | `FixedWavelengthSelector` | (542, 902, 886) | 0.151 | 0.995 | `dinomaly_multifile_custom_two_stage.pt` |

The `image_threshold` is the gate on the mean of the top-0.1 % per-pixel anomaly scores; only frames above it are pixel-thresholded against the per-frame `quantile` (0.995). Both knobs are persisted in each pipeline YAML and loaded automatically by `restore-pipeline`.

## Pipeline diagram

![Cuvis.AI pipeline — Dinomaly anomaly detection over hyperspectral video](assets/dinomaly_pipeline.png)

1. **LentilsAnomalyDataNode** — reads the `.cu3s` cube, casts to float32, and maps the multi-class GT mask to a binary anomaly mask (used at training/eval; inference reads the cube directly from the dataloader's batch).
2. **MinMaxNormalizer** — running per-cube min-max so the selector sees data in a consistent dynamic range.
3. **rgb_selector / cir_selector** — picks three target wavelengths and emits a 3-channel image. This is the only node that differs between the three methods.
4. **DinomalyDetector** — pre-trained DINOv2 backbone + Dinomaly anomaly head; produces a per-pixel anomaly score map.
5. **TwoStageBinaryDecider** — gates the frame on the mean of the top-k per-pixel scores (image-level threshold), then quantile-thresholds the score map per pixel.

## How to use

The pipelines load with the standard Cuvis.AI restore helpers. Minimal end-to-end snippet:

```python
import torch
from huggingface_hub import hf_hub_download
from cuvis_ai_core.utils.node_registry import NodeRegistry
from cuvis_ai_core.pipeline.pipeline import CuvisPipeline

REPO = "cubert-gmbh/XMR_Demo_Industrial_Foreign_Object_Detection_Lentils"

yaml_path = hf_hub_download(
    repo_id=REPO,
    subfolder="dinomaly_custom_selector_full_pipeline",
    filename="dinomaly_multifile_custom_two_stage.yaml",
)
pt_path = hf_hub_download(
    repo_id=REPO,
    subfolder="dinomaly_custom_selector_full_pipeline",
    filename="dinomaly_multifile_custom_two_stage.pt",
)

# The dinomaly plugin (cuvis-ai-dinomaly v0.1.3) auto-fetches its dependencies
# and the DINOv2 backbone on first use.
registry = NodeRegistry()
registry.load_plugins("configs/plugins/dinomaly.yaml")  # ships with cuvis-ai

device = "cuda" if torch.cuda.is_available() else "cpu"
pipeline = CuvisPipeline.load_pipeline(
    yaml_path,
    weights_path=str(pt_path),
    device=device,
    strict_weight_loading=False,
    node_registry=registry,
)
pipeline.torch_layers.eval()
# ... feed a CU3S batch into pipeline.forward(...)
```

For the full notebook walk-through (preview frames, video export, side-by-side comparison) see `notebooks/use_cases/lentils_dinomaly.ipynb` in the [cuvis-ai repo](https://github.com/cubert-hyperspectral/cuvis-ai).

## Dataset

Recorded on the same product line and conveyor rig described on the companion dataset card:

[**cubert-gmbh/XMR_Demo_Industrial_Foreign_Object_Detection_Lentils**](https://huggingface.co/datasets/cubert-gmbh/XMR_Demo_Industrial_Foreign_Object_Detection_Lentils) (dataset)

Acquisition: Ultris XMR · 50 mm · video mode · 15 ms integration · 4 fps · white & dark references recorded.

## Training details

- **Framework:** [Cuvis.AI](https://github.com/cubert-hyperspectral/cuvis-ai) + [cuvis-ai-dinomaly](https://github.com/cubert-hyperspectral/cuvis-ai-dinomaly) v0.1.3.
- **Backbone:** DINOv2 ViT-B/14 with 4 register tokens (`dinov2reg_vit_base_14`), frozen.
- **Head:** Dinomaly anomaly head from anomalib v2.1.0, image size 448, decoder depth 8, bottleneck dropout 0.2.
- **Loss / optimizer:** Standard Dinomaly training schedule; 50 epochs, weight 1.0 on the train-loss bridge.
- **Decider:** TwoStageBinaryDecider tuned on a held-out NPZ split (pixel F1) — see each yaml's `decider.hparams` for the exact threshold values.
- **Data:** Multi-file conveyor lentil sequences under controlled industrial belt lighting. The published CU3S in the companion dataset is one held-out evaluation session.

## Contact

Trained and packaged by the [AI Team @ Cubert](mailto:cuvis.ai@cubert-gmbh.de). For pilot enquiries on your own product line, reach out via email or the contact details on <https://www.cubert-hyperspectral.com>.