| --- |
| license: apache-2.0 |
| library_name: konfai |
| pipeline_tag: image-to-image |
| tags: |
| - medical-imaging |
| - registration |
| - deformable |
| - multimodal |
| - ct |
| - mri |
| - cbct |
| - konfai |
| --- |
| |
| # IMPACT-Reg — Multimodal Medical Image Registration |
|
|
| Robust **multimodal** (**MR / CT / CBCT**) deformable **registration** presets, built with |
| [**KonfAI**](https://github.com/vboussot/KonfAI). Alignment is driven by the **IMPACT** semantic |
| similarity metric — deep features from pretrained segmentation / foundation models (**MIND**, |
| **TotalSegmentator**, **MRSegmentator**) — so cross-modality pairs align while the deformation |
| stays smooth and physically plausible. |
|
|
| Each preset is a **self-contained KonfAI app**: on the **fixed** grid it produces the moving image |
| resampled onto the fixed image (**`MovedImage`**) and the **`DisplacementField`**. Presets can be |
| **ensembled** (their displacement fields are averaged into one transform). |
|
|
| ## 🧩 Presets |
|
|
| | Preset | Pair | Engine | Description | |
| |:--|:--|:--|:--| |
| | `Generic_Rigid` | any | elastix | Rigid alignment (mutual information, multi-resolution) | |
| | `Generic_Rigid_BSpline` | any | elastix | Rigid, then B-spline deformable refinement | |
| | `MR_CT_HeadNeck` | MR/CT | elastix + IMPACT | MR/CT head & neck preset | |
| | `MR_CT_TS` | MR/CT | elastix + IMPACT | MR/CT with MIND + TotalSegmentator features | |
| | `MR_CT_MRSeg` | MR/CT | elastix + IMPACT | MR/CT with MIND + MRSegmentator features | |
| | `CBCT_CT_HeadNeck` | CBCT/CT | elastix + IMPACT | CBCT/CT head & neck preset | |
| | `CBCT_CT_MRSeg` | CBCT/CT | elastix + IMPACT | CBCT/CT with MRSegmentator features | |
| | `ConvexAdam_Coarse` | any | itk-impact (native) | Global coarse coupled-convex init (IMPACT/MIND) | |
| | `ConvexAdam_Fine` | any | itk-impact (native) | Adam instance-optimisation (tileable; expects a pre-aligned start) | |
| | `ConvexAdam_Composite` | any | itk-impact (native) | Coarse + fine ConvexAdam in one app (IMPACT/MIND) | |
| | `FireANTs_Affine` | any | FireANTs (native) | Rigid + Affine linear alignment (Riemannian Adam, GPU) | |
| | `FireANTs_SyN` | any | FireANTs (native) | Rigid + Affine + deformable — SyN or Greedy via `deformable_method` (Riemannian Adam, GPU) | |
| | `FireANTs_IMPACT` | any | FireANTs + IMPACT | Rigid + Affine + SyN driven by the IMPACT deep-feature metric (multi-model, GPU) | |
|
|
| Inputs: **Fixed**, **Moving**, and optional **FixedMask** / **MovingMask** (restrict the metric region). |
|
|
| ## 🚀 Usage |
|
|
| ```bash |
| pip install impact-reg-konfai |
| # Register a moving image onto a fixed image (ensemble several presets by listing them): |
| impact-reg-konfai register ConvexAdam_Composite -f fixed.nii.gz -m moving.nii.gz -o ./Output --gpu 0 |
| ``` |
|
|
| - **Generic runner (single preset):** `konfai-apps infer VBoussot/ImpactReg:ConvexAdam_Composite -i fixed.nii.gz -i moving.nii.gz -o output/` |
| - **Interactive:** [**SlicerImpactReg**](https://github.com/vboussot/SlicerImpactReg) — a 3D Slicer extension driving these presets. |
|
|
| > The `ConvexAdam_*` presets depend on [`itk-impact`](https://pypi.org/project/itk-impact/); resolving the app installs it automatically (it reuses your existing PyTorch, CPU or GPU). |
| > |
| > The `FireANTs_*` presets depend on [`fireants`](https://pypi.org/project/fireants/) (installed at resolve time; **GPU required**). FireANTs is distributed under the **FireANTs License v1.0** — this app calls its public API without copying its source, and ships its license and citation in each preset's `NOTICE`; please cite FireANTs if you use them. `FireANTs_IMPACT` additionally drives the deformable stage with the KonfAI **IMPACT** metric, fetching its feature models from `VBoussot/impact-torchscript-models`. |
| |
| ## ⚡ Performance & VRAM |
| |
| `ConvexAdam` presets (native, GPU) benchmarked on an **NVIDIA RTX PRO 5000 (24 GB)** with a real |
| abdomen **MR→CT** pair, **222 × 226 × 124 @ 2 mm** (single pass, no TTA): |
| |
| | Preset | Stages | Time / case | Peak VRAM | |
| |:--|:--|:--:|:--:| |
| | `ConvexAdam_Fine` | fine (150 Adam iters) | **≈ 0.5 s** | ~2.1 GB | |
| | `ConvexAdam_Coarse` | linear + coarse | **≈ 4.6 s** | ~2.1 GB | |
| | `ConvexAdam_Composite` | linear + coarse + fine | **≈ 5.1 s** | ~2.1 GB | |
| |
| Per-stage breakdown: linear pre-align **≈ 4.2 s** (ITK affine, MI) · coarse **≈ 0.4 s** · fine **≈ 0.5 s**. |
| One-time TorchScript feature-model load **≈ 7 s** (amortised across a batch). Times scale with case size; |
| `--tta k` multiplies runtime. The `elastix + IMPACT` presets run through elastix and scale differently. |
| |
| ## 🔗 Links & Citation |
| |
| - 🧠 **KonfAI:** [github.com/vboussot/KonfAI](https://github.com/vboussot/KonfAI) |
| - 📦 **PyPI:** [impact_reg_konfai](https://pypi.org/project/impact_reg_konfai/) |
| - 🩻 **Slicer:** [SlicerImpactReg](https://github.com/vboussot/SlicerImpactReg) |
| - 📄 **Paper:** KonfAI — [arXiv:2508.09823](https://arxiv.org/abs/2508.09823) |
| - 🔥 **FireANTs** (used by `FireANTs_SyN`): [github.com/rohitrango/FireANTs](https://github.com/rohitrango/FireANTs) — Jena et al., *FireANTs: Adaptive Riemannian Optimization for Multi-Scale Diffeomorphic Registration*, Nature Communications, 2024. See [`FireANTs_SyN/NOTICE`](FireANTs_SyN/NOTICE) for the full citation and license. |
| </content> |
| </invoke> |
| |
