File size: 2,881 Bytes
cd99f4e f7038f8 cd99f4e 8a1b910 cd99f4e f7038f8 cd99f4e | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 | ---
license: apache-2.0
library_name: diffusers
pipeline_tag: image-to-image
tags:
- zoomldm
- histopathology
- brca
- latent-diffusion
- custom-pipeline
- arxiv:2411.16969
widget:
- src: demo_images/input.jpeg
prompt: BRCA sample conditioned on demo SSL feature (mag=0)
output:
url: demo_images/output.jpeg
---
> [!WARNING] we do not have a full checkpoint conversion validation, if you encounter pipeline loading failure and unsidered output, please contact me via bili_sakura@zju.edu.cn
# BiliSakura/ZoomLDM-brca
Diffusers-format **BRCA** variant of ZoomLDM with a bundled custom pipeline and local `ldm` modules.
## Model Description
- **Architecture:** ZoomLDM latent diffusion pipeline (`UNet + VAE + conditioning encoder`)
- **Domain:** Histopathology (BRCA)
- **Conditioning:** UNI-style SSL feature maps + magnification level (`0..4`)
- **Format:** Self-contained local folder for `DiffusionPipeline.from_pretrained(...)`
## Intended Use
Use this model for conditional multi-scale BRCA patch generation when you have compatible pre-extracted SSL features.
## Out-of-Scope Use
- Not intended for diagnosis, treatment planning, or other clinical decisions.
- Not a general-purpose text-to-image model.
- Not validated for data outside the expected acquisition/distribution range.
## Files
- `unet/`, `vae/`, `conditioning_encoder/`, `scheduler/`
- `model_index.json`
- `pipeline_zoomldm.py`
- `ldm/` (bundled dependency modules)
## Usage
```python
import torch
from diffusers import DiffusionPipeline
pipe = DiffusionPipeline.from_pretrained(
"BiliSakura/ZoomLDM-brca",
custom_pipeline="pipeline_zoomldm.py",
trust_remote_code=True,
).to("cuda")
out = pipe(
ssl_features=ssl_feat_tensor.to("cuda"), # BRCA UNI-style SSL embeddings
magnification=torch.tensor([0]).to("cuda"), # 0..4
num_inference_steps=50,
guidance_scale=2.0,
)
images = out.images
```
## Demo Generation (dataset-backed)
This repo includes `run_demo_inference.py`, which uses local repo assets only:
- image: `demo_images/input.jpeg`
- SSL feature: `demo_data/0_ssl_feat.npy`
- magnification label: `0`
Run:
```bash
python run_demo_inference.py
```
## Limitations
- Requires correctly precomputed BRCA conditioning features.
- Magnification conditioning must match expected integer codes.
- Generated content may reflect biases and artifacts from training data.
## Citation
```bibtex
@InProceedings{Yellapragada_2025_CVPR,
author = {Yellapragada, Srikar and Graikos, Alexandros and Triaridis, Kostas and Prasanna, Prateek and Gupta, Rajarsi and Saltz, Joel and Samaras, Dimitris},
title = {ZoomLDM: Latent Diffusion Model for Multi-scale Image Generation},
booktitle = {Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR)},
month = {June},
year = {2025},
pages = {23453-23463}
}
```
|