---
license: mit
tags:
- image-to-image
- generative-adversarial-network
- conditional-gan
- pix2pix
- resnet
- pytorch
- grad-cam
- explainability
- computer-vision
- image-synthesis
- image-translation

pipeline_tag: image-to-image
language:
- en
metrics:
- mae
- precision
- recall
- f1
---

# FLAIR → T1 MRI Synthesis with Explainable Residual GANs

## What this solves

In clinical MRI workflows, a patient may have a FLAIR scan but not a T1-weighted scan — re-acquiring is costly, time-consuming, or sometimes impossible. This model **synthesises a high-fidelity T1-weighted image directly from a FLAIR scan** using a conditional GAN, eliminating the need for an additional acquisition.

Beyond synthesis quality, existing models are black boxes. We embed **Grad-CAM into the generator**, producing voxel-level saliency maps that show *which FLAIR regions drove each T1 synthesis* — making the model usable in interpretability-sensitive settings.

---
## Output

![image](https://cdn-uploads.huggingface.co/production/uploads/679f9a4c2ca91c9401e9edd6/ikaJr6M6rm6eiNCPbdU7S.png)

---

## Architecture

| Component | Detail |
|---|---|
| Generator | ResNet-9 encoder-decoder · 9 residual blocks · InstanceNorm · Tanh |
| Discriminator | PatchGAN · 4×4 convolutions · 31×31 output map |
| Input → Output | `(B, 3, 256, 256)` FLAIR → `(B, 3, 256, 256)` T1, normalised to `[-1, 1]` |
| Generator params | 11.4M |
| Discriminator params | 2.8M |
| Interpretability | Grad-CAM on final generator conv layer |

**7-component generator loss:**

$$\mathcal{L}_G = \mathcal{L}_{adv} + 10\mathcal{L}_{L1} + 10\mathcal{L}_{MS\text{-}SSIM} + 10\mathcal{L}_{percep} + 10\mathcal{L}_{feat} + 20\mathcal{L}_{edge} + 5\mathcal{L}_{contrast}$$

Edge loss ($\lambda=20$) combines Sobel + Laplacian at two scales to preserve anatomical boundaries. Perceptual loss uses VGG19 at `relu1_2`, `relu2_2`, `relu3_4`.

---

## Results

**BraTS 2021 held-out test — 251 subjects**

| PSNR (dB) | SSIM | MAE | RMSE | F1 |
|---|---|---|---|---|
| 24.33 `[23.94, 24.74]` | 0.8936 `[0.8928, 0.8946]` | 0.0302 | 0.0719 | 0.9945 |

**BraTS 2023 GLI external validation — 219 subjects (unseen distribution)**

| PSNR (dB) | SSIM | MAE | RMSE |
|---|---|---|---|
| 24.57 `[24.11, 24.98]` | 0.8940 `[0.8871, 0.9003]` | 0.0288 | 0.0670 |

95% CIs via bootstrap resampling (1,000 iterations).

**vs. baselines (same train/test split, BraTS 2021)**

| Method | SSIM | MAE | Params |
|---|---|---|---|
| CycleGAN | 0.798 | 0.042 | ~23M |
| AttentionGAN | 0.795 | 0.043 | ~23M |
| Pix2Pix (U-Net) | 0.885 | 0.031 | 54M |
| **Ours (V6)** | **0.894** | **0.030** | **11.4M** |

5× fewer parameters than Pix2Pix, +12% SSIM and −29% MAE over CycleGAN.

---

## Training evolution

All 6 versions are in this repo under `resnet9/v1/` → `resnet9/v6/`.

| Version | Key change | SSIM | PSNR |
|---|---|---|---|
| V1 | L1 + SSIM baseline | 0.807 | 21.49 |
| V2 | + VGG perceptual, LR decay | 0.850 | 22.26 |
| V3 | Higher perceptual weight, fine-tuning | 0.862 | 21.63 |
| V4 | + Feature matching, LSGAN | 0.876 | 23.85 |
| V5 | + Multi-scale SSIM | 0.886 | 24.05 |
| **V6** | **+ Edge loss, local contrast loss** | **0.894** | **24.33** |

Trained on NVIDIA L4 · 600 cumulative epochs · 4.11 hours · mixed precision FP16/FP32.

---

## Usage

```python
from huggingface_hub import hf_hub_download
import torch

# Load V6 — best model
path = hf_hub_download(
    repo_id="atchusg/flair-to-t1-mri-synthesis",
    filename="resnet9/v6/generator.pth"
)

# Instantiate architecture (copy models.py from GitHub)
from models import ResNet9Generator

gen = ResNet9Generator(in_channels=3, out_channels=3)
gen.load_state_dict(torch.load(path, map_location="cpu"))
gen.eval()

# Inference
# flair: torch.Tensor of shape (B, 3, 256, 256), values in [-1, 1]
with torch.no_grad():
    synthetic_t1 = gen(flair)
```

Code & training scripts: [GitHub](https://github.com/atchudhansg/Explainable-FLAIR-to-T1-MRI-Synthesis)

---

## Repo structure

```
resnet9/
  v1/ → v6/
    generator.pth          # Generator weights only (11.4M params)
    training_report.json   # Full metrics, hyperparams, CI
baselines/
  pix2pix/
  cyclegan/
  attentiongan/
    generator.pth
    training_report.json
```