atchusg/flair-to-t1-mri-synthesis

FLAIR → T1 MRI Synthesis with Explainable Residual GANs

What this solves

In clinical MRI workflows, a patient may have a FLAIR scan but not a T1-weighted scan — re-acquiring is costly, time-consuming, or sometimes impossible. This model synthesises a high-fidelity T1-weighted image directly from a FLAIR scan using a conditional GAN, eliminating the need for an additional acquisition.

Beyond synthesis quality, existing models are black boxes. We embed Grad-CAM into the generator, producing voxel-level saliency maps that show which FLAIR regions drove each T1 synthesis — making the model usable in interpretability-sensitive settings.

---

Output

---

Architecture

Component	Detail
Generator	ResNet-9 encoder-decoder · 9 residual blocks · InstanceNorm · Tanh
Discriminator	PatchGAN · 4×4 convolutions · 31×31 output map
Input → Output	(B, 3, 256, 256) FLAIR → (B, 3, 256, 256) T1, normalised to [-1, 1]
Generator params	11.4M
Discriminator params	2.8M
Interpretability	Grad-CAM on final generator conv layer

7-component generator loss:

$${\mathcal{L}}_G={\mathcal{L}}_{adv}+10{\mathcal{L}}_{L1}+10{\mathcal{L}}_{MS\text{-}SSIM}+10{\mathcal{L}}_{percep}+10{\mathcal{L}}_{feat}+20{\mathcal{L}}_{edge}+5{\mathcal{L}}_{contrast}\backslash mathcal\{L\}\_G\; =\; \backslash mathcal\{L\}\_\{adv\}\; +\; 10\backslash mathcal\{L\}\_\{L1\}\; +\; 10\backslash mathcal\{L\}\_\{MS\backslash text\{-\}SSIM\}\; +\; 10\backslash mathcal\{L\}\_\{percep\}\; +\; 10\backslash mathcal\{L\}\_\{feat\}\; +\; 20\backslash mathcal\{L\}\_\{edge\}\; +\; 5\backslash mathcal\{L\}\_\{contrast\}$$

Edge loss ($\lambda=20$) combines Sobel + Laplacian at two scales to preserve anatomical boundaries. Perceptual loss uses VGG19 at relu1_2, relu2_2, relu3_4.

---

Results

BraTS 2021 held-out test — 251 subjects

PSNR (dB)	SSIM	MAE	RMSE	F1
24.33 [23.94, 24.74]	0.8936 [0.8928, 0.8946]	0.0302	0.0719	0.9945

BraTS 2023 GLI external validation — 219 subjects (unseen distribution)

PSNR (dB)	SSIM	MAE	RMSE
24.57 [24.11, 24.98]	0.8940 [0.8871, 0.9003]	0.0288	0.0670

95% CIs via bootstrap resampling (1,000 iterations).

vs. baselines (same train/test split, BraTS 2021)

Method	SSIM	MAE	Params
CycleGAN	0.798	0.042	~23M
AttentionGAN	0.795	0.043	~23M
Pix2Pix (U-Net)	0.885	0.031	54M
Ours (V6)	0.894	0.030	11.4M

5× fewer parameters than Pix2Pix, +12% SSIM and −29% MAE over CycleGAN.

---

Training evolution

All 6 versions are in this repo under resnet9/v1/ → resnet9/v6/.

Version	Key change	SSIM	PSNR
V1	L1 + SSIM baseline	0.807	21.49
V2	+ VGG perceptual, LR decay	0.850	22.26
V3	Higher perceptual weight, fine-tuning	0.862	21.63
V4	+ Feature matching, LSGAN	0.876	23.85
V5	+ Multi-scale SSIM	0.886	24.05
V6	+ Edge loss, local contrast loss	0.894	24.33

Trained on NVIDIA L4 · 600 cumulative epochs · 4.11 hours · mixed precision FP16/FP32.

---

Usage

from huggingface_hub import hf_hub_download
import torch

# Load V6 — best model
path = hf_hub_download(
    repo_id="atchusg/flair-to-t1-mri-synthesis",
    filename="resnet9/v6/generator.pth"
)

# Instantiate architecture (copy models.py from GitHub)
from models import ResNet9Generator

gen = ResNet9Generator(in_channels=3, out_channels=3)
gen.load_state_dict(torch.load(path, map_location="cpu"))
gen.eval()

# Inference
# flair: torch.Tensor of shape (B, 3, 256, 256), values in [-1, 1]
with torch.no_grad():
    synthetic_t1 = gen(flair)

Code & training scripts: GitHub

---

Repo structure

resnet9/
  v1/ → v6/
    generator.pth          # Generator weights only (11.4M params)
    training_report.json   # Full metrics, hyperparams, CI
baselines/
  pix2pix/
  cyclegan/
  attentiongan/
    generator.pth
    training_report.json