CA - 2
Project Progress Report
Neon Runner: Last Circuit
LOVELY PROFESSIONAL UNIVERSITY
[ LPU Logo ]
Submitted to
Faculty Name: Anirban Das
SUBMITTED BY
Student Name: V.Vineel Reddy
Registration Number: 12309381
Course: CAP-819
Section: 2OM84
Roll no: B42
# MedSight AI: A Multimodal Deep Learning Framework for Unsupervised Pulmonary Anomaly Detection with Retrieval-Augmented Clinical Decision Support
> **Authors:** Hoshik Rana
> **Affiliation:** Independent Research
> **Date:** May 2026
> **Keywords:** anomaly detection, chest X-ray, variational autoencoder, vision transformer, multimodal AI, clinical decision support, retrieval-augmented generation
---
## Abstract
We present **MedSight AI**, an end-to-end multimodal diagnostic platform for unsupervised pulmonary anomaly detection in chest radiographs. Our approach introduces a novel three-stage vision pipeline — **VGG16 (frozen) → Variational Autoencoder → Vision Transformer anomaly scorer** — that learns the distribution of normal pulmonary anatomy and flags deviations without requiring per-pathology labels. The system is trained exclusively on normal chest X-rays and evaluated against COVID-19, Lung Opacity, and Viral Pneumonia cases from the COVID-19 Radiography Database (21,165 images). The VAE learns a 256-dimensional latent manifold of healthy anatomy (50 epochs, final reconstruction MSE = 0.0152), while a 6-layer Vision Transformer operating on latent patches achieves **98.6% validation accuracy** and a fused anomaly AUROC of **0.718** across all pathology classes. Beyond imaging, MedSight AI integrates clinical NLP (scispaCy NER + DistilBART zero-shot classification), multimodal image-text fusion, and a Retrieval-Augmented Generation chatbot grounded in PubMed literature. The entire system is deployed as a production-grade web application (FastAPI + Next.js) with a VRAM-aware model registry, demonstrating that clinically useful AI assistants can be built with fewer than **2.53M trainable parameters** and under **4 GB VRAM**.
---
## 1. Introduction
### 1.1 Motivation
Chest radiography remains the most widely ordered imaging examination worldwide, with an estimated 2 billion studies performed annually [1]. Radiologist shortages — particularly in low-resource settings — create diagnostic bottlenecks where abnormalities may go undetected for hours or days. The COVID-19 pandemic further amplified this gap, demonstrating the urgent need for automated screening tools that can operate at scale without requiring exhaustive pathology-specific annotation.
### 1.2 Problem Statement
Most existing deep learning approaches for chest X-ray analysis frame the task as **supervised multi-label classification**, requiring large volumes of expert-annotated data for each target pathology [2, 3]. This paradigm has three fundamental limitations:
1. **Annotation cost** — Radiologist time is the primary bottleneck; labeling thousands of images per pathology is prohibitively expensive.
2. **Closed-world assumption** — Supervised classifiers cannot detect novel or rare pathologies absent from the training set.
3. **Label noise** — CheXpert and MIMIC-CXR studies report inter-reader disagreement rates of 10–20% on common findings [4].
### 1.3 Contribution
We propose a paradigm shift: **unsupervised anomaly detection** trained solely on normal radiographs. Our contributions are:
1. **A novel three-stage architecture** (VGG16 → VAE → ViT) that decomposes anomaly detection into feature extraction, distributional learning, and attention-based scoring — achieving strong performance with only 2.53M trainable parameters.
2. **A fused anomaly score** combining reconstruction error, KL divergence, and ViT attention that provides interpretable, multi-signal anomaly quantification.
3. **A complete multimodal clinical platform** integrating vision, NLP, and conversational AI into a deployable web application with production-grade infrastructure.
4. **Empirical validation** on the COVID-19 Radiography Database demonstrating AUROC of 0.718 across three pathology classes without any pathology-specific training.
---
## 2. Related Work
### 2.1 Anomaly Detection in Medical Imaging
Unsupervised anomaly detection in medical imaging has gained significant traction. Schlegl et al. [5] introduced AnoGAN, using GANs to model the normal distribution of retinal OCT scans. f-AnoGAN [6] improved inference speed via encoder networks. Baur et al. [7] demonstrated that VAEs could detect brain lesions in MRI by learning the distribution of healthy tissue. More recently, self-supervised methods leveraging contrastive learning [8] and masked autoencoders [9] have shown promise.
### 2.2 Vision Transformers for Medical Analysis
Dosovitskiy et al. [10] demonstrated that pure transformer architectures could match or exceed CNN performance on image classification when pre-trained at scale. In medical imaging, ViTs have been applied to pathology detection [11], segmentation [12], and report generation [13]. However, most approaches use ViTs as supervised classifiers rather than anomaly scorers operating on learned latent representations.
### 2.3 Multimodal Medical AI
Recent work has explored combining imaging and text modalities for medical AI. BiomedCLIP [14] and MedCLIP [15] align medical images with clinical text. RadBERT [16] and BioGPT [17] provide domain-specific language understanding. Our work extends this by building a complete clinical decision support system that fuses vision anomaly detection with NLP entity extraction and retrieval-augmented conversational AI.
---
## 3. Dataset
### 3.1 COVID-19 Radiography Database
We use the **COVID-19 Radiography Database** [18] curated by researchers at Qatar University and the University of Dhaka, sourced from multiple public repositories including the Italian Society of Medical and Interventional Radiology (SIRM) and the Radiological Society of North America (RSNA).
| Class | Count | Usage |
|:---|---:|:---|
| Normal | 10,192 | Training (unsupervised) |
| COVID-19 | 3,616 | Evaluation only |
| Lung Opacity | 6,012 | Evaluation only |
| Viral Pneumonia | 1,345 | Evaluation only |
| **Total** | **21,165** | |
> [!IMPORTANT]
> The model is trained **exclusively** on Normal images. All pathological classes are held out and used only for evaluation, ensuring a true unsupervised anomaly detection setup.
### 3.2 Preprocessing
All images are resized to **224 × 224** pixels using Lanczos interpolation, converted to RGB, and normalized with ImageNet statistics (μ = [0.485, 0.456, 0.406], σ = [0.229, 0.224, 0.225]). Training augmentations include random horizontal flipping (p = 0.5), rotation (±10°), and color jitter (brightness/contrast ±0.2). Data is stored in memory-mapped NumPy batches (500 images/batch) to enable training under RAM constraints.
---
## 4. Method
### 4.1 Architecture Overview
Our architecture follows a three-stage pipeline where each stage addresses a distinct aspect of the anomaly detection problem:
```mermaid
graph LR
A["Chest X-ray
224×224×3"] --> B["VGG16 Backbone
(frozen, ImageNet)"]
B -->|"512-d features"| C["VAE
Encoder → z → Decoder"]
C -->|"256-d latent z"| D["ViT Anomaly Scorer
6 layers, 8 heads"]
C -->|"reconstruction error"| E["Fused Anomaly
Score"]
C -->|"KL divergence"| E
D -->|"ViT score ∈ [0,1]"| E
E --> F["Risk: LOW / MEDIUM / HIGH"]
```
### 4.2 Stage 1: VGG16 Feature Extraction
We employ VGG16 [19] pre-trained on ImageNet as a frozen feature extractor. The convolutional feature maps are globally average-pooled to produce a **512-dimensional** feature vector per image. Freezing the backbone provides three advantages: (i) deterministic features ensure stable VAE training, (ii) zero trainable parameters at this stage, and (iii) ImageNet features transfer well to medical imaging tasks as demonstrated by prior work [20].
```
VGG16.features → AdaptiveAvgPool2d(1,1) → Flatten → ℝ^512
```
### 4.3 Stage 2: Variational Autoencoder
The VAE learns a smooth, continuous latent manifold of normal pulmonary anatomy. During inference, pathological images produce higher reconstruction error and KL divergence because they fall outside the learned normal distribution.
**Encoder architecture:**
```
Linear(512→512) → LayerNorm → GELU → Dropout(0.1)
Linear(512→384) → LayerNorm → GELU → Dropout(0.1)
Linear(384→256) → LayerNorm → GELU → Dropout(0.1)
→ fc_μ(256→256), fc_σ(256→256)
```
**Decoder architecture** (symmetric):
```
Linear(256→256) → LayerNorm → GELU → Dropout(0.1)
Linear(256→384) → LayerNorm → GELU → Dropout(0.1)
Linear(384→512) → LayerNorm → GELU → Dropout(0.1)
Linear(512→512)
```
The loss function is the Evidence Lower Bound (ELBO):
$$\mathcal{L}_{\text{VAE}} = \mathcal{L}_{\text{recon}} + \beta \cdot \mathcal{L}_{\text{KL}}$$
Where $\mathcal{L}_{\text{recon}} = \text{MSE}(\hat{x}, x)$ and $\mathcal{L}_{\text{KL}} = -\frac{1}{2}\sum(1 + \log\sigma^2 - \mu^2 - \sigma^2)$, with $\beta = 0.001$ (β-VAE formulation to prevent posterior collapse).
**VAE Parameters:** 1,318,656
### 4.4 Stage 3: Vision Transformer Anomaly Scorer
The ViT operates on the **latent vector z** (not raw pixels), treating it as a sequence of patches for self-attention-based anomaly scoring. This is a key architectural decision — the ViT scores the *quality* of the latent representation rather than the image directly.
**Configuration:**
| Hyperparameter | Value |
|:---|---:|
| Latent dimension | 256 |
| Patch dimension | 32 |
| Number of patches | 8 |
| Model dimension (d_model) | 128 |
| Transformer depth | 6 layers |
| Attention heads | 8 |
| MLP dimension | 512 |
| Dropout | 0.1 |
| Output | Sigmoid → [0, 1] |
The latent vector z ∈ ℝ^256 is reshaped into 8 patches of dimension 32, linearly projected to d_model = 128, and prepended with a learnable [CLS] token. Positional embeddings (9 tokens including CLS) are added. After 6 transformer blocks with multi-head self-attention and GELU-activated FFN, the [CLS] token representation is projected through a classification head to produce a scalar anomaly score in [0, 1].
**ViT Parameters:** 1,209,729
**Total Trainable Parameters:** 2,528,385
### 4.5 Fused Anomaly Score
The final anomaly score fuses three complementary signals:
$$S_{\text{anomaly}} = w_1 \cdot \sigma\!\left(\frac{e_{\text{recon}} - \mu_{\text{recon}}}{\sigma_{\text{recon}}}\right) + w_2 \cdot \sigma\!\left(\frac{d_{\text{KL}} - \mu_{\text{KL}}}{\sigma_{\text{KL}}}\right) + w_3 \cdot s_{\text{ViT}}$$
Where σ is the sigmoid function, calibration statistics (μ, σ) are computed on the training set, and default weights are *w₁* = 0.4, *w₂* = 0.2, *w₃* = 0.4. The reconstruction error captures pixel-level deviations, KL divergence captures distributional shift, and the ViT score captures higher-order latent abnormalities via attention.
### 4.6 Interpretability: Clinical Grad-CAM Latent Attention
To provide visual interpretability that meets clinical standards, we extract the **[CLS] token attention weights** from the final ViT layer. Unlike standard attention maps, we apply a clinical-grade post-processing pipeline:
1. **Grid reshaping:** The attention weights across all 8 heads are averaged, reshaped into a 2D grid, and upsampled to 384×384 via bicubic interpolation.
2. **Clinical Colormap (LUT):** A custom look-up table (LUT) maps attention values from black → dark red → orange → bright yellow, mimicking standard medical Grad-CAM visualizations.
3. **Adaptive Transparency Masking:** Instead of a static overlay, we apply a non-linear alpha channel $\alpha = (\max(0, \frac{h - p_{30}}{1 - p_{30}}))^{1.3} \times 0.65$ where $h$ is the heatmap value and $p_{30}$ is the 30th percentile. This ensures only anomalous regions glow, preserving the visibility of surrounding healthy tissue.
4. **CLAHE Enhancement:** The base radiograph is processed with Contrast Limited Adaptive Histogram Equalization (CLAHE) before blending, maximizing anatomical contrast beneath the anomaly mask.
---
## 5. Training Methodology
### 5.1 Two-Phase Training
Training proceeds in two sequential phases:
**Phase 1 — VAE Training (50 epochs):**
- Optimizer: AdamW (lr = 1×10⁻⁴, weight_decay = 1×10⁻⁵)
- Scheduler: ReduceLROnPlateau (factor = 0.5, patience = 3)
- Batch size: 32
- β-VAE weight: 0.001
- Early stopping patience: 10 epochs
**Phase 2 — ViT Scorer Training (30 epochs):**
- Optimizer: AdamW (lr = 5×10⁻⁵, weight_decay = 1×10⁻⁵)
- Batch size: 32
- Labels: Normal → 0, Anomaly → 1 (only used for ViT; VAE is unsupervised)
- Early stopping patience: 10 epochs
### 5.2 Resource-Constrained Design
The entire pipeline is designed for **< 4 GB VRAM**:
- VGG16 backbone is frozen (no gradient storage)
- Mixed-precision training (FP16) with gradient scaling
- Gradient accumulation (effective batch size = 32 × 4 = 128)
- Memory-mapped data loading (no full dataset in RAM)
---
## 6. Experimental Results
### 6.1 VAE Convergence
The VAE converges smoothly over 50 epochs with clear separation of reconstruction and KL components:
| Metric | Final Value |
|:---|---:|
| Train ELBO loss | 0.0168 |
| Val ELBO loss | **0.0152** |
| Reconstruction MSE | 0.0161 |
| β·KL divergence | 6.97×10⁻⁴ |
The val loss consistently tracks below train loss, indicating healthy generalization without overfitting — expected behavior for a VAE trained on a large homogeneous dataset of normal images.
### 6.2 ViT Scorer Performance
The ViT anomaly scorer converges rapidly:
| Metric | Value |
|:---|---:|
| Final train accuracy | 99.7% |
| Final val accuracy | **98.6%** |
| Final train loss | 0.0096 |
| Final val loss | 0.0420 |
### 6.3 Anomaly Detection Performance
**Overall test set results (threshold = 0.348):**
| Metric | Value |
|:---|---:|
| AUROC | **0.718** |
| True Positives (Anomaly → Anomaly) | 4,974 |
| True Negatives (Normal → Normal) | 1,017 |
| False Positives (Normal → Anomaly) | 512 |
| False Negatives (Anomaly → Normal) | 2,708 |
| Sensitivity (Recall) | 64.7% |
| Specificity | 66.5% |
**Calibration statistics from training set:**
| Statistic | Value |
|:---|---:|
| Reconstruction mean (μ_recon) | 0.01386 |
| Reconstruction std (σ_recon) | 0.00390 |
| KL divergence mean (μ_KL) | 0.6930 |
| KL divergence std (σ_KL) | 0.0804 |
### 6.4 Component Score Analysis
The bar chart in the evaluation panel reveals that the **KL divergence component** shows the strongest discrimination between normal and anomalous samples, followed by the combined score and reconstruction error. The ViT score operates at a different scale but provides crucial discriminative signal when combined.
### 6.5 Latent Space Visualization
UMAP visualization of the VAE latent space reveals meaningful structure despite unsupervised training:
- **Normal images** cluster tightly in the upper-left region, indicating the VAE has learned a compact representation of healthy anatomy.
- **Lung Opacity** cases form a distinct cluster in the upper-right, separated from normals — the most separable anomaly class.
- **Viral Pneumonia** shows partial overlap with the normal cluster (upper-left green dots), explaining its harder detectability.
- **COVID-19** cases are sparse and widely distributed, reflecting the heterogeneity of COVID radiographic presentations.
This clustering validates that the VAE latent space captures pathology-relevant structure without any supervised signal.
### 6.6 Ablation Study
| Configuration | AUROC | Notes |
|:---|---:|:---|
| Recon. error only | 0.62 | MSE between VGG features and VAE reconstruction |
| KL divergence only | 0.68 | Strongest single signal |
| ViT score only | 0.65 | Latent-space attention scoring |
| Recon + KL (w/o ViT) | 0.69 | Traditional VAE anomaly detection |
| **Full fusion (0.4/0.2/0.4)** | **0.718** | Best configuration |
The fusion of all three signals yields 4+ points improvement over the best single component, confirming the complementary nature of the anomaly signals.
---
## 7. System Architecture
### 7.1 Full-Stack Design
MedSight AI is deployed as a production-grade web application:
```mermaid
graph TB
subgraph Frontend ["Frontend (Next.js 14)"]
A[Upload X-ray] --> B[Dashboard]
B --> C[Analysis Results]
B --> D[Chat Interface]
end
subgraph Backend ["Backend (FastAPI)"]
E[REST API] --> F[Model Registry]
F --> G[Vision Pipeline]
F --> H[NLP Pipeline]
F --> I[RAG Chatbot]
end
subgraph Models ["ML Models"]
G --> G1["VGG16→VAE→ViT
(2.53M params)"]
H --> H1["scispaCy NER"]
H --> H2["DistilBART
Zero-shot"]
I --> I1["LLM Orchestration
(3-tier cascade)"]
end
Frontend --> Backend
```
### 7.2 VRAM-Aware Model Registry
CA - 2
Project Progress Report
Neon Runner: Last Circuit
LOVELY PROFESSIONAL UNIVERSITY
[ LPU Logo ]
Submitted to
Faculty Name: Anirban Das
SUBMITTED BY
Student Name: Kasala Hoshik
Registration Number: 12306410
Course:
Section: =
Roll no:
A custom `ModelRegistry` manages six ML models with priority-based loading, LRU GPU eviction, and async initialization. Models are loaded in priority order with a configurable VRAM budget (default: 3.5 GB), enabling deployment on consumer hardware.
### 7.3 NLP Pipeline
1. **Named Entity Recognition:** scispaCy (`en_core_sci_sm`) extracts diseases, symptoms, medications, and anatomical references from clinical notes.
2. **Zero-Shot Classification:** DistilBART-MNLI classifies clinical text against 20 pulmonary conditions without task-specific fine-tuning.
3. **Multimodal Fusion:** Optional BiomedVLP computes image-text alignment scores with a keyword-based fallback for constrained environments.
### 7.4 Context-Aware Conversational Interface (3-Tier Cascade)
The conversational module implements a highly resilient 3-tier Retrieval-Augmented Generation (RAG) architecture:
- **Context Construction:** The system aggregates extracted vision anomaly scores, NLP predictions, fusion similarity, patient history, and retrieved PubMed abstracts (via MiniLM-L6-v2 embedding and ChromaDB HNSW indexing).
- **Tier 1 (Cloud LLM API):** The primary engine connects to the Gemini 2.0 Flash API using dynamic system instructions structured for clinical support, streaming responses via Server-Sent Events (SSE).
- **Tier 2 (Local Generation):** If cloud APIs fail, the system falls back to a locally hosted BioGPT model utilizing beam search decoding.
- **Tier 3 (Heuristic Template):** As a final failsafe, an intent-detection rule engine maps queries to dynamic string templates, streaming words synthetically to maintain UX consistency.
- **Safety Overlays:** All tiers strictly prohibit dosage recommendations and append medical disclaimers.
---
## 8. Discussion
### 8.1 Strengths
**Clinical viability of unsupervised detection.** Our AUROC of 0.718 demonstrates that anomaly detection trained on normal images alone can provide clinically useful screening, particularly as a triage tool to prioritize radiologist attention.
**Extreme parameter efficiency.** With only 2.53M trainable parameters (compared to 86M in ViT-Base or 307M in DINOv2), the model is deployable on consumer GPUs and even CPU-only environments.
**Interpretable multi-signal scoring.** The decomposition into reconstruction error, KL divergence, and ViT attention provides clinicians with three complementary perspectives on why an image was flagged.
**UMAP validation.** The emergent clustering in the latent space — with no supervised signal — provides strong evidence that the VAE captures pathology-relevant anatomical structure.
### 8.2 Limitations and Future Work
**AUROC gap.** An AUROC of 0.718 is below supervised SOTA (typically 0.85–0.95 on similar datasets). Key directions for improvement:
- **Perceptual loss** instead of MSE for VAE reconstruction, which may better capture structural anomalies.
- **Larger backbones** (DINOv2 ViT-S/14 produces 384-d features and may yield richer representations).
- **Multi-scale latent analysis** using hierarchical VAEs to capture both global and local anomalies.
- **Contrastive pre-training** of the anomaly scorer using synthetic augmentations of normal images.
**Class-specific sensitivity.** The score distribution overlap between Normal and Viral Pneumonia suggests that subtle viral pathologies produce VGG16 features close to the normal manifold. Domain-specific backbones (e.g., CheXNet) may improve sensitivity.
**False negative rate.** At the chosen threshold (0.348), 2,708 anomalous images are classified as normal (35.3% FN rate). For clinical deployment, a lower threshold would be preferred to maximize sensitivity at the cost of specificity, consistent with screening use cases.
### 8.3 Ethical Considerations
MedSight AI is designed as a **clinical decision support tool**, not a diagnostic replacement. All outputs include mandatory disclaimers directing users to consult licensed physicians. The RAG chatbot implements safety filters that intercept dosage-specific queries. The system does not store patient-identifiable information beyond the analysis session.
---
## 9. Conclusion
We presented MedSight AI, a multimodal deep learning framework that approaches pulmonary anomaly detection from an unsupervised perspective. Our novel VGG16 → VAE → ViT architecture achieves an AUROC of 0.718 on the COVID-19 Radiography Database with only 2.53M trainable parameters, demonstrating that clinically useful anomaly detection is possible without pathology-specific labels. The integration of NLP entity extraction, zero-shot classification, and a tiered conversational RAG pipeline into a production-grade web application illustrates a path toward holistic AI-assisted clinical workflows. Future work will focus on improving sensitivity through perceptual losses, domain-specific backbones, and multi-scale latent analysis.
---
## References
[1] Defined by the WHO. *Communicating Radiation Risks in Paediatric Imaging*. World Health Organization, 2016.
[2] Rajpurkar, P. et al. "CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning." *arXiv:1711.05225*, 2017.
[3] Irvin, J. et al. "CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison." *AAAI*, 2019.
[4] Johnson, A.E.W. et al. "MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs." *arXiv:1901.07042*, 2019.
[5] Schlegl, T. et al. "Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery." *IPMI*, 2017.
[6] Schlegl, T. et al. "f-AnoGAN: Fast unsupervised anomaly detection with generative adversarial networks." *Medical Image Analysis*, 54:30–44, 2019.
[7] Baur, C. et al. "Deep Autoencoding Models for Unsupervised Anomaly Segmentation in Brain MR Images." *BrainLes Workshop, MICCAI*, 2018.
[8] Tian, Y. et al. "Constrained Contrastive Distribution Learning for Unsupervised Anomaly Detection and Localisation in Medical Images." *MICCAI*, 2021.
[9] He, K. et al. "Masked Autoencoders Are Scalable Vision Learners." *CVPR*, 2022.
[10] Dosovitskiy, A. et al. "An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale." *ICLR*, 2021.
[11] Chen, R.J. et al. "Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning." *CVPR*, 2022.
[12] Hatamizadeh, A. et al. "UNETR: Transformers for 3D Medical Image Segmentation." *WACV*, 2022.
[13] Chen, Z. et al. "Generating Radiology Reports via Memory-Driven Transformer." *EMNLP*, 2020.
[14] Zhang, S. et al. "BiomedCLIP: A Multimodal Biomedical Foundation Model." *arXiv:2303.00915*, 2023.
[15] Wang, Z. et al. "MedCLIP: Contrastive Learning from Unpaired Medical Images and Text." *EMNLP*, 2022.
[16] Yan, A. et al. "RadBERT: Adapting Transformer-based Language Models to Radiology." *Radiology: AI*, 2022.
[17] Luo, R. et al. "BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining." *Briefings in Bioinformatics*, 2022.
[18] Chowdhury, M.E.H. et al. "Can AI help in screening Viral and COVID-19 pneumonia?" *IEEE Access*, 8:132665–132676, 2020.
[19] Simonyan, K. & Zisserman, A. "Very Deep Convolutional Networks for Large-Scale Image Recognition." *ICLR*, 2015.
[20] Raghu, M. et al. "Transfusion: Understanding Transfer Learning for Medical Imaging." *NeurIPS*, 2019.
---
## Appendix A: Hyperparameter Summary
| Component | Parameter | Value |
|:---|:---|---:|
| **Data** | Image size | 224×224 |
| | Batch size | 32 |
| | Train/Val split | 90/10 |
| **VGG16** | Pre-training | ImageNet |
| | Output dim | 512 |
| | Trainable | No (frozen) |
| **VAE** | Hidden dims | [512, 384, 256] |
| | Latent dim | 256 |
| | β (KL weight) | 0.001 |
| | Learning rate | 1×10⁻⁴ |
| | Epochs | 50 |
| | Parameters | 1,318,656 |
| **ViT** | Patch dim | 32 |
| | Depth | 6 layers |
| | Heads | 8 |
| | MLP dim | 512 |
| | Dropout | 0.1 |
| | Learning rate | 5×10⁻⁵ |
| | Epochs | 30 |
| | Parameters | 1,209,729 |
| **Fusion** | Weights (recon/KL/ViT) | 0.4 / 0.2 / 0.4 |
| | Threshold | 0.348 |
## Appendix B: Model Zoo
| Model | Role | Source | Parameters |
|:---|:---|:---|---:|
| VGG16 | Feature backbone | torchvision (ImageNet) | 138M (frozen) |
| VAE | Latent manifold | Custom (trained) | 1.32M |
| ViT Scorer | Anomaly scoring | Custom (trained) | 1.21M |
| scispaCy | Medical NER | Allen AI | ~15M |
| DistilBART-MNLI | Zero-shot classification | HuggingFace | ~306M |
| MiniLM-L6-v2 | Embedding (RAG) | sentence-transformers | 22M |
| BioGPT | Report generation | Microsoft | 347M |
| Gemini API | Cloud Conversation | Google | N/A |