Summarization
Transformers
Safetensors
English
bart
text2text-generation
radiology
medical
healthcare
Eval Results (legacy)
Instructions to use Kumud2k16/radiology-expression-summarizer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Kumud2k16/radiology-expression-summarizer with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "summarization" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # 'pip install "transformers<5.0.0' from transformers import pipeline pipe = pipeline("summarization", model="Kumud2k16/radiology-expression-summarizer")# Load model directly from transformers import AutoTokenizer, AutoModelForSeq2SeqLM tokenizer = AutoTokenizer.from_pretrained("Kumud2k16/radiology-expression-summarizer") model = AutoModelForSeq2SeqLM.from_pretrained("Kumud2k16/radiology-expression-summarizer", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,6 +1,10 @@
|
|
| 1 |
---
|
| 2 |
language: en
|
| 3 |
license: mit
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
tags:
|
| 5 |
- summarization
|
| 6 |
- radiology
|
|
@@ -17,6 +21,9 @@ model-index:
|
|
| 17 |
- task:
|
| 18 |
type: summarization
|
| 19 |
name: Radiology Report Summarization
|
|
|
|
|
|
|
|
|
|
| 20 |
metrics:
|
| 21 |
- type: rouge
|
| 22 |
value: 0.5174
|
|
@@ -27,200 +34,4 @@ model-index:
|
|
| 27 |
- type: rouge
|
| 28 |
value: 0.5092
|
| 29 |
name: ROUGE-L F1
|
| 30 |
-
---
|
| 31 |
-
|
| 32 |
-
# Radiology Expression Summarizer
|
| 33 |
-
|
| 34 |
-
A fine-tuned BART-base model that generates the IMPRESSION section of chest X-ray radiology reports from the FINDINGS section. Built as a portfolio project to demonstrate end-to-end NLP pipeline development, from data preparation through model training to live deployment.
|
| 35 |
-
|
| 36 |
-
## Model Description
|
| 37 |
-
|
| 38 |
-
This model takes detailed radiological FINDINGS as input and produces a concise IMPRESSION summary, mimicking the workflow that radiologists follow when writing clinical reports. The IMPRESSION is the most important section of a radiology report — it summarizes the key clinical findings and is what physicians refer to most often.
|
| 39 |
-
|
| 40 |
-
- **Developed by:** Kumud Shorey
|
| 41 |
-
- **Model type:** Sequence-to-Sequence (Encoder-Decoder Transformer)
|
| 42 |
-
- **Language:** English
|
| 43 |
-
- **Base model:** [facebook/bart-base](https://huggingface.co/facebook/bart-base)
|
| 44 |
-
- **Fine-tuned for:** Medical text summarization
|
| 45 |
-
- **Live API:** [https://radiology-expression-summarizer.onrender.com](https://radiology-expression-summarizer.onrender.com)
|
| 46 |
-
|
| 47 |
-
## Intended Use
|
| 48 |
-
|
| 49 |
-
### Primary Use Cases
|
| 50 |
-
- Automated drafting of preliminary IMPRESSION sections from FINDINGS
|
| 51 |
-
- Research and educational demonstrations of medical NLP
|
| 52 |
-
- Baseline model for further fine-tuning on domain-specific data
|
| 53 |
-
|
| 54 |
-
### Out-of-Scope Use
|
| 55 |
-
**⚠️ This model is NOT intended for clinical use.** It has not been validated by medical professionals, is not approved by any regulatory body (FDA, CE, etc.), and should never be used to make medical decisions. All output should be reviewed by qualified radiologists.
|
| 56 |
-
|
| 57 |
-
## Training Data
|
| 58 |
-
|
| 59 |
-
- **Source:** Open-i NLMCXR dataset (Indiana University Chest X-ray Collection)
|
| 60 |
-
- **Original size:** 3,955 XML files
|
| 61 |
-
- **After cleaning:** 2,712 examples (removed missing values and duplicates)
|
| 62 |
-
- **Splits:**
|
| 63 |
-
- Train: 1,952 examples (72%)
|
| 64 |
-
- Validation: 217 examples (8%)
|
| 65 |
-
- Test: 543 examples (20%)
|
| 66 |
-
- **No data leakage:** Verified that no examples appear in multiple splits
|
| 67 |
-
|
| 68 |
-
### Data Preprocessing
|
| 69 |
-
1. XML files parsed to extract FINDINGS and IMPRESSION sections
|
| 70 |
-
2. Removed examples with missing FINDINGS or IMPRESSION
|
| 71 |
-
3. Removed duplicate examples
|
| 72 |
-
4. Stratified random split with seed=42 for reproducibility
|
| 73 |
-
|
| 74 |
-
## Training Procedure
|
| 75 |
-
|
| 76 |
-
### Hyperparameters
|
| 77 |
-
|
| 78 |
-
| Parameter | Value |
|
| 79 |
-
|-----------|-------|
|
| 80 |
-
| Base model | facebook/bart-base |
|
| 81 |
-
| Number of parameters | 139M |
|
| 82 |
-
| Number of epochs | 5 |
|
| 83 |
-
| Per-device batch size | 4 |
|
| 84 |
-
| Gradient accumulation steps | 4 |
|
| 85 |
-
| Effective batch size | 16 |
|
| 86 |
-
| Learning rate | 3e-5 |
|
| 87 |
-
| Warmup steps | 100 |
|
| 88 |
-
| Weight decay | 0.01 |
|
| 89 |
-
| Mixed precision | FP16 (when GPU available) |
|
| 90 |
-
| Max source length | 512 tokens |
|
| 91 |
-
| Max target length | 128 tokens |
|
| 92 |
-
| Generation beams | 4 |
|
| 93 |
-
| Random seed | 42 |
|
| 94 |
-
|
| 95 |
-
### Training Framework
|
| 96 |
-
- **Library:** Hugging Face Transformers
|
| 97 |
-
- **Trainer:** `Seq2SeqTrainer`
|
| 98 |
-
- **Hardware:** CPU (training); GPU recommended for production
|
| 99 |
-
|
| 100 |
-
### Model Selection
|
| 101 |
-
The best checkpoint was selected based on highest ROUGE-1 F1 score on the validation set using `load_best_model_at_end=True`.
|
| 102 |
-
|
| 103 |
-
## Evaluation Results
|
| 104 |
-
|
| 105 |
-
Evaluated on the held-out test set (543 examples):
|
| 106 |
-
|
| 107 |
-
| Metric | F1 Score |
|
| 108 |
-
|--------|----------|
|
| 109 |
-
| ROUGE-1 | **0.5174** |
|
| 110 |
-
| ROUGE-2 | **0.3881** |
|
| 111 |
-
| ROUGE-L | **0.5092** |
|
| 112 |
-
|
| 113 |
-
These scores exceed the project requirement of ROUGE-1 ≥ 0.3 by 72%.
|
| 114 |
-
|
| 115 |
-
## How to Use
|
| 116 |
-
|
| 117 |
-
### Quick Start
|
| 118 |
-
|
| 119 |
-
```python
|
| 120 |
-
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
|
| 121 |
-
|
| 122 |
-
tokenizer = AutoTokenizer.from_pretrained("Kumud2k16/radiology-expression-summarizer")
|
| 123 |
-
model = AutoModelForSeq2SeqLM.from_pretrained("Kumud2k16/radiology-expression-summarizer")
|
| 124 |
-
|
| 125 |
-
findings = "The lungs are clear bilaterally. Heart size is normal. No pleural effusion or pneumothorax."
|
| 126 |
-
|
| 127 |
-
inputs = tokenizer(findings, return_tensors="pt", max_length=512, truncation=True)
|
| 128 |
-
outputs = model.generate(**inputs, max_length=128, num_beams=4)
|
| 129 |
-
impression = tokenizer.decode(outputs[0], skip_special_tokens=True)
|
| 130 |
-
|
| 131 |
-
print(impression)
|
| 132 |
-
# Expected output: "No acute cardiopulmonary abnormalities."
|
| 133 |
-
```
|
| 134 |
-
|
| 135 |
-
### Live API
|
| 136 |
-
|
| 137 |
-
The model is deployed as a REST API on Render:
|
| 138 |
-
|
| 139 |
-
```bash
|
| 140 |
-
curl -X POST https://radiology-expression-summarizer.onrender.com/summarize \
|
| 141 |
-
-H "Content-Type: application/json" \
|
| 142 |
-
-d '{"findings": "The lungs are clear bilaterally. Heart size is normal."}'
|
| 143 |
-
```
|
| 144 |
-
|
| 145 |
-
Response:
|
| 146 |
-
```json
|
| 147 |
-
{"impression": "No acute cardiopulmonary abnormalities."}
|
| 148 |
-
```
|
| 149 |
-
|
| 150 |
-
**Note:** The free-tier deployment may take ~50 seconds to wake up after periods of inactivity (cold start).
|
| 151 |
-
|
| 152 |
-
## Limitations and Recommendations
|
| 153 |
-
|
| 154 |
-
### Known Limitations
|
| 155 |
-
|
| 156 |
-
1. **Dataset bias:** Trained on a single institution's data (Indiana University). May not generalize to reports from other institutions with different writing styles or terminology conventions.
|
| 157 |
-
|
| 158 |
-
2. **Hallucination risk:** Like all generative models, may occasionally produce plausible-sounding but incorrect medical content. All output must be reviewed by qualified clinicians.
|
| 159 |
-
|
| 160 |
-
3. **Domain narrow:** Trained only on chest X-ray reports. Performance on other modalities (CT, MRI, ultrasound) or other anatomical regions is untested.
|
| 161 |
-
|
| 162 |
-
4. **English only:** Cannot handle reports in other languages.
|
| 163 |
-
|
| 164 |
-
5. **No clinical validation:** Has not been evaluated against clinical correctness metrics (e.g., CheXbert F1). ROUGE measures n-gram overlap, not factual accuracy.
|
| 165 |
-
|
| 166 |
-
6. **Small training set:** Only 1,952 training examples. A larger dataset would likely improve performance.
|
| 167 |
-
|
| 168 |
-
7. **Free-tier deployment:** The live API uses Render's free tier, which causes cold starts (~50 second initial response time after inactivity).
|
| 169 |
-
|
| 170 |
-
### Recommendations for Future Work
|
| 171 |
-
|
| 172 |
-
- Fine-tune on **BioBART** or **BioGPT** instead of BART-base for better baseline medical understanding
|
| 173 |
-
- Evaluate using domain-specific metrics like **CheXbert F1** to measure clinical correctness
|
| 174 |
-
- Expand training data with reports from multiple institutions
|
| 175 |
-
- Add safety guardrails to detect uncertain predictions
|
| 176 |
-
- Conduct clinical validation studies before any real-world use
|
| 177 |
-
- Deploy on production infrastructure (GPU, no cold starts) for clinical-grade latency
|
| 178 |
-
|
| 179 |
-
## Reproducibility
|
| 180 |
-
|
| 181 |
-
Full source code, training scripts, and evaluation code are available at:
|
| 182 |
-
**[GitHub Repository](https://github.com/Kumud2k16/radiology-expression-summarizer)**
|
| 183 |
-
|
| 184 |
-
The codebase includes:
|
| 185 |
-
- Data parsing and cleaning scripts
|
| 186 |
-
- Tokenization pipeline
|
| 187 |
-
- Training script with full hyperparameter logging
|
| 188 |
-
- Evaluation script with ROUGE computation
|
| 189 |
-
- FastAPI deployment code
|
| 190 |
-
- Dockerfile for containerized deployment
|
| 191 |
-
|
| 192 |
-
All random seeds are set to 42 for reproducibility.
|
| 193 |
-
|
| 194 |
-
## Citation
|
| 195 |
-
|
| 196 |
-
If you use this model, please cite the original BART paper:
|
| 197 |
-
|
| 198 |
-
```bibtex
|
| 199 |
-
@article{lewis2019bart,
|
| 200 |
-
title={BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension},
|
| 201 |
-
author={Lewis, Mike and Liu, Yinhan and Goyal, Naman and Ghazvininejad, Marjan and Mohamed, Abdelrahman and Levy, Omer and Stoyanov, Ves and Zettlemoyer, Luke},
|
| 202 |
-
journal={arXiv preprint arXiv:1910.13461},
|
| 203 |
-
year={2019}
|
| 204 |
-
}
|
| 205 |
-
```
|
| 206 |
-
|
| 207 |
-
And the Open-i dataset:
|
| 208 |
-
|
| 209 |
-
```bibtex
|
| 210 |
-
@article{demner2016preparing,
|
| 211 |
-
title={Preparing a collection of radiology examinations for distribution and retrieval},
|
| 212 |
-
author={Demner-Fushman, Dina and Kohli, Marc D and Rosenman, Marc B and Shooshan, Sonya E and Rodriguez, Laritza and Antani, Sameer and Thoma, George R and McDonald, Clement J},
|
| 213 |
-
journal={Journal of the American Medical Informatics Association},
|
| 214 |
-
volume={23},
|
| 215 |
-
number={2},
|
| 216 |
-
pages={304--310},
|
| 217 |
-
year={2016}
|
| 218 |
-
}
|
| 219 |
-
```
|
| 220 |
-
|
| 221 |
-
## Author
|
| 222 |
-
|
| 223 |
-
**Kumud Shorey**
|
| 224 |
-
Project completed as part of IT Career Switch training program.
|
| 225 |
-
GitHub: [@Kumud2k16](https://github.com/Kumud2k16)
|
| 226 |
-
Hugging Face: [@Kumud2k16](https://huggingface.co/Kumud2k16)
|
|
|
|
| 1 |
---
|
| 2 |
language: en
|
| 3 |
license: mit
|
| 4 |
+
library_name: transformers
|
| 5 |
+
pipeline_tag: summarization
|
| 6 |
+
base_model: facebook/bart-base
|
| 7 |
+
base_model_relation: finetune
|
| 8 |
tags:
|
| 9 |
- summarization
|
| 10 |
- radiology
|
|
|
|
| 21 |
- task:
|
| 22 |
type: summarization
|
| 23 |
name: Radiology Report Summarization
|
| 24 |
+
dataset:
|
| 25 |
+
type: openi
|
| 26 |
+
name: Open-i NLMCXR (Indiana University Chest X-ray)
|
| 27 |
metrics:
|
| 28 |
- type: rouge
|
| 29 |
value: 0.5174
|
|
|
|
| 34 |
- type: rouge
|
| 35 |
value: 0.5092
|
| 36 |
name: ROUGE-L F1
|
| 37 |
+
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|