Kumud2k16 commited on
Commit
f530eaf
·
verified ·
1 Parent(s): 7316d61

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -197
README.md CHANGED
@@ -1,6 +1,10 @@
1
  ---
2
  language: en
3
  license: mit
 
 
 
 
4
  tags:
5
  - summarization
6
  - radiology
@@ -17,6 +21,9 @@ model-index:
17
  - task:
18
  type: summarization
19
  name: Radiology Report Summarization
 
 
 
20
  metrics:
21
  - type: rouge
22
  value: 0.5174
@@ -27,200 +34,4 @@ model-index:
27
  - type: rouge
28
  value: 0.5092
29
  name: ROUGE-L F1
30
- ---
31
-
32
- # Radiology Expression Summarizer
33
-
34
- A fine-tuned BART-base model that generates the IMPRESSION section of chest X-ray radiology reports from the FINDINGS section. Built as a portfolio project to demonstrate end-to-end NLP pipeline development, from data preparation through model training to live deployment.
35
-
36
- ## Model Description
37
-
38
- This model takes detailed radiological FINDINGS as input and produces a concise IMPRESSION summary, mimicking the workflow that radiologists follow when writing clinical reports. The IMPRESSION is the most important section of a radiology report — it summarizes the key clinical findings and is what physicians refer to most often.
39
-
40
- - **Developed by:** Kumud Shorey
41
- - **Model type:** Sequence-to-Sequence (Encoder-Decoder Transformer)
42
- - **Language:** English
43
- - **Base model:** [facebook/bart-base](https://huggingface.co/facebook/bart-base)
44
- - **Fine-tuned for:** Medical text summarization
45
- - **Live API:** [https://radiology-expression-summarizer.onrender.com](https://radiology-expression-summarizer.onrender.com)
46
-
47
- ## Intended Use
48
-
49
- ### Primary Use Cases
50
- - Automated drafting of preliminary IMPRESSION sections from FINDINGS
51
- - Research and educational demonstrations of medical NLP
52
- - Baseline model for further fine-tuning on domain-specific data
53
-
54
- ### Out-of-Scope Use
55
- **⚠️ This model is NOT intended for clinical use.** It has not been validated by medical professionals, is not approved by any regulatory body (FDA, CE, etc.), and should never be used to make medical decisions. All output should be reviewed by qualified radiologists.
56
-
57
- ## Training Data
58
-
59
- - **Source:** Open-i NLMCXR dataset (Indiana University Chest X-ray Collection)
60
- - **Original size:** 3,955 XML files
61
- - **After cleaning:** 2,712 examples (removed missing values and duplicates)
62
- - **Splits:**
63
- - Train: 1,952 examples (72%)
64
- - Validation: 217 examples (8%)
65
- - Test: 543 examples (20%)
66
- - **No data leakage:** Verified that no examples appear in multiple splits
67
-
68
- ### Data Preprocessing
69
- 1. XML files parsed to extract FINDINGS and IMPRESSION sections
70
- 2. Removed examples with missing FINDINGS or IMPRESSION
71
- 3. Removed duplicate examples
72
- 4. Stratified random split with seed=42 for reproducibility
73
-
74
- ## Training Procedure
75
-
76
- ### Hyperparameters
77
-
78
- | Parameter | Value |
79
- |-----------|-------|
80
- | Base model | facebook/bart-base |
81
- | Number of parameters | 139M |
82
- | Number of epochs | 5 |
83
- | Per-device batch size | 4 |
84
- | Gradient accumulation steps | 4 |
85
- | Effective batch size | 16 |
86
- | Learning rate | 3e-5 |
87
- | Warmup steps | 100 |
88
- | Weight decay | 0.01 |
89
- | Mixed precision | FP16 (when GPU available) |
90
- | Max source length | 512 tokens |
91
- | Max target length | 128 tokens |
92
- | Generation beams | 4 |
93
- | Random seed | 42 |
94
-
95
- ### Training Framework
96
- - **Library:** Hugging Face Transformers
97
- - **Trainer:** `Seq2SeqTrainer`
98
- - **Hardware:** CPU (training); GPU recommended for production
99
-
100
- ### Model Selection
101
- The best checkpoint was selected based on highest ROUGE-1 F1 score on the validation set using `load_best_model_at_end=True`.
102
-
103
- ## Evaluation Results
104
-
105
- Evaluated on the held-out test set (543 examples):
106
-
107
- | Metric | F1 Score |
108
- |--------|----------|
109
- | ROUGE-1 | **0.5174** |
110
- | ROUGE-2 | **0.3881** |
111
- | ROUGE-L | **0.5092** |
112
-
113
- These scores exceed the project requirement of ROUGE-1 ≥ 0.3 by 72%.
114
-
115
- ## How to Use
116
-
117
- ### Quick Start
118
-
119
- ```python
120
- from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
121
-
122
- tokenizer = AutoTokenizer.from_pretrained("Kumud2k16/radiology-expression-summarizer")
123
- model = AutoModelForSeq2SeqLM.from_pretrained("Kumud2k16/radiology-expression-summarizer")
124
-
125
- findings = "The lungs are clear bilaterally. Heart size is normal. No pleural effusion or pneumothorax."
126
-
127
- inputs = tokenizer(findings, return_tensors="pt", max_length=512, truncation=True)
128
- outputs = model.generate(**inputs, max_length=128, num_beams=4)
129
- impression = tokenizer.decode(outputs[0], skip_special_tokens=True)
130
-
131
- print(impression)
132
- # Expected output: "No acute cardiopulmonary abnormalities."
133
- ```
134
-
135
- ### Live API
136
-
137
- The model is deployed as a REST API on Render:
138
-
139
- ```bash
140
- curl -X POST https://radiology-expression-summarizer.onrender.com/summarize \
141
- -H "Content-Type: application/json" \
142
- -d '{"findings": "The lungs are clear bilaterally. Heart size is normal."}'
143
- ```
144
-
145
- Response:
146
- ```json
147
- {"impression": "No acute cardiopulmonary abnormalities."}
148
- ```
149
-
150
- **Note:** The free-tier deployment may take ~50 seconds to wake up after periods of inactivity (cold start).
151
-
152
- ## Limitations and Recommendations
153
-
154
- ### Known Limitations
155
-
156
- 1. **Dataset bias:** Trained on a single institution's data (Indiana University). May not generalize to reports from other institutions with different writing styles or terminology conventions.
157
-
158
- 2. **Hallucination risk:** Like all generative models, may occasionally produce plausible-sounding but incorrect medical content. All output must be reviewed by qualified clinicians.
159
-
160
- 3. **Domain narrow:** Trained only on chest X-ray reports. Performance on other modalities (CT, MRI, ultrasound) or other anatomical regions is untested.
161
-
162
- 4. **English only:** Cannot handle reports in other languages.
163
-
164
- 5. **No clinical validation:** Has not been evaluated against clinical correctness metrics (e.g., CheXbert F1). ROUGE measures n-gram overlap, not factual accuracy.
165
-
166
- 6. **Small training set:** Only 1,952 training examples. A larger dataset would likely improve performance.
167
-
168
- 7. **Free-tier deployment:** The live API uses Render's free tier, which causes cold starts (~50 second initial response time after inactivity).
169
-
170
- ### Recommendations for Future Work
171
-
172
- - Fine-tune on **BioBART** or **BioGPT** instead of BART-base for better baseline medical understanding
173
- - Evaluate using domain-specific metrics like **CheXbert F1** to measure clinical correctness
174
- - Expand training data with reports from multiple institutions
175
- - Add safety guardrails to detect uncertain predictions
176
- - Conduct clinical validation studies before any real-world use
177
- - Deploy on production infrastructure (GPU, no cold starts) for clinical-grade latency
178
-
179
- ## Reproducibility
180
-
181
- Full source code, training scripts, and evaluation code are available at:
182
- **[GitHub Repository](https://github.com/Kumud2k16/radiology-expression-summarizer)**
183
-
184
- The codebase includes:
185
- - Data parsing and cleaning scripts
186
- - Tokenization pipeline
187
- - Training script with full hyperparameter logging
188
- - Evaluation script with ROUGE computation
189
- - FastAPI deployment code
190
- - Dockerfile for containerized deployment
191
-
192
- All random seeds are set to 42 for reproducibility.
193
-
194
- ## Citation
195
-
196
- If you use this model, please cite the original BART paper:
197
-
198
- ```bibtex
199
- @article{lewis2019bart,
200
- title={BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension},
201
- author={Lewis, Mike and Liu, Yinhan and Goyal, Naman and Ghazvininejad, Marjan and Mohamed, Abdelrahman and Levy, Omer and Stoyanov, Ves and Zettlemoyer, Luke},
202
- journal={arXiv preprint arXiv:1910.13461},
203
- year={2019}
204
- }
205
- ```
206
-
207
- And the Open-i dataset:
208
-
209
- ```bibtex
210
- @article{demner2016preparing,
211
- title={Preparing a collection of radiology examinations for distribution and retrieval},
212
- author={Demner-Fushman, Dina and Kohli, Marc D and Rosenman, Marc B and Shooshan, Sonya E and Rodriguez, Laritza and Antani, Sameer and Thoma, George R and McDonald, Clement J},
213
- journal={Journal of the American Medical Informatics Association},
214
- volume={23},
215
- number={2},
216
- pages={304--310},
217
- year={2016}
218
- }
219
- ```
220
-
221
- ## Author
222
-
223
- **Kumud Shorey**
224
- Project completed as part of IT Career Switch training program.
225
- GitHub: [@Kumud2k16](https://github.com/Kumud2k16)
226
- Hugging Face: [@Kumud2k16](https://huggingface.co/Kumud2k16)
 
1
  ---
2
  language: en
3
  license: mit
4
+ library_name: transformers
5
+ pipeline_tag: summarization
6
+ base_model: facebook/bart-base
7
+ base_model_relation: finetune
8
  tags:
9
  - summarization
10
  - radiology
 
21
  - task:
22
  type: summarization
23
  name: Radiology Report Summarization
24
+ dataset:
25
+ type: openi
26
+ name: Open-i NLMCXR (Indiana University Chest X-ray)
27
  metrics:
28
  - type: rouge
29
  value: 0.5174
 
34
  - type: rouge
35
  value: 0.5092
36
  name: ROUGE-L F1
37
+ ---