Kumud2k16
/

radiology-expression-summarizer

@@ -1,199 +1,226 @@
 ---
-library_name: transformers
-tags: []
 ---
-# Model Card for Model ID
-<!-- Provide a quick summary of what the model is/does. -->
-## Model Details
-### Model Description
-<!-- Provide a longer summary of what this model is. -->
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-### Model Sources [optional]
-<!-- Provide the basic links for the model. -->
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-## Uses
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-### Direct Use
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-[More Information Needed]
-### Downstream Use [optional]
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-[More Information Needed]
 ### Out-of-Scope Use
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-[More Information Needed]
-## Bias, Risks, and Limitations
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-[More Information Needed]
-### Recommendations
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-## How to Get Started with the Model
-Use the code below to get started with the model.
-[More Information Needed]
-## Training Details
-### Training Data
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-[More Information Needed]
-### Training Procedure
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-#### Preprocessing [optional]
-[More Information Needed]
-#### Training Hyperparameters
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-#### Speeds, Sizes, Times [optional]
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-[More Information Needed]
-## Evaluation
-<!-- This section describes the evaluation protocols and provides the results. -->
-### Testing Data, Factors & Metrics
-#### Testing Data
-<!-- This should link to a Dataset Card if possible. -->
-[More Information Needed]
-#### Factors
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-[More Information Needed]
-#### Metrics
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-[More Information Needed]
-### Results
-[More Information Needed]
-#### Summary
-## Model Examination [optional]
-<!-- Relevant interpretability work for the model goes here -->
-[More Information Needed]
-## Environmental Impact
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-## Technical Specifications [optional]
-### Model Architecture and Objective
-[More Information Needed]
-### Compute Infrastructure
-[More Information Needed]
-#### Hardware
-[More Information Needed]
-#### Software
-[More Information Needed]
-## Citation [optional]
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-**BibTeX:**
-[More Information Needed]
-**APA:**
-[More Information Needed]
-## Glossary [optional]
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-[More Information Needed]
-## More Information [optional]
-[More Information Needed]
-## Model Card Authors [optional]
-[More Information Needed]
-## Model Card Contact
-[More Information Needed]

 ---
+language: en
+license: mit
+tags:
+  - summarization
+  - radiology
+  - medical
+  - bart
+  - healthcare
+datasets:
+  - openi
+metrics:
+  - rouge
+model-index:
+  - name: radiology-expression-summarizer
+    results:
+      - task:
+          type: summarization
+          name: Radiology Report Summarization
+        metrics:
+          - type: rouge
+            value: 0.5174
+            name: ROUGE-1 F1
+          - type: rouge
+            value: 0.3881
+            name: ROUGE-2 F1
+          - type: rouge
+            value: 0.5092
+            name: ROUGE-L F1
 ---
+# Radiology Expression Summarizer
+A fine-tuned BART-base model that generates the IMPRESSION section of chest X-ray radiology reports from the FINDINGS section. Built as a portfolio project to demonstrate end-to-end NLP pipeline development, from data preparation through model training to live deployment.
+## Model Description
+This model takes detailed radiological FINDINGS as input and produces a concise IMPRESSION summary, mimicking the workflow that radiologists follow when writing clinical reports. The IMPRESSION is the most important section of a radiology report — it summarizes the key clinical findings and is what physicians refer to most often.
+- **Developed by:** Kumud Shorey
+- **Model type:** Sequence-to-Sequence (Encoder-Decoder Transformer)
+- **Language:** English
+- **Base model:** [facebook/bart-base](https://huggingface.co/facebook/bart-base)
+- **Fine-tuned for:** Medical text summarization
+- **Live API:** [https://radiology-expression-summarizer.onrender.com](https://radiology-expression-summarizer.onrender.com)
+## Intended Use
+### Primary Use Cases
+- Automated drafting of preliminary IMPRESSION sections from FINDINGS
+- Research and educational demonstrations of medical NLP
+- Baseline model for further fine-tuning on domain-specific data
 ### Out-of-Scope Use
+**⚠️ This model is NOT intended for clinical use.** It has not been validated by medical professionals, is not approved by any regulatory body (FDA, CE, etc.), and should never be used to make medical decisions. All output should be reviewed by qualified radiologists.
+## Training Data
+- **Source:** Open-i NLMCXR dataset (Indiana University Chest X-ray Collection)
+- **Original size:** 3,955 XML files
+- **After cleaning:** 2,712 examples (removed missing values and duplicates)
+- **Splits:**
+  - Train: 1,952 examples (72%)
+  - Validation: 217 examples (8%)
+  - Test: 543 examples (20%)
+- **No data leakage:** Verified that no examples appear in multiple splits
+### Data Preprocessing
+1. XML files parsed to extract FINDINGS and IMPRESSION sections
+2. Removed examples with missing FINDINGS or IMPRESSION
+3. Removed duplicate examples
+4. Stratified random split with seed=42 for reproducibility
+## Training Procedure
+### Hyperparameters
+| Parameter | Value |
+|-----------|-------|
+| Base model | facebook/bart-base |
+| Number of parameters | 139M |
+| Number of epochs | 5 |
+| Per-device batch size | 4 |
+| Gradient accumulation steps | 4 |
+| Effective batch size | 16 |
+| Learning rate | 3e-5 |
+| Warmup steps | 100 |
+| Weight decay | 0.01 |
+| Mixed precision | FP16 (when GPU available) |
+| Max source length | 512 tokens |
+| Max target length | 128 tokens |
+| Generation beams | 4 |
+| Random seed | 42 |
+### Training Framework
+- **Library:** Hugging Face Transformers
+- **Trainer:** `Seq2SeqTrainer`
+- **Hardware:** CPU (training); GPU recommended for production
+### Model Selection
+The best checkpoint was selected based on highest ROUGE-1 F1 score on the validation set using `load_best_model_at_end=True`.
+## Evaluation Results
+Evaluated on the held-out test set (543 examples):
+| Metric | F1 Score |
+|--------|----------|
+| ROUGE-1 | **0.5174** |
+| ROUGE-2 | **0.3881** |
+| ROUGE-L | **0.5092** |
+These scores exceed the project requirement of ROUGE-1 ≥ 0.3 by 72%.
+## How to Use
+### Quick Start
+```python
+from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
+tokenizer = AutoTokenizer.from_pretrained("Kumud2k16/radiology-expression-summarizer")
+model = AutoModelForSeq2SeqLM.from_pretrained("Kumud2k16/radiology-expression-summarizer")
+findings = "The lungs are clear bilaterally. Heart size is normal. No pleural effusion or pneumothorax."
+inputs = tokenizer(findings, return_tensors="pt", max_length=512, truncation=True)
+outputs = model.generate(**inputs, max_length=128, num_beams=4)
+impression = tokenizer.decode(outputs[0], skip_special_tokens=True)
+print(impression)
+# Expected output: "No acute cardiopulmonary abnormalities."
+```
+### Live API
+The model is deployed as a REST API on Render:
+```bash
+curl -X POST https://radiology-expression-summarizer.onrender.com/summarize \
+  -H "Content-Type: application/json" \
+  -d '{"findings": "The lungs are clear bilaterally. Heart size is normal."}'
+```
+Response:
+```json
+{"impression": "No acute cardiopulmonary abnormalities."}
+```
+**Note:** The free-tier deployment may take ~50 seconds to wake up after periods of inactivity (cold start).
+## Limitations and Recommendations
+### Known Limitations
+1. **Dataset bias:** Trained on a single institution's data (Indiana University). May not generalize to reports from other institutions with different writing styles or terminology conventions.
+2. **Hallucination risk:** Like all generative models, may occasionally produce plausible-sounding but incorrect medical content. All output must be reviewed by qualified clinicians.
+3. **Domain narrow:** Trained only on chest X-ray reports. Performance on other modalities (CT, MRI, ultrasound) or other anatomical regions is untested.
+4. **English only:** Cannot handle reports in other languages.
+5. **No clinical validation:** Has not been evaluated against clinical correctness metrics (e.g., CheXbert F1). ROUGE measures n-gram overlap, not factual accuracy.
+6. **Small training set:** Only 1,952 training examples. A larger dataset would likely improve performance.
+7. **Free-tier deployment:** The live API uses Render's free tier, which causes cold starts (~50 second initial response time after inactivity).
+### Recommendations for Future Work
+- Fine-tune on **BioBART** or **BioGPT** instead of BART-base for better baseline medical understanding
+- Evaluate using domain-specific metrics like **CheXbert F1** to measure clinical correctness
+- Expand training data with reports from multiple institutions
+- Add safety guardrails to detect uncertain predictions
+- Conduct clinical validation studies before any real-world use
+- Deploy on production infrastructure (GPU, no cold starts) for clinical-grade latency
+## Reproducibility
+Full source code, training scripts, and evaluation code are available at:
+**[GitHub Repository](https://github.com/Kumud2k16/radiology-expression-summarizer)**
+The codebase includes:
+- Data parsing and cleaning scripts
+- Tokenization pipeline
+- Training script with full hyperparameter logging
+- Evaluation script with ROUGE computation
+- FastAPI deployment code
+- Dockerfile for containerized deployment
+All random seeds are set to 42 for reproducibility.
+## Citation
+If you use this model, please cite the original BART paper:
+```bibtex
+@article{lewis2019bart,
+  title={BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension},
+  author={Lewis, Mike and Liu, Yinhan and Goyal, Naman and Ghazvininejad, Marjan and Mohamed, Abdelrahman and Levy, Omer and Stoyanov, Ves and Zettlemoyer, Luke},
+  journal={arXiv preprint arXiv:1910.13461},
+  year={2019}
+}
+```
+And the Open-i dataset:
+```bibtex
+@article{demner2016preparing,
+  title={Preparing a collection of radiology examinations for distribution and retrieval},
+  author={Demner-Fushman, Dina and Kohli, Marc D and Rosenman, Marc B and Shooshan, Sonya E and Rodriguez, Laritza and Antani, Sameer and Thoma, George R and McDonald, Clement J},
+  journal={Journal of the American Medical Informatics Association},
+  volume={23},
+  number={2},
+  pages={304--310},
+  year={2016}
+}
+```
+## Author
+**Kumud Shorey**
+Project completed as part of IT Career Switch training program.
+GitHub: [@Kumud2k16](https://github.com/Kumud2k16)
+Hugging Face: [@Kumud2k16](https://huggingface.co/Kumud2k16)