---
license: apache-2.0
language:
  - zh
tags:
  - medical
  - breast-cancer
  - llm
  - qwen
  - lora
  - clinical-ai
pipeline_tag: text-generation
base_model: Qwen/Qwen2.5-7B-Instruct
---

# Qwen2.5-7B-Breast-CRAG

## Breast Cancer Specialized LLM (Full Fine-tuned / LoRA-ready)

**Qwen2.5-7B-Breast-CRAG** is a **7B-parameter** large language model **fully fine-tuned** for breast cancer clinical consultation, developed as part of the **Breast-CRAG** system. It is optimized for **human-like doctor–patient dialogue** and **professional breast cancer domain knowledge**.

This model can be loaded **directly** or as a **LoRA adapter** via **LLaMA Factory**.

## 🔗 Links

- HF Repo: https://huggingface.co/MaxinT23/Qwen2.5-7B-Breast-CRAG
- Paper: https://link.springer.com/chapter/10.1007/978-3-031-95841-0_19
- Code: https://github.com/Maxin-C/Breast-CRAG
- Framework: LLaMA Factory

## ✨ Model Description

- **Base Model**: Qwen2.5-7B-Instruct
- **Task**: Breast cancer medical dialogue generation
- **Parameters**: 7B
- **Language**: Chinese
- **Training**: Full fine-tuned + LoRA (PEFT) optimized
- **Purpose**: Clinical consultation assistance (research only)
- **System**: Core generator of **Breast-CRAG (RAG-enhanced)**

## 📊 Training Data

- Curated breast cancer dialogues: **268K**
- Train split: **91K** (30K MedDialog-BC + 61K Huatuo-BC)
- Data source: MedDialog-CN, Huatuo-26M (filtered/cleaned)
- Pipeline: Keyword filter → GPT-4o quality filter → dialogue summarization

## ⚙️ Training Hyperparameters

- LoRA alpha: 32
- LoRA rank: 16
- Dropout: 0.1
- Learning rate: 5e-5
- Scheduler: cosine
- Batch size: 2
- Gradient accumulation: 8
- Epochs: 8
- GPU: RTX 3090 (24GB)
- Training time: ~34.2 hours

## 🧪 Key Results

### Dialogue (Humanization)

- Outperforms similar-size open-source LLMs
- Matches/exceeds **GPT-4o** on **70%** of dialogue metrics

### Exam (Specialization)

- USMLE-BC: **81% accuracy (on par with GPT-4o)**
- Exam-BC Simple: 63% | Hard: 52%

## 🚀 Usage (LLaMA Factory LoRA Load)

### 1. Install

```bash
pip install "llamafactory[torch]" transformers peft
```

### 2. Load via LLaMA Factory

```python
from llamafactory import load_model
from transformers import AutoTokenizer

model_name = "MaxinT23/Qwen2.5-7B-Breast-CRAG"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = load_model(model_name, adapter_name="lora", device="auto")

prompt = "请以乳腺癌专科医生身份，回答以下问题：乳腺癌术后多久可以恢复正常生活？"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.95, top_p=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## ⚠️ Limitations

1. Text-only (no multimodal support)
2. For **research purposes only**, not for direct clinical use
3. Weak performance on multiple-choice exams
4. Best used with **Breast-CRAG Retriever + 1M knowledge chunks**

## 📚 Citation

```Plain
Chen, Z., Wang, Q., Liu, J., Sun, Y., Zheng, H., Li, H., Duan, H., Lu, X. (2025). Breast-CRAG: A Breast Cancer Large Language Model Leveraging Retrieval-Augmented Generation. In: Artificial Intelligence in Medicine. AIME 2025. Lecture Notes in Computer Science, vol 15735. Springer, Cham. https://doi.org/10.1007/978-3-031-95841-0_19
```

## 🙏 Acknowledgments

Supported by the National Key Technologies R&D Program of China (2022YFF1203002) and Sir Run Run Shaw Hospital.