Text Generation
Transformers
Safetensors
biomedical
genomics
variant-interpretation
lora
clinical-genomics
bioinformatics
research
conversational
Instructions to use Babajaan/KAU-BioMedLLM with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Babajaan/KAU-BioMedLLM with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Babajaan/KAU-BioMedLLM") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Babajaan/KAU-BioMedLLM", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Babajaan/KAU-BioMedLLM with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Babajaan/KAU-BioMedLLM" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Babajaan/KAU-BioMedLLM", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Babajaan/KAU-BioMedLLM
- SGLang
How to use Babajaan/KAU-BioMedLLM with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Babajaan/KAU-BioMedLLM" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Babajaan/KAU-BioMedLLM", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Babajaan/KAU-BioMedLLM" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Babajaan/KAU-BioMedLLM", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use Babajaan/KAU-BioMedLLM with Docker Model Runner:
docker model run hf.co/Babajaan/KAU-BioMedLLM
Update model card for Llama root and Qwen subfolder adapters
Browse files
README.md
CHANGED
|
@@ -17,11 +17,17 @@ pipeline_tag: text-generation
|
|
| 17 |
|
| 18 |
KAU-BioMedLLM is a research prototype for source-grounded biomedical variant interpretation. The current public release contains the LoRA adapter and documentation for a guarded report-generation system built around biomedical evidence retrieval, label-free variant scoring, citation enforcement, and abstention when evidence is insufficient.
|
| 19 |
|
| 20 |
-
This repository
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
|
| 22 |
## Table of Contents
|
| 23 |
|
| 24 |
- [Model Details](#model-details)
|
|
|
|
| 25 |
- [Model Description](#model-description)
|
| 26 |
- [System Overview](#system-overview)
|
| 27 |
- [Uses](#uses)
|
|
@@ -49,6 +55,15 @@ This repository does **not** redistribute the full Meta Llama-3.1-8B-Instruct ba
|
|
| 49 |
- **Release type:** Research-use adapter release
|
| 50 |
- **Clinical status:** Not clinically validated; not for diagnosis, treatment, or patient-management decisions
|
| 51 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 52 |
## Model Description
|
| 53 |
|
| 54 |
KAU-BioMedLLM was developed as part of a broader biomedical AI project to generate structured, source-grounded variant interpretation reports. The system combines two deliberately separated components:
|
|
@@ -251,7 +266,9 @@ Full guarded report examples and summaries are included in the repository under
|
|
| 251 |
- Generated text can still contain errors and must be reviewed by domain experts.
|
| 252 |
- The uploaded artifact is an adapter, not a standalone full model.
|
| 253 |
|
| 254 |
-
## How to Load the
|
|
|
|
|
|
|
| 255 |
|
| 256 |
```python
|
| 257 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
@@ -299,6 +316,33 @@ with torch.no_grad():
|
|
| 299 |
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
| 300 |
```
|
| 301 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 302 |
## Compute Infrastructure
|
| 303 |
|
| 304 |
The project was developed and evaluated using the King Abdulaziz University Aziz High Performance Computing environment. The workflow used the Aziz HPC software environment, PBS job scheduling, and GPU resources including A100 GPU jobs where available.
|
|
@@ -332,4 +376,4 @@ Suggested acknowledgement:
|
|
| 332 |
|
| 333 |
## License and Base Model Notes
|
| 334 |
|
| 335 |
-
This repository contains
|
|
|
|
| 17 |
|
| 18 |
KAU-BioMedLLM is a research prototype for source-grounded biomedical variant interpretation. The current public release contains the LoRA adapter and documentation for a guarded report-generation system built around biomedical evidence retrieval, label-free variant scoring, citation enforcement, and abstention when evidence is insufficient.
|
| 19 |
|
| 20 |
+
This repository contains two adapter releases:
|
| 21 |
+
|
| 22 |
+
- **Root adapter:** Llama-3.1-8B-Instruct v0.2 LoRA adapter, recommended baseline for report generation.
|
| 23 |
+
- **`qwen_v0_1/` adapter:** Qwen2.5-1.5B-Instruct v0.1 LoRA adapter, lightweight prototype/demo fallback.
|
| 24 |
+
|
| 25 |
+
This repository does **not** redistribute full Llama or Qwen base model weights. Users must separately load the relevant base model according to its license terms.
|
| 26 |
|
| 27 |
## Table of Contents
|
| 28 |
|
| 29 |
- [Model Details](#model-details)
|
| 30 |
+
- [Repository Layout](#repository-layout)
|
| 31 |
- [Model Description](#model-description)
|
| 32 |
- [System Overview](#system-overview)
|
| 33 |
- [Uses](#uses)
|
|
|
|
| 55 |
- **Release type:** Research-use adapter release
|
| 56 |
- **Clinical status:** Not clinically validated; not for diagnosis, treatment, or patient-management decisions
|
| 57 |
|
| 58 |
+
## Repository Layout
|
| 59 |
+
|
| 60 |
+
| Path | Adapter | Base model | Intended role |
|
| 61 |
+
|---|---|---|---|
|
| 62 |
+
| repository root | Llama v0.2 LoRA | `meta-llama/Llama-3.1-8B-Instruct` | Recommended report-generation baseline |
|
| 63 |
+
| `qwen_v0_1/` | Qwen v0.1 LoRA | `Qwen/Qwen2.5-1.5B-Instruct` | Lightweight prototype and demo fallback |
|
| 64 |
+
|
| 65 |
+
The root adapter is kept as the primary Hugging Face adapter so standard PEFT loading with `Babajaan/KAU-BioMedLLM` loads the Llama v0.2 adapter. The Qwen adapter is included in the same repository under `qwen_v0_1/` to avoid maintaining a second model repo.
|
| 66 |
+
|
| 67 |
## Model Description
|
| 68 |
|
| 69 |
KAU-BioMedLLM was developed as part of a broader biomedical AI project to generate structured, source-grounded variant interpretation reports. The system combines two deliberately separated components:
|
|
|
|
| 266 |
- Generated text can still contain errors and must be reviewed by domain experts.
|
| 267 |
- The uploaded artifact is an adapter, not a standalone full model.
|
| 268 |
|
| 269 |
+
## How to Load the Adapters
|
| 270 |
+
|
| 271 |
+
### Llama v0.2 Adapter
|
| 272 |
|
| 273 |
```python
|
| 274 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
|
|
| 316 |
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
| 317 |
```
|
| 318 |
|
| 319 |
+
### Qwen v0.1 Adapter
|
| 320 |
+
|
| 321 |
+
The Qwen adapter is stored in the same repository under `qwen_v0_1/`. Use it for lightweight demos when Llama-3.1-8B hardware is unavailable.
|
| 322 |
+
|
| 323 |
+
```python
|
| 324 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 325 |
+
from peft import PeftModel
|
| 326 |
+
import torch
|
| 327 |
+
|
| 328 |
+
base_model = "Qwen/Qwen2.5-1.5B-Instruct"
|
| 329 |
+
adapter_model = "Babajaan/KAU-BioMedLLM"
|
| 330 |
+
adapter_subfolder = "qwen_v0_1"
|
| 331 |
+
|
| 332 |
+
tokenizer = AutoTokenizer.from_pretrained(base_model)
|
| 333 |
+
model = AutoModelForCausalLM.from_pretrained(
|
| 334 |
+
base_model,
|
| 335 |
+
torch_dtype=torch.float16,
|
| 336 |
+
device_map="auto",
|
| 337 |
+
)
|
| 338 |
+
model = PeftModel.from_pretrained(
|
| 339 |
+
model,
|
| 340 |
+
adapter_model,
|
| 341 |
+
subfolder=adapter_subfolder,
|
| 342 |
+
)
|
| 343 |
+
model.eval()
|
| 344 |
+
```
|
| 345 |
+
|
| 346 |
## Compute Infrastructure
|
| 347 |
|
| 348 |
The project was developed and evaluated using the King Abdulaziz University Aziz High Performance Computing environment. The workflow used the Aziz HPC software environment, PBS job scheduling, and GPU resources including A100 GPU jobs where available.
|
|
|
|
| 376 |
|
| 377 |
## License and Base Model Notes
|
| 378 |
|
| 379 |
+
This repository contains LoRA adapters and project documentation. It does not redistribute full Llama or Qwen base model weights. Use of the root adapter with `meta-llama/Llama-3.1-8B-Instruct` is subject to the Meta Llama 3.1 Community License and Acceptable Use Policy. Use of the `qwen_v0_1/` adapter with `Qwen/Qwen2.5-1.5B-Instruct` is subject to Qwen license terms. Users are responsible for verifying license compatibility for their own use case.
|