---
license: llama3.1
base_model:
  - meta-llama/Llama-3.1-8B-Instruct
  - Qwen/Qwen2.5-1.5B-Instruct
tags:
  - biomedical
  - genomics
  - variant-interpretation
  - lora
  - research
library_name: transformers
pipeline_tag: text-generation
---

# KAUBioMED-LLM

KAUBioMED-LLM is a research prototype for guarded biomedical variant interpretation plus leakage-aware temporal variant scoring.

## Components

- Qwen2.5-1.5B-Instruct v0.1: lightweight prototype report generator.
- Llama-3.1-8B-Instruct v0.2: stronger guarded report-generation baseline.
- score_model: scikit-learn HistGradientBoostingClassifier with isotonic calibration, using label-free broad variant features.

## Intended Use

Research use only. Not for clinical diagnosis, treatment decisions, or patient management.

## Data and Provenance

See `reports/DATA_PROVENANCE.md`.

## Key Results

First-appearance temporal broad score_model test:

- AUROC: 0.99125
- AUPRC: 0.96938
- Brier: 0.02903
- ECE 10-bin: 0.03428

Missense-only limitation:

- score_model v0 missense AUROC: 0.78497
- missense v1 protein-feature AUROC: 0.83644
- AlphaMissense same-row AUROC: 0.97214
- REVEL same-row AUROC: 0.97832

## Known Limitations

- Broad AUROC is driven partly by easy consequence classes; see `reports/score_model/per_consequence_breakdown.md`.
- Missense performance remains below REVEL and AlphaMissense.
- Evidence panel is limited to 20 genes and is not genome-wide.
- Calibration is measured on a first-appearance ClinVar temporal split, not prospective clinical data.
- Not validated for clinical deployment.

## Base Model Attribution and License Notes

Llama-derived artifacts are subject to the Meta Llama 3.1 Community License and Acceptable Use Policy. Qwen-derived prototype artifacts are subject to Qwen2.5 license terms. Verify license terms before public release.

## Reproducibility Commands

```bash
cd /ddn/data/generic/bbabajan/KAUBioMED_LLM
/ddn/data/generic/bbabajan/envs/kaubiomed-llm/bin/python pipeline/build_label_free_features.py
/ddn/data/generic/bbabajan/envs/kaubiomed-llm/bin/python pipeline/train_label_free_score_model.py
/ddn/data/generic/bbabajan/envs/kaubiomed-llm/bin/python pipeline/evaluate_first_appearance_temporal.py
/ddn/data/generic/bbabajan/envs/kaubiomed-llm/bin/python pipeline/compare_external_predictors_first_appearance.py
/ddn/data/generic/bbabajan/envs/kaubiomed-llm/bin/python pipeline/kaubiomed_full_report.py
/ddn/data/generic/bbabajan/envs/kaubiomed-llm/bin/python pipeline/apply_report_guards.py
```