File size: 2,526 Bytes
dc0b8ee | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 | ---
license: llama3.1
base_model:
- meta-llama/Llama-3.1-8B-Instruct
- Qwen/Qwen2.5-1.5B-Instruct
tags:
- biomedical
- genomics
- variant-interpretation
- lora
- research
library_name: transformers
pipeline_tag: text-generation
---
# KAUBioMED-LLM
KAUBioMED-LLM is a research prototype for guarded biomedical variant interpretation plus leakage-aware temporal variant scoring.
## Components
- Qwen2.5-1.5B-Instruct v0.1: lightweight prototype report generator.
- Llama-3.1-8B-Instruct v0.2: stronger guarded report-generation baseline.
- score_model: scikit-learn HistGradientBoostingClassifier with isotonic calibration, using label-free broad variant features.
## Intended Use
Research use only. Not for clinical diagnosis, treatment decisions, or patient management.
## Data and Provenance
See `reports/DATA_PROVENANCE.md`.
## Key Results
First-appearance temporal broad score_model test:
- AUROC: 0.99125
- AUPRC: 0.96938
- Brier: 0.02903
- ECE 10-bin: 0.03428
Missense-only limitation:
- score_model v0 missense AUROC: 0.78497
- missense v1 protein-feature AUROC: 0.83644
- AlphaMissense same-row AUROC: 0.97214
- REVEL same-row AUROC: 0.97832
## Known Limitations
- Broad AUROC is driven partly by easy consequence classes; see `reports/score_model/per_consequence_breakdown.md`.
- Missense performance remains below REVEL and AlphaMissense.
- Evidence panel is limited to 20 genes and is not genome-wide.
- Calibration is measured on a first-appearance ClinVar temporal split, not prospective clinical data.
- Not validated for clinical deployment.
## Base Model Attribution and License Notes
Llama-derived artifacts are subject to the Meta Llama 3.1 Community License and Acceptable Use Policy. Qwen-derived prototype artifacts are subject to Qwen2.5 license terms. Verify license terms before public release.
## Reproducibility Commands
```bash
cd /ddn/data/generic/bbabajan/KAUBioMED_LLM
/ddn/data/generic/bbabajan/envs/kaubiomed-llm/bin/python pipeline/build_label_free_features.py
/ddn/data/generic/bbabajan/envs/kaubiomed-llm/bin/python pipeline/train_label_free_score_model.py
/ddn/data/generic/bbabajan/envs/kaubiomed-llm/bin/python pipeline/evaluate_first_appearance_temporal.py
/ddn/data/generic/bbabajan/envs/kaubiomed-llm/bin/python pipeline/compare_external_predictors_first_appearance.py
/ddn/data/generic/bbabajan/envs/kaubiomed-llm/bin/python pipeline/kaubiomed_full_report.py
/ddn/data/generic/bbabajan/envs/kaubiomed-llm/bin/python pipeline/apply_report_guards.py
```
|