---
license: llama2
base_model: meta-llama/Llama-2-70b-chat-hf
library_name: peft
pipeline_tag: text-generation
tags:
- rare-disease
- clinical-nlp
- phenotype-extraction
- hpo
- peft
- lora
- llama-2
---

# RARE-PHENIX Module 1: Llama-2-70B RareDis Adapter

This repository contains a PEFT/LoRA adapter for **RARE-PHENIX Module 1**, the phenotype extraction module of RARE-PHENIX.

## What this model does

This adapter is intended to extract rare disease phenotype mentions from clinical text.

Input:

~~~text
The patient has global developmental delay and hypotonia.
~~~

Output:

~~~text
The patient has <span class="condition">global developmental delay</span> and <span class="condition">hypotonia</span>.
~~~

## Important scope note

This is **Module 1 only**.

It does not perform:

- HPO standardization
- HPO term ranking
- disease diagnosis
- gene or variant prioritization

The full RARE-PHENIX workflow includes additional downstream modules for HPO standardization and rare disease-relevant HPO prioritization.

## Training data

This adapter was fine-tuned on the public RareDis corpus.

It is **not** the full Undiagnosed Diseases Network (UDN)-trained RARE-PHENIX model described in the manuscript. Controlled-access Undiagnosed Diseases Network (UDN) data and UDN-trained model artifacts are not included.

## Base model

This is a LoRA adapter and requires access to the corresponding Meta Llama-2 base model:

~~~text
meta-llama/Llama-2-70b-chat-hf
~~~

Users must separately request and obtain access to the gated Meta Llama-2 model through Hugging Face.

## Example loading code

~~~python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "meta-llama/Llama-2-70b-chat-hf"
adapter_id = "shyrcathy/rare-phenix-llama2-70b-raredis"

tokenizer = AutoTokenizer.from_pretrained(adapter_id)

base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    dtype=torch.float16,
    low_cpu_mem_usage=True,
    device_map="auto",
)

model = PeftModel.from_pretrained(base_model, adapter_id)
model.eval()
~~~

## Compute note

The 70B model requires substantially more GPU memory than the 7B model. For most users, start with:

~~~text
shyrcathy/rare-phenix-llama2-7b-raredis
~~~

Use this 70B adapter only in an appropriate multi-GPU or high-memory GPU environment.

## GitHub repository

Code and usage examples are available at:

~~~text
https://github.com/cathyshyr/RARE_PHENIX_for_rare_disease_phenotyping
~~~

See:

~~~text
docs/module1_hf_quickstart.md
~~~

## Intended use

This adapter is intended for research use in rare disease phenotype extraction workflows.

## Limitations

- This model is not intended for autonomous clinical diagnosis.
- Outputs should be reviewed by domain experts.
- The public adapter may perform differently from the UDN-trained RARE-PHENIX model described in the manuscript.
- The model may incorrectly tag negated findings, family history, or non-patient conditions; downstream post-processing and expert review are recommended.
- Do not send protected health information to environments that are not approved for PHI.

## Citation

If you use this adapter, please cite:

Shyr, C., Hu, Y., Tinker, R.J., Cassini, T.A., Byram, K.W., Hamid, R., Fabbri, D.V., Wright, A., Peterson, J.F., Bastarache, L., and Xu, H. 2026. *An artificial intelligence framework for end-to-end rare disease phenotyping from clinical notes using large language models*. arXiv preprint arXiv:2602.20324.