| --- |
| library_name: peft |
| base_model: |
| - unsloth/Llama-3.2-11B-Vision-Instruct |
| datasets: |
| - eltorio/ROCOv2-radiology |
| --- |
| |
| # Model Card for Llama-3.2 11b Vision Medical |
|
|
| <img src="https://i5.walmartimages.com/seo/DolliBu-Beige-Llama-Doctor-Plush-Toy-Super-Soft-Stuffed-Animal-Dress-Up-Cute-Scrub-Uniform-Cap-Outfit-Fluffy-Gift-11-Inches_e78392b2-71ef-4e26-a23f-8bb0b0e2043a.70c3b5988d390cf43d799758a826f2a5.jpeg" alt="drawing" width="400"/> |
|
|
| <font color="FF0000" size="5"><b> |
| This is a vision-language model fine-tuned for radiographic image analysis</b></font> |
| <br><b>Foundation Model: https://huggingface.co/unsloth/Llama-3.2-11B-Vision-Instruct<br/> |
| Dataset: https://huggingface.co/datasets/eltorio/ROCOv2-radiology<br/></b> |
|
|
| The model has been fine-tuned using CUDA-enabled GPU hardware. |
|
|
| ## Model Details |
|
|
| The model is based upon the foundation model: unsloth/Llama-3.2-11B-Vision-Instruct.<br/> |
| It has been tuned with Supervised Fine-tuning Trainer and PEFT LoRA with vision-language capabilities. |
|
|
| ### Libraries |
| - unsloth |
| - transformers |
| - torch |
| - datasets |
| - trl |
| - peft |
|
|
| ## Bias, Risks, and Limitations |
|
|
| To optimize training efficiency, the model has been trained on a subset of the ROCOv2-radiology dataset (1/7th of the total dataset).<br/> |
|
|
| <font color="FF0000"> |
| Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model.<br/> |
| The model's performance is directly dependent on the quality and diversity of the training data. Medical diagnosis should always be performed by qualified healthcare professionals.<br/> |
| Generation of plausible yet incorrect medical interpretations could occur and should not be used as the sole basis for clinical decisions. |
| </font> |
|
|
| ## Training Details |
|
|
| ### Training Parameters |
| - per_device_train_batch_size = 2 |
| - gradient_accumulation_steps = 16 |
| - num_train_epochs = 3 |
| - learning_rate = 5e-5 |
| - weight_decay = 0.02 |
| - lr_scheduler_type = "linear" |
| - max_seq_length = 2048 |
|
|
| ### LoRA Configuration |
| - r = 32 |
| - lora_alpha = 32 |
| - lora_dropout = 0 |
| - bias = "none" |
|
|
| ### Hardware Requirements |
| The model was trained using CUDA-enabled GPU hardware. |
|
|
| ### Training Statistics |
| - Training duration: 40,989 seconds (approximately 683 minutes) |
| - Peak reserved memory: 12.8 GB |
| - Peak reserved memory for training: 3.975 GB |
| - Peak reserved memory % of max memory: 32.3% |
| - Peak reserved memory for training % of max memory: 10.1% |
|
|
| ### Training Data |
| The model was trained on the ROCOv2-radiology dataset, which contains radiographic images and their corresponding medical descriptions. . |
|
|
| The training set was reduced to 1/7th of the original size for computational efficiency. |
|
|
| ## Usage |
|
|
| The model is designed to provide detailed descriptions of radiographic images. It can be prompted with: |
| ```python |
| instruction = "You are an expert radiographer. Describe accurately what you see in this image." |
| ``` |
|
|
| ## Model Access |
|
|
| The model is available on Hugging Face Hub at: bouthros/llma32_11b_vision_medical |
| |
| ## Citation |
| |
| If you use this model, please cite the original ROCOv2-radiology dataset and the Llama-3.2-11B-Vision-Instruct base model. |
