Improve model card: add pipeline tag, library name, and paper link

README.md

@@ -1,38 +1,43 @@
 ---
-license: apache-2.0
 base_model:
-  - Qwen/Qwen2-VL-7B-Instruct
-  - openai/whisper-large-v3-turbo
+- Qwen/Qwen2-VL-7B-Instruct
+- openai/whisper-large-v3-turbo
 language:
-  - en
-pipeline_tag: visual-question-answering
+- en
+license: apache-2.0
+pipeline_tag: video-text-to-text
+library_name: transformers
 tags:
-  - medical
-  - surgical
-  - multimodal
-  - audio
-  - video
-  - vision-language
-  - qwen2-vl
-  - whisper
-  - colonoscopy
-  - healthcare-ai
-  - qlora
+- medical
+- surgical
+- multimodal
+- audio
+- video
+- vision-language
+- qwen2-vl
+- whisper
+- colonoscopy
+- healthcare-ai
+- qlora
 ---
 
 # SurgViVQA-Audio: Audio-Adapted Qwen2-VL for Surgical Video QA
 
+This repository contains the weights for **SurgViVQA-Audio**, presented in the paper [SurgViVQA: Temporally-Grounded Video Question Answering for Surgical Scene Understanding](https://huggingface.co/papers/2511.03325).
+
 This model adapts a Whisper audio encoder to feed directly into Qwen2-VL, enabling **hands-free surgical video question answering** without intermediate ASR transcription.
 
 > ⚠️ **Research prototype only. Not for clinical use.**
 
 ## Model Description
 
+- **Paper:** [SurgViVQA: Temporally-Grounded Video Question Answering for Surgical Scene Understanding](https://huggingface.co/papers/2511.03325)
 - **Base Vision-Language Model:** [Qwen2-VL-7B-Instruct](https://huggingface.co/Qwen/Qwen2-VL-7B-Instruct)
 - **Audio Encoder:** [Whisper Large v3 Turbo](https://huggingface.co/openai/whisper-large-v3-turbo) (frozen)
 - **Audio Projector:** Linear layer (1280 → 3584) mapping Whisper features to Qwen2-VL embedding space
 - **Training Method:** QLoRA (4-bit base model + BF16 LoRA adapters)
 - **Domain:** Colonoscopy surgical procedures
+- **Code:** [GitHub Repository](https://github.com/kulsoom-abdullah/SurgViVQA-Audio)
 
 ### Why Skip ASR?
 
@@ -70,10 +75,6 @@ Built on the [SurgViVQA](https://github.com/madratak/SurgViVQA/) benchmark by Dr
 | Eval | 398 | 002-001, 002-002, 002-003 | Validation |
 | Test | 1,000 | 002-004 (held-out) | Generalization testing |
 
-**Split methodology:** Train/eval are split by question instances (stratified by question type); test is a completely held-out video (002-004) to evaluate generalization to unseen patient anatomy.
-
-**20 question types** across safety checks, tool detection, motion reasoning, and lesion assessment.
-
 ## Results
 
 ### Overall Performance
@@ -94,33 +95,6 @@ Built on the [SurgViVQA](https://github.com/madratak/SurgViVQA/) benchmark by Dr
 - Motion direction (5-way): **20%**
 - Spatial localization (4-way): **20%**
 
-### Known Limitations
-
-- **Temporal reasoning:** Frame-based processing limits motion understanding (20-50% on direction questions)
-- **Spatial precision:** 384px resolution constrains fine-grained localization
-- **Single TTS voice:** Trained on one synthetic voice; real speech robustness untested
-- **Limited procedures:** Trained on 3 colonoscopy videos; generalization to other surgical domains untested
-
-## Reproducing Results
-
-To evaluate on the held-out test set:
-
-```bash
-# Clone the repo
-git clone https://github.com/kulsoom-abdullah/SurgViVQA-Audio
-cd SurgViVQA-Audio
-
-# Run evaluation
-python3 src/evaluate_checkpoint.py \
-    --checkpoint_path ./checkpoints/surgical_vqa_multivideo \
-    --eval_data_path test_multivideo.jsonl \
-    --frames_dir data/frames \
-    --audio_dir data/audio \
-    --output_file results/eval_results.jsonl
-```
-
-See the [GitHub repository](https://github.com/kulsoom-abdullah/SurgViVQA-Audio) for full setup instructions.
-
 ## Usage
 
 ### Requirements
@@ -150,14 +124,6 @@ processor = AutoProcessor.from_pretrained("Qwen/Qwen2-VL-7B-Instruct")
 
 For full inference code including audio processing, see the [GitHub repository](https://github.com/kulsoom-abdullah/SurgViVQA-Audio).
 
-## Training Details
-
-- **Hardware:** 2× NVIDIA RTX 4090 (24GB each)
-- **Training Time:** ~6 hours
-- **LoRA Config:** Rank 64, Alpha 16
-- **Target Modules:** All attention + MLP projections
-- **Precision:** 4-bit NF4 quantization + BF16 adapters
-
 ## Citation
 
 ```bibtex
@@ -186,8 +152,4 @@ For full inference code including audio processing, see the [GitHub repository](
 
 - **Code:** [GitHub Repository](https://github.com/kulsoom-abdullah/SurgViVQA-Audio)
 - **Demo Video:** [Loom Walkthrough](https://www.loom.com/share/e6259484ed0f4ad2aac584860c0d32f0)
-- **Base Audio Adapter:** [Qwen2-VL-Audio-Adapter](https://github.com/kulsoom-abdullah/Qwen2-VL-Audio-Adapter)
-
-## Contact
-
-**[Kulsoom Abdullah](https://www.linkedin.com/in/kulsoomabdullah/)** — PhD in ECE, 10+ years in AI/ML, healthcare AI since 2020
+- **Contact:** [Kulsoom Abdullah](https://www.linkedin.com/in/kulsoomabdullah/)