amibaid/acpo-videollama2

ACPO: Audio-Contrastive Preference Optimization

VideoLLaMA2.1-7B-AV, fine-tuned with contrastive DPO on the audio projector so it grounds captions in the audio that's actually present instead of hallucinating sounds from visual cues.

Checkpoint from "Don't Let the Video Speak: Audio-Contrastive Preference Optimization for Audio-Visual Language Models" (ECCV 2026).

Project page: https://vision.cs.utexas.edu/projects/acpo/
Code and benchmark: github.com/amibaid/ACPO

Training: 5k VALOR clips, audio projector only, β=0.1, lr=2e-5, 10 frames.

Usage

Standard VideoLLaMA2 checkpoint — load with the VideoLLaMA2 codebase.

huggingface-cli download amibaid/acpo-videollama2 --local-dir ckpt

Citation

@inproceedings{acpo2026,
  title  = {Don't Let the Video Speak: Audio-Contrastive Preference Optimization for Audio-Visual Language Models},
  author = {Baid, Ami and Xue, Zihui and Grauman, Kristen},
  booktitle = {ECCV},
  year   = {2026}
}

Released under Apache-2.0. The weights are additionally subject to the VideoLLaMA2 base-model license; the benchmark to AVHBench; training data to VALOR.