VibeVoice 7B - Low VRAM Quantized Models
Pre-quantized versions of VibeVoice 7B for low VRAM GPUs.
Available Versions
- 4bit/ - 4-bit quantized model (~8GB VRAM needed)
- 8bit/ - 8-bit quantized model (~12GB VRAM needed) - NOTE: Removed 8 bit until I can test it again. I'll re-up it soon.
Usage
from vibevoice.modular.modeling_vibevoice_inference import VibeVoiceForConditionalGenerationInference
from vibevoice.processor.vibevoice_processor import VibeVoiceProcessor
# For 4-bit model
model = VibeVoiceForConditionalGenerationInference.from_pretrained(
"Dannidee/VibeVoice7b-low-vram/4bit",
device_map='cuda',
torch_dtype=torch.bfloat16,
)
processor = VibeVoiceProcessor.from_pretrained("Dannidee/VibeVoice7b-low-vram/4bit")
VRAM Requirements
- 4-bit: ~8 GB total VRAM
- 8-bit: ~12 GB total VRAM
- Original: ~19 GB total VRAM
See individual model folders for detailed information.
import torch, soundfile as sf, librosa, numpy as np from vibevoice.processor.vibevoice_processor import VibeVoiceProcessor from vibevoice.modular.modeling_vibevoice_inference import VibeVoiceForConditionalGenerationInference # Load voice sample (should be 24kHz mono) voice, sr = sf.read("path/to/voice_sample.wav") if voice.ndim > 1: voice = voice.mean(axis=1) if sr != 24000: voice = librosa.resample(voice, sr, 24000) processor = VibeVoiceProcessor.from_pretrained("DevParker/VibeVoice7b-low-vram") model = VibeVoiceForConditionalGenerationInference.from_pretrained( "DevParker/VibeVoice7b-low-vram", torch_dtype=torch.bfloat16 ).to("cuda").eval() model.set_ddpm_inference_steps(5) inputs = processor(text=["Speaker 0: Hello!\nSpeaker 1: Hi there!"], voice_samples=[[voice]], return_tensors="pt") audio = model.generate(**inputs, cfg_scale=1.3, tokenizer=processor.tokenizer).speech_outputs[0] sf.write("output.wav", audio.cpu().numpy().squeeze(), 24000)