File size: 4,726 Bytes
832fee5 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 | ---
license: apache-2.0
base_model:
- ByteDance/Bernini-R
tags:
- text-to-video
- image-to-video
- video-to-video
- image-to-image
- audio-to-video
- wan-s2v
- bernini-r-s2v
---
# Bernini-R-S2V
## ComfyUI Bernini-R-S2V custom node v2 (update)
**Bernini S2V Conditioning v2** - Bernini in-context video/image conditioning with masked lip-sync for one or two speakers.
<video controls width="100%" height="480">
<source src="https://huggingface.co/rzgar/Bernini-R-S2V/resolve/main/video/ComfyUI__00003-audio.mp4" type="video/mp4">
Your browser does not support the video tag.
</video>
Unzip [ComfyUI-WanBerniniS2V_v2.zip](https://huggingface.co/rzgar/Bernini-R-S2V/resolve/main/ComfyUI-WanBerniniS2V_v2/ComfyUI-WanBerniniS2V_v2.zip) into `ComfyUI/custom_nodes/`, then restart ComfyUI.
```
or save the Python files in: ComfyUI/custom_nodes/ComfyUI-WanBerniniS2V_v2/
```
Disable or remove the older `ComfyUI-WanBerniniS2V` folder if you only want v2.
### One speaker
- `audio_1` + `mask_1`
- Leave `audio_2` / `mask_2` unwired
### Two speakers (dialog)
- `audio_1` + `mask_1` - first speaker
- `audio_2` + `mask_2` - second speaker
- `speaker_2_start_frame = -1` - second audio starts when the first clip ends
### Masks
Paint on the **output frame** where each speaker's face appears. Masks control lip-sync placement only, they are not tied to `reference_image_N` slots.
<img src="https://huggingface.co/rzgar/Bernini-R-S2V/resolve/main/ComfyUI-WanBerniniS2V_v2/demo_assets/Screenshot_ComfyUI-WanBerniniS2V_v2.png" width="1280" height="720" />
________
<video controls width="100%" height="480">
<source src="https://huggingface.co/rzgar/Bernini-R-S2V/resolve/main/Bernini-R-S2V-FP8/video/ComfyUI__00005-audio.mp4" type="video/mp4">
Your browser does not support the video tag.
</video>
*Speech-driven video on Bernini-R , T2V, I2V, and V2V with lip-sync.*
This model adds single-speaker audio support to Bernini-R, so you can drive video with speech in text-to-video, image-to-video, and video-to-video setups.
It is not state-of-the-art audio-to-video, but it removes the need for post-processing or extra models just to add speech to Wan videos.
For basic talking-head work, or longer videos built from short sequences, it is a handy all-in-one option on top of Bernini's motion and editing strengths.
| File | Role |
|---|---|
| '[wan2.2_bernini_r_high_noise_fp16_s2v.safetensors](https://huggingface.co/rzgar/Bernini-R-S2V/resolve/main/Bernini-R-S2V-FP16/wan2.2_bernini_r_high_noise_fp16_s2v.safetensors)'| High noise |
| '[wan2.2_bernini_r_low_noise_fp16_s2v.safetensors](https://huggingface.co/rzgar/Bernini-R-S2V/resolve/main/Bernini-R-S2V-FP16/wan2.2_bernini_r_low_noise_fp16_s2v.safetensors)'| Low noise |
| '[wan2.2_bernini_r_high_noise_fp8_scaled_s2v.safetensors](https://huggingface.co/rzgar/Bernini-R-S2V/resolve/main/Bernini-R-S2V-FP8/wan2.2_bernini_r_high_noise_fp8_scaled_s2v.safetensors)' | High noise |
| '[wan2.2_bernini_r_low_noise_fp8_scaled_s2v.safetensors](https://huggingface.co/rzgar/Bernini-R-S2V/resolve/main/Bernini-R-S2V-FP8/wan2.2_bernini_r_low_noise_fp8_scaled_s2v.safetensors)' | Low noise |
| '[wan2.2_bernini_r_high_noise_int8_convrot_s2v.safetensors](https://huggingface.co/rzgar/Bernini-R-S2V/resolve/main/Bernini-R-S2V-int8-convrot/wan2.2_bernini_r_high_noise_int8_convrot_s2v.safetensors)' | High noise |
| '[wan2.2_bernini_r_low_noise_int8_convrot_s2v.safetensors](https://huggingface.co/rzgar/Bernini-R-S2V/resolve/main/Bernini-R-S2V-int8-convrot/wan2.2_bernini_r_low_noise_int8_convrot_s2v.safetensors)' | Low noise |
ComfyUI detects these as **`WAN22_S2V`** / 'WanModel_S2V' (audio keys trigger S2V model type).
## Usage
1. Download the checkpoints.
2. Place in:
```
ComfyUI/models/diffusion_models/
```
3. Add [wav2vec2](https://huggingface.co/rzgar/Bernini-R-S2V/resolve/main/audio_encoders/wav2vec2_large_english_fp16.safetensors) to:
```
ComfyUI/models/audio_encoders/
```
4. Install **[ComfyUI-WanBerniniS2V](https://huggingface.co/rzgar/Bernini-R-S2V/tree/main/ComfyUI-WanBerniniS2V)** custom node:
```
(Create a folder in 'ComfyUI/custom_nodes/' named 'ComfyUI-WanBerniniS2V', then save the Python files in that folder.)
```
5. Restart ComfyUI.
6. Search for the Bernini S2V Conditioning node or use the example [workfllow](https://huggingface.co/rzgar/Bernini-R-S2V/blob/main/Workflow/Bernini-R-S2V_Workflow.json)
## Audio tips
| Setting | Recommendation |
|---|---|
| **Channels** | **Mono**, wav2vec2 downmixes stereo internally |
| **Sample rate** | 44.1 khz or 48 khz (resampled to 16 kHz) |
| **Content** | Clear speech, less background music = better sync |
| **Length** | in my testing, max 9 to 15 seconds |
|