multimodalart HF Staff commited on
Commit
dbff0a2
·
verified ·
1 Parent(s): 20ade0d

Document measured runtime and the soundfile WAV-write delta

Browse files
Files changed (1) hide show
  1. README.md +5 -1
README.md CHANGED
@@ -34,7 +34,8 @@ so the picture and the soundtrack are denoised together rather than dubbed after
34
  ## What it produces
35
 
36
  The checkpoint's native shape: **121 frames · 256×256 · 24 fps · 48 kHz audio (~5 s)**,
37
- muxed into a single MP4. A run takes a few minutes on ZeroGPU.
 
38
 
39
  ## Faithfulness to the reference implementation
40
 
@@ -65,6 +66,9 @@ Deployment-only deltas from the upstream tree, all of which leave the numerics u
65
  optional.
66
  - `joint_av/validation.py` — the denoising loop is wrapped in `tqdm` so the UI can show
67
  progress.
 
 
 
68
  - Components are loaded on CPU and then moved with `.to("cuda")`; upstream's
69
  `device="cuda"` path uses `device_map={"": "cuda:0"}` + `torch.cuda.set_device`, which
70
  ZeroGPU does not support.
 
34
  ## What it produces
35
 
36
  The checkpoint's native shape: **121 frames · 256×256 · 24 fps · 48 kHz audio (~5 s)**,
37
+ muxed into a single MP4. A default 50-step run takes ~97 s on ZeroGPU (measured:
38
+ ~1.9 s/step plus ~3 s for text encoding, VAE decoding and muxing).
39
 
40
  ## Faithfulness to the reference implementation
41
 
 
66
  optional.
67
  - `joint_av/validation.py` — the denoising loop is wrapped in `tqdm` so the UI can show
68
  progress.
69
+ - `torchaudio.save` is routed through `soundfile` in `app.py`: torchaudio ≥ 2.10 delegates
70
+ `save` to `torchcodec`, whose wheels are bound to a specific FFmpeg ABI. Both write the
71
+ same 48 kHz PCM WAV.
72
  - Components are loaded on CPU and then moved with `.to("cuda")`; upstream's
73
  `device="cuda"` path uses `device_map={"": "cuda:0"}` + `torch.cuda.set_device`, which
74
  ZeroGPU does not support.