thewh1teagle commited on
Commit
45caba9
·
verified ·
1 Parent(s): 8bab5d0

Upload folder using huggingface_hub

Browse files
Files changed (2) hide show
  1. README.md +45 -0
  2. last.ckpt +3 -0
README.md ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ language:
4
+ - en
5
+ - he
6
+ tags:
7
+ - vocoder
8
+ - audio
9
+ - speech
10
+ - streaming
11
+ - vocos
12
+ base_model: charactr/vocos-mel-24khz
13
+ ---
14
+
15
+ # Vocos (mel, 24 kHz) with causal streaming support
16
+
17
+ Causal fine-tune of [charactr/vocos-mel-24khz](https://huggingface.co/charactr/vocos-mel-24khz) for real-time streaming inference.
18
+ Mel frames go in chunk by chunk, audio comes out with ~85 ms algorithmic latency (4-frame lookahead + ISTFT overlap-add),
19
+ numerically identical to the offline forward pass.
20
+
21
+ Code, streaming inference (`VocosStreamer`), and training setup: **https://github.com/thewh1teagle/vocos**
22
+
23
+ ## Model details
24
+
25
+ - Backbone: `VocosBackbone` (dim 512, intermediate 1536, 8 ConvNeXt layers) with `causal=True`, `lookahead_frames=4`
26
+ - Head: `ISTFTHead`, n_fft 1024, hop 256, padding "same"
27
+ - Features: 100-bin mel, 24 kHz
28
+ - Initialized from the pretrained non-causal `charactr/vocos-mel-24khz` weights (weight-compatible)
29
+
30
+ ## Training data
31
+
32
+ | Phase | Data | Steps |
33
+ |---|---|---|
34
+ | 1 | LibriTTS-R (train-clean, 148,694 files, ~460 h) | 0 → 235k |
35
+ | 2 | LibriTTS-R + 20% Hebrew podcast speech (37,174 segments, ~76 h, quality-filtered, upsampled 16→24 kHz) | 235k → current |
36
+
37
+ Trained with the standard Vocos GAN objective (multi-period + multi-resolution discriminators, mel + adversarial + feature-matching losses),
38
+ batch 16, 16384-sample crops. This checkpoint is a full PyTorch Lightning checkpoint (~epoch 14, step ~277k) including generator,
39
+ discriminators, and optimizer states — usable both for inference (filter the `feature_extractor.`/`backbone.`/`head.` keys) and to
40
+ resume training.
41
+
42
+ ## Usage
43
+
44
+ See [examples/streaming_infer_ckpt.py](https://github.com/thewh1teagle/vocos/blob/main/examples/streaming_infer_ckpt.py)
45
+ for loading this checkpoint and streaming audio chunk by chunk.
last.ckpt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ccd8bfedd6511cb38278f620e4d441ec62808de65e3106c924c3c5a05fd67088
3
+ size 673556924