---
license: mit
language:
  - en
  - he
tags:
  - vocoder
  - audio
  - speech
  - streaming
  - vocos
base_model: charactr/vocos-mel-24khz
---

# Vocos (mel, 24 kHz) with causal streaming support

Causal fine-tune of [charactr/vocos-mel-24khz](https://huggingface.co/charactr/vocos-mel-24khz) for real-time streaming inference.
Mel frames go in chunk by chunk, audio comes out with ~85 ms algorithmic latency (4-frame lookahead + ISTFT overlap-add),
numerically identical to the offline forward pass.

Code, streaming inference (`VocosStreamer`), and training setup: **https://github.com/thewh1teagle/vocos**

## Model details

- Backbone: `VocosBackbone` (dim 512, intermediate 1536, 8 ConvNeXt layers) with `causal=True`, `lookahead_frames=4`
- Head: `ISTFTHead`, n_fft 1024, hop 256, padding "same"
- Features: 100-bin mel, 24 kHz
- Initialized from the pretrained non-causal `charactr/vocos-mel-24khz` weights (weight-compatible)

## Training data

| Phase | Data | Steps |
|---|---|---|
| 1 | LibriTTS-R (train-clean, 148,694 files, ~460 h) | 0 → 235k |
| 2 | LibriTTS-R + 20% Hebrew podcast speech (37,174 segments, ~76 h, quality-filtered, upsampled 16→24 kHz) | 235k → current |

Trained with the standard Vocos GAN objective (multi-period + multi-resolution discriminators, mel + adversarial + feature-matching losses),
batch 16, 16384-sample crops. This checkpoint is a full PyTorch Lightning checkpoint (~epoch 14, step ~277k) including generator,
discriminators, and optimizer states — usable both for inference (filter the `feature_extractor.`/`backbone.`/`head.` keys) and to
resume training.

## Usage

See [examples/streaming_infer_ckpt.py](https://github.com/thewh1teagle/vocos/blob/main/examples/streaming_infer_ckpt.py)
for loading this checkpoint and streaming audio chunk by chunk.