reiscook/pocket-voice-cleanup-demucs-htdemucs-coreml

HTDemucs Core ML Voice Cleanup

Core ML export of the real-valued separator core from Meta's Demucs 4.0.1 htdemucs model.

Model choice

Demucs 4.0.1 is still the latest stable Demucs release. The upstream repository documents an unreleased 4.1.0a1 section, but the stable package remains 4.0.1. The base htdemucs checkpoint is used here instead of htdemucs_ft because the fine-tuned variant is a heavier bag of per-source models.

Upstream model details:

• Demucs version: 4.0.1
• Model id: htdemucs
• Checkpoint: 955717e8
• Source order: drums, bass, other, vocals
• Vocals source index: 3
• Weights: upstream Demucs htdemucs

Why this is a separator-core export

The full waveform HTDemucs graph uses complex-valued STFT and ISTFT tensors. Core ML does not accept those complex tensors through the PyTorch converter, so the export boundary is the internal cac=True separator boundary:

• Preprocessing performs the Demucs STFT and feeds real/imaginary channels.
• Core ML runs the learned separator core.
• Postprocessing reconstructs the vocals waveform and downmixes it.

Files

• manifest.json: runtime preprocessing contract.
• htdemucs_separator_core.mlpackage: Core ML model package.
• scripts/export_htdemucs_coreml_core.py: export script used to generate the Core ML package and manifest.

The package was exported with coremltools as an ML Program targeting iOS 26 and checked with xcrun coremlcompiler compile.

Reference processing flow

1. Decode the input sample to PCM.
2. Resample to 44.1 kHz.
3. Duplicate mono to stereo.
4. Chunk and pad to 343980 samples.
5. Build the Demucs STFT tensor.
6. Run the Core ML model.
7. Reconstruct the vocals stem.
8. Downmix to mono.

License

This Core ML package contains the upstream Demucs htdemucs weights, which are provided for research purposes only under the upstream Demucs terms.