export: add triaan-vc ONNX artifacts (CPC encoder + TriAAN decoder + PWG vocoder)
Browse files- PROVENANCE.md +42 -0
- README.md +88 -0
- config.json +38 -0
- cpc_encoder.onnx +3 -0
- cpc_encoder_q8.onnx +3 -0
- pwg_vocoder.onnx +3 -0
- pwg_vocoder_q8.onnx +3 -0
- triaan_vc.onnx +3 -0
- triaan_vc_q8.onnx +3 -0
PROVENANCE.md
ADDED
|
@@ -0,0 +1,42 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# PROVENANCE
|
| 2 |
+
|
| 3 |
+
| Field | Value |
|
| 4 |
+
|-------|-------|
|
| 5 |
+
| upstream_repo | https://github.com/winddori2002/TriAAN-VC |
|
| 6 |
+
| upstream_ref | v1.0 |
|
| 7 |
+
| export_script_sha | ae79a62aad4ccf2f8276559c2d42d0edfb8a5f35 |
|
| 8 |
+
| export_date | 2026-06-11T16:01:45Z |
|
| 9 |
+
| torch_version | 2.10.0+cu128 |
|
| 10 |
+
| onnx_version | 1.20.1 |
|
| 11 |
+
| platform | Linux-6.18.20-1-lts-x86_64-with-glibc2.43 |
|
| 12 |
+
|
| 13 |
+
## Upstream licence
|
| 14 |
+
|
| 15 |
+
```
|
| 16 |
+
MIT License
|
| 17 |
+
|
| 18 |
+
Copyright (c) 2022 winddori2002
|
| 19 |
+
|
| 20 |
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
| 21 |
+
of this software and associated documentation files (the "Software"), to deal
|
| 22 |
+
in the Software without restriction, including without limitation the rights
|
| 23 |
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
| 24 |
+
copies of the Software, and to permit persons to whom the Software is
|
| 25 |
+
furnished to do so, subject to the following conditions:
|
| 26 |
+
|
| 27 |
+
The above copyright notice and this permission notice shall be included in all
|
| 28 |
+
copies or substantial portions of the Software.
|
| 29 |
+
|
| 30 |
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
| 31 |
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
| 32 |
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
| 33 |
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
| 34 |
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
| 35 |
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
| 36 |
+
SOFTWARE.
|
| 37 |
+
|
| 38 |
+
Portions derived from:
|
| 39 |
+
facebookresearch/CPC_audio (MIT) — CPC encoder architecture and checkpoint
|
| 40 |
+
kan-bayashi/ParallelWaveGAN (MIT) — vocoder (via parallel_wavegan package)
|
| 41 |
+
Wendison/VQMIVC (MIT) — vocoder checkpoint
|
| 42 |
+
```
|
README.md
ADDED
|
@@ -0,0 +1,88 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
tags:
|
| 4 |
+
- voice-conversion
|
| 5 |
+
- any-to-any
|
| 6 |
+
- onnx
|
| 7 |
+
- vconnx
|
| 8 |
+
language:
|
| 9 |
+
- en
|
| 10 |
+
pipeline_tag: audio-to-audio
|
| 11 |
+
---
|
| 12 |
+
|
| 13 |
+
# vconnx-triaan-vc
|
| 14 |
+
|
| 15 |
+
Any-to-any voice conversion using TriAAN-VC (Triple Adaptive Attention Normalization,
|
| 16 |
+
ICASSP 2023), exported to ONNX for CPU-first inference via the
|
| 17 |
+
[vconnx](https://github.com/TigreGotico/vconnx) engine adapter.
|
| 18 |
+
|
| 19 |
+
## Model card
|
| 20 |
+
|
| 21 |
+
| Field | Value |
|
| 22 |
+
|---|---|
|
| 23 |
+
| Paper | [TriAAN-VC: Triple Adaptive Attention Normalization for Any-to-Any Voice Conversion](https://arxiv.org/abs/2303.09057) |
|
| 24 |
+
| Original code | [winddori2002/TriAAN-VC](https://github.com/winddori2002/TriAAN-VC) |
|
| 25 |
+
| License | MIT (original code and weights) |
|
| 26 |
+
| Sample rate | 16 000 Hz |
|
| 27 |
+
| Mel channels | 80 |
|
| 28 |
+
| FFT / hop / win | 400 / 160 / 400 samples |
|
| 29 |
+
|
| 30 |
+
## Architecture
|
| 31 |
+
|
| 32 |
+
Three ONNX components chained in sequence:
|
| 33 |
+
|
| 34 |
+
1. **CPC encoder** — 5-layer strided Conv1d (160× downsample) + LSTM; encodes speech
|
| 35 |
+
to 256-dim content features at 100 Hz. Derived from
|
| 36 |
+
[facebookresearch/CPC\_audio](https://github.com/facebookresearch/CPC_audio).
|
| 37 |
+
2. **TriAAN-VC decoder** — SpeakerEncoder + ContentEncoder + BiGRU fusion +
|
| 38 |
+
TriAAN decoder + PostNet; maps CPC features + log-F0 → 80-band mel spectrogram.
|
| 39 |
+
3. **ParallelWaveGAN vocoder** — WaveNet-style neural vocoder (MIT, kan-bayashi); mel
|
| 40 |
+
→ 16 kHz waveform.
|
| 41 |
+
|
| 42 |
+
## Files
|
| 43 |
+
|
| 44 |
+
| File | Size | Description |
|
| 45 |
+
|---|---|---|
|
| 46 |
+
| `cpc_encoder.onnx` | 7.0 MB | CPC content encoder (FP32) |
|
| 47 |
+
| `cpc_encoder_q8.onnx` | 1.8 MB | CPC encoder INT8 dynamic quantization |
|
| 48 |
+
| `triaan_vc.onnx` | 266.3 MB | TriAAN-VC decoder (FP32) |
|
| 49 |
+
| `triaan_vc_q8.onnx` | 76.3 MB | TriAAN-VC decoder INT8 dynamic quantization |
|
| 50 |
+
| `pwg_vocoder.onnx` | 7.0 MB | ParallelWaveGAN vocoder (FP32) |
|
| 51 |
+
| `pwg_vocoder_q8.onnx` | 2.0 MB | PWG vocoder INT8 dynamic quantization |
|
| 52 |
+
| `config.json` | — | vconnx component manifest |
|
| 53 |
+
| `PROVENANCE.md` | — | Upstream provenance + license text |
|
| 54 |
+
|
| 55 |
+
## Parity (PyTorch vs ONNX Runtime, random input, T=50 frames)
|
| 56 |
+
|
| 57 |
+
| Component | max\_abs Δ | mean\_abs Δ | Verdict |
|
| 58 |
+
|---|---|---|---|
|
| 59 |
+
| CPC encoder | 1.06e-05 | 1.46e-07 | PASS |
|
| 60 |
+
| TriAAN-VC | 3.76e-06 | 6.39e-07 | PASS |
|
| 61 |
+
| ParallelWaveGAN | 4.39e-05 | 1.19e-06 | PASS |
|
| 62 |
+
|
| 63 |
+
All components pass with max\_abs < 5e-05 (well within perceptual threshold).
|
| 64 |
+
|
| 65 |
+
## Usage
|
| 66 |
+
|
| 67 |
+
```python
|
| 68 |
+
from vconnx import VoiceCloner
|
| 69 |
+
|
| 70 |
+
vc = VoiceCloner(engine="triaan")
|
| 71 |
+
output_wav = vc.clone_voice(source_wav, target_wav)
|
| 72 |
+
```
|
| 73 |
+
|
| 74 |
+
Install:
|
| 75 |
+
|
| 76 |
+
```bash
|
| 77 |
+
pip install "vconnx[triaan]"
|
| 78 |
+
```
|
| 79 |
+
|
| 80 |
+
## Provenance
|
| 81 |
+
|
| 82 |
+
Weights converted from the GitHub release v1.0 artifacts:
|
| 83 |
+
|
| 84 |
+
- `cpc.pt` — CPC encoder checkpoint (facebookresearch/CPC\_audio)
|
| 85 |
+
- `model-cpc-split.pth` — TriAAN-VC decoder checkpoint
|
| 86 |
+
- `vocoder.pkl` — ParallelWaveGAN vocoder checkpoint (originally from VQMIVC)
|
| 87 |
+
|
| 88 |
+
See `PROVENANCE.md` for full license text.
|
config.json
ADDED
|
@@ -0,0 +1,38 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"engine": "triaan-vc",
|
| 3 |
+
"components": {
|
| 4 |
+
"cpc_encoder": "cpc_encoder.onnx",
|
| 5 |
+
"cpc_encoder_q8": "cpc_encoder_q8.onnx",
|
| 6 |
+
"triaan_vc": "triaan_vc.onnx",
|
| 7 |
+
"triaan_vc_q8": "triaan_vc_q8.onnx",
|
| 8 |
+
"pwg_vocoder": "pwg_vocoder.onnx",
|
| 9 |
+
"pwg_vocoder_q8": "pwg_vocoder_q8.onnx"
|
| 10 |
+
},
|
| 11 |
+
"sample_rates": {
|
| 12 |
+
"output": 16000
|
| 13 |
+
},
|
| 14 |
+
"distributable": true,
|
| 15 |
+
"metadata": {
|
| 16 |
+
"opset": 14,
|
| 17 |
+
"cpc_hidden": 256,
|
| 18 |
+
"cpc_downsample": 160,
|
| 19 |
+
"n_mels": 80,
|
| 20 |
+
"n_fft": 400,
|
| 21 |
+
"hop_length": 160,
|
| 22 |
+
"win_length": 400,
|
| 23 |
+
"parity": {
|
| 24 |
+
"cpc_encoder": {
|
| 25 |
+
"max_abs": 1.0609626770019531e-05,
|
| 26 |
+
"mean_abs": 1.455821347917663e-07
|
| 27 |
+
},
|
| 28 |
+
"triaan_vc": {
|
| 29 |
+
"max_abs": 3.7550926208496094e-06,
|
| 30 |
+
"mean_abs": 6.39064239749132e-07
|
| 31 |
+
},
|
| 32 |
+
"pwg_vocoder": {
|
| 33 |
+
"max_abs": 5.415081977844238e-05,
|
| 34 |
+
"mean_abs": 1.355652671009011e-06
|
| 35 |
+
}
|
| 36 |
+
}
|
| 37 |
+
}
|
| 38 |
+
}
|
cpc_encoder.onnx
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:78c763ca8195ab577abcf888628d7a14fb2461bf1a1da4368d820a1798cd4fd0
|
| 3 |
+
size 7378296
|
cpc_encoder_q8.onnx
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:433ce9bdff0714f80c3ecd08055453498bbeedcd4886409c9c3700707bc86c98
|
| 3 |
+
size 1873271
|
pwg_vocoder.onnx
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:fc8aa30187347b78f3bee523a96ab7a79a344432361342ea800a5ffe0ba59786
|
| 3 |
+
size 7377434
|
pwg_vocoder_q8.onnx
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9e458f7a5022e5e50bdcd62fc3bab8c5816bb74e0acb5aa3678363b36c33edac
|
| 3 |
+
size 2146031
|
triaan_vc.onnx
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c48a7126ef8ab8880c114943016a963ded050740538261c14d7013544c23295d
|
| 3 |
+
size 279255768
|
triaan_vc_q8.onnx
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:35838da8a3ee5f2a1352ae9f33547e72209707bdb6e60dbbb4195af5bea5b98b
|
| 3 |
+
size 80040688
|