Jarbas commited on
Commit
08211ff
·
verified ·
1 Parent(s): cb02e05

export: add triaan-vc ONNX artifacts (CPC encoder + TriAAN decoder + PWG vocoder)

Browse files
PROVENANCE.md ADDED
@@ -0,0 +1,42 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # PROVENANCE
2
+
3
+ | Field | Value |
4
+ |-------|-------|
5
+ | upstream_repo | https://github.com/winddori2002/TriAAN-VC |
6
+ | upstream_ref | v1.0 |
7
+ | export_script_sha | ae79a62aad4ccf2f8276559c2d42d0edfb8a5f35 |
8
+ | export_date | 2026-06-11T16:01:45Z |
9
+ | torch_version | 2.10.0+cu128 |
10
+ | onnx_version | 1.20.1 |
11
+ | platform | Linux-6.18.20-1-lts-x86_64-with-glibc2.43 |
12
+
13
+ ## Upstream licence
14
+
15
+ ```
16
+ MIT License
17
+
18
+ Copyright (c) 2022 winddori2002
19
+
20
+ Permission is hereby granted, free of charge, to any person obtaining a copy
21
+ of this software and associated documentation files (the "Software"), to deal
22
+ in the Software without restriction, including without limitation the rights
23
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
24
+ copies of the Software, and to permit persons to whom the Software is
25
+ furnished to do so, subject to the following conditions:
26
+
27
+ The above copyright notice and this permission notice shall be included in all
28
+ copies or substantial portions of the Software.
29
+
30
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
31
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
32
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
33
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
34
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
35
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
36
+ SOFTWARE.
37
+
38
+ Portions derived from:
39
+ facebookresearch/CPC_audio (MIT) — CPC encoder architecture and checkpoint
40
+ kan-bayashi/ParallelWaveGAN (MIT) — vocoder (via parallel_wavegan package)
41
+ Wendison/VQMIVC (MIT) — vocoder checkpoint
42
+ ```
README.md ADDED
@@ -0,0 +1,88 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ tags:
4
+ - voice-conversion
5
+ - any-to-any
6
+ - onnx
7
+ - vconnx
8
+ language:
9
+ - en
10
+ pipeline_tag: audio-to-audio
11
+ ---
12
+
13
+ # vconnx-triaan-vc
14
+
15
+ Any-to-any voice conversion using TriAAN-VC (Triple Adaptive Attention Normalization,
16
+ ICASSP 2023), exported to ONNX for CPU-first inference via the
17
+ [vconnx](https://github.com/TigreGotico/vconnx) engine adapter.
18
+
19
+ ## Model card
20
+
21
+ | Field | Value |
22
+ |---|---|
23
+ | Paper | [TriAAN-VC: Triple Adaptive Attention Normalization for Any-to-Any Voice Conversion](https://arxiv.org/abs/2303.09057) |
24
+ | Original code | [winddori2002/TriAAN-VC](https://github.com/winddori2002/TriAAN-VC) |
25
+ | License | MIT (original code and weights) |
26
+ | Sample rate | 16 000 Hz |
27
+ | Mel channels | 80 |
28
+ | FFT / hop / win | 400 / 160 / 400 samples |
29
+
30
+ ## Architecture
31
+
32
+ Three ONNX components chained in sequence:
33
+
34
+ 1. **CPC encoder** — 5-layer strided Conv1d (160× downsample) + LSTM; encodes speech
35
+ to 256-dim content features at 100 Hz. Derived from
36
+ [facebookresearch/CPC\_audio](https://github.com/facebookresearch/CPC_audio).
37
+ 2. **TriAAN-VC decoder** — SpeakerEncoder + ContentEncoder + BiGRU fusion +
38
+ TriAAN decoder + PostNet; maps CPC features + log-F0 → 80-band mel spectrogram.
39
+ 3. **ParallelWaveGAN vocoder** — WaveNet-style neural vocoder (MIT, kan-bayashi); mel
40
+ → 16 kHz waveform.
41
+
42
+ ## Files
43
+
44
+ | File | Size | Description |
45
+ |---|---|---|
46
+ | `cpc_encoder.onnx` | 7.0 MB | CPC content encoder (FP32) |
47
+ | `cpc_encoder_q8.onnx` | 1.8 MB | CPC encoder INT8 dynamic quantization |
48
+ | `triaan_vc.onnx` | 266.3 MB | TriAAN-VC decoder (FP32) |
49
+ | `triaan_vc_q8.onnx` | 76.3 MB | TriAAN-VC decoder INT8 dynamic quantization |
50
+ | `pwg_vocoder.onnx` | 7.0 MB | ParallelWaveGAN vocoder (FP32) |
51
+ | `pwg_vocoder_q8.onnx` | 2.0 MB | PWG vocoder INT8 dynamic quantization |
52
+ | `config.json` | — | vconnx component manifest |
53
+ | `PROVENANCE.md` | — | Upstream provenance + license text |
54
+
55
+ ## Parity (PyTorch vs ONNX Runtime, random input, T=50 frames)
56
+
57
+ | Component | max\_abs Δ | mean\_abs Δ | Verdict |
58
+ |---|---|---|---|
59
+ | CPC encoder | 1.06e-05 | 1.46e-07 | PASS |
60
+ | TriAAN-VC | 3.76e-06 | 6.39e-07 | PASS |
61
+ | ParallelWaveGAN | 4.39e-05 | 1.19e-06 | PASS |
62
+
63
+ All components pass with max\_abs < 5e-05 (well within perceptual threshold).
64
+
65
+ ## Usage
66
+
67
+ ```python
68
+ from vconnx import VoiceCloner
69
+
70
+ vc = VoiceCloner(engine="triaan")
71
+ output_wav = vc.clone_voice(source_wav, target_wav)
72
+ ```
73
+
74
+ Install:
75
+
76
+ ```bash
77
+ pip install "vconnx[triaan]"
78
+ ```
79
+
80
+ ## Provenance
81
+
82
+ Weights converted from the GitHub release v1.0 artifacts:
83
+
84
+ - `cpc.pt` — CPC encoder checkpoint (facebookresearch/CPC\_audio)
85
+ - `model-cpc-split.pth` — TriAAN-VC decoder checkpoint
86
+ - `vocoder.pkl` — ParallelWaveGAN vocoder checkpoint (originally from VQMIVC)
87
+
88
+ See `PROVENANCE.md` for full license text.
config.json ADDED
@@ -0,0 +1,38 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "engine": "triaan-vc",
3
+ "components": {
4
+ "cpc_encoder": "cpc_encoder.onnx",
5
+ "cpc_encoder_q8": "cpc_encoder_q8.onnx",
6
+ "triaan_vc": "triaan_vc.onnx",
7
+ "triaan_vc_q8": "triaan_vc_q8.onnx",
8
+ "pwg_vocoder": "pwg_vocoder.onnx",
9
+ "pwg_vocoder_q8": "pwg_vocoder_q8.onnx"
10
+ },
11
+ "sample_rates": {
12
+ "output": 16000
13
+ },
14
+ "distributable": true,
15
+ "metadata": {
16
+ "opset": 14,
17
+ "cpc_hidden": 256,
18
+ "cpc_downsample": 160,
19
+ "n_mels": 80,
20
+ "n_fft": 400,
21
+ "hop_length": 160,
22
+ "win_length": 400,
23
+ "parity": {
24
+ "cpc_encoder": {
25
+ "max_abs": 1.0609626770019531e-05,
26
+ "mean_abs": 1.455821347917663e-07
27
+ },
28
+ "triaan_vc": {
29
+ "max_abs": 3.7550926208496094e-06,
30
+ "mean_abs": 6.39064239749132e-07
31
+ },
32
+ "pwg_vocoder": {
33
+ "max_abs": 5.415081977844238e-05,
34
+ "mean_abs": 1.355652671009011e-06
35
+ }
36
+ }
37
+ }
38
+ }
cpc_encoder.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:78c763ca8195ab577abcf888628d7a14fb2461bf1a1da4368d820a1798cd4fd0
3
+ size 7378296
cpc_encoder_q8.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:433ce9bdff0714f80c3ecd08055453498bbeedcd4886409c9c3700707bc86c98
3
+ size 1873271
pwg_vocoder.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fc8aa30187347b78f3bee523a96ab7a79a344432361342ea800a5ffe0ba59786
3
+ size 7377434
pwg_vocoder_q8.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9e458f7a5022e5e50bdcd62fc3bab8c5816bb74e0acb5aa3678363b36c33edac
3
+ size 2146031
triaan_vc.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c48a7126ef8ab8880c114943016a963ded050740538261c14d7013544c23295d
3
+ size 279255768
triaan_vc_q8.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:35838da8a3ee5f2a1352ae9f33547e72209707bdb6e60dbbb4195af5bea5b98b
3
+ size 80040688