ONNX
voice-conversion
speech
audio
Jarbas commited on
Commit
14d48bc
·
verified ·
1 Parent(s): 7d30c31

export: add FACodec V2 ONNX artifacts (apache-2.0)

Browse files
PROVENANCE.md ADDED
@@ -0,0 +1,25 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # PROVENANCE
2
+
3
+ | Field | Value |
4
+ |-------|-------|
5
+ | upstream_repo | https://github.com/open-mmlab/Amphion |
6
+ | upstream_ref | main |
7
+ | export_script_sha | 120a3035cced6cb7dc1fc5458efe6efa5ac049db |
8
+ | export_date | 2026-06-12T15:01:04Z |
9
+ | torch_version | 2.10.0+cu128 |
10
+ | onnx_version | 1.20.1 |
11
+ | platform | Linux-6.18.20-1-lts-x86_64-with-glibc2.43 |
12
+
13
+ ## Upstream licence
14
+
15
+ ```
16
+ Apache License
17
+ Version 2.0, January 2004
18
+ http://www.apache.org/licenses/
19
+
20
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
21
+ <...full Apache-2.0 text; see https://www.apache.org/licenses/LICENSE-2.0>
22
+
23
+ Copyright 2023 Amphion Authors (open-mmlab/Amphion).
24
+ Weights: amphion/naturalspeech3_facodec on Hugging Face (apache-2.0).
25
+ ```
README.md ADDED
@@ -0,0 +1,70 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ tags:
4
+ - voice-conversion
5
+ - speech
6
+ - onnx
7
+ - audio
8
+ ---
9
+
10
+ # TigreGotico/vconnx-facodec
11
+
12
+ ONNX artifacts for the FACodec (NaturalSpeech 3) voice-conversion engine,
13
+ part of [vconnx](https://github.com/TigreGotico/vconnx).
14
+
15
+ ## License
16
+
17
+ Upstream weights: **Apache-2.0** — [amphion/naturalspeech3_facodec](https://huggingface.co/amphion/naturalspeech3_facodec).
18
+ Code: Amphion (open-mmlab/Amphion) — Apache-2.0.
19
+ ONNX exports: Apache-2.0 (same upstream license, stated here per vconnx publish-all policy).
20
+
21
+ ## Architecture
22
+
23
+ FACodec (Ju et al., NaturalSpeech 3, ICML 2024) disentangles speech into
24
+ content, prosody, timbre, and acoustic detail subspaces. Voice conversion is
25
+ zero-shot: encode source → quantize → swap reference timbre embedding → decode.
26
+
27
+ | Component | Input → Output | Description |
28
+ |---|---|---|
29
+ | | wav(1,1,N) → enc_feats(1,256,T) | Convolutional encoder (hop=200) |
30
+ | | enc_feats(1,256,T) → spk_embs(1,256) | TransformerEncoder timbre extractor |
31
+ | | (enc_feats, mel_20(1,20,T)) → vq_ids(6,1,T) | Factorised VQ — prosody(1)+content(2)+residual(3) |
32
+ | | (vq_ids, spk_embs) → wav(1,1,N) | vq2emb + AdaIN + conv decoder |
33
+
34
+ ## Parity (fp32 torch vs ORT)
35
+
36
+ | Component | max_abs Δ | mean_abs Δ | Verdict |
37
+ |---|---|---|---|
38
+ | encoder | 1.62e-05 | 2.36e-06 | PASS |
39
+ | timbre | 1.43e-06 | 6.40e-08 | PASS |
40
+ | quantize | exact int64 match | — | PASS |
41
+ | decoder | 7.50e-09 | 1.46e-09 | PASS |
42
+
43
+ ## Model sizes
44
+
45
+ | File | Size |
46
+ |---|---|
47
+ | (fp32) | 16.5 MB |
48
+ | (INT8) | 4.7 MB |
49
+ | (fp32) | 33.0 MB |
50
+ | (INT8) | 12.1 MB |
51
+ | (fp32) | 33.4 MB |
52
+ | (INT8) | 12.5 MB |
53
+ | (fp32) | 66.2 MB |
54
+ | (INT8) | 36.8 MB |
55
+
56
+ ## Intelligibility (WER gate ≤ 25%)
57
+
58
+ Tested with edge-tts source (en-US-GuyNeural) converted to two reference voices
59
+ (en-US-AriaNeural, en-GB-SoniaNeural) using Whisper base.en transcription.
60
+
61
+ | Reference voice | WER | Gate |
62
+ |---|---|---|
63
+ | en-US-AriaNeural | 0% | ✓ PASS |
64
+ | en-GB-SoniaNeural | 0% | ✓ PASS |
65
+
66
+ ## References
67
+
68
+ - https://arxiv.org/abs/2403.03100
69
+ - https://huggingface.co/amphion/naturalspeech3_facodec
70
+ - https://github.com/TigreGotico/vconnx
config.json ADDED
@@ -0,0 +1,66 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "engine": "facodec",
3
+ "components": {
4
+ "facodec_encoder": "facodec_encoder.onnx",
5
+ "facodec_encoder_q8": "facodec_encoder_q8.onnx",
6
+ "facodec_timbre": "facodec_timbre.onnx",
7
+ "facodec_timbre_q8": "facodec_timbre_q8.onnx",
8
+ "facodec_quantize": "facodec_quantize.onnx",
9
+ "facodec_quantize_q8": "facodec_quantize_q8.onnx",
10
+ "facodec_decoder": "facodec_decoder.onnx",
11
+ "facodec_decoder_q8": "facodec_decoder_q8.onnx"
12
+ },
13
+ "sample_rates": {
14
+ "output": 16000
15
+ },
16
+ "distributable": true,
17
+ "metadata": {
18
+ "opset": 14,
19
+ "upstream_repo": "amphion/naturalspeech3_facodec",
20
+ "license": "apache-2.0",
21
+ "n_quantizers": 6,
22
+ "vq_num_q_p": 1,
23
+ "vq_num_q_c": 2,
24
+ "vq_num_q_r": 3,
25
+ "hop_length": 200,
26
+ "parity": {
27
+ "encoder": {
28
+ "max_abs": 1.621246337890625e-05,
29
+ "mean_abs": 2.3622947082913015e-06,
30
+ "pass": true
31
+ },
32
+ "timbre": {
33
+ "max_abs": 1.430511474609375e-06,
34
+ "mean_abs": 6.403433872037567e-08,
35
+ "pass": true
36
+ },
37
+ "quantize": {
38
+ "exact_match": true,
39
+ "pass": true
40
+ },
41
+ "decoder": {
42
+ "max_abs": 7.49787432141602e-09,
43
+ "mean_abs": 1.4579514262536009e-09,
44
+ "pass": true
45
+ }
46
+ },
47
+ "sizes_mb": {
48
+ "facodec_encoder": {
49
+ "fp32_mb": 16.5,
50
+ "q8_mb": 4.7
51
+ },
52
+ "facodec_timbre": {
53
+ "fp32_mb": 33.0,
54
+ "q8_mb": 12.1
55
+ },
56
+ "facodec_quantize": {
57
+ "fp32_mb": 33.4,
58
+ "q8_mb": 12.5
59
+ },
60
+ "facodec_decoder": {
61
+ "fp32_mb": 66.2,
62
+ "q8_mb": 36.8
63
+ }
64
+ }
65
+ }
66
+ }
facodec_decoder.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b49cbf948c06f98fe7a642f3a081c2992276c350da00ab15e0b45b1a63b0bf73
3
+ size 69398668
facodec_decoder_q8.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:75316291882b9d14653ffc6da4c3f85ad724fe8a71f40f8096b039cdd09bda69
3
+ size 38604746
facodec_encoder.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d0d327a05d7bc1ec2a53cd6f386b7dd6b0c6c01d23a915eb640403ab44eec0ab
3
+ size 17285976
facodec_encoder_q8.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:db6c42b9d5e0ad10353ba5a193d7794ba6dfe039c958571ede56c1eb2777a658
3
+ size 4979562
facodec_quantize.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:197e5f4cb36c3bc3dcb74bd4a4d9ecea6becb685c2bc13ee0058d1ed991763bc
3
+ size 35022902
facodec_quantize_q8.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0db5c5243d08285f4609e623df202936a4bfab04bf1b338ba275b6e49132f8b2
3
+ size 13061686
facodec_timbre.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:865b106d26932fc33f53173cb6a392dd5f9d117a8a51fb9ed592f87508c107ca
3
+ size 34621660
facodec_timbre_q8.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7102d36a9621de25a52a46c9b3ec1e7c731429d7140d7de22de8e57d0dcef73f
3
+ size 12659511