Arsh9210 commited on
Commit
fa44d96
·
verified ·
1 Parent(s): c44d570

Added enhancement_VAE/README.md

Browse files
Files changed (1) hide show
  1. enhancement_VAE/README.md +85 -0
enhancement_VAE/README.md ADDED
@@ -0,0 +1,85 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Audex Enhancement VAE
2
+
3
+ This directory contains the Audex enhancement VAE used to convert
4
+ XCodec1-decoded 16 kHz mono WAV audio into enhanced 48 kHz mono WAV audio.
5
+
6
+ The enhancement model is a postprocessor for Audex text-to-audio generation:
7
+
8
+ ```text
9
+ Audex generation -> XCodec1 decode -> 16 kHz WAV -> Enhancement VAE -> 48 kHz WAV
10
+ ```
11
+
12
+ The model is intended for XCodec1-decoded Audex audio as inputs. It is not a general
13
+ purpose enhancer for arbitrary audio.
14
+
15
+
16
+ ## Requirements
17
+ - `torch, numpy, scipy`
18
+
19
+ ## Command Line Usage
20
+
21
+ For a folder of outputs:
22
+
23
+ ```bash
24
+ python enhancement_VAE/enhance_audio_48k.py \
25
+ --input tta_outputs \
26
+ --output-dir tta_outputs_enhanced_48k
27
+ ```
28
+
29
+ For a single WAV file:
30
+
31
+ ```bash
32
+ python enhancement_VAE/enhance_audio_48k.py \
33
+ --input tta_outputs/example.wav \
34
+ --output-dir tta_outputs_enhanced_48k
35
+ ```
36
+
37
+ Outputs are written as:
38
+
39
+ ```text
40
+ <input_stem>_enhanced_48k.wav
41
+ ```
42
+
43
+ Supported options:
44
+
45
+ - `--device`: inference device. Defaults to `cuda` when available, otherwise `cpu`.
46
+ - `--seed`: torch seed for stochastic VAE sampling. Defaults to `0`.
47
+ - `--deterministic`: use the posterior mean instead of VAE sampling.
48
+
49
+ ## Python API
50
+
51
+ ```python
52
+ from pathlib import Path
53
+ import torch
54
+
55
+ from enhancement_VAE.enhancement_vae import enhance_file, load_model
56
+
57
+ root = Path("enhancement_VAE")
58
+ device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
59
+
60
+ model = load_model(
61
+ checkpoint_path=root / "XCodec_RVQ4_mono_causal_fp32.safetensors",
62
+ config_path=root / "config.json",
63
+ device=device,
64
+ )
65
+
66
+ enhance_file(
67
+ model=model,
68
+ input_path=Path("input_16k.wav"),
69
+ output_path=Path("input_16k_enhanced_48k.wav"),
70
+ deterministic=False,
71
+ )
72
+ ```
73
+
74
+ ## Input and Output
75
+
76
+ Input:
77
+
78
+ - XCodec1-decoded Mono 16 kHz WAV file
79
+
80
+ Output:
81
+
82
+ - Mono 48 kHz audio WAV file
83
+
84
+ If a directory is passed to `--input`, all `.wav` files directly inside that
85
+ directory are processed. Directory traversal is not recursive.