Upload 7 files

README.md

@@ -1,14 +1,285 @@
+# VieNeu-TTS
+
+[![GitHub](https://img.shields.io/badge/GitHub-Repository-blue)](https://github.com/pnnbao97/VieNeu-TTS)
+[![Hugging Face](https://img.shields.io/badge/Hugging%20Face-0.5B-yellow)](https://huggingface.co/pnnbao-ump/VieNeu-TTS)
+[![Hugging Face](https://img.shields.io/badge/Hugging%20Face-0.3B-orange)](https://huggingface.co/pnnbao-ump/VieNeu-TTS-0.3B)
+[![Hugging Face](https://img.shields.io/badge/Hugging%20Face-0.3B--GGUF-green)](https://huggingface.co/pnnbao-ump/VieNeu-TTS-0.3B-q8-gguf)
+[![Discord](https://img.shields.io/badge/Discord-Join%20Us-5865F2?logo=discord&logoColor=white)](https://discord.gg/mQWr4cp3)
+[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1V1DjG-KdmurCAhvXrxxTLsa9tteDxSVO?usp=sharing) 
+
+<img width="899" height="615" alt="Untitled" src="https://github.com/user-attachments/assets/7eb9b816-6ab7-4049-866f-f85e36cb9c6f" />
+
+**VieNeu-TTS** is an advanced on-device Vietnamese Text-to-Speech (TTS) model with **instant voice cloning**.
+
+> [!TIP]
+> **Voice Cloning:** All model variants (including GGUF) support instant voice cloning with just **3-5 seconds** of reference audio. 
+
+This project features two core architectures trained on the [VieNeu-TTS-1000h](https://huggingface.co/datasets/pnnbao-ump/VieNeu-TTS-1000h) dataset:
+- **VieNeu-TTS (0.5B):** An enhanced model fine-tuned from the NeuTTS Air architecture for maximum stability.
+- **VieNeu-TTS-0.3B:** A specialized model **trained from scratch**, delivering 2x faster inference and ultra-low latency.
+
+These represent a significant upgrade from the previous VieNeu-TTS-140h with the following improvements:
+
+- **Enhanced pronunciation**: More accurate and stable Vietnamese pronunciation
+- **Code-switching support**: Seamless transitions between Vietnamese and English
+- **Better voice cloning**: Higher fidelity and speaker consistency
+- **Real-time synthesis**: 24 kHz waveform generation on CPU or GPU
+- **Multiple model formats**: Support for PyTorch, GGUF Q4/Q8 (CPU optimized), and ONNX codec
+
+VieNeu-TTS delivers production-ready speech synthesis fully offline.
+
+**Author:** Phạm Nguyễn Ngọc Bảo
+
+---
+
+[<img width="600" height="595" alt="VieNeu-TTS" src="https://github.com/user-attachments/assets/6b32df9d-7e2e-474f-94c8-43d6fa586d15" />](https://github.com/user-attachments/assets/6b32df9d-7e2e-474f-94c8-43d6fa586d15)
+
+---
+
+## 🔬 Model Overview
+
+- **Backbone:** 
+  - **VieNeu-TTS (0.5B):** Qwen-0.5B fine-tuned from [NeuTTS Air](https://huggingface.co/neuphonic/neutts-air).
+  - **VieNeu-TTS-0.3B:** Custom 0.3B model **trained from scratch**, optimized for extreme speed (2x faster).
+- **Audio codec:** NeuCodec (torch implementation; ONNX & quantized variants supported)
+- **Context window:** 2,048 tokens shared by prompt text and speech tokens
+- **Output watermark:** Enabled by default
+- **Training data:** [VieNeu-TTS-1000h](https://huggingface.co/datasets/pnnbao-ump/VieNeu-TTS-1000h) — 443,641 curated Vietnamese samples (Used for both versions).
+
+### Model Variants
+
+| Model                   | Format  | Device  | Quality    | Speed                   |
+| ----------------------- | ------- | ------- | ---------- | ----------------------- |
+| VieNeu-TTS              | PyTorch | GPU/CPU | ⭐⭐⭐⭐⭐ | Very Fast with lmdeploy |
+| VieNeu-TTS-0.3B         | PyTorch | GPU/CPU | ⭐⭐⭐⭐   | **Ultra Fast (2x)**     |
+| VieNeu-TTS-q8-gguf      | GGUF Q8 | CPU/GPU | ⭐⭐⭐⭐   | Fast                    |
+| VieNeu-TTS-q4-gguf      | GGUF Q4 | CPU/GPU | ⭐⭐⭐     | Very Fast               |
+| VieNeu-TTS-0.3B-q8-gguf | GGUF Q8 | CPU/GPU | ⭐⭐⭐⭐   | **Ultra Fast (1.5x)**   |
+| VieNeu-TTS-0.3B-q4-gguf | GGUF Q4 | CPU/GPU | ⭐⭐⭐     | **Extreme Speed (2x)**  |
+
+**Recommendations:**
+
+- **GPU users**: Use `VieNeu-TTS` (PyTorch) for best quality
+- **CPU users**: Use `VieNeu-TTS-0.3B-q4-gguf` for fastest inference or `VieNeu-TTS-0.3B-q8-gguf` for best CPU quality.
+- **Streaming**: Only GGUF models support streaming inference (Requires `llama-cpp-python >= 0.3.16`)
+
+---
+
+## ✅ Todo & Status
+
+- [x] Publish safetensor artifacts
+- [x] Release GGUF Q4 / Q8 models
+- [x] Release datasets (1000h and 140h)
+- [x] Enable streaming on GPU
+- [x] Provide Dockerized setup
+- [ ] Release fine-tuning code
+
+---
+
+## 🏁 Getting Started
+
+### 1. Clone the repository
+```bash
+git clone https://github.com/pnnbao97/VieNeu-TTS.git
+cd VieNeu-TTS
+```
+
+### 2. Install eSpeak NG (Required)
+Phonemizer requires eSpeak NG to function.
+
+- **Windows:** Download installer from [eSpeak NG Releases](https://github.com/espeak-ng/espeak-ng/releases) (Recommended: `.msi`).
+- **macOS:** `brew install espeak`
+- **Ubuntu/Debian:** `sudo apt install espeak-ng`
+- **Arch Linux:** `paru -S aur/espeak-ng`
+
+---
+
+### 3. Environment Setup (Choose ONE method)
+
+#### Method 1: Standard with `uv` (Recommended)
+This is the fastest and most reliable way to manage dependencies.
+
+**A. Install `uv`** (If you haven't already):
+- **Windows:** `powershell -c "irm https://astral.sh/uv/install.ps1 | iex"`
+- **Linux/macOS:** `curl -LsSf https://astral.sh/uv/install.sh | sh`
+
+**B. Choose your hardware:**
+
+**Option A: For GPU Users (NVIDIA 30xx/40xx/50xx)**
+
+> [!IMPORTANT]
+> **Update your NVIDIA Drivers & Install CUDA Toolkit!**
+> This project uses **CUDA 12.8**. Please ensure your NVIDIA driver is up-to-date (support CUDA 12.8 or newer) to avoid compatibility issues, especially on RTX 30 series.
+>
+> To use `lmdeploy`, you **MUST** install the **NVIDIA GPU Computing Toolkit**: [https://developer.nvidia.com/cuda-downloads](https://developer.nvidia.com/cuda-downloads).
+
+```bash
+uv sync
+```
+
+**Option B: For CPU-only Users**
+
+1. Switch to CPU configuration:
+   ```bash
+   # Windows:
+   ren pyproject.toml pyproject.toml.bak
+   copy pyproject.toml.cpu pyproject.toml
+   
+   # Linux/macOS:
+   mv pyproject.toml pyproject.toml.bak
+   cp pyproject.toml.cpu pyproject.toml
+   ```
+2. Install dependencies:
+   ```bash
+   uv sync
+   ```
+
+**C. Run the Application:**
+```bash
+uv run gradio_app.py
+```
+
+Then access the Web UI at `http://127.0.0.1:7860`.
+
+---
+
+#### Method 2: Automatic with Makefile (Alternative)
+Best if you have `make` installed (standard on Linux/macOS, or via Git Bash on Windows). It handles configuration swaps automatically.
+
+- **Setup GPU:** `make setup-gpu`
+- **Setup CPU:** `make setup-cpu`
+- **Run Demo:** `make demo`
+
+
+Then access the Web UI at `http://127.0.0.1:7860`.
+
+---
+
+---
+
+## 🐋 Docker Deployment
+
+For a quick start or production deployment without manually installing dependencies, use Docker.
+
+### Quick Start
+
+Copy .env.example to .env
+
+```
+cp .env.example .env
+```
+
+Build and start container
+
+```bash
+# Run with CPU
+docker compose --profile cpu up
+
+# Run with GPU (requires NVIDIA Container Toolkit)
+docker compose --profile gpu up
+```
+
+Access the Web UI at `http://localhost:7860`.
+
+For detailed deployment instructions, including production setup, see [docs/Deploy.md](docs/Deploy.md).
+
+---
+
+## 📦 Project Structure
+
+```
+VieNeu-TTS/
+├── examples/
+│   ├── infer_long_text.py     # CLI for long-form synthesis (chunked)
+│   └── sample_long_text.txt   # Example paragraph for testing
+├── gradio_app.py              # Local Gradio web demo with LMDeploy support
+├── main.py                    # Basic batch inference script
+├── config.yaml                # Configuration for models, codecs, and voices
+├── output_audio/              # Generated audio (created when running scripts)
+├── sample/                    # Reference voices (audio + transcript + codes)
+│   ├── Bình (nam miền Bắc).wav/txt/pt
+│   ├── Đoan (nữ miền Nam).wav/txt/pt
+│   ├── Dung (nữ miền Nam).wav/txt/pt
+│   ├── Hương (nữ miền Bắc).wav/txt/pt
+│   ├── Ly (nữ miền Bắc).wav/txt/pt
+│   ├── Ngọc (nữ miền Bắc).wav/txt/pt
+│   ├── Nguyên (nam miền Nam).wav/txt/pt
+│   ├── Sơn (nam miền Nam).wav/txt/pt
+│   ├── Tuyên (nam miền Bắc).wav/txt/pt
+│   └── Vĩnh (nam miền Nam).wav/txt/pt
+├── utils/
+│   ├── __init__.py
+│   ├── core_utils.py          # Text chunking utilities
+│   ├── normalize_text.py      # Vietnamese text normalization pipeline
+│   ├── phonemize_text.py      # Text to phoneme conversion
+│   └── phoneme_dict.json      # Phoneme dictionary
+├── vieneu_tts/
+│   ├── __init__.py            # Exports VieNeuTTS and FastVieNeuTTS
+│   └── vieneu_tts.py          # Core VieNeuTTS implementation (VieNeuTTS & FastVieNeuTTS)
+├── README.md
+├── requirements.txt           # Basic dependencies (legacy)
+├── pyproject.toml             # Project configuration with full dependencies (UV)
+└── uv.lock                    # UV lock file for dependency management
+```
+
 ---
-title: Tts
-emoji: 😻
-colorFrom: pink
-colorTo: gray
-sdk: gradio
-sdk_version: 6.3.0
-app_file: app.py
-pinned: false
-license: apache-2.0
-short_description: vieneu_tts/
+
+## 📚 References
+
+- [GitHub Repository](https://github.com/pnnbao97/VieNeu-TTS)
+- [Hugging Face Model (0.5B)](https://huggingface.co/pnnbao-ump/VieNeu-TTS)
+- [Hugging Face Model (0.3B)](https://huggingface.co/pnnbao-ump/VieNeu-TTS-0.3B)
+- [VieNeuTTS Fine-tuning Guide](https://github.com/pnnbao-ump/VieNeuTTS/blob/main/finetune.ipynb)
+- [VieNeuCodec dataset](https://huggingface.co/datasets/pnnbao-ump/VieNeuCodec-dataset)
+
+---
+
+## 📄 License
+
+- **VieNeu-TTS (0.5B):** Original terms (Apache 2.0).
+- **VieNeu-TTS-0.3B:** Released under **CC BY-NC 4.0** (Non-Commercial). 
+  - This version is currently **experimental**.
+  - **Commercial use is prohibited** without authorization. Please contact the author for commercial licensing.
+
+---
+
+## 📑 Citation
+
+```bibtex
+@misc{vieneutts2026,
+  title        = {VieNeu-TTS: Vietnamese Text-to-Speech with Instant Voice Cloning},
+  author       = {Pham Nguyen Ngoc Bao},
+  year         = {2026},
+  publisher    = {Hugging Face},
+  howpublished = {\url{https://huggingface.co/pnnbao-ump/VieNeu-TTS}}
+}
+```
+
+## 🤝 Contributing
+
+Contributions are welcome!
+
+1. Fork the repository
+2. Create a feature branch: `git checkout -b feature/amazing-feature`
+3. Commit your changes: `git commit -m "Add amazing feature"`
+4. Push the branch: `git push origin feature/amazing-feature`
+5. Open a pull request
+
+---
+
+## 📞 Support
+
+- GitHub Issues: [github.com/pnnbao97/VieNeu-TTS/issues](https://github.com/pnnbao97/VieNeu-TTS/issues)
+- Hugging Face: [huggingface.co/pnnbao-ump](https://huggingface.co/pnnbao-ump)
+- Discord: [Join with us](https://discord.gg/mQWr4cp3)
+- Facebook: [Phạm Nguyễn Ngọc Bảo](https://www.facebook.com/bao.phamnguyenngoc.5)
+
+---
+
+## 🙏 Acknowledgements
+
+This project builds upon [NeuTTS Air](https://huggingface.co/neuphonic/neutts-air) for the original 0.5B model. The 0.3B version is a custom architecture trained from scratch using the [VieNeu-TTS-1000h](https://huggingface.co/datasets/pnnbao-ump/VieNeu-TTS-1000h) dataset.
+
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+**Made with ❤️ for the Vietnamese TTS community**

config.yaml

@@ -0,0 +1,81 @@
+text_settings:
+  max_chars_per_chunk: 256
+  max_total_chars_streaming: 3000
+
+backbone_configs:
+  "VieNeu-TTS (GPU)":
+    repo: pnnbao-ump/VieNeu-TTS
+    supports_streaming: false
+    description: Chất lượng cao nhất, yêu cầu GPU
+  "VieNeu-TTS-0.3B (GPU)":
+    repo: pnnbao-ump/VieNeu-TTS-0.3B
+    supports_streaming: false
+    description: Phiên bản nhẹ cho GPU, tốc độ nhanh x2 so với phiên bản gốc 
+  "VieNeu-TTS-q8-gguf":
+    repo: pnnbao-ump/VieNeu-TTS-q8-gguf
+    supports_streaming: true
+    description: Phiên bản GGUF có chất lượng cao nhất
+  "VieNeu-TTS-q4-gguf":
+    repo: pnnbao-ump/VieNeu-TTS-q4-gguf
+    supports_streaming: true
+    description: Cân bằng giữa chất lượng và tốc độ
+  "VieNeu-TTS-0.3B-q4-gguf":
+    repo: pnnbao-ump/VieNeu-TTS-0.3B-q4-gguf
+    supports_streaming: true
+    description: Phiên bản cực nhẹ, chạy mượt trên CPU
+  "VieNeu-TTS-0.3B-q8-gguf":
+    repo: pnnbao-ump/VieNeu-TTS-0.3B-q8-gguf
+    supports_streaming: true
+    description: Phiên bản 0.3B nén Q8, chất lượng cao và tốc độ nhanh trên CPU
+
+codec_configs:
+  "NeuCodec (Standard)":
+    repo: neuphonic/neucodec
+    description: Codec chuẩn, tốc độ trung bình
+    use_preencoded: false
+  "NeuCodec (Distill)":
+    repo: neuphonic/distill-neucodec
+    description: Codec tối ưu, tốc độ cao
+    use_preencoded: false
+  "NeuCodec ONNX (Fast CPU)":
+    repo: neuphonic/neucodec-onnx-decoder-int8
+    description: Tối ưu cho CPU, cần pre-encoded codes
+    use_preencoded: true
+
+voice_samples:
+  "Tuyên (nam miền Bắc)":
+    audio: ./sample/Tuyên (nam miền Bắc).wav
+    text: ./sample/Tuyên (nam miền Bắc).txt
+    codes: ./sample/Tuyên (nam miền Bắc).pt
+  "Vĩnh (nam miền Nam)":
+    audio: ./sample/Vĩnh (nam miền Nam).wav
+    text: ./sample/Vĩnh (nam miền Nam).txt
+    codes: ./sample/Vĩnh (nam miền Nam).pt
+  "Bình (nam miền Bắc)":
+    audio: ./sample/Bình (nam miền Bắc).wav
+    text: ./sample/Bình (nam miền Bắc).txt
+    codes: ./sample/Bình (nam miền Bắc).pt
+  "Nguyên (nam miền Nam)":
+    audio: ./sample/Nguyên (nam miền Nam).wav
+    text: ./sample/Nguyên (nam miền Nam).txt
+    codes: ./sample/Nguyên (nam miền Nam).pt
+  "Sơn (nam miền Nam)":
+    audio: ./sample/Sơn (nam miền Nam).wav
+    text: ./sample/Sơn (nam miền Nam).txt
+    codes: ./sample/Sơn (nam miền Nam).pt
+  "Đoan (nữ miền Nam)":
+    audio: ./sample/Đoan (nữ miền Nam).wav
+    text: ./sample/Đoan (nữ miền Nam).txt
+    codes: ./sample/Đoan (nữ miền Nam).pt
+  "Ngọc (nữ miền Bắc)":
+    audio: ./sample/Ngọc (nữ miền Bắc).wav
+    text: ./sample/Ngọc (nữ miền Bắc).txt
+    codes: ./sample/Ngọc (nữ miền Bắc).pt
+  "Ly (nữ miền Bắc)":
+    audio: ./sample/Ly (nữ miền Bắc).wav
+    text: ./sample/Ly (nữ miền Bắc).txt
+    codes: ./sample/Ly (nữ miền Bắc).pt
+  "Dung (nữ miền Nam)":
+    audio: ./sample/Dung (nữ miền Nam).wav
+    text: ./sample/Dung (nữ miền Nam).txt
+    codes: ./sample/Dung (nữ miền Nam).pt

gradio_app.py

@@ -0,0 +1,965 @@
+import gradio as gr
+print("⏳ Đang khởi động... Vui lòng chờ...")
+import soundfile as sf
+import tempfile
+import torch
+from vieneu_tts import VieNeuTTS, FastVieNeuTTS
+import os
+import sys
+import time
+import numpy as np
+from typing import Generator, Optional, Tuple
+import queue
+import threading
+import yaml
+from utils.core_utils import split_text_into_chunks, env_bool
+from functools import lru_cache
+import gc
+
+print("⏳ Đang khởi động VieNeu-TTS...")
+
+# --- CONSTANTS & CONFIG ---
+CONFIG_PATH = os.path.join(os.path.dirname(__file__), "config.yaml")
+try:
+    with open(CONFIG_PATH, "r", encoding="utf-8") as f:
+        _config = yaml.safe_load(f) or {}
+except Exception as e:
+    raise RuntimeError(f"Không thể đọc config.yaml: {e}")
+
+BACKBONE_CONFIGS = _config.get("backbone_configs", {})
+CODEC_CONFIGS = _config.get("codec_configs", {})
+VOICE_SAMPLES = _config.get("voice_samples", {})
+
+_text_settings = _config.get("text_settings", {})
+MAX_CHARS_PER_CHUNK = _text_settings.get("max_chars_per_chunk", 256)
+MAX_TOTAL_CHARS_STREAMING = _text_settings.get("max_total_chars_streaming", 3000)
+
+if not BACKBONE_CONFIGS or not CODEC_CONFIGS:
+    raise ValueError("config.yaml thiếu backbone_configs hoặc codec_configs")
+if not VOICE_SAMPLES:
+    raise ValueError("config.yaml thiếu voice_samples")
+
+# --- 1. MODEL CONFIGURATION ---
+# Global model instance
+tts = None
+current_backbone = None
+current_codec = None
+model_loaded = False
+using_lmdeploy = False
+
+# Cache for reference texts
+_ref_text_cache = {}
+
+def get_available_devices() -> list[str]:
+    """Get list of available devices for current platform."""
+    devices = ["Auto", "CPU"]
+
+    if sys.platform == "darwin":
+        # macOS - check MPS
+        if torch.backends.mps.is_available():
+            devices.append("MPS")
+    else:
+        # Windows/Linux - check CUDA
+        if torch.cuda.is_available():
+            devices.append("CUDA")
+
+    return devices
+
+def get_model_status_message() -> str:
+    """Reconstruct status message from global state"""
+    global model_loaded, tts, using_lmdeploy, current_backbone, current_codec
+    if not model_loaded or tts is None:
+        return "⏳ Chưa tải model."
+    
+    backbone_config = BACKBONE_CONFIGS.get(current_backbone, {})
+    codec_config = CODEC_CONFIGS.get(current_codec, {})
+    
+    backend_name = "🚀 LMDeploy (Optimized)" if using_lmdeploy else "📦 Standard"
+    
+    # We don't track the exact device strings perfectly in global state, so we estimate
+    device_info = "GPU" if using_lmdeploy else "Auto"
+    codec_device = "CPU" if "ONNX" in (current_codec or "") else ("GPU/MPS" if torch.cuda.is_available() or torch.backends.mps.is_available() else "CPU")
+    
+    preencoded_note = "\n⚠️ Codec ONNX không hỗ trợ chức năng clone giọng nói." if codec_config.get('use_preencoded') else ""
+    
+    opt_info = ""
+    if using_lmdeploy and hasattr(tts, 'get_optimization_stats'):
+        stats = tts.get_optimization_stats()
+        opt_info = (
+            f"\n\n🔧 Tối ưu hóa:"
+            f"\n  • Triton: {'✅' if stats['triton_enabled'] else '❌'}"
+            f"\n  • Max Batch Size (Default): {stats.get('max_batch_size', 'N/A')}"
+            f"\n  • Reference Cache: {stats['cached_references']} voices"
+            f"\n  • Prefix Caching: ✅"
+        )
+
+    return (
+        f"✅ Model đã tải thành công!\n\n"
+        f"🔧 Backend: {backend_name}\n"
+        f"🦜 Backbone: {current_backbone}\n"
+        f"🎵 Codec: {current_codec}{preencoded_note}{opt_info}"
+    )
+
+def restore_ui_state():
+    """Update UI components based on persistence"""
+    global model_loaded
+    msg = get_model_status_message()
+    return (
+        msg, 
+        gr.update(interactive=model_loaded), # btn_generate
+        gr.update(interactive=False)         # btn_stop
+    )
+
+def should_use_lmdeploy(backbone_choice: str, device_choice: str) -> bool:
+    """Determine if we should use LMDeploy backend."""
+    # LMDeploy not supported on macOS
+    if sys.platform == "darwin":
+        return False
+
+    if "gguf" in backbone_choice.lower():
+        return False
+
+    if device_choice == "Auto":
+        has_gpu = torch.cuda.is_available()
+    elif device_choice == "CUDA":
+        has_gpu = torch.cuda.is_available()
+    else:
+        has_gpu = False
+
+    return has_gpu
+
+@lru_cache(maxsize=32)
+def get_ref_text_cached(text_path: str) -> str:
+    """Cache reference text loading"""
+    with open(text_path, "r", encoding="utf-8") as f:
+        return f.read()
+
+def cleanup_gpu_memory():
+    """Aggressively cleanup GPU memory"""
+    if torch.cuda.is_available():
+        torch.cuda.empty_cache()
+        torch.cuda.synchronize()
+    elif torch.backends.mps.is_available():
+        torch.mps.empty_cache()
+    gc.collect()
+
+def load_model(backbone_choice: str, codec_choice: str, device_choice: str, 
+               force_lmdeploy: bool):
+    """Load model with optimizations and max batch size control"""
+    global tts, current_backbone, current_codec, model_loaded, using_lmdeploy
+    lmdeploy_error_reason = None
+    
+    yield (
+        "⏳ Đang tải model với tối ưu hóa... Lưu ý: Quá trình này sẽ tốn thời gian. Vui lòng kiên nhẫn.",
+        gr.update(interactive=False),
+        gr.update(interactive=False),
+        gr.update(interactive=False)
+    )
+    
+    try:
+        # Cleanup before loading new model
+        if model_loaded and tts is not None:
+            del tts
+            cleanup_gpu_memory()
+        
+        backbone_config = BACKBONE_CONFIGS[backbone_choice]
+        codec_config = CODEC_CONFIGS[codec_choice]
+        
+        use_lmdeploy = force_lmdeploy and should_use_lmdeploy(backbone_choice, device_choice)
+        
+        if use_lmdeploy:
+            lmdeploy_error_reason = None
+            print(f"🚀 Using LMDeploy backend with optimizations")
+            
+            backbone_device = "cuda"
+            
+            if "ONNX" in codec_choice:
+                codec_device = "cpu"
+            else:
+                codec_device = "cuda" if torch.cuda.is_available() else "cpu"
+            
+            print(f"📦 Loading optimized model...")
+            print(f"   Backbone: {backbone_config['repo']} on {backbone_device}")
+            print(f"   Codec: {codec_config['repo']} on {codec_device}")
+            print(f"   Triton: Enabled")
+            
+            try:
+                tts = FastVieNeuTTS(
+                    backbone_repo=backbone_config["repo"],
+                    backbone_device=backbone_device,
+                    codec_repo=codec_config["repo"],
+                    codec_device=codec_device,
+                    memory_util=0.3,
+                    tp=1,
+                    enable_prefix_caching=True,
+                    enable_triton=True,
+                )
+                using_lmdeploy = True
+                
+                # Pre-cache voice references
+                print("📝 Pre-caching voice references...")
+                for voice_name, voice_info in VOICE_SAMPLES.items():
+                    audio_path = voice_info["audio"]
+                    text_path = voice_info["text"]
+                    if os.path.exists(audio_path) and os.path.exists(text_path):
+                        ref_text = get_ref_text_cached(text_path)
+                        tts.get_cached_reference(voice_name, audio_path, ref_text)
+                print(f"   ✅ Cached {len(VOICE_SAMPLES)} voices")
+                
+            except Exception as e:
+                import traceback
+                traceback.print_exc()
+                
+                error_str = str(e)
+                if "$env:CUDA_PATH" in error_str:
+                    lmdeploy_error_reason = "Không tìm thấy biến môi trường CUDA_PATH. Vui lòng cài đặt NVIDIA GPU Computing Toolkit."
+                else:
+                    lmdeploy_error_reason = f"{error_str}"
+                
+                yield (
+                    f"⚠️ LMDeploy Init Error: {lmdeploy_error_reason}. Đang loading model với backend mặc định - tốc độ chậm hơn so với lmdeploy...",
+                    gr.update(interactive=False),
+                    gr.update(interactive=False)
+                )
+                time.sleep(1)
+                use_lmdeploy = False
+                using_lmdeploy = False
+        
+        if not use_lmdeploy:
+            print(f"📦 Using original backend")
+
+            if device_choice == "Auto":
+                if "gguf" in backbone_choice.lower():
+                    # GGUF: uses Metal on Mac, CUDA on Windows/Linux
+                    if sys.platform == "darwin":
+                        backbone_device = "gpu"  # llama-cpp-python uses Metal
+                    else:
+                        backbone_device = "gpu" if torch.cuda.is_available() else "cpu"
+                else:
+                    # PyTorch model
+                    if sys.platform == "darwin":
+                        backbone_device = "mps" if torch.backends.mps.is_available() else "cpu"
+                    else:
+                        backbone_device = "cuda" if torch.cuda.is_available() else "cpu"
+
+                # Codec device
+                if "ONNX" in codec_choice:
+                    codec_device = "cpu"
+                elif sys.platform == "darwin":
+                    codec_device = "mps" if torch.backends.mps.is_available() else "cpu"
+                else:
+                    codec_device = "cuda" if torch.cuda.is_available() else "cpu"
+
+            elif device_choice == "MPS":
+                backbone_device = "mps"
+                codec_device = "mps" if "ONNX" not in codec_choice else "cpu"
+
+            else:
+                backbone_device = device_choice.lower()
+                codec_device = device_choice.lower()
+
+                if "ONNX" in codec_choice:
+                    codec_device = "cpu"
+
+            if "gguf" in backbone_choice.lower() and backbone_device == "cuda":
+                backbone_device = "gpu"
+            
+            print(f"📦 Loading model...")
+            print(f"   Backbone: {backbone_config['repo']} on {backbone_device}")
+            print(f"   Codec: {codec_config['repo']} on {codec_device}")
+            
+            tts = VieNeuTTS(
+                backbone_repo=backbone_config["repo"],
+                backbone_device=backbone_device,
+                codec_repo=codec_config["repo"],
+                codec_device=codec_device
+            )
+            using_lmdeploy = False
+        
+        current_backbone = backbone_choice
+        current_codec = codec_choice
+        model_loaded = True
+        
+        # Success message with optimization info
+        backend_name = "🚀 LMDeploy (Optimized)" if using_lmdeploy else "📦 Standard"
+        device_info = "cuda" if use_lmdeploy else (backbone_device if not use_lmdeploy else "N/A")
+        
+        streaming_support = "✅ Có" if backbone_config['supports_streaming'] else "❌ Không"
+        preencoded_note = "\n⚠️ Codec này cần sử dụng pre-encoded codes (.pt files)" if codec_config['use_preencoded'] else ""
+        
+        opt_info = ""
+        if using_lmdeploy and hasattr(tts, 'get_optimization_stats'):
+            stats = tts.get_optimization_stats()
+            opt_info = (
+                f"\n\n🔧 Tối ưu hóa:"
+                f"\n  • Triton: {'✅' if stats['triton_enabled'] else '❌'}"
+                f"\n  • Max Batch Size (Default): {stats.get('max_batch_size', 'N/A')}"
+                f"\n  • Reference Cache: {stats['cached_references']} voices"
+                f"\n  • Prefix Caching: ✅"
+            )
+        
+        warning_msg = ""
+        if lmdeploy_error_reason:
+             warning_msg = (
+                 f"\n\n⚠️ **Cảnh báo:** Không thể kích hoạt LMDeploy (Optimized Backend) do lỗi sau:\n"
+                 f"👉 {lmdeploy_error_reason}\n"
+                 f"💡 Hệ thống đã tự động chuyển về chế độ Standard (chậm hơn)."
+             )
+
+        success_msg = get_model_status_message()
+        if warning_msg:
+            success_msg += warning_msg
+            
+        yield (
+            success_msg,
+            gr.update(interactive=True), # btn_generate
+            gr.update(interactive=True), # btn_load
+            gr.update(interactive=False) # btn_stop
+        )
+        
+    except Exception as e:
+        import traceback
+        traceback.print_exc()
+        model_loaded = False
+        using_lmdeploy = False
+
+        if "$env:CUDA_PATH" in str(e):
+            yield (
+                "❌ Lỗi khi tải model: Không tìm thấy biến môi trường CUDA_PATH. Vui lòng cài đặt NVIDIA GPU Computing Toolkit (https://developer.nvidia.com/cuda/toolkit)",
+                gr.update(interactive=False),
+                gr.update(interactive=True),
+                gr.update(interactive=False)
+            )
+        else: 
+            yield (
+                f"❌ Lỗi khi tải model: {str(e)}",
+                gr.update(interactive=False),
+                gr.update(interactive=True),
+                gr.update(interactive=False)
+            )
+
+
+# --- 2. DATA & HELPERS ---
+GGUF_ALLOWED_VOICES = [
+    "Vĩnh (nam miền Nam)",
+    "Bình (nam miền Bắc)",
+    "Ngọc (nữ miền Bắc)",
+    "Dung (nữ miền Nam)",
+]
+
+def get_voice_options(backbone_choice: str):
+    """Filter voice options: GGUF only shows the 4 allowed voices."""
+    if "gguf" in backbone_choice.lower():
+        return [v for v in GGUF_ALLOWED_VOICES if v in VOICE_SAMPLES]
+    return list(VOICE_SAMPLES.keys())
+
+def update_voice_dropdown(backbone_choice: str, current_voice: str):
+    options = get_voice_options(backbone_choice)
+    new_value = current_voice if current_voice in options else (options[0] if options else None)
+    return gr.update(choices=options, value=new_value)
+
+# --- 3. CORE LOGIC FUNCTIONS ---
+def load_reference_info(voice_choice: str) -> Tuple[Optional[str], str]:
+    """Load reference audio and text with caching"""
+    if voice_choice in VOICE_SAMPLES:
+        audio_path = VOICE_SAMPLES[voice_choice]["audio"]
+        text_path = VOICE_SAMPLES[voice_choice]["text"]
+        try:
+            if os.path.exists(text_path):
+                ref_text = get_ref_text_cached(text_path)
+                return audio_path, ref_text
+            else:
+                return audio_path, "⚠️ Không tìm thấy file text mẫu."
+        except Exception as e:
+            return None, f"❌ Lỗi: {str(e)}"
+    return None, ""
+
+def synthesize_speech(text: str, voice_choice: str, custom_audio, custom_text: str, 
+                     mode_tab: str, generation_mode: str, use_batch: bool, max_batch_size_run: int):
+    """Synthesis with optimization support and max batch size control"""
+    global tts, current_backbone, current_codec, model_loaded, using_lmdeploy
+    
+    if not model_loaded or tts is None:
+        yield None, "⚠️ Vui lòng tải model trước!"
+        return
+    
+    if not text or text.strip() == "":
+        yield None, "⚠️ Vui lòng nhập văn bản!"
+        return
+    
+    raw_text = text.strip()
+    
+    codec_config = CODEC_CONFIGS[current_codec]
+    use_preencoded = codec_config['use_preencoded']
+    
+    # Setup Reference
+    if mode_tab == "custom_mode":
+        if custom_audio is None or not custom_text:
+            yield None, "⚠️ Thiếu Audio hoặc Text mẫu custom."
+            return
+        ref_audio_path = custom_audio
+        ref_text_raw = custom_text
+        ref_codes_path = None
+    else:
+        if voice_choice not in VOICE_SAMPLES:
+            yield None, "⚠️ Vui lòng chọn giọng mẫu."
+            return
+        ref_audio_path = VOICE_SAMPLES[voice_choice]["audio"]
+        text_path = VOICE_SAMPLES[voice_choice]["text"]
+        ref_codes_path = VOICE_SAMPLES[voice_choice]["codes"]
+        
+        if not os.path.exists(ref_audio_path):
+            yield None, "❌ Không tìm thấy file audio mẫu."
+            return
+        
+        ref_text_raw = get_ref_text_cached(text_path)
+    
+    yield None, "📄 Đang xử lý Reference..."
+    
+    # Encode or get cached reference
+    try:
+        if use_preencoded and ref_codes_path and os.path.exists(ref_codes_path):
+            ref_codes = torch.load(ref_codes_path, map_location="cpu", weights_only=True)
+        else:
+            # Use cached reference if available (LMDeploy only)
+            if using_lmdeploy and hasattr(tts, 'get_cached_reference') and mode_tab == "preset_mode":
+                ref_codes = tts.get_cached_reference(voice_choice, ref_audio_path, ref_text_raw)
+            else:
+                ref_codes = tts.encode_reference(ref_audio_path)
+        
+        if isinstance(ref_codes, torch.Tensor):
+            ref_codes = ref_codes.cpu().numpy()
+    except Exception as e:
+        yield None, f"❌ Lỗi xử lý reference: {e}"
+        return
+    
+    text_chunks = split_text_into_chunks(raw_text, max_chars=MAX_CHARS_PER_CHUNK)
+    total_chunks = len(text_chunks)
+    
+    # === STANDARD MODE ===
+    if generation_mode == "Standard (Một lần)":
+        backend_name = "LMDeploy" if using_lmdeploy else "Standard"
+        batch_info = " (Batch Mode)" if use_batch and using_lmdeploy and total_chunks > 1 else ""
+        
+        # Show batch size info
+        batch_size_info = ""
+        if use_batch and using_lmdeploy and hasattr(tts, 'max_batch_size'):
+            batch_size_info = f" [Max batch: {tts.max_batch_size}]"
+        
+        yield None, f"🚀 Bắt đầu tổng hợp {backend_name}{batch_info}{batch_size_info} ({total_chunks} đoạn)..."
+        
+        all_audio_segments = []
+        sr = 24000
+        silence_pad = np.zeros(int(sr * 0.15), dtype=np.float32)
+        
+        start_time = time.time()
+        
+        try:
+            # Use batch processing if enabled and using LMDeploy
+            if use_batch and using_lmdeploy and hasattr(tts, 'infer_batch') and total_chunks > 1:
+                # Show how many mini-batches will be processed
+                num_batches = (total_chunks + max_batch_size_run - 1) // max_batch_size_run
+                
+                yield None, f"⚡ Xử lý {num_batches} mini-batch(es) (max {max_batch_size_run} đoạn/batch)..."
+                
+                chunk_wavs = tts.infer_batch(text_chunks, ref_codes, ref_text_raw, max_batch_size=max_batch_size_run)
+                
+                for i, chunk_wav in enumerate(chunk_wavs):
+                    if chunk_wav is not None and len(chunk_wav) > 0:
+                        all_audio_segments.append(chunk_wav)
+                        if i < total_chunks - 1:
+                            all_audio_segments.append(silence_pad)
+            else:
+                # Sequential processing
+                for i, chunk in enumerate(text_chunks):
+                    yield None, f"⏳ Đang xử lý đoạn {i+1}/{total_chunks}..."
+                    
+                    chunk_wav = tts.infer(chunk, ref_codes, ref_text_raw)
+                    
+                    if chunk_wav is not None and len(chunk_wav) > 0:
+                        all_audio_segments.append(chunk_wav)
+                        if i < total_chunks - 1:
+                            all_audio_segments.append(silence_pad)
+            
+            if not all_audio_segments:
+                yield None, "❌ Không sinh được audio nào."
+                return
+            
+            yield None, "💾 Đang ghép file và lưu..."
+            
+            final_wav = np.concatenate(all_audio_segments)
+            with tempfile.NamedTemporaryFile(delete=False, suffix=".wav") as tmp:
+                sf.write(tmp.name, final_wav, sr)
+                output_path = tmp.name
+            
+            process_time = time.time() - start_time
+            backend_info = f" (Backend: {'LMDeploy 🚀' if using_lmdeploy else 'Standard 📦'})"
+            speed_info = f", Tốc độ: {len(final_wav)/sr/process_time:.2f}x realtime" if process_time > 0 else ""
+            
+            yield output_path, f"✅ Hoàn tất! (Thời gian: {process_time:.2f}s{speed_info}){backend_info}"
+            
+            # Cleanup memory
+            if using_lmdeploy and hasattr(tts, 'cleanup_memory'):
+                tts.cleanup_memory()
+            cleanup_gpu_memory()
+            
+        except torch.cuda.OutOfMemoryError as e:
+            cleanup_gpu_memory()
+            yield None, (
+                f"❌ GPU hết VRAM! Hãy thử:\n"
+                f"• Giảm Max Batch Size (hiện tại: {tts.max_batch_size if hasattr(tts, 'max_batch_size') else 'N/A'})\n"
+                f"• Giảm độ dài văn bản\n\n"
+                f"Chi tiết: {str(e)}"
+            )
+            return
+            
+        except Exception as e:
+            import traceback
+            traceback.print_exc()
+            cleanup_gpu_memory()
+            yield None, f"❌ Lỗi Standard Mode: {str(e)}"
+            return
+    
+    # === STREAMING MODE ===
+    else:
+        sr = 24000
+        crossfade_samples = int(sr * 0.03)
+        audio_queue = queue.Queue(maxsize=100)
+        PRE_BUFFER_SIZE = 3
+        
+        end_event = threading.Event()
+        error_event = threading.Event()
+        error_msg = ""
+        
+        def producer_thread():
+            nonlocal error_msg
+            try:
+                previous_tail = None
+                
+                for i, chunk_text in enumerate(text_chunks):
+                    stream_gen = tts.infer_stream(chunk_text, ref_codes, ref_text_raw)
+                    
+                    for part_idx, audio_part in enumerate(stream_gen):
+                        if audio_part is None or len(audio_part) == 0:
+                            continue
+                        
+                        if previous_tail is not None and len(previous_tail) > 0:
+                            overlap = min(len(previous_tail), len(audio_part), crossfade_samples)
+                            if overlap > 0:
+                                fade_out = np.linspace(1.0, 0.0, overlap, dtype=np.float32)
+                                fade_in = np.linspace(0.0, 1.0, overlap, dtype=np.float32)
+                                
+                                blended = (audio_part[:overlap] * fade_in + 
+                                         previous_tail[-overlap:] * fade_out)
+                                
+                                processed = np.concatenate([
+                                    previous_tail[:-overlap] if len(previous_tail) > overlap else np.array([]),
+                                    blended,
+                                    audio_part[overlap:]
+                                ])
+                            else:
+                                processed = np.concatenate([previous_tail, audio_part])
+                            
+                            tail_size = min(crossfade_samples, len(processed))
+                            previous_tail = processed[-tail_size:].copy()
+                            output_chunk = processed[:-tail_size] if len(processed) > tail_size else processed
+                        else:
+                            tail_size = min(crossfade_samples, len(audio_part))
+                            previous_tail = audio_part[-tail_size:].copy()
+                            output_chunk = audio_part[:-tail_size] if len(audio_part) > tail_size else audio_part
+                        
+                        if len(output_chunk) > 0:
+                            audio_queue.put((sr, output_chunk))
+                
+                if previous_tail is not None and len(previous_tail) > 0:
+                    audio_queue.put((sr, previous_tail))
+                    
+            except Exception as e:
+                import traceback
+                traceback.print_exc()
+                error_msg = str(e)
+                error_event.set()
+            finally:
+                end_event.set()
+                audio_queue.put(None)
+        
+        threading.Thread(target=producer_thread, daemon=True).start()
+        
+        yield (sr, np.zeros(int(sr * 0.05))), "📄 Đang buffering..."
+        
+        pre_buffer = []
+        while len(pre_buffer) < PRE_BUFFER_SIZE:
+            try:
+                item = audio_queue.get(timeout=5.0)
+                if item is None:
+                    break
+                pre_buffer.append(item)
+            except queue.Empty:
+                if error_event.is_set():
+                    yield None, f"❌ Lỗi: {error_msg}"
+                    return
+                break
+        
+        full_audio_buffer = []
+        backend_info = "🚀 LMDeploy" if using_lmdeploy else "📦 Standard"
+        for sr, audio_data in pre_buffer:
+            full_audio_buffer.append(audio_data)
+            yield (sr, audio_data), f"🔊 Đang phát ({backend_info})..."
+        
+        while True:
+            try:
+                item = audio_queue.get(timeout=0.05)
+                if item is None:
+                    break
+                sr, audio_data = item
+                full_audio_buffer.append(audio_data)
+                yield (sr, audio_data), f"🔊 Đang phát ({backend_info})..."
+            except queue.Empty:
+                if error_event.is_set():
+                    yield None, f"❌ Lỗi: {error_msg}"
+                    break
+                if end_event.is_set() and audio_queue.empty():
+                    break
+                continue
+        
+        if full_audio_buffer:
+            final_wav = np.concatenate(full_audio_buffer)
+            with tempfile.NamedTemporaryFile(delete=False, suffix=".wav") as tmp:
+                sf.write(tmp.name, final_wav, sr)
+                yield tmp.name, f"✅ Hoàn tất Streaming! ({backend_info})"
+            
+            # Cleanup memory
+            if using_lmdeploy and hasattr(tts, 'cleanup_memory'):
+                tts.cleanup_memory()
+            cleanup_gpu_memory()
+
+
+# --- 4. UI SETUP ---
+theme = gr.themes.Soft(
+    primary_hue="indigo",
+    secondary_hue="cyan",
+    neutral_hue="slate",
+    font=[gr.themes.GoogleFont('Inter'), 'ui-sans-serif', 'system-ui'],
+).set(
+    button_primary_background_fill="linear-gradient(90deg, #6366f1 0%, #0ea5e9 100%)",
+    button_primary_background_fill_hover="linear-gradient(90deg, #4f46e5 0%, #0284c7 100%)",
+)
+
+css = """
+.container { max-width: 1400px; margin: auto; }
+.header-box {
+    text-align: center;
+    margin-bottom: 25px;
+    padding: 25px;
+    background: linear-gradient(135deg, #0f172a 0%, #1e293b 100%);
+    border-radius: 12px;
+    color: white !important;
+}
+.header-title {
+    font-size: 2.5rem;
+    font-weight: 800;
+    color: white !important;
+}
+.gradient-text {
+    background: -webkit-linear-gradient(45deg, #60A5FA, #22D3EE);
+    -webkit-background-clip: text;
+    -webkit-text-fill-color: transparent;
+}
+.header-icon {
+    color: white;
+}
+.status-box {
+    font-weight: bold;
+    text-align: center;
+    border: none;
+    background: transparent;
+}
+.model-card-content {
+    display: flex;
+    flex-wrap: wrap;
+    justify-content: center;
+    align-items: center;
+    gap: 15px;
+    font-size: 0.9rem;
+    text-align: center;
+    color: white !important;
+}
+.model-card-item {
+    display: flex;
+    align-items: center;
+    justify-content: center;
+    gap: 6px;
+    color: white !important;
+}
+.model-card-item strong {
+    color: white !important;
+}
+.model-card-item span {
+    color: white !important;
+}
+.model-card-link {
+    color: #60A5FA;
+    text-decoration: none;
+    font-weight: 500;
+    transition: color 0.2s;
+}
+.model-card-link:hover {
+    color: #22D3EE;
+    text-decoration: underline;
+}
+.warning-banner {
+    background-color: #fffbeb;
+    border: 1px solid #fef3c7;
+    border-radius: 12px;
+    padding: 16px;
+    margin-bottom: 20px;
+}
+.warning-banner-title {
+    color: #92400e;
+    font-weight: 700;
+    font-size: 1.1rem;
+    display: flex;
+    align-items: center;
+    gap: 8px;
+    margin-bottom: 12px;
+}
+.warning-banner-grid {
+    display: flex;
+    gap: 15px;
+    flex-wrap: wrap;
+}
+.warning-banner-item {
+    flex: 1;
+    min-width: 240px;
+    background: #fef3c7;
+    padding: 12px;
+    border-radius: 8px;
+    border: 1px solid #fde68a;
+}
+.warning-banner-item strong {
+    color: #b45309;
+    display: block;
+    margin-bottom: 4px;
+    font-size: 0.95rem;
+}
+.warning-banner-content {
+    color: #78350f;
+    font-size: 0.9rem;
+    line-height: 1.5;
+}
+.warning-banner-content b {
+    color: #451a03;
+    background: rgba(251, 191, 36, 0.2);
+    padding: 1px 4px;
+    border-radius: 4px;
+}
+"""
+
+EXAMPLES_LIST = [
+    ["Về miền Tây không chỉ để ngắm nhìn sông nước hữu tình, mà còn để cảm nhận tấm chân tình của người dân nơi đây.", "Vĩnh (nam miền Nam)"],
+    ["Hà Nội những ngày vào thu mang một vẻ đẹp trầm mặc và cổ kính đến lạ thường.", "Bình (nam miền Bắc)"],
+]
+
+with gr.Blocks(theme=theme, css=css, title="VieNeu-TTS") as demo:
+    with gr.Column(elem_classes="container"):
+        gr.HTML("""
+<div class="header-box">
+    <h1 class="header-title">
+        <span class="header-icon">🦜</span>
+        <span class="gradient-text">VieNeu-TTS Studio</span>
+    </h1>
+    <div class="model-card-content">
+        <div class="model-card-item">
+            <strong>Models:</strong>
+            <a href="https://huggingface.co/pnnbao-ump/VieNeu-TTS" target="_blank" class="model-card-link">VieNeu-TTS</a>
+            <span>•</span>
+            <a href="https://huggingface.co/pnnbao-ump/VieNeu-TTS-0.3B" target="_blank" class="model-card-link">VieNeu-TTS-0.3B</a>
+        </div>
+        <div class="model-card-item">
+            <strong>Repository:</strong>
+            <a href="https://github.com/pnnbao97/VieNeu-TTS" target="_blank" class="model-card-link">GitHub</a>
+        </div>
+        <div class="model-card-item">
+            <strong>Tác giả:</strong>
+            <a href="https://www.facebook.com/bao.phamnguyenngoc.5" target="_blank" class="model-card-link">Phạm Nguyễn Ngọc Bảo</a>
+        </div>
+    </div>
+</div>
+        """)
+        
+        # --- CONFIGURATION ---
+        with gr.Group():
+            with gr.Row():
+                backbone_select = gr.Dropdown(
+                    list(BACKBONE_CONFIGS.keys()), 
+                    value="VieNeu-TTS (GPU)", 
+                    label="🦜 Backbone"
+                )
+                codec_select = gr.Dropdown(list(CODEC_CONFIGS.keys()), value="NeuCodec (Distill)", label="🎵 Codec")
+                device_choice = gr.Radio(get_available_devices(), value="Auto", label="🖥️ Device")
+            
+            with gr.Row():
+                use_lmdeploy_cb = gr.Checkbox(
+                    value=True, 
+                    label="🚀 Optimize with LMDeploy (Khuyên dùng cho NVIDIA GPU)",
+                    info="Tick nếu bạn dùng GPU để tăng tốc độ tổng hợp đáng kể."
+                )
+            
+            gr.HTML("""
+            <div class="warning-banner">
+                <div class="warning-banner-title">
+                    🦜 Gợi ý tối ưu hiệu năng
+                </div>
+                <div class="warning-banner-grid">
+                    <div class="warning-banner-item">
+                        <strong>🐢 Hệ máy CPU</strong>
+                        <div class="warning-banner-content">
+                            Sử dụng <b>VieNeu-TTS-0.3B-q4-gguf</b> để đạt tốc độ xử lý nhanh nhất. Nếu ưu tiên độ chính xác thì dùng <b>VieNeu-TTS-0.3B-q8-gguf</b>.
+                        </div>
+                    </div>
+                    <div class="warning-banner-item">
+                        <strong>🐆 Hệ máy GPU</strong>
+                        <div class="warning-banner-content">
+                            Chọn <b>VieNeu-TTS-0.3B (GPU)</b> để x2 tốc độ (độ chính xác ~95% bản gốc).
+                        </div>
+                    </div>
+                </div>
+            </div>
+            """)
+
+            btn_load = gr.Button("🔄 Tải Model", variant="primary")
+            model_status = gr.Markdown("⏳ Chưa tải model.")
+        
+        with gr.Row(elem_classes="container"):
+            # --- INPUT ---
+            with gr.Column(scale=3):
+                text_input = gr.Textbox(
+                    label=f"Văn bản",
+                    lines=4,
+                    value="Hà Nội, trái tim của Việt Nam, là một thành phố ngàn năm văn hiến với bề dày lịch sử và văn hóa độc đáo. Bước chân trên những con phố cổ kính quanh Hồ Hoàn Kiếm, du khách như được du hành ngược thời gian, chiêm ngưỡng kiến trúc Pháp cổ điển hòa quyện với nét kiến trúc truyền thống Việt Nam. Mỗi con phố trong khu phố cổ mang một tên gọi đặc trưng, phản ánh nghề thủ công truyền thống từng thịnh hành nơi đây như phố Hàng Bạc, Hàng Đào, Hàng Mã. Ẩm thực Hà Nội cũng là một điểm nhấn đặc biệt, từ tô phở nóng hổi buổi sáng, bún chả thơm lừng trưa hè, đến chè Thái ngọt ngào chiều thu. Những món ăn dân dã này đã trở thành biểu tượng của văn hóa ẩm thực Việt, được cả thế giới yêu mến. Người Hà Nội nổi tiếng với tính cách hiền hòa, lịch thiệp nhưng cũng rất cầu toàn trong từng chi tiết nhỏ, từ cách pha trà sen cho đến cách chọn hoa sen tây để thưởng trà.",
+                )
+                
+                with gr.Tabs() as tabs:
+                    with gr.TabItem("👤 Preset", id="preset_mode") as tab_preset:
+                        initial_voices = get_voice_options("VieNeu-TTS (GPU)")
+                        default_voice = initial_voices[0] if initial_voices else None
+                        voice_select = gr.Dropdown(initial_voices, value=default_voice, label="Giọng mẫu")
+                    
+                    with gr.TabItem("🦜 Voice Cloning", id="custom_mode") as tab_custom:
+                        custom_audio = gr.Audio(label="Audio giọng mẫu (3-5 giây) (.wav)", type="filepath")
+                        custom_text = gr.Textbox(label="Nội dung audio mẫu - vui lòng gõ đúng nội dung của audio mẫu - kể cả dấu câu vì model rất nhạy cảm với dấu câu (.,?!)")
+                        gr.Examples(
+                            examples=[
+                                [os.path.join("examples", "audio_ref", "example.wav"), "Ví dụ 2. Tính trung bình của dãy số."],
+                                [os.path.join("examples", "audio_ref", "example_2.wav"), "Trên thực tế, các nghi ngờ đã bắt đầu xuất hiện."]
+                            ],
+                            inputs=[custom_audio, custom_text],
+                            label="Ví dụ mẫu để thử nghiệm clone giọng"
+                        )
+
+                
+                generation_mode = gr.Radio(
+                    ["Standard (Một lần)"],
+                    value="Standard (Một lần)",
+                    label="Chế độ sinh"
+                )
+                with gr.Row():
+                    use_batch = gr.Checkbox(
+                        value=True, 
+                        label="⚡ Batch Processing",
+                        info="Xử lý nhiều đoạn cùng lúc (chỉ áp dụng khi sử dụng GPU và đã cài đặt LMDeploy)"
+                    )
+                    max_batch_size_run = gr.Slider(
+                        minimum=1, 
+                        maximum=16, 
+                        value=4, 
+                        step=1, 
+                        label="📊 Batch Size (Generation)",
+                        info="Số lượng đoạn văn bản xử lý cùng lúc. Giá trị cao = nhanh hơn nhưng tốn VRAM hơn. Giảm xuống nếu gặp lỗi Out of Memory."
+                    )
+                
+                # State to track current mode (replaces unreliable Textbox/Tabs input)
+                current_mode_state = gr.State("preset_mode")
+                
+                with gr.Row():
+                    btn_generate = gr.Button("🎵 Bắt đầu", variant="primary", scale=2, interactive=False)
+                    btn_stop = gr.Button("⏹️ Dừng", variant="stop", scale=1, interactive=False)
+            
+            # --- OUTPUT ---
+            with gr.Column(scale=2):
+                audio_output = gr.Audio(
+                    label="Kết quả",
+                    type="filepath",
+                    autoplay=True
+                )
+                status_output = gr.Textbox(label="Trạng thái", elem_classes="status-box")
+                gr.Markdown("<div style='text-align: center; color: #64748b; font-size: 0.8rem;'>🔒 Audio được đóng dấu bản quyền ẩn (Watermarker) để bảo mật và định danh AI.</div>")
+        
+        # # --- EVENT HANDLERS ---
+        # def update_info(backbone: str) -> str:
+        #     return f"Streaming: {'✅' if BACKBONE_CONFIGS[backbone]['supports_streaming'] else '❌'}"
+        
+        # backbone_select.change(update_info, backbone_select, model_status)
+        backbone_select.change(update_voice_dropdown, [backbone_select, voice_select], voice_select)
+        
+        # Handler to show/hide Voice Cloning tab
+        def on_codec_change(codec: str):
+            is_onnx = "onnx" in codec.lower()
+            # If switching to ONNX and we are on custom mode, switch back to preset
+            return gr.update(visible=not is_onnx), gr.update(selected="preset_mode" if is_onnx else None)
+        
+        codec_select.change(
+            on_codec_change, 
+            inputs=[codec_select], 
+            outputs=[tab_custom, tabs]
+        )
+        
+        # Bind tab events to update state
+        tab_preset.select(lambda: "preset_mode", outputs=current_mode_state)
+        tab_custom.select(lambda: "custom_mode", outputs=current_mode_state)
+        
+        btn_load.click(
+            fn=load_model,
+            inputs=[backbone_select, codec_select, device_choice, use_lmdeploy_cb],
+            outputs=[model_status, btn_generate, btn_load, btn_stop]
+        )
+        
+        generate_event = btn_generate.click(
+            fn=synthesize_speech,
+            inputs=[text_input, voice_select, custom_audio, custom_text, current_mode_state, generation_mode, use_batch, max_batch_size_run],
+            outputs=[audio_output, status_output]
+        )
+        
+        # When generation starts, enable stop button
+        btn_generate.click(lambda: gr.update(interactive=True), outputs=btn_stop)
+        # When generation ends/stops, disable stop button
+        generate_event.then(lambda: gr.update(interactive=False), outputs=btn_stop)
+        
+        btn_stop.click(fn=None, cancels=[generate_event])
+        btn_stop.click(lambda: (None, "⏹️ Đã dừng tạo giọng nói."), outputs=[audio_output, status_output])
+        btn_stop.click(lambda: gr.update(interactive=False), outputs=btn_stop)
+
+        # Persistence: Restore UI state on load
+        demo.load(
+            fn=restore_ui_state,
+            outputs=[model_status, btn_generate, btn_stop]
+        )
+
+if __name__ == "__main__":
+    # Cho phép override từ biến môi trường (hữu ích cho Docker)
+    server_name = os.getenv("GRADIO_SERVER_NAME", "127.0.0.1")
+    server_port = int(os.getenv("GRADIO_SERVER_PORT", "7860"))
+
+    # Check running in Colab
+    is_on_colab = os.getenv("COLAB_RELEASE_TAG") is not None
+
+    # Default:
+    # - Colab: share=True (convenient)
+    # - Docker/local: share=False (safe)
+    share = env_bool("GRADIO_SHARE", default=is_on_colab)
+    #
+    # If server_name is "0.0.0.0" and GRADIO_SHARE is not set, disable sharing
+    if server_name == "0.0.0.0" and os.getenv("GRADIO_SHARE") is None:
+        share = False
+
+    demo.queue().launch(server_name=server_name, server_port=server_port, share=share)

main.py

@@ -0,0 +1,80 @@
+from vieneu_tts import VieNeuTTS
+import soundfile as sf
+import torch
+import os
+
+device = "cpu"
+
+input_texts = [
+    "Các khóa học trực tuyến đang giúp học sinh tiếp cận kiến thức mọi lúc mọi nơi. Giáo viên sử dụng video, bài tập tương tác và thảo luận trực tuyến để nâng cao hiệu quả học tập.",
+
+    "Các nghiên cứu về bệnh Alzheimer cho thấy tác dụng tích cực của các bài tập trí não và chế độ dinh dưỡng lành mạnh, giúp giảm tốc độ suy giảm trí nhớ ở người cao tuổi.",
+
+    "Một tiểu thuyết trinh thám hiện đại dẫn dắt độc giả qua những tình tiết phức tạp, bí ẩn, kết hợp yếu tố tâm lý sâu sắc khiến người đọc luôn hồi hộp theo dõi diễn biến câu chuyện.",
+
+    "Các nhà khoa học nghiên cứu gen người phát hiện những đột biến mới liên quan đến bệnh di truyền. Điều này giúp nâng cao khả năng chẩn đoán và điều trị.",
+]
+
+output_dir = "./output_audio"
+os.makedirs(output_dir, exist_ok=True)
+
+def main(backbone="pnnbao-ump/VieNeu-TTS-q4-gguf", codec="neuphonic/neucodec-onnx-decoder"):
+    """
+    In the sample directory, there are wav files and txt files with matching names.
+    These are pre-prepared reference files for testing with Vietnamese names:
+    - Bình (nam miền Bắc) - Male, North accent
+    - Tuyên (nam miền Bắc) - Male, North accent
+    - Nguyên (nam miền Nam) - Male, South accent
+    - Sơn (nam miền Nam) - Male, South accent
+    - Vĩnh (nam miền Nam) - Male, South accent
+    - Hương (nữ miền Bắc) - Female, North accent
+    - Ly (nữ miền Bắc) - Female, North accent
+    - Ngọc (nữ miền Bắc) - Female, North accent
+    - Đoan (nữ miền Nam) - Female, South accent
+    - Dung (nữ miền Nam) - Female, South accent
+    
+    Note: The model can clone any voice you provide (with corresponding text).
+    However, quality may not match the sample files. For best results, finetune
+    the model on your target voice. See finetune guide at:
+    https://github.com/pnnbao-ump/VieNeuTTS/blob/main/finetune.ipynb
+    """
+    # Male voice (South accent)
+    ref_audio_path = "./sample/Vĩnh (nam miền Nam).wav"
+    ref_text_path = "./sample/Vĩnh (nam miền Nam).txt"
+    ref_codes_path = "./sample/Vĩnh (nam miền Nam).pt"
+    
+    # Female voice (South accent) - uncomment to use
+    # ref_audio_path = "./sample/Đoan (nữ miền Nam).wav"
+    # ref_text_path = "./sample/Đoan (nữ miền Nam).txt"
+
+    ref_text_raw = open(ref_text_path, "r", encoding="utf-8").read()
+    
+    if not ref_audio_path or not ref_text_raw:
+        print("No reference audio or text provided.")
+        return None
+
+    # Initialize VieNeuTTS-1000h
+    tts = VieNeuTTS(
+        backbone_repo=backbone,
+        backbone_device=device,
+        codec_repo=codec,
+        codec_device=device
+    )
+
+    if codec == "neuphonic/neucodec-onnx-decoder":
+        print("Load reference codes...")
+        ref_codes = torch.load(ref_codes_path, map_location=device)
+    else:
+        print("Encoding reference audio...")
+        ref_codes = tts.encode_reference(ref_audio_path)
+
+    # Generate speech for all input texts
+    for i, text in enumerate(input_texts, 1):
+        print(f"Generating audio {i}/{len(input_texts)}: {text[:50]}...")
+        wav = tts.infer(text, ref_codes, ref_text_raw)
+        output_path = os.path.join(output_dir, f"output_{i}.wav")
+        sf.write(output_path, wav, 24000)
+        print(f"✓ Saved to {output_path}")
+
+if __name__ == "__main__":
+    main()

pyproject.toml

@@ -0,0 +1,70 @@
+[tool.uv]
+index-strategy = "unsafe-best-match"
+required-environments = [
+    "sys_platform == 'win32' and platform_machine == 'AMD64'",
+    "sys_platform == 'linux' and platform_machine == 'x86_64'",
+    "sys_platform == 'darwin' and platform_machine == 'arm64'",
+]
+override-dependencies = [
+    "nvidia-nccl-cu12; sys_platform == 'linux'",
+]
+
+[[tool.uv.index]]
+name = "pytorch"
+url = "https://download.pytorch.org/whl/cu128"
+explicit = true
+
+[[tool.uv.index]]
+name = "pypi"
+url = "https://pypi.org/simple"
+
+[project]
+name = "VieNeu-TTS"
+version = "0.1.0"
+description = "Advanced on-device Vietnamese TTS with instant voice cloning"
+readme = "README.md"
+requires-python = "==3.12.*"
+dependencies = [
+    "phonemizer>=3.3.0",
+    "neucodec>=0.0.4",
+    "librosa>=0.11.0",
+    "gradio>=5.49.1",
+    "onnxruntime>=1.23.2",
+    "datasets>=3.2.0",
+    "lmdeploy; sys_platform != 'darwin'",
+    "triton-windows; sys_platform == 'win32'",
+    "triton; sys_platform == 'linux'",
+    "transformers; sys_platform == 'darwin'",
+    "accelerate; sys_platform == 'darwin'",
+    "torch",
+    "torchvision",
+    "torchaudio",
+    "perth>=0.2.0",
+    "llama-cpp-python==0.3.16",
+]
+
+[tool.uv.sources]
+
+torch = [
+    { index = "pytorch", marker = "sys_platform != 'darwin'" },
+    { index = "pypi", marker = "sys_platform == 'darwin'" } 
+]
+torchvision = [
+    { index = "pytorch", marker = "sys_platform != 'darwin'" },
+    { index = "pypi", marker = "sys_platform == 'darwin'" }
+]
+torchaudio = [
+    { index = "pytorch", marker = "sys_platform != 'darwin'" },
+    { index = "pypi", marker = "sys_platform == 'darwin'" }
+]
+
+lmdeploy = [
+    { url = "https://github.com/InternLM/lmdeploy/releases/download/v0.11.0/lmdeploy-0.11.0+cu128-cp312-cp312-win_amd64.whl", marker = "sys_platform == 'win32' and python_version == '3.12'" },
+    { url = "https://github.com/InternLM/lmdeploy/releases/download/v0.11.0/lmdeploy-0.11.0+cu128-cp312-cp312-manylinux2014_x86_64.whl", marker = "sys_platform == 'linux' and python_version == '3.12'" },
+    { index = "pypi", marker = "sys_platform == 'darwin'" }
+]
+
+llama-cpp-python = [
+    { url = "https://github.com/pnnbao97/VieNeu-TTS/releases/download/wheels-v0.3.16/llama_cpp_python-0.3.16-cp312-cp312-win_amd64.whl", marker = "sys_platform == 'win32' and python_version == '3.12'" },
+    { index = "pypi", marker = "sys_platform != 'win32'" }
+]

requirements.txt

@@ -0,0 +1,162 @@
+accelerate==1.12.0
+addict==2.4.0
+aiofiles==24.1.0
+aiohappyeyeballs==2.6.1
+aiohttp==3.13.1
+aiosignal==1.4.0
+annotated-doc==0.0.3
+annotated-types==0.7.0
+antlr4-python3-runtime==4.9.3
+anyio==4.11.0
+attrs==25.4.0
+audioread==3.1.0
+babel==2.17.0
+blobfile==3.1.0
+brotli==1.1.0
+certifi==2025.10.5
+cffi==2.0.0
+charset-normalizer==3.4.4
+click==8.3.0
+cloudpickle==3.1.2
+colorama==0.4.6
+coloredlogs==15.0.1
+csvw==3.7.0
+datasets==4.3.0
+decorator==5.2.1
+dill==0.4.0
+distro==1.9.0
+dlinfo==2.0.0
+einops==0.8.1
+einx==0.3.0
+fastapi==0.120.2
+ffmpy==0.6.4
+filelock==3.20.0
+fire==0.7.1
+flatbuffers==25.9.23
+frozendict==2.4.6
+frozenlist==1.8.0
+fsspec==2025.9.0
+gradio==5.49.1
+gradio-client==1.13.3
+groovy==0.1.2
+h11==0.16.0
+hf-transfer==0.1.9
+httpcore==1.0.9
+httpx==0.28.1
+huggingface-hub==0.36.0
+humanfriendly==10.0
+hyper-connections==0.2.1
+idna==3.11
+inquirerpy==0.3.4
+isodate==0.7.2
+jinja2==3.1.6
+jiter==0.12.0
+joblib==1.5.2
+jsonschema==4.25.1
+jsonschema-specifications==2025.9.1
+kagglehub==0.3.13
+language-tags==1.2.0
+lazy-loader==0.4
+librosa==0.11.0
+llvmlite==0.45.1
+lmdeploy==0.11.0
+local-attention==1.11.2
+lxml==6.0.2
+markdown-it-py==4.0.0
+markupsafe==3.0.3
+mdurl==0.1.2
+mmengine-lite==0.10.7
+mpmath==1.3.0
+msgpack==1.1.2
+multidict==6.7.0
+multiprocess==0.70.16
+networkx==3.5
+neucodec==0.0.4
+numba==0.62.1
+numpy==2.3.4
+omegaconf==2.3.0
+onnxruntime==1.23.2
+openai==2.12.0
+openai-harmony==0.0.8
+orjson==3.11.4
+packaging==25.0
+pandas==2.3.3
+partial-json-parser==0.2.1.1.post7
+peft==0.14.0
+pfzy==0.3.4
+phonemizer==3.3.0
+pillow==11.3.0
+platformdirs==4.5.0
+pooch==1.8.2
+prometheus-client==0.23.1
+prompt-toolkit==3.0.52
+propcache==0.4.1
+protobuf==6.33.2
+psutil==7.1.2
+pyarrow==22.0.0
+pycparser==2.23
+pycryptodomex==3.23.0
+pydantic==2.11.10
+pydantic-core==2.33.2
+pydub==0.25.1
+pygments==2.19.2
+pyparsing==3.2.5
+pyreadline3==3.5.4
+python-dateutil==2.9.0.post0
+python-multipart==0.0.20
+pytz==2025.2
+pyyaml==6.0.3
+pyzmq==27.1.0
+ray==2.52.1
+rdflib==7.3.0
+referencing==0.37.0
+regex==2025.10.23
+requests==2.32.5
+rfc3986==1.5.0
+rich==14.2.0
+rpds-py==0.28.0
+ruff==0.14.2
+safehttpx==0.1.7
+safetensors==0.6.2
+scikit-learn==1.7.2
+scipy==1.16.2
+segments==2.3.0
+semantic-version==2.10.0
+sentencepiece==0.2.1
+setuptools==80.9.0
+shellingham==1.5.4
+shortuuid==1.0.13
+six==1.17.0
+sniffio==1.3.1
+soundfile==0.13.1
+soxr==1.0.0
+starlette==0.49.1
+sympy==1.14.0
+termcolor==3.2.0
+threadpoolctl==3.6.0
+tiktoken==0.12.0
+tokenizers==0.22.1
+tomlkit==0.13.3
+torch==2.7.1+cu118
+torchao==0.14.1
+torchaudio==2.7.1+cu118
+torchdata==0.11.0
+torchtune==0.6.1
+torchvision==0.22.1+cu118
+tqdm==4.67.1
+transformers==4.57.1
+triton-windows==3.5.1.post22
+typer==0.20.0
+typing-extensions==4.15.0
+typing-inspection==0.4.2
+tzdata==2025.2
+uritemplate==4.2.0
+urllib3==2.5.0
+uvicorn==0.38.0
+vector-quantize-pytorch==1.17.8
+wcwidth==0.2.14
+websockets==15.0.1
+xgrammar==0.1.28
+xxhash==3.6.0
+yapf==0.43.0
+yarl==1.22.0