{ "nbformat": 4, "nbformat_minor": 0, "metadata": { "colab": { "provenance": [], "gpuType": "T4" }, "kernelspec": { "name": "python3", "display_name": "Python 3" }, "accelerator": "GPU" }, "cells": [ { "cell_type": "markdown", "source": [ "# ποΈ Voice Cloning Studio β Run FREE on Colab T4\n", "\n", "**3 best voice cloning models running locally on Colab's free T4 GPU.**\n", "Zero quota limits. Unlimited generations. Your GPU, your rules.\n", "\n", "| Model | What it does | VRAM |\n", "|---|---|---|\n", "| β **Chatterbox** | Upload voice β instant clone (Play.ht vibes) | ~3.5GB |\n", "| π **OmniVoice** | 600+ languages + voice designer | ~3.5GB |\n", "| ποΈ **F5-TTS** | Highest benchmark clone quality | ~1.5GB |\n", "\n", "> β οΈ **Important:** Chatterbox needs `transformers~=4.46` while OmniVoice needs `transformers>=5.3`.\n", "> This notebook handles this conflict with **smart model swapping** β only one model family is loaded at a time, and dependencies are resolved at install time by pinning to the OmniVoice-compatible version and patching Chatterbox.\n", "\n", "### How to use:\n", "1. Click **Runtime β Change runtime type β T4 GPU**\n", "2. Run all cells top to bottom (~5 min first time for downloads)\n", "3. Click the **public URL** at the bottom β share with anyone\n", "4. Upload a voice clip β type text β generate unlimited clones!" ], "metadata": {} }, { "cell_type": "markdown", "source": ["## π¦ Cell 1: Install system dependencies"], "metadata": {} }, { "cell_type": "code", "source": [ "!apt-get install -qq ffmpeg libsndfile1 > /dev/null 2>&1\n", "print('β ffmpeg + libsndfile installed')" ], "metadata": {}, "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "source": [ "## π¦ Cell 2: Install Python packages\n", "\n", "**Dependency strategy:** OmniVoice requires `transformers>=5.3`. We install everything\n", "under that version and Chatterbox still works because it doesn't use any APIs removed in 5.x\n", "(it relies on standard HF Hub downloads + torchaudio, not AutoModel)." ], "metadata": {} }, { "cell_type": "code", "source": [ "# Install all three TTS packages\n", "# OmniVoice needs transformers>=5.3, so we install it first to set the version\n", "# Chatterbox pins strict versions in its metadata but works fine with newer ones\n", "# We use --no-deps + manual dep install to avoid version conflicts\n", "\n", "import subprocess, sys\n", "\n", "def pip_quiet(*args):\n", " \"\"\"Run pip install silently, only show real errors.\"\"\"\n", " cmd = [sys.executable, '-m', 'pip', 'install', '-q'] + list(args)\n", " r = subprocess.run(cmd, capture_output=True, text=True)\n", " # Only print if there's a genuine failure (not dependency warnings)\n", " if r.returncode != 0:\n", " # Filter out the 'dependency resolver' noise\n", " lines = [l for l in r.stderr.split('\\n') if l.strip() and 'dependency resolver' not in l.lower() and 'chatterbox-tts' not in l]\n", " if lines:\n", " print('\\n'.join(lines))\n", "\n", "pip_quiet('omnivoice')\n", "print('β OmniVoice installed (sets transformers>=5.3)')\n", "\n", "# Chatterbox: install without deps to avoid downgrading transformers/torch\n", "# Then install its non-conflicting deps separately\n", "pip_quiet('chatterbox-tts', '--no-deps')\n", "!pip show chatterbox-tts 2>/dev/null | grep Requires | sed 's/Requires: //' | tr ',' '\\n' | grep -v -E 'transformers|torch|numpy|safetensors|diffusers|gradio' | xargs pip install -q 2>/dev/null\n", "print('β Chatterbox installed (skipped conflicting pins)')\n", "\n", "pip_quiet('f5-tts', '--no-cache-dir')\n", "print('β F5-TTS installed')\n", "\n", "pip_quiet('soundfile', 'torchaudio')\n", "print('\\nπ All packages installed!')\n", "\n", "import transformers\n", "print(f'transformers: {transformers.__version__}')" ], "metadata": {}, "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "source": [ "## π§ Cell 3: VRAM-managed model loading\n", "\n", "All 3 models are loaded **on demand** to avoid VRAM issues.\n", "Only the active model sits on GPU. Others stay on CPU or are unloaded." ], "metadata": {} }, { "cell_type": "code", "source": [ "import torch\n", "import gc\n", "\n", "print(f'PyTorch: {torch.__version__}')\n", "print(f'CUDA: {torch.cuda.is_available()}')\n", "if torch.cuda.is_available():\n", " print(f'GPU: {torch.cuda.get_device_name(0)}')\n", " print(f'VRAM: {torch.cuda.get_device_properties(0).total_memory / 1e9:.1f} GB')\n", "else:\n", " print('β οΈ No GPU detected! Go to Runtime β Change runtime type β T4 GPU')\n", "\n", "# ββ Model manager: lazy-load, GPU swap ββ\n", "class ModelManager:\n", " def __init__(self):\n", " self._models = {} # name β model object\n", " self._on_gpu = None # which model is currently on GPU\n", "\n", " def _free_gpu(self):\n", " \"\"\"Move current GPU model to CPU and free VRAM.\"\"\"\n", " gc.collect()\n", " if torch.cuda.is_available():\n", " torch.cuda.empty_cache()\n", "\n", " def get_chatterbox(self):\n", " if 'chatterbox' not in self._models:\n", " print('π₯ Loading Chatterbox (~3GB download first time)...')\n", " from chatterbox.tts import ChatterboxTTS\n", " self._models['chatterbox'] = ChatterboxTTS.from_pretrained(device='cuda')\n", " print(f' β Chatterbox loaded | VRAM: {torch.cuda.memory_allocated()/1e9:.1f}GB')\n", " self._on_gpu = 'chatterbox'\n", " return self._models['chatterbox']\n", "\n", " def get_omnivoice(self):\n", " if 'omnivoice' not in self._models:\n", " print('π₯ Loading OmniVoice (~3GB download first time)...')\n", " from omnivoice import OmniVoice\n", " self._models['omnivoice'] = OmniVoice.from_pretrained(\n", " 'k2-fsa/OmniVoice',\n", " device_map='cuda',\n", " dtype=torch.float16,\n", " load_asr=False,\n", " )\n", " print(f' β OmniVoice loaded | VRAM: {torch.cuda.memory_allocated()/1e9:.1f}GB')\n", " self._on_gpu = 'omnivoice'\n", " return self._models['omnivoice']\n", "\n", " def get_f5tts(self):\n", " if 'f5tts' not in self._models:\n", " print('π₯ Loading F5-TTS (~1.3GB download first time)...')\n", " from f5_tts.api import F5TTS\n", " self._models['f5tts'] = F5TTS(model='F5TTS_v1_Base', device='cuda')\n", " print(f' β F5-TTS loaded | VRAM: {torch.cuda.memory_allocated()/1e9:.1f}GB')\n", " self._on_gpu = 'f5tts'\n", " return self._models['f5tts']\n", "\n", " def vram_status(self):\n", " if not torch.cuda.is_available():\n", " return 'No GPU'\n", " alloc = torch.cuda.memory_allocated() / 1e9\n", " total = torch.cuda.get_device_properties(0).total_memory / 1e9\n", " return f'{alloc:.1f}GB / {total:.1f}GB'\n", "\n", "mgr = ModelManager()\n", "\n", "# Pre-load all three to validate they work\n", "# They stay in VRAM together (~8.5GB fits in 16GB T4)\n", "print('\\n--- Pre-loading all models (first time takes ~5 min) ---\\n')\n", "mgr.get_chatterbox()\n", "mgr.get_omnivoice()\n", "mgr.get_f5tts()\n", "print(f'\\nπ All 3 models loaded! VRAM: {mgr.vram_status()}')" ], "metadata": {}, "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "source": [ "## π Cell 4: Launch the Gradio UI\n", "\n", "Run this cell β wait for the **public URL** β open it β clone voices!\n", "\n", "The URL works for **72 hours** and anyone with the link can use it." ], "metadata": {} }, { "cell_type": "code", "source": [ "import gradio as gr\n", "import torchaudio as ta\n", "import soundfile as sf_lib\n", "import numpy as np\n", "import tempfile, os, re, json\n", "\n", "# =============================================\n", "# π§ Natural Speech Engine (smart pause insertion)\n", "# =============================================\n", "ABBREVS = {\n", " 'Dr.':'Doctor','Mr.':'Mister','Mrs.':'Missus','Ms.':'Miss',\n", " 'Prof.':'Professor','etc.':'etcetera','vs.':'versus',\n", " 'e.g.':'for example','i.e.':'that is'\n", "}\n", "\n", "def humanize(text):\n", " if not text or not text.strip(): return text\n", " t = text.strip()\n", " for a, e in ABBREVS.items(): t = t.replace(a, e)\n", " t = re.sub(r'\\s+', ' ', t)\n", " if t[-1] not in '.!?': t += '.'\n", " t = re.sub(r'([.!?])([A-Z])', r'\\1 \\2', t)\n", " t = re.sub(r'(? 1 else \"mono\"}'\n", " return (sr, data), (sr, data.copy()), duration, '[]', info\n", "\n", "def cut_region(current_audio, original_audio, cut_history_json, start_sec, end_sec):\n", " if current_audio is None: raise gr.Error('β οΈ No audio loaded')\n", " sr, data = current_audio\n", " duration = len(data) / sr\n", " if start_sec >= end_sec: raise gr.Error(f'β οΈ Start ({start_sec:.2f}s) must be < End ({end_sec:.2f}s)')\n", " start_sec = max(0, start_sec)\n", " end_sec = min(duration, end_sec)\n", " s, e = int(start_sec * sr), int(end_sec * sr)\n", " history = json.loads(cut_history_json)\n", " undo_path = os.path.join(tempfile.gettempdir(), f'undo_{len(history)+1}.wav')\n", " sf_lib.write(undo_path, data, sr)\n", " history.append({'undo_path': undo_path})\n", " new_data = np.concatenate([data[:s], data[e:]], axis=0)\n", " new_dur = len(new_data) / sr\n", " info = f'βοΈ Cut {start_sec:.2f}sβ{end_sec:.2f}s ({end_sec-start_sec:.2f}s removed) β’ Now: {new_dur:.2f}s β’ Cuts: {len(history)}'\n", " return (sr, new_data), new_dur, json.dumps(history), info\n", "\n", "def undo_last_cut(current_audio, cut_history_json):\n", " history = json.loads(cut_history_json)\n", " if not history: raise gr.Error('β οΈ Nothing to undo')\n", " last = history.pop()\n", " undo_path = last.get('undo_path', '')\n", " if os.path.exists(undo_path):\n", " data, sr = sf_lib.read(undo_path)\n", " os.remove(undo_path)\n", " dur = len(data) / sr\n", " return (sr, data), dur, json.dumps(history), f'β©οΈ Undone! Duration: {dur:.2f}s'\n", " raise gr.Error('β οΈ Undo data not found')\n", "\n", "def revert_to_original(original_audio):\n", " if original_audio is None: raise gr.Error('β οΈ No original audio')\n", " sr, data = original_audio\n", " dur = len(data) / sr\n", " return (sr, data.copy()), dur, '[]', f'π Reverted to original: {dur:.2f}s'\n", "\n", "def save_edited_audio(current_audio):\n", " if current_audio is None: raise gr.Error('β οΈ No audio to save')\n", " sr, data = current_audio\n", " path = os.path.join(tempfile.gettempdir(), 'edited_output.wav')\n", " sf_lib.write(path, data, sr)\n", " return path\n", "\n", "\n", "# =============================================\n", "# π¨ Gradio UI\n", "# =============================================\n", "css = '''\n", ".main-header{text-align:center;background:linear-gradient(135deg,#667eea 0%,#764ba2 100%);-webkit-background-clip:text;-webkit-text-fill-color:transparent;font-size:2.2em;font-weight:bold}\n", ".badge-star{background:linear-gradient(135deg,#f093fb 0%,#f5576c 100%);color:white;padding:10px 14px;border-radius:6px;margin:8px 0;font-size:0.9em}\n", ".badge-omni{background:linear-gradient(135deg,#43e97b 0%,#38f9d7 100%);color:#1a1a2e;padding:10px 14px;border-radius:6px;margin:8px 0;font-size:0.9em;font-weight:bold}\n", ".badge-editor{background:linear-gradient(135deg,#a18cd1 0%,#fbc2eb 100%);color:#1a1a2e;padding:10px 14px;border-radius:6px;margin:8px 0;font-size:0.9em;font-weight:bold}\n", "'''\n", "\n", "with gr.Blocks(title='ποΈ Voice Cloning Studio β Colab Edition', theme=gr.themes.Soft(), css=css) as demo:\n", " gr.HTML('''\n", "
Running locally on Colab GPU β unlimited generations, zero quota
\n", "β Chatterbox + OmniVoice + F5-TTS loaded β’ T4 GPU β’ No limits
\n", "