Mungert/kani-tts-400m-en-GGUF

kani-tts-400m-en GGUF Models

Model Generation Details

This model was generated using llama.cpp at commit c5023daf6.

---

Click here to get info on choosing the right GGUF model format

---

KaniTTS English

A high-speed, high-fidelity Text-to-Speech model optimized for real-time conversational AI applications.

Overview

KaniTTS uses a two-stage pipeline combining a large language model with an efficient audio codec for exceptional speed and audio quality. The architecture generates compressed token representations through a backbone LLM, then rapidly synthesizes waveforms via neural audio codec, achieving extremely low latency.

Key Specifications:

• Model Size: 400M parameters
• Sample Rate: 22kHz
• Language: English
• License: Apache 2.0

Performance

On NovitaAI RTX 5090 using vLLM:

• RTF: ~0.2 (5 times faster than realtime)
• Memory: 16GB GPU VRAM used
• Source Code: https://github.com/nineninesix-ai/kanitts-vllm

GPU Benchmark Results

GPU Model	VRAM	Cost ($/hr)	RTF
RTX 5090	32GB	$0.423	0.190
RTX 4080	16GB	$0.220	0.200
RTX 5060 Ti	16GB	$0.138	0.529
RTX 4060 Ti	16GB	$0.122	0.537
RTX 3060	12GB	$0.093	0.600

Lower RTF is better (< 1.0 means faster than real-time). Benchmarks conducted on Vast AI.

Quickstart: Install from PyPI & Run Inference

It’s a lightweight so you can install, load a model, and speak in minutes. Designed for quick starts and simple workflows—no heavy setup, just pip install and run. More detailes...

Install

pip install kani-tts
pip install -U "transformers==4.57.1" # for LFM2 !!!

Quick Start

from kani_tts import KaniTTS

model = KaniTTS('nineninesix/kani-tts-400m-en')

# Generate audio from text
audio, text = model("Hello, world!")

# Save to file (requires soundfile)
model.save_audio(audio, "output.wav")

Working with Multi-Speaker Models

This model support multiple speakers. You can check if your model supports speakers and select a specific voice:

from kani_tts import KaniTTS

model = KaniTTS('nineninesix/kani-tts-400m-en')

# Check if model supports multiple speakers
print(f"Model type: {model.status}")  # 'singlspeaker' or 'multispeaker'

# Display available speakers (pretty formatted)
model.show_speakers()

# Or access the speaker list directly
print(model.speaker_list)  # ['andrew', 'katie']

# Generate audio with a specific speaker
audio, text = model("Hello, world!", speaker_id="andrew")

Custom Configuration

from kani_tts import KaniTTS

model = KaniTTS(
    'your-model-name',
    temperature=0.7,           # Control randomness (default: 1.0)
    top_p=0.9,                 # Nucleus sampling (default: 0.95)
    max_new_tokens=2000,       # Max audio length (default: 1200)
    repetition_penalty=1.2,    # Prevent repetition (default: 1.1)
    suppress_logs=True,        # Suppress library logs (default: True)
    show_info=True,            # Show model info on init (default: True)
)

audio, text = model("Your text here")

Playing Audio in Jupyter Notebooks

You can listen to generated audio directly in Jupyter notebooks or IPython:

from kani_tts import KaniTTS
from IPython.display import Audio as aplay

model = KaniTTS('your-model-name')
audio, text = model("Hello, world!")

# Play audio in notebook
aplay(audio, rate=model.sample_rate)

---

Datasets

• https://huggingface.co/datasets/laion/Emolia
• https://huggingface.co/datasets/nytopop/expresso-conversational
• https://huggingface.co/datasets/zeeshanparvez/andrew-v3

Voices:

• andrew
• katie

Audio Examples

Text	Audio
Holy fu- Oh my God! Don't you understand how dangerous it is?
Colleges of Oxford, Cambridge, Durham and the University of the Highlands and Islands UHI are 'listed bodies', as bodies that appear to the Secretary of State to be constituent colleges, schools, halls or other institutions of a university.
A joyful flock of sparrows chirped merrily in the old oak tree outside my window this morning.
Darlin', I still ain't feelin' so well. I'm goin' to bed.

Use Cases

• Conversational AI: Real-time speech for chatbots and virtual assistants
• Edge/Server Deployment: Resource-efficient inference on affordable hardware
• Accessibility: Screen readers and language learning applications
• Research: Fine-tuning for specific voices, accents, or emotions

Limitations

• Performance degrades with inputs exceeding 15 seconds (need to use sliding window chunking)
• Limited expressivity without fine-tuning for specific emotions
• May inherit biases from training data in prosody or pronunciation
• Optimized primarily for English; other languages may require additional training

Optimization Tips

• Multilingual Performance: Continually pretrain on target language datasets and fine-tune NanoCodec
• Batch Processing: Use batches of 8-16 for high-throughput scenarios
• Hardware: Optimized for NVIDIA Blackwell architecture GPUs

Resources

Models:

• Pretrained Model: https://huggingface.co/nineninesix/kani-tts-500m-0.3-pt
• Space: https://huggingface.co/spaces/nineninesix/KaniTTS

Examples:

• OpenAI compatible API Example: https://github.com/nineninesix-ai/kanitts-vllm
• Finetuning code pipeline: https://github.com/nineninesix-ai/KaniTTS-Finetune-pipeline
• Dataset preparation pipeline: https://github.com/nineninesix-ai/nano-codec-dataset-pipeline
• Example Dataset: https://huggingface.co/datasets/nineninesix/expresso-conversational-en-nano-codec-dataset
• GitHub Repository: https://github.com/nineninesix-ai/kani-tts
• ComfyUI node: https://github.com/wildminder/ComfyUI-KaniTTS by WildAi
• NextJS basic app: https://github.com/nineninesix-ai/open-audio. It uses the OpenAI npm package to connect to the OpenAI-compatible server API provided by kanitts-vllm.

Links:

• Website: https://www.nineninesix.ai
• Contact Form: https://airtable.com/appX2G2TpoRk4M5Bf/pagO2xbIOjiwulPcP/form

Acknowledgments

Built on top of LiquidAI LFM2 350M as the backbone and Nvidia NanoCodec for audio processing.

Responsible Use

Prohibited activities include:

• Illegal content or harmful, threatening, defamatory, or obscene material
• Hate speech, harassment, or incitement of violence
• Generating false or misleading information
• Impersonating individuals without consent
• Malicious activities such as spamming, phishing, or fraud By using this model, you agree to comply with these restrictions and all applicable laws.

Contact

Have a question, feedback, or need support? Please fill out our contact form and we'll get back to you as soon as possible.

Citation

@inproceedings{emilialarge,
  author={He, Haorui and Shang, Zengqiang and Wang, Chaoren and Li, Xuyuan and Gu, Yicheng and Hua, Hua and Liu, Liwei and Yang, Chen and Li, Jiaqi and Shi, Peiyang and Wang, Yuancheng and Chen, Kai and Zhang, Pengyuan and Wu, Zhizheng},
  title={Emilia: A Large-Scale, Extensive, Multilingual, and Diverse Dataset for Speech Generation},
  booktitle={arXiv:2501.15907},
  year={2025}
}

@article{emonet_voice_2025,
  author={Schuhmann, Christoph and Kaczmarczyk, Robert and Rabby, Gollam and Friedrich, Felix and Kraus, Maurice and Nadi, Kourosh and Nguyen, Huu and Kersting, Kristian and Auer, Sören},
  title={EmoNet-Voice: A Fine-Grained, Expert-Verified Benchmark for Speech Emotion Detection},
  journal={arXiv preprint arXiv:2506.09827},
  year={2025}
}

---

🚀 If you find these models useful

Help me test my AI-Powered Quantum Network Monitor Assistant with quantum-ready security checks:

👉 Quantum Network Monitor

The full Open Source Code for the Quantum Network Monitor Service available at my github repos ( repos with NetworkMonitor in the name) : Source Code Quantum Network Monitor. You will also find the code I use to quantize the models if you want to do it yourself GGUFModelBuilder

💬 How to test:
Choose an AI assistant type:

• TurboLLM (GPT-4.1-mini)
• HugLLM (Hugginface Open-source models)
• TestLLM (Experimental CPU-only)

What I’m Testing

I’m pushing the limits of small open-source models for AI network monitoring, specifically:

• Function calling against live network services
• How small can a model go while still handling:
  • Automated Nmap security scans
  • Quantum-readiness checks
  • Network Monitoring tasks

🟡 TestLLM – Current experimental model (llama.cpp on 2 CPU threads on huggingface docker space):

• ✅ Zero-configuration setup
• ⏳ 30s load time (slow inference but no API costs) . No token limited as the cost is low.
• 🔧 Help wanted! If you’re into edge-device AI, let’s collaborate!

Other Assistants

🟢 TurboLLM – Uses gpt-4.1-mini :

• **It performs very well but unfortunatly OpenAI charges per token. For this reason tokens usage is limited.
• Create custom cmd processors to run .net code on Quantum Network Monitor Agents
• Real-time network diagnostics and monitoring
• Security Audits
• Penetration testing (Nmap/Metasploit)

🔵 HugLLM – Latest Open-source models:

• 🌐 Runs on Hugging Face Inference API. Performs pretty well using the lastest models hosted on Novita.

💡 Example commands you could test:

1. "Give me info on my websites SSL certificate"
2. "Check if my server is using quantum safe encyption for communication"
3. "Run a comprehensive security audit on my server"
4. '"Create a cmd processor to .. (what ever you want)" Note you need to install a Quantum Network Monitor Agent to run the .net code on. This is a very flexible and powerful feature. Use with caution!

Final Word

I fund the servers used to create these model files, run the Quantum Network Monitor service, and pay for inference from Novita and OpenAI—all out of my own pocket. All the code behind the model creation and the Quantum Network Monitor project is open source. Feel free to use whatever you find helpful.

If you appreciate the work, please consider buying me a coffee ☕. Your support helps cover service costs and allows me to raise token limits for everyone.

I'm also open to job opportunities or sponsorship.

Thank you! 😊