---
license: mit
library_name: onnx
tags:
  - text-to-speech
  - chatterbox
  - lora
  - onnx
  - mobile
base_model: ResembleAI/chatterbox
---

# chatterbox-turbo-lora

LoRA-aware ONNX export of the **complete Chatterbox-Turbo TTS pipeline**,
quantized to **q4f16** for on-device inference (Android / iOS via ORT).

The `language_model` subgraph is a custom fork that exposes 192 extra
input ports so a PEFT-trained LoRA adapter can be bound at runtime. The
other three subgraphs (`speech_encoder`, `embed_tokens`,
`conditional_decoder`) are byte-identical mirrors of the corresponding
files from `ResembleAI/chatterbox-turbo-ONNX/onnx/`, hosted here so an
on-device app has a single-repo source for the full pipeline.

## What's new in v2.0.0

v1.0.0 of the language-model subgraph had no LoRA input ports — adapter
weights could not be bound at runtime. v2.0.0 wraps every `Conv1D`
projection inside the transformer (`attn.c_attn`, `attn.c_proj`,
`mlp.c_fc`, `mlp.c_proj`) in a `LoraConv1D` that exposes two extra
input tensors per layer:

lora.layers.{N}.{attn_c_attn|attn_c_proj|mlp_c_fc|mlp_c_proj}.{A|B}

→ **24 layers × 4 projections × 2 tensors = 192 extra graph inputs.**

At inference time the host runtime binds a PEFT-trained LoRA adapter
(rank, scaling baked in) to these inputs via the ONNX Runtime
`OrtLoraAdapter` API. Unbound inputs default to zero-sized initializers
so the graph still runs without any adapter loaded — the base voice
synthesizes as-is.

## Files

Four subgraphs, each a `.onnx` protobuf + a `.onnx_data` external weight
blob. Every `.onnx` file references its `.onnx_data` companion by
relative filename, so both files of a pair must be downloaded and kept
next to each other.

| File | Size | Purpose | Source |
|---|---|---|---|
| `speech_encoder_q4f16.onnx` | ~1.2 MB | Extracts speaker embedding + audio tokens from a voiceprint clip | Mirrored from upstream |
| `speech_encoder_q4f16.onnx_data` | ~169 MB | External weights | Mirrored from upstream |
| `embed_tokens_q4f16.onnx` | ~2.5 KB | Token embedding lookup | Mirrored from upstream |
| `embed_tokens_q4f16.onnx_data` | ~32 MB | External weights | Mirrored from upstream |
| `language_model_q4f16.onnx` | ~0.8 MB | Autoregressive LM with LoRA input ports (see above) | Custom (v2.0.0) |
| `language_model_q4f16.onnx_data` | ~222 MB | External weights | Custom (v2.0.0) |
| `conditional_decoder_q4f16.onnx` | ~2.3 MB | Speech-token → waveform vocoder | Mirrored from upstream |
| `conditional_decoder_q4f16.onnx_data` | ~155 MB | External weights | Mirrored from upstream |

Total: ~581 MB.

## Quantization details

- **Format:** q4f16 (4-bit asymmetric weights + fp16 scales)
- **Block size:** 32 (matches Chatterbox's original 1.0.0 layout)
- **Quantizer:** `onnxruntime.quantization.MatMulNBitsQuantizer`
- **Excluded from quantization** (language_model only): all 192 LoRA-path
  `MatMul`/`Add` nodes (kept fp32 so adapter weights can be hot-swapped
  without retraining scales)
- The 96 GPT-2 projection layers in `language_model` are first rewritten
  from `Gemm` → `MatMul + Add` so they fall inside the quantizer's
  coverage; without this step the file would stay at ~1.2 GB.

Embedding tables (`wte`, `wpe`) remain fp32 because they are read via
`Gather`, not `MatMul`, and `MatMulNBitsQuantizer` does not touch them.

The three mirrored subgraphs (`speech_encoder`, `embed_tokens`,
`conditional_decoder`) use whatever quantization the upstream ResembleAI
build applied. They are bit-for-bit identical to
`ResembleAI/chatterbox-turbo-ONNX/onnx/*_q4f16.onnx{,_data}` — you can
verify with sha256 against the upstream repo.

## License

MIT, inherited from the upstream Chatterbox project.