README.md

metadata

language:
  - en
tags:
  - coreml
  - core-ml
  - openvoice
  - voice-conversion
  - tone-conversion
  - apple
  - ios
  - macos
license: mit
library_name: coremltools
pipeline_tag: text-to-speech

OpenVoice V2 — ToneColorConverter (Core ML)

Core ML (.mlpackage) exports of the ToneColorConverter from myshell-ai/OpenVoiceV2, for on-device inference on macOS and iOS with Apple’s Core ML runtime.

This repository is private and intended for TranslateBlue / internal workflows. It does not include the PyTorch checkpoints or MeloTTS—only the converted Core ML packages and small JSON sidecars describing tensor layouts.

What’s included

Artifact	Description
ToneColorConverter_ReferenceEncoder.mlpackage	Reference mel → tone embedding g
ToneColorConverter_PosteriorEncoder.mlpackage	Base linear STFT → latent z
ToneColorConverter_Flow_Forward.mlpackage	Source tone removal (z → z_p)
ToneColorConverter_Flow_Reverse.mlpackage	Target tone injection (z_p → z_hat)
ToneColorConverter_Decoder.mlpackage	HiFi-GAN decoder (z_hat → waveform)
ToneColorConverter_E2E.mlpackage	Single-shot pipeline: STFT + sid_src / sid_tgt → waveform
pipeline_info.json	Shapes and execution order for the 5-split path
pipeline_info_e2e.json	Inputs/outputs for the E2E model

Sampling rate and STFT settings match the OpenVoice V2 converter (e.g. 22050 Hz, linear STFT layout consistent with the PyTorch reference implementation). See the JSON files for exact ranks and notes.

Intended use

• Voice / tone conversion on Apple Silicon or Apple Neural Engine–capable devices, using embeddings extracted from a short reference clip.
• E2E path: supply linear magnitude STFT y, frame count, and precomputed source/target embeddings (sid_src, sid_tgt).
• 5-split path: orchestrate the five models in order (see pipeline_info.json) when you need intermediate tensors or tighter control.

Base speech synthesis (e.g. MeloTTS) is not part of these packages; generate or load a base waveform (or spectrogram) in your app or server, then run ToneColorConverter.

Limitations

• Posterior path uses a deterministic (tau≈0 style) formulation suitable for Core ML export; behaviour may differ slightly from full stochastic sampling in PyTorch.
• You may see MILCompilerForANE / E5RT warnings on some macOS versions; inference can still fall back to CPU/GPU.
• Large .mlpackage weights are binary blobs—clone with Git LFS if you mirror this repo outside the Hub.

Download

export HF_TOKEN="***"   # read token with access to this private repo
huggingface-cli download aoiandroid/OpenVoiceV2-CoreML-TranslateBlue --local-dir ./OpenVoiceV2-CoreML-TranslateBlue

Or with Python:

from huggingface_hub import snapshot_download
snapshot_download(
    "aoiandroid/OpenVoiceV2-CoreML-TranslateBlue",
    local_dir="OpenVoiceV2-CoreML-TranslateBlue",
    token=os.environ["HF_TOKEN"],
)

Source & license

• Upstream model: myshell-ai/OpenVoiceV2 (research / demo use per upstream terms).
• OpenVoice code & weights are subject to the original MIT license (see OpenVoice LICENSE).
• This Core ML conversion is a derivative packaging for Apple platforms; retain upstream notices when redistributing.

Citation

If you use OpenVoice, cite the upstream project as in the official repository.

---

Model card maintained for the TranslateBlue OpenVoice V2 Core ML export (2026).