The piece MiniMax Music 3 was published without — audio in, as ONNX
MiniMax Music 3 writes music from a description and lyrics. What it cannot do out of the box is read music: the model was released without the encoder that turns audio into the codes it generates, so a finished track cannot be handed back to it.
That one missing piece is the difference between a slot machine and an editor. Without it, a chorus you dislike means generating the whole song again and losing everything you liked. With it:
- continue a track — give it forty seconds, get the rest, with the opening untouched
- replace a section — regenerate only the chorus; everything around it stays byte for byte
- write an intro in front of a piece that already exists
- re-render a song through the model — the same song, a different performance
It is not a style or voice reference. Identity is decided upstream in the language model; handing it audio says "work with this material", not "imitate this artist".
What this repository is
The encoder was reconstructed by the community and published as PyTorch weights. This is the 169M v4 checkpoint exported to ONNX, so it runs anywhere ONNX Runtime runs — on the card or the processor, with no Python and no PyTorch.
Verified against the PyTorch reference on random input: every one of the eight codes per frame identical, not merely close.
| Source weights | SimpleTuner/open-rvq-encoder-minimax-music3-169m-v4, final/ |
| Parameters | 169M, f32, 676 MB |
| Exported by | scripts/export-rvq-encoder-onnx.py |
| Runs in | MiniMax Music3 Studio |
Inputs and outputs
| name | shape | meaning |
|---|---|---|
latents |
[1, T, 128] |
DAC-VAE latents at 86.13 Hz — what neural-codec writes from a WAV or MP3 |
pool |
[1, F, T] |
pooling matrix onto the model's 25 Hz frame grid |
codes |
[1, F, 8] |
per frame: one semantic code of 16384, then seven acoustic codes of 1024 |
Those eight codes per frame are exactly what the engine accepts as
audio_codes, which skips the autoregressive stage and renders from them.
The whole path
track.mp3
-> neural-codec (audio -> VAE latents, 86.13 Hz) [in minimaxmusic.cpp]
-> this model (latents -> 8 codes per frame) [ONNX Runtime]
-> mm-server audio_codes (codes -> audio) [in minimaxmusic.cpp]
Architecture
Read from the checkpoint, not assumed: conv_in (kernel 7, 128 → 1088), three
residual blocks with dilations 1/3/9, pooling onto the 25 Hz grid, eight
transformer layers (17 heads, d_model 1088, muP attention scale 8.0), a
semantic head over 16384 entries, then a two-layer depth decoder (d 512) that
produces the seven acoustic codes in a greedy chain.
Credit
The reconstruction is the community's work — SimpleTuner for the weights and the reference implementation, and the discussion around MiniMaxAI/MiniMax-Music3 for working out that it was possible at all. This repository only converts it to a format that runs without PyTorch.