---
license: other
license_name: minimax-music3-terms
license_link: https://huggingface.co/MiniMaxAI/MiniMax-Music3
tags:
- minimax-music3
- composer-lm
- depth-pruning
- distillation
---

# MiniMax Music3 composer LM — 5.7B depth-pruned distilled students (LR ablation pair)

The Music3 composer (Qwen3-8B post-trained by MiniMax to emit RVQ audio codes) **depth-pruned from
36 to 21 layers (5.69B params)** and repair-distilled against the frozen teacher. Both checkpoints
here are the same recipe differing ONLY in learning rate — an A/B pair, both of which pass listening
tests (recognizable genre, structure, and vocals on held-out prompts) while composing **~2x faster
than the teacher** (single-row / CFG-free: the distillation targets are the CFG-guided
distributions, so guidance is baked in and no unconditional pass is needed).

## Listen — same prompt, same seed, same renderer (UK grime, 40 s)

Only the composer differs. Teacher = stock 8B dual-row CFG path; students compose single-row, ~2x faster.

**Teacher (8B, reference):**

<audio controls src="https://huggingface.co/Mothersuperior/minimax-music3-composer-5.7b-distilled/resolve/main/samples/teacher_8B_reference_grime.wav"></audio>

**Student 5.7B — lr 3e-5 arm:**

<audio controls src="https://huggingface.co/Mothersuperior/minimax-music3-composer-5.7b-distilled/resolve/main/samples/student_5.7B_lr-3e-5_grime.wav"></audio>

**Student 5.7B — lr 6e-5 arm:**

<audio controls src="https://huggingface.co/Mothersuperior/minimax-music3-composer-5.7b-distilled/resolve/main/samples/student_5.7B_lr-6e-5_grime.wav"></audio>

| subfolder | LR | notes |
|---|---|---|
| `lr-3e-5/` | 3e-5 | conservative anneal arm |
| `lr-6e-5/` | 6e-5 | doubled-LR arm |

Recipe: ShortGPT-style layer scoring over a 18.7k-track self-distillation corpus
([Mothersuperior/minimax-music3-rvq-distill-corpus-8k](https://huggingface.co/datasets/Mothersuperior/minimax-music3-rvq-distill-corpus-8k)),
drop 15/36 layers (first/last two protected, consecutive-run cap 3), then repair with
CE vs teacher top-50 guided distributions + teacher-normalized hidden-state MSE (final-layer,
cached fp16 targets) + END-token supervision. bf16, torch-optimi AdamW, batch 8, single RTX PRO 6000.
Hidden-state cosine to teacher ~0.94 at upload (teacher self-consistency ceiling 0.99994).

Drop-in: standard `Qwen3ForCausalLM.from_pretrained(...)`, same embeddings/width — compatible with
the stock Music3 depth decoder + DiT. Compose single-row (sample top-50 from masked logits; no CFG).
Use is subject to the MiniMax Music 3 model terms.