Rreitsma commited on
Commit
7898562
·
verified ·
1 Parent(s): 67a9a59

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +88 -0
README.md ADDED
@@ -0,0 +1,88 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ tags:
4
+ - diffusers
5
+ - modular-diffusers
6
+ - custom-blocks
7
+ - minimax-music3
8
+ - text-to-audio
9
+ - music-generation
10
+ ---
11
+
12
+ # MiniMax Music 3 — Ensemble Blocks
13
+
14
+ **Render K variations of one prompt in a single batched pass — the marginal
15
+ variation is nearly free.**
16
+
17
+ The MiniMax Music 3 autoregressive stage reads the full 8B language model plus
18
+ seven 0.6B depth-decoder passes (~23GB of weights) for *every* audio frame at
19
+ 25fps: it is memory-bandwidth bound, and that read costs the same whether it
20
+ serves one variation or four. These blocks decode K variations in lockstep
21
+ (batch 2K rows), sharing the dominant cost. The flow-matching stage batches
22
+ across variations too (grouped by exact frame count — no padding).
23
+
24
+ Measured with THIS block on an RTX 4090 (24GB, bf16, diffusers main, warm,
25
+ 15s songs, same prompt/seeds):
26
+
27
+ | variations | total | per variation |
28
+ |---|---|---|
29
+ | 1 | 28.9s | 28.9s |
30
+ | 2 | 34.9s | 17.5s (1.65x) |
31
+ | 3 | 41.5s | **13.8s (2.1x)** |
32
+
33
+ Each additional variation costs ~6s on a ~29s base — the marginal take is
34
+ ~21% of a solo render. Discussion:
35
+ [diffusers#14486](https://github.com/huggingface/diffusers/issues/14486).
36
+
37
+ ## Usage
38
+
39
+ ```python
40
+ import torch
41
+ from diffusers.modular_pipelines import ModularPipelineBlocks, ComponentsManager
42
+
43
+ blocks = ModularPipelineBlocks.from_pretrained(
44
+ "Rreitsma/minimax-music3-ensemble-blocks", trust_remote_code=True
45
+ )
46
+ manager = ComponentsManager()
47
+ manager.enable_auto_cpu_offload(device="cuda")
48
+ pipe = blocks.init_pipeline("MiniMaxAI/MiniMax-Music3", components_manager=manager)
49
+ pipe.load_components(dtype=torch.bfloat16)
50
+
51
+ out = pipe(
52
+ prompt="Genre: acoustic pop. BPM: 96. Key: C major. Warm female vocals, fingerpicked guitar.",
53
+ lyrics="[verse]\nMorning light filtering through the pine\n[chorus]\nSoftly the world begins to breathe",
54
+ audio_duration=60.0,
55
+ num_variations=3,
56
+ seed=7, # variation i uses seed + i; omit for random seeds
57
+ output="audios",
58
+ )
59
+ # out: list of float32 stereo waveforms, one per variation, 44.1kHz, (channels, samples)
60
+ ```
61
+
62
+ ## Notes & honest caveats
63
+
64
+ - **Quality:** every variation's math is row-independent and draws from its
65
+ own seeded generator using the reference sampling recipe — identical in
66
+ distribution to solo generation. Batched kernels differ from solo kernels at
67
+ the floating-point-ulp level, so a given seed may take a different (equally
68
+ valid) trajectory than it would solo.
69
+ - **Guidance** is standard CFG with the checkpoint's guidance scale. For other
70
+ guidance techniques, use the default MiniMax Music 3 blocks (with the
71
+ guider) instead.
72
+ - **VRAM** scales with `duration x num_variations` (KV cache). On 24GB:
73
+ 3 variations up to ~1 minute is comfortable; use fewer variations for longer
74
+ songs. An early EOS in one variation freezes its rows at no cost.
75
+ - Deliberately **plain eager PyTorch** — no torch.compile, no extra
76
+ dependencies — so it runs wherever diffusers runs. A further-optimized local
77
+ studio (compiled AR decode, ~2.9x total on a 4090) lives at
78
+ [minimax-music3-studio](https://github.com/TheDutchRuler/minimax-music3-studio).
79
+
80
+ ## License & credits
81
+
82
+ Apache-2.0 (portions derived from the diffusers MiniMax Music 3 modular
83
+ pipeline, Copyright 2026 The MiniMax Team and The HuggingFace Team). Model
84
+ weights are MiniMax's, CC BY 4.0, fetched separately from
85
+ [MiniMaxAI/MiniMax-Music3](https://huggingface.co/MiniMaxAI/MiniMax-Music3).
86
+
87
+ Built with Claude (Fable 5 Max) following a request from the diffusers
88
+ maintainers in [#14486](https://github.com/huggingface/diffusers/issues/14486).