jangq commited on
Commit
81bd1ae
·
verified ·
1 Parent(s): eaacca1

Fix README: correct 228.7B params, add MLX Studio branding

Browse files
Files changed (1) hide show
  1. README.md +28 -20
README.md CHANGED
@@ -23,16 +23,16 @@ pipeline_tag: text-generation
23
 
24
  # MiniMax-M2.7 JANG_3L
25
 
26
- **MiniMax M2.7 456B MoE — 3-bit mixed precision, 89 GB**
27
 
28
- Best balance of quality and size for Apple Silicon. Fits on 128 GB+ Macs.
29
  </div>
30
 
31
  > **Recommended: Run in [MLX Studio](https://mlxstudio.com)** for best experience including thinking mode support and optimized MoE inference.
32
 
33
  ## Important Settings
34
 
35
- MiniMax M2.7 requires specific inference settings:
36
 
37
  | Setting | Value | Notes |
38
  |---------|-------|-------|
@@ -46,14 +46,16 @@ MiniMax M2.7 requires specific inference settings:
46
  | Metric | Value |
47
  |--------|-------|
48
  | Source | `MiniMaxAI/MiniMax-M2.7` (FP8 E4M3) |
49
- | Architecture | MoE (256 experts, top-8 active), GQA, partial RoPE |
 
 
50
  | Profile | JANG_3L (CRITICAL=8-bit, IMPORTANT=4-bit, COMPRESS=3-bit) |
51
  | Actual avg bits | 3.08 |
52
  | Model size | 89 GB |
53
- | Parameters | 456B total, ~46B active per token |
54
  | Format | JANG v2 (MLX-native safetensors, instant load) |
55
  | group_size | 128 (speed-optimized for 256 experts) |
56
- | Routing | Sigmoid + bias correction |
 
57
  | Context | 192K tokens |
58
 
59
  ## JANG_3L Bit Allocation
@@ -62,28 +64,33 @@ MiniMax M2.7 requires specific inference settings:
62
  |------|-----------|------|
63
  | CRITICAL | Attention (Q/K/V/O), lm_head | 8 |
64
  | IMPORTANT | Embeddings | 4 |
65
- | COMPRESS | Expert MLP (w1/w2/w3) — 98.2% of params | 3 |
66
- | Passthrough | MoE router/gate (float16), norms | 16 |
67
 
68
- ## MMLU Benchmarks
69
 
70
- *Coming soon benchmarks will be added after all profiles are converted and tested.*
71
 
72
- ## Why JANG
73
 
74
- Standard MLX quantization on MiniMax M2.5 produced **completely broken output at ALL bit levels** (~25% MMLU = random guessing). JANG's mixed-precision approach is the **only working quantized MiniMax on Apple Silicon**, achieving 74% MMLU on M2.5. M2.7 results pending.
75
 
76
- ## Other Quantizations
77
 
78
- | Model | Profile | Size | Avg Bits | Status |
79
- |-------|---------|------|----------|--------|
80
- | JANG_2L | (8, 6, 2) | 63 GB | 2.10 | Ready |
81
- | **JANG_3L** (this) | **(8, 4, 3)** | **89 GB** | **3.08** | **Ready** |
82
- | JANG_4M | (8, 4, 4) | ~120 GB | ~4.1 | Converting |
 
 
 
 
 
83
 
84
  ## Requirements
85
 
86
- - Apple Silicon Mac with 128+ GB unified memory
87
  - MLX framework
88
  - [MLX Studio](https://mlxstudio.com) recommended
89
 
@@ -101,7 +108,7 @@ prompt = tokenizer.apply_chat_template(
101
  [{"role": "user", "content": "What is photosynthesis?"}],
102
  tokenize=False, add_generation_prompt=True
103
  )
104
- output = generate(model, tokenizer, prompt=prompt, max_tokens=500, sampler=sampler)
105
  print(output)
106
  ```
107
 
@@ -116,3 +123,4 @@ Quantized by Jinho Jang (eric@jangq.ai) using JANG Tools v2.4.1.
116
  ---
117
 
118
  *This model is provided for research and personal use. Users are responsible for ensuring their use complies with applicable laws and the MiniMax license.*
 
 
23
 
24
  # MiniMax-M2.7 JANG_3L
25
 
26
+ **MiniMax M2.7 228B MoE — 3.08-bit mixed precision, 89 GB**
27
 
28
+ Best balance of quality and size fits on 128 GB+ Macs.
29
  </div>
30
 
31
  > **Recommended: Run in [MLX Studio](https://mlxstudio.com)** for best experience including thinking mode support and optimized MoE inference.
32
 
33
  ## Important Settings
34
 
35
+ MiniMax M2.7 is an always-reasoning model. It thinks before answering on every prompt.
36
 
37
  | Setting | Value | Notes |
38
  |---------|-------|-------|
 
46
  | Metric | Value |
47
  |--------|-------|
48
  | Source | `MiniMaxAI/MiniMax-M2.7` (FP8 E4M3) |
49
+ | Architecture | MoE (256 experts, top-8 active), GQA (48 heads / 8 KV), partial RoPE |
50
+ | Total Parameters | 228.7B |
51
+ | Active Parameters | ~1.4B per token |
52
  | Profile | JANG_3L (CRITICAL=8-bit, IMPORTANT=4-bit, COMPRESS=3-bit) |
53
  | Actual avg bits | 3.08 |
54
  | Model size | 89 GB |
 
55
  | Format | JANG v2 (MLX-native safetensors, instant load) |
56
  | group_size | 128 (speed-optimized for 256 experts) |
57
+ | Routing | Sigmoid + bias correction (not softmax) |
58
+ | QK-norm | Full vector RMSNorm |
59
  | Context | 192K tokens |
60
 
61
  ## JANG_3L Bit Allocation
 
64
  |------|-----------|------|
65
  | CRITICAL | Attention (Q/K/V/O), lm_head | 8 |
66
  | IMPORTANT | Embeddings | 4 |
67
+ | COMPRESS | Expert MLP (w1/w2/w3) — 98%+ of params | 3 |
68
+ | Passthrough | MoE router/gate (float16), norms, QK-norms | 16 |
69
 
70
+ JANG protects routing and attention at full precision while compressing the 256 expert MLPs — where MoE models are most tolerant of quantization. The router is kept at float16 (no quantization) for maximum routing precision.
71
 
72
+ ## MMLU Benchmarks (200q, 10 subjects, reasoning ON)
73
 
74
+ *Coming soon — benchmarks in progress.*
75
 
76
+ ## Why JANG for MiniMax
77
 
78
+ Standard MLX quantization on MiniMax produces **completely broken output at ALL bit levels** (~25% MMLU = random guessing). JANG's mixed-precision approach is the **only working quantized MiniMax on Apple Silicon**.
79
 
80
+ On M2.5, JANG_2L achieved **74% MMLU** vs MLX's 25% (random). M2.7 results pending.
81
+
82
+ ## All Quantizations
83
+
84
+ | Model | Profile | Size | Avg Bits |
85
+ |-------|---------|------|----------|
86
+ | [JANG_2L](https://huggingface.co/JANGQ-AI/MiniMax-M2.7-JANG_2L) | (8, 6, 2) | 63 GB | 2.10 |
87
+ | [JANG_3L](https://huggingface.co/JANGQ-AI/MiniMax-M2.7-JANG_3L) | (8, 4, 3) | 89 GB | 3.08 |
88
+ | [JANG_4M](https://huggingface.co/JANGQ-AI/MiniMax-M2.7-JANG_4M) | (8, 4, 4) | 115 GB | 4.06 |
89
+ | [JANG_6M](https://huggingface.co/JANGQ-AI/MiniMax-M2.7-JANG_6M) | (8, 6, 6) | 167 GB | 6.03 |
90
 
91
  ## Requirements
92
 
93
+ - Apple Silicon Mac with 128 GB unified memory
94
  - MLX framework
95
  - [MLX Studio](https://mlxstudio.com) recommended
96
 
 
108
  [{"role": "user", "content": "What is photosynthesis?"}],
109
  tokenize=False, add_generation_prompt=True
110
  )
111
+ output = generate(model, tokenizer, prompt=prompt, max_tokens=2048, sampler=sampler)
112
  print(output)
113
  ```
114
 
 
123
  ---
124
 
125
  *This model is provided for research and personal use. Users are responsible for ensuring their use complies with applicable laws and the MiniMax license.*
126
+