# Headroom Eval Report — MiniMax M3

**Date:** 2026-06-15
**Model:** `minimaxai/minimax-m3` (NVIDIA endpoint)
**Proxy:** Headroom local build (`/v1/compress`, Rust core enabled)
**Scoring (LCB criteria):** pass = no regression, big win = headroom solves baseline fail, fail = regression

---

## 1. LiveCodeBench (n=20)

Two runs: with `compress_user_messages=True` vs `False`.

| Run | No-regress | Regress | Avg comp | Headroom solves |
|---|---|---|---|---|
| User-msg compress ON | 16/20 | 4 | 38.8% | 1 |
| User-msg compress OFF | 17/20 | 3 | 0% | 5 |

**Takeaway:** User-message compression hurt codegen on minimax-m3 — baseline solved 5 vs headroom 1 in run 1. Compressing problem statements drops signal the model needs. Disabled in LCB runner.

3/20 noise floor (regressions at 0% compression = stochastic baseline variance).

---

## 2. Long Context (n=40, narrativeqa + long_context_qa)

Two runs in `ground_truth` mode: true baseline (`--no-compress`) vs headroom proxy.

| Metric | Baseline | Headroom | Delta |
|---|---|---|---|
| Total passed | 5/40 (12.5%) | 7/40 (17.5%) | +2 (+5.0pp) |
| NarrativeQA pass | 1/20 (5.0%) | 1/20 (5.0%) | +0 |
| LongContextQA pass | 4/20 (20.0%) | 6/20 (30.0%) | +2 |
| Tokens used | 62,390 | 48,650 | -13,740 (-22%) |

**Compression by dataset:**
- NarrativeQA: 17.6% avg, 2,680 tokens saved
- LongContextQA: 23.5% avg, 11,060 tokens saved

### Regression status (per LCB criteria)

- **Regressions: 0** (no baseline pass lost)
- **Big wins: 2** (baseline failed, headroom solved)
- **No regressions introduced by compression**

### Why headroom wins here

minimax-m3 chokes on long noisy context. Headroom strips noise → model focuses on relevant signal → accuracy goes up, not down. LongContextQA improves from 20% → 30%.

---

## 3. Conclusion

- **Long context: modest win.** 22% token savings + 5.0pp accuracy gain. No regressions.
- **LCB codegen: mixed.** Token savings real (38.8%) but 1 real regression traced to user-message compression. Disabling that flag = noise floor only.
- **Recommendation:** Enable headroom for long-context / tool-output / RAG workloads. Keep `compress_user_messages=False` for codegen prompts where the problem statement must stay verbatim.

---

## Artifacts

- `evals/lcb-minimax-m3/` — LCB run 1 (user-msg compress ON)
- `evals/lcb-minimax-m3-no-compress/` — LCB run 2 (user-msg compress OFF)
- `evals/long-context-minimax-m3/` — headroom proxy run
- `evals/long-context-minimax-m3-baseline/` — true baseline (`--no-compress`)