# Headroom Eval Report — MiniMax M2.7

**Date:** 2026-06-16
**Model:** `minimaxai/minimax-m2.7` (NVIDIA endpoint)
**Proxy:** Headroom local build (`http://localhost:1345/v1/compress`, Rust core enabled)
**Scoring (LCB criteria):** pass = no regression, big win = headroom solves baseline fail, fail = regression

---

## 1. LiveCodeBench (n=20)

Two runs: baseline (no proxy) vs headroom proxy (default config).

| Run | No-regress | Regress | Avg comp | GT solved | Notes |
|---|---|---|---|---|---|
| Baseline (no proxy) | 20/20 | 0 | 0% | 1/20 (5%) | Full 20-case run, 3086s |
| Headroom proxy | 15/20 | 5 | 0% | 3/20 (15%) | Full 20-case run, 5494s |

**Takeaway:** Proxy default config leaves user messages uncompressed on LCB prompts — same behavior as minimax-m3 run 2 (`compress_user_messages=False`). All regressions are stochastic: 5 cases where baseline PASSed but headroom FAILed (GT mismatch). No compression applied — delta is pure LLM variance.

---

## 2. Long Context (n=40, narrativeqa + long_context_qa)

Two runs in `ground_truth` mode: true baseline (`--no-compress`) vs headroom proxy.

| Metric | Baseline | Headroom | Delta |
|---|---|---|---|
| Total passed | 2/40 (5.0%) | 0/40 (0.0%) | -2 (-5.0pp) |
| NarrativeQA pass | 2/20 (10.0%) | 0/20 (0.0%) | -2 |
| LongContextQA pass | 0/20 (0.0%) | 0/20 (0.0%) | +0 |
| Tokens used | 62,390 | 48,650 | -13,740 (-22%) |
| F1 Score | 0.100 | 0.000 | -0.100 |

**Compression by dataset:**
- NarrativeQA: 17.6% avg, 2,680 tokens saved
- LongContextQA: 23.5% avg, 11,060 tokens saved

### Regression status (per LCB criteria)

- **Regressions: 2** (baseline passed 2 NarrativeQA cases; headroom failed both)
- **Big wins: 0** (no baseline failure converted to success)
- **Note:** All regressions occurred in the headroom run with 22% token compression applied. While the root cause (compression-induced signal loss vs stochastic variance) cannot be isolated without deeper analysis, compression was active during these failures.

### Why headroom loses here

Unlike minimax-m3 (which scored 12.5% baseline → 17.5% headroom on same suite), minimax-m2.7 shows marginal capability on NarrativeQA (2/20 = 10% baseline passes). Headroom lost both, suggesting compression degraded the fragile signal this model relies on. Token savings (22%) still land at the same rate as minimax-m3 because compression is model-independent.

---

## 3. Conclusion

- **Long context: regression.** 22% token savings but -5.0pp accuracy loss on NarrativeQA. Headroom degrades performance on long-context QA for this model.
- **LCB codegen: mixed.** 5 regressions (all stochastic, 0% compression), 3 GT solved vs baseline's 1. Solve rate 15% vs minimax-m3's 25%.
- **Recommendation:** Disable headroom for `minimaxai/minimax-m2.7` on NarrativeQA-style workloads until compression fidelity improves. For LCB, the proxy default leaves prompts uncompressed so no compression-related regression — the 5 observed failures are noise floor.

---

## Artifacts

- `evals/lcb-minimax-m2.7-baseline/` — LCB baseline (full 20 cases)
- `evals/lcb-minimax-m2.7/` — LCB headroom (full 20 cases)
- `evals/long-context-minimax-m2.7-baseline/` — long-context baseline (`--no-compress`)
- `evals/long-context-minimax-m2.7/` — long-context headroom proxy run