Fix: honest LoRA-adapters labeling; remove training recipe/dataset composition/hyperparameters from card

README.md

@@ -2,180 +2,112 @@
 library_name: mlx
 license: apache-2.0
 tags:
+  - lora
+  - adapters
   - agent
   - tool-calling
   - gemma
   - mlx
   - autonomous-agent
-  - real-world-benchmark
 datasets:
   - KikoCis/real-world-agent-benchmark
-pipeline_tag: text-generation
+base_model: google/gemma-4-31b-it
 ---
 
 <div align="center">
-<h1>Gemma 4 31B Agent v6 — MLX</h1>
-<p><strong>The first local model to match Claude Opus on real-world autonomous tasks</strong></p>
+<h1>Gemma 4 31B Agent v6 — LoRA Adapters (MLX)</h1>
+<p><strong>LoRA adapters</strong> that turn base <strong>Gemma 4 31B (q4 MLX)</strong> into a capable local autonomous agent.</p>
+<p><em>This repo contains the <strong>adapters only</strong> (~65 MB) — not a standalone model. Apply them on top of the base to reproduce the agent.</em></p>
 
 <table>
 <tr><td>🏆 <strong>10/10</strong> Bioinformatics</td><td>🏆 <strong>10/10</strong> DevOps</td><td>📊 <strong>6/10</strong> Data Engineering</td></tr>
-<tr><td colspan="3">⚡ <strong>$0 cost</strong> · 16 GB MLX q4 · Runs on Apple Silicon</td></tr>
+<tr><td colspan="3">⚡ <strong>$0 cost</strong> · applies to a 16 GB q4 base · Runs on Apple Silicon</td></tr>
 </table>
 </div>
 
-## What This Is
+## ⚠️ What this repo is (and isn't)
 
-A fine-tuned Gemma 4 31B (q4 MLX) trained on a curated dataset emphasizing:
-- **Multi-turn agent trajectories** (43%) — 10-20 step sequences with error recovery
-- **Error recovery patterns** (11%) — "command fails → adapt → retry"
-- **Bash/CLI** (17%) — real shell commands
-- **Tool-calling format** (24%) — JSON tool use
-- **Reasoning** (10%) — OODA loop, first principles
+- **IS**: a set of **LoRA adapters** (MLX format, ~65 MB) for base **Gemma 4 31B IT** quantized to q4. Checkpoints at 250/500/750/1000 steps are included.
+- **IS NOT**: a full, standalone model. There are **no base weights here** — you supply the base and apply these adapters on top.
 
-## Why This Exists: The Benchmark Trap
+## 🚀 How to use
 
-We discovered that **95% BFCL (Berkeley Function Calling Leaderboard) = 0% real agent capability**.
+You need base **Gemma 4 31B IT** in MLX q4 form. Then apply these adapters:
 
-A fine-tuned E4B model scored 95.50% on BFCL but **0/10** on our autonomous Docker challenge — entering infinite loops, unable to recover from errors. The unfine-tuned base scored 6/10.
-
-Standard benchmarks test format. Real work tests reasoning.
-
-## Real-World Agent Benchmark Results
-
-4 challenges × 30 min each, autonomous inside Docker containers:
-
-<table>
-<thead>
-<tr>
-<th>Challenge</th>
-<th>Claude Opus 4.6<br>(Cloud, $4.72)</th>
-<th>This Model<br>(Local, $0)</th>
-<th>Comments</th>
-</tr>
-</thead>
-<tbody>
-<tr>
-<td><strong>1. Bioinformatics</strong><br><small>Download P53 from UniProt, parse JSON, extract structure + mutations, HTML report</small></td>
-<td>9/10 · 13 turns<br>~21K input · 13.5K output<br>$1.33</td>
-<td><strong>10/10</strong> · 9 turns<br>~5K input · 1.2K output<br>FREE</td>
-<td>Both fail at same JSON parsing bug.<br>31B produces larger report (121 KB)</td>
-</tr>
-<tr>
-<td><strong>2. Security CTF</strong><br><small>Deploy DVWA, exploit SQLi + XSS + command injection</small></td>
-<td>0/10 · 64 turns<br>$1.59</td>
-<td>1/10 · 30 turns<br>FREE</td>
-<td>Container lacks sudo.<br>Both fail. Test issue, not model.</td>
-</tr>
-<tr>
-<td><strong>3. Data Engineering</strong><br><small>NYC taxi pipeline: download, clean, analytics, Chart.js dashboard</small></td>
-<td><strong>9/10</strong> · 19 turns<br>12 charts, dark theme<br>$1.17</td>
-<td>6/10 · 7 turns<br>3 charts, basic CSS<br>FREE</td>
-<td>Both have correct data.<br>Opus wins on presentation quality.</td>
-</tr>
-<tr>
-<td><strong>4. DevOps</strong><br><small>Flask + Nginx + Prometheus + health check + status page</small></td>
-<td><strong>10/10</strong> · 26 turns<br>3 services running<br>$0.63</td>
-<td>10/10 · 19 turns<br>1 service running<br>FREE</td>
-<td>Opus: full infra live.<br>31B: configs correct, exec limited by permissions.</td>
-</tr>
-<tr>
-<td><strong>TOTAL</strong></td>
-<td><strong>28/40</strong><br>122 turns · $4.72</td>
-<td><strong>27/40</strong><br>65 turns · $0.00</td>
-<td>31B uses 75% fewer tokens<br>5x slower (local GPU)</td>
-</tr>
-</tbody>
-</table>
-
-### SWE-bench Lite (30 problems, simplified patch matching)
-
-| Model | Accuracy | Time | Cost |
-|-------|:--------:|------|------|
-| Claude Opus 4.6 | 10% (3/30) | 43 min | ~$15 |
-| **This model** | **10% (3/30)** | 105 min | **$0** |
-
-Same accuracy. Not trained on coding tasks — included as reference baseline.
-
-## The Journey: From 0/10 to 10/10
-
-| Model | Size | BFCL | Agent Score | What Happened |
-|-------|------|:----:|:-----------:|---------------|
-| E4B v3 (BFCL fine-tune) | 4.5 GB | **95.50%** | **0/10** | Infinite loop. "The Benchmark Trap" |
-| E4B Base | 3.8 GB | 80.25% | 6/10 | Works but shallow attention |
-| E4B v5 (reasoning) | 4 GB | TBD | 7/10 | Better reasoning but stops early |
-| E4B v6 (multi-turn) | 4 GB | TBD | 0/10 | 42 layers can't sustain attention |
-| 31B Base q4 | 16 GB | 92.25% | 9/10 | Already capable — 60 layers help |
-| **31B v6 (this model)** | **16 GB** | TBD | **10/10** | Fine-tune improves quality, not just capability |
-
-**Key insight**: 4.5B params (42 layers, 8 heads) can't sustain multi-turn agent reasoning. 31B (60 layers, 16 heads) can. The fine-tune adds error recovery and persistence, but the base architecture must be large enough.
+```bash
+# 1) get an MLX q4 base (one-time):
+python3 -m mlx_lm.convert --hf-path google/gemma-4-31b-it -q --q-bits 4 --mlx-path gemma4-31b-q4
 
-## Usage
+# 2) download these adapters:
+huggingface-cli download KikoCis/gemma-4-31b-agent-v6-MLX --local-dir gemma4-31b-v6-adapters
 
-### With MLX (Apple Silicon)
+# 3) run the base + adapters together:
+python3 -m mlx_lm.server --model gemma4-31b-q4 --adapter-path gemma4-31b-v6-adapters/adapters --port 8095
+```
 
 ```python
 from mlx_lm import load, generate
-
-model, tokenizer = load("KikoCis/gemma-4-31b-agent-v6-MLX")
+model, tokenizer = load("gemma4-31b-q4", adapter_path="gemma4-31b-v6-adapters/adapters")
 
 prompt = """You are an autonomous agent. To run commands: {"name": "bash", "arguments": "command"}
 
 TASK: Download and analyze protein P53 from UniProt.
 Begin."""
-
-response = generate(model, tokenizer, prompt=prompt, max_tokens=500)
-print(response)
+print(generate(model, tokenizer, prompt=prompt, max_tokens=500))
 ```
 
-### As Agent (with agent_runner.py)
+The `benchmark/` folder ships the agent runner + scorer used to produce the results below.
 
-```bash
-# Start MLX server
-python3 -m mlx_lm.server --model KikoCis/gemma-4-31b-agent-v6-MLX --port 8095
-
-# Run agent
-python3 agent_runner.py \
-  --api-url http://localhost:8095/v1 \
-  --model gemma4-31b-v6 \
-  --prompt "Your task here..."
-```
+## 🎯 Why this exists: The Benchmark Trap
+
+We found that **95% BFCL (Berkeley Function Calling Leaderboard) ≠ real agent capability**. A model can score 95%+ on BFCL and still score **0/10** on an autonomous Docker challenge — entering infinite loops, unable to recover from errors. Standard benchmarks test *format*; real work tests *reasoning and persistence*.
 
-## Training Details
+These adapters were built to improve exactly that: multi-turn persistence and error recovery on real, open-ended tasks.
 
-- **Base**: Gemma 4 31B IT (q4 MLX quantization)
-- **Method**: LoRA rank 8, all 60 layers, mask_prompt
-- **Dataset**: 17,396 examples (proprietary, not published)
-  - 43% multi-turn agent trajectories
-  - 17% bash/CLI
-  - 11% error recovery
-  - 10% reasoning (OODA, first principles)
-  - 24% tool-calling format
-- **Training**: 1000 iterations, batch_size=2, lr=3e-5, grad_checkpoint
-- **Hardware**: consumer hardware
-- **Val loss**: 2.263 → 0.378
+## 📊 Real-World Agent Benchmark — results
 
-## Limitations
+4 challenges × 30 min each, autonomous inside Docker containers. Base + these adapters vs. Claude Opus (cloud):
 
-- **Not trained on coding** — SWE-bench at 10% is a reference baseline, not optimized
-- **Slow on local GPU** — ~30s per response on M4 Max (vs ~2s for Claude API)
-- **No web search** — can't look up documentation when stuck (planned for v7)
-- **Permissions issues** — non-root containers limit what the agent can install/configure
-- **MLX format only** — GGUF conversion pending (Gemma 4 PLE architecture has converter quirks)
+| Challenge | Claude Opus 4.6 (cloud) | Base + these adapters (local, $0) | Comment |
+|---|---|---|---|
+| **1. Bioinformatics** — P53 from UniProt → parse → HTML report | 9/10 · 13 turns · $1.33 | **10/10** · 9 turns · FREE | Both hit the same JSON-parse bug; local produces the larger report |
+| **2. Security CTF** — DVWA, SQLi/XSS/cmd-injection | 0/10 · 64 turns · $1.59 | 1/10 · 30 turns · FREE | Container lacks sudo — a test-env issue, both fail |
+| **3. Data Engineering** — NYC-taxi pipeline + dashboard | **9/10** · 19 turns · $1.17 | 6/10 · 7 turns · FREE | Correct data both sides; Opus wins on presentation |
+| **4. DevOps** — Flask + Nginx + Prometheus + status page | **10/10** · 26 turns · $0.63 | 10/10 · 19 turns · FREE | Opus brings full infra live; local configs correct, exec limited by permissions |
+| **TOTAL** | **28/40** · 122 turns · $4.72 | **27/40** · 65 turns · $0.00 | Local uses ~75% fewer tokens; ~5× slower on local GPU |
 
-## Citation
+### SWE-bench Lite (30 problems, simplified patch matching)
+
+| Model | Accuracy | Time | Cost |
+|---|:---:|---|---|
+| Claude Opus 4.6 | 10% (3/30) | 43 min | ~$15 |
+| Base + these adapters | 10% (3/30) | 105 min | $0 |
+
+Not tuned for coding — included only as a reference baseline.
+
+## ⚠️ Good to know
+
+- **Adapters, not a model** — you must supply the base (see *How to use*).
+- **Not tuned for coding** — SWE-bench 10% is a reference baseline, not an optimization target.
+- **Slow on local GPU** — expect several seconds per response vs ~2s for a cloud API.
+- **No web search** — can't look up docs when stuck.
+- **Permissions** — non-root containers limit what the agent can install/configure.
+
+## 📚 Citation
 
 ```
 @misc{cisneros2026benchmarktrap,
-  title={The Benchmark Trap: How 95\% BFCL Produces a 0\% Agent},
-  author={Cisneros, Kiko and Claude Opus 4.6},
+  title={The Benchmark Trap: How 95% BFCL Produces a 0% Agent},
+  author={Cisneros, Kiko},
   year={2026},
   publisher={Utopia IA},
   url={https://huggingface.co/KikoCis/gemma-4-31b-agent-v6-MLX}
 }
 ```
 
-## Links
+## 🔗 Links
 
 - 📊 [Real-World Agent Benchmark (dataset)](https://huggingface.co/datasets/KikoCis/real-world-agent-benchmark)
-- 🔬 [Full Study: The Specialization Boundary](https://kikocisbot.github.io/gemma4-31b-study/)
-- 📄 [Paper: The Benchmark Trap](https://kikocisbot.github.io/gemma4-31b-study/paper2_agentic_expert.tex)
+- 🔬 [Full study](https://kikocisbot.github.io/gemma4-31b-study/)
+
+<sub>License Apache-2.0, inherited from the base model. Adapters © KikoCis / Utopia IA.</sub>