---
license: apache-2.0
base_model: Qwen/Qwen2.5-7B-Instruct
tags:
- loop-engineering
- orchestrator
- function-calling
- kompress
- fine-tuned
- lora
- neftune
language:
- en
---

# kompress-superpower-orchestrator

LoRA + NEFTune fine-tune of Qwen2.5-7B-Instruct on 117 conversation pairs encoding all 17 kompress experiment outcomes. A loop engineering agent that designs experiments, diagnoses failures, spawns sub-agents, and decides next actions.

[Blog post →](https://pocoo.vaked.dev/posts/2026-06-26-orchestrator) · [Paper →](https://kompress.vaked.dev) · [Models →](https://huggingface.co/PeetPedro) · [LoopKit →](https://github.com/peterlodri-sec/loopkit)

## Superpowers

| Capability | Example |
|---|---|
| Experiment design | "Try GLM-5.2 as teacher" → spawns training script, estimates $0.15 |
| Failure diagnosis | heretic=0.878 → "v15 dilution pattern, reduce to 300 pairs" |
| Council decisions | Review metrics → SHIP / RETRAIN / PIVOT with reasoning |
| Sub-agent spawning | spawn_train(), spawn_eval(), spawn_label() |
| Budget tracking | Knows costs: $0.13/run, $0.15/version total |
| State keeping | Remembers all 17 versions, 11 dead ends, Pareto at λ=3/5/10 |

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-7B-Instruct",
    device_map="auto",
    torch_dtype="auto"
)
model = PeftModel.from_pretrained(base, "PeetPedro/kompress-superpower-orchestrator")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-7B-Instruct")

messages = [
    {"role": "system", "content": "You are kompress-superpower-orchestrator, a loop engineering agent with tools: check_status, spawn_train, spawn_eval, spawn_label, council_review. 17 models, v8=production (0.955), Pareto λ=3/5/10, label quality bottleneck."},
    {"role": "user", "content": "My model regressed to 0.878. 983 training pairs. What happened?"}
]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt")
outputs = model.generate(inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0]))
```

## Training

| Parameter | Value |
|---|---|
| Base model | Qwen/Qwen2.5-7B-Instruct |
| Method | LoRA (r=16, alpha=32) + NEFTune (α=5) |
| Quantization | 4-bit NF4 (BitsAndBytes) |
| Trainable params | 40M / 7.6B (0.53%) |
| Data | 117 pairs (diagnosis, planning, council, spawn, multi-turn) |
| Epochs | 3 |
| Hardware | RTX 4090 24GB |
| Cost | ~$0.30 |

DoRA was attempted but OOM'd on 24GB — needs A100+. NEFTune (noisy embeddings) improves chat naturalness at zero memory cost.


## CONCLUSION

LoRA + NEFTune on 117 pairs encoding 17 experiments. First model to encode the entire loop engineering decision history.

## USECASE

Use as a loop engineering assistant. Ask about experiment design, failure diagnosis, or council decisions.

## Series

This is the 20th model on [PeetPedro](https://huggingface.co/PeetPedro). See also:

| Model | Type | Heretic |
|---|---|---|
| [kompress-v8](https://huggingface.co/PeetPedro/kompress-v8) | Compression (production) | 0.955 |
| [kompress-v16](https://huggingface.co/PeetPedro/kompress-v16) | Pareto endpoint | 0.972 |
| **orchestrator** | **Loop engineering agent** | — |

[Full story →](https://pocoo.vaked.dev/posts/2026-06-26-orchestrator) · [All experiments →](https://pocoo.vaked.dev/posts/2026-06-25-kompress-heretic-eval) · [Interactive paper →](https://kompress.vaked.dev)