---
license: apache-2.0
language:
- en
base_model:
- empero-ai/Qwen3.5-9B-Claude-Opus-4.6-Distill
library_name: transformers
datasets:
- peteromallet/dataclaw-peteromallet
- woctordho/dataclaw
- zhiyaowang/dataclaw-zhiyaowang
- parani01/dataclaw-parani01
- vaynelee/dataclaw-vaynelee
- Batman787/dataclaw-Batman787
- sunsun123new/dataclaw-sunsun123new
- GolienHzmsr/dataclaw-GolienHzmsr
- tillg/dataclaw-tillg
- DJTRIXUK/dataclaw-DJTRIXUK
- emperorfutures/dataclaw-code2
- woctordho/dataclaw-windows
- MRiabov/dataclaw-march-26
- peteromallet/my-personal-codex-data
- misterkerns/my-personal-claude-code-data
- REXX-NEW/my-personal-claude-code-data
- REXX-NEW/my-personal-codex-data
- xuechengjiang/my-personal-codex-data
- akenove/my-personal-codex-data
- michaelwaves/my-personal-codex-data
- introvoyz041/my-personal-codex-data
- gutenbergpbc/john-masterclass-cc
---
# CodeClawd — Qwen3.5-9B Claude Opus 4.6 Distill (SFT on DataClaw)

A supervised fine-tune of [`empero-ai/Qwen3.5-9B-Claude-Opus-4.6-Distill`](https://huggingface.co/empero-ai/Qwen3.5-9B-Claude-Opus-4.6-Distill) on real **Claude Code / Codex** agentic sessions from the DataClaw dataset collection.

The model is trained to behave like a capable coding agent: thinking through problems, calling tools, interpreting results, and responding clearly — mirroring the full agentic loop of a real Claude Code session.

---

## Model Details

| Property | Value |
|---|---|
| **Base model** | `empero-ai/Qwen3.5-9B-Claude-Opus-4.6-Distill` |
| **Fine-tune type** | SFT (LoRA, merged) |
| **LoRA rank / alpha** | r=64, alpha=128 |
| **Training data** | DataClaw (Claude Code + Codex traces) |
| **Sequence length** | 4096 tokens |
| **Quantization (train)** | 4-bit NF4 (bitsandbytes) |
| **Optimizer** | paged_adamw_8bit |
| **LR scheduler** | cosine |
| **Learning rate** | 2e-4 |

---

## Training Data

Trained on **29 DataClaw datasets** containing real agentic coding sessions. Sessions were quality-filtered to require:

- At least 4 messages per session
- At least 2 assistant turns
- At least 1 tool use
- Formatted length between 200–100,000 characters

Datasets include contributions from:
`peteromallet`, `woctordho`, `zhiyaowang`, `parani01`, `vaynelee`, `Batman787`,
`sunsun123new`, `GolienHzmsr`, `tillg`, `DJTRIXUK`, `emperorfutures`, `MRiabov`,
`misterkerns`, `REXX-NEW`, `xuechengjiang`, `akenove`, `michaelwaves`,
`introvoyz041`, `gutenbergpbc`, and others.

---

## Agentic Format

The model is trained on the full agentic loop:

```
user → <think>...</think> → <tool_call> → <tool_result> → response → repeat
```

### Tool Call Format

```xml
<tool_call>
<tool>Read</tool>
<input>
{"file_path": "/path/to/file.py"}
</input>
</tool_call>

<tool_result>
<tool>Read</tool>
<output>
... file contents ...
</output>
</tool_result>
```

### Thinking Format

The model reasons inside `<think>` tags before acting:

```xml
<think>
The user wants to refactor the auth module. I should first read the existing
implementation before suggesting changes.
</think>
```

### System Prompt

```
You are an expert AI coding assistant. You help users with software engineering
tasks including writing code, debugging, refactoring, explaining code, and more.
You think through problems carefully inside <think> tags before acting.
You have access to tools like Read, Edit, Write, Bash, Grep, Glob, and others
to interact with the user's codebase. You use tools when needed and provide
clear, concise responses. You show your reasoning process and use tools
methodically to accomplish tasks.
```

---

## Usage

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "empero-ai/Qwen3.5-9B-Claude-Code"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

messages = [
    {
        "role": "system",
        "content": (
            "You are an expert AI coding assistant. You help users with software engineering "
            "tasks including writing code, debugging, refactoring, explaining code, and more. "
            "You think through problems carefully inside <think> tags before acting. "
            "You have access to tools like Read, Edit, Write, Bash, Grep, Glob, and others "
            "to interact with the user's codebase. You use tools when needed and provide "
            "clear, concise responses. You show your reasoning process and use tools "
            "methodically to accomplish tasks."
        ),
    },
    {"role": "user", "content": "Read the file main.py and explain what it does."},
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=1024,
        temperature=0.7,
        top_p=0.9,
        do_sample=True,
    )

print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
```

---

## Intended Use

- Agentic coding assistants and IDE integrations
- Tool-use / function-calling capable code models
- Research into agentic SFT from real human-AI coding sessions
- Local deployment of a capable coding agent (fits on 24GB+ GPU in fp16)

## Limitations

- Trained on traces from Claude Code / Codex sessions — tool format is XML-based, not OpenAI function-call JSON
- Quality of outputs reflects quality of DataClaw contributors' sessions
- Not RLHF'd or DPO'd — raw SFT only
- May hallucinate tool outputs if not connected to an actual tool executor

---

## License

Inherit's the base model's license. Please check [`empero-ai/Qwen3.5-9B-Claude-Opus-4.6-Distill`](https://huggingface.co/empero-ai/Qwen3.5-9B-Claude-Opus-4.6-Distill) and the underlying Qwen3.5 license before commercial use.

---

## Citation

If you use this model or the DataClaw datasets in your work, please credit the DataClaw contributors and the original Claude Code trace collection effort.