---
base_model: Qwen/Qwen3.5-9B
library_name: peft
pipeline_tag: text-generation
tags:
- base_model:adapter:Qwen/Qwen3.5-9B
- lora
- peft
- sft
- trl
---

# sinopia-b57f6cad

A LoRA adapter on **[Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B)**,
trained as an SFT cold start: it distils a frozen teacher's *thoughts* into the
policy, so that a later RL stage has something better than a cold model to
refine.

`sinopia` is the red underdrawing laid down beneath a fresco — the sketch the
finished painting is built over. That is what this checkpoint is.

## The prompt contract (read this first)

This model is trained to continue from **inside an already-open `<think>`
block**. The prompt must end with the assistant header followed by `<think>\n`;
the chat template will not add that for you. Completions are shaped:

```
</think>
THOUGHT: {thought}

{action}
```

so a downstream parser can recover `(thought, action)` by splitting on the
closing `</think>` and taking the last fenced ```bash block as the action. If
you render the prompt with `apply_chat_template(..., add_generation_prompt=True)`
and stop there, the model will not behave as trained.

## Usage

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

BASE = "Qwen/Qwen3.5-9B"
tok = AutoTokenizer.from_pretrained(BASE)
model = AutoModelForCausalLM.from_pretrained(
    BASE, dtype=torch.bfloat16, device_map="auto",
    attn_implementation="flash_attention_2",
)
model = PeftModel.from_pretrained(model, "iamPi/sinopia-b57f6cad")
model = model.merge_and_unload()          # optional; needed to serve with vLLM

prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
if not prompt.rstrip().endswith("<think>"):
    prompt += "<think>\n"                 # the contract above

ids = tok(prompt, return_tensors="pt", add_special_tokens=False).to(model.device)
out = model.generate(**ids, max_new_tokens=1792, do_sample=True, temperature=0.8)
print(tok.decode(out[0][ids["input_ids"].shape[1]:], skip_special_tokens=True))
```

## Adapter

| | |
|---|---|
| rank / alpha / dropout | 32 / 128 / 0.05 (α/r = 4) |
| trainable params | 80,216,064 |
| tensors | 400 |
| base | `Qwen/Qwen3.5-9B` (bf16, 8.95 B) |

### Target modules — why there are ten, not seven

Qwen3.5-9B is a **hybrid linear-attention** model: `layer_types` alternates
three `linear_attention` layers with one `full_attention`, so only **8 of 32**
layers carry `q_proj`/`k_proj`/`v_proj`/`o_proj`. The conventional LoRA target
list therefore adapts every MLP but the *mixing operator* of just a quarter of
the layers.

This adapter adds the Gated DeltaNet projections so all 32 layers are covered:

```
q_proj  k_proj  v_proj  o_proj          -> 8 full-attention layers
gate_proj  up_proj  down_proj           -> all 32 MLPs
in_proj_qkv  in_proj_z  out_proj        -> 24 linear-attention layers
```

`in_proj_a` / `in_proj_b` are deliberately excluded: they emit 32 values, so
r=32 would be full-rank on them for negligible capacity, and they gate the
recurrence's decay/beta dynamics where a perturbation destabilises more easily
than it helps. `conv1d` is depthwise, which low-rank factorisation does not fit.

## Training

| | |
|---|---|
| objective | SFT, loss on the completion only (prompt masked) |
| epochs | 1 (567 optimizer steps) |
| global batch | 32 (4×DDP × micro 2 × grad-accum 4) |
| lr / schedule | 1e-4, cosine, 3% warmup |
| precision | bf16, gradient checkpointing, flash-attention-2 |
| max length | unbounded (`padding_free`, no truncation) |

Trained on 4×A100 80GB with TRL 1.10 / transformers 5.15 / PEFT 0.20 / torch 2.13.

## Limitations

- **Not evaluated here.** This card reports how it was trained, not how well it
  scores. Treat it as a starting point for an RL stage, not a finished model.
- **One epoch.** No convergence claim is made.
- Trained on agentic shell-tool trajectories, so behaviour outside that shape —
  and outside the `<think>` contract above — is untested.

## License

The adapter carries no license of its own; it is derivative of
`Qwen/Qwen3.5-9B` and inherits that model's terms. Consult the base model card
before use.