---
license: apache-2.0
base_model: Qwen/Qwen2.5-Coder-3B-Instruct
tags:
- qwen
- qwen2.5
- code
- coding-agent
- lora
- qlora
- 4bit
- software-engineering
- swe
- tool-use
- transformers
- peft
language:
- en
pipeline_tag: text-generation
library_name: transformers
datasets:
- suhas9545/Multi_Turn_SWE_dataset
---

# Qwen2.5-3B-SWE-Agent-QLoRA

A QLoRA adapter trained on top of Qwen2.5-Coder-3B-Instruct for software engineering agent workflows, repository reasoning, and structured tool-based coding tasks.

This adapter is optimized for:

- multi-step repository reasoning
- debugging workflows
- codebase navigation
- structured tool generation
- autonomous coding agents
- SWE-agent style trajectories
- JSON-based tool planning

---

# Base Model

- Qwen/Qwen2.5-Coder-3B-Instruct

Base model link:

https://huggingface.co/Qwen/Qwen2.5-Coder-3B-Instruct

---

# Training Dataset

Trained on:

- suhas9545/Multi_Turn_SWE_dataset

Dataset link:

https://huggingface.co/datasets/suhas9545/Multi_Turn_SWE_dataset

The dataset contains structured multi-turn software engineering trajectories derived from SWE-agent style repository interactions and tool-use workflows.

---

# Quantization & Training

This adapter was trained using QLoRA with:

- 4-bit NF4 quantization
- PEFT LoRA adapters
- bitsandbytes
- Transformers

Recommended inference dtype:

- float16
- bfloat16

---

# Intended Use

Recommended for:

- coding assistants
- SWE-agents
- autonomous debugging systems
- repository interaction agents
- tool-calling agents
- structured JSON generation
- software engineering research

---

# Example Usage

```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel

BASE_MODEL = "Qwen/Qwen2.5-Coder-3B-Instruct"
ADAPTER = "YOUR_USERNAME/Qwen2.5-3B-SWE-Agent-QLoRA"

tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
    bnb_4bit_compute_dtype=torch.float16,
)

model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    device_map="auto",
    quantization_config=bnb_config,
)

model = PeftModel.from_pretrained(model, ADAPTER)

prompt = "Fix failing tests in a Python repository."

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=512,
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))
````

---

# Prompting Style

The model performs best with concise task-oriented prompts.

Examples:

```text
Fix failing tests in the repository.
```

```text
Create a JSON tool plan to debug the issue.
```

```text
Analyze the codebase and modify the failing function.
```

---

# Limitations

* Generated commands and patches should be reviewed before execution.
* The model may hallucinate repository structure or tool outputs.
* Performance depends heavily on prompt quality and inference settings.
* Optimized primarily for coding and SWE-agent style tasks rather than general conversation.

---

# Citation


```text
@article{baumann2026swechat,
  title={SWE-chat: Coding Agent Interactions From Real Users in the Wild},
  author={Baumann, Joachim and Padmakumar, Vishakh and Li, Xiang and Yang, John and Yang, Diyi and Koyejo, Sanmi},
  year={2026},
  journal={arXiv preprint arXiv:2604.20779},
  url={https://arxiv.org/abs/2604.20779}
}
```

```
```