| --- |
| license: apache-2.0 |
| base_model: Qwen/Qwen2.5-Coder-3B-Instruct |
| tags: |
| - qwen |
| - qwen2.5 |
| - code |
| - coding-agent |
| - lora |
| - qlora |
| - 4bit |
| - software-engineering |
| - swe |
| - tool-use |
| - transformers |
| - peft |
| language: |
| - en |
| pipeline_tag: text-generation |
| library_name: transformers |
| datasets: |
| - suhas9545/Multi_Turn_SWE_dataset |
| --- |
| |
| # Qwen2.5-3B-SWE-Agent-QLoRA |
|
|
| A QLoRA adapter trained on top of Qwen2.5-Coder-3B-Instruct for software engineering agent workflows, repository reasoning, and structured tool-based coding tasks. |
|
|
| This adapter is optimized for: |
|
|
| - multi-step repository reasoning |
| - debugging workflows |
| - codebase navigation |
| - structured tool generation |
| - autonomous coding agents |
| - SWE-agent style trajectories |
| - JSON-based tool planning |
|
|
| --- |
|
|
| # Base Model |
|
|
| - Qwen/Qwen2.5-Coder-3B-Instruct |
|
|
| Base model link: |
|
|
| https://huggingface.co/Qwen/Qwen2.5-Coder-3B-Instruct |
|
|
| --- |
|
|
| # Training Dataset |
|
|
| Trained on: |
|
|
| - suhas9545/Multi_Turn_SWE_dataset |
| |
| Dataset link: |
| |
| https://huggingface.co/datasets/suhas9545/Multi_Turn_SWE_dataset |
|
|
| The dataset contains structured multi-turn software engineering trajectories derived from SWE-agent style repository interactions and tool-use workflows. |
|
|
| --- |
|
|
| # Quantization & Training |
|
|
| This adapter was trained using QLoRA with: |
|
|
| - 4-bit NF4 quantization |
| - PEFT LoRA adapters |
| - bitsandbytes |
| - Transformers |
|
|
| Recommended inference dtype: |
|
|
| - float16 |
| - bfloat16 |
|
|
| --- |
|
|
| # Intended Use |
|
|
| Recommended for: |
|
|
| - coding assistants |
| - SWE-agents |
| - autonomous debugging systems |
| - repository interaction agents |
| - tool-calling agents |
| - structured JSON generation |
| - software engineering research |
|
|
| --- |
|
|
| # Example Usage |
|
|
| ```python |
| import torch |
| from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig |
| from peft import PeftModel |
| |
| BASE_MODEL = "Qwen/Qwen2.5-Coder-3B-Instruct" |
| ADAPTER = "YOUR_USERNAME/Qwen2.5-3B-SWE-Agent-QLoRA" |
| |
| tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL) |
| |
| bnb_config = BitsAndBytesConfig( |
| load_in_4bit=True, |
| bnb_4bit_quant_type="nf4", |
| bnb_4bit_use_double_quant=True, |
| bnb_4bit_compute_dtype=torch.float16, |
| ) |
| |
| model = AutoModelForCausalLM.from_pretrained( |
| BASE_MODEL, |
| device_map="auto", |
| quantization_config=bnb_config, |
| ) |
| |
| model = PeftModel.from_pretrained(model, ADAPTER) |
| |
| prompt = "Fix failing tests in a Python repository." |
| |
| inputs = tokenizer(prompt, return_tensors="pt").to(model.device) |
| |
| outputs = model.generate( |
| **inputs, |
| max_new_tokens=512, |
| ) |
| |
| print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
| ```` |
|
|
| --- |
|
|
| # Prompting Style |
|
|
| The model performs best with concise task-oriented prompts. |
|
|
| Examples: |
|
|
| ```text |
| Fix failing tests in the repository. |
| ``` |
|
|
| ```text |
| Create a JSON tool plan to debug the issue. |
| ``` |
|
|
| ```text |
| Analyze the codebase and modify the failing function. |
| ``` |
|
|
| --- |
|
|
| # Limitations |
|
|
| * Generated commands and patches should be reviewed before execution. |
| * The model may hallucinate repository structure or tool outputs. |
| * Performance depends heavily on prompt quality and inference settings. |
| * Optimized primarily for coding and SWE-agent style tasks rather than general conversation. |
|
|
| --- |
|
|
| # Citation |
|
|
|
|
| ```text |
| @article{baumann2026swechat, |
| title={SWE-chat: Coding Agent Interactions From Real Users in the Wild}, |
| author={Baumann, Joachim and Padmakumar, Vishakh and Li, Xiang and Yang, John and Yang, Diyi and Koyejo, Sanmi}, |
| year={2026}, |
| journal={arXiv preprint arXiv:2604.20779}, |
| url={https://arxiv.org/abs/2604.20779} |
| } |
| ``` |
|
|
| ``` |
| ``` |
