micymike/codemate-qwen3.5-2b-production-debugger

CodeMate Qwen3.5 2B Production Debugger

CodeMate Qwen3.5 2B Production Debugger is a coding-focused LoRA adapter built on top of Qwen/Qwen3.5-2B.

It is specialized for diagnosing and repairing real-world software issues across Python, JavaScript and TypeScript projects.

Intended capabilities

The model is trained to assist with:

• Python bug fixing
• JavaScript and TypeScript debugging
• Production issue diagnosis
• Minimal unified-diff generation
• Regression-test generation
• API and backend debugging
• Repository maintenance
• Edge-case identification
• Production reliability improvements

Training data

The adapter was trained using curated examples from:

nebius/SWE-rebench-V2-PRs

The source dataset contains real-world GitHub pull requests and software-engineering tasks, including:

• Issue descriptions
• Repository metadata
• Base commit references
• Human-authored patches
• Test patches
• Python, JavaScript and TypeScript projects

Training examples were filtered to remove:

• Documentation-only changes
• Generated and minified files
• Lockfile-only changes
• Empty or malformed patches
• Unsupported programming languages
• Duplicate examples
• Examples whose complete answer exceeded the training sequence limit

Gold patches were preserved without cutting off their endings.

Language distribution

Language	Target examples
Python	480
TypeScript	420
JavaScript	300
Total	1,200

Actual counts may differ slightly if the source stream did not contain enough qualifying examples for a specific language.

Training configuration

Setting	Value
Base model	Qwen/Qwen3.5-2B
Training method	4-bit QLoRA
Quantization	NF4 with double quantization
LoRA rank	32
LoRA alpha	64
LoRA dropout	0.05
Maximum training length	2,048 tokens
Epochs	1
Learning rate	5e-5
Scheduler	Cosine
Effective batch size	8
Optimizer	Paged AdamW 8-bit
Precision	FP16 with FP32 LoRA parameters
Hardware	NVIDIA Tesla T4

Training results

Metric	Result
Training examples	REPLACE_WITH_TRAIN_SIZE
Evaluation examples	REPLACE_WITH_EVAL_SIZE
Training loss	REPLACE_WITH_TRAIN_LOSS
Evaluation loss	REPLACE_WITH_EVAL_LOSS
Epochs completed	1

These loss values measure next-token prediction performance. They do not, by themselves, prove that generated patches compile or pass repository tests.

Model format

This repository contains a PEFT LoRA adapter, not standalone merged model weights.

The adapter must be loaded together with:

Qwen/Qwen3.5-2B

Installation

pip install -U "transformers>=5.2.0" peft accelerate bitsandbytes

Loading in 4-bit

import torch

from peft import PeftModel
from transformers import (
    AutoTokenizer,
    BitsAndBytesConfig,
    Qwen3_5ForCausalLM,
)

BASE_MODEL_ID = "Qwen/Qwen3.5-2B"
ADAPTER_ID = "micymike/codemate-qwen3.5-2b-production-debugger"

quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=True,
)

tokenizer = AutoTokenizer.from_pretrained(
    ADAPTER_ID,
)

base_model = Qwen3_5ForCausalLM.from_pretrained(
    BASE_MODEL_ID,
    quantization_config=quantization_config,
    device_map="auto",
    trust_remote_code=True,
)

model = PeftModel.from_pretrained(
    base_model,
    ADAPTER_ID,
)

model.eval()

Inference example

import torch

messages = [
    {
        "role": "system",
        "content": (
            "You are CodeMate, a production software debugging "
            "assistant. Diagnose the root cause, produce the smallest "
            "correct patch, preserve unrelated behavior and add "
            "regression tests when necessary."
        ),
    },
    {
        "role": "user",
        "content": """
Language: Python

Find and fix the production bug:

```python
from fastapi import FastAPI, HTTPException

app = FastAPI()
users = {1: {"name": "Michael"}}

@app.get("/users/{user_id}")
async def get_user(user_id: int):
    user = users.get(user_id)

    if user is None:
        HTTPException(
            status_code=404,
            detail="User not found"
        )

    return {"id": user_id, "name": user["name"]}

Return the root cause, minimal unified diff and regression test. """, }, ]

prompt = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True, )

inputs = tokenizer( prompt, return_tensors="pt", ).to(model.device)

with torch.inference_mode(): output_ids = model.generate( **inputs, max_new_tokens=800, do_sample=False, repetition_penalty=1.05, )

generated_ids = output_ids[ :, inputs["input_ids"].shape[1]: ]

response = tokenizer.batch_decode( generated_ids, skip_special_tokens=True, )[0]

print(response)

## Recommended prompt format

For repository tasks, provide as much relevant evidence as possible:

```text
Repository: owner/repository
Base commit: commit-sha
Language: Python | JavaScript | TypeScript

Issue:
Describe the observed behavior, expected behavior and available errors.

Relevant files:
<file path="src/example.py">
...
</file>

Failing tests:
...

Return:
1. Root-cause diagnosis
2. Minimal unified diff
3. Regression tests
4. Validation notes

Context length

The Qwen3.5 foundation supports long-context inference. However, this adapter’s supervised fine-tuning examples were limited to 2,048 tokens due to training hardware constraints.

Long-context support inherited from the base model does not guarantee equally strong debugging performance at every context length.

This release has not yet been comprehensively validated on:

• 16K repository contexts
• 32K repository contexts
• 64K repository contexts
• Full repository-scale agent workflows

The repository name and documentation should not claim verified 64K debugging performance until those evaluations are completed.

Evaluation status

Planned evaluations include:

• HumanEval+
• MBPP+
• SWE-bench-style patch generation
• Python repository debugging
• JavaScript and TypeScript repository debugging
• Unified-diff validity
• Regression-test generation
• 8K, 16K, 32K and 64K retrieval tests
• Executable patch validation

Evaluation benchmarks should remain separate from training data.

Limitations

CodeMate may:

• Generate patches that do not compile
• Misdiagnose incomplete issue reports
• Invent repository details when context is missing
• Produce syntactically valid but behaviorally incorrect changes
• Miss security, concurrency or compatibility issues
• Modify more code than necessary
• Generate incomplete regression tests

Always review and execute generated patches in an isolated development environment before production deployment.

Intended use

CodeMate is intended for:

• Software-engineering research
• Developer assistance
• Debugging experiments
• Patch-generation evaluation
• Educational coding workflows

It should not replace human code review, automated tests, security analysis or production deployment checks.

License

The adapter is released under the Apache 2.0 license.

Users are responsible for reviewing the licenses and terms associated with the base model, source dataset and any repositories represented in the training data.

Author

Created by micymike as part of the CodeMate model series.