jalpan04/qwen-researcher

Qwen Researcher (0.5B - GGUF)

A specialized version of Qwen2.5-0.5B-Instruct fine-tuned for Computer Science research assistance. This model has been instruction-tuned on a curated subset of 2,000 arXiv Computer Science papers to provide academic summaries and technical insights.

Model Details

• Developed by: Jalpan04
• Model type: Causal Language Model
• Language(s): English
• License: Apache-2.0
• Fine-tuned from model: Qwen/Qwen2.5-0.5B-Instruct
• Training Method: QLoRA (Rank 16, Alpha 32)
• Dataset: arXiv Computer Science Metadata (Instruction-formatted)

Usage

Local Deployment (Ollama)

1. Create a Modelfile:

FROM ./qwen-resercher.gguf
TEMPLATE """{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .User }}<|im_start|>user
{{ .User }}<|im_end|>
{{ end }}<|im_start|>assistant
{{ .Output }}<|im_end|>"""
PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"
SYSTEM "You are a professional computer science researcher. Provide academic, detailed information based on research abstracts."

2. Run in terminal:

ollama create qwen-researcher -f Modelfile
ollama run qwen-researcher

Python (llama-cpp-python)

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="jalpan04/qwen-researcher",
    filename="qwen-resercher.gguf",
)

response = llm.create_chat_completion(
    messages = [
        {"role": "system", "content": "You are a CS researcher."},
        {"role": "user", "content": "Summarize the latest trends in Neural Program Synthesis."}
    ]
)
print(response["choices"][0]["message"]["content"])

Training Procedure

The model was trained on an NVIDIA RTX 4060 (8GB) using the following setup:

• Optimization: 4-bit NormalFloat (nf4) quantization.
• Precision: BFloat16 for stability on 40-series hardware.
• Learning Rate: 2e-4.
• Batch Size: 1 with Gradient Accumulation (16 steps).
• Sequence Length: 1024 tokens.

Limitations

As a 0.5B parameter model, this is highly efficient but may exhibit hallucinations compared to larger models (7B+). It is best used for summarization and quick technical lookups rather than complex logical reasoning.