---
license: apache-2.0
base_model: Qwen/Qwen3.5-9B
tags:
- qwen3.5
- reasoning
- distillation
- lora
- sft
datasets:
- bespokelabs/Bespoke-Stratos-17k
- allenai/tulu-3-sft-mixture
- Open-Orca/SlimOrca
- PrimeIntellect/SYNTHETIC-1-SFT-Data
pipeline_tag: text-generation
---
> ## ⚠️ CRITICAL: Ollama Inference Flag Required
>
> **If you serve this model via Ollama with the qwen3.5 RENDERER (the standard
> recommended setup), you MUST pass `"think": false` in the `/api/chat`
> request body for chat / instruction following / tool use.**
>
> ```bash
> curl -X POST http://localhost:11434/api/chat \
>   -d '{"model": "...", "think": false, "messages": [...], "stream": false}'
> ```
>
> Without this flag, the renderer auto-injects `<think>` tags into every chat
> completion. On longer prompts the model can stay inside the `<think>` block
> past the response budget, never emit `</think>`, and produce **zero answer
> tokens** on 25-46% of requests.
>
> Set `think: true` (or omit) only when you DO want chain-of-thought reasoning
> (math, planning, complex multi-step). This is Qwen3 dual-mode operation per
> https://qwenlm.github.io/blog/qwen3/.
>
> See the dataset `cudabenchmarktest/r9-research-framework`
> `_OLLAMA_INFERENCE_WARNING.md` for the full explanation.

---


# Qwen3.5-9B R7 Research (FP16)

Fine-tuned [Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B) with distilled reasoning from research-backed datasets. Text-only model (no vision encoder). For the vision-capable version, see [cudabenchmarktest/qwen3.5-9b-r7-research-vision](https://huggingface.co/cudabenchmarktest/qwen3.5-9b-r7-research-vision).

This is the pre-GGUF FP16 safetensors checkpoint. For quantized GGUF versions ready for Ollama, see [robit/qwen3.5-9b-r7-research on Ollama](https://ollama.com/robit/qwen3.5-9b-r7-research).

## Capabilities

- **Thinking** — produces structured reasoning in `<think>` blocks
- **Tool calling** — structured function calls when given tool definitions
- **Instruction following** — concise answers, format constraints, system prompt adherence

## Eval Results

| Benchmark | Score |
|-----------|-------|
| Diverse stochastic eval (38 tests, 9 categories) | **86.8%** |
| Base qwen3.5:9b on same eval | 79.0% |

## Training Details

- **Method**: LoRA SFT (r=32, alpha=64, LR=1e-4, 1 epoch, completion-only loss masking)
- **Hardware**: 3x NVIDIA A100 80GB (DDP)
- **Data**: Additive mix of 4043 samples:
  - [bespokelabs/Bespoke-Stratos-17k](https://huggingface.co/datasets/bespokelabs/Bespoke-Stratos-17k) — DeepSeek-R1 reasoning traces (2000 samples)
  - [allenai/tulu-3-sft-mixture](https://huggingface.co/datasets/allenai/tulu-3-sft-mixture) — instruction diversity (1358 samples)
  - [Open-Orca/SlimOrca](https://huggingface.co/datasets/Open-Orca/SlimOrca) — curated GPT-4 instructions (451 samples)
  - [PrimeIntellect/SYNTHETIC-1-SFT-Data](https://huggingface.co/datasets/PrimeIntellect/SYNTHETIC-1-SFT-Data) — verified math/code/STEM (312 samples)
  - Static format-constrained, conversational, and code examples (remaining)
- **Best eval loss**: 0.4950

## Training Suite

Full training pipeline, evaluation scripts, and documentation: [robit-man/fine_tuning_suite](https://github.com/robit-man/fine_tuning_suite)

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "cudabenchmarktest/qwen3.5-9b-r7-research",
    torch_dtype="auto",
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained("cudabenchmarktest/qwen3.5-9b-r7-research")

messages = [{"role": "user", "content": "What is the capital of France?"}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## License

Apache 2.0 (inherited from Qwen3.5-9B). Training data licenses vary by source.