Qwen3.5-122B-A10B-Claude-Distill-v2

A Qwen3.5-122B-A10B model fine-tuned on Claude-Distills, a high-quality reasoning dataset distilled from Claude.

Model Overview

• Base Model: Qwen3.5-122B-A10B (MoE, 122B total / 10B active parameters)
• Training Data: Claude-Distills (125,175 train / 6,588 val)
• Training Method: Full supervised fine-tuning (SFT)
• Training Framework: DeepSpeed ZeRO-3 Offload

Training Details

Item	Value
Epochs	1
Learning Rate	5e-6
Train Loss	0.674 → 0.537
Eval Loss	0.537
Dataset	Kassadin88/Claude-Distills

Data Distribution

The Claude-Distills dataset covers diverse reasoning domains:

Category	Percentage
Math	65.5%
Code	15.1%
Knowledge	5.1%
Science	5.0%
Conversation	2.6%
Reasoning	2.1%
Writing	1.4%
Instruction	1.4%
Other	1.8%

Quickstart

Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "Kassadin88/Qwen3.5-122B-A10B-Claude-Distill-v2"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype="auto", device_map="auto")

messages = [{"role": "user", "content": "Explain the chain rule in calculus."}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=2048)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

vLLM / SGLang

from vllm import LLM, SamplingParams

llm = LLM(model="Kassadin88/Qwen3.5-122B-A10B-Claude-Distill-v2")
params = SamplingParams(temperature=0.7, max_tokens=2048)
outputs = llm.chat([{"role": "user", "content": "Solve: x² + 5x + 6 = 0"}], sampling_params=params)
print(outputs[0].outputs[0].text)

OpenAI-compatible SDK

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
response = client.chat.completions.create(
    model="Kassadin88/Qwen3.5-122B-A10B-Claude-Distill-v2",
    messages=[{"role": "user", "content": "Write a Python quicksort."}],
    temperature=0.7,
)
print(response.choices[0].message.content)

Non-Thinking Mode

To disable the thinking/reasoning output, add /no_think to your prompt or set the system message accordingly:

messages = [
    {"role": "system", "content": "Reply without thinking."},
    {"role": "user", "content": "What is 2+2? /no_think"}
]

Recommended Sampling Parameters

Parameter	Value
Temperature	0.6
Top-P	0.95
Top-K	20
Min-P	0.01

For non-thinking mode:

• Temperature: 0.7
• Top-P: 0.8

Acknowledgments

• Qwen3.5-122B-A10B — The base model
• Claude-Distills — The training dataset

Limitations

• This is a distilled model and may inherit biases from the teacher model
• Performance on tasks outside the training distribution may vary
• The model may occasionally generate incorrect reasoning steps

Citation

@model{qwen35_122b_claude_distill_v2,
  title={Qwen3.5-122B-A10B-Claude-Distill-v2},
  author={Kassadin88},
  year={2026},
  url={https://huggingface.co/Kassadin88/Qwen3.5-122B-A10B-Claude-Distill-v2}
}