Instructions to use robinzixuan/LT-Tuning-Qwen2.5-1.5B-Instruct-GSM8K with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use robinzixuan/LT-Tuning-Qwen2.5-1.5B-Instruct-GSM8K with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="robinzixuan/LT-Tuning-Qwen2.5-1.5B-Instruct-GSM8K")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("robinzixuan/LT-Tuning-Qwen2.5-1.5B-Instruct-GSM8K")
model = AutoModelForCausalLM.from_pretrained("robinzixuan/LT-Tuning-Qwen2.5-1.5B-Instruct-GSM8K", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use robinzixuan/LT-Tuning-Qwen2.5-1.5B-Instruct-GSM8K with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "robinzixuan/LT-Tuning-Qwen2.5-1.5B-Instruct-GSM8K"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "robinzixuan/LT-Tuning-Qwen2.5-1.5B-Instruct-GSM8K",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/robinzixuan/LT-Tuning-Qwen2.5-1.5B-Instruct-GSM8K

SGLang

How to use robinzixuan/LT-Tuning-Qwen2.5-1.5B-Instruct-GSM8K with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "robinzixuan/LT-Tuning-Qwen2.5-1.5B-Instruct-GSM8K" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "robinzixuan/LT-Tuning-Qwen2.5-1.5B-Instruct-GSM8K",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "robinzixuan/LT-Tuning-Qwen2.5-1.5B-Instruct-GSM8K" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "robinzixuan/LT-Tuning-Qwen2.5-1.5B-Instruct-GSM8K",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use robinzixuan/LT-Tuning-Qwen2.5-1.5B-Instruct-GSM8K with Docker Model Runner:
```
docker model run hf.co/robinzixuan/LT-Tuning-Qwen2.5-1.5B-Instruct-GSM8K
```

LT-Tuning Qwen2.5-1.5B-Instruct (GSM8K)

Latent-Thoughts-Tuning applied to Qwen/Qwen2.5-1.5B-Instruct on GSM8K.

A <thinking> token (id 151665) is added to the vocabulary and inserted into the training sequences at positions chosen by the model's own token confidence, so the model learns to spend latent computation there before emitting an answer.

Status

Not evaluated. No GSM8K accuracy was measured for this checkpoint, so there is no evidence here that it improves over the base model. Benchmark it before relying on it.

Usage

Loads as a plain Qwen2ForCausalLM — the run used thinking_use_mlp: false, so the checkpoint contains no modules beyond the base architecture.

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "robinzixuan/LT-Tuning-Qwen2.5-1.5B-Instruct-GSM8K"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="bfloat16", device_map="auto")

messages = [{"role": "user", "content": "Natalia sold clips to 48 friends in April, "
                                        "and then she sold half as many clips in May. "
                                        "How many clips did she sell altogether?"}]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
print(tok.decode(model.generate(inputs, max_new_tokens=512)[0], skip_special_tokens=True))

Training

Three stages of 1 epoch each, run back-to-back with the optimizer reset between stages:

Stage	Name	Mode	Thinking insertion prob
0	`stage0-cot`	`common`	0.0
1	`stage1-hidden-state`	`hidden_state`	0.85
2	`stage2-soft-fusion`	`soft_fusion`	0.95

Between stages the dataset is regenerated: every training example is scored with a forward pass (thinking_strategy: confidence) to pick where the <thinking> tokens go.


Hardware	8× H100 80GB
Precision	bf16, FlashAttention-2
Global batch	32 (4 per device × 8 GPUs × 1 grad accum)
Steps	702 (234 per stage)
LR	5e-5, warmup ratio 0.05, weight decay 0.01
Grad clipping	1.0
Wall clock	4h08m
Train data	GSM8K socratic, 7473 train / 1319 val

Loss

Final stage2-soft-fusion loss settled around 0.36 (grad_norm ~1.7, stable throughout).

The reported train_loss of 1.297 is the mean across all three stages and is not a meaningful convergence figure — the stages optimize different objectives, so their losses are not comparable to each other.

Limitations

Trained only on GSM8K grade-school word problems; nothing here transfers to other domains by default.
No held-out evaluation was run, on GSM8K or anything else.
Inherits the limitations and license terms of the Qwen2.5-1.5B-Instruct base model.

Downloads last month: 10

Safetensors

Model size

2B params

Tensor type

BF16

Model tree for robinzixuan/LT-Tuning-Qwen2.5-1.5B-Instruct-GSM8K

Base model

Qwen/Qwen2.5-1.5B

Finetuned

Qwen/Qwen2.5-1.5B-Instruct

Finetuned

(1764)

this model

robinzixuan
/

LT-Tuning-Qwen2.5-1.5B-Instruct-GSM8K