README.md

---
license: apache-2.0
base_model: Qwen/Qwen3-VL-8B-Thinking
tags:
- physics
- tool-use
- sympy
- reasoning
- sft
- cold-start
- multimodal-rl
- qwen
language:
- en
---

# Qwen3-VL-8B-Thinking — Physics-R2 SFT Cold-Start v1

Stage-1 cold-start checkpoint for the **Physics-R2** project (a tool-use RL paper for VLMs, targeting ICLR 2027).
Fine-tuned from `Qwen/Qwen3-VL-8B-Thinking` on 1,776 audited tool-using physics
trajectories from [shanyangmie/physr1corp-cold-start](https://huggingface.co/datasets/shanyangmie/physr1corp-cold-start).

## What this model does

Given a physics problem, the model emits a structured reasoning trajectory that uses a SymPy tool mid-stream:

```
{brief reasoning}
<tool>sympy: {symbolic expression}</tool>
<tool_result>{result}</tool_result>
{continued reasoning}
<answer>\boxed{ANSWER}</answer>
```

At inference time, a runtime harness intercepts each `<tool>...</tool>` call, executes the SymPy expression in a sandboxed subprocess, injects the result back as `<tool_result>...</tool_result>`, and lets the model continue.
The harness lives in the [physics-r2-progress repo](https://github.com/shanyang-me/physics-r2-progress) under `phase_e/harness/`.

## Training details

| Hyperparameter | Value |
|---|---|
| Base model | `Qwen/Qwen3-VL-8B-Thinking` |
| Training framework | TRL `SFTTrainer` on FSDP1 |
| GPUs | 4× H200 SXM 141GB |
| Optimizer | AdamW (β=[0.9, 0.95], wd=0.01) |
| Learning rate | 2e-5, cosine decay to 10%, 30 warmup steps |
| Effective batch | 16 (2 per device × 2 grad accum × 4 GPUs) |
| Sequence length | 4096 |
| Epochs | 3 (note: eval loss minimum at epoch 2) |
| Precision | bfloat16 mixed |
| Visual tower | **frozen** (text-only SFT for v1) |
| Wall time | ~18 minutes |

## Results

| Metric | Value |
|---|---|
| Train loss (final) | 0.13 |
| Eval loss (epoch 1) | ~0.54 |
| Eval loss (epoch 2) | ~0.55 |
| Eval loss (epoch 3, final) | 0.61 |
| Eval token accuracy | 84.7% |

**Note**: eval loss bottomed at epoch 2 (~0.55) and rose slightly at epoch 3 (0.61). The v1 checkpoint here is the epoch-3 model; a future v2 may use the epoch-2 checkpoint or 2-epoch training.

## Usage

```python
from transformers import AutoProcessor, Qwen3VLForConditionalGeneration
import torch

model = Qwen3VLForConditionalGeneration.from_pretrained(
    "shanyangmie/qwen3-vl-8b-thinking-physics-r2-sft-v1",
    dtype=torch.bfloat16,
    device_map="auto",
)
processor = AutoProcessor.from_pretrained("shanyangmie/qwen3-vl-8b-thinking-physics-r2-sft-v1")
```

For inference with the SymPy tool actually executing, use the harness:

```python
import sys; sys.path.insert(0, "/path/to/physics-r2-progress/phase_e")
from harness.runner import run_with_tools
# ... see phase_e/sft/eval_sft.py for full example
```

## Limitations & known issues

- **Visual tower frozen during SFT v1.** The 233 multimodal training rows (image-bearing physics problems) were trained text-only. The model has not learned to use figure content; multimodal Q&A will not benefit from the figure. A v2 multimodal SFT is planned.
- **Mild overfit at epoch 3.** Eval loss rose ~10% from epoch 2 to epoch 3. Use with that caveat or wait for v2.
- **Result hallucination risk at inference.** The model was trained to emit the full `<tool>...</tool_result>` pattern from training data. At inference, the harness is the only legitimate source of `<tool_result>` content. If the inference loop is not properly wired, the model may emit hallucinated tool results. The harness in `phase_e/harness/runner.py` handles this correctly.

## Citation

Anonymous (in preparation), 2026. Physics-R2: Tool-Use RL for Physics VLMs. ICLR 2027.

Underlying corpus and cold-start dataset:
- [shanyangmie/physr1corp-cold-start](https://huggingface.co/datasets/shanyangmie/physr1corp-cold-start) — the SFT training dataset (1973 audited trajectories)
- Anonymous, 2026. Physics-R1: An Audited Olympiad Corpus and Recipe for Visual Physics Reasoning. NeurIPS 2026 D&B.