---
license: apache-2.0
base_model: Qwen/Qwen3.5-4B
tags:
- reinforcement-learning
- rlhf
- cispo
- terminal-agent
- tmax
library_name: transformers
pipeline_tag: text-generation
---

# Qwen3.5-4B-TMax-CISPO

Qwen3.5-4B fine-tuned with **CISPO** (Clipped IS-weight Policy Optimization) on the
**TMax-15K** terminal-agent RL environment, using a fully-asynchronous rollout/trainer
setup (verl).

## Training
- **Base model:** `Qwen/Qwen3.5-4B`
- **Algorithm:** CISPO (rollout-anchored), clip high `0.28` / low `10`
- **Sampling:** temperature `1.0`, top_p `1.0`, group size `16`
- **Data:** TMax-15K, text-only **short/moderate** complexity split (AppTainer-compatible allowlist)
- **Agent:** `terminal_echo_tool_agent` (Terminus-2 command interface)
- **Precision:** fp32 generation/LM head + fused chunked cross-entropy
- Exported from trainer checkpoint (global_step 51).

## Intended use
Research checkpoint for terminal/agentic RL. Not instruction-tuned for general chat.