Improve model card with usage, evaluation and limitations

README.md

@@ -2,86 +2,196 @@
 library_name: transformers
 pipeline_tag: text-generation
 license: apache-2.0
+license_link: https://huggingface.co/Qwen/Qwen2.5-7B-Instruct/blob/main/LICENSE
+language:
+- en
 base_model: Qwen/Qwen2.5-7B-Instruct
+base_model_relation: finetune
 tags:
 - search-agent
 - tool-use
 - function-calling
 - react
 - deep-search
+- multi-hop-qa
 - qwen
 ---
 
 # SearchQwen2.5-7B
 
-SearchQwen2.5-7B is a search-agent model post-trained on environment-aligned multi-hop search tasks and verified ReAct trajectories generated by the EasyDistill2 pipeline. The model is designed for structured `search` / `browse` tool interaction and long-horizon evidence integration.
+**SearchQwen2.5-7B** is a compact Search Agent model from the Alibaba Cloud PAI team. It is post-trained on environment-aligned multi-hop search tasks and solver-verified ReAct trajectories generated with the EasyDistill2 pipeline.
+
+The model is optimized for iterative **search → browse → evidence integration → answer** behavior. It supports both structured Tool-Call interaction and Search-R1-style text interaction (Qwen2.5 variants).
+
+> 中文简介：SearchQwen2.5-7B 是面向多跳问答与深度搜索任务训练的 Search Agent 小模型，重点增强结构化工具调用、网页浏览和跨来源信息整合能力。
 
 ## Model details
 
-- Base model: `Qwen/Qwen2.5-7B-Instruct`
-- Parameters: 7.62B
-- Training: trajectory SFT followed by process-aware post-training
-- Interaction: structured Tool-Call (recommended); Search-R1-style text interaction is also evaluated for Qwen2.5 models
-- Companion dataset: [`alibaba-pai/SynSearch-Data`](https://huggingface.co/datasets/alibaba-pai/SynSearch-Data)
+| Item | Value |
+|---|---|
+| Base model | [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) |
+| Parameters | 7.62B |
+| Architecture | Qwen2, 28 layers |
+| Context length | 32,768 tokens in the released config |
+| Primary interface | Structured `search` / `browse` Tool-Call |
+| Training | Environment-aligned trajectories, Solver-in-the-Loop verification, and process-aware post-training |
+| Companion dataset | [alibaba-pai/SynSearch-Data](https://huggingface.co/datasets/alibaba-pai/SynSearch-Data) |
+
 
-## Evaluation results
 
-LLM-judge accuracy (%).
+## Evaluation
+
+We report LLM-judge accuracy (%) over two benchmark groups:
+
+- **Multi-hop QA:** 2WikiMultiHopQA, Bamboogle, HotpotQA, MuSiQue.
+- **Deep Search:** GAIA, WebWalkerQA, xbench-deepsearch, BrowseComp-ZH.
 
 ### Search-R1-style text interaction
 
 | Model | Multi-hop QA Avg. | Deep Search Avg. | Overall Avg. |
 |---|---:|---:|---:|
-| Qwen2.5-7B-Instruct (Base) | 45.28 | 18.65 | 31.96 |
+| Qwen2.5-7B-Instruct | 45.28 | 18.65 | 31.96 |
 | **SearchQwen2.5-7B** | **52.95** | **26.23** | **39.59** |
-| Qwen2.5-3B-Instruct (Base) | 30.12 | 14.95 | 22.54 |
-| **SearchQwen2.5-3B** | **39.55** | **21.15** | **30.35** |
 
 ### Structured Tool-Call interaction
 
 | Model | Multi-hop QA Avg. | Deep Search Avg. | Overall Avg. |
 |---|---:|---:|---:|
-| Qwen2.5-7B-Instruct (Base) | 45.23 | 24.35 | 33.33 |
+| Qwen2.5-7B-Instruct | 45.23 | 24.35 | 33.33 |
 | **SearchQwen2.5-7B** | **55.90** | **33.33** | **44.61** |
-| Qwen2.5-3B-Instruct (Base) | 36.10 | 7.05 | 21.60 |
-| **SearchQwen2.5-3B** | **48.58** | **21.40** | **35.00** |
 
-### Cross-backbone generalization (Structured Tool-Call)
+Deep Search gains are consistently larger than the gains on conventional multi-hop QA, indicating improved search, browsing, and evidence-integration behavior rather than answer memorization alone.
 
-| Model | Deep Search Avg. | Overall Avg. |
-|---|---:|---:|
-| Qwen3-8B (Base) | 24.50 | 40.31 |
-| **SearchQwen3-8B** | **35.42** | **50.31** |
+## Requirements
 
+```bash
+pip install -U transformers accelerate torch
+```
 
-## Usage
+## Transformers quickstart
 
 ```python
+import torch
 from transformers import AutoModelForCausalLM, AutoTokenizer
 
 model_id = "alibaba-pai/SearchQwen2.5-7B"
 tokenizer = AutoTokenizer.from_pretrained(model_id)
 model = AutoModelForCausalLM.from_pretrained(
     model_id,
-    torch_dtype="auto",
+    torch_dtype=torch.bfloat16,
     device_map="auto",
-)
+).eval()
+
+tools = [
+    {
+        "type": "function",
+        "function": {
+            "name": "search",
+            "description": "Search the web for relevant pages.",
+            "parameters": {
+                "type": "object",
+                "properties": {"query": {"type": "string"}},
+                "required": ["query"],
+            },
+        },
+    },
+    {
+        "type": "function",
+        "function": {
+            "name": "browse",
+            "description": "Read the content of a web page.",
+            "parameters": {
+                "type": "object",
+                "properties": {"url": {"type": "string"}},
+                "required": ["url"],
+            },
+        },
+    },
+]
+
+messages = [
+    {"role": "system", "content": "You are a search agent. Use tools before answering."},
+    {"role": "user", "content": "Which city is the birthplace of the author of The Old Man and the Sea?"},
+]
+
+inputs = tokenizer.apply_chat_template(
+    messages,
+    tools=tools,
+    add_generation_prompt=True,
+    tokenize=True,
+    return_tensors="pt",
+    return_dict=True,
+).to(model.device)
+
+with torch.inference_mode():
+    output = model.generate(**inputs, max_new_tokens=256, do_sample=False)
+
+new_tokens = output[0, inputs["input_ids"].shape[-1]:]
+print(tokenizer.decode(new_tokens, skip_special_tokens=False))
+```
+
+A typical first response is a structured call:
+
+```xml
+<tool_call>
+{"name": "search", "arguments": {"query": "..."}}
+</tool_call>
+```
+
+Execute the requested tool, append the tool response to the conversation, and call the model again until it returns a final answer.
+
+## vLLM deployment
+
+```bash
+vllm serve alibaba-pai/SearchQwen2.5-7B \
+  --served-model-name SearchQwen2.5-7B \
+  --enable-auto-tool-choice \
+  --tool-call-parser hermes \
+  --max-model-len 8192
 ```
 
-For Qwen3, pass `enable_thinking=False` to `apply_chat_template` when reproducing the non-thinking Tool-Call evaluation setting.
+For `SearchQwen2.5-7B`, the release was validated with Transformers and an OpenAI-compatible vLLM endpoint on NVIDIA L20 GPUs. The test covered checkpoint loading, structured `search` emission, tool-result consumption, and final-answer generation.
+
+## Training data
+
+The released companion dataset contains 5,000 audited multi-hop ReAct trajectories covering 4,182 unique tasks. Each example includes a complete `messages` history and metadata for seed provenance, hop count, difficulty, rollout statistics, and answer-equivalence auditing.
+
+## Intended use
+
+- Research on search agents and retrieval-augmented reasoning.
+- Search/browse tool-use experiments.
+- Fine-tuning, evaluation, and deployment of compact agent models.
 
-## Verified deployment
+## Limitations
 
-The release checkpoint was validated by:
+- The model requires an external search/browse runtime; it does not provide a search index itself.
+- Final-answer quality depends on tool availability and retrieved evidence quality.
+- Tool schemas and prompt templates should remain compatible with the released chat template.
+- LLM-judge scores may vary with evaluator model, decoding settings, and search backend.
+- Do not use the model as the sole authority for safety-critical, legal, medical, or financial decisions.
 
-1. loading every safetensors shard;
-2. loading the full model on an NVIDIA L20 GPU;
-3. serving it with vLLM's OpenAI-compatible API;
-4. emitting a structured `search` tool call;
-5. consuming the tool response and producing the final answer.
+## Artifact integrity
 
-See `SHA256SUMS` for artifact checksums.
+`SHA256SUMS` contains checksums for all weight shards, the weight index, configuration, and tokenizer.
+
+## Related releases
+
+- [SearchQwen2.5-7B](https://huggingface.co/alibaba-pai/SearchQwen2.5-7B)
+- [SearchQwen2.5-3B](https://huggingface.co/alibaba-pai/SearchQwen2.5-3B)
+- [SearchQwen3-8B](https://huggingface.co/alibaba-pai/SearchQwen3-8B)
+- [SynSearch-Data](https://huggingface.co/datasets/alibaba-pai/SynSearch-Data)
+
+## Citation
+
+```bibtex
+@misc{searchqwen2026,
+  title  = {Fake It Till You Make It: Training Deep Search Agents on Synthetic Reality},
+  author = {Alibaba Cloud PAI Team},
+  year   = {2026},
+  note   = {SearchQwen model family and SynSearch-Data}
+}
+```
 
 ## License
 
-This derivative model follows the license included in this repository and the terms of its base model.
+This derivative model is released under the license file included in this repository and remains subject to the terms of its base model.