# gpt-oss-120b-Fable-5-Distilled — Tool Calling Guide

**A 120B MoE agent model fine-tuned on real-world Claude Code programming sessions. Uses OpenAI Harmony protocol with native commentary-channel tool calling.**

---

## Overview

`gpt-oss-120b-Fable-5-Distilled` supports tool calling via the **Harmony commentary channel** — a structured, streaming-friendly protocol that separates reasoning (`analysis`), tool invocation (`commentary`), and final responses (`final`) into distinct channels within a single generation.

Unlike traditional function-calling models that produce separate `tool_calls` API fields, this model expresses tool calls **inline as part of its text generation**, which means:

- Tool calls and text responses coexist in a single coherent stream
- No separate API call / tool-call round-trip metadata required at the model level
- The model can reason about *which* tool to call in the `analysis` channel before committing

Compatible servers (e.g. `mlx-openai-server`, LM Studio) parse the commentary channel and expose it as standard OpenAI `tool_calls` for client consumption.

---

## Quick Start

### Server

```bash
mlx_lm.server --model gpt-oss-120b-Fable-5-Distilled --temp 1.0 --min-p 0.01 --max-tokens 4096
```

### Client (OpenAI SDK)

```python
from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8080/v1", api_key="not-needed")

response = client.chat.completions.create(
    model="gpt-oss-120b-Fable-5-Distilled",
    messages=[{"role": "user", "content": "北京天气怎么样？"}],
    tools=[{
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "获取指定城市的当前天气信息",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {"type": "string", "description": "城市名称，如 Beijing"},
                    "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
                },
                "required": ["city"],
            },
        },
    }],
    temperature=0.0,
)

msg = response.choices[0].message
print("Tool calls:", msg.tool_calls)
print("Text:", msg.content)
```

---

## Harmony Protocol Reference

### Message Format

Every message follows the Harmony token structure:

| Token | Role | Example |
|-------|------|---------|
| `<\|start\|>` | Begin message | `<\|start\|>user` |
| `<\|channel\|>` | Separate role from channel type | `<\|channel\|>analysis` |
| `<\|message\|>` | Separate header from body | `<\|message\|>Hello world` |
| `<\|end\|>` | End intermediate message | `...<\|end\|>` |
| `<\|return\|>` | End final assistant message (EOS) | `...<\|return\|>` |
| `<\|constrain\|>` | Constraint type annotation | `<\|constrain\|>json` |

### Channel Types

| Channel | Purpose |
|---------|---------|
| `analysis` | Internal chain-of-thought (can be hidden) |
| `final` | Deliverable response to user |
| `commentary` | Tool call invocation |

### System Message

```
<|start|>system<|message|>You are a helpful assistant.
Reasoning: high
# Valid channels: analysis, commentary, final. Channel must be included for every message.<|end|>
```

### User Message

```
<|start|>user<|message|>北京今天天气怎么样？<|end|>
```

---

## Tool Calling Format

### Single Tool Call

When the model decides to invoke a tool, it generates a `commentary` channel:

```
<|start|>assistant<|channel|>analysis<|message|>
用户想知道北京天气，应该调用 get_weather，参数 city="北京"。
<|end|>
<|start|>assistant<|channel|>commentary to=functions.get_weather <|constrain|>json<|message|>
{
  "city": "北京",
  "unit": "celsius"
}
```

**Key syntax:**

| Element | Meaning |
|---------|---------|
| `<\|channel\|>commentary` | Declares this is a tool invocation |
| `to=functions.get_weather` | Target tool name (must match a defined function) |
| `<\|constrain\|>json` | Declares JSON-constrained output |
| `<\|message\|>{JSON}` | Tool arguments in JSON format |

**Important:** The commentary channel does NOT end with `<|end|>` or `<|return|>`. The model expects the server to pause generation, execute the tool, and feed the result back as a `tool` role message.

### Multi-Tool Call Scenario

For multi-tool scenarios, the model can generate multiple commentary channels sequentially:

```
<|start|>assistant<|channel|>analysis<|message|>
需要同时搜索和读取文件...
<|end|>
<|start|>assistant<|channel|>commentary to=functions.search_web <|constrain|>json<|message|>
{"query": "GPT-OSS benchmark 2025", "max_results": 5}
<|start|>assistant<|channel|>commentary to=functions.read_file <|constrain|>json<|message|>
{"path": "/home/user/results.json"}
```

### No-Tool Response

When no tool is needed, the model uses the `final` channel directly:

```
<|start|>assistant<|channel|>final<|message|>
1+1等于2。
<|return|>
```

---

## Full Multi-Turn Workflow

### Turn 1 — User → Tool Call

**Request:**
```json
{
  "messages": [{"role": "user", "content": "北京天气怎么样？"}],
  "tools": [{"type": "function", "function": {"name": "get_weather", ...}}]
}
```

**Model Output (raw Harmony):**
```
<|channel|>analysis<|message|>需要调用 get_weather，参数 city="北京"<|end|>
<|start|>assistant<|channel|>commentary to=functions.get_weather <|constrain|>json<|message|>
{"city": "北京", "unit": "celsius"}
```

**Server Parses as:**
```json
{
  "choices": [{
    "message": {
      "content": null,
      "tool_calls": [{
        "id": "call_001",
        "type": "function",
        "function": {
          "name": "get_weather",
          "arguments": "{\"city\":\"北京\",\"unit\":\"celsius\"}"
        }
      }]
    }
  }]
}
```

### Turn 2 — Tool Result → Final Response

**Request (append tool result):**
```json
{
  "messages": [
    {"role": "user", "content": "北京天气怎么样？"},
    {
      "role": "assistant",
      "content": null,
      "tool_calls": [{"id": "call_001", "type": "function", "function": {"name": "get_weather", "arguments": "{\"city\":\"北京\",\"unit\":\"celsius\"}"}}]
    },
    {
      "role": "tool",
      "tool_call_id": "call_001",
      "content": "{\"city\":\"北京\",\"temperature\":23,\"unit\":\"celsius\",\"condition\":\"晴转多云\",\"humidity\":42,\"wind\":\"东南风3级\"}"
    }
  ]
}
```

**Model Output (raw Harmony):**
```
<|channel|>analysis<|message|>天气数据已返回，温度23°C，晴转多云<|end|>
<|start|>assistant<|channel|>final<|message|>
北京今天晴转多云，气温23°C，湿度42%，东南风3级。
<|return|>
```

---

## Python — Full Multi-Turn Example

```python
from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8080/v1", api_key="not-needed")

TOOLS = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "获取指定城市的当前天气信息",
        "parameters": {
            "type": "object",
            "properties": {
                "city": {"type": "string", "description": "城市名称"},
                "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
            },
            "required": ["city"],
        },
    },
}]


def run_agent(user_message: str) -> str:
    messages = [{"role": "user", "content": user_message}]

    # Turn 1: model decides to call tool
    resp = client.chat.completions.create(
        model="gpt-oss-120b-Fable-5-Distilled",
        messages=messages,
        tools=TOOLS,
        temperature=0.0,
    )
    msg = resp.choices[0].message

    # If model called a tool, execute it
    if msg.tool_calls:
        # Append assistant's tool call to history
        messages.append({
            "role": "assistant",
            "content": msg.content,
            "tool_calls": [
                {
                    "id": tc.id,
                    "type": "function",
                    "function": {
                        "name": tc.function.name,
                        "arguments": tc.function.arguments,
                    },
                }
                for tc in msg.tool_calls
            ],
        })

        # Execute each tool and append results
        import json

        for tc in msg.tool_calls:
            args = json.loads(tc.function.arguments)
            if tc.function.name == "get_weather":
                result = json.dumps({
                    "city": args["city"],
                    "temperature": 23,
                    "unit": args.get("unit", "celsius"),
                    "condition": "晴转多云",
                    "humidity": 42,
                    "wind": "东南风3级",
                }, ensure_ascii=False)
            else:
                result = json.dumps({"error": "unknown tool"})
            messages.append({
                "role": "tool",
                "tool_call_id": tc.id,
                "content": result,
            })

        # Turn 2: model processes result
        resp2 = client.chat.completions.create(
            model="gpt-oss-120b-Fable-5-Distilled",
            messages=messages,
            temperature=0.0,
        )
        return resp2.choices[0].message.content

    return msg.content


print(run_agent("北京天气怎么样？"))
# 输出: 北京今天晴转多云，气温23°C，湿度42%，东南风3级。
```

---

## Tool Definition Best Practices

| Practice | Example | Why |
|----------|---------|-----|
| **Clear `name`** | `get_weather` not `tool_1` | Model uses name to select correct function |
| **Descriptive `description`** | "获取指定城市的当前天气信息" | Helps model decide WHEN to call |
| **Typed parameters** | `"city": {"type": "string"}` | Model fills correct types |
| **`required` array** | `"required": ["city"]` | Model always includes critical params |
| **`enum` for constrained values** | `"enum": ["celsius", "fahrenheit"]` | Prevents invalid values |

### Complete Tool Schema Template

```json
{
    "type": "function",
    "function": {
        "name": "search_web",
        "description": "搜索互联网上的信息，返回相关结果列表",
        "parameters": {
            "type": "object",
            "properties": {
                "query": {
                    "type": "string",
                    "description": "搜索关键词"
                },
                "max_results": {
                    "type": "integer",
                    "description": "返回结果数量，默认5，最大20"
                },
                "language": {
                    "type": "string",
                    "enum": ["zh", "en"],
                    "description": "结果语言偏好"
                }
            },
            "required": ["query"]
        }
    }
}
```

---

## System Prompt Guidelines

The default system prompt is tuned for this model:

```
You are a helpful assistant with access to tools.

CRITICAL RULES — follow these exactly:
1. When a tool is available and relevant, you MUST call it using tool_calls.
2. NEVER write tool arguments as inline JSON text in your response.
3. Only answer in plain text when no tool is needed.
4. After receiving a tool result, summarize it clearly for the user.

Remember: tool_calls first, text explanation second.
```

- **Temperature 0.0** is recommended for production tool calling — ensures deterministic, stable behavior (20/20 consistency in tests)
- **Temperature 0.95** is recommended for creative coding tasks that benefit from generation diversity
- The model respects `tool_choice: "auto"` (skip tools when irrelevant) and correctly handles hallucination prevention (does NOT call tools on questions like "1+1=?")

---

## Performance Benchmarks

Tested on Apple Silicon M-series with MLX, temperature=0.0:

| Metric | Value |
|--------|-------|
| Tool call format accuracy | **100%** (6/6 test categories) |
| Tool call stability (20-round) | **100%** (20/20 consistent) |
| Tool name selection | **100%** (correct tool from multi-tool list) |
| Required parameter filling | **100%** (no missing required params) |
| No hallucinated tool calls | **100%** (no phantom calls on non-tool queries) |
| Tool result followup | **100%** (correct summarization after tool return) |
| Avg latency (tool call) | ~1,000 ms |
| Throughput | ~95 tok/s |

---

## Comparison: Commentary vs Classic Function Calling

| Aspect | Classic Function Calling | Harmony Commentary |
|--------|-------------------------|-------------------|
| Tool call location | Separate `tool_calls` API field | Inline in generation stream |
| Reasoning visibility | Hidden (if any) | `analysis` channel — inspectable |
| Streaming support | Requires special handling | Native — channels stream sequentially |
| Multi-turn | Separate requests per round | Same pattern, parsed by server |
| Model training | Requires function-calling fine-tune data | Trained on real agent traces |

---

## Troubleshooting

| Problem | Likely Cause | Fix |
|---------|-------------|-----|
| Tool calls not detected | Pre-v4 test script or client not parsing commentary | Use OpenAI SDK; server should auto-parse |
| Model outputs JSON in text instead of `tool_calls` | Server not parsing commentary channel | Ensure server supports Harmony protocol |
| Wrong tool selected | Tool descriptions too similar | Make descriptions more distinct |
| Missing required params | Parameter not in `required` array | Add to `required` |
| Temperature too high → unstable calls | T > 0.3 introduces variance | Use `temperature=0.0` for tool calling |
| `all_proxy` connection error | SOCKS proxy env var set | `unset all_proxy http_proxy https_proxy` |