MLX
jinja
chat-template
qwen
qwen3.5
qwen3.6
qwen3.8
llama.cpp
lm-studio
vllm
tool-calling
thinking
token-efficient
Instructions to use peculiar-ragdoll/Qwen-Sharp-Chat-Templates with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use peculiar-ragdoll/Qwen-Sharp-Chat-Templates with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] huggingface-cli download --local-dir Qwen-Sharp-Chat-Templates peculiar-ragdoll/Qwen-Sharp-Chat-Templates
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
v22.3.1: rebase onto froggeric v22.3, vendor upstream test suite, third fast-mode fix
Browse files- .DS_Store +0 -0
- README.md +143 -48
- archive/v22.1.1-sharp/chat_template.jinja +379 -0
- archive/v22.1.1-sharp/chat_template_oneline.txt +5 -0
- chat_template.jinja +100 -42
- chat_template_oneline.txt +2 -2
- scripts/check_applied.py +56 -9
- scripts/fuzz_template.py +284 -0
- scripts/test_v21.py +184 -0
- scripts/test_v22.py +1736 -0
- scripts/verify_template.py +39 -9
.DS_Store
ADDED
|
Binary file (6.15 kB). View file
|
|
|
README.md
CHANGED
|
@@ -37,40 +37,74 @@ With the *Sharp* template, Qwen3.8-27b (medium effort) gets smarter **and** uses
|
|
| 37 |
## Straight to the point
|
| 38 |
|
| 39 |
This is froggeric's [Qwen-Fixed-Chat-Templates](https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates)
|
| 40 |
-
`v22.
|
| 41 |
-
The base fixes issues, the addition makes it better, and `v22.
|
| 42 |
coherent — see the changelog below.
|
| 43 |
|
| 44 |
-
> **`v22.
|
| 45 |
-
> **thinking-off (fast) mode**; the thinking-**on** path is byte-identical to
|
| 46 |
-
> running with thinking on
|
|
|
|
| 47 |
>
|
| 48 |
-
> 1. **Fast-mode `<think>` contradiction fixed.** With tools and thinking *off*,
|
| 49 |
-
> model to put its reasoning inside `<think></think>` — while the generation prompt
|
| 50 |
-
> closed thinking.
|
| 51 |
-
>
|
| 52 |
-
> 2. **Terseness lead split by mode.**
|
| 53 |
-
> incoherent when thinking is off
|
| 54 |
-
>
|
| 55 |
-
>
|
|
|
|
|
|
|
|
|
|
|
|
|
| 56 |
>
|
| 57 |
-
>
|
| 58 |
-
>
|
| 59 |
-
>
|
| 60 |
-
>
|
| 61 |
-
>
|
| 62 |
-
>
|
| 63 |
-
>
|
| 64 |
-
>
|
| 65 |
-
|
| 66 |
-
> **
|
| 67 |
-
>
|
| 68 |
-
>
|
| 69 |
-
>
|
| 70 |
-
>
|
| 71 |
-
>
|
| 72 |
-
>
|
| 73 |
-
>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 74 |
>
|
| 75 |
> ```json
|
| 76 |
> {"messages": [...], "chat_template_kwargs": {"reasoning_effort": "low"}}
|
|
@@ -81,19 +115,20 @@ coherent — see the changelog below.
|
|
| 81 |
the **v1** template baked into those builds — that is the exact template embedded in those GGUF and
|
| 82 |
MLX builds (`template_version = "qwen3.6-froggeric-v21.3"`, terseness, no reasoning-effort steering),
|
| 83 |
and it lives here in [`archive/v1-qwen3.6-froggeric-v21.3/`](archive/v1-qwen3.6-froggeric-v21.3). The
|
| 84 |
-
`chat_template.jinja` at the root of this repo is the newest **v22.
|
| 85 |
move a model onto it. The template is published separately because it is the portable part — the
|
| 86 |
thing worth reusing is not tied to either model.
|
| 87 |
|
| 88 |
Superseded versions are kept verbatim under [`archive/`](archive): the **v1** froggeric-v21.3 build
|
| 89 |
-
(Dagger/Nail),
|
| 90 |
-
[`archive/v22.1-sharp/`](archive/v22.1-sharp) —
|
|
|
|
| 91 |
|
| 92 |
## What it changes
|
| 93 |
|
| 94 |
A terseness block, force-appended after your own system prompt. The **lead** now varies by thinking
|
| 95 |
-
mode (the v22.
|
| 96 |
-
thinking-on path is byte-identical to upstream v22.
|
| 97 |
|
| 98 |
```jinja
|
| 99 |
{%- if ns_state.thinking %}
|
|
@@ -117,9 +152,9 @@ If a user request is genuinely ambiguous, ask a sharp question, don't guess.
|
|
| 117 |
Two things happen here: the `if/else` on `_sc` keeps **your own system prompt** — the terseness block
|
| 118 |
is appended after it, nothing you pass in is replaced; and the lead line matches the reasoning mode so
|
| 119 |
a fast-mode model isn't told to "answer after thinking" when it isn't thinking. Separately, the
|
| 120 |
-
tool-calling instructions gate
|
| 121 |
-
v22.
|
| 122 |
-
reasoning-effort line unless you ask for one (earlier v22 forced `xhigh`; see the v22.
|
| 123 |
An explicit `reasoning_effort` still renders.
|
| 124 |
|
| 125 |
## Impact
|
|
@@ -169,7 +204,7 @@ renders identically to the full template (verified by `scripts/verify_template.p
|
|
| 169 |
**llama.cpp at runtime, without touching the file** — pass it per-run instead:
|
| 170 |
|
| 171 |
```bash
|
| 172 |
-
llama-server -m model.gguf --chat-template-file chat_template.jinja -ngl 99
|
| 173 |
llama-cli -m model.gguf --chat-template-file chat_template.jinja -ngl 99
|
| 174 |
```
|
| 175 |
|
|
@@ -179,12 +214,22 @@ byte-identical to this file and the model name is gone. Check it yourself with
|
|
| 179 |
`curl localhost:8080/props | jq -r .chat_template`, or render a prompt through
|
| 180 |
`POST /apply-template`.
|
| 181 |
|
| 182 |
-
|
| 183 |
it — on older builds you do, and it must come *before* `--chat-template-file`. And the flag is
|
| 184 |
per-invocation: forget it once and you silently get the embedded template back. Rewriting the
|
| 185 |
GGUF with `gguf-new-metadata` is the durable version; the flag is right for trying it out or for
|
| 186 |
running one template across several models.
|
| 187 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 188 |
## Did it actually apply?
|
| 189 |
|
| 190 |
Point `check_applied.py` at a model directory or a `.gguf`. It finds every template source,
|
|
@@ -197,23 +242,36 @@ python3 scripts/check_applied.py model.gguf
|
|
| 197 |
```
|
| 198 |
|
| 199 |
```
|
| 200 |
-
[chat_template.jinja]
|
| 201 |
terseness prompt ......... yes
|
| 202 |
keeps your system prompt . yes
|
|
|
|
| 203 |
|
| 204 |
-
[tokenizer_config.json]
|
| 205 |
terseness prompt ......... NO (found 0x)
|
|
|
|
|
|
|
| 206 |
|
| 207 |
*** THE TWO SOURCES DISAGREE ***
|
| 208 |
Recent transformers uses chat_template.jinja; oMLX and others read the
|
| 209 |
-
copy embedded in tokenizer_config.json. Right now those
|
| 210 |
-
you get depends on your runtime. Patch both to the same
|
| 211 |
```
|
| 212 |
|
| 213 |
That case — a fresh `.jinja` dropped in next to a stale embedded copy — is the most common way
|
| 214 |
this silently does nothing. It also warns if the template names a specific model, which happens
|
| 215 |
when the file was taken from a model repo rather than from here.
|
| 216 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 217 |
## Setting reasoning effort
|
| 218 |
|
| 219 |
By default there is no reasoning-effort instruction — you get the tuned terseness behavior and
|
|
@@ -244,6 +302,37 @@ Verified on both runtimes: with `chat_template_kwargs` the steering line renders
|
|
| 244 |
+38 tokens for `xhigh`, +26 for `low`; llama.cpp/minja `POST /apply-template` shows the same line);
|
| 245 |
with the bare top-level field it does not.
|
| 246 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 247 |
## What it doesn't do
|
| 248 |
|
| 249 |
- **It is not a fine-tune**, despite the `base_model_relation: finetune` tag — that is the closest
|
|
@@ -261,9 +350,15 @@ with the bare top-level field it does not.
|
|
| 261 |
## Credits
|
| 262 |
|
| 263 |
Everything structural here is [froggeric](https://huggingface.co/froggeric)'s work — the retention
|
| 264 |
-
fix, the tool-calling handling, the whole template. This repo adds a
|
| 265 |
-
|
| 266 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 267 |
|
| 268 |
Apache-2.0, matching upstream.
|
| 269 |
|
|
@@ -275,6 +370,6 @@ Apache-2.0, matching upstream.
|
|
| 275 |
author = {Saga Ishtardottir},
|
| 276 |
year = {2026},
|
| 277 |
url = {https://huggingface.co/peculiar-ragdoll/Qwen-Sharp-Chat-Templates},
|
| 278 |
-
note = {froggeric's fixed Qwen3.5/3.6/3.8 chat template with an always-on terseness system prompt (v22.
|
| 279 |
}
|
| 280 |
```
|
|
|
|
| 37 |
## Straight to the point
|
| 38 |
|
| 39 |
This is froggeric's [Qwen-Fixed-Chat-Templates](https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates)
|
| 40 |
+
`v22.3` with a force-appended system prompt spliced in, plus this repo's **`v22.3.1`** bugfix on top.
|
| 41 |
+
The base fixes issues, the addition makes it better, and `v22.3.1` makes the fast (thinking-off) path
|
| 42 |
coherent — see the changelog below.
|
| 43 |
|
| 44 |
+
> **`v22.3.1` (current — this repo's bugfix, rebased onto froggeric v22.3).** Three changes, all
|
| 45 |
+
> scoped to **thinking-off (fast) mode**; the thinking-**on** path is byte-identical to upstream
|
| 46 |
+
> v22.3 plus the terseness block, so anything already running with thinking on gets only upstream's
|
| 47 |
+
> fixes.
|
| 48 |
>
|
| 49 |
+
> 1. **Fast-mode `<think>` contradiction fixed.** With tools and thinking *off*, upstream still tells
|
| 50 |
+
> the model to put its reasoning inside `<think></think>` — while the generation prompt has already
|
| 51 |
+
> closed thinking. This gates every `<think>`-related tool-call instruction on thinking being on, so
|
| 52 |
+
> fast mode no longer asks for a block it can't open. (Tool-call *format* rules are untouched.)
|
| 53 |
+
> 2. **Terseness lead split by mode.** The single lead — *"Answer directly, after thinking"* — is
|
| 54 |
+
> incoherent when thinking is off, so thinking-off gets a terse-neutral *"Answer directly and
|
| 55 |
+
> concisely"* lead instead. The terseness core (the never/always rules) is identical in both.
|
| 56 |
+
> 3. **The last thinking reference on the fast path removed.** One tool-call rule still read *"output
|
| 57 |
+
> the `<tool_call>` block IMMEDIATELY **after thinking**"* in fast mode — the same contradiction as
|
| 58 |
+
> (1), surviving in a line the first fix didn't cover. Only the two-word fragment is gated, so
|
| 59 |
+
> thinking-off reads *"…IMMEDIATELY, with NO conversational text before it"* and thinking-on is
|
| 60 |
+
> unchanged to the byte. With this, a fast-mode prompt contains no reference to thinking at all.
|
| 61 |
>
|
| 62 |
+
> (1) and (2) carry over unchanged from `v22.1.1`; (3) is new in `v22.3.1`. None is fixed upstream as
|
| 63 |
+
> of v22.3.
|
| 64 |
+
>
|
| 65 |
+
> **Version-string break.** `v22.3.1` does **not** contain `v22.1` as a substring, so any checker
|
| 66 |
+
> matching on the old id stops matching. This project's re-embed scripts (`publish/retemplate_dirk.py`,
|
| 67 |
+
> `publish/retemplate_dirk_v3.py`) now read the expected version out of the template file at run time
|
| 68 |
+
> instead of hardcoding it, so the next rebase won't break them again. Already-published GGUF/MLX
|
| 69 |
+
> builds still carry v22.1.1 and are unaffected until deliberately re-templated.
|
| 70 |
+
|
| 71 |
+
> **What upstream added in v22.2 / v22.3 (and what it closed for us).** Rebasing picked up, verified
|
| 72 |
+
> by upstream's own suite:
|
| 73 |
+
>
|
| 74 |
+
> - **Long tool errors escalate again.** The old `content|length < 500` gate meant a multi-line
|
| 75 |
+
> traceback — the exact case where escalation matters — silently *never* escalated. v22.2+ replaces
|
| 76 |
+
> it with two tiers: structural signals (`"error":`, `"status": "error"`, a nonzero exit code, a
|
| 77 |
+
> real `Traceback (most recent call last):`) escalate at any payload size, while weak signals stay
|
| 78 |
+
> size-gated. This is one of the two issues this repo previously listed as open and deferred —
|
| 79 |
+
> **it is now fixed**, upstream, and better than the patch that regressed here.
|
| 80 |
+
> - **False retry loops on code search killed.** Grep hits containing `throw new Error(...)`,
|
| 81 |
+
> `console.error`, `logger.error` no longer count as tool failures.
|
| 82 |
+
> - **Multiple leading system/developer messages merge** into one system turn joined by blank lines,
|
| 83 |
+
> instead of only the first being treated as a system prompt.
|
| 84 |
+
> - **In-content reasoning extraction widened** to content that *starts* with `<think>`/`<thinking>`,
|
| 85 |
+
> and de-duplicated when `reasoning_content`/`thinking` is supplied alongside inline tags.
|
| 86 |
+
> - **Preserved assistant turns always render the `<think>` wrapper**, even when the thought was
|
| 87 |
+
> empty, so rendered history matches what was actually generated and the prefix cache stays valid.
|
| 88 |
+
> - **Tool arguments serialize correctly.** Booleans, nulls and numbers now go through `tojson`
|
| 89 |
+
> instead of `| string` (which emitted Python `True`/`None`); raw string args honour
|
| 90 |
+
> `max_tool_arg_chars`; JSON tool format no longer truncates tool responses.
|
| 91 |
+
> - **More effort aliases:** `off`, `max`, `ultracode`, `extreme`, with matching `<|think_…|>` tags.
|
| 92 |
+
>
|
| 93 |
+
> **Still open (so you don't over-trust it):** a literal `<|think_off|>` arriving in tool output still
|
| 94 |
+
> disables reasoning **if** your harness packs tool results into a `user` message — upstream's tag
|
| 95 |
+
> scanner reads `system`/`developer`/`user` roles, so a proper `tool`-role message is safe, and that is
|
| 96 |
+
> unchanged in v22.3. And a separately-reported *mid-answer* `<think>` tag remains unreproduced in our
|
| 97 |
+
> stack (2,863 corpus messages + 14 fresh llama.cpp generations, zero repros), with MTP speculative
|
| 98 |
+
> decoding the leading suspect; v22.3.1 does not target it.
|
| 99 |
+
|
| 100 |
+
> **v22.3 (upstream base).** Covers Qwen 3.8 alongside 3.5/3.6 and adds prompt-directed
|
| 101 |
+
> reasoning-effort steering (`none`/`minimal`/`low`/`medium`/`high`/`xhigh`, plus the aliases above)
|
| 102 |
+
> and inline `<|think_…|>` control tags. The default effort is `medium` — a neutral baseline that
|
| 103 |
+
> injects **no** steering line when the caller asks for nothing. (Earlier v22 forced `xhigh` by
|
| 104 |
+
> default; froggeric fixed that upstream, so this Sharp build no longer suppresses anything — out of
|
| 105 |
+
> the box you get the tuned terseness behavior and nothing else, exactly as v1.) An *explicit* effort
|
| 106 |
+
> still renders; pass it via `chat_template_kwargs` (a bare top-level `reasoning_effort` field is
|
| 107 |
+
> dropped by OpenAI-style servers before the template sees it):
|
| 108 |
>
|
| 109 |
> ```json
|
| 110 |
> {"messages": [...], "chat_template_kwargs": {"reasoning_effort": "low"}}
|
|
|
|
| 115 |
the **v1** template baked into those builds — that is the exact template embedded in those GGUF and
|
| 116 |
MLX builds (`template_version = "qwen3.6-froggeric-v21.3"`, terseness, no reasoning-effort steering),
|
| 117 |
and it lives here in [`archive/v1-qwen3.6-froggeric-v21.3/`](archive/v1-qwen3.6-froggeric-v21.3). The
|
| 118 |
+
`chat_template.jinja` at the root of this repo is the newest **v22.3.1** described above; drop it in to
|
| 119 |
move a model onto it. The template is published separately because it is the portable part — the
|
| 120 |
thing worth reusing is not tied to either model.
|
| 121 |
|
| 122 |
Superseded versions are kept verbatim under [`archive/`](archive): the **v1** froggeric-v21.3 build
|
| 123 |
+
(Dagger/Nail), the **v22.1** build in [`archive/v22.1-sharp/`](archive/v22.1-sharp), and the
|
| 124 |
+
immediately-prior **v22.1.1** build in [`archive/v22.1.1-sharp/`](archive/v22.1.1-sharp) — the last
|
| 125 |
+
one before the v22.3 rebase, and the version embedded in the published Dirk builds.
|
| 126 |
|
| 127 |
## What it changes
|
| 128 |
|
| 129 |
A terseness block, force-appended after your own system prompt. The **lead** now varies by thinking
|
| 130 |
+
mode (the v22.3.1 fix); the **core** never/always rules are identical on both paths, and the
|
| 131 |
+
thinking-on path is byte-identical to upstream v22.3 + the original terseness block.
|
| 132 |
|
| 133 |
```jinja
|
| 134 |
{%- if ns_state.thinking %}
|
|
|
|
| 152 |
Two things happen here: the `if/else` on `_sc` keeps **your own system prompt** — the terseness block
|
| 153 |
is appended after it, nothing you pass in is replaced; and the lead line matches the reasoning mode so
|
| 154 |
a fast-mode model isn't told to "answer after thinking" when it isn't thinking. Separately, the
|
| 155 |
+
tool-calling instructions gate every `<think>` reference — and the phrase *"IMMEDIATELY after
|
| 156 |
+
thinking"* — on thinking being on (the other half of the v22.3.1 fix). No effort-suppression is needed: v22.3 already defaults to `medium`, which injects no
|
| 157 |
+
reasoning-effort line unless you ask for one (earlier v22 forced `xhigh`; see the v22.3 note above).
|
| 158 |
An explicit `reasoning_effort` still renders.
|
| 159 |
|
| 160 |
## Impact
|
|
|
|
| 204 |
**llama.cpp at runtime, without touching the file** — pass it per-run instead:
|
| 205 |
|
| 206 |
```bash
|
| 207 |
+
llama-server -m model.gguf --chat-template-file chat_template.jinja --reasoning-format deepseek -ngl 99
|
| 208 |
llama-cli -m model.gguf --chat-template-file chat_template.jinja -ngl 99
|
| 209 |
```
|
| 210 |
|
|
|
|
| 214 |
`curl localhost:8080/props | jq -r .chat_template`, or render a prompt through
|
| 215 |
`POST /apply-template`.
|
| 216 |
|
| 217 |
+
Three caveats. `--jinja` is enabled by default in current llama.cpp, so you usually do not need
|
| 218 |
it — on older builds you do, and it must come *before* `--chat-template-file`. And the flag is
|
| 219 |
per-invocation: forget it once and you silently get the embedded template back. Rewriting the
|
| 220 |
GGUF with `gguf-new-metadata` is the durable version; the flag is right for trying it out or for
|
| 221 |
running one template across several models.
|
| 222 |
|
| 223 |
+
Third, `--reasoning-format deepseek` (shown on the server line; it is an API-response setting, so
|
| 224 |
+
it does nothing for `llama-cli`). It puts the model's `<think>` block in the OpenAI
|
| 225 |
+
`reasoning_content` field instead of leaving it inline in `content` — which is what keeps a coding
|
| 226 |
+
agent from stalling on raw thinking tokens mid-stream. **On current llama.cpp it is already a
|
| 227 |
+
no-op:** `--reasoning-format` defaults to `auto`, which the source defines as *"same as deepseek"*
|
| 228 |
+
— verified at build 9890 (`74976e1ae`), where `COMMON_REASONING_FORMAT_AUTO` appears in no
|
| 229 |
+
behavioural branch at all and every extraction site gates on `!= none`. Pass it anyway if you may
|
| 230 |
+
be on an older build. The setting that genuinely breaks agents is `--reasoning-format none`, which
|
| 231 |
+
leaves the tags inline — don't use it except to inspect raw output.
|
| 232 |
+
|
| 233 |
## Did it actually apply?
|
| 234 |
|
| 235 |
Point `check_applied.py` at a model directory or a `.gguf`. It finds every template source,
|
|
|
|
| 242 |
```
|
| 243 |
|
| 244 |
```
|
| 245 |
+
[chat_template.jinja] 28162 bytes
|
| 246 |
terseness prompt ......... yes
|
| 247 |
keeps your system prompt . yes
|
| 248 |
+
retains thinking* ........ yes
|
| 249 |
|
| 250 |
+
[tokenizer_config.json] 8952 bytes
|
| 251 |
terseness prompt ......... NO (found 0x)
|
| 252 |
+
keeps your system prompt . yes
|
| 253 |
+
retains thinking* ........ yes
|
| 254 |
|
| 255 |
*** THE TWO SOURCES DISAGREE ***
|
| 256 |
Recent transformers uses chat_template.jinja; oMLX and others read the
|
| 257 |
+
copy embedded in tokenizer_config.json. Right now those RENDER DIFFERENTLY,
|
| 258 |
+
so what you get depends on your runtime. Patch both to the same template.
|
| 259 |
```
|
| 260 |
|
| 261 |
That case — a fresh `.jinja` dropped in next to a stale embedded copy — is the most common way
|
| 262 |
this silently does nothing. It also warns if the template names a specific model, which happens
|
| 263 |
when the file was taken from a model repo rather than from here.
|
| 264 |
|
| 265 |
+
**It compares what the sources *render*, not how they are spelled.** That matters because the
|
| 266 |
+
documented way to patch both places is to paste `chat_template_oneline.txt` into
|
| 267 |
+
`tokenizer_config.json` — the minified form of the same template, byte-different by construction.
|
| 268 |
+
A text comparison flags that recommended state as broken; this one reports:
|
| 269 |
+
|
| 270 |
+
```
|
| 271 |
+
Both sources render the SAME prompts — whichever your runtime prefers,
|
| 272 |
+
you get the same behaviour (they differ only as full vs. minified text).
|
| 273 |
+
```
|
| 274 |
+
|
| 275 |
## Setting reasoning effort
|
| 276 |
|
| 277 |
By default there is no reasoning-effort instruction — you get the tuned terseness behavior and
|
|
|
|
| 302 |
+38 tokens for `xhigh`, +26 for `low`; llama.cpp/minja `POST /apply-template` shows the same line);
|
| 303 |
with the bare top-level field it does not.
|
| 304 |
|
| 305 |
+
## Tests
|
| 306 |
+
|
| 307 |
+
froggeric ships a test suite upstream; this repo vendors it under `scripts/` and runs it against the
|
| 308 |
+
Sharp template rather than a stock one, so the fork is held to upstream's own invariants.
|
| 309 |
+
|
| 310 |
+
| Script | Covers | v22.3.1 |
|
| 311 |
+
|---|---|:--:|
|
| 312 |
+
| `test_v22.py` | 100 cases — effort steering and aliases, inline `<\|think_…\|>` tags, tool-call wire formats, system merging, error escalation, vision parts | 100 / 100 |
|
| 313 |
+
| `test_v21.py` | 9 cases — the v21-era retention and rendering baseline | 9 / 9 |
|
| 314 |
+
| `fuzz_template.py` | property fuzzer, 9 invariants: render, oneline parity, tag balance, content, XML fidelity, JSON validity, warning precision, prefix stability, empty-think prefill | clean over 2,000 conversations |
|
| 315 |
+
|
| 316 |
+
```bash
|
| 317 |
+
pip install jinja2
|
| 318 |
+
python3 scripts/test_v22.py
|
| 319 |
+
python3 scripts/test_v21.py
|
| 320 |
+
python3 scripts/fuzz_template.py --cases 2000
|
| 321 |
+
```
|
| 322 |
+
|
| 323 |
+
`test_v22.py` carries one local change, marked in the file: a shim that strips Sharp's appended
|
| 324 |
+
terseness block before each assertion. Sixteen upstream tests pin the exact *end* of the system turn,
|
| 325 |
+
which is precisely where Sharp appends — without the shim they fail on a difference this repo makes on
|
| 326 |
+
purpose, and sixteen permanently-red tests would hide a real regression the next time upstream bumps.
|
| 327 |
+
The shim removes only the block Sharp adds; every other upstream assertion still runs against our
|
| 328 |
+
rendering. It keys off the terseness marker, so the same file scores a *pristine* upstream template
|
| 329 |
+
100/100 as well, which is how it was checked for being a genuine no-op. Run against the pre-rebase
|
| 330 |
+
v22.1.1 template it still reports 70/100 — it hides Sharp's intended divergence, not real breakage.
|
| 331 |
+
|
| 332 |
+
`scripts/verify_template.py` is this repo's own check and complements those: it re-fetches upstream
|
| 333 |
+
live, asserts the thinking-on path is still byte-identical to upstream-plus-terseness, and fails if
|
| 334 |
+
froggeric has moved past the base recorded in `BASE` — which is what caught the v22.1 → v22.3 drift.
|
| 335 |
+
|
| 336 |
## What it doesn't do
|
| 337 |
|
| 338 |
- **It is not a fine-tune**, despite the `base_model_relation: finetune` tag — that is the closest
|
|
|
|
| 350 |
## Credits
|
| 351 |
|
| 352 |
Everything structural here is [froggeric](https://huggingface.co/froggeric)'s work — the retention
|
| 353 |
+
fix, the tool-calling handling, the error-escalation tiers, the whole template. This repo adds a
|
| 354 |
+
system prompt, the two fast-mode fixes, and nothing else.
|
| 355 |
+
|
| 356 |
+
`scripts/test_v22.py`, `scripts/test_v21.py` and `scripts/fuzz_template.py` are froggeric's test
|
| 357 |
+
suite, vendored so this fork is measured against upstream's invariants; `test_v22.py` carries the
|
| 358 |
+
documented shim described under [Tests](#tests). `scripts/minify_jinja.py` is froggeric's with one
|
| 359 |
+
patch: it now preserves newlines inside `{% set %}…{% endset %}` blocks, which upstream's template
|
| 360 |
+
doesn't contain and this one does. `scripts/check_applied.py` and `scripts/verify_template.py` are
|
| 361 |
+
this repo's.
|
| 362 |
|
| 363 |
Apache-2.0, matching upstream.
|
| 364 |
|
|
|
|
| 370 |
author = {Saga Ishtardottir},
|
| 371 |
year = {2026},
|
| 372 |
url = {https://huggingface.co/peculiar-ragdoll/Qwen-Sharp-Chat-Templates},
|
| 373 |
+
note = {froggeric's fixed Qwen3.5/3.6/3.8 chat template with an always-on terseness system prompt (v22.3.1)}
|
| 374 |
}
|
| 375 |
```
|
archive/v22.1.1-sharp/chat_template.jinja
ADDED
|
@@ -0,0 +1,379 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{%- set template_version = "qwen3.8-froggeric-v22.1.1" %}
|
| 2 |
+
{%- set _tool_format = tool_call_format if tool_call_format is defined else 'xml' %}
|
| 3 |
+
{%- set image_count = namespace(value=0) %}
|
| 4 |
+
{%- set video_count = namespace(value=0) %}
|
| 5 |
+
{%- set add_vision_id = add_vision_id if add_vision_id is defined else false %}
|
| 6 |
+
{%- set enable_thinking = enable_thinking if enable_thinking is defined else true %}
|
| 7 |
+
{%- set auto_disable_thinking_with_tools = auto_disable_thinking_with_tools if auto_disable_thinking_with_tools is defined else false %}
|
| 8 |
+
{%- if preserve_reasoning is defined and preserve_reasoning is not none %}
|
| 9 |
+
{%- set _preserve_thinking = preserve_reasoning %}
|
| 10 |
+
{%- elif preserve_thinking is defined and preserve_thinking is not none %}
|
| 11 |
+
{%- set _preserve_thinking = preserve_thinking %}
|
| 12 |
+
{%- else %}
|
| 13 |
+
{%- set _preserve_thinking = true %}
|
| 14 |
+
{%- endif %}
|
| 15 |
+
{%- set max_tool_arg_chars = max_tool_arg_chars if max_tool_arg_chars is defined else 0 %}
|
| 16 |
+
{%- set max_tool_response_chars = max_tool_response_chars if max_tool_response_chars is defined else 0 %}
|
| 17 |
+
{%- set _has_tools = (tools is defined and tools and tools is iterable and tools is not mapping) %}
|
| 18 |
+
{%- set _effort_raw = (reasoning_effort | string | lower) if reasoning_effort is defined and reasoning_effort is not none else 'medium' %}
|
| 19 |
+
{%- set _initial_thinking = enable_thinking %}
|
| 20 |
+
{%- set _initial_effort = 'medium' %}
|
| 21 |
+
{%- if _effort_raw == 'none' %}
|
| 22 |
+
{%- set _initial_thinking = false %}
|
| 23 |
+
{%- set _initial_effort = 'medium' %}
|
| 24 |
+
{%- elif _effort_raw == 'minimal' or _effort_raw == 'low' %}
|
| 25 |
+
{%- set _initial_effort = 'low' %}
|
| 26 |
+
{%- elif _effort_raw == 'high' or _effort_raw == 'xhigh' or _effort_raw == 'max' %}
|
| 27 |
+
{%- set _initial_effort = 'xhigh' %}
|
| 28 |
+
{%- else %}
|
| 29 |
+
{%- set _initial_effort = 'medium' %}
|
| 30 |
+
{%- endif %}
|
| 31 |
+
{%- set ns_state = namespace(thinking=_initial_thinking, effort=_initial_effort) %}
|
| 32 |
+
{%- if auto_disable_thinking_with_tools and _has_tools %}
|
| 33 |
+
{%- set ns_state.thinking = false %}
|
| 34 |
+
{%- endif %}
|
| 35 |
+
{%- for msg in messages %}
|
| 36 |
+
{%- if msg.role == 'system' or msg.role == 'developer' or msg.role == 'user' %}
|
| 37 |
+
{%- if msg.content is string %}
|
| 38 |
+
{%- if '<|think_off|>' in msg.content %}
|
| 39 |
+
{%- set ns_state.thinking = false %}
|
| 40 |
+
{%- elif '<|think_on|>' in msg.content %}
|
| 41 |
+
{%- set ns_state.thinking = true %}
|
| 42 |
+
{%- elif '<|think_xhigh|>' in msg.content or '<|think_high|>' in msg.content %}
|
| 43 |
+
{%- set ns_state.thinking = true %}
|
| 44 |
+
{%- set ns_state.effort = 'xhigh' %}
|
| 45 |
+
{%- elif '<|think_low|>' in msg.content or '<|think_minimal|>' in msg.content %}
|
| 46 |
+
{%- set ns_state.thinking = true %}
|
| 47 |
+
{%- set ns_state.effort = 'low' %}
|
| 48 |
+
{%- elif '<|think_medium|>' in msg.content %}
|
| 49 |
+
{%- set ns_state.thinking = true %}
|
| 50 |
+
{%- set ns_state.effort = 'medium' %}
|
| 51 |
+
{%- endif %}
|
| 52 |
+
{%- elif msg.content is iterable and msg.content is not mapping %}
|
| 53 |
+
{%- for item in msg.content %}
|
| 54 |
+
{%- if item is string %}
|
| 55 |
+
{%- set _item_text = item %}
|
| 56 |
+
{%- elif item is mapping and 'text' in item and item.text is string %}
|
| 57 |
+
{%- set _item_text = item.text %}
|
| 58 |
+
{%- else %}
|
| 59 |
+
{%- set _item_text = '' %}
|
| 60 |
+
{%- endif %}
|
| 61 |
+
{%- if _item_text %}
|
| 62 |
+
{%- if '<|think_off|>' in _item_text %}
|
| 63 |
+
{%- set ns_state.thinking = false %}
|
| 64 |
+
{%- elif '<|think_on|>' in _item_text %}
|
| 65 |
+
{%- set ns_state.thinking = true %}
|
| 66 |
+
{%- elif '<|think_xhigh|>' in _item_text or '<|think_high|>' in _item_text %}
|
| 67 |
+
{%- set ns_state.thinking = true %}
|
| 68 |
+
{%- set ns_state.effort = 'xhigh' %}
|
| 69 |
+
{%- elif '<|think_low|>' in _item_text or '<|think_minimal|>' in _item_text %}
|
| 70 |
+
{%- set ns_state.thinking = true %}
|
| 71 |
+
{%- set ns_state.effort = 'low' %}
|
| 72 |
+
{%- elif '<|think_medium|>' in _item_text %}
|
| 73 |
+
{%- set ns_state.thinking = true %}
|
| 74 |
+
{%- set ns_state.effort = 'medium' %}
|
| 75 |
+
{%- endif %}
|
| 76 |
+
{%- endif %}
|
| 77 |
+
{%- endfor %}
|
| 78 |
+
{%- endif %}
|
| 79 |
+
{%- endif %}
|
| 80 |
+
{%- endfor %}
|
| 81 |
+
{%- set reasoning_instructions = '' %}
|
| 82 |
+
{%- if ns_state.thinking %}
|
| 83 |
+
{%- if ns_state.effort == 'xhigh' %}
|
| 84 |
+
{%- set reasoning_instructions = 'Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.' %}
|
| 85 |
+
{%- elif ns_state.effort == 'low' %}
|
| 86 |
+
{%- set reasoning_instructions = 'Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration.' %}
|
| 87 |
+
{%- endif %}
|
| 88 |
+
{%- endif %}
|
| 89 |
+
{%- macro render_content(content, do_vision_count, is_system_content=false) %}
|
| 90 |
+
{%- if content is string %}
|
| 91 |
+
{{- content }}
|
| 92 |
+
{%- elif content is iterable and content is not mapping %}
|
| 93 |
+
{%- for item in content %}
|
| 94 |
+
{%- if item is mapping %}
|
| 95 |
+
{%- if item.type == 'image' or 'image' in item or 'image_url' in item %}
|
| 96 |
+
{%- if is_system_content %}
|
| 97 |
+
{{- raise_exception('System message cannot contain images.') }}
|
| 98 |
+
{%- endif %}
|
| 99 |
+
{%- if do_vision_count %}
|
| 100 |
+
{%- set image_count.value = image_count.value + 1 %}
|
| 101 |
+
{%- endif %}
|
| 102 |
+
{%- if add_vision_id %}
|
| 103 |
+
{{- 'Picture ' ~ image_count.value ~ ': ' }}
|
| 104 |
+
{%- endif %}
|
| 105 |
+
{{- '<|vision_start|><|image_pad|><|vision_end|>' }}
|
| 106 |
+
{%- elif item.type == 'video' or 'video' in item %}
|
| 107 |
+
{%- if is_system_content %}
|
| 108 |
+
{{- raise_exception('System message cannot contain videos.') }}
|
| 109 |
+
{%- endif %}
|
| 110 |
+
{%- if do_vision_count %}
|
| 111 |
+
{%- set video_count.value = video_count.value + 1 %}
|
| 112 |
+
{%- endif %}
|
| 113 |
+
{%- if add_vision_id %}
|
| 114 |
+
{{- 'Video ' ~ video_count.value ~ ': ' }}
|
| 115 |
+
{%- endif %}
|
| 116 |
+
{{- '<|vision_start|><|video_pad|><|vision_end|>' }}
|
| 117 |
+
{%- elif 'text' in item %}
|
| 118 |
+
{{- item.text }}
|
| 119 |
+
{%- else %}
|
| 120 |
+
{{- raise_exception('Unexpected item type in content.') }}
|
| 121 |
+
{%- endif %}
|
| 122 |
+
{%- else %}
|
| 123 |
+
{{- item | string }}
|
| 124 |
+
{%- endif %}
|
| 125 |
+
{%- endfor %}
|
| 126 |
+
{%- elif content is none or content is undefined %}
|
| 127 |
+
{{- '' }}
|
| 128 |
+
{%- else %}
|
| 129 |
+
{{- raise_exception('Unexpected content type.') }}
|
| 130 |
+
{%- endif %}
|
| 131 |
+
{%- endmacro %}
|
| 132 |
+
{%- if not messages %}
|
| 133 |
+
{{- raise_exception('No messages provided.') }}
|
| 134 |
+
{%- endif %}
|
| 135 |
+
{%- set _first_role = messages[0].role %}
|
| 136 |
+
{%- if _first_role == 'system' or _first_role == 'developer' %}
|
| 137 |
+
{%- set _sys_msg = messages[0] %}
|
| 138 |
+
{%- set _msgs = messages[1:] %}
|
| 139 |
+
{%- else %}
|
| 140 |
+
{%- set _sys_msg = none %}
|
| 141 |
+
{%- set _msgs = messages %}
|
| 142 |
+
{%- endif %}
|
| 143 |
+
{%- set _sc = '' %}
|
| 144 |
+
{%- if _sys_msg is not none %}
|
| 145 |
+
{%- set _sc = render_content(_sys_msg.content, false, true) | trim %}
|
| 146 |
+
{%- if '<|think_off|>' in _sc %}{%- set _sc = _sc.split('<|think_off|>') | join('') | trim %}{%- endif %}
|
| 147 |
+
{%- if '<|think_on|>' in _sc %}{%- set _sc = _sc.split('<|think_on|>') | join('') | trim %}{%- endif %}
|
| 148 |
+
{%- if '<|think_xhigh|>' in _sc %}{%- set _sc = _sc.split('<|think_xhigh|>') | join('') | trim %}{%- endif %}
|
| 149 |
+
{%- if '<|think_high|>' in _sc %}{%- set _sc = _sc.split('<|think_high|>') | join('') | trim %}{%- endif %}
|
| 150 |
+
{%- if '<|think_medium|>' in _sc %}{%- set _sc = _sc.split('<|think_medium|>') | join('') | trim %}{%- endif %}
|
| 151 |
+
{%- if '<|think_low|>' in _sc %}{%- set _sc = _sc.split('<|think_low|>') | join('') | trim %}{%- endif %}
|
| 152 |
+
{%- if '<|think_minimal|>' in _sc %}{%- set _sc = _sc.split('<|think_minimal|>') | join('') | trim %}{%- endif %}
|
| 153 |
+
{%- endif %}
|
| 154 |
+
{%- if ns_state.thinking %}
|
| 155 |
+
{%- set _terse_lead = 'Answer directly, after thinking. Lead with the answer, then only what it needs to be correct and usable.' %}
|
| 156 |
+
{%- else %}
|
| 157 |
+
{%- set _terse_lead = 'Answer directly and concisely. Give the answer with only what it needs to be correct and usable.' %}
|
| 158 |
+
{%- endif %}
|
| 159 |
+
{%- set _terse_core %}
|
| 160 |
+
Never: open with preamble or pleasantries; restate the question; add filler transitions; hedge with niceties; or repeat a point you've already made.
|
| 161 |
+
Always: keep essential steps, caveats, uncertainties, and specifics — never drop correctness or a needed warning for brevity. Keep the final answer lean. Use the least structure that conveys it (plain prose when short; lists or code only when they earn their place). If genuinely uncertain, say so and explain why — never omit uncertainty for the sake of brevity.
|
| 162 |
+
If a user request is genuinely ambiguous, ask a sharp question, don't guess.
|
| 163 |
+
{%- endset %}
|
| 164 |
+
{%- set _terse = _terse_lead ~ '\n' ~ (_terse_core | trim) %}
|
| 165 |
+
{%- if not _sc %}
|
| 166 |
+
{%- set _sc = _terse | trim %}
|
| 167 |
+
{%- else %}
|
| 168 |
+
{%- set _sc = (_sc | trim) ~ '\n\n' ~ (_terse | trim) %}
|
| 169 |
+
{%- endif %}
|
| 170 |
+
{%- if _has_tools %}
|
| 171 |
+
{{- '<|im_start|>system\n' }}
|
| 172 |
+
{%- if reasoning_instructions %}
|
| 173 |
+
{{- reasoning_instructions + '\n\n' }}
|
| 174 |
+
{%- endif %}
|
| 175 |
+
{{- '# Tools\n\nYou have access to the following functions:\n\n<tools>' }}
|
| 176 |
+
{%- for tool in tools %}
|
| 177 |
+
{{- '\n' }}
|
| 178 |
+
{{- tool | tojson }}
|
| 179 |
+
{%- endfor %}
|
| 180 |
+
{{- '\n</tools>' }}
|
| 181 |
+
{%- if _tool_format == 'json' %}
|
| 182 |
+
{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n' }}{%- if ns_state.thinking %}{{- '<think>\nBrief explanation of tool call\n</think>\n' }}{%- endif %}{{- '<tool_call>\n{"name": "example_function_name", "arguments": {"example_parameter_1": "value_1", "example_parameter_2": "This is the value for the second parameter"}}\n</tool_call>\n\n<IMPORTANT>\nReminder:\n' }}{%- if ns_state.thinking %}{{- '- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n' }}{%- endif %}{{- '- Function calls MUST follow the specified format: a single JSON object with "name" and "arguments" keys inside <tool_call></tool_call> XML tags.\n- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after thinking, with NO conversational text before it.\n- The <tool_call> tag MUST be at the very beginning of a new line, with NO spaces or indentation before it.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}
|
| 183 |
+
{%- else %}
|
| 184 |
+
{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n' }}{%- if ns_state.thinking %}{{- '<think>\nBrief explanation of tool call\n</think>\n' }}{%- endif %}{{- '<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n' }}{%- if ns_state.thinking %}{{- '- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n' }}{%- endif %}{{- '- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags.\n- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after thinking, with NO conversational text before it.\n- The <tool_call> and <function> tags MUST be at the very beginning of a new line, with NO spaces or indentation before them.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}
|
| 185 |
+
{%- endif %}
|
| 186 |
+
{%- if _sc %}
|
| 187 |
+
{{- '\n\n' + _sc }}
|
| 188 |
+
{%- endif %}
|
| 189 |
+
{{- '<|im_end|>\n' }}
|
| 190 |
+
{%- else %}
|
| 191 |
+
{%- if _sc %}
|
| 192 |
+
{{- '<|im_start|>system\n' + (reasoning_instructions + '\n\n' if reasoning_instructions else '') + _sc + '<|im_end|>\n' }}
|
| 193 |
+
{%- elif reasoning_instructions %}
|
| 194 |
+
{{- '<|im_start|>system\n' + reasoning_instructions + '<|im_end|>\n' }}
|
| 195 |
+
{%- endif %}
|
| 196 |
+
{%- endif %}
|
| 197 |
+
{%- set _last_idx = _msgs | length - 1 %}
|
| 198 |
+
{%- set ns = namespace(multi_step_tool=true, last_query_index=_last_idx) %}
|
| 199 |
+
{%- for message in _msgs[::-1] %}
|
| 200 |
+
{%- set index = (_msgs | length - 1) - loop.index0 %}
|
| 201 |
+
{%- if ns.multi_step_tool and message.role == 'user' %}
|
| 202 |
+
{%- set _rc = render_content(message.content, false) | trim %}
|
| 203 |
+
{%- if not (_rc.startswith('<tool_response>') and _rc.endswith('</tool_response>')) %}
|
| 204 |
+
{%- set ns.multi_step_tool = false %}
|
| 205 |
+
{%- set ns.last_query_index = index %}
|
| 206 |
+
{%- endif %}
|
| 207 |
+
{%- endif %}
|
| 208 |
+
{%- endfor %}
|
| 209 |
+
{%- if ns.multi_step_tool %}
|
| 210 |
+
{%- if _last_idx > 50 %}
|
| 211 |
+
{%- set ns.last_query_index = _last_idx %}
|
| 212 |
+
{%- else %}
|
| 213 |
+
{%- set ns.last_query_index = 0 %}
|
| 214 |
+
{%- endif %}
|
| 215 |
+
{%- endif %}
|
| 216 |
+
{%- set ns2 = namespace(prev_role='', consecutive_failures=0) %}
|
| 217 |
+
{%- for message in _msgs %}
|
| 218 |
+
{%- set is_system = (message.role == "system" or message.role == "developer") %}
|
| 219 |
+
{%- set content = render_content(message.content, true, is_system) | trim %}
|
| 220 |
+
{%- if is_system or message.role == 'user' %}
|
| 221 |
+
{%- if '<|think_off|>' in content %}{%- set content = content.split('<|think_off|>') | join('') | trim %}{%- endif %}
|
| 222 |
+
{%- if '<|think_on|>' in content %}{%- set content = content.split('<|think_on|>') | join('') | trim %}{%- endif %}
|
| 223 |
+
{%- if '<|think_xhigh|>' in content %}{%- set content = content.split('<|think_xhigh|>') | join('') | trim %}{%- endif %}
|
| 224 |
+
{%- if '<|think_high|>' in content %}{%- set content = content.split('<|think_high|>') | join('') | trim %}{%- endif %}
|
| 225 |
+
{%- if '<|think_medium|>' in content %}{%- set content = content.split('<|think_medium|>') | join('') | trim %}{%- endif %}
|
| 226 |
+
{%- if '<|think_low|>' in content %}{%- set content = content.split('<|think_low|>') | join('') | trim %}{%- endif %}
|
| 227 |
+
{%- if '<|think_minimal|>' in content %}{%- set content = content.split('<|think_minimal|>') | join('') | trim %}{%- endif %}
|
| 228 |
+
{%- endif %}
|
| 229 |
+
{%- if is_system %}
|
| 230 |
+
{{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
|
| 231 |
+
{%- elif message.role == 'user' %}
|
| 232 |
+
{%- set ns2.consecutive_failures = 0 %}
|
| 233 |
+
{{- '<|im_start|>user\n' + content + '<|im_end|>\n' }}
|
| 234 |
+
{%- elif message.role == 'assistant' %}
|
| 235 |
+
{%- set reasoning_content = '' %}
|
| 236 |
+
{%- if message.reasoning_content is defined and message.reasoning_content is not none %}
|
| 237 |
+
{%- if message.reasoning_content is string %}
|
| 238 |
+
{%- set reasoning_content = message.reasoning_content %}
|
| 239 |
+
{%- else %}
|
| 240 |
+
{%- set reasoning_content = message.reasoning_content | string %}
|
| 241 |
+
{%- endif %}
|
| 242 |
+
{%- elif message.thinking is defined and message.thinking is not none %}
|
| 243 |
+
{%- if message.thinking is string %}
|
| 244 |
+
{%- set reasoning_content = message.thinking %}
|
| 245 |
+
{%- else %}
|
| 246 |
+
{%- set reasoning_content = message.thinking | string %}
|
| 247 |
+
{%- endif %}
|
| 248 |
+
{%- else %}
|
| 249 |
+
{%- set _think_end = '' %}
|
| 250 |
+
{%- if content.startswith('</think>') %}
|
| 251 |
+
{%- set _think_end = '</think>' %}
|
| 252 |
+
{%- elif content.startswith('</thinking>') %}
|
| 253 |
+
{%- set _think_end = '</thinking>' %}
|
| 254 |
+
{%- elif '\n</think>' in content %}
|
| 255 |
+
{%- set _think_end = '\n</think>' %}
|
| 256 |
+
{%- elif '\n</thinking>' in content %}
|
| 257 |
+
{%- set _think_end = '\n</thinking>' %}
|
| 258 |
+
{%- elif '\n</ think>' in content %}
|
| 259 |
+
{%- set _think_end = '\n</ think>' %}
|
| 260 |
+
{%- elif '\n</think >' in content %}
|
| 261 |
+
{%- set _think_end = '\n</think >' %}
|
| 262 |
+
{%- endif %}
|
| 263 |
+
{%- if _think_end %}
|
| 264 |
+
{%- if 'thinking' in _think_end %}
|
| 265 |
+
{%- set _think_start = '<thinking>' %}
|
| 266 |
+
{%- else %}
|
| 267 |
+
{%- set _think_start = '<think>' %}
|
| 268 |
+
{%- endif %}
|
| 269 |
+
{%- set reasoning_content = content.split(_think_end)[0].rstrip('\n') %}
|
| 270 |
+
{%- if _think_start in reasoning_content %}
|
| 271 |
+
{%- set reasoning_content = reasoning_content.split(_think_start)[-1].lstrip('\n') %}
|
| 272 |
+
{%- endif %}
|
| 273 |
+
{%- set content = content.split(_think_end)[-1].lstrip('\n') %}
|
| 274 |
+
{%- endif %}
|
| 275 |
+
{%- endif %}
|
| 276 |
+
{%- set reasoning_content = reasoning_content | trim %}
|
| 277 |
+
{%- if (_preserve_thinking or loop.index0 > ns.last_query_index) and reasoning_content %}
|
| 278 |
+
{{- '<|im_start|>assistant\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
|
| 279 |
+
{%- else %}
|
| 280 |
+
{{- '<|im_start|>assistant\n' + content }}
|
| 281 |
+
{%- endif %}
|
| 282 |
+
{%- if message.tool_calls is defined and message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
|
| 283 |
+
{%- for tool_call in message.tool_calls %}
|
| 284 |
+
{%- if tool_call.function is defined and tool_call.function is not none %}
|
| 285 |
+
{%- set tc = tool_call.function %}
|
| 286 |
+
{%- else %}
|
| 287 |
+
{%- set tc = tool_call %}
|
| 288 |
+
{%- endif %}
|
| 289 |
+
{%- set tc_name = tc.name if (tc.name is defined and tc.name is not none) else '' %}
|
| 290 |
+
{%- if _tool_format == 'json' %}
|
| 291 |
+
{%- if not loop.first or content | trim %}
|
| 292 |
+
{{- '\n\n' }}
|
| 293 |
+
{%- endif %}
|
| 294 |
+
{%- set _args = '{}' %}
|
| 295 |
+
{%- if tc.arguments is defined and tc.arguments is not none %}
|
| 296 |
+
{%- if tc.arguments is mapping %}
|
| 297 |
+
{%- set _args = tc.arguments | tojson %}
|
| 298 |
+
{%- elif tc.arguments is string and tc.arguments %}
|
| 299 |
+
{%- set _args = tc.arguments %}
|
| 300 |
+
{%- endif %}
|
| 301 |
+
{%- endif %}
|
| 302 |
+
{{- '<tool_call>\n{"name": ' }}{{- tc_name | tojson }}{{- ', "arguments": ' }}{{- _args }}{{- '}\n</tool_call>' }}
|
| 303 |
+
{%- else %}
|
| 304 |
+
{%- if loop.first %}
|
| 305 |
+
{%- if content | trim %}
|
| 306 |
+
{{- '\n\n<tool_call>\n<function=' + tc_name + '>\n' }}
|
| 307 |
+
{%- else %}
|
| 308 |
+
{{- '<tool_call>\n<function=' + tc_name + '>\n' }}
|
| 309 |
+
{%- endif %}
|
| 310 |
+
{%- else %}
|
| 311 |
+
{{- '\n\n<tool_call>\n<function=' + tc_name + '>\n' }}
|
| 312 |
+
{%- endif %}
|
| 313 |
+
{%- if tc.arguments is defined and tc.arguments is not none %}
|
| 314 |
+
{%- if tc.arguments is mapping %}
|
| 315 |
+
{%- for args_name, args_value in tc.arguments.items() %}
|
| 316 |
+
{{- '<parameter=' + args_name + '>\n' }}
|
| 317 |
+
{%- if args_value is mapping or (args_value is sequence and args_value is not string) %}
|
| 318 |
+
{%- set _av = args_value | tojson %}
|
| 319 |
+
{%- else %}
|
| 320 |
+
{%- set _av = args_value | string %}
|
| 321 |
+
{%- endif %}
|
| 322 |
+
{%- if max_tool_arg_chars > 0 and _av | length > max_tool_arg_chars %}
|
| 323 |
+
{{- _av[:max_tool_arg_chars] + '\n[TRUNCATED - original length ' ~ (_av | length | string) ~ ' chars]' }}
|
| 324 |
+
{%- else %}
|
| 325 |
+
{{- _av }}
|
| 326 |
+
{%- endif %}
|
| 327 |
+
{{- '\n</parameter>\n' }}
|
| 328 |
+
{%- endfor %}
|
| 329 |
+
{%- elif tc.arguments is string and tc.arguments %}
|
| 330 |
+
{{- tc.arguments }}
|
| 331 |
+
{%- endif %}
|
| 332 |
+
{%- endif %}
|
| 333 |
+
{{- '</function>\n</tool_call>' }}
|
| 334 |
+
{%- endif %}
|
| 335 |
+
{%- endfor %}
|
| 336 |
+
{%- endif %}
|
| 337 |
+
{{- '<|im_end|>\n' }}
|
| 338 |
+
{%- elif message.role == 'tool' %}
|
| 339 |
+
{%- set _content_lower = content | lower %}
|
| 340 |
+
{%- set _content_head = _content_lower[:80] %}
|
| 341 |
+
{%- if content | length < 500 and '$ ' not in content and 'took ' not in _content_lower and ('"error":' in _content_head or 'error:' in _content_head or 'err!' in _content_head or 'fatal:' in _content_head or 'exception:' in _content_head or 'traceback' in _content_head or 'command not found' in _content_head or 'invalid syntax' in _content_head or 'failed to' in _content_head) %}
|
| 342 |
+
{%- set ns2.consecutive_failures = ns2.consecutive_failures + 1 %}
|
| 343 |
+
{%- else %}
|
| 344 |
+
{%- set ns2.consecutive_failures = 0 %}
|
| 345 |
+
{%- endif %}
|
| 346 |
+
{%- if ns2.prev_role != 'tool' %}
|
| 347 |
+
{{- '<|im_start|>user' }}
|
| 348 |
+
{%- endif %}
|
| 349 |
+
{%- if max_tool_response_chars > 0 and content | length > max_tool_response_chars %}
|
| 350 |
+
{%- set content = content[:max_tool_response_chars] + '\n[TRUNCATED - original length ' ~ (content | length | string) ~ ' chars]' %}
|
| 351 |
+
{%- endif %}
|
| 352 |
+
{{- '\n<tool_response>\n' + content }}
|
| 353 |
+
{%- if ns2.consecutive_failures >= 2 %}
|
| 354 |
+
{{- '\n\n⚠️ SYSTEM WARNING: ' ~ ns2.consecutive_failures ~ ' consecutive tool errors detected. Your previous approach is incorrect. You MUST use a fundamentally different approach or corrected arguments.' }}
|
| 355 |
+
{%- elif ns2.consecutive_failures == 1 %}
|
| 356 |
+
{{- '\n\n⚠️ SYSTEM WARNING: The previous tool call returned an error. Diagnose the failure and retry with completely corrected arguments.' }}
|
| 357 |
+
{%- endif %}
|
| 358 |
+
{{- '\n</tool_response>' }}
|
| 359 |
+
{%- if loop.last %}
|
| 360 |
+
{{- '<|im_end|>\n' }}
|
| 361 |
+
{%- else %}
|
| 362 |
+
{%- set _next_role = _msgs[loop.index0 + 1].role %}
|
| 363 |
+
{%- if _next_role != 'tool' %}
|
| 364 |
+
{{- '<|im_end|>\n' }}
|
| 365 |
+
{%- endif %}
|
| 366 |
+
{%- endif %}
|
| 367 |
+
{%- else %}
|
| 368 |
+
{{- '<|im_start|>user\n[' + message.role + ']: ' + content + '<|im_end|>\n' }}
|
| 369 |
+
{%- endif %}
|
| 370 |
+
{%- set ns2.prev_role = message.role %}
|
| 371 |
+
{%- endfor %}
|
| 372 |
+
{%- if add_generation_prompt %}
|
| 373 |
+
{{- '<|im_start|>assistant\n' }}
|
| 374 |
+
{%- if not ns_state.thinking %}
|
| 375 |
+
{{- '<think>\n\n</think>\n\n' }}
|
| 376 |
+
{%- else %}
|
| 377 |
+
{{- '<think>\n' }}
|
| 378 |
+
{%- endif %}
|
| 379 |
+
{%- endif %}
|
archive/v22.1.1-sharp/chat_template_oneline.txt
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{%- set template_version = "qwen3.8-froggeric-v22.1.1" %}{%- set _tool_format = tool_call_format if tool_call_format is defined else 'xml' %}{%- set image_count = namespace(value=0) %}{%- set video_count = namespace(value=0) %}{%- set add_vision_id = add_vision_id if add_vision_id is defined else false %}{%- set enable_thinking = enable_thinking if enable_thinking is defined else true %}{%- set auto_disable_thinking_with_tools = auto_disable_thinking_with_tools if auto_disable_thinking_with_tools is defined else false %}{%- if preserve_reasoning is defined and preserve_reasoning is not none %}{%- set _preserve_thinking = preserve_reasoning %}{%- elif preserve_thinking is defined and preserve_thinking is not none %}{%- set _preserve_thinking = preserve_thinking %}{%- else %}{%- set _preserve_thinking = true %}{%- endif %}{%- set max_tool_arg_chars = max_tool_arg_chars if max_tool_arg_chars is defined else 0 %}{%- set max_tool_response_chars = max_tool_response_chars if max_tool_response_chars is defined else 0 %}{%- set _has_tools = (tools is defined and tools and tools is iterable and tools is not mapping) %}{%- set _effort_raw = (reasoning_effort | string | lower) if reasoning_effort is defined and reasoning_effort is not none else 'medium' %}{%- set _initial_thinking = enable_thinking %}{%- set _initial_effort = 'medium' %}{%- if _effort_raw == 'none' %}{%- set _initial_thinking = false %}{%- set _initial_effort = 'medium' %}{%- elif _effort_raw == 'minimal' or _effort_raw == 'low' %}{%- set _initial_effort = 'low' %}{%- elif _effort_raw == 'high' or _effort_raw == 'xhigh' or _effort_raw == 'max' %}{%- set _initial_effort = 'xhigh' %}{%- else %}{%- set _initial_effort = 'medium' %}{%- endif %}{%- set ns_state = namespace(thinking=_initial_thinking, effort=_initial_effort) %}{%- if auto_disable_thinking_with_tools and _has_tools %}{%- set ns_state.thinking = false %}{%- endif %}{%- for msg in messages %}{%- if msg.role == 'system' or msg.role == 'developer' or msg.role == 'user' %}{%- if msg.content is string %}{%- if '<|think_off|>' in msg.content %}{%- set ns_state.thinking = false %}{%- elif '<|think_on|>' in msg.content %}{%- set ns_state.thinking = true %}{%- elif '<|think_xhigh|>' in msg.content or '<|think_high|>' in msg.content %}{%- set ns_state.thinking = true %}{%- set ns_state.effort = 'xhigh' %}{%- elif '<|think_low|>' in msg.content or '<|think_minimal|>' in msg.content %}{%- set ns_state.thinking = true %}{%- set ns_state.effort = 'low' %}{%- elif '<|think_medium|>' in msg.content %}{%- set ns_state.thinking = true %}{%- set ns_state.effort = 'medium' %}{%- endif %}{%- elif msg.content is iterable and msg.content is not mapping %}{%- for item in msg.content %}{%- if item is string %}{%- set _item_text = item %}{%- elif item is mapping and 'text' in item and item.text is string %}{%- set _item_text = item.text %}{%- else %}{%- set _item_text = '' %}{%- endif %}{%- if _item_text %}{%- if '<|think_off|>' in _item_text %}{%- set ns_state.thinking = false %}{%- elif '<|think_on|>' in _item_text %}{%- set ns_state.thinking = true %}{%- elif '<|think_xhigh|>' in _item_text or '<|think_high|>' in _item_text %}{%- set ns_state.thinking = true %}{%- set ns_state.effort = 'xhigh' %}{%- elif '<|think_low|>' in _item_text or '<|think_minimal|>' in _item_text %}{%- set ns_state.thinking = true %}{%- set ns_state.effort = 'low' %}{%- elif '<|think_medium|>' in _item_text %}{%- set ns_state.thinking = true %}{%- set ns_state.effort = 'medium' %}{%- endif %}{%- endif %}{%- endfor %}{%- endif %}{%- endif %}{%- endfor %}{%- set reasoning_instructions = '' %}{%- if ns_state.thinking %}{%- if ns_state.effort == 'xhigh' %}{%- set reasoning_instructions = 'Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.' %}{%- elif ns_state.effort == 'low' %}{%- set reasoning_instructions = 'Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration.' %}{%- endif %}{%- endif %}{%- macro render_content(content, do_vision_count, is_system_content=false) %}{%- if content is string %}{{- content }}{%- elif content is iterable and content is not mapping %}{%- for item in content %}{%- if item is mapping %}{%- if item.type == 'image' or 'image' in item or 'image_url' in item %}{%- if is_system_content %}{{- raise_exception('System message cannot contain images.') }}{%- endif %}{%- if do_vision_count %}{%- set image_count.value = image_count.value + 1 %}{%- endif %}{%- if add_vision_id %}{{- 'Picture ' ~ image_count.value ~ ': ' }}{%- endif %}{{- '<|vision_start|><|image_pad|><|vision_end|>' }}{%- elif item.type == 'video' or 'video' in item %}{%- if is_system_content %}{{- raise_exception('System message cannot contain videos.') }}{%- endif %}{%- if do_vision_count %}{%- set video_count.value = video_count.value + 1 %}{%- endif %}{%- if add_vision_id %}{{- 'Video ' ~ video_count.value ~ ': ' }}{%- endif %}{{- '<|vision_start|><|video_pad|><|vision_end|>' }}{%- elif 'text' in item %}{{- item.text }}{%- else %}{{- raise_exception('Unexpected item type in content.') }}{%- endif %}{%- else %}{{- item | string }}{%- endif %}{%- endfor %}{%- elif content is none or content is undefined %}{{- '' }}{%- else %}{{- raise_exception('Unexpected content type.') }}{%- endif %}{%- endmacro %}{%- if not messages %}{{- raise_exception('No messages provided.') }}{%- endif %}{%- set _first_role = messages[0].role %}{%- if _first_role == 'system' or _first_role == 'developer' %}{%- set _sys_msg = messages[0] %}{%- set _msgs = messages[1:] %}{%- else %}{%- set _sys_msg = none %}{%- set _msgs = messages %}{%- endif %}{%- set _sc = '' %}{%- if _sys_msg is not none %}{%- set _sc = render_content(_sys_msg.content, false, true) | trim %}{%- if '<|think_off|>' in _sc %}{%- set _sc = _sc.split('<|think_off|>') | join('') | trim %}{%- endif %}{%- if '<|think_on|>' in _sc %}{%- set _sc = _sc.split('<|think_on|>') | join('') | trim %}{%- endif %}{%- if '<|think_xhigh|>' in _sc %}{%- set _sc = _sc.split('<|think_xhigh|>') | join('') | trim %}{%- endif %}{%- if '<|think_high|>' in _sc %}{%- set _sc = _sc.split('<|think_high|>') | join('') | trim %}{%- endif %}{%- if '<|think_medium|>' in _sc %}{%- set _sc = _sc.split('<|think_medium|>') | join('') | trim %}{%- endif %}{%- if '<|think_low|>' in _sc %}{%- set _sc = _sc.split('<|think_low|>') | join('') | trim %}{%- endif %}{%- if '<|think_minimal|>' in _sc %}{%- set _sc = _sc.split('<|think_minimal|>') | join('') | trim %}{%- endif %}{%- endif %}{%- if ns_state.thinking %}{%- set _terse_lead = 'Answer directly, after thinking. Lead with the answer, then only what it needs to be correct and usable.' %}{%- else %}{%- set _terse_lead = 'Answer directly and concisely. Give the answer with only what it needs to be correct and usable.' %}{%- endif %} {%- set _terse_core %}
|
| 2 |
+
Never: open with preamble or pleasantries; restate the question; add filler transitions; hedge with niceties; or repeat a point you've already made.
|
| 3 |
+
Always: keep essential steps, caveats, uncertainties, and specifics — never drop correctness or a needed warning for brevity. Keep the final answer lean. Use the least structure that conveys it (plain prose when short; lists or code only when they earn their place). If genuinely uncertain, say so and explain why — never omit uncertainty for the sake of brevity.
|
| 4 |
+
If a user request is genuinely ambiguous, ask a sharp question, don't guess.
|
| 5 |
+
{%- endset %} {%- set _terse = _terse_lead ~ '\n' ~ (_terse_core | trim) %}{%- if not _sc %}{%- set _sc = _terse | trim %}{%- else %}{%- set _sc = (_sc | trim) ~ '\n\n' ~ (_terse | trim) %}{%- endif %}{%- if _has_tools %}{{- '<|im_start|>system\n' }}{%- if reasoning_instructions %}{{- reasoning_instructions + '\n\n' }}{%- endif %}{{- '# Tools\n\nYou have access to the following functions:\n\n<tools>' }}{%- for tool in tools %}{{- '\n' }}{{- tool | tojson }}{%- endfor %}{{- '\n</tools>' }}{%- if _tool_format == 'json' %}{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n' }}{%- if ns_state.thinking %}{{- '<think>\nBrief explanation of tool call\n</think>\n' }}{%- endif %}{{- '<tool_call>\n{"name": "example_function_name", "arguments": {"example_parameter_1": "value_1", "example_parameter_2": "This is the value for the second parameter"}}\n</tool_call>\n\n<IMPORTANT>\nReminder:\n' }}{%- if ns_state.thinking %}{{- '- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n' }}{%- endif %}{{- '- Function calls MUST follow the specified format: a single JSON object with "name" and "arguments" keys inside <tool_call></tool_call> XML tags.\n- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after thinking, with NO conversational text before it.\n- The <tool_call> tag MUST be at the very beginning of a new line, with NO spaces or indentation before it.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}{%- else %}{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n' }}{%- if ns_state.thinking %}{{- '<think>\nBrief explanation of tool call\n</think>\n' }}{%- endif %}{{- '<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n' }}{%- if ns_state.thinking %}{{- '- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n' }}{%- endif %}{{- '- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags.\n- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after thinking, with NO conversational text before it.\n- The <tool_call> and <function> tags MUST be at the very beginning of a new line, with NO spaces or indentation before them.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}{%- endif %}{%- if _sc %}{{- '\n\n' + _sc }}{%- endif %}{{- '<|im_end|>\n' }}{%- else %}{%- if _sc %}{{- '<|im_start|>system\n' + (reasoning_instructions + '\n\n' if reasoning_instructions else '') + _sc + '<|im_end|>\n' }}{%- elif reasoning_instructions %}{{- '<|im_start|>system\n' + reasoning_instructions + '<|im_end|>\n' }}{%- endif %}{%- endif %}{%- set _last_idx = _msgs | length - 1 %}{%- set ns = namespace(multi_step_tool=true, last_query_index=_last_idx) %}{%- for message in _msgs[::-1] %}{%- set index = (_msgs | length - 1) - loop.index0 %}{%- if ns.multi_step_tool and message.role == 'user' %}{%- set _rc = render_content(message.content, false) | trim %}{%- if not (_rc.startswith('<tool_response>') and _rc.endswith('</tool_response>')) %}{%- set ns.multi_step_tool = false %}{%- set ns.last_query_index = index %}{%- endif %}{%- endif %}{%- endfor %}{%- if ns.multi_step_tool %}{%- if _last_idx > 50 %}{%- set ns.last_query_index = _last_idx %}{%- else %}{%- set ns.last_query_index = 0 %}{%- endif %}{%- endif %}{%- set ns2 = namespace(prev_role='', consecutive_failures=0) %}{%- for message in _msgs %}{%- set is_system = (message.role == "system" or message.role == "developer") %}{%- set content = render_content(message.content, true, is_system) | trim %}{%- if is_system or message.role == 'user' %}{%- if '<|think_off|>' in content %}{%- set content = content.split('<|think_off|>') | join('') | trim %}{%- endif %}{%- if '<|think_on|>' in content %}{%- set content = content.split('<|think_on|>') | join('') | trim %}{%- endif %}{%- if '<|think_xhigh|>' in content %}{%- set content = content.split('<|think_xhigh|>') | join('') | trim %}{%- endif %}{%- if '<|think_high|>' in content %}{%- set content = content.split('<|think_high|>') | join('') | trim %}{%- endif %}{%- if '<|think_medium|>' in content %}{%- set content = content.split('<|think_medium|>') | join('') | trim %}{%- endif %}{%- if '<|think_low|>' in content %}{%- set content = content.split('<|think_low|>') | join('') | trim %}{%- endif %}{%- if '<|think_minimal|>' in content %}{%- set content = content.split('<|think_minimal|>') | join('') | trim %}{%- endif %}{%- endif %}{%- if is_system %}{{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}{%- elif message.role == 'user' %}{%- set ns2.consecutive_failures = 0 %}{{- '<|im_start|>user\n' + content + '<|im_end|>\n' }}{%- elif message.role == 'assistant' %}{%- set reasoning_content = '' %}{%- if message.reasoning_content is defined and message.reasoning_content is not none %}{%- if message.reasoning_content is string %}{%- set reasoning_content = message.reasoning_content %}{%- else %}{%- set reasoning_content = message.reasoning_content | string %}{%- endif %}{%- elif message.thinking is defined and message.thinking is not none %}{%- if message.thinking is string %}{%- set reasoning_content = message.thinking %}{%- else %}{%- set reasoning_content = message.thinking | string %}{%- endif %}{%- else %}{%- set _think_end = '' %}{%- if content.startswith('</think>') %}{%- set _think_end = '</think>' %}{%- elif content.startswith('</thinking>') %}{%- set _think_end = '</thinking>' %}{%- elif '\n</think>' in content %}{%- set _think_end = '\n</think>' %}{%- elif '\n</thinking>' in content %}{%- set _think_end = '\n</thinking>' %}{%- elif '\n</ think>' in content %}{%- set _think_end = '\n</ think>' %}{%- elif '\n</think >' in content %}{%- set _think_end = '\n</think >' %}{%- endif %}{%- if _think_end %}{%- if 'thinking' in _think_end %}{%- set _think_start = '<thinking>' %}{%- else %}{%- set _think_start = '<think>' %}{%- endif %}{%- set reasoning_content = content.split(_think_end)[0].rstrip('\n') %}{%- if _think_start in reasoning_content %}{%- set reasoning_content = reasoning_content.split(_think_start)[-1].lstrip('\n') %}{%- endif %}{%- set content = content.split(_think_end)[-1].lstrip('\n') %}{%- endif %}{%- endif %}{%- set reasoning_content = reasoning_content | trim %}{%- if (_preserve_thinking or loop.index0 > ns.last_query_index) and reasoning_content %}{{- '<|im_start|>assistant\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}{%- else %}{{- '<|im_start|>assistant\n' + content }}{%- endif %}{%- if message.tool_calls is defined and message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}{%- for tool_call in message.tool_calls %}{%- if tool_call.function is defined and tool_call.function is not none %}{%- set tc = tool_call.function %}{%- else %}{%- set tc = tool_call %}{%- endif %}{%- set tc_name = tc.name if (tc.name is defined and tc.name is not none) else '' %}{%- if _tool_format == 'json' %}{%- if not loop.first or content | trim %}{{- '\n\n' }}{%- endif %}{%- set _args = '{}' %}{%- if tc.arguments is defined and tc.arguments is not none %}{%- if tc.arguments is mapping %}{%- set _args = tc.arguments | tojson %}{%- elif tc.arguments is string and tc.arguments %}{%- set _args = tc.arguments %}{%- endif %}{%- endif %}{{- '<tool_call>\n{"name": ' }}{{- tc_name | tojson }}{{- ', "arguments": ' }}{{- _args }}{{- '}\n</tool_call>' }}{%- else %}{%- if loop.first %}{%- if content | trim %}{{- '\n\n<tool_call>\n<function=' + tc_name + '>\n' }}{%- else %}{{- '<tool_call>\n<function=' + tc_name + '>\n' }}{%- endif %}{%- else %}{{- '\n\n<tool_call>\n<function=' + tc_name + '>\n' }}{%- endif %}{%- if tc.arguments is defined and tc.arguments is not none %}{%- if tc.arguments is mapping %}{%- for args_name, args_value in tc.arguments.items() %}{{- '<parameter=' + args_name + '>\n' }}{%- if args_value is mapping or (args_value is sequence and args_value is not string) %}{%- set _av = args_value | tojson %}{%- else %}{%- set _av = args_value | string %}{%- endif %}{%- if max_tool_arg_chars > 0 and _av | length > max_tool_arg_chars %}{{- _av[:max_tool_arg_chars] + '\n[TRUNCATED - original length ' ~ (_av | length | string) ~ ' chars]' }}{%- else %}{{- _av }}{%- endif %}{{- '\n</parameter>\n' }}{%- endfor %}{%- elif tc.arguments is string and tc.arguments %}{{- tc.arguments }}{%- endif %}{%- endif %}{{- '</function>\n</tool_call>' }}{%- endif %}{%- endfor %}{%- endif %}{{- '<|im_end|>\n' }}{%- elif message.role == 'tool' %}{%- set _content_lower = content | lower %}{%- set _content_head = _content_lower[:80] %}{%- if content | length < 500 and '$ ' not in content and 'took ' not in _content_lower and ('"error":' in _content_head or 'error:' in _content_head or 'err!' in _content_head or 'fatal:' in _content_head or 'exception:' in _content_head or 'traceback' in _content_head or 'command not found' in _content_head or 'invalid syntax' in _content_head or 'failed to' in _content_head) %}{%- set ns2.consecutive_failures = ns2.consecutive_failures + 1 %}{%- else %}{%- set ns2.consecutive_failures = 0 %}{%- endif %}{%- if ns2.prev_role != 'tool' %}{{- '<|im_start|>user' }}{%- endif %}{%- if max_tool_response_chars > 0 and content | length > max_tool_response_chars %}{%- set content = content[:max_tool_response_chars] + '\n[TRUNCATED - original length ' ~ (content | length | string) ~ ' chars]' %}{%- endif %}{{- '\n<tool_response>\n' + content }}{%- if ns2.consecutive_failures >= 2 %}{{- '\n\n⚠️ SYSTEM WARNING: ' ~ ns2.consecutive_failures ~ ' consecutive tool errors detected. Your previous approach is incorrect. You MUST use a fundamentally different approach or corrected arguments.' }}{%- elif ns2.consecutive_failures == 1 %}{{- '\n\n⚠️ SYSTEM WARNING: The previous tool call returned an error. Diagnose the failure and retry with completely corrected arguments.' }}{%- endif %}{{- '\n</tool_response>' }}{%- if loop.last %}{{- '<|im_end|>\n' }}{%- else %}{%- set _next_role = _msgs[loop.index0 + 1].role %}{%- if _next_role != 'tool' %}{{- '<|im_end|>\n' }}{%- endif %}{%- endif %}{%- else %}{{- '<|im_start|>user\n[' + message.role + ']: ' + content + '<|im_end|>\n' }}{%- endif %}{%- set ns2.prev_role = message.role %}{%- endfor %}{%- if add_generation_prompt %}{{- '<|im_start|>assistant\n' }}{%- if not ns_state.thinking %}{{- '<think>\n\n</think>\n\n' }}{%- else %}{{- '<think>\n' }}{%- endif %}{%- endif %}
|
chat_template.jinja
CHANGED
|
@@ -1,4 +1,4 @@
|
|
| 1 |
-
{%- set template_version = "qwen3.8-froggeric-v22.
|
| 2 |
{%- set _tool_format = tool_call_format if tool_call_format is defined else 'xml' %}
|
| 3 |
{%- set image_count = namespace(value=0) %}
|
| 4 |
{%- set video_count = namespace(value=0) %}
|
|
@@ -18,12 +18,12 @@
|
|
| 18 |
{%- set _effort_raw = (reasoning_effort | string | lower) if reasoning_effort is defined and reasoning_effort is not none else 'medium' %}
|
| 19 |
{%- set _initial_thinking = enable_thinking %}
|
| 20 |
{%- set _initial_effort = 'medium' %}
|
| 21 |
-
{%- if _effort_raw
|
| 22 |
{%- set _initial_thinking = false %}
|
| 23 |
{%- set _initial_effort = 'medium' %}
|
| 24 |
-
{%- elif _effort_raw
|
| 25 |
{%- set _initial_effort = 'low' %}
|
| 26 |
-
{%- elif _effort_raw
|
| 27 |
{%- set _initial_effort = 'xhigh' %}
|
| 28 |
{%- else %}
|
| 29 |
{%- set _initial_effort = 'medium' %}
|
|
@@ -39,7 +39,7 @@
|
|
| 39 |
{%- set ns_state.thinking = false %}
|
| 40 |
{%- elif '<|think_on|>' in msg.content %}
|
| 41 |
{%- set ns_state.thinking = true %}
|
| 42 |
-
{%- elif '<|think_xhigh|>' in msg.content or '<|think_high|>' in msg.content %}
|
| 43 |
{%- set ns_state.thinking = true %}
|
| 44 |
{%- set ns_state.effort = 'xhigh' %}
|
| 45 |
{%- elif '<|think_low|>' in msg.content or '<|think_minimal|>' in msg.content %}
|
|
@@ -63,7 +63,7 @@
|
|
| 63 |
{%- set ns_state.thinking = false %}
|
| 64 |
{%- elif '<|think_on|>' in _item_text %}
|
| 65 |
{%- set ns_state.thinking = true %}
|
| 66 |
-
{%- elif '<|think_xhigh|>' in _item_text or '<|think_high|>' in _item_text %}
|
| 67 |
{%- set ns_state.thinking = true %}
|
| 68 |
{%- set ns_state.effort = 'xhigh' %}
|
| 69 |
{%- elif '<|think_low|>' in _item_text or '<|think_minimal|>' in _item_text %}
|
|
@@ -132,25 +132,38 @@
|
|
| 132 |
{%- if not messages %}
|
| 133 |
{{- raise_exception('No messages provided.') }}
|
| 134 |
{%- endif %}
|
| 135 |
-
{%- set
|
| 136 |
-
{%-
|
| 137 |
-
{%- set
|
| 138 |
-
{%-
|
| 139 |
-
{%-
|
| 140 |
-
{%-
|
| 141 |
-
|
| 142 |
-
{%- endif %}
|
| 143 |
-
{%-
|
| 144 |
-
{%-
|
| 145 |
-
|
| 146 |
-
{%-
|
| 147 |
-
{%- if '<|
|
| 148 |
-
{%- if '<|
|
| 149 |
-
{%- if '<|
|
| 150 |
-
{%- if '<|
|
| 151 |
-
{%- if '<|
|
| 152 |
-
{%- if '<|
|
| 153 |
-
{%- endif %}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 154 |
{%- if ns_state.thinking %}
|
| 155 |
{%- set _terse_lead = 'Answer directly, after thinking. Lead with the answer, then only what it needs to be correct and usable.' %}
|
| 156 |
{%- else %}
|
|
@@ -179,9 +192,9 @@ If a user request is genuinely ambiguous, ask a sharp question, don't guess.
|
|
| 179 |
{%- endfor %}
|
| 180 |
{{- '\n</tools>' }}
|
| 181 |
{%- if _tool_format == 'json' %}
|
| 182 |
-
{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n' }}{%- if ns_state.thinking %}{{- '<think>\nBrief explanation of tool call\n</think>\n' }}{%- endif %}{{- '<tool_call>\n{"name": "example_function_name", "arguments": {"example_parameter_1": "value_1", "example_parameter_2": "This is the value for the second parameter"}}\n</tool_call>\n\n<IMPORTANT>\nReminder:\n' }}{%- if ns_state.thinking %}{{- '- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n' }}{%- endif %}{{- '- Function calls MUST follow the specified format: a single JSON object with "name" and "arguments" keys inside <tool_call></tool_call> XML tags.\n- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after thinking, with NO conversational text before it.\n- The <tool_call> tag MUST be at the very beginning of a new line, with NO spaces or indentation before it.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}
|
| 183 |
{%- else %}
|
| 184 |
-
{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n' }}{%- if ns_state.thinking %}{{- '<think>\nBrief explanation of tool call\n</think>\n' }}{%- endif %}{{- '<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n' }}{%- if ns_state.thinking %}{{- '- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n' }}{%- endif %}{{- '- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags.\n- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after thinking, with NO conversational text before it.\n- The <tool_call> and <function> tags MUST be at the very beginning of a new line, with NO spaces or indentation before them.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}
|
| 185 |
{%- endif %}
|
| 186 |
{%- if _sc %}
|
| 187 |
{{- '\n\n' + _sc }}
|
|
@@ -222,6 +235,9 @@ If a user request is genuinely ambiguous, ask a sharp question, don't guess.
|
|
| 222 |
{%- if '<|think_on|>' in content %}{%- set content = content.split('<|think_on|>') | join('') | trim %}{%- endif %}
|
| 223 |
{%- if '<|think_xhigh|>' in content %}{%- set content = content.split('<|think_xhigh|>') | join('') | trim %}{%- endif %}
|
| 224 |
{%- if '<|think_high|>' in content %}{%- set content = content.split('<|think_high|>') | join('') | trim %}{%- endif %}
|
|
|
|
|
|
|
|
|
|
| 225 |
{%- if '<|think_medium|>' in content %}{%- set content = content.split('<|think_medium|>') | join('') | trim %}{%- endif %}
|
| 226 |
{%- if '<|think_low|>' in content %}{%- set content = content.split('<|think_low|>') | join('') | trim %}{%- endif %}
|
| 227 |
{%- if '<|think_minimal|>' in content %}{%- set content = content.split('<|think_minimal|>') | join('') | trim %}{%- endif %}
|
|
@@ -233,18 +249,35 @@ If a user request is genuinely ambiguous, ask a sharp question, don't guess.
|
|
| 233 |
{{- '<|im_start|>user\n' + content + '<|im_end|>\n' }}
|
| 234 |
{%- elif message.role == 'assistant' %}
|
| 235 |
{%- set reasoning_content = '' %}
|
|
|
|
| 236 |
{%- if message.reasoning_content is defined and message.reasoning_content is not none %}
|
| 237 |
{%- if message.reasoning_content is string %}
|
| 238 |
-
{%- set
|
| 239 |
{%- else %}
|
| 240 |
-
{%- set
|
| 241 |
{%- endif %}
|
| 242 |
{%- elif message.thinking is defined and message.thinking is not none %}
|
| 243 |
{%- if message.thinking is string %}
|
| 244 |
-
{%- set
|
| 245 |
{%- else %}
|
| 246 |
-
{%- set
|
| 247 |
{%- endif %}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 248 |
{%- else %}
|
| 249 |
{%- set _think_end = '' %}
|
| 250 |
{%- if content.startswith('</think>') %}
|
|
@@ -259,6 +292,10 @@ If a user request is genuinely ambiguous, ask a sharp question, don't guess.
|
|
| 259 |
{%- set _think_end = '\n</ think>' %}
|
| 260 |
{%- elif '\n</think >' in content %}
|
| 261 |
{%- set _think_end = '\n</think >' %}
|
|
|
|
|
|
|
|
|
|
|
|
|
| 262 |
{%- endif %}
|
| 263 |
{%- if _think_end %}
|
| 264 |
{%- if 'thinking' in _think_end %}
|
|
@@ -274,7 +311,7 @@ If a user request is genuinely ambiguous, ask a sharp question, don't guess.
|
|
| 274 |
{%- endif %}
|
| 275 |
{%- endif %}
|
| 276 |
{%- set reasoning_content = reasoning_content | trim %}
|
| 277 |
-
{%- if (_preserve_thinking or loop.index0 > ns.last_query_index)
|
| 278 |
{{- '<|im_start|>assistant\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
|
| 279 |
{%- else %}
|
| 280 |
{{- '<|im_start|>assistant\n' + content }}
|
|
@@ -295,8 +332,12 @@ If a user request is genuinely ambiguous, ask a sharp question, don't guess.
|
|
| 295 |
{%- if tc.arguments is defined and tc.arguments is not none %}
|
| 296 |
{%- if tc.arguments is mapping %}
|
| 297 |
{%- set _args = tc.arguments | tojson %}
|
| 298 |
-
{%- elif tc.arguments is string
|
| 299 |
-
{%-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 300 |
{%- endif %}
|
| 301 |
{%- endif %}
|
| 302 |
{{- '<tool_call>\n{"name": ' }}{{- tc_name | tojson }}{{- ', "arguments": ' }}{{- _args }}{{- '}\n</tool_call>' }}
|
|
@@ -314,10 +355,10 @@ If a user request is genuinely ambiguous, ask a sharp question, don't guess.
|
|
| 314 |
{%- if tc.arguments is mapping %}
|
| 315 |
{%- for args_name, args_value in tc.arguments.items() %}
|
| 316 |
{{- '<parameter=' + args_name + '>\n' }}
|
| 317 |
-
{%- if args_value is
|
| 318 |
-
{%- set _av = args_value
|
| 319 |
{%- else %}
|
| 320 |
-
{%- set _av = args_value |
|
| 321 |
{%- endif %}
|
| 322 |
{%- if max_tool_arg_chars > 0 and _av | length > max_tool_arg_chars %}
|
| 323 |
{{- _av[:max_tool_arg_chars] + '\n[TRUNCATED - original length ' ~ (_av | length | string) ~ ' chars]' }}
|
|
@@ -326,8 +367,19 @@ If a user request is genuinely ambiguous, ask a sharp question, don't guess.
|
|
| 326 |
{%- endif %}
|
| 327 |
{{- '\n</parameter>\n' }}
|
| 328 |
{%- endfor %}
|
| 329 |
-
{%-
|
| 330 |
-
{
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 331 |
{%- endif %}
|
| 332 |
{%- endif %}
|
| 333 |
{{- '</function>\n</tool_call>' }}
|
|
@@ -337,8 +389,14 @@ If a user request is genuinely ambiguous, ask a sharp question, don't guess.
|
|
| 337 |
{{- '<|im_end|>\n' }}
|
| 338 |
{%- elif message.role == 'tool' %}
|
| 339 |
{%- set _content_lower = content | lower %}
|
| 340 |
-
{%- set _content_head = _content_lower[:
|
| 341 |
-
{%-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 342 |
{%- set ns2.consecutive_failures = ns2.consecutive_failures + 1 %}
|
| 343 |
{%- else %}
|
| 344 |
{%- set ns2.consecutive_failures = 0 %}
|
|
@@ -346,7 +404,7 @@ If a user request is genuinely ambiguous, ask a sharp question, don't guess.
|
|
| 346 |
{%- if ns2.prev_role != 'tool' %}
|
| 347 |
{{- '<|im_start|>user' }}
|
| 348 |
{%- endif %}
|
| 349 |
-
{%- if max_tool_response_chars > 0 and content | length > max_tool_response_chars %}
|
| 350 |
{%- set content = content[:max_tool_response_chars] + '\n[TRUNCATED - original length ' ~ (content | length | string) ~ ' chars]' %}
|
| 351 |
{%- endif %}
|
| 352 |
{{- '\n<tool_response>\n' + content }}
|
|
|
|
| 1 |
+
{%- set template_version = "qwen3.8-froggeric-v22.3.1" %}
|
| 2 |
{%- set _tool_format = tool_call_format if tool_call_format is defined else 'xml' %}
|
| 3 |
{%- set image_count = namespace(value=0) %}
|
| 4 |
{%- set video_count = namespace(value=0) %}
|
|
|
|
| 18 |
{%- set _effort_raw = (reasoning_effort | string | lower) if reasoning_effort is defined and reasoning_effort is not none else 'medium' %}
|
| 19 |
{%- set _initial_thinking = enable_thinking %}
|
| 20 |
{%- set _initial_effort = 'medium' %}
|
| 21 |
+
{%- if _effort_raw in ('none', 'off') %}
|
| 22 |
{%- set _initial_thinking = false %}
|
| 23 |
{%- set _initial_effort = 'medium' %}
|
| 24 |
+
{%- elif _effort_raw in ('minimal', 'low') %}
|
| 25 |
{%- set _initial_effort = 'low' %}
|
| 26 |
+
{%- elif _effort_raw in ('high', 'xhigh', 'max', 'ultracode', 'extreme') %}
|
| 27 |
{%- set _initial_effort = 'xhigh' %}
|
| 28 |
{%- else %}
|
| 29 |
{%- set _initial_effort = 'medium' %}
|
|
|
|
| 39 |
{%- set ns_state.thinking = false %}
|
| 40 |
{%- elif '<|think_on|>' in msg.content %}
|
| 41 |
{%- set ns_state.thinking = true %}
|
| 42 |
+
{%- elif '<|think_xhigh|>' in msg.content or '<|think_high|>' in msg.content or '<|think_ultracode|>' in msg.content or '<|think_extreme|>' in msg.content or '<|think_max|>' in msg.content %}
|
| 43 |
{%- set ns_state.thinking = true %}
|
| 44 |
{%- set ns_state.effort = 'xhigh' %}
|
| 45 |
{%- elif '<|think_low|>' in msg.content or '<|think_minimal|>' in msg.content %}
|
|
|
|
| 63 |
{%- set ns_state.thinking = false %}
|
| 64 |
{%- elif '<|think_on|>' in _item_text %}
|
| 65 |
{%- set ns_state.thinking = true %}
|
| 66 |
+
{%- elif '<|think_xhigh|>' in _item_text or '<|think_high|>' in _item_text or '<|think_ultracode|>' in _item_text or '<|think_extreme|>' in _item_text or '<|think_max|>' in _item_text %}
|
| 67 |
{%- set ns_state.thinking = true %}
|
| 68 |
{%- set ns_state.effort = 'xhigh' %}
|
| 69 |
{%- elif '<|think_low|>' in _item_text or '<|think_minimal|>' in _item_text %}
|
|
|
|
| 132 |
{%- if not messages %}
|
| 133 |
{{- raise_exception('No messages provided.') }}
|
| 134 |
{%- endif %}
|
| 135 |
+
{%- set head = namespace(count=0, seen_non_system=false) %}
|
| 136 |
+
{%- for message in messages %}
|
| 137 |
+
{%- set _is_sys = (message.role == 'system' or message.role == 'developer') %}
|
| 138 |
+
{%- if _is_sys and not head.seen_non_system %}
|
| 139 |
+
{%- set head.count = head.count + 1 %}
|
| 140 |
+
{%- else %}
|
| 141 |
+
{%- set head.seen_non_system = true %}
|
| 142 |
+
{%- endif %}
|
| 143 |
+
{%- endfor %}
|
| 144 |
+
{%- set sys_state = namespace(content='') %}
|
| 145 |
+
{%- for message in messages[:head.count] %}
|
| 146 |
+
{%- set _part = render_content(message.content, false, true) | trim %}
|
| 147 |
+
{%- if '<|think_off|>' in _part %}{%- set _part = _part.split('<|think_off|>') | join('') | trim %}{%- endif %}
|
| 148 |
+
{%- if '<|think_on|>' in _part %}{%- set _part = _part.split('<|think_on|>') | join('') | trim %}{%- endif %}
|
| 149 |
+
{%- if '<|think_xhigh|>' in _part %}{%- set _part = _part.split('<|think_xhigh|>') | join('') | trim %}{%- endif %}
|
| 150 |
+
{%- if '<|think_high|>' in _part %}{%- set _part = _part.split('<|think_high|>') | join('') | trim %}{%- endif %}
|
| 151 |
+
{%- if '<|think_ultracode|>' in _part %}{%- set _part = _part.split('<|think_ultracode|>') | join('') | trim %}{%- endif %}
|
| 152 |
+
{%- if '<|think_extreme|>' in _part %}{%- set _part = _part.split('<|think_extreme|>') | join('') | trim %}{%- endif %}
|
| 153 |
+
{%- if '<|think_max|>' in _part %}{%- set _part = _part.split('<|think_max|>') | join('') | trim %}{%- endif %}
|
| 154 |
+
{%- if '<|think_medium|>' in _part %}{%- set _part = _part.split('<|think_medium|>') | join('') | trim %}{%- endif %}
|
| 155 |
+
{%- if '<|think_low|>' in _part %}{%- set _part = _part.split('<|think_low|>') | join('') | trim %}{%- endif %}
|
| 156 |
+
{%- if '<|think_minimal|>' in _part %}{%- set _part = _part.split('<|think_minimal|>') | join('') | trim %}{%- endif %}
|
| 157 |
+
{%- if _part %}
|
| 158 |
+
{%- if sys_state.content %}
|
| 159 |
+
{%- set sys_state.content = sys_state.content ~ '\n\n' ~ _part %}
|
| 160 |
+
{%- else %}
|
| 161 |
+
{%- set sys_state.content = _part %}
|
| 162 |
+
{%- endif %}
|
| 163 |
+
{%- endif %}
|
| 164 |
+
{%- endfor %}
|
| 165 |
+
{%- set _sc = sys_state.content %}
|
| 166 |
+
{%- set _msgs = messages[head.count:] %}
|
| 167 |
{%- if ns_state.thinking %}
|
| 168 |
{%- set _terse_lead = 'Answer directly, after thinking. Lead with the answer, then only what it needs to be correct and usable.' %}
|
| 169 |
{%- else %}
|
|
|
|
| 192 |
{%- endfor %}
|
| 193 |
{{- '\n</tools>' }}
|
| 194 |
{%- if _tool_format == 'json' %}
|
| 195 |
+
{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n' }}{%- if ns_state.thinking %}{{- '<think>\nBrief explanation of tool call\n</think>\n' }}{%- endif %}{{- '<tool_call>\n{"name": "example_function_name", "arguments": {"example_parameter_1": "value_1", "example_parameter_2": "This is the value for the second parameter"}}\n</tool_call>\n\n<IMPORTANT>\nReminder:\n' }}{%- if ns_state.thinking %}{{- '- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n' }}{%- endif %}{{- '- Function calls MUST follow the specified format: a single JSON object with "name" and "arguments" keys inside <tool_call></tool_call> XML tags.\n- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY' }}{%- if ns_state.thinking %}{{- ' after thinking' }}{%- endif %}{{- ', with NO conversational text before it.\n- The <tool_call> tag MUST be at the very beginning of a new line, with NO spaces or indentation before it.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}
|
| 196 |
{%- else %}
|
| 197 |
+
{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n' }}{%- if ns_state.thinking %}{{- '<think>\nBrief explanation of tool call\n</think>\n' }}{%- endif %}{{- '<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n' }}{%- if ns_state.thinking %}{{- '- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n' }}{%- endif %}{{- '- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags.\n- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY' }}{%- if ns_state.thinking %}{{- ' after thinking' }}{%- endif %}{{- ', with NO conversational text before it.\n- The <tool_call> and <function> tags MUST be at the very beginning of a new line, with NO spaces or indentation before them.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}
|
| 198 |
{%- endif %}
|
| 199 |
{%- if _sc %}
|
| 200 |
{{- '\n\n' + _sc }}
|
|
|
|
| 235 |
{%- if '<|think_on|>' in content %}{%- set content = content.split('<|think_on|>') | join('') | trim %}{%- endif %}
|
| 236 |
{%- if '<|think_xhigh|>' in content %}{%- set content = content.split('<|think_xhigh|>') | join('') | trim %}{%- endif %}
|
| 237 |
{%- if '<|think_high|>' in content %}{%- set content = content.split('<|think_high|>') | join('') | trim %}{%- endif %}
|
| 238 |
+
{%- if '<|think_ultracode|>' in content %}{%- set content = content.split('<|think_ultracode|>') | join('') | trim %}{%- endif %}
|
| 239 |
+
{%- if '<|think_extreme|>' in content %}{%- set content = content.split('<|think_extreme|>') | join('') | trim %}{%- endif %}
|
| 240 |
+
{%- if '<|think_max|>' in content %}{%- set content = content.split('<|think_max|>') | join('') | trim %}{%- endif %}
|
| 241 |
{%- if '<|think_medium|>' in content %}{%- set content = content.split('<|think_medium|>') | join('') | trim %}{%- endif %}
|
| 242 |
{%- if '<|think_low|>' in content %}{%- set content = content.split('<|think_low|>') | join('') | trim %}{%- endif %}
|
| 243 |
{%- if '<|think_minimal|>' in content %}{%- set content = content.split('<|think_minimal|>') | join('') | trim %}{%- endif %}
|
|
|
|
| 249 |
{{- '<|im_start|>user\n' + content + '<|im_end|>\n' }}
|
| 250 |
{%- elif message.role == 'assistant' %}
|
| 251 |
{%- set reasoning_content = '' %}
|
| 252 |
+
{%- set _explicit_reasoning = '' %}
|
| 253 |
{%- if message.reasoning_content is defined and message.reasoning_content is not none %}
|
| 254 |
{%- if message.reasoning_content is string %}
|
| 255 |
+
{%- set _explicit_reasoning = message.reasoning_content %}
|
| 256 |
{%- else %}
|
| 257 |
+
{%- set _explicit_reasoning = message.reasoning_content | string %}
|
| 258 |
{%- endif %}
|
| 259 |
{%- elif message.thinking is defined and message.thinking is not none %}
|
| 260 |
{%- if message.thinking is string %}
|
| 261 |
+
{%- set _explicit_reasoning = message.thinking %}
|
| 262 |
{%- else %}
|
| 263 |
+
{%- set _explicit_reasoning = message.thinking | string %}
|
| 264 |
{%- endif %}
|
| 265 |
+
{%- endif %}
|
| 266 |
+
{%- if _explicit_reasoning %}
|
| 267 |
+
{%- set _lead_end = '' %}
|
| 268 |
+
{%- if content.startswith('<think>') and '</think>' in content %}
|
| 269 |
+
{%- set _lead_end = '</think>' %}
|
| 270 |
+
{%- elif content.startswith('<thinking>') and '</thinking>' in content %}
|
| 271 |
+
{%- set _lead_end = '</thinking>' %}
|
| 272 |
+
{%- elif content.startswith('</think>') %}
|
| 273 |
+
{%- set _lead_end = '</think>' %}
|
| 274 |
+
{%- elif content.startswith('</thinking>') %}
|
| 275 |
+
{%- set _lead_end = '</thinking>' %}
|
| 276 |
+
{%- endif %}
|
| 277 |
+
{%- if _lead_end %}
|
| 278 |
+
{%- set content = content.split(_lead_end)[-1].lstrip('\n') %}
|
| 279 |
+
{%- endif %}
|
| 280 |
+
{%- set reasoning_content = _explicit_reasoning %}
|
| 281 |
{%- else %}
|
| 282 |
{%- set _think_end = '' %}
|
| 283 |
{%- if content.startswith('</think>') %}
|
|
|
|
| 292 |
{%- set _think_end = '\n</ think>' %}
|
| 293 |
{%- elif '\n</think >' in content %}
|
| 294 |
{%- set _think_end = '\n</think >' %}
|
| 295 |
+
{%- elif content.startswith('<think>') and '</think>' in content %}
|
| 296 |
+
{%- set _think_end = '</think>' %}
|
| 297 |
+
{%- elif content.startswith('<thinking>') and '</thinking>' in content %}
|
| 298 |
+
{%- set _think_end = '</thinking>' %}
|
| 299 |
{%- endif %}
|
| 300 |
{%- if _think_end %}
|
| 301 |
{%- if 'thinking' in _think_end %}
|
|
|
|
| 311 |
{%- endif %}
|
| 312 |
{%- endif %}
|
| 313 |
{%- set reasoning_content = reasoning_content | trim %}
|
| 314 |
+
{%- if (_preserve_thinking or loop.index0 > ns.last_query_index) %}
|
| 315 |
{{- '<|im_start|>assistant\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
|
| 316 |
{%- else %}
|
| 317 |
{{- '<|im_start|>assistant\n' + content }}
|
|
|
|
| 332 |
{%- if tc.arguments is defined and tc.arguments is not none %}
|
| 333 |
{%- if tc.arguments is mapping %}
|
| 334 |
{%- set _args = tc.arguments | tojson %}
|
| 335 |
+
{%- elif tc.arguments is string %}
|
| 336 |
+
{%- if tc.arguments %}
|
| 337 |
+
{%- set _args = tc.arguments %}
|
| 338 |
+
{%- endif %}
|
| 339 |
+
{%- else %}
|
| 340 |
+
{%- set _args = tc.arguments | tojson %}
|
| 341 |
{%- endif %}
|
| 342 |
{%- endif %}
|
| 343 |
{{- '<tool_call>\n{"name": ' }}{{- tc_name | tojson }}{{- ', "arguments": ' }}{{- _args }}{{- '}\n</tool_call>' }}
|
|
|
|
| 355 |
{%- if tc.arguments is mapping %}
|
| 356 |
{%- for args_name, args_value in tc.arguments.items() %}
|
| 357 |
{{- '<parameter=' + args_name + '>\n' }}
|
| 358 |
+
{%- if args_value is string %}
|
| 359 |
+
{%- set _av = args_value %}
|
| 360 |
{%- else %}
|
| 361 |
+
{%- set _av = args_value | tojson %}
|
| 362 |
{%- endif %}
|
| 363 |
{%- if max_tool_arg_chars > 0 and _av | length > max_tool_arg_chars %}
|
| 364 |
{{- _av[:max_tool_arg_chars] + '\n[TRUNCATED - original length ' ~ (_av | length | string) ~ ' chars]' }}
|
|
|
|
| 367 |
{%- endif %}
|
| 368 |
{{- '\n</parameter>\n' }}
|
| 369 |
{%- endfor %}
|
| 370 |
+
{%- else %}
|
| 371 |
+
{%- if tc.arguments is string %}
|
| 372 |
+
{%- set _raw_args = tc.arguments %}
|
| 373 |
+
{%- else %}
|
| 374 |
+
{%- set _raw_args = tc.arguments | tojson %}
|
| 375 |
+
{%- endif %}
|
| 376 |
+
{%- if _raw_args %}
|
| 377 |
+
{%- if max_tool_arg_chars > 0 and _raw_args | length > max_tool_arg_chars %}
|
| 378 |
+
{{- _raw_args[:max_tool_arg_chars] + '\n[TRUNCATED - original length ' ~ (_raw_args | length | string) ~ ' chars]' }}
|
| 379 |
+
{%- else %}
|
| 380 |
+
{{- _raw_args }}
|
| 381 |
+
{%- endif %}
|
| 382 |
+
{%- endif %}
|
| 383 |
{%- endif %}
|
| 384 |
{%- endif %}
|
| 385 |
{{- '</function>\n</tool_call>' }}
|
|
|
|
| 389 |
{{- '<|im_end|>\n' }}
|
| 390 |
{%- elif message.role == 'tool' %}
|
| 391 |
{%- set _content_lower = content | lower %}
|
| 392 |
+
{%- set _content_head = _content_lower[:120] %}
|
| 393 |
+
{%- set _is_code_or_grep = ('throw new ' in _content_lower or 'throw error' in _content_lower or 'console.error' in _content_lower or 'logger.error' in _content_lower or 'logging.error' in _content_lower or 'import ' in _content_head or 'def ' in _content_head or 'function ' in _content_head) %}
|
| 394 |
+
{%- set _exit_code_zero = ('exit code: 0' in _content_head or 'process exited with code 0' in _content_head) %}
|
| 395 |
+
{%- set _error_field_ok = ('"error": null' in _content_head or '"error":null' in _content_head or '"error": false' in _content_head or '"error":false' in _content_head or '"error": ""' in _content_head or '"error":""' in _content_head) %}
|
| 396 |
+
{%- set _strong_error = (('"error":' in _content_head and not _error_field_ok) or '"status": "error"' in _content_head or '"status":"error"' in _content_head or 'traceback (most recent call last):' in _content_head or 'command not found' in _content_head or 'invalid syntax' in _content_head or 'fatal:' in _content_head or (('exit code: ' in _content_head or 'process exited with code' in _content_head) and not _exit_code_zero) or _content_head.startswith('exception:') or _content_head.startswith('failed to ')) %}
|
| 397 |
+
{%- set _weak_error = ('error:' in _content_head or 'err!' in _content_head) %}
|
| 398 |
+
{%- set _weak_suppressed = ('$ ' in _content_head or 'took ' in _content_head or content | length >= 600) %}
|
| 399 |
+
{%- if not _is_code_or_grep and (_strong_error or (_weak_error and not _weak_suppressed)) %}
|
| 400 |
{%- set ns2.consecutive_failures = ns2.consecutive_failures + 1 %}
|
| 401 |
{%- else %}
|
| 402 |
{%- set ns2.consecutive_failures = 0 %}
|
|
|
|
| 404 |
{%- if ns2.prev_role != 'tool' %}
|
| 405 |
{{- '<|im_start|>user' }}
|
| 406 |
{%- endif %}
|
| 407 |
+
{%- if _tool_format != 'json' and max_tool_response_chars > 0 and content | length > max_tool_response_chars %}
|
| 408 |
{%- set content = content[:max_tool_response_chars] + '\n[TRUNCATED - original length ' ~ (content | length | string) ~ ' chars]' %}
|
| 409 |
{%- endif %}
|
| 410 |
{{- '\n<tool_response>\n' + content }}
|
chat_template_oneline.txt
CHANGED
|
@@ -1,5 +1,5 @@
|
|
| 1 |
-
{%- set template_version = "qwen3.8-froggeric-v22.
|
| 2 |
Never: open with preamble or pleasantries; restate the question; add filler transitions; hedge with niceties; or repeat a point you've already made.
|
| 3 |
Always: keep essential steps, caveats, uncertainties, and specifics — never drop correctness or a needed warning for brevity. Keep the final answer lean. Use the least structure that conveys it (plain prose when short; lists or code only when they earn their place). If genuinely uncertain, say so and explain why — never omit uncertainty for the sake of brevity.
|
| 4 |
If a user request is genuinely ambiguous, ask a sharp question, don't guess.
|
| 5 |
-
{%- endset %} {%- set _terse = _terse_lead ~ '\n' ~ (_terse_core | trim) %}{%- if not _sc %}{%- set _sc = _terse | trim %}{%- else %}{%- set _sc = (_sc | trim) ~ '\n\n' ~ (_terse | trim) %}{%- endif %}{%- if _has_tools %}{{- '<|im_start|>system\n' }}{%- if reasoning_instructions %}{{- reasoning_instructions + '\n\n' }}{%- endif %}{{- '# Tools\n\nYou have access to the following functions:\n\n<tools>' }}{%- for tool in tools %}{{- '\n' }}{{- tool | tojson }}{%- endfor %}{{- '\n</tools>' }}{%- if _tool_format == 'json' %}{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n' }}{%- if ns_state.thinking %}{{- '<think>\nBrief explanation of tool call\n</think>\n' }}{%- endif %}{{- '<tool_call>\n{"name": "example_function_name", "arguments": {"example_parameter_1": "value_1", "example_parameter_2": "This is the value for the second parameter"}}\n</tool_call>\n\n<IMPORTANT>\nReminder:\n' }}{%- if ns_state.thinking %}{{- '- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n' }}{%- endif %}{{- '- Function calls MUST follow the specified format: a single JSON object with "name" and "arguments" keys inside <tool_call></tool_call> XML tags.\n- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after thinking, with NO conversational text before it.\n- The <tool_call> tag MUST be at the very beginning of a new line, with NO spaces or indentation before it.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}{%- else %}{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n' }}{%- if ns_state.thinking %}{{- '<think>\nBrief explanation of tool call\n</think>\n' }}{%- endif %}{{- '<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n' }}{%- if ns_state.thinking %}{{- '- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n' }}{%- endif %}{{- '- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags.\n- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after thinking, with NO conversational text before it.\n- The <tool_call> and <function> tags MUST be at the very beginning of a new line, with NO spaces or indentation before them.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}{%- endif %}{%- if _sc %}{{- '\n\n' + _sc }}{%- endif %}{{- '<|im_end|>\n' }}{%- else %}{%- if _sc %}{{- '<|im_start|>system\n' + (reasoning_instructions + '\n\n' if reasoning_instructions else '') + _sc + '<|im_end|>\n' }}{%- elif reasoning_instructions %}{{- '<|im_start|>system\n' + reasoning_instructions + '<|im_end|>\n' }}{%- endif %}{%- endif %}{%- set _last_idx = _msgs | length - 1 %}{%- set ns = namespace(multi_step_tool=true, last_query_index=_last_idx) %}{%- for message in _msgs[::-1] %}{%- set index = (_msgs | length - 1) - loop.index0 %}{%- if ns.multi_step_tool and message.role == 'user' %}{%- set _rc = render_content(message.content, false) | trim %}{%- if not (_rc.startswith('<tool_response>') and _rc.endswith('</tool_response>')) %}{%- set ns.multi_step_tool = false %}{%- set ns.last_query_index = index %}{%- endif %}{%- endif %}{%- endfor %}{%- if ns.multi_step_tool %}{%- if _last_idx > 50 %}{%- set ns.last_query_index = _last_idx %}{%- else %}{%- set ns.last_query_index = 0 %}{%- endif %}{%- endif %}{%- set ns2 = namespace(prev_role='', consecutive_failures=0) %}{%- for message in _msgs %}{%- set is_system = (message.role == "system" or message.role == "developer") %}{%- set content = render_content(message.content, true, is_system) | trim %}{%- if is_system or message.role == 'user' %}{%- if '<|think_off|>' in content %}{%- set content = content.split('<|think_off|>') | join('') | trim %}{%- endif %}{%- if '<|think_on|>' in content %}{%- set content = content.split('<|think_on|>') | join('') | trim %}{%- endif %}{%- if '<|think_xhigh|>' in content %}{%- set content = content.split('<|think_xhigh|>') | join('') | trim %}{%- endif %}{%- if '<|think_high|>' in content %}{%- set content = content.split('<|think_high|>') | join('') | trim %}{%- endif %}{%- if '<|think_medium|>' in content %}{%- set content = content.split('<|think_medium|>') | join('') | trim %}{%- endif %}{%- if '<|think_low|>' in content %}{%- set content = content.split('<|think_low|>') | join('') | trim %}{%- endif %}{%- if '<|think_minimal|>' in content %}{%- set content = content.split('<|think_minimal|>') | join('') | trim %}{%- endif %}{%- endif %}{%- if is_system %}{{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}{%- elif message.role == 'user' %}{%- set ns2.consecutive_failures = 0 %}{{- '<|im_start|>user\n' + content + '<|im_end|>\n' }}{%- elif message.role == 'assistant' %}{%- set reasoning_content = '' %}{%- if message.reasoning_content is defined and message.reasoning_content is not none %}{%- if message.reasoning_content is string %}{%- set reasoning_content = message.reasoning_content %}{%- else %}{%- set reasoning_content = message.reasoning_content | string %}{%- endif %}{%- elif message.thinking is defined and message.thinking is not none %}{%- if message.thinking is string %}{%- set reasoning_content = message.thinking %}{%- else %}{%- set reasoning_content = message.thinking | string %}{%- endif %}{%- else %}{%- set _think_end = '' %}{%- if content.startswith('</think>') %}{%- set _think_end = '</think>' %}{%- elif content.startswith('</thinking>') %}{%- set _think_end = '</thinking>' %}{%- elif '\n</think>' in content %}{%- set _think_end = '\n</think>' %}{%- elif '\n</thinking>' in content %}{%- set _think_end = '\n</thinking>' %}{%- elif '\n</ think>' in content %}{%- set _think_end = '\n</ think>' %}{%- elif '\n</think >' in content %}{%- set _think_end = '\n</think >' %}{%- endif %}{%- if _think_end %}{%- if 'thinking' in _think_end %}{%- set _think_start = '<thinking>' %}{%- else %}{%- set _think_start = '<think>' %}{%- endif %}{%- set reasoning_content = content.split(_think_end)[0].rstrip('\n') %}{%- if _think_start in reasoning_content %}{%- set reasoning_content = reasoning_content.split(_think_start)[-1].lstrip('\n') %}{%- endif %}{%- set content = content.split(_think_end)[-1].lstrip('\n') %}{%- endif %}{%- endif %}{%- set reasoning_content = reasoning_content | trim %}{%- if (_preserve_thinking or loop.index0 > ns.last_query_index) and reasoning_content %}{{- '<|im_start|>assistant\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}{%- else %}{{- '<|im_start|>assistant\n' + content }}{%- endif %}{%- if message.tool_calls is defined and message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}{%- for tool_call in message.tool_calls %}{%- if tool_call.function is defined and tool_call.function is not none %}{%- set tc = tool_call.function %}{%- else %}{%- set tc = tool_call %}{%- endif %}{%- set tc_name = tc.name if (tc.name is defined and tc.name is not none) else '' %}{%- if _tool_format == 'json' %}{%- if not loop.first or content | trim %}{{- '\n\n' }}{%- endif %}{%- set _args = '{}' %}{%- if tc.arguments is defined and tc.arguments is not none %}{%- if tc.arguments is mapping %}{%- set _args = tc.arguments | tojson %}{%- elif tc.arguments is string and tc.arguments %}{%- set _args = tc.arguments %}{%- endif %}{%- endif %}{{- '<tool_call>\n{"name": ' }}{{- tc_name | tojson }}{{- ', "arguments": ' }}{{- _args }}{{- '}\n</tool_call>' }}{%- else %}{%- if loop.first %}{%- if content | trim %}{{- '\n\n<tool_call>\n<function=' + tc_name + '>\n' }}{%- else %}{{- '<tool_call>\n<function=' + tc_name + '>\n' }}{%- endif %}{%- else %}{{- '\n\n<tool_call>\n<function=' + tc_name + '>\n' }}{%- endif %}{%- if tc.arguments is defined and tc.arguments is not none %}{%- if tc.arguments is mapping %}{%- for args_name, args_value in tc.arguments.items() %}{{- '<parameter=' + args_name + '>\n' }}{%- if args_value is mapping or (args_value is sequence and args_value is not string) %}{%- set _av = args_value | tojson %}{%- else %}{%- set _av = args_value | string %}{%- endif %}{%- if max_tool_arg_chars > 0 and _av | length > max_tool_arg_chars %}{{- _av[:max_tool_arg_chars] + '\n[TRUNCATED - original length ' ~ (_av | length | string) ~ ' chars]' }}{%- else %}{{- _av }}{%- endif %}{{- '\n</parameter>\n' }}{%- endfor %}{%- elif tc.arguments is string and tc.arguments %}{{- tc.arguments }}{%- endif %}{%- endif %}{{- '</function>\n</tool_call>' }}{%- endif %}{%- endfor %}{%- endif %}{{- '<|im_end|>\n' }}{%- elif message.role == 'tool' %}{%- set _content_lower = content | lower %}{%- set _content_head = _content_lower[:80] %}{%- if content | length < 500 and '$ ' not in content and 'took ' not in _content_lower and ('"error":' in _content_head or 'error:' in _content_head or 'err!' in _content_head or 'fatal:' in _content_head or 'exception:' in _content_head or 'traceback' in _content_head or 'command not found' in _content_head or 'invalid syntax' in _content_head or 'failed to' in _content_head) %}{%- set ns2.consecutive_failures = ns2.consecutive_failures + 1 %}{%- else %}{%- set ns2.consecutive_failures = 0 %}{%- endif %}{%- if ns2.prev_role != 'tool' %}{{- '<|im_start|>user' }}{%- endif %}{%- if max_tool_response_chars > 0 and content | length > max_tool_response_chars %}{%- set content = content[:max_tool_response_chars] + '\n[TRUNCATED - original length ' ~ (content | length | string) ~ ' chars]' %}{%- endif %}{{- '\n<tool_response>\n' + content }}{%- if ns2.consecutive_failures >= 2 %}{{- '\n\n⚠️ SYSTEM WARNING: ' ~ ns2.consecutive_failures ~ ' consecutive tool errors detected. Your previous approach is incorrect. You MUST use a fundamentally different approach or corrected arguments.' }}{%- elif ns2.consecutive_failures == 1 %}{{- '\n\n⚠️ SYSTEM WARNING: The previous tool call returned an error. Diagnose the failure and retry with completely corrected arguments.' }}{%- endif %}{{- '\n</tool_response>' }}{%- if loop.last %}{{- '<|im_end|>\n' }}{%- else %}{%- set _next_role = _msgs[loop.index0 + 1].role %}{%- if _next_role != 'tool' %}{{- '<|im_end|>\n' }}{%- endif %}{%- endif %}{%- else %}{{- '<|im_start|>user\n[' + message.role + ']: ' + content + '<|im_end|>\n' }}{%- endif %}{%- set ns2.prev_role = message.role %}{%- endfor %}{%- if add_generation_prompt %}{{- '<|im_start|>assistant\n' }}{%- if not ns_state.thinking %}{{- '<think>\n\n</think>\n\n' }}{%- else %}{{- '<think>\n' }}{%- endif %}{%- endif %}
|
|
|
|
| 1 |
+
{%- set template_version = "qwen3.8-froggeric-v22.3.1" %}{%- set _tool_format = tool_call_format if tool_call_format is defined else 'xml' %}{%- set image_count = namespace(value=0) %}{%- set video_count = namespace(value=0) %}{%- set add_vision_id = add_vision_id if add_vision_id is defined else false %}{%- set enable_thinking = enable_thinking if enable_thinking is defined else true %}{%- set auto_disable_thinking_with_tools = auto_disable_thinking_with_tools if auto_disable_thinking_with_tools is defined else false %}{%- if preserve_reasoning is defined and preserve_reasoning is not none %}{%- set _preserve_thinking = preserve_reasoning %}{%- elif preserve_thinking is defined and preserve_thinking is not none %}{%- set _preserve_thinking = preserve_thinking %}{%- else %}{%- set _preserve_thinking = true %}{%- endif %}{%- set max_tool_arg_chars = max_tool_arg_chars if max_tool_arg_chars is defined else 0 %}{%- set max_tool_response_chars = max_tool_response_chars if max_tool_response_chars is defined else 0 %}{%- set _has_tools = (tools is defined and tools and tools is iterable and tools is not mapping) %}{%- set _effort_raw = (reasoning_effort | string | lower) if reasoning_effort is defined and reasoning_effort is not none else 'medium' %}{%- set _initial_thinking = enable_thinking %}{%- set _initial_effort = 'medium' %}{%- if _effort_raw in ('none', 'off') %}{%- set _initial_thinking = false %}{%- set _initial_effort = 'medium' %}{%- elif _effort_raw in ('minimal', 'low') %}{%- set _initial_effort = 'low' %}{%- elif _effort_raw in ('high', 'xhigh', 'max', 'ultracode', 'extreme') %}{%- set _initial_effort = 'xhigh' %}{%- else %}{%- set _initial_effort = 'medium' %}{%- endif %}{%- set ns_state = namespace(thinking=_initial_thinking, effort=_initial_effort) %}{%- if auto_disable_thinking_with_tools and _has_tools %}{%- set ns_state.thinking = false %}{%- endif %}{%- for msg in messages %}{%- if msg.role == 'system' or msg.role == 'developer' or msg.role == 'user' %}{%- if msg.content is string %}{%- if '<|think_off|>' in msg.content %}{%- set ns_state.thinking = false %}{%- elif '<|think_on|>' in msg.content %}{%- set ns_state.thinking = true %}{%- elif '<|think_xhigh|>' in msg.content or '<|think_high|>' in msg.content or '<|think_ultracode|>' in msg.content or '<|think_extreme|>' in msg.content or '<|think_max|>' in msg.content %}{%- set ns_state.thinking = true %}{%- set ns_state.effort = 'xhigh' %}{%- elif '<|think_low|>' in msg.content or '<|think_minimal|>' in msg.content %}{%- set ns_state.thinking = true %}{%- set ns_state.effort = 'low' %}{%- elif '<|think_medium|>' in msg.content %}{%- set ns_state.thinking = true %}{%- set ns_state.effort = 'medium' %}{%- endif %}{%- elif msg.content is iterable and msg.content is not mapping %}{%- for item in msg.content %}{%- if item is string %}{%- set _item_text = item %}{%- elif item is mapping and 'text' in item and item.text is string %}{%- set _item_text = item.text %}{%- else %}{%- set _item_text = '' %}{%- endif %}{%- if _item_text %}{%- if '<|think_off|>' in _item_text %}{%- set ns_state.thinking = false %}{%- elif '<|think_on|>' in _item_text %}{%- set ns_state.thinking = true %}{%- elif '<|think_xhigh|>' in _item_text or '<|think_high|>' in _item_text or '<|think_ultracode|>' in _item_text or '<|think_extreme|>' in _item_text or '<|think_max|>' in _item_text %}{%- set ns_state.thinking = true %}{%- set ns_state.effort = 'xhigh' %}{%- elif '<|think_low|>' in _item_text or '<|think_minimal|>' in _item_text %}{%- set ns_state.thinking = true %}{%- set ns_state.effort = 'low' %}{%- elif '<|think_medium|>' in _item_text %}{%- set ns_state.thinking = true %}{%- set ns_state.effort = 'medium' %}{%- endif %}{%- endif %}{%- endfor %}{%- endif %}{%- endif %}{%- endfor %}{%- set reasoning_instructions = '' %}{%- if ns_state.thinking %}{%- if ns_state.effort == 'xhigh' %}{%- set reasoning_instructions = 'Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.' %}{%- elif ns_state.effort == 'low' %}{%- set reasoning_instructions = 'Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration.' %}{%- endif %}{%- endif %}{%- macro render_content(content, do_vision_count, is_system_content=false) %}{%- if content is string %}{{- content }}{%- elif content is iterable and content is not mapping %}{%- for item in content %}{%- if item is mapping %}{%- if item.type == 'image' or 'image' in item or 'image_url' in item %}{%- if is_system_content %}{{- raise_exception('System message cannot contain images.') }}{%- endif %}{%- if do_vision_count %}{%- set image_count.value = image_count.value + 1 %}{%- endif %}{%- if add_vision_id %}{{- 'Picture ' ~ image_count.value ~ ': ' }}{%- endif %}{{- '<|vision_start|><|image_pad|><|vision_end|>' }}{%- elif item.type == 'video' or 'video' in item %}{%- if is_system_content %}{{- raise_exception('System message cannot contain videos.') }}{%- endif %}{%- if do_vision_count %}{%- set video_count.value = video_count.value + 1 %}{%- endif %}{%- if add_vision_id %}{{- 'Video ' ~ video_count.value ~ ': ' }}{%- endif %}{{- '<|vision_start|><|video_pad|><|vision_end|>' }}{%- elif 'text' in item %}{{- item.text }}{%- else %}{{- raise_exception('Unexpected item type in content.') }}{%- endif %}{%- else %}{{- item | string }}{%- endif %}{%- endfor %}{%- elif content is none or content is undefined %}{{- '' }}{%- else %}{{- raise_exception('Unexpected content type.') }}{%- endif %}{%- endmacro %}{%- if not messages %}{{- raise_exception('No messages provided.') }}{%- endif %}{%- set head = namespace(count=0, seen_non_system=false) %}{%- for message in messages %}{%- set _is_sys = (message.role == 'system' or message.role == 'developer') %}{%- if _is_sys and not head.seen_non_system %}{%- set head.count = head.count + 1 %}{%- else %}{%- set head.seen_non_system = true %}{%- endif %}{%- endfor %}{%- set sys_state = namespace(content='') %}{%- for message in messages[:head.count] %}{%- set _part = render_content(message.content, false, true) | trim %}{%- if '<|think_off|>' in _part %}{%- set _part = _part.split('<|think_off|>') | join('') | trim %}{%- endif %}{%- if '<|think_on|>' in _part %}{%- set _part = _part.split('<|think_on|>') | join('') | trim %}{%- endif %}{%- if '<|think_xhigh|>' in _part %}{%- set _part = _part.split('<|think_xhigh|>') | join('') | trim %}{%- endif %}{%- if '<|think_high|>' in _part %}{%- set _part = _part.split('<|think_high|>') | join('') | trim %}{%- endif %}{%- if '<|think_ultracode|>' in _part %}{%- set _part = _part.split('<|think_ultracode|>') | join('') | trim %}{%- endif %}{%- if '<|think_extreme|>' in _part %}{%- set _part = _part.split('<|think_extreme|>') | join('') | trim %}{%- endif %}{%- if '<|think_max|>' in _part %}{%- set _part = _part.split('<|think_max|>') | join('') | trim %}{%- endif %}{%- if '<|think_medium|>' in _part %}{%- set _part = _part.split('<|think_medium|>') | join('') | trim %}{%- endif %}{%- if '<|think_low|>' in _part %}{%- set _part = _part.split('<|think_low|>') | join('') | trim %}{%- endif %}{%- if '<|think_minimal|>' in _part %}{%- set _part = _part.split('<|think_minimal|>') | join('') | trim %}{%- endif %}{%- if _part %}{%- if sys_state.content %}{%- set sys_state.content = sys_state.content ~ '\n\n' ~ _part %}{%- else %}{%- set sys_state.content = _part %}{%- endif %}{%- endif %}{%- endfor %}{%- set _sc = sys_state.content %}{%- set _msgs = messages[head.count:] %}{%- if ns_state.thinking %}{%- set _terse_lead = 'Answer directly, after thinking. Lead with the answer, then only what it needs to be correct and usable.' %}{%- else %}{%- set _terse_lead = 'Answer directly and concisely. Give the answer with only what it needs to be correct and usable.' %}{%- endif %} {%- set _terse_core %}
|
| 2 |
Never: open with preamble or pleasantries; restate the question; add filler transitions; hedge with niceties; or repeat a point you've already made.
|
| 3 |
Always: keep essential steps, caveats, uncertainties, and specifics — never drop correctness or a needed warning for brevity. Keep the final answer lean. Use the least structure that conveys it (plain prose when short; lists or code only when they earn their place). If genuinely uncertain, say so and explain why — never omit uncertainty for the sake of brevity.
|
| 4 |
If a user request is genuinely ambiguous, ask a sharp question, don't guess.
|
| 5 |
+
{%- endset %} {%- set _terse = _terse_lead ~ '\n' ~ (_terse_core | trim) %}{%- if not _sc %}{%- set _sc = _terse | trim %}{%- else %}{%- set _sc = (_sc | trim) ~ '\n\n' ~ (_terse | trim) %}{%- endif %}{%- if _has_tools %}{{- '<|im_start|>system\n' }}{%- if reasoning_instructions %}{{- reasoning_instructions + '\n\n' }}{%- endif %}{{- '# Tools\n\nYou have access to the following functions:\n\n<tools>' }}{%- for tool in tools %}{{- '\n' }}{{- tool | tojson }}{%- endfor %}{{- '\n</tools>' }}{%- if _tool_format == 'json' %}{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n' }}{%- if ns_state.thinking %}{{- '<think>\nBrief explanation of tool call\n</think>\n' }}{%- endif %}{{- '<tool_call>\n{"name": "example_function_name", "arguments": {"example_parameter_1": "value_1", "example_parameter_2": "This is the value for the second parameter"}}\n</tool_call>\n\n<IMPORTANT>\nReminder:\n' }}{%- if ns_state.thinking %}{{- '- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n' }}{%- endif %}{{- '- Function calls MUST follow the specified format: a single JSON object with "name" and "arguments" keys inside <tool_call></tool_call> XML tags.\n- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY' }}{%- if ns_state.thinking %}{{- ' after thinking' }}{%- endif %}{{- ', with NO conversational text before it.\n- The <tool_call> tag MUST be at the very beginning of a new line, with NO spaces or indentation before it.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}{%- else %}{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n' }}{%- if ns_state.thinking %}{{- '<think>\nBrief explanation of tool call\n</think>\n' }}{%- endif %}{{- '<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n' }}{%- if ns_state.thinking %}{{- '- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n' }}{%- endif %}{{- '- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags.\n- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY' }}{%- if ns_state.thinking %}{{- ' after thinking' }}{%- endif %}{{- ', with NO conversational text before it.\n- The <tool_call> and <function> tags MUST be at the very beginning of a new line, with NO spaces or indentation before them.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}{%- endif %}{%- if _sc %}{{- '\n\n' + _sc }}{%- endif %}{{- '<|im_end|>\n' }}{%- else %}{%- if _sc %}{{- '<|im_start|>system\n' + (reasoning_instructions + '\n\n' if reasoning_instructions else '') + _sc + '<|im_end|>\n' }}{%- elif reasoning_instructions %}{{- '<|im_start|>system\n' + reasoning_instructions + '<|im_end|>\n' }}{%- endif %}{%- endif %}{%- set _last_idx = _msgs | length - 1 %}{%- set ns = namespace(multi_step_tool=true, last_query_index=_last_idx) %}{%- for message in _msgs[::-1] %}{%- set index = (_msgs | length - 1) - loop.index0 %}{%- if ns.multi_step_tool and message.role == 'user' %}{%- set _rc = render_content(message.content, false) | trim %}{%- if not (_rc.startswith('<tool_response>') and _rc.endswith('</tool_response>')) %}{%- set ns.multi_step_tool = false %}{%- set ns.last_query_index = index %}{%- endif %}{%- endif %}{%- endfor %}{%- if ns.multi_step_tool %}{%- if _last_idx > 50 %}{%- set ns.last_query_index = _last_idx %}{%- else %}{%- set ns.last_query_index = 0 %}{%- endif %}{%- endif %}{%- set ns2 = namespace(prev_role='', consecutive_failures=0) %}{%- for message in _msgs %}{%- set is_system = (message.role == "system" or message.role == "developer") %}{%- set content = render_content(message.content, true, is_system) | trim %}{%- if is_system or message.role == 'user' %}{%- if '<|think_off|>' in content %}{%- set content = content.split('<|think_off|>') | join('') | trim %}{%- endif %}{%- if '<|think_on|>' in content %}{%- set content = content.split('<|think_on|>') | join('') | trim %}{%- endif %}{%- if '<|think_xhigh|>' in content %}{%- set content = content.split('<|think_xhigh|>') | join('') | trim %}{%- endif %}{%- if '<|think_high|>' in content %}{%- set content = content.split('<|think_high|>') | join('') | trim %}{%- endif %}{%- if '<|think_ultracode|>' in content %}{%- set content = content.split('<|think_ultracode|>') | join('') | trim %}{%- endif %}{%- if '<|think_extreme|>' in content %}{%- set content = content.split('<|think_extreme|>') | join('') | trim %}{%- endif %}{%- if '<|think_max|>' in content %}{%- set content = content.split('<|think_max|>') | join('') | trim %}{%- endif %}{%- if '<|think_medium|>' in content %}{%- set content = content.split('<|think_medium|>') | join('') | trim %}{%- endif %}{%- if '<|think_low|>' in content %}{%- set content = content.split('<|think_low|>') | join('') | trim %}{%- endif %}{%- if '<|think_minimal|>' in content %}{%- set content = content.split('<|think_minimal|>') | join('') | trim %}{%- endif %}{%- endif %}{%- if is_system %}{{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}{%- elif message.role == 'user' %}{%- set ns2.consecutive_failures = 0 %}{{- '<|im_start|>user\n' + content + '<|im_end|>\n' }}{%- elif message.role == 'assistant' %}{%- set reasoning_content = '' %}{%- set _explicit_reasoning = '' %}{%- if message.reasoning_content is defined and message.reasoning_content is not none %}{%- if message.reasoning_content is string %}{%- set _explicit_reasoning = message.reasoning_content %}{%- else %}{%- set _explicit_reasoning = message.reasoning_content | string %}{%- endif %}{%- elif message.thinking is defined and message.thinking is not none %}{%- if message.thinking is string %}{%- set _explicit_reasoning = message.thinking %}{%- else %}{%- set _explicit_reasoning = message.thinking | string %}{%- endif %}{%- endif %}{%- if _explicit_reasoning %}{%- set _lead_end = '' %}{%- if content.startswith('<think>') and '</think>' in content %}{%- set _lead_end = '</think>' %}{%- elif content.startswith('<thinking>') and '</thinking>' in content %}{%- set _lead_end = '</thinking>' %}{%- elif content.startswith('</think>') %}{%- set _lead_end = '</think>' %}{%- elif content.startswith('</thinking>') %}{%- set _lead_end = '</thinking>' %}{%- endif %}{%- if _lead_end %}{%- set content = content.split(_lead_end)[-1].lstrip('\n') %}{%- endif %}{%- set reasoning_content = _explicit_reasoning %}{%- else %}{%- set _think_end = '' %}{%- if content.startswith('</think>') %}{%- set _think_end = '</think>' %}{%- elif content.startswith('</thinking>') %}{%- set _think_end = '</thinking>' %}{%- elif '\n</think>' in content %}{%- set _think_end = '\n</think>' %}{%- elif '\n</thinking>' in content %}{%- set _think_end = '\n</thinking>' %}{%- elif '\n</ think>' in content %}{%- set _think_end = '\n</ think>' %}{%- elif '\n</think >' in content %}{%- set _think_end = '\n</think >' %}{%- elif content.startswith('<think>') and '</think>' in content %}{%- set _think_end = '</think>' %}{%- elif content.startswith('<thinking>') and '</thinking>' in content %}{%- set _think_end = '</thinking>' %}{%- endif %}{%- if _think_end %}{%- if 'thinking' in _think_end %}{%- set _think_start = '<thinking>' %}{%- else %}{%- set _think_start = '<think>' %}{%- endif %}{%- set reasoning_content = content.split(_think_end)[0].rstrip('\n') %}{%- if _think_start in reasoning_content %}{%- set reasoning_content = reasoning_content.split(_think_start)[-1].lstrip('\n') %}{%- endif %}{%- set content = content.split(_think_end)[-1].lstrip('\n') %}{%- endif %}{%- endif %}{%- set reasoning_content = reasoning_content | trim %}{%- if (_preserve_thinking or loop.index0 > ns.last_query_index) %}{{- '<|im_start|>assistant\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}{%- else %}{{- '<|im_start|>assistant\n' + content }}{%- endif %}{%- if message.tool_calls is defined and message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}{%- for tool_call in message.tool_calls %}{%- if tool_call.function is defined and tool_call.function is not none %}{%- set tc = tool_call.function %}{%- else %}{%- set tc = tool_call %}{%- endif %}{%- set tc_name = tc.name if (tc.name is defined and tc.name is not none) else '' %}{%- if _tool_format == 'json' %}{%- if not loop.first or content | trim %}{{- '\n\n' }}{%- endif %}{%- set _args = '{}' %}{%- if tc.arguments is defined and tc.arguments is not none %}{%- if tc.arguments is mapping %}{%- set _args = tc.arguments | tojson %}{%- elif tc.arguments is string %}{%- if tc.arguments %}{%- set _args = tc.arguments %}{%- endif %}{%- else %}{%- set _args = tc.arguments | tojson %}{%- endif %}{%- endif %}{{- '<tool_call>\n{"name": ' }}{{- tc_name | tojson }}{{- ', "arguments": ' }}{{- _args }}{{- '}\n</tool_call>' }}{%- else %}{%- if loop.first %}{%- if content | trim %}{{- '\n\n<tool_call>\n<function=' + tc_name + '>\n' }}{%- else %}{{- '<tool_call>\n<function=' + tc_name + '>\n' }}{%- endif %}{%- else %}{{- '\n\n<tool_call>\n<function=' + tc_name + '>\n' }}{%- endif %}{%- if tc.arguments is defined and tc.arguments is not none %}{%- if tc.arguments is mapping %}{%- for args_name, args_value in tc.arguments.items() %}{{- '<parameter=' + args_name + '>\n' }}{%- if args_value is string %}{%- set _av = args_value %}{%- else %}{%- set _av = args_value | tojson %}{%- endif %}{%- if max_tool_arg_chars > 0 and _av | length > max_tool_arg_chars %}{{- _av[:max_tool_arg_chars] + '\n[TRUNCATED - original length ' ~ (_av | length | string) ~ ' chars]' }}{%- else %}{{- _av }}{%- endif %}{{- '\n</parameter>\n' }}{%- endfor %}{%- else %}{%- if tc.arguments is string %}{%- set _raw_args = tc.arguments %}{%- else %}{%- set _raw_args = tc.arguments | tojson %}{%- endif %}{%- if _raw_args %}{%- if max_tool_arg_chars > 0 and _raw_args | length > max_tool_arg_chars %}{{- _raw_args[:max_tool_arg_chars] + '\n[TRUNCATED - original length ' ~ (_raw_args | length | string) ~ ' chars]' }}{%- else %}{{- _raw_args }}{%- endif %}{%- endif %}{%- endif %}{%- endif %}{{- '</function>\n</tool_call>' }}{%- endif %}{%- endfor %}{%- endif %}{{- '<|im_end|>\n' }}{%- elif message.role == 'tool' %}{%- set _content_lower = content | lower %}{%- set _content_head = _content_lower[:120] %}{%- set _is_code_or_grep = ('throw new ' in _content_lower or 'throw error' in _content_lower or 'console.error' in _content_lower or 'logger.error' in _content_lower or 'logging.error' in _content_lower or 'import ' in _content_head or 'def ' in _content_head or 'function ' in _content_head) %}{%- set _exit_code_zero = ('exit code: 0' in _content_head or 'process exited with code 0' in _content_head) %}{%- set _error_field_ok = ('"error": null' in _content_head or '"error":null' in _content_head or '"error": false' in _content_head or '"error":false' in _content_head or '"error": ""' in _content_head or '"error":""' in _content_head) %}{%- set _strong_error = (('"error":' in _content_head and not _error_field_ok) or '"status": "error"' in _content_head or '"status":"error"' in _content_head or 'traceback (most recent call last):' in _content_head or 'command not found' in _content_head or 'invalid syntax' in _content_head or 'fatal:' in _content_head or (('exit code: ' in _content_head or 'process exited with code' in _content_head) and not _exit_code_zero) or _content_head.startswith('exception:') or _content_head.startswith('failed to ')) %}{%- set _weak_error = ('error:' in _content_head or 'err!' in _content_head) %}{%- set _weak_suppressed = ('$ ' in _content_head or 'took ' in _content_head or content | length >= 600) %}{%- if not _is_code_or_grep and (_strong_error or (_weak_error and not _weak_suppressed)) %}{%- set ns2.consecutive_failures = ns2.consecutive_failures + 1 %}{%- else %}{%- set ns2.consecutive_failures = 0 %}{%- endif %}{%- if ns2.prev_role != 'tool' %}{{- '<|im_start|>user' }}{%- endif %}{%- if _tool_format != 'json' and max_tool_response_chars > 0 and content | length > max_tool_response_chars %}{%- set content = content[:max_tool_response_chars] + '\n[TRUNCATED - original length ' ~ (content | length | string) ~ ' chars]' %}{%- endif %}{{- '\n<tool_response>\n' + content }}{%- if ns2.consecutive_failures >= 2 %}{{- '\n\n⚠️ SYSTEM WARNING: ' ~ ns2.consecutive_failures ~ ' consecutive tool errors detected. Your previous approach is incorrect. You MUST use a fundamentally different approach or corrected arguments.' }}{%- elif ns2.consecutive_failures == 1 %}{{- '\n\n⚠️ SYSTEM WARNING: The previous tool call returned an error. Diagnose the failure and retry with completely corrected arguments.' }}{%- endif %}{{- '\n</tool_response>' }}{%- if loop.last %}{{- '<|im_end|>\n' }}{%- else %}{%- set _next_role = _msgs[loop.index0 + 1].role %}{%- if _next_role != 'tool' %}{{- '<|im_end|>\n' }}{%- endif %}{%- endif %}{%- else %}{{- '<|im_start|>user\n[' + message.role + ']: ' + content + '<|im_end|>\n' }}{%- endif %}{%- set ns2.prev_role = message.role %}{%- endfor %}{%- if add_generation_prompt %}{{- '<|im_start|>assistant\n' }}{%- if not ns_state.thinking %}{{- '<think>\n\n</think>\n\n' }}{%- else %}{{- '<think>\n' }}{%- endif %}{%- endif %}
|
scripts/check_applied.py
CHANGED
|
@@ -19,9 +19,11 @@ from __future__ import annotations
|
|
| 19 |
|
| 20 |
import json
|
| 21 |
import pathlib
|
|
|
|
| 22 |
import sys
|
| 23 |
|
| 24 |
MARKER = "Never: open with preamble"
|
|
|
|
| 25 |
SYSTEM_PROBE = "Be a pirate."
|
| 26 |
|
| 27 |
|
|
@@ -34,12 +36,47 @@ def render(src: str, msgs: list[dict], **kw) -> str:
|
|
| 34 |
messages=msgs, add_generation_prompt=True, **kw)
|
| 35 |
|
| 36 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 37 |
def describe(src: str) -> dict:
|
| 38 |
"""Render the cases that matter and report what the template does."""
|
| 39 |
user = [{"role": "user", "content": "hi"}]
|
| 40 |
with_sys = [{"role": "system", "content": SYSTEM_PROBE}, {"role": "user", "content": "hi"}]
|
| 41 |
multi = [{"role": "user", "content": "Q1"},
|
| 42 |
-
{"role": "assistant", "content": "<think>
|
| 43 |
{"role": "user", "content": "Q2"}]
|
| 44 |
try:
|
| 45 |
plain, sysd, mt = render(src, user), render(src, with_sys), render(src, multi)
|
|
@@ -48,7 +85,7 @@ def describe(src: str) -> dict:
|
|
| 48 |
return {
|
| 49 |
"terse_count": plain.count(MARKER),
|
| 50 |
"keeps_system": SYSTEM_PROBE in sysd,
|
| 51 |
-
"retains_think":
|
| 52 |
"identity": next((n for n in ("Nail-35b-a3b", "Dagger-27b") if n in plain), None),
|
| 53 |
"bytes": len(src),
|
| 54 |
}
|
|
@@ -120,18 +157,28 @@ def main() -> int:
|
|
| 120 |
ok = all([report(name, src) for name, src in sources.items()])
|
| 121 |
|
| 122 |
if len(sources) > 1:
|
| 123 |
-
|
| 124 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 125 |
print()
|
| 126 |
-
if
|
| 127 |
-
|
| 128 |
-
print("
|
|
|
|
|
|
|
| 129 |
else:
|
| 130 |
ok = False
|
| 131 |
print(" *** THE TWO SOURCES DISAGREE ***")
|
| 132 |
print(" Recent transformers uses chat_template.jinja; oMLX and others read the")
|
| 133 |
-
print(" copy embedded in tokenizer_config.json. Right now those
|
| 134 |
-
print(" you get depends on your runtime. Patch both to the same
|
| 135 |
|
| 136 |
print("\n * retention is inferred from the rendered output; a template that merely echoes"
|
| 137 |
"\n message content passes it without implementing retention. Trust the 'no'.")
|
|
|
|
| 19 |
|
| 20 |
import json
|
| 21 |
import pathlib
|
| 22 |
+
import re
|
| 23 |
import sys
|
| 24 |
|
| 25 |
MARKER = "Never: open with preamble"
|
| 26 |
+
THINK_PROBE = "kept-thought-4f2a"
|
| 27 |
SYSTEM_PROBE = "Be a pirate."
|
| 28 |
|
| 29 |
|
|
|
|
| 36 |
messages=msgs, add_generation_prompt=True, **kw)
|
| 37 |
|
| 38 |
|
| 39 |
+
def render_probe(src: str) -> tuple:
|
| 40 |
+
"""What this template actually produces, across the cases the differences would show up in."""
|
| 41 |
+
user = [{"role": "user", "content": "hi"}]
|
| 42 |
+
cases = [
|
| 43 |
+
(user, {}),
|
| 44 |
+
([{"role": "system", "content": SYSTEM_PROBE}] + user, {}),
|
| 45 |
+
(user, {"enable_thinking": False}),
|
| 46 |
+
(user, {"reasoning_effort": "low"}),
|
| 47 |
+
([{"role": "user", "content": "Q1"},
|
| 48 |
+
{"role": "assistant", "content": f"<think>{THINK_PROBE}</think>A1"},
|
| 49 |
+
{"role": "user", "content": "Q2"}], {}),
|
| 50 |
+
]
|
| 51 |
+
out = []
|
| 52 |
+
for msgs, kw in cases:
|
| 53 |
+
try:
|
| 54 |
+
out.append(render(src, msgs, **kw))
|
| 55 |
+
except Exception as e: # a template that throws differs from one that does not
|
| 56 |
+
out.append(f"__ERROR__{type(e).__name__}")
|
| 57 |
+
return tuple(out)
|
| 58 |
+
|
| 59 |
+
|
| 60 |
+
def think_kept(rendered: str) -> bool:
|
| 61 |
+
"""Did last turn's reasoning survive into this prompt?
|
| 62 |
+
|
| 63 |
+
Two shapes count. froggeric <= v22.1 passes the assistant's `<think>` tags through
|
| 64 |
+
verbatim; v22.2+ extracts in-content reasoning and re-emits it as a canonical
|
| 65 |
+
`<think>\\n...\\n</think>` block (that extraction is the fix for duplicated tags). Both
|
| 66 |
+
retain the thought, which is the thing being measured -- so match on the probe text
|
| 67 |
+
living inside a think block, not on either literal tag layout. A stock template drops
|
| 68 |
+
the reasoning entirely and fails both.
|
| 69 |
+
"""
|
| 70 |
+
return any(THINK_PROBE in blk
|
| 71 |
+
for blk in re.findall(r"<think>(.*?)</think>", rendered, re.DOTALL))
|
| 72 |
+
|
| 73 |
+
|
| 74 |
def describe(src: str) -> dict:
|
| 75 |
"""Render the cases that matter and report what the template does."""
|
| 76 |
user = [{"role": "user", "content": "hi"}]
|
| 77 |
with_sys = [{"role": "system", "content": SYSTEM_PROBE}, {"role": "user", "content": "hi"}]
|
| 78 |
multi = [{"role": "user", "content": "Q1"},
|
| 79 |
+
{"role": "assistant", "content": f"<think>{THINK_PROBE}</think>A1"},
|
| 80 |
{"role": "user", "content": "Q2"}]
|
| 81 |
try:
|
| 82 |
plain, sysd, mt = render(src, user), render(src, with_sys), render(src, multi)
|
|
|
|
| 85 |
return {
|
| 86 |
"terse_count": plain.count(MARKER),
|
| 87 |
"keeps_system": SYSTEM_PROBE in sysd,
|
| 88 |
+
"retains_think": think_kept(mt),
|
| 89 |
"identity": next((n for n in ("Nail-35b-a3b", "Dagger-27b") if n in plain), None),
|
| 90 |
"bytes": len(src),
|
| 91 |
}
|
|
|
|
| 157 |
ok = all([report(name, src) for name, src in sources.items()])
|
| 158 |
|
| 159 |
if len(sources) > 1:
|
| 160 |
+
# Compare what the sources DO, not how they are spelled. The documented way to patch both
|
| 161 |
+
# places is to paste chat_template_oneline.txt into tokenizer_config.json, which is the
|
| 162 |
+
# minified form of the same template -- byte-different by construction, behaviourally
|
| 163 |
+
# identical. Comparing raw text flagged that recommended state as broken; comparing
|
| 164 |
+
# renderings asks the question that actually matters: does the prompt depend on which
|
| 165 |
+
# source your runtime picked?
|
| 166 |
+
names = list(sources)
|
| 167 |
+
renders = {n: render_probe(sources[n]) for n in names}
|
| 168 |
+
base = renders[names[0]]
|
| 169 |
+
differing = [n for n in names[1:] if renders[n] != base]
|
| 170 |
print()
|
| 171 |
+
if not differing:
|
| 172 |
+
same_text = len({sources[n] for n in names}) == 1
|
| 173 |
+
print(" Both sources render the SAME prompts — whichever your runtime prefers,")
|
| 174 |
+
print(" you get the same behaviour" + ("." if same_text else
|
| 175 |
+
" (they differ only as full vs. minified text)."))
|
| 176 |
else:
|
| 177 |
ok = False
|
| 178 |
print(" *** THE TWO SOURCES DISAGREE ***")
|
| 179 |
print(" Recent transformers uses chat_template.jinja; oMLX and others read the")
|
| 180 |
+
print(" copy embedded in tokenizer_config.json. Right now those RENDER DIFFERENTLY,")
|
| 181 |
+
print(" so what you get depends on your runtime. Patch both to the same template.")
|
| 182 |
|
| 183 |
print("\n * retention is inferred from the rendered output; a template that merely echoes"
|
| 184 |
"\n message content passes it without implementing retention. Trust the 'no'.")
|
scripts/fuzz_template.py
ADDED
|
@@ -0,0 +1,284 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Deterministic property-based fuzzer for the fixed Qwen chat templates.
|
| 2 |
+
|
| 3 |
+
Generates structurally valid conversations (system merging, multipart vision
|
| 4 |
+
content, tool loops with mixed argument shapes, explicit and in-content
|
| 5 |
+
reasoning, planted error payloads) and asserts nine invariants:
|
| 6 |
+
|
| 7 |
+
1. render Rendering never raises on valid input.
|
| 8 |
+
2. parity chat_template.jinja and chat_template_oneline.txt render
|
| 9 |
+
byte-identically.
|
| 10 |
+
3. balance <|im_start|> and <|im_end|> counts match
|
| 11 |
+
(add_generation_prompt=False).
|
| 12 |
+
4. content Planted user text, assistant answers, and (when preserved)
|
| 13 |
+
reasoning appear verbatim in the output.
|
| 14 |
+
5. xml-fidelity In XML mode every mapping tool-argument key appears as
|
| 15 |
+
<parameter=key>; values appear verbatim when truncation
|
| 16 |
+
is disabled.
|
| 17 |
+
6. json-validity In JSON mode every emitted <tool_call> body parses as JSON.
|
| 18 |
+
7. warning The tool-error warning appears iff an error payload was
|
| 19 |
+
planted (no false positives, no false negatives).
|
| 20 |
+
8. prefix render(messages[:k]) is a strict prefix of
|
| 21 |
+
render(messages[:k+1]) at every generation boundary under
|
| 22 |
+
default preserve_thinking. Prefixes that split a merged
|
| 23 |
+
system block or a consecutive tool-result batch are
|
| 24 |
+
skipped: those intermediate states are never rendered in
|
| 25 |
+
real serving, where the model generates only after the
|
| 26 |
+
full tool-result batch is appended.
|
| 27 |
+
9. prefill enable_thinking=False ends the generation prompt with the
|
| 28 |
+
empty think prefill.
|
| 29 |
+
|
| 30 |
+
Usage:
|
| 31 |
+
python3 scripts/fuzz_template.py [--cases 500] [--seed 0] [--template chat_template.jinja]
|
| 32 |
+
|
| 33 |
+
Exits nonzero on any invariant failure and prints a JSON repro for each.
|
| 34 |
+
"""
|
| 35 |
+
import argparse
|
| 36 |
+
import json
|
| 37 |
+
import os
|
| 38 |
+
import random
|
| 39 |
+
import sys
|
| 40 |
+
import traceback
|
| 41 |
+
|
| 42 |
+
from jinja2 import Environment, FileSystemLoader, StrictUndefined
|
| 43 |
+
|
| 44 |
+
# Alphabet deliberately excludes every error-detector keyword, control-tag
|
| 45 |
+
# fragment, and special-token prefix so invariant 7 stays two-sided.
|
| 46 |
+
WORDS = ['alpha', 'bravo', 'delta', 'gamma', 'lumen', 'quartz', 'river',
|
| 47 |
+
'stone', 'matrix', 'vector', 'naïve', 'café', '数据', '結果']
|
| 48 |
+
|
| 49 |
+
|
| 50 |
+
def _phrase(rnd, lo=2, hi=5):
|
| 51 |
+
sep = '\n' if rnd.random() < 0.1 else ' '
|
| 52 |
+
return sep.join(rnd.choice(WORDS) + str(rnd.randint(0, 99))
|
| 53 |
+
for _ in range(rnd.randint(lo, hi)))
|
| 54 |
+
|
| 55 |
+
|
| 56 |
+
def _gen_case(rnd):
|
| 57 |
+
kwargs = {}
|
| 58 |
+
fmt = 'json' if rnd.random() < 0.3 else 'xml'
|
| 59 |
+
if fmt == 'json':
|
| 60 |
+
kwargs['tool_call_format'] = 'json'
|
| 61 |
+
if rnd.random() < 0.3:
|
| 62 |
+
kwargs['reasoning_effort'] = rnd.choice(['low', 'medium', 'high', 'xhigh'])
|
| 63 |
+
no_think = rnd.random() < 0.15
|
| 64 |
+
if no_think:
|
| 65 |
+
kwargs['enable_thinking'] = False
|
| 66 |
+
preserve = True
|
| 67 |
+
if rnd.random() < 0.2:
|
| 68 |
+
kwargs['preserve_thinking'] = False
|
| 69 |
+
preserve = False
|
| 70 |
+
if rnd.random() < 0.2:
|
| 71 |
+
kwargs['add_vision_id'] = True
|
| 72 |
+
trunc_args = False
|
| 73 |
+
if fmt == 'xml' and rnd.random() < 0.2:
|
| 74 |
+
kwargs['max_tool_arg_chars'] = rnd.choice([8, 40])
|
| 75 |
+
trunc_args = True
|
| 76 |
+
if fmt == 'xml' and rnd.random() < 0.15:
|
| 77 |
+
kwargs['max_tool_response_chars'] = 80
|
| 78 |
+
|
| 79 |
+
msgs = []
|
| 80 |
+
user_texts, answers, reasonings, xml_keys, xml_vals = [], [], [], [], []
|
| 81 |
+
planted_error = False
|
| 82 |
+
|
| 83 |
+
for _ in range(rnd.randint(0, 2)):
|
| 84 |
+
msgs.append({'role': 'system', 'content': _phrase(rnd)})
|
| 85 |
+
|
| 86 |
+
for _ in range(rnd.randint(1, 3)):
|
| 87 |
+
text = _phrase(rnd)
|
| 88 |
+
if rnd.random() < 0.25:
|
| 89 |
+
parts = [{'type': 'image', 'image': 'x'}
|
| 90 |
+
for _ in range(rnd.randint(1, 2))]
|
| 91 |
+
parts.append({'type': 'text', 'text': text})
|
| 92 |
+
msgs.append({'role': 'user', 'content': parts})
|
| 93 |
+
else:
|
| 94 |
+
msgs.append({'role': 'user', 'content': text})
|
| 95 |
+
user_texts.append(text)
|
| 96 |
+
|
| 97 |
+
for _ in range(rnd.randint(0, 2)):
|
| 98 |
+
calls = []
|
| 99 |
+
for c in range(rnd.randint(1, 2)):
|
| 100 |
+
roll = rnd.random()
|
| 101 |
+
if roll < 0.5:
|
| 102 |
+
value = _phrase(rnd)
|
| 103 |
+
args = {'k%d' % c: value}
|
| 104 |
+
xml_keys.append('k%d' % c)
|
| 105 |
+
if fmt == 'xml' and not trunc_args:
|
| 106 |
+
xml_vals.append(value)
|
| 107 |
+
elif roll < 0.7:
|
| 108 |
+
args = json.dumps({'q': _phrase(rnd)})
|
| 109 |
+
elif roll < 0.8:
|
| 110 |
+
args = rnd.randint(0, 999)
|
| 111 |
+
elif roll < 0.9:
|
| 112 |
+
args = [1, 2, 3]
|
| 113 |
+
else:
|
| 114 |
+
args = {}
|
| 115 |
+
fn = {'name': 'fn%d' % c, 'arguments': args}
|
| 116 |
+
calls.append({'type': 'function', 'function': fn}
|
| 117 |
+
if rnd.random() < 0.5 else dict(fn))
|
| 118 |
+
amsg = {'role': 'assistant',
|
| 119 |
+
'content': _phrase(rnd) if rnd.random() < 0.4 else '',
|
| 120 |
+
'tool_calls': calls}
|
| 121 |
+
if rnd.random() < 0.5:
|
| 122 |
+
rz = _phrase(rnd)
|
| 123 |
+
if rnd.random() < 0.5:
|
| 124 |
+
amsg['content'] = '<think>\n' + rz + '\n</think>\n\n' + amsg['content']
|
| 125 |
+
else:
|
| 126 |
+
amsg['reasoning_content'] = rz
|
| 127 |
+
if preserve:
|
| 128 |
+
reasonings.append(rz)
|
| 129 |
+
msgs.append(amsg)
|
| 130 |
+
for _ in calls:
|
| 131 |
+
if rnd.random() < 0.12:
|
| 132 |
+
msgs.append({'role': 'tool', 'content': '{"error": "boom"}'})
|
| 133 |
+
planted_error = True
|
| 134 |
+
else:
|
| 135 |
+
msgs.append({'role': 'tool', 'content': 'result ' + _phrase(rnd)})
|
| 136 |
+
|
| 137 |
+
ans = _phrase(rnd)
|
| 138 |
+
amsg = {'role': 'assistant', 'content': ans}
|
| 139 |
+
if rnd.random() < 0.6:
|
| 140 |
+
rz = _phrase(rnd)
|
| 141 |
+
if rnd.random() < 0.5:
|
| 142 |
+
amsg['content'] = '<think>\n' + rz + '\n</think>\n\n' + ans
|
| 143 |
+
else:
|
| 144 |
+
amsg['reasoning_content'] = rz
|
| 145 |
+
if preserve:
|
| 146 |
+
reasonings.append(rz)
|
| 147 |
+
msgs.append(amsg)
|
| 148 |
+
answers.append(ans)
|
| 149 |
+
|
| 150 |
+
return dict(msgs=msgs, kwargs=kwargs, fmt=fmt, preserve=preserve,
|
| 151 |
+
no_think=no_think, user_texts=user_texts, answers=answers,
|
| 152 |
+
reasonings=reasonings, xml_keys=xml_keys, xml_vals=xml_vals,
|
| 153 |
+
planted_error=planted_error)
|
| 154 |
+
|
| 155 |
+
|
| 156 |
+
def _check(case, tpl, other, failures, idx):
|
| 157 |
+
msgs, kw = case['msgs'], case['kwargs']
|
| 158 |
+
|
| 159 |
+
def fail(inv, detail):
|
| 160 |
+
failures.append({
|
| 161 |
+
'case': idx, 'invariant': inv, 'detail': detail,
|
| 162 |
+
'repro': json.dumps({'messages': msgs, 'kwargs': kw},
|
| 163 |
+
ensure_ascii=False, default=str)})
|
| 164 |
+
|
| 165 |
+
try:
|
| 166 |
+
out = tpl.render(messages=msgs, add_generation_prompt=False, **kw)
|
| 167 |
+
except Exception:
|
| 168 |
+
fail('render', traceback.format_exc().strip().splitlines()[-1])
|
| 169 |
+
return
|
| 170 |
+
try:
|
| 171 |
+
out_b = other.render(messages=msgs, add_generation_prompt=False, **kw)
|
| 172 |
+
if out != out_b:
|
| 173 |
+
i = next((j for j in range(min(len(out), len(out_b)))
|
| 174 |
+
if out[j] != out_b[j]), min(len(out), len(out_b)))
|
| 175 |
+
fail('parity', 'first diff at char %d: %r vs %r'
|
| 176 |
+
% (i, out[i:i + 40], out_b[i:i + 40]))
|
| 177 |
+
except Exception:
|
| 178 |
+
fail('parity', traceback.format_exc().strip().splitlines()[-1])
|
| 179 |
+
|
| 180 |
+
if out.count('<|im_start|>') != out.count('<|im_end|>'):
|
| 181 |
+
fail('balance', '%d starts vs %d ends'
|
| 182 |
+
% (out.count('<|im_start|>'), out.count('<|im_end|>')))
|
| 183 |
+
|
| 184 |
+
for text in case['user_texts'] + case['answers'] + case['reasonings']:
|
| 185 |
+
if text not in out:
|
| 186 |
+
fail('content', 'missing %r' % text[:60])
|
| 187 |
+
break
|
| 188 |
+
|
| 189 |
+
if case['fmt'] == 'xml':
|
| 190 |
+
for key in case['xml_keys']:
|
| 191 |
+
if ('<parameter=%s>' % key) not in out:
|
| 192 |
+
fail('xml-fidelity', 'missing key %s' % key)
|
| 193 |
+
break
|
| 194 |
+
for value in case['xml_vals']:
|
| 195 |
+
if value not in out:
|
| 196 |
+
fail('xml-fidelity', 'missing value %r' % value[:60])
|
| 197 |
+
break
|
| 198 |
+
|
| 199 |
+
if case['fmt'] == 'json':
|
| 200 |
+
for block in out.split('<tool_call>\n')[1:]:
|
| 201 |
+
body = block.split('\n</tool_call>')[0]
|
| 202 |
+
try:
|
| 203 |
+
json.loads(body)
|
| 204 |
+
except Exception:
|
| 205 |
+
fail('json-validity', body[:80])
|
| 206 |
+
break
|
| 207 |
+
|
| 208 |
+
warned = '⚠️ SYSTEM WARNING' in out
|
| 209 |
+
if warned != case['planted_error']:
|
| 210 |
+
fail('warning', 'warned=%s planted=%s' % (warned, case['planted_error']))
|
| 211 |
+
|
| 212 |
+
if case['preserve']:
|
| 213 |
+
prev = None
|
| 214 |
+
for k in range(1, len(msgs) + 1):
|
| 215 |
+
if k < len(msgs) and msgs[k]['role'] == msgs[k - 1]['role'] \
|
| 216 |
+
and msgs[k]['role'] in ('system', 'tool'):
|
| 217 |
+
continue
|
| 218 |
+
cur = tpl.render(messages=msgs[:k], add_generation_prompt=False, **kw)
|
| 219 |
+
|
| 220 |
+
if k > 1 and msgs[k - 1].get('role') == 'assistant':
|
| 221 |
+
prompt = tpl.render(messages=msgs[:k - 1], add_generation_prompt=True, **kw)
|
| 222 |
+
if not cur.startswith(prompt):
|
| 223 |
+
if prompt.endswith('<think>\n\n</think>\n\n') and cur.startswith(prompt[:-11]):
|
| 224 |
+
# Fuzzer randomly injected reasoning into a non-thinking turn. KV cache naturally breaks here.
|
| 225 |
+
pass
|
| 226 |
+
else:
|
| 227 |
+
fail('prefix', 'generation prompt at turn %d not prefix of history at turn %d' % (k - 1, k))
|
| 228 |
+
break
|
| 229 |
+
|
| 230 |
+
if prev is not None and not cur.startswith(prev):
|
| 231 |
+
fail('prefix', 'history mutated at turn %d' % k)
|
| 232 |
+
break
|
| 233 |
+
prev = cur
|
| 234 |
+
|
| 235 |
+
if case['no_think']:
|
| 236 |
+
gen = tpl.render(messages=msgs, add_generation_prompt=True, **kw)
|
| 237 |
+
if not gen.endswith('<think>\n\n</think>\n\n'):
|
| 238 |
+
fail('prefill', repr(gen[-40:]))
|
| 239 |
+
|
| 240 |
+
|
| 241 |
+
def run_fuzz(cases=500, seed=0, template_dir=None,
|
| 242 |
+
template_file='chat_template.jinja', max_failures=10):
|
| 243 |
+
template_dir = template_dir or os.path.dirname(
|
| 244 |
+
os.path.dirname(os.path.abspath(__file__)))
|
| 245 |
+
env = Environment(loader=FileSystemLoader(template_dir),
|
| 246 |
+
undefined=StrictUndefined, keep_trailing_newline=True,
|
| 247 |
+
lstrip_blocks=True, trim_blocks=True)
|
| 248 |
+
env.globals['raise_exception'] = \
|
| 249 |
+
lambda m: (_ for _ in ()).throw(Exception(m))
|
| 250 |
+
tpl = env.get_template(template_file)
|
| 251 |
+
other_name = ('chat_template.jinja'
|
| 252 |
+
if template_file == 'chat_template_oneline.txt'
|
| 253 |
+
else 'chat_template_oneline.txt')
|
| 254 |
+
other = env.get_template(other_name)
|
| 255 |
+
rnd = random.Random(seed)
|
| 256 |
+
failures = []
|
| 257 |
+
for i in range(cases):
|
| 258 |
+
_check(_gen_case(rnd), tpl, other, failures, i)
|
| 259 |
+
if len(failures) >= max_failures:
|
| 260 |
+
break
|
| 261 |
+
return failures
|
| 262 |
+
|
| 263 |
+
|
| 264 |
+
def main():
|
| 265 |
+
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
| 266 |
+
parser.add_argument('--cases', type=int, default=500)
|
| 267 |
+
parser.add_argument('--seed', type=int, default=0)
|
| 268 |
+
parser.add_argument('--template', default='chat_template.jinja')
|
| 269 |
+
args = parser.parse_args()
|
| 270 |
+
failures = run_fuzz(cases=args.cases, seed=args.seed,
|
| 271 |
+
template_file=args.template)
|
| 272 |
+
if failures:
|
| 273 |
+
for f in failures:
|
| 274 |
+
print('FAIL case %d [%s]: %s' % (f['case'], f['invariant'], f['detail']))
|
| 275 |
+
print(' repro: %s' % f['repro'][:800])
|
| 276 |
+
print('\n%d invariant violation(s) in %d cases (seed %d)'
|
| 277 |
+
% (len(failures), args.cases, args.seed))
|
| 278 |
+
sys.exit(1)
|
| 279 |
+
print('All invariants held over %d generated conversations (seed %d).'
|
| 280 |
+
% (args.cases, args.seed))
|
| 281 |
+
|
| 282 |
+
|
| 283 |
+
if __name__ == '__main__':
|
| 284 |
+
main()
|
scripts/test_v21.py
ADDED
|
@@ -0,0 +1,184 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import os
|
| 2 |
+
import sys
|
| 3 |
+
import json
|
| 4 |
+
import traceback
|
| 5 |
+
|
| 6 |
+
try:
|
| 7 |
+
from jinja2 import Environment, FileSystemLoader, StrictUndefined
|
| 8 |
+
except ImportError:
|
| 9 |
+
print("Error: jinja2 is required to run tests. Please install it using 'pip install jinja2'")
|
| 10 |
+
sys.exit(1)
|
| 11 |
+
|
| 12 |
+
TEMPLATE_FILE = 'chat_template.jinja'
|
| 13 |
+
TEMPLATE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
| 14 |
+
|
| 15 |
+
env = Environment(
|
| 16 |
+
loader=FileSystemLoader(TEMPLATE_DIR),
|
| 17 |
+
undefined=StrictUndefined,
|
| 18 |
+
keep_trailing_newline=True,
|
| 19 |
+
lstrip_blocks=True,
|
| 20 |
+
trim_blocks=True
|
| 21 |
+
)
|
| 22 |
+
|
| 23 |
+
def raise_exception(msg):
|
| 24 |
+
raise Exception(msg)
|
| 25 |
+
|
| 26 |
+
env.globals['raise_exception'] = raise_exception
|
| 27 |
+
|
| 28 |
+
try:
|
| 29 |
+
template = env.get_template(TEMPLATE_FILE)
|
| 30 |
+
except Exception as e:
|
| 31 |
+
print(f"Error loading template: {e}")
|
| 32 |
+
sys.exit(1)
|
| 33 |
+
|
| 34 |
+
def run_test(name, messages, tools=None, kwargs=None, expected_in=None, expected_not_in=None, expect_error=False):
|
| 35 |
+
if kwargs is None:
|
| 36 |
+
kwargs = {}
|
| 37 |
+
|
| 38 |
+
print(f"\n--- Running Test: {name} ---")
|
| 39 |
+
|
| 40 |
+
try:
|
| 41 |
+
render_kwargs = {'messages': messages, 'add_generation_prompt': True}
|
| 42 |
+
if tools is not None:
|
| 43 |
+
render_kwargs['tools'] = tools
|
| 44 |
+
render_kwargs.update(kwargs)
|
| 45 |
+
|
| 46 |
+
rendered = template.render(**render_kwargs)
|
| 47 |
+
|
| 48 |
+
if expect_error:
|
| 49 |
+
print("❌ FAILED: Expected an exception but got none.")
|
| 50 |
+
return False
|
| 51 |
+
|
| 52 |
+
success = True
|
| 53 |
+
|
| 54 |
+
if expected_in:
|
| 55 |
+
for ex in expected_in:
|
| 56 |
+
if ex not in rendered:
|
| 57 |
+
print(f"❌ FAILED: Missing expected string:\n'''{ex}'''")
|
| 58 |
+
print(f"Rendered:\n{rendered}")
|
| 59 |
+
success = False
|
| 60 |
+
|
| 61 |
+
if expected_not_in:
|
| 62 |
+
for n_ex in expected_not_in:
|
| 63 |
+
if n_ex in rendered:
|
| 64 |
+
print(f"❌ FAILED: Found string that should NOT be present:\n'''{n_ex}'''")
|
| 65 |
+
print(f"Rendered:\n{rendered}")
|
| 66 |
+
success = False
|
| 67 |
+
|
| 68 |
+
if success:
|
| 69 |
+
print("✅ PASSED")
|
| 70 |
+
return True
|
| 71 |
+
return False
|
| 72 |
+
|
| 73 |
+
except Exception as e:
|
| 74 |
+
if expect_error:
|
| 75 |
+
print(f"✅ PASSED (Caught expected error: {e})")
|
| 76 |
+
return True
|
| 77 |
+
print(f"❌ FAILED with exception:\n{traceback.format_exc()}")
|
| 78 |
+
return False
|
| 79 |
+
|
| 80 |
+
# Tests
|
| 81 |
+
tests_passed = 0
|
| 82 |
+
tests_total = 0
|
| 83 |
+
|
| 84 |
+
def execute_test(*args, **kwargs):
|
| 85 |
+
global tests_passed, tests_total
|
| 86 |
+
tests_total += 1
|
| 87 |
+
if run_test(*args, **kwargs):
|
| 88 |
+
tests_passed += 1
|
| 89 |
+
|
| 90 |
+
# 1. auto_disable_thinking_with_tools (Default logic)
|
| 91 |
+
execute_test(
|
| 92 |
+
"auto_disable_thinking_with_tools (enabled via kwarg)",
|
| 93 |
+
messages=[{"role": "user", "content": "Hello!"}],
|
| 94 |
+
tools=[{"name": "test_tool"}],
|
| 95 |
+
kwargs={"auto_disable_thinking_with_tools": True},
|
| 96 |
+
expected_in=["<think>\n\n</think>\n\n"], # Should be stripped
|
| 97 |
+
)
|
| 98 |
+
|
| 99 |
+
execute_test(
|
| 100 |
+
"auto_disable_thinking_with_tools (disabled via kwarg -> allows thinking)",
|
| 101 |
+
messages=[{"role": "user", "content": "Hello!"}],
|
| 102 |
+
tools=[{"name": "test_tool"}],
|
| 103 |
+
kwargs={"auto_disable_thinking_with_tools": False},
|
| 104 |
+
expected_in=["<think>\n"],
|
| 105 |
+
expected_not_in=["<think>\n</think>\n"]
|
| 106 |
+
)
|
| 107 |
+
|
| 108 |
+
# 2. inline <|think_on|> overrides auto_disable_thinking_with_tools
|
| 109 |
+
execute_test(
|
| 110 |
+
"inline <|think_on|> overrides auto_disable",
|
| 111 |
+
messages=[{"role": "user", "content": "Hello! <|think_on|>"}],
|
| 112 |
+
tools=[{"name": "test_tool"}],
|
| 113 |
+
kwargs={"auto_disable_thinking_with_tools": True},
|
| 114 |
+
expected_in=["<think>\n"],
|
| 115 |
+
expected_not_in=["<think>\n</think>\n", "<|think_on|>"] # Tag must be stripped
|
| 116 |
+
)
|
| 117 |
+
|
| 118 |
+
# 3. Payload truncation
|
| 119 |
+
execute_test(
|
| 120 |
+
"max_tool_arg_chars truncation",
|
| 121 |
+
messages=[{"role": "user", "content": "Call tool"}, {"role": "assistant", "content": "", "tool_calls": [{"function": {"name": "test", "arguments": {"param": "1234567890"}}}]}],
|
| 122 |
+
kwargs={"max_tool_arg_chars": 5},
|
| 123 |
+
expected_in=["<parameter=param>\n12345\n[TRUNCATED"]
|
| 124 |
+
)
|
| 125 |
+
|
| 126 |
+
execute_test(
|
| 127 |
+
"max_tool_response_chars truncation",
|
| 128 |
+
messages=[{"role": "user", "content": "Do it"}, {"role": "assistant", "content": "calling", "tool_calls": [{"name": "test"}]}, {"role": "tool", "content": "1234567890"}],
|
| 129 |
+
kwargs={"max_tool_response_chars": 5},
|
| 130 |
+
expected_in=["<tool_response>\n12345\n[TRUNCATED"]
|
| 131 |
+
)
|
| 132 |
+
|
| 133 |
+
# 4. Mid-conversation System Prompt
|
| 134 |
+
execute_test(
|
| 135 |
+
"mid-conversation system prompt",
|
| 136 |
+
messages=[{"role": "user", "content": "Hello"}, {"role": "system", "content": "Reminder: Be polite"}],
|
| 137 |
+
expected_in=["<|im_start|>system\nReminder: Be polite<|im_end|>"]
|
| 138 |
+
)
|
| 139 |
+
|
| 140 |
+
# 5. Parallel tools delimiter
|
| 141 |
+
execute_test(
|
| 142 |
+
"parallel tools delimiter",
|
| 143 |
+
messages=[{"role": "user", "content": "x"}, {"role": "assistant", "content": "", "tool_calls": [{"name": "t1"}, {"name": "t2"}]}],
|
| 144 |
+
expected_in=["</function>\n</tool_call>\n\n<tool_call>\n<function=t2>"]
|
| 145 |
+
)
|
| 146 |
+
|
| 147 |
+
# 6. Deep Agent Fallback
|
| 148 |
+
execute_test(
|
| 149 |
+
"deep agent fallback (no user message)",
|
| 150 |
+
messages=[{"role": "system", "content": "Sys"}, {"role": "tool", "content": "test"}],
|
| 151 |
+
expected_in=["<|im_start|>system\nSys", "<|im_start|>user\n<tool_response>"]
|
| 152 |
+
)
|
| 153 |
+
|
| 154 |
+
# 7. Error Escalation
|
| 155 |
+
execute_test(
|
| 156 |
+
"error escalation warnings",
|
| 157 |
+
messages=[
|
| 158 |
+
{"role": "user", "content": "Do it"},
|
| 159 |
+
{"role": "tool", "content": "error: something failed"},
|
| 160 |
+
{"role": "assistant", "content": "calling"},
|
| 161 |
+
{"role": "tool", "content": "error: failed again"}
|
| 162 |
+
],
|
| 163 |
+
expected_in=["⚠️ SYSTEM WARNING: 2 consecutive tool errors", "<think>\n"]
|
| 164 |
+
)
|
| 165 |
+
|
| 166 |
+
run_test(
|
| 167 |
+
"tool_call_format='json' (override)",
|
| 168 |
+
messages=[{"role": "user", "content": "Call tool"}, {"role": "assistant", "content": "", "tool_calls": [{"function": {"name": "test", "arguments": {"par": "1234567890"}}}]}],
|
| 169 |
+
tools=[{"type": "function", "function": {"name": "test", "description": "test tool"}}],
|
| 170 |
+
kwargs={'tool_call_format': 'json'},
|
| 171 |
+
expected_in=[
|
| 172 |
+
'Function calls MUST follow the specified format: a single JSON object with "name" and "arguments"',
|
| 173 |
+
'{"name": "test", "arguments": {"par": "1234567890"}}'
|
| 174 |
+
]
|
| 175 |
+
)
|
| 176 |
+
|
| 177 |
+
print(f"\n=============================")
|
| 178 |
+
print(f"Test Summary: {tests_passed} / {tests_total} passed.")
|
| 179 |
+
if tests_passed == tests_total:
|
| 180 |
+
print("All tests passed successfully! 🎉")
|
| 181 |
+
sys.exit(0)
|
| 182 |
+
else:
|
| 183 |
+
print("Some tests failed.")
|
| 184 |
+
sys.exit(1)
|
scripts/test_v22.py
ADDED
|
@@ -0,0 +1,1736 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import os
|
| 2 |
+
import pathlib
|
| 3 |
+
import sys
|
| 4 |
+
import re
|
| 5 |
+
import json
|
| 6 |
+
import traceback
|
| 7 |
+
|
| 8 |
+
try:
|
| 9 |
+
from jinja2 import Environment, FileSystemLoader, StrictUndefined
|
| 10 |
+
except ImportError:
|
| 11 |
+
print("Error: jinja2 is required to run tests. Please install it using 'pip install jinja2'")
|
| 12 |
+
sys.exit(1)
|
| 13 |
+
|
| 14 |
+
TEMPLATE_FILE = os.environ.get('QWEN_TEMPLATE_FILE', 'chat_template.jinja')
|
| 15 |
+
TEMPLATE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
| 16 |
+
|
| 17 |
+
env = Environment(
|
| 18 |
+
loader=FileSystemLoader(TEMPLATE_DIR),
|
| 19 |
+
undefined=StrictUndefined,
|
| 20 |
+
keep_trailing_newline=True,
|
| 21 |
+
lstrip_blocks=True,
|
| 22 |
+
trim_blocks=True
|
| 23 |
+
)
|
| 24 |
+
|
| 25 |
+
def raise_exception(msg):
|
| 26 |
+
raise Exception(msg)
|
| 27 |
+
|
| 28 |
+
env.globals['raise_exception'] = raise_exception
|
| 29 |
+
|
| 30 |
+
try:
|
| 31 |
+
template = env.get_template(TEMPLATE_FILE)
|
| 32 |
+
except Exception as e:
|
| 33 |
+
print(f"Error loading template: {e}")
|
| 34 |
+
sys.exit(1)
|
| 35 |
+
|
| 36 |
+
# --- Sharp fork shim -------------------------------------------------------
|
| 37 |
+
# This suite is froggeric's, written against the upstream template. Sharp's only
|
| 38 |
+
# rendering difference is that it force-appends a terseness block to the system
|
| 39 |
+
# turn, so every upstream assertion that pins the exact END of that turn fails on
|
| 40 |
+
# a difference we introduced on purpose. Rather than carry 16 permanently-red
|
| 41 |
+
# tests (which would hide a real regression the next time upstream bumps), strip
|
| 42 |
+
# exactly the block Sharp adds and assert upstream's invariants on the remainder.
|
| 43 |
+
# Nothing else is touched: if Sharp broke a genuine upstream behaviour, it still
|
| 44 |
+
# fails here. With a pristine upstream template the marker is absent and this is
|
| 45 |
+
# a no-op, so the same file doubles as an upstream control.
|
| 46 |
+
_TPL_SRC = pathlib.Path(TEMPLATE_DIR, TEMPLATE_FILE).read_text()
|
| 47 |
+
_SHARP_BLOCKS = []
|
| 48 |
+
if 'Never: open with preamble' in _TPL_SRC:
|
| 49 |
+
_leads = re.findall(r"_terse_lead = '([^']*)'", _TPL_SRC)
|
| 50 |
+
_core = re.search(r"\{%- set _terse_core %\}\n(.*?)\n\{%- endset %\}",
|
| 51 |
+
_TPL_SRC, re.DOTALL).group(1).strip()
|
| 52 |
+
_SHARP_BLOCKS = [lead + '\n' + _core for lead in _leads]
|
| 53 |
+
assert _SHARP_BLOCKS, 'Sharp marker present but terseness block not parseable'
|
| 54 |
+
|
| 55 |
+
def strip_sharp(rendered):
|
| 56 |
+
"""Remove Sharp's appended terseness block, leaving upstream's rendering."""
|
| 57 |
+
for block in _SHARP_BLOCKS:
|
| 58 |
+
# sole occupant of a system turn (no user system prompt, no tools)
|
| 59 |
+
rendered = rendered.replace('<|im_start|>system\n' + block + '<|im_end|>\n', '')
|
| 60 |
+
# appended after existing system content
|
| 61 |
+
rendered = rendered.replace('\n\n' + block, '')
|
| 62 |
+
return rendered
|
| 63 |
+
# --- end Sharp fork shim ---------------------------------------------------
|
| 64 |
+
|
| 65 |
+
def run_test(name, messages, tools=None, kwargs=None, expected_in=None, expected_not_in=None, expect_error=False):
|
| 66 |
+
if kwargs is None:
|
| 67 |
+
kwargs = {}
|
| 68 |
+
|
| 69 |
+
print(f"\n--- Running Test: {name} ---")
|
| 70 |
+
|
| 71 |
+
try:
|
| 72 |
+
render_kwargs = {'messages': messages, 'add_generation_prompt': True}
|
| 73 |
+
if tools is not None:
|
| 74 |
+
render_kwargs['tools'] = tools
|
| 75 |
+
render_kwargs.update(kwargs)
|
| 76 |
+
|
| 77 |
+
rendered = template.render(**render_kwargs)
|
| 78 |
+
rendered = strip_sharp(rendered)
|
| 79 |
+
|
| 80 |
+
if expect_error:
|
| 81 |
+
print("❌ FAILED: Expected an exception but got none.")
|
| 82 |
+
return False
|
| 83 |
+
|
| 84 |
+
success = True
|
| 85 |
+
|
| 86 |
+
if expected_in:
|
| 87 |
+
for ex in expected_in:
|
| 88 |
+
if ex not in rendered:
|
| 89 |
+
print(f"❌ FAILED: Missing expected string:\n'''{ex}'''")
|
| 90 |
+
print(f"Rendered:\n{rendered}")
|
| 91 |
+
success = False
|
| 92 |
+
|
| 93 |
+
if expected_not_in:
|
| 94 |
+
for n_ex in expected_not_in:
|
| 95 |
+
if n_ex in rendered:
|
| 96 |
+
print(f"❌ FAILED: Found string that should NOT be present:\n'''{n_ex}'''")
|
| 97 |
+
print(f"Rendered:\n{rendered}")
|
| 98 |
+
success = False
|
| 99 |
+
|
| 100 |
+
if success:
|
| 101 |
+
print("✅ PASSED")
|
| 102 |
+
return True
|
| 103 |
+
return False
|
| 104 |
+
|
| 105 |
+
except Exception as e:
|
| 106 |
+
if expect_error:
|
| 107 |
+
print(f"✅ PASSED (Caught expected error: {e})")
|
| 108 |
+
return True
|
| 109 |
+
print(f"❌ FAILED with exception:\n{traceback.format_exc()}")
|
| 110 |
+
return False
|
| 111 |
+
|
| 112 |
+
def run_prefix_test(name, messages, kwargs=None):
|
| 113 |
+
"""Asserts render(messages[:k]) is always a strict prefix of render(messages[:k+1]).
|
| 114 |
+
|
| 115 |
+
This is the direct verification of the 100% Prefix KV Cache claim: if any past turn
|
| 116 |
+
is mutated when a new turn arrives, the cached prefix is invalidated from that point.
|
| 117 |
+
"""
|
| 118 |
+
if kwargs is None:
|
| 119 |
+
kwargs = {}
|
| 120 |
+
|
| 121 |
+
print(f"\n--- Running Test: {name} ---")
|
| 122 |
+
|
| 123 |
+
previous = None
|
| 124 |
+
for k in range(1, len(messages) + 1):
|
| 125 |
+
# Checkpoint only at generation boundaries: prefixes splitting a merged
|
| 126 |
+
# system block or a consecutive tool-result batch are never rendered in
|
| 127 |
+
# real serving, so they are not required to be stable.
|
| 128 |
+
if k < len(messages) and messages[k].get('role') == messages[k - 1].get('role') \
|
| 129 |
+
and messages[k].get('role') in ('system', 'tool'):
|
| 130 |
+
continue
|
| 131 |
+
try:
|
| 132 |
+
current = template.render(messages=messages[:k], add_generation_prompt=False, **kwargs)
|
| 133 |
+
except Exception:
|
| 134 |
+
print(f"❌ FAILED with exception:\n{traceback.format_exc()}")
|
| 135 |
+
return False
|
| 136 |
+
|
| 137 |
+
if previous is not None and not current.startswith(previous):
|
| 138 |
+
idx = min(len(previous), len(current))
|
| 139 |
+
for i in range(min(len(previous), len(current))):
|
| 140 |
+
if previous[i] != current[i]:
|
| 141 |
+
idx = i
|
| 142 |
+
break
|
| 143 |
+
print(f"❌ FAILED: turn {k} mutated rendered history at char {idx}.")
|
| 144 |
+
print(f"Before: {previous[max(0, idx - 80):idx + 80]!r}")
|
| 145 |
+
print(f"After: {current[max(0, idx - 80):idx + 80]!r}")
|
| 146 |
+
return False
|
| 147 |
+
previous = current
|
| 148 |
+
|
| 149 |
+
print("✅ PASSED")
|
| 150 |
+
return True
|
| 151 |
+
|
| 152 |
+
|
| 153 |
+
def run_oneline_parity_test(name, cases):
|
| 154 |
+
"""Asserts chat_template_oneline.txt renders byte-identically to chat_template.jinja."""
|
| 155 |
+
print(f"\n--- Running Test: {name} ---")
|
| 156 |
+
|
| 157 |
+
oneline_path = os.path.join(TEMPLATE_DIR, 'chat_template_oneline.txt')
|
| 158 |
+
if not os.path.exists(oneline_path):
|
| 159 |
+
print(f"❌ FAILED: {oneline_path} not found.")
|
| 160 |
+
return False
|
| 161 |
+
|
| 162 |
+
with open(oneline_path, 'r', encoding='utf-8') as f:
|
| 163 |
+
oneline_source = f.read()
|
| 164 |
+
with open(os.path.join(TEMPLATE_DIR, TEMPLATE_FILE), 'r', encoding='utf-8') as f:
|
| 165 |
+
jinja_source = f.read()
|
| 166 |
+
|
| 167 |
+
version_pattern = r'template_version\s*=\s*["\']([^"\']+)["\']'
|
| 168 |
+
jinja_version = re.search(version_pattern, jinja_source)
|
| 169 |
+
oneline_version = re.search(version_pattern, oneline_source)
|
| 170 |
+
if not jinja_version or not oneline_version or jinja_version.group(1) != oneline_version.group(1):
|
| 171 |
+
print("❌ FAILED: template_version mismatch between jinja and oneline builds.")
|
| 172 |
+
print("Regenerate with: python3 scripts/minify_jinja.py chat_template.jinja chat_template_oneline.txt")
|
| 173 |
+
return False
|
| 174 |
+
|
| 175 |
+
oneline_template = env.from_string(oneline_source)
|
| 176 |
+
|
| 177 |
+
for label, messages, kwargs in cases:
|
| 178 |
+
try:
|
| 179 |
+
a = template.render(messages=messages, add_generation_prompt=True, **kwargs)
|
| 180 |
+
b = oneline_template.render(messages=messages, add_generation_prompt=True, **kwargs)
|
| 181 |
+
except Exception:
|
| 182 |
+
print(f"❌ FAILED with exception on case '{label}':\n{traceback.format_exc()}")
|
| 183 |
+
return False
|
| 184 |
+
if a != b:
|
| 185 |
+
print(f"❌ FAILED: oneline output diverges from jinja on case '{label}'.")
|
| 186 |
+
print("Regenerate with: python3 scripts/minify_jinja.py chat_template.jinja chat_template_oneline.txt")
|
| 187 |
+
return False
|
| 188 |
+
|
| 189 |
+
print("✅ PASSED")
|
| 190 |
+
return True
|
| 191 |
+
|
| 192 |
+
|
| 193 |
+
tests_passed = 0
|
| 194 |
+
tests_total = 0
|
| 195 |
+
|
| 196 |
+
def execute_test(*args, **kwargs):
|
| 197 |
+
global tests_passed, tests_total
|
| 198 |
+
tests_total += 1
|
| 199 |
+
if run_test(*args, **kwargs):
|
| 200 |
+
tests_passed += 1
|
| 201 |
+
|
| 202 |
+
def execute_prefix_test(*args, **kwargs):
|
| 203 |
+
global tests_passed, tests_total
|
| 204 |
+
tests_total += 1
|
| 205 |
+
if run_prefix_test(*args, **kwargs):
|
| 206 |
+
tests_passed += 1
|
| 207 |
+
|
| 208 |
+
def execute_parity_test(*args, **kwargs):
|
| 209 |
+
global tests_passed, tests_total
|
| 210 |
+
tests_total += 1
|
| 211 |
+
if run_oneline_parity_test(*args, **kwargs):
|
| 212 |
+
tests_passed += 1
|
| 213 |
+
|
| 214 |
+
# ==========================================
|
| 215 |
+
# 1. Qwen 3.8 Reasoning Effort Controls (v22.1 Default: medium)
|
| 216 |
+
# ==========================================
|
| 217 |
+
|
| 218 |
+
# 1. Default reasoning_effort="medium" (no system message -> zero system message emitted)
|
| 219 |
+
execute_test(
|
| 220 |
+
"1. reasoning_effort='medium' (v22.1 default, no system message)",
|
| 221 |
+
messages=[{"role": "user", "content": "Hello!"}],
|
| 222 |
+
expected_in=[
|
| 223 |
+
"<|im_start|>user\nHello!<|im_end|>\n<|im_start|>assistant\n<think>\n"
|
| 224 |
+
],
|
| 225 |
+
expected_not_in=[
|
| 226 |
+
"<|im_start|>system\n"
|
| 227 |
+
]
|
| 228 |
+
)
|
| 229 |
+
|
| 230 |
+
# 2. Explicit reasoning_effort="xhigh"
|
| 231 |
+
execute_test(
|
| 232 |
+
"2. reasoning_effort='xhigh'",
|
| 233 |
+
messages=[{"role": "user", "content": "Hello!"}],
|
| 234 |
+
kwargs={"reasoning_effort": "xhigh"},
|
| 235 |
+
expected_in=[
|
| 236 |
+
"<|im_start|>system\nReasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.<|im_end|>\n",
|
| 237 |
+
"<|im_start|>user\nHello!<|im_end|>\n",
|
| 238 |
+
"<|im_start|>assistant\n<think>\n"
|
| 239 |
+
]
|
| 240 |
+
)
|
| 241 |
+
|
| 242 |
+
# 3. Explicit reasoning_effort="high" (OpenAI alias -> xhigh)
|
| 243 |
+
execute_test(
|
| 244 |
+
"3. reasoning_effort='high' (OpenAI alias)",
|
| 245 |
+
messages=[{"role": "user", "content": "Hello!"}],
|
| 246 |
+
kwargs={"reasoning_effort": "high"},
|
| 247 |
+
expected_in=[
|
| 248 |
+
"<|im_start|>system\nReasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.<|im_end|>\n"
|
| 249 |
+
]
|
| 250 |
+
)
|
| 251 |
+
|
| 252 |
+
# 4. Explicit reasoning_effort="max" (API max alias -> xhigh)
|
| 253 |
+
execute_test(
|
| 254 |
+
"4. reasoning_effort='max' (API alias)",
|
| 255 |
+
messages=[{"role": "user", "content": "Hello!"}],
|
| 256 |
+
kwargs={"reasoning_effort": "max"},
|
| 257 |
+
expected_in=[
|
| 258 |
+
"<|im_start|>system\nReasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.<|im_end|>\n"
|
| 259 |
+
]
|
| 260 |
+
)
|
| 261 |
+
|
| 262 |
+
# 5. Explicit reasoning_effort="low"
|
| 263 |
+
execute_test(
|
| 264 |
+
"5. reasoning_effort='low'",
|
| 265 |
+
messages=[{"role": "user", "content": "Hello!"}],
|
| 266 |
+
kwargs={"reasoning_effort": "low"},
|
| 267 |
+
expected_in=[
|
| 268 |
+
"<|im_start|>system\nReasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration.<|im_end|>\n",
|
| 269 |
+
"<|im_start|>user\nHello!<|im_end|>\n"
|
| 270 |
+
]
|
| 271 |
+
)
|
| 272 |
+
|
| 273 |
+
# 6. Explicit reasoning_effort="minimal" (API minimal alias -> low)
|
| 274 |
+
execute_test(
|
| 275 |
+
"6. reasoning_effort='minimal' (API alias)",
|
| 276 |
+
messages=[{"role": "user", "content": "Hello!"}],
|
| 277 |
+
kwargs={"reasoning_effort": "minimal"},
|
| 278 |
+
expected_in=[
|
| 279 |
+
"<|im_start|>system\nReasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration.<|im_end|>\n"
|
| 280 |
+
]
|
| 281 |
+
)
|
| 282 |
+
|
| 283 |
+
# 7. Explicit reasoning_effort="none" (disables thinking)
|
| 284 |
+
execute_test(
|
| 285 |
+
"7. reasoning_effort='none' (disables thinking)",
|
| 286 |
+
messages=[{"role": "user", "content": "Hello!"}],
|
| 287 |
+
kwargs={"reasoning_effort": "none"},
|
| 288 |
+
expected_in=[
|
| 289 |
+
"<|im_start|>user\nHello!<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n"
|
| 290 |
+
],
|
| 291 |
+
expected_not_in=[
|
| 292 |
+
"Reasoning effort is set to"
|
| 293 |
+
]
|
| 294 |
+
)
|
| 295 |
+
|
| 296 |
+
# 8. Explicit reasoning_effort="unknown_val" (safe fallback to medium)
|
| 297 |
+
execute_test(
|
| 298 |
+
"8. reasoning_effort='unknown_val' (safe fallback to medium)",
|
| 299 |
+
messages=[{"role": "user", "content": "Hello!"}],
|
| 300 |
+
kwargs={"reasoning_effort": "unrecognized_str"},
|
| 301 |
+
expected_in=[
|
| 302 |
+
"<|im_start|>user\nHello!<|im_end|>\n<|im_start|>assistant\n<think>\n"
|
| 303 |
+
],
|
| 304 |
+
expected_not_in=[
|
| 305 |
+
"Reasoning effort is set to"
|
| 306 |
+
]
|
| 307 |
+
)
|
| 308 |
+
|
| 309 |
+
# 9. reasoning_effort='xhigh' with user system prompt
|
| 310 |
+
execute_test(
|
| 311 |
+
"9. reasoning_effort='xhigh' with user system prompt",
|
| 312 |
+
messages=[
|
| 313 |
+
{"role": "system", "content": "You are an expert coder."},
|
| 314 |
+
{"role": "user", "content": "Write quicksort in C++"}
|
| 315 |
+
],
|
| 316 |
+
kwargs={"reasoning_effort": "xhigh"},
|
| 317 |
+
expected_in=[
|
| 318 |
+
"<|im_start|>system\nReasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.\n\nYou are an expert coder.<|im_end|>\n",
|
| 319 |
+
"<|im_start|>user\nWrite quicksort in C++<|im_end|>\n"
|
| 320 |
+
]
|
| 321 |
+
)
|
| 322 |
+
|
| 323 |
+
# 10. reasoning_effort='xhigh' with tools
|
| 324 |
+
tools_sample = [
|
| 325 |
+
{
|
| 326 |
+
"type": "function",
|
| 327 |
+
"function": {
|
| 328 |
+
"name": "get_weather",
|
| 329 |
+
"description": "Get weather for city",
|
| 330 |
+
"parameters": {
|
| 331 |
+
"type": "object",
|
| 332 |
+
"properties": {"city": {"type": "string"}},
|
| 333 |
+
"required": ["city"]
|
| 334 |
+
}
|
| 335 |
+
}
|
| 336 |
+
}
|
| 337 |
+
]
|
| 338 |
+
|
| 339 |
+
execute_test(
|
| 340 |
+
"10. reasoning_effort='xhigh' with tools",
|
| 341 |
+
messages=[{"role": "user", "content": "What's the weather in Tokyo?"}],
|
| 342 |
+
tools=tools_sample,
|
| 343 |
+
kwargs={"reasoning_effort": "xhigh"},
|
| 344 |
+
expected_in=[
|
| 345 |
+
"<|im_start|>system\nReasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.\n\n# Tools\n\nYou have access to the following functions:\n\n<tools>\n"
|
| 346 |
+
]
|
| 347 |
+
)
|
| 348 |
+
|
| 349 |
+
# ==========================================
|
| 350 |
+
# 2. Inline Chat Tags for Reasoning Effort Steering (v22.1)
|
| 351 |
+
# ==========================================
|
| 352 |
+
|
| 353 |
+
# 11. Inline <|think_low|> in user message
|
| 354 |
+
execute_test(
|
| 355 |
+
"11. Inline <|think_low|> in user string",
|
| 356 |
+
messages=[{"role": "user", "content": "What is 2+2? <|think_low|>"}],
|
| 357 |
+
expected_in=[
|
| 358 |
+
"<|im_start|>system\nReasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration.<|im_end|>\n",
|
| 359 |
+
"<|im_start|>user\nWhat is 2+2?<|im_end|>\n",
|
| 360 |
+
"<|im_start|>assistant\n<think>\n"
|
| 361 |
+
],
|
| 362 |
+
expected_not_in=[
|
| 363 |
+
"<|think_low|>"
|
| 364 |
+
]
|
| 365 |
+
)
|
| 366 |
+
|
| 367 |
+
# 12. Inline <|think_xhigh|> in user message
|
| 368 |
+
execute_test(
|
| 369 |
+
"12. Inline <|think_xhigh|> in user string",
|
| 370 |
+
messages=[{"role": "user", "content": "Prove Fermat's Last Theorem <|think_xhigh|>"}],
|
| 371 |
+
expected_in=[
|
| 372 |
+
"<|im_start|>system\nReasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.<|im_end|>\n",
|
| 373 |
+
"<|im_start|>user\nProve Fermat's Last Theorem<|im_end|>\n",
|
| 374 |
+
"<|im_start|>assistant\n<think>\n"
|
| 375 |
+
],
|
| 376 |
+
expected_not_in=[
|
| 377 |
+
"<|think_xhigh|>"
|
| 378 |
+
]
|
| 379 |
+
)
|
| 380 |
+
|
| 381 |
+
# 13. Inline <|think_medium|> in user message
|
| 382 |
+
execute_test(
|
| 383 |
+
"13. Inline <|think_medium|> in user string",
|
| 384 |
+
messages=[{"role": "user", "content": "Hello <|think_medium|>"}],
|
| 385 |
+
expected_in=[
|
| 386 |
+
"<|im_start|>user\nHello<|im_end|>\n",
|
| 387 |
+
"<|im_start|>assistant\n<think>\n"
|
| 388 |
+
],
|
| 389 |
+
expected_not_in=[
|
| 390 |
+
"<|think_medium|>",
|
| 391 |
+
"<|im_start|>system\n"
|
| 392 |
+
]
|
| 393 |
+
)
|
| 394 |
+
|
| 395 |
+
# 14. Inline <|think_off|> in user message
|
| 396 |
+
execute_test(
|
| 397 |
+
"14. Inline <|think_off|> in user string",
|
| 398 |
+
messages=[{"role": "user", "content": "Quick answer: what is capital of France? <|think_off|>"}],
|
| 399 |
+
expected_in=[
|
| 400 |
+
"<|im_start|>user\nQuick answer: what is capital of France?<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n"
|
| 401 |
+
],
|
| 402 |
+
expected_not_in=[
|
| 403 |
+
"<|think_off|>",
|
| 404 |
+
"Reasoning effort is set to"
|
| 405 |
+
]
|
| 406 |
+
)
|
| 407 |
+
|
| 408 |
+
# 15. Inline <|think_low|> in multi-part list[dict]
|
| 409 |
+
execute_test(
|
| 410 |
+
"15. Inline <|think_low|> in multi-part list[dict]",
|
| 411 |
+
messages=[
|
| 412 |
+
{"role": "user", "content": [{"type": "text", "text": "Solve this riddle <|think_low|>"}]}
|
| 413 |
+
],
|
| 414 |
+
expected_in=[
|
| 415 |
+
"<|im_start|>system\nReasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration.<|im_end|>\n",
|
| 416 |
+
"<|im_start|>user\nSolve this riddle<|im_end|>\n"
|
| 417 |
+
],
|
| 418 |
+
expected_not_in=[
|
| 419 |
+
"<|think_low|>"
|
| 420 |
+
]
|
| 421 |
+
)
|
| 422 |
+
|
| 423 |
+
# 16. Inline <|think_xhigh|> in multi-part list[str]
|
| 424 |
+
execute_test(
|
| 425 |
+
"16. Inline <|think_xhigh|> in multi-part list[str]",
|
| 426 |
+
messages=[
|
| 427 |
+
{"role": "user", "content": ["Solve this deeply", "<|think_xhigh|>"]}
|
| 428 |
+
],
|
| 429 |
+
expected_in=[
|
| 430 |
+
"<|im_start|>system\nReasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.<|im_end|>\n",
|
| 431 |
+
"<|im_start|>user\nSolve this deeply<|im_end|>\n"
|
| 432 |
+
],
|
| 433 |
+
expected_not_in=[
|
| 434 |
+
"<|think_xhigh|>"
|
| 435 |
+
]
|
| 436 |
+
)
|
| 437 |
+
|
| 438 |
+
# 17. Clean tag stripping across multiple tags in same string
|
| 439 |
+
execute_test(
|
| 440 |
+
"17. Clean tag stripping across multiple tags in same string",
|
| 441 |
+
messages=[
|
| 442 |
+
{"role": "user", "content": "Hello <|think_on|> <|think_minimal|> world"}
|
| 443 |
+
],
|
| 444 |
+
expected_in=[
|
| 445 |
+
"<|im_start|>user\nHello world<|im_end|>\n"
|
| 446 |
+
],
|
| 447 |
+
expected_not_in=[
|
| 448 |
+
"<|think_on|>",
|
| 449 |
+
"<|think_minimal|>"
|
| 450 |
+
]
|
| 451 |
+
)
|
| 452 |
+
|
| 453 |
+
# ==========================================
|
| 454 |
+
# 3. Thinking Toggles & Preserves
|
| 455 |
+
# ==========================================
|
| 456 |
+
|
| 457 |
+
# 18. enable_thinking=false kwarg
|
| 458 |
+
execute_test(
|
| 459 |
+
"18. enable_thinking=false kwarg",
|
| 460 |
+
messages=[{"role": "user", "content": "Hello!"}],
|
| 461 |
+
kwargs={"enable_thinking": False},
|
| 462 |
+
expected_in=[
|
| 463 |
+
"<|im_start|>user\nHello!<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n"
|
| 464 |
+
]
|
| 465 |
+
)
|
| 466 |
+
|
| 467 |
+
# 19. auto_disable_thinking_with_tools=true
|
| 468 |
+
execute_test(
|
| 469 |
+
"19. auto_disable_thinking_with_tools=true",
|
| 470 |
+
messages=[{"role": "user", "content": "What's the weather?"}],
|
| 471 |
+
tools=tools_sample,
|
| 472 |
+
kwargs={"auto_disable_thinking_with_tools": True},
|
| 473 |
+
expected_in=[
|
| 474 |
+
"<|im_start|>assistant\n<think>\n\n</think>\n\n"
|
| 475 |
+
]
|
| 476 |
+
)
|
| 477 |
+
|
| 478 |
+
# 20. preserve_reasoning=True preserves thinking
|
| 479 |
+
execute_test(
|
| 480 |
+
"20. preserve_reasoning=True preserves thinking",
|
| 481 |
+
messages=[
|
| 482 |
+
{"role": "user", "content": "Question 1"},
|
| 483 |
+
{"role": "assistant", "content": "<think>\nThinking 1\n</think>\n\nAnswer 1"},
|
| 484 |
+
{"role": "user", "content": "Question 2"}
|
| 485 |
+
],
|
| 486 |
+
kwargs={"preserve_reasoning": True},
|
| 487 |
+
expected_in=[
|
| 488 |
+
"<|im_start|>assistant\n<think>\nThinking 1\n</think>\n\nAnswer 1<|im_end|>\n"
|
| 489 |
+
]
|
| 490 |
+
)
|
| 491 |
+
|
| 492 |
+
# 21. preserve_reasoning=False strips past thinking
|
| 493 |
+
execute_test(
|
| 494 |
+
"21. preserve_reasoning=False strips past thinking",
|
| 495 |
+
messages=[
|
| 496 |
+
{"role": "user", "content": "Question 1"},
|
| 497 |
+
{"role": "assistant", "content": "<think>\nThinking 1\n</think>\n\nAnswer 1"},
|
| 498 |
+
{"role": "user", "content": "Question 2"}
|
| 499 |
+
],
|
| 500 |
+
kwargs={"preserve_reasoning": False},
|
| 501 |
+
expected_in=[
|
| 502 |
+
"<|im_start|>assistant\nAnswer 1<|im_end|>\n"
|
| 503 |
+
],
|
| 504 |
+
expected_not_in=[
|
| 505 |
+
"Thinking 1"
|
| 506 |
+
]
|
| 507 |
+
)
|
| 508 |
+
|
| 509 |
+
# 22. In-content <think> parsing (Curing official 3.8 empty think poisoning)
|
| 510 |
+
execute_test(
|
| 511 |
+
"22. In-content <think> parsing (Curing official 3.8 empty think poisoning)",
|
| 512 |
+
messages=[
|
| 513 |
+
{"role": "user", "content": "Solve 1+1"},
|
| 514 |
+
{"role": "assistant", "content": "<think>\n1+1 is 2\n</think>\n\nResult is 2"},
|
| 515 |
+
{"role": "user", "content": "Now 2+2"}
|
| 516 |
+
],
|
| 517 |
+
kwargs={"preserve_thinking": True},
|
| 518 |
+
expected_in=[
|
| 519 |
+
"<|im_start|>assistant\n<think>\n1+1 is 2\n</think>\n\nResult is 2<|im_end|>\n"
|
| 520 |
+
],
|
| 521 |
+
expected_not_in=[
|
| 522 |
+
"<think>\n\n</think>\n\n<think>"
|
| 523 |
+
]
|
| 524 |
+
)
|
| 525 |
+
|
| 526 |
+
# 23. OpenAI reasoning_content field
|
| 527 |
+
execute_test(
|
| 528 |
+
"23. OpenAI reasoning_content field",
|
| 529 |
+
messages=[
|
| 530 |
+
{"role": "user", "content": "Question 1"},
|
| 531 |
+
{"role": "assistant", "content": "Answer 1", "reasoning_content": "Deep thought 1"},
|
| 532 |
+
{"role": "user", "content": "Question 2"}
|
| 533 |
+
],
|
| 534 |
+
kwargs={"preserve_thinking": True},
|
| 535 |
+
expected_in=[
|
| 536 |
+
"<|im_start|>assistant\n<think>\nDeep thought 1\n</think>\n\nAnswer 1<|im_end|>\n"
|
| 537 |
+
]
|
| 538 |
+
)
|
| 539 |
+
|
| 540 |
+
# 24. Anthropic message.thinking field
|
| 541 |
+
execute_test(
|
| 542 |
+
"24. Anthropic message.thinking field",
|
| 543 |
+
messages=[
|
| 544 |
+
{"role": "user", "content": "Question 1"},
|
| 545 |
+
{"role": "assistant", "content": "Answer 1", "thinking": "Anthropic thought 1"},
|
| 546 |
+
{"role": "user", "content": "Question 2"}
|
| 547 |
+
],
|
| 548 |
+
kwargs={"preserve_thinking": True},
|
| 549 |
+
expected_in=[
|
| 550 |
+
"<|im_start|>assistant\n<think>\nAnthropic thought 1\n</think>\n\nAnswer 1<|im_end|>\n"
|
| 551 |
+
]
|
| 552 |
+
)
|
| 553 |
+
|
| 554 |
+
# ==========================================
|
| 555 |
+
# 4. Tool Calling (XML & JSON)
|
| 556 |
+
# ==========================================
|
| 557 |
+
|
| 558 |
+
# 25. Tool calling with dict arguments (XML)
|
| 559 |
+
execute_test(
|
| 560 |
+
"25. Tool calling with dict arguments (XML)",
|
| 561 |
+
messages=[
|
| 562 |
+
{"role": "user", "content": "Weather in Paris?"},
|
| 563 |
+
{
|
| 564 |
+
"role": "assistant",
|
| 565 |
+
"content": "",
|
| 566 |
+
"tool_calls": [
|
| 567 |
+
{
|
| 568 |
+
"type": "function",
|
| 569 |
+
"function": {
|
| 570 |
+
"name": "get_weather",
|
| 571 |
+
"arguments": {"city": "Paris"}
|
| 572 |
+
}
|
| 573 |
+
}
|
| 574 |
+
]
|
| 575 |
+
}
|
| 576 |
+
],
|
| 577 |
+
expected_in=[
|
| 578 |
+
"<|im_start|>assistant\n<think>\n\n</think>\n\n<tool_call>\n<function=get_weather>\n<parameter=city>\nParis\n</parameter>\n</function>\n</tool_call><|im_end|>\n"
|
| 579 |
+
]
|
| 580 |
+
)
|
| 581 |
+
|
| 582 |
+
# 26. Tool calling with JSON string arguments (XML)
|
| 583 |
+
execute_test(
|
| 584 |
+
"26. Tool calling with JSON string arguments (XML)",
|
| 585 |
+
messages=[
|
| 586 |
+
{"role": "user", "content": "Weather in Paris?"},
|
| 587 |
+
{
|
| 588 |
+
"role": "assistant",
|
| 589 |
+
"content": "",
|
| 590 |
+
"tool_calls": [
|
| 591 |
+
{
|
| 592 |
+
"type": "function",
|
| 593 |
+
"function": {
|
| 594 |
+
"name": "get_weather",
|
| 595 |
+
"arguments": '{"city": "Paris"}'
|
| 596 |
+
}
|
| 597 |
+
}
|
| 598 |
+
]
|
| 599 |
+
}
|
| 600 |
+
],
|
| 601 |
+
kwargs={"tool_call_format": "xml"},
|
| 602 |
+
expected_in=[
|
| 603 |
+
"<|im_start|>assistant\n<think>\n\n</think>\n\n<tool_call>\n<function=get_weather>\n{\"city\": \"Paris\"}</function>\n</tool_call><|im_end|>\n"
|
| 604 |
+
]
|
| 605 |
+
)
|
| 606 |
+
|
| 607 |
+
# 27. Tool calling with dict arguments (JSON format)
|
| 608 |
+
execute_test(
|
| 609 |
+
"27. Tool calling with dict arguments (JSON format)",
|
| 610 |
+
messages=[
|
| 611 |
+
{"role": "user", "content": "Weather in Paris?"},
|
| 612 |
+
{
|
| 613 |
+
"role": "assistant",
|
| 614 |
+
"content": "",
|
| 615 |
+
"tool_calls": [
|
| 616 |
+
{
|
| 617 |
+
"type": "function",
|
| 618 |
+
"function": {
|
| 619 |
+
"name": "get_weather",
|
| 620 |
+
"arguments": {"city": "Paris"}
|
| 621 |
+
}
|
| 622 |
+
}
|
| 623 |
+
]
|
| 624 |
+
}
|
| 625 |
+
],
|
| 626 |
+
kwargs={"tool_call_format": "json"},
|
| 627 |
+
expected_in=[
|
| 628 |
+
'<|im_start|>assistant\n<think>\n\n</think>\n\n<tool_call>\n{"name": "get_weather", "arguments": {"city": "Paris"}}\n</tool_call><|im_end|>\n'
|
| 629 |
+
]
|
| 630 |
+
)
|
| 631 |
+
|
| 632 |
+
# 28. Tool calling with JSON string arguments (JSON format)
|
| 633 |
+
execute_test(
|
| 634 |
+
"28. Tool calling with JSON string arguments (JSON format)",
|
| 635 |
+
messages=[
|
| 636 |
+
{"role": "user", "content": "Weather in Paris?"},
|
| 637 |
+
{
|
| 638 |
+
"role": "assistant",
|
| 639 |
+
"content": "",
|
| 640 |
+
"tool_calls": [
|
| 641 |
+
{
|
| 642 |
+
"type": "function",
|
| 643 |
+
"function": {
|
| 644 |
+
"name": "get_weather",
|
| 645 |
+
"arguments": '{"city": "Paris"}'
|
| 646 |
+
}
|
| 647 |
+
}
|
| 648 |
+
]
|
| 649 |
+
}
|
| 650 |
+
],
|
| 651 |
+
kwargs={"tool_call_format": "json"},
|
| 652 |
+
expected_in=[
|
| 653 |
+
'<|im_start|>assistant\n<think>\n\n</think>\n\n<tool_call>\n{"name": "get_weather", "arguments": {"city": "Paris"}}\n</tool_call><|im_end|>\n'
|
| 654 |
+
]
|
| 655 |
+
)
|
| 656 |
+
|
| 657 |
+
# 29. Tool calling with empty arguments string
|
| 658 |
+
execute_test(
|
| 659 |
+
"29. Tool calling with empty arguments string",
|
| 660 |
+
messages=[
|
| 661 |
+
{"role": "user", "content": "Call tool without args"},
|
| 662 |
+
{
|
| 663 |
+
"role": "assistant",
|
| 664 |
+
"content": "",
|
| 665 |
+
"tool_calls": [
|
| 666 |
+
{
|
| 667 |
+
"type": "function",
|
| 668 |
+
"function": {
|
| 669 |
+
"name": "no_arg_tool",
|
| 670 |
+
"arguments": ""
|
| 671 |
+
}
|
| 672 |
+
}
|
| 673 |
+
]
|
| 674 |
+
}
|
| 675 |
+
],
|
| 676 |
+
expected_in=[
|
| 677 |
+
"<|im_start|>assistant\n<think>\n\n</think>\n\n<tool_call>\n<function=no_arg_tool>\n</function>\n</tool_call><|im_end|>\n"
|
| 678 |
+
]
|
| 679 |
+
)
|
| 680 |
+
|
| 681 |
+
# ==========================================
|
| 682 |
+
# 5. Payload Truncation & Error Escalation
|
| 683 |
+
# ==========================================
|
| 684 |
+
|
| 685 |
+
# 30. Dynamic parameter truncation (max_tool_arg_chars)
|
| 686 |
+
execute_test(
|
| 687 |
+
"30. Dynamic parameter truncation (max_tool_arg_chars)",
|
| 688 |
+
messages=[
|
| 689 |
+
{"role": "user", "content": "Execute SQL"},
|
| 690 |
+
{
|
| 691 |
+
"role": "assistant",
|
| 692 |
+
"content": "",
|
| 693 |
+
"tool_calls": [
|
| 694 |
+
{
|
| 695 |
+
"type": "function",
|
| 696 |
+
"function": {
|
| 697 |
+
"name": "run_sql",
|
| 698 |
+
"arguments": {"query": "SELECT * FROM users WHERE id = 1234567890 AND active = true"}
|
| 699 |
+
}
|
| 700 |
+
}
|
| 701 |
+
]
|
| 702 |
+
}
|
| 703 |
+
],
|
| 704 |
+
kwargs={"max_tool_arg_chars": 20},
|
| 705 |
+
expected_in=[
|
| 706 |
+
"[TRUNCATED - original length"
|
| 707 |
+
]
|
| 708 |
+
)
|
| 709 |
+
|
| 710 |
+
# 31. Dynamic response truncation (max_tool_response_chars)
|
| 711 |
+
execute_test(
|
| 712 |
+
"31. Dynamic response truncation (max_tool_response_chars)",
|
| 713 |
+
messages=[
|
| 714 |
+
{"role": "user", "content": "Search files"},
|
| 715 |
+
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "search", "arguments": {}}}]},
|
| 716 |
+
{"role": "tool", "content": "A" * 200}
|
| 717 |
+
],
|
| 718 |
+
kwargs={"max_tool_response_chars": 50},
|
| 719 |
+
expected_in=[
|
| 720 |
+
"[TRUNCATED - original length 200 chars]"
|
| 721 |
+
]
|
| 722 |
+
)
|
| 723 |
+
|
| 724 |
+
# 32. Consecutive tool error warning 1
|
| 725 |
+
execute_test(
|
| 726 |
+
"32. Consecutive tool error warning 1",
|
| 727 |
+
messages=[
|
| 728 |
+
{"role": "user", "content": "Run tool"},
|
| 729 |
+
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "run", "arguments": {}}}]},
|
| 730 |
+
{"role": "tool", "content": '{"error": "file not found"}'}
|
| 731 |
+
],
|
| 732 |
+
expected_in=[
|
| 733 |
+
"⚠️ SYSTEM WARNING: The previous tool call returned an error. Diagnose the failure and retry with completely corrected arguments."
|
| 734 |
+
]
|
| 735 |
+
)
|
| 736 |
+
|
| 737 |
+
# 33. Consecutive tool error warning 2 (retaining reasoning for error correction)
|
| 738 |
+
execute_test(
|
| 739 |
+
"33. Consecutive tool error warning 2 (retaining reasoning for error correction)",
|
| 740 |
+
messages=[
|
| 741 |
+
{"role": "user", "content": "Run tool"},
|
| 742 |
+
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "run", "arguments": {}}}]},
|
| 743 |
+
{"role": "tool", "content": '{"error": "file not found"}'},
|
| 744 |
+
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "run", "arguments": {}}}]},
|
| 745 |
+
{"role": "tool", "content": '{"error": "permission denied"}'}
|
| 746 |
+
],
|
| 747 |
+
expected_in=[
|
| 748 |
+
"⚠️ SYSTEM WARNING: 2 consecutive tool errors detected. Your previous approach is incorrect. You MUST use a fundamentally different approach or corrected arguments.",
|
| 749 |
+
"<|im_start|>assistant\n<think>\n"
|
| 750 |
+
]
|
| 751 |
+
)
|
| 752 |
+
|
| 753 |
+
# 34. Mid-conversation system & developer messages
|
| 754 |
+
execute_test(
|
| 755 |
+
"34. Mid-conversation system & developer messages",
|
| 756 |
+
messages=[
|
| 757 |
+
{"role": "user", "content": "Hello"},
|
| 758 |
+
{"role": "assistant", "content": "Hi"},
|
| 759 |
+
{"role": "developer", "content": "Mid-conversation update: user changed context."},
|
| 760 |
+
{"role": "user", "content": "Continue"}
|
| 761 |
+
],
|
| 762 |
+
expected_in=[
|
| 763 |
+
"<|im_start|>system\nMid-conversation update: user changed context.<|im_end|>\n",
|
| 764 |
+
"<|im_start|>user\nContinue<|im_end|>\n"
|
| 765 |
+
]
|
| 766 |
+
)
|
| 767 |
+
|
| 768 |
+
# ==========================================
|
| 769 |
+
# 3. v22.2 Enhancements & Community Fixes
|
| 770 |
+
# ==========================================
|
| 771 |
+
|
| 772 |
+
# 35. reasoning_effort='ultracode' (Discussion #78)
|
| 773 |
+
execute_test(
|
| 774 |
+
"35. reasoning_effort='ultracode'",
|
| 775 |
+
messages=[{"role": "user", "content": "Analyze algorithm"}],
|
| 776 |
+
kwargs={"reasoning_effort": "ultracode"},
|
| 777 |
+
expected_in=[
|
| 778 |
+
"<|im_start|>system\nReasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.<|im_end|>\n"
|
| 779 |
+
]
|
| 780 |
+
)
|
| 781 |
+
|
| 782 |
+
# 36. reasoning_effort='extreme'
|
| 783 |
+
execute_test(
|
| 784 |
+
"36. reasoning_effort='extreme'",
|
| 785 |
+
messages=[{"role": "user", "content": "Analyze algorithm"}],
|
| 786 |
+
kwargs={"reasoning_effort": "extreme"},
|
| 787 |
+
expected_in=[
|
| 788 |
+
"<|im_start|>system\nReasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.<|im_end|>\n"
|
| 789 |
+
]
|
| 790 |
+
)
|
| 791 |
+
|
| 792 |
+
# 37. Inline <|think_ultracode|> in user message
|
| 793 |
+
execute_test(
|
| 794 |
+
"37. Inline <|think_ultracode|> in user message",
|
| 795 |
+
messages=[{"role": "user", "content": "Optimize this shader <|think_ultracode|>"}],
|
| 796 |
+
expected_in=[
|
| 797 |
+
"<|im_start|>system\nReasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.<|im_end|>\n",
|
| 798 |
+
"<|im_start|>user\nOptimize this shader<|im_end|>\n"
|
| 799 |
+
],
|
| 800 |
+
expected_not_in=[
|
| 801 |
+
"<|think_ultracode|>"
|
| 802 |
+
]
|
| 803 |
+
)
|
| 804 |
+
|
| 805 |
+
# 38. Multiple leading system and developer messages merging
|
| 806 |
+
execute_test(
|
| 807 |
+
"38. Multiple leading system and developer messages merging",
|
| 808 |
+
messages=[
|
| 809 |
+
{"role": "system", "content": "Base instructions."},
|
| 810 |
+
{"role": "developer", "content": "Developer constraints."},
|
| 811 |
+
{"role": "system", "content": "Additional guidelines."},
|
| 812 |
+
{"role": "user", "content": "Hello"}
|
| 813 |
+
],
|
| 814 |
+
expected_in=[
|
| 815 |
+
"<|im_start|>system\nBase instructions.\n\nDeveloper constraints.\n\nAdditional guidelines.<|im_end|>\n",
|
| 816 |
+
"<|im_start|>user\nHello<|im_end|>\n"
|
| 817 |
+
]
|
| 818 |
+
)
|
| 819 |
+
|
| 820 |
+
# 39. Grep code search with throw Error not false-positiving on tool error (Discussion #66)
|
| 821 |
+
execute_test(
|
| 822 |
+
"39. Grep code search with throw Error not false-positiving on tool error (Discussion #66)",
|
| 823 |
+
messages=[
|
| 824 |
+
{"role": "user", "content": "Search for error handlers"},
|
| 825 |
+
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "grep", "arguments": {"pattern": "failed to"}}}]},
|
| 826 |
+
{"role": "tool", "content": "src/lib/api.ts:42: throw new Error('failed to fetch');"},
|
| 827 |
+
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "grep", "arguments": {"pattern": "failed to"}}}]},
|
| 828 |
+
{"role": "tool", "content": "src/lib/auth.ts:18: throw new Error('failed to authenticate');"}
|
| 829 |
+
],
|
| 830 |
+
expected_not_in=[
|
| 831 |
+
"⚠️ SYSTEM WARNING"
|
| 832 |
+
]
|
| 833 |
+
)
|
| 834 |
+
|
| 835 |
+
# 40. Grep code search with console.error not false-positiving
|
| 836 |
+
execute_test(
|
| 837 |
+
"40. Grep code search with console.error not false-positiving",
|
| 838 |
+
messages=[
|
| 839 |
+
{"role": "user", "content": "Search for error logging"},
|
| 840 |
+
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "grep", "arguments": {"pattern": "console.error"}}}]},
|
| 841 |
+
{"role": "tool", "content": "src/utils.js:5: console.error('failed to load config');"}
|
| 842 |
+
],
|
| 843 |
+
expected_not_in=[
|
| 844 |
+
"⚠️ SYSTEM WARNING"
|
| 845 |
+
]
|
| 846 |
+
)
|
| 847 |
+
|
| 848 |
+
# 41. Python Traceback legitimately triggers tool error warning
|
| 849 |
+
execute_test(
|
| 850 |
+
"41. Python Traceback legitimately triggers tool error warning",
|
| 851 |
+
messages=[
|
| 852 |
+
{"role": "user", "content": "Run script"},
|
| 853 |
+
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "python", "arguments": {"code": "1/0"}}}]},
|
| 854 |
+
{"role": "tool", "content": "Traceback (most recent call last):\n File 'test.py', line 1\nZeroDivisionError: division by zero"}
|
| 855 |
+
],
|
| 856 |
+
expected_in=[
|
| 857 |
+
"⚠️ SYSTEM WARNING: The previous tool call returned an error. Diagnose the failure and retry with completely corrected arguments."
|
| 858 |
+
]
|
| 859 |
+
)
|
| 860 |
+
|
| 861 |
+
# 42. Safe XML parameter formatting for booleans, nulls, and numbers
|
| 862 |
+
execute_test(
|
| 863 |
+
"42. Safe XML parameter formatting for booleans, nulls, and numbers",
|
| 864 |
+
messages=[
|
| 865 |
+
{"role": "user", "content": "Call config tool"},
|
| 866 |
+
{"role": "assistant", "content": "", "tool_calls": [{
|
| 867 |
+
"type": "function",
|
| 868 |
+
"function": {
|
| 869 |
+
"name": "set_config",
|
| 870 |
+
"arguments": {
|
| 871 |
+
"is_active": True,
|
| 872 |
+
"is_debug": False,
|
| 873 |
+
"retries": 5,
|
| 874 |
+
"metadata": None
|
| 875 |
+
}
|
| 876 |
+
}
|
| 877 |
+
}]}
|
| 878 |
+
],
|
| 879 |
+
expected_in=[
|
| 880 |
+
"<parameter=is_active>\ntrue\n</parameter>",
|
| 881 |
+
"<parameter=is_debug>\nfalse\n</parameter>",
|
| 882 |
+
"<parameter=retries>\n5\n</parameter>",
|
| 883 |
+
"<parameter=metadata>\nnull\n</parameter>"
|
| 884 |
+
]
|
| 885 |
+
)
|
| 886 |
+
|
| 887 |
+
# 43. KV Cache Stability: History with JSON string arguments preserves canonical XML system prompt
|
| 888 |
+
execute_test(
|
| 889 |
+
"43. KV Cache Stability: History with JSON string arguments preserves canonical XML system prompt",
|
| 890 |
+
messages=[
|
| 891 |
+
{"role": "user", "content": "What's the weather?"},
|
| 892 |
+
{"role": "assistant", "content": "", "tool_calls": [{
|
| 893 |
+
"type": "function",
|
| 894 |
+
"function": {
|
| 895 |
+
"name": "get_weather",
|
| 896 |
+
"arguments": '{"city": "Paris", "unit": "celsius"}'
|
| 897 |
+
}
|
| 898 |
+
}]},
|
| 899 |
+
{"role": "tool", "content": '{"temp": 22}'}
|
| 900 |
+
],
|
| 901 |
+
tools=tools_sample,
|
| 902 |
+
expected_in=[
|
| 903 |
+
"<function=example_function_name>",
|
| 904 |
+
"<tool_call>\n<function=get_weather>\n{\"city\": \"Paris\", \"unit\": \"celsius\"}</function>\n</tool_call>"
|
| 905 |
+
]
|
| 906 |
+
)
|
| 907 |
+
|
| 908 |
+
# ==========================================
|
| 909 |
+
# 6. Control Tag & Alias Completeness (v22.3)
|
| 910 |
+
# ==========================================
|
| 911 |
+
|
| 912 |
+
# 45. Inline <|think_high|>
|
| 913 |
+
execute_test(
|
| 914 |
+
"45. Inline <|think_high|>",
|
| 915 |
+
messages=[{"role": "user", "content": "Analyze this <|think_high|>"}],
|
| 916 |
+
expected_in=[
|
| 917 |
+
"<|im_start|>system\nReasoning effort is set to xhigh."
|
| 918 |
+
],
|
| 919 |
+
expected_not_in=[
|
| 920 |
+
"<|think_high|>"
|
| 921 |
+
]
|
| 922 |
+
)
|
| 923 |
+
|
| 924 |
+
# 46. Inline <|think_extreme|>
|
| 925 |
+
execute_test(
|
| 926 |
+
"46. Inline <|think_extreme|>",
|
| 927 |
+
messages=[{"role": "user", "content": "Analyze this <|think_extreme|>"}],
|
| 928 |
+
expected_in=[
|
| 929 |
+
"<|im_start|>system\nReasoning effort is set to xhigh."
|
| 930 |
+
],
|
| 931 |
+
expected_not_in=[
|
| 932 |
+
"<|think_extreme|>"
|
| 933 |
+
]
|
| 934 |
+
)
|
| 935 |
+
|
| 936 |
+
# 47. Inline <|think_max|>
|
| 937 |
+
execute_test(
|
| 938 |
+
"47. Inline <|think_max|>",
|
| 939 |
+
messages=[{"role": "user", "content": "Analyze this <|think_max|>"}],
|
| 940 |
+
expected_in=[
|
| 941 |
+
"<|im_start|>system\nReasoning effort is set to xhigh."
|
| 942 |
+
],
|
| 943 |
+
expected_not_in=[
|
| 944 |
+
"<|think_max|>"
|
| 945 |
+
]
|
| 946 |
+
)
|
| 947 |
+
|
| 948 |
+
# 48. Inline <|think_minimal|> injects low reasoning instructions
|
| 949 |
+
execute_test(
|
| 950 |
+
"48. Inline <|think_minimal|> injects low reasoning instructions",
|
| 951 |
+
messages=[{"role": "user", "content": "Answer fast <|think_minimal|>"}],
|
| 952 |
+
expected_in=[
|
| 953 |
+
"<|im_start|>system\nReasoning effort is set to low."
|
| 954 |
+
],
|
| 955 |
+
expected_not_in=[
|
| 956 |
+
"<|think_minimal|>"
|
| 957 |
+
]
|
| 958 |
+
)
|
| 959 |
+
|
| 960 |
+
# 49. Inline <|think_on|> overrides enable_thinking=False
|
| 961 |
+
execute_test(
|
| 962 |
+
"49. Inline <|think_on|> overrides enable_thinking=False",
|
| 963 |
+
messages=[{"role": "user", "content": "Think about this <|think_on|>"}],
|
| 964 |
+
kwargs={"enable_thinking": False},
|
| 965 |
+
expected_in=[
|
| 966 |
+
"<|im_start|>assistant\n<think>\n"
|
| 967 |
+
],
|
| 968 |
+
expected_not_in=[
|
| 969 |
+
"<think>\n\n</think>",
|
| 970 |
+
"<|think_on|>"
|
| 971 |
+
]
|
| 972 |
+
)
|
| 973 |
+
|
| 974 |
+
# 50. reasoning_effort is case-insensitive
|
| 975 |
+
execute_test(
|
| 976 |
+
"50. reasoning_effort is case-insensitive",
|
| 977 |
+
messages=[{"role": "user", "content": "Hello!"}],
|
| 978 |
+
kwargs={"reasoning_effort": "XHigh"},
|
| 979 |
+
expected_in=[
|
| 980 |
+
"<|im_start|>system\nReasoning effort is set to xhigh."
|
| 981 |
+
]
|
| 982 |
+
)
|
| 983 |
+
|
| 984 |
+
# 51. reasoning_effort='off' disables thinking
|
| 985 |
+
execute_test(
|
| 986 |
+
"51. reasoning_effort='off' disables thinking",
|
| 987 |
+
messages=[{"role": "user", "content": "Hello!"}],
|
| 988 |
+
kwargs={"reasoning_effort": "off"},
|
| 989 |
+
expected_in=[
|
| 990 |
+
"<|im_start|>assistant\n<think>\n\n</think>\n\n"
|
| 991 |
+
],
|
| 992 |
+
expected_not_in=[
|
| 993 |
+
"Reasoning effort is set to"
|
| 994 |
+
]
|
| 995 |
+
)
|
| 996 |
+
|
| 997 |
+
# 52. reasoning_effort=None falls back to medium
|
| 998 |
+
execute_test(
|
| 999 |
+
"52. reasoning_effort=None falls back to medium",
|
| 1000 |
+
messages=[{"role": "user", "content": "Hello!"}],
|
| 1001 |
+
kwargs={"reasoning_effort": None},
|
| 1002 |
+
expected_in=[
|
| 1003 |
+
"<|im_start|>assistant\n<think>\n"
|
| 1004 |
+
],
|
| 1005 |
+
expected_not_in=[
|
| 1006 |
+
"Reasoning effort is set to"
|
| 1007 |
+
]
|
| 1008 |
+
)
|
| 1009 |
+
|
| 1010 |
+
# 53. Non-string reasoning_effort does not crash
|
| 1011 |
+
execute_test(
|
| 1012 |
+
"53. Non-string reasoning_effort does not crash",
|
| 1013 |
+
messages=[{"role": "user", "content": "Hello!"}],
|
| 1014 |
+
kwargs={"reasoning_effort": 3},
|
| 1015 |
+
expected_in=[
|
| 1016 |
+
"<|im_start|>assistant\n<think>\n"
|
| 1017 |
+
],
|
| 1018 |
+
expected_not_in=[
|
| 1019 |
+
"Reasoning effort is set to"
|
| 1020 |
+
]
|
| 1021 |
+
)
|
| 1022 |
+
|
| 1023 |
+
# 54. Control tag inside a system message is honored and stripped
|
| 1024 |
+
execute_test(
|
| 1025 |
+
"54. Control tag inside a system message is honored and stripped",
|
| 1026 |
+
messages=[
|
| 1027 |
+
{"role": "system", "content": "You are terse. <|think_low|>"},
|
| 1028 |
+
{"role": "user", "content": "Hello"}
|
| 1029 |
+
],
|
| 1030 |
+
expected_in=[
|
| 1031 |
+
"Reasoning effort is set to low.",
|
| 1032 |
+
"You are terse."
|
| 1033 |
+
],
|
| 1034 |
+
expected_not_in=[
|
| 1035 |
+
"<|think_low|>"
|
| 1036 |
+
]
|
| 1037 |
+
)
|
| 1038 |
+
|
| 1039 |
+
# 55. Most recent control tag wins across turns
|
| 1040 |
+
execute_test(
|
| 1041 |
+
"55. Most recent control tag wins across turns",
|
| 1042 |
+
messages=[
|
| 1043 |
+
{"role": "user", "content": "First <|think_xhigh|>"},
|
| 1044 |
+
{"role": "assistant", "content": "Ok"},
|
| 1045 |
+
{"role": "user", "content": "Second <|think_low|>"}
|
| 1046 |
+
],
|
| 1047 |
+
expected_in=[
|
| 1048 |
+
"Reasoning effort is set to low."
|
| 1049 |
+
],
|
| 1050 |
+
expected_not_in=[
|
| 1051 |
+
"Reasoning effort is set to xhigh."
|
| 1052 |
+
]
|
| 1053 |
+
)
|
| 1054 |
+
|
| 1055 |
+
# 56. enable_thinking=False suppresses reasoning effort injection
|
| 1056 |
+
execute_test(
|
| 1057 |
+
"56. enable_thinking=False suppresses reasoning effort injection",
|
| 1058 |
+
messages=[{"role": "user", "content": "Hello!"}],
|
| 1059 |
+
kwargs={"enable_thinking": False, "reasoning_effort": "xhigh"},
|
| 1060 |
+
expected_in=[
|
| 1061 |
+
"<|im_start|>assistant\n<think>\n\n</think>\n\n"
|
| 1062 |
+
],
|
| 1063 |
+
expected_not_in=[
|
| 1064 |
+
"Reasoning effort is set to"
|
| 1065 |
+
]
|
| 1066 |
+
)
|
| 1067 |
+
|
| 1068 |
+
# 57. <|think_off|> takes precedence over an effort tag in the same message
|
| 1069 |
+
execute_test(
|
| 1070 |
+
"57. <|think_off|> takes precedence over an effort tag in the same message",
|
| 1071 |
+
messages=[{"role": "user", "content": "Quick <|think_xhigh|> <|think_off|>"}],
|
| 1072 |
+
expected_in=[
|
| 1073 |
+
"<|im_start|>assistant\n<think>\n\n</think>\n\n"
|
| 1074 |
+
],
|
| 1075 |
+
expected_not_in=[
|
| 1076 |
+
"Reasoning effort is set to",
|
| 1077 |
+
"<|think_"
|
| 1078 |
+
]
|
| 1079 |
+
)
|
| 1080 |
+
|
| 1081 |
+
# ==========================================
|
| 1082 |
+
# 7. Malformed Input & Exception Paths (v22.3)
|
| 1083 |
+
# ==========================================
|
| 1084 |
+
|
| 1085 |
+
# 58. Empty message list raises
|
| 1086 |
+
execute_test(
|
| 1087 |
+
"58. Empty message list raises",
|
| 1088 |
+
messages=[],
|
| 1089 |
+
expect_error=True
|
| 1090 |
+
)
|
| 1091 |
+
|
| 1092 |
+
# 59. Image inside a leading system message raises
|
| 1093 |
+
execute_test(
|
| 1094 |
+
"59. Image inside a leading system message raises",
|
| 1095 |
+
messages=[
|
| 1096 |
+
{"role": "system", "content": [{"type": "image", "image": "data"}]},
|
| 1097 |
+
{"role": "user", "content": "Hello"}
|
| 1098 |
+
],
|
| 1099 |
+
expect_error=True
|
| 1100 |
+
)
|
| 1101 |
+
|
| 1102 |
+
# 60. Scalar (non-string, non-list) content raises
|
| 1103 |
+
execute_test(
|
| 1104 |
+
"60. Scalar content raises",
|
| 1105 |
+
messages=[{"role": "user", "content": 12345}],
|
| 1106 |
+
expect_error=True
|
| 1107 |
+
)
|
| 1108 |
+
|
| 1109 |
+
# 61. Unknown multi-part item type raises
|
| 1110 |
+
execute_test(
|
| 1111 |
+
"61. Unknown multi-part item type raises",
|
| 1112 |
+
messages=[{"role": "user", "content": [{"type": "audio", "audio": "data"}]}],
|
| 1113 |
+
expect_error=True
|
| 1114 |
+
)
|
| 1115 |
+
|
| 1116 |
+
# ==========================================
|
| 1117 |
+
# 8. Vision & Multi-Part Content (v22.3)
|
| 1118 |
+
# ==========================================
|
| 1119 |
+
|
| 1120 |
+
# 62. Image part renders vision tokens inline with text
|
| 1121 |
+
execute_test(
|
| 1122 |
+
"62. Image part renders vision tokens inline with text",
|
| 1123 |
+
messages=[{"role": "user", "content": [
|
| 1124 |
+
{"type": "image", "image": "data"},
|
| 1125 |
+
{"type": "text", "text": "What is this?"}
|
| 1126 |
+
]}],
|
| 1127 |
+
expected_in=[
|
| 1128 |
+
"<|im_start|>user\n<|vision_start|><|image_pad|><|vision_end|>What is this?<|im_end|>\n"
|
| 1129 |
+
]
|
| 1130 |
+
)
|
| 1131 |
+
|
| 1132 |
+
# 63. add_vision_id numbers images across turns
|
| 1133 |
+
execute_test(
|
| 1134 |
+
"63. add_vision_id numbers images across turns",
|
| 1135 |
+
messages=[
|
| 1136 |
+
{"role": "user", "content": [{"type": "image", "image": "a"}]},
|
| 1137 |
+
{"role": "assistant", "content": "Ok"},
|
| 1138 |
+
{"role": "user", "content": [{"type": "image", "image": "b"}]}
|
| 1139 |
+
],
|
| 1140 |
+
kwargs={"add_vision_id": True},
|
| 1141 |
+
expected_in=[
|
| 1142 |
+
"Picture 1: <|vision_start|><|image_pad|><|vision_end|>",
|
| 1143 |
+
"Picture 2: <|vision_start|><|image_pad|><|vision_end|>"
|
| 1144 |
+
]
|
| 1145 |
+
)
|
| 1146 |
+
|
| 1147 |
+
# 64. Video part renders video tokens
|
| 1148 |
+
execute_test(
|
| 1149 |
+
"64. Video part renders video tokens",
|
| 1150 |
+
messages=[{"role": "user", "content": [{"type": "video", "video": "data"}]}],
|
| 1151 |
+
expected_in=[
|
| 1152 |
+
"<|vision_start|><|video_pad|><|vision_end|>"
|
| 1153 |
+
]
|
| 1154 |
+
)
|
| 1155 |
+
|
| 1156 |
+
# ==========================================
|
| 1157 |
+
# 9. Agentic Structure & Wire Format Shapes (v22.3)
|
| 1158 |
+
# ==========================================
|
| 1159 |
+
|
| 1160 |
+
# 65. Assistant text combined with a tool call
|
| 1161 |
+
execute_test(
|
| 1162 |
+
"65. Assistant text combined with a tool call",
|
| 1163 |
+
messages=[
|
| 1164 |
+
{"role": "user", "content": "Weather?"},
|
| 1165 |
+
{"role": "assistant", "content": "Let me check.", "tool_calls": [
|
| 1166 |
+
{"type": "function", "function": {"name": "get_weather", "arguments": {"city": "Paris"}}}
|
| 1167 |
+
]}
|
| 1168 |
+
],
|
| 1169 |
+
expected_in=[
|
| 1170 |
+
"<|im_start|>assistant\n<think>\n\n</think>\n\nLet me check.\n\n<tool_call>\n<function=get_weather>\n"
|
| 1171 |
+
]
|
| 1172 |
+
)
|
| 1173 |
+
|
| 1174 |
+
# 66. Parallel tool calls in a single assistant message
|
| 1175 |
+
execute_test(
|
| 1176 |
+
"66. Parallel tool calls in a single assistant message",
|
| 1177 |
+
messages=[
|
| 1178 |
+
{"role": "user", "content": "Compare"},
|
| 1179 |
+
{"role": "assistant", "content": "", "tool_calls": [
|
| 1180 |
+
{"type": "function", "function": {"name": "f1", "arguments": {"a": "1"}}},
|
| 1181 |
+
{"type": "function", "function": {"name": "f2", "arguments": {"b": "2"}}}
|
| 1182 |
+
]}
|
| 1183 |
+
],
|
| 1184 |
+
expected_in=[
|
| 1185 |
+
"<|im_start|>assistant\n<think>\n\n</think>\n\n<tool_call>\n<function=f1>\n",
|
| 1186 |
+
"</tool_call>\n\n<tool_call>\n<function=f2>\n"
|
| 1187 |
+
]
|
| 1188 |
+
)
|
| 1189 |
+
|
| 1190 |
+
# 67. Consecutive tool results collapse into a single user turn
|
| 1191 |
+
execute_test(
|
| 1192 |
+
"67. Consecutive tool results collapse into a single user turn",
|
| 1193 |
+
messages=[
|
| 1194 |
+
{"role": "user", "content": "Compare"},
|
| 1195 |
+
{"role": "assistant", "content": "", "tool_calls": [
|
| 1196 |
+
{"type": "function", "function": {"name": "f1", "arguments": {}}},
|
| 1197 |
+
{"type": "function", "function": {"name": "f2", "arguments": {}}}
|
| 1198 |
+
]},
|
| 1199 |
+
{"role": "tool", "content": "result one"},
|
| 1200 |
+
{"role": "tool", "content": "result two"},
|
| 1201 |
+
{"role": "user", "content": "Thanks"}
|
| 1202 |
+
],
|
| 1203 |
+
expected_in=[
|
| 1204 |
+
"<|im_start|>user\n<tool_response>\nresult one\n</tool_response>\n<tool_response>\nresult two\n</tool_response><|im_end|>\n"
|
| 1205 |
+
]
|
| 1206 |
+
)
|
| 1207 |
+
|
| 1208 |
+
# 68. Flat tool_call shape without a 'function' wrapper
|
| 1209 |
+
execute_test(
|
| 1210 |
+
"68. Flat tool_call shape without a 'function' wrapper",
|
| 1211 |
+
messages=[
|
| 1212 |
+
{"role": "user", "content": "Weather?"},
|
| 1213 |
+
{"role": "assistant", "content": "", "tool_calls": [
|
| 1214 |
+
{"name": "get_weather", "arguments": {"city": "Paris"}}
|
| 1215 |
+
]}
|
| 1216 |
+
],
|
| 1217 |
+
expected_in=[
|
| 1218 |
+
"<tool_call>\n<function=get_weather>\n<parameter=city>\nParis\n</parameter>\n</function>\n</tool_call>"
|
| 1219 |
+
]
|
| 1220 |
+
)
|
| 1221 |
+
|
| 1222 |
+
# 69. OpenAI tool_call id/index fields are tolerated
|
| 1223 |
+
execute_test(
|
| 1224 |
+
"69. OpenAI tool_call id/index fields are tolerated",
|
| 1225 |
+
messages=[
|
| 1226 |
+
{"role": "user", "content": "Weather?"},
|
| 1227 |
+
{"role": "assistant", "content": "", "tool_calls": [
|
| 1228 |
+
{"id": "call_abc", "index": 0, "type": "function",
|
| 1229 |
+
"function": {"name": "get_weather", "arguments": {"city": "Paris"}}}
|
| 1230 |
+
]},
|
| 1231 |
+
{"role": "tool", "tool_call_id": "call_abc", "content": "22C"}
|
| 1232 |
+
],
|
| 1233 |
+
expected_in=[
|
| 1234 |
+
"<function=get_weather>",
|
| 1235 |
+
"<tool_response>\n22C\n</tool_response>"
|
| 1236 |
+
]
|
| 1237 |
+
)
|
| 1238 |
+
|
| 1239 |
+
# 70. Unknown roles fall back to a labelled user turn
|
| 1240 |
+
execute_test(
|
| 1241 |
+
"70. Unknown roles fall back to a labelled user turn",
|
| 1242 |
+
messages=[
|
| 1243 |
+
{"role": "user", "content": "Hello"},
|
| 1244 |
+
{"role": "critic", "content": "Needs work"}
|
| 1245 |
+
],
|
| 1246 |
+
expected_in=[
|
| 1247 |
+
"<|im_start|>user\n[critic]: Needs work<|im_end|>\n"
|
| 1248 |
+
]
|
| 1249 |
+
)
|
| 1250 |
+
|
| 1251 |
+
# 71. add_generation_prompt=False emits no assistant header
|
| 1252 |
+
execute_test(
|
| 1253 |
+
"71. add_generation_prompt=False emits no assistant header",
|
| 1254 |
+
messages=[{"role": "user", "content": "Hello"}],
|
| 1255 |
+
kwargs={"add_generation_prompt": False},
|
| 1256 |
+
expected_in=[
|
| 1257 |
+
"<|im_start|>user\nHello<|im_end|>\n"
|
| 1258 |
+
],
|
| 1259 |
+
expected_not_in=[
|
| 1260 |
+
"<|im_start|>assistant"
|
| 1261 |
+
]
|
| 1262 |
+
)
|
| 1263 |
+
|
| 1264 |
+
# 72. Empty tools list emits no tool system block
|
| 1265 |
+
execute_test(
|
| 1266 |
+
"72. Empty tools list emits no tool system block",
|
| 1267 |
+
messages=[{"role": "user", "content": "Hello"}],
|
| 1268 |
+
tools=[],
|
| 1269 |
+
expected_not_in=[
|
| 1270 |
+
"# Tools"
|
| 1271 |
+
]
|
| 1272 |
+
)
|
| 1273 |
+
|
| 1274 |
+
# 73. Assistant content=None does not crash
|
| 1275 |
+
execute_test(
|
| 1276 |
+
"73. Assistant content=None does not crash",
|
| 1277 |
+
messages=[
|
| 1278 |
+
{"role": "user", "content": "Hi"},
|
| 1279 |
+
{"role": "assistant", "content": None},
|
| 1280 |
+
{"role": "user", "content": "Again"}
|
| 1281 |
+
],
|
| 1282 |
+
expected_in=[
|
| 1283 |
+
"<|im_start|>assistant\n<think>\n\n</think>\n\n<|im_end|>\n"
|
| 1284 |
+
]
|
| 1285 |
+
)
|
| 1286 |
+
|
| 1287 |
+
# 74. Scalar tool arguments are serialized, not dropped
|
| 1288 |
+
execute_test(
|
| 1289 |
+
"74. Scalar tool arguments are serialized, not dropped",
|
| 1290 |
+
messages=[
|
| 1291 |
+
{"role": "user", "content": "Call"},
|
| 1292 |
+
{"role": "assistant", "content": "", "tool_calls": [
|
| 1293 |
+
{"type": "function", "function": {"name": "f", "arguments": 42}}
|
| 1294 |
+
]}
|
| 1295 |
+
],
|
| 1296 |
+
expected_in=[
|
| 1297 |
+
"<tool_call>\n<function=f>\n42</function>\n</tool_call>"
|
| 1298 |
+
]
|
| 1299 |
+
)
|
| 1300 |
+
|
| 1301 |
+
# ==========================================
|
| 1302 |
+
# 10. Reasoning Extraction Edge Cases (v22.3)
|
| 1303 |
+
# ==========================================
|
| 1304 |
+
|
| 1305 |
+
# 75. <thinking> variant is extracted and normalized
|
| 1306 |
+
execute_test(
|
| 1307 |
+
"75. <thinking> variant is extracted and normalized",
|
| 1308 |
+
messages=[
|
| 1309 |
+
{"role": "user", "content": "Q"},
|
| 1310 |
+
{"role": "assistant", "content": "<thinking>\nT\n</thinking>\n\nA"},
|
| 1311 |
+
{"role": "user", "content": "Q2"}
|
| 1312 |
+
],
|
| 1313 |
+
expected_in=[
|
| 1314 |
+
"<|im_start|>assistant\n<think>\nT\n</think>\n\nA<|im_end|>\n"
|
| 1315 |
+
]
|
| 1316 |
+
)
|
| 1317 |
+
|
| 1318 |
+
# 76. Spaced </think > variant is extracted
|
| 1319 |
+
execute_test(
|
| 1320 |
+
"76. Spaced </think > variant is extracted",
|
| 1321 |
+
messages=[
|
| 1322 |
+
{"role": "user", "content": "Q"},
|
| 1323 |
+
{"role": "assistant", "content": "<think>\nT\n</think >\n\nA"},
|
| 1324 |
+
{"role": "user", "content": "Q2"}
|
| 1325 |
+
],
|
| 1326 |
+
expected_in=[
|
| 1327 |
+
"<|im_start|>assistant\n<think>\nT\n</think>\n\nA<|im_end|>\n"
|
| 1328 |
+
]
|
| 1329 |
+
)
|
| 1330 |
+
|
| 1331 |
+
# 77. Single-line think block is extracted
|
| 1332 |
+
execute_test(
|
| 1333 |
+
"77. Single-line think block is extracted",
|
| 1334 |
+
messages=[
|
| 1335 |
+
{"role": "user", "content": "Q"},
|
| 1336 |
+
{"role": "assistant", "content": "<think>T</think>A"},
|
| 1337 |
+
{"role": "user", "content": "Q2"}
|
| 1338 |
+
],
|
| 1339 |
+
expected_in=[
|
| 1340 |
+
"<|im_start|>assistant\n<think>\nT\n</think>\n\nA<|im_end|>\n"
|
| 1341 |
+
]
|
| 1342 |
+
)
|
| 1343 |
+
|
| 1344 |
+
# 78. reasoning_content plus in-content tags must not duplicate think blocks
|
| 1345 |
+
execute_test(
|
| 1346 |
+
"78. reasoning_content plus in-content tags must not duplicate think blocks",
|
| 1347 |
+
messages=[
|
| 1348 |
+
{"role": "user", "content": "Q"},
|
| 1349 |
+
{"role": "assistant", "reasoning_content": "R", "content": "<think>\nT\n</think>\n\nA"},
|
| 1350 |
+
{"role": "user", "content": "Q2"}
|
| 1351 |
+
],
|
| 1352 |
+
expected_in=[
|
| 1353 |
+
"<|im_start|>assistant\n<think>\nR\n</think>\n\nA<|im_end|>\n"
|
| 1354 |
+
],
|
| 1355 |
+
expected_not_in=[
|
| 1356 |
+
"</think>\n\n<think>"
|
| 1357 |
+
]
|
| 1358 |
+
)
|
| 1359 |
+
|
| 1360 |
+
# 79. preserve_thinking=False retains reasoning inside the active tool loop
|
| 1361 |
+
execute_test(
|
| 1362 |
+
"79. preserve_thinking=False retains reasoning inside the active tool loop",
|
| 1363 |
+
messages=[
|
| 1364 |
+
{"role": "user", "content": "Fix it"},
|
| 1365 |
+
{"role": "assistant", "content": "<think>\nplan the fix\n</think>\n\n", "tool_calls": [
|
| 1366 |
+
{"type": "function", "function": {"name": "run", "arguments": {}}}
|
| 1367 |
+
]},
|
| 1368 |
+
{"role": "tool", "content": "output"}
|
| 1369 |
+
],
|
| 1370 |
+
kwargs={"preserve_thinking": False},
|
| 1371 |
+
expected_in=[
|
| 1372 |
+
"plan the fix"
|
| 1373 |
+
]
|
| 1374 |
+
)
|
| 1375 |
+
|
| 1376 |
+
# 80. preserve_thinking=False strips reasoning once a new user turn starts
|
| 1377 |
+
execute_test(
|
| 1378 |
+
"80. preserve_thinking=False strips reasoning once a new user turn starts",
|
| 1379 |
+
messages=[
|
| 1380 |
+
{"role": "user", "content": "Fix it"},
|
| 1381 |
+
{"role": "assistant", "content": "<think>\nplan the fix\n</think>\n\n", "tool_calls": [
|
| 1382 |
+
{"type": "function", "function": {"name": "run", "arguments": {}}}
|
| 1383 |
+
]},
|
| 1384 |
+
{"role": "tool", "content": "output"},
|
| 1385 |
+
{"role": "assistant", "content": "<think>\nwrap up\n</think>\n\nDone."},
|
| 1386 |
+
{"role": "user", "content": "Thanks"}
|
| 1387 |
+
],
|
| 1388 |
+
kwargs={"preserve_thinking": False},
|
| 1389 |
+
expected_not_in=[
|
| 1390 |
+
"plan the fix",
|
| 1391 |
+
"wrap up"
|
| 1392 |
+
]
|
| 1393 |
+
)
|
| 1394 |
+
|
| 1395 |
+
# 81. Explicit reasoning preserves literal think tags inside the final answer
|
| 1396 |
+
execute_test(
|
| 1397 |
+
"81. Explicit reasoning preserves literal think tags inside the final answer",
|
| 1398 |
+
messages=[
|
| 1399 |
+
{"role": "user", "content": "How do I close the think block?"},
|
| 1400 |
+
{"role": "assistant", "reasoning_content": "R",
|
| 1401 |
+
"content": "Use this closing tag:\n```\n</think>\n```\nDone."},
|
| 1402 |
+
{"role": "user", "content": "thanks"}
|
| 1403 |
+
],
|
| 1404 |
+
expected_in=[
|
| 1405 |
+
"<|im_start|>assistant\n<think>\nR\n</think>\n\nUse this closing tag:\n```\n</think>\n```\nDone.<|im_end|>\n"
|
| 1406 |
+
]
|
| 1407 |
+
)
|
| 1408 |
+
|
| 1409 |
+
# ==========================================
|
| 1410 |
+
# 11. Tool Error Detection Precision (v22.3)
|
| 1411 |
+
# ==========================================
|
| 1412 |
+
|
| 1413 |
+
# 82. Structural error signal fires regardless of payload length
|
| 1414 |
+
execute_test(
|
| 1415 |
+
"82. Structural error signal fires regardless of payload length",
|
| 1416 |
+
messages=[
|
| 1417 |
+
{"role": "user", "content": "Run"},
|
| 1418 |
+
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "f", "arguments": {}}}]},
|
| 1419 |
+
{"role": "tool", "content": '{"error": "boom"}\n' + "detail line\n" * 80}
|
| 1420 |
+
],
|
| 1421 |
+
expected_in=[
|
| 1422 |
+
"⚠️ SYSTEM WARNING: The previous tool call returned an error."
|
| 1423 |
+
]
|
| 1424 |
+
)
|
| 1425 |
+
|
| 1426 |
+
# 83. Long traceback beyond the weak-signal length gate still fires
|
| 1427 |
+
execute_test(
|
| 1428 |
+
"83. Long traceback beyond the weak-signal length gate still fires",
|
| 1429 |
+
messages=[
|
| 1430 |
+
{"role": "user", "content": "Run"},
|
| 1431 |
+
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "f", "arguments": {}}}]},
|
| 1432 |
+
{"role": "tool", "content": "Traceback (most recent call last):\n" +
|
| 1433 |
+
' File "/app/handler.py", line 118, in process\n result = self.client.fetch(payload)\n' * 6 +
|
| 1434 |
+
"ConnectionResetError: [Errno 104] Connection reset by peer"}
|
| 1435 |
+
],
|
| 1436 |
+
expected_in=[
|
| 1437 |
+
"⚠️ SYSTEM WARNING: The previous tool call returned an error."
|
| 1438 |
+
]
|
| 1439 |
+
)
|
| 1440 |
+
|
| 1441 |
+
# 84. Shell-echoed command output still reports a real failure
|
| 1442 |
+
execute_test(
|
| 1443 |
+
"84. Shell-echoed command output still reports a real failure",
|
| 1444 |
+
messages=[
|
| 1445 |
+
{"role": "user", "content": "Build"},
|
| 1446 |
+
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "bash", "arguments": {}}}]},
|
| 1447 |
+
{"role": "tool", "content": "$ npm run build\nError: command not found"}
|
| 1448 |
+
],
|
| 1449 |
+
expected_in=[
|
| 1450 |
+
"⚠️ SYSTEM WARNING: The previous tool call returned an error."
|
| 1451 |
+
]
|
| 1452 |
+
)
|
| 1453 |
+
|
| 1454 |
+
# 85. Timing metadata does not suppress a structural error
|
| 1455 |
+
execute_test(
|
| 1456 |
+
"85. Timing metadata does not suppress a structural error",
|
| 1457 |
+
messages=[
|
| 1458 |
+
{"role": "user", "content": "Query"},
|
| 1459 |
+
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "db", "arguments": {}}}]},
|
| 1460 |
+
{"role": "tool", "content": '{"error": "db timeout", "note": "took 3ms"}'}
|
| 1461 |
+
],
|
| 1462 |
+
expected_in=[
|
| 1463 |
+
"⚠️ SYSTEM WARNING: The previous tool call returned an error."
|
| 1464 |
+
]
|
| 1465 |
+
)
|
| 1466 |
+
|
| 1467 |
+
# 86. Successful exit code report does not trigger an error warning
|
| 1468 |
+
execute_test(
|
| 1469 |
+
"86. Successful exit code report does not trigger an error warning",
|
| 1470 |
+
messages=[
|
| 1471 |
+
{"role": "user", "content": "Run"},
|
| 1472 |
+
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "bash", "arguments": {}}}]},
|
| 1473 |
+
{"role": "tool", "content": "Command completed successfully.\nExit code: 0\n" + "log line\n" * 40}
|
| 1474 |
+
],
|
| 1475 |
+
expected_not_in=[
|
| 1476 |
+
"⚠️ SYSTEM WARNING"
|
| 1477 |
+
]
|
| 1478 |
+
)
|
| 1479 |
+
|
| 1480 |
+
# 87. Nonzero exit code escalates regardless of payload length
|
| 1481 |
+
execute_test(
|
| 1482 |
+
"87. Nonzero exit code escalates regardless of payload length",
|
| 1483 |
+
messages=[
|
| 1484 |
+
{"role": "user", "content": "Run"},
|
| 1485 |
+
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "bash", "arguments": {}}}]},
|
| 1486 |
+
{"role": "tool", "content": "Exit code: 1\n" + "stack frame\n" * 60}
|
| 1487 |
+
],
|
| 1488 |
+
expected_in=[
|
| 1489 |
+
"⚠️ SYSTEM WARNING: The previous tool call returned an error."
|
| 1490 |
+
]
|
| 1491 |
+
)
|
| 1492 |
+
|
| 1493 |
+
# 88. Failure counter resets after a successful call
|
| 1494 |
+
execute_test(
|
| 1495 |
+
"88. Failure counter resets after a successful call",
|
| 1496 |
+
messages=[
|
| 1497 |
+
{"role": "user", "content": "Run"},
|
| 1498 |
+
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "f", "arguments": {}}}]},
|
| 1499 |
+
{"role": "tool", "content": '{"error": "e1"}'},
|
| 1500 |
+
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "f", "arguments": {}}}]},
|
| 1501 |
+
{"role": "tool", "content": '{"ok": true}'},
|
| 1502 |
+
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "f", "arguments": {}}}]},
|
| 1503 |
+
{"role": "tool", "content": '{"error": "e2"}'}
|
| 1504 |
+
],
|
| 1505 |
+
expected_not_in=[
|
| 1506 |
+
"2 consecutive tool errors"
|
| 1507 |
+
]
|
| 1508 |
+
)
|
| 1509 |
+
|
| 1510 |
+
# 89. Failure counter resets on a new user turn
|
| 1511 |
+
execute_test(
|
| 1512 |
+
"89. Failure counter resets on a new user turn",
|
| 1513 |
+
messages=[
|
| 1514 |
+
{"role": "user", "content": "Run"},
|
| 1515 |
+
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "f", "arguments": {}}}]},
|
| 1516 |
+
{"role": "tool", "content": '{"error": "e1"}'},
|
| 1517 |
+
{"role": "user", "content": "Try again"},
|
| 1518 |
+
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "f", "arguments": {}}}]},
|
| 1519 |
+
{"role": "tool", "content": '{"error": "e2"}'}
|
| 1520 |
+
],
|
| 1521 |
+
expected_not_in=[
|
| 1522 |
+
"2 consecutive tool errors"
|
| 1523 |
+
]
|
| 1524 |
+
)
|
| 1525 |
+
|
| 1526 |
+
# 90. Tool response truncation is bypassed in JSON format
|
| 1527 |
+
execute_test(
|
| 1528 |
+
"90. Tool response truncation is bypassed in JSON format",
|
| 1529 |
+
messages=[
|
| 1530 |
+
{"role": "user", "content": "Query"},
|
| 1531 |
+
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "f", "arguments": {}}}]},
|
| 1532 |
+
{"role": "tool", "content": json.dumps({"rows": ["x"] * 200})}
|
| 1533 |
+
],
|
| 1534 |
+
kwargs={"tool_call_format": "json", "max_tool_response_chars": 50},
|
| 1535 |
+
expected_not_in=[
|
| 1536 |
+
"[TRUNCATED"
|
| 1537 |
+
]
|
| 1538 |
+
)
|
| 1539 |
+
|
| 1540 |
+
# 91. max_tool_arg_chars applies to serialized JSON string arguments
|
| 1541 |
+
execute_test(
|
| 1542 |
+
"91. max_tool_arg_chars applies to serialized JSON string arguments",
|
| 1543 |
+
messages=[
|
| 1544 |
+
{"role": "user", "content": "Query"},
|
| 1545 |
+
{"role": "assistant", "content": "", "tool_calls": [
|
| 1546 |
+
{"type": "function", "function": {"name": "f", "arguments": json.dumps({"q": "S" * 400})}}
|
| 1547 |
+
]}
|
| 1548 |
+
],
|
| 1549 |
+
kwargs={"max_tool_arg_chars": 20},
|
| 1550 |
+
expected_in=[
|
| 1551 |
+
"[TRUNCATED - original length"
|
| 1552 |
+
]
|
| 1553 |
+
)
|
| 1554 |
+
|
| 1555 |
+
# ==========================================
|
| 1556 |
+
# 12. KV Cache Prefix Stability & Build Parity (v22.3)
|
| 1557 |
+
# ==========================================
|
| 1558 |
+
|
| 1559 |
+
agentic_session = [
|
| 1560 |
+
{"role": "system", "content": "You are a coding agent."},
|
| 1561 |
+
{"role": "user", "content": "Fix the build"},
|
| 1562 |
+
{"role": "assistant", "content": "<think>\ninspect the logs\n</think>\n\n", "tool_calls": [
|
| 1563 |
+
{"type": "function", "function": {"name": "run", "arguments": {"cmd": "make"}}}
|
| 1564 |
+
]},
|
| 1565 |
+
{"role": "tool", "content": "Error: missing header"},
|
| 1566 |
+
{"role": "assistant", "content": "<think>\nadd the include\n</think>\n\nAdding it now.", "tool_calls": [
|
| 1567 |
+
{"type": "function", "function": {"name": "edit", "arguments": {"file": "main.c"}}}
|
| 1568 |
+
]},
|
| 1569 |
+
{"role": "tool", "content": "ok"},
|
| 1570 |
+
{"role": "assistant", "content": "<think>\nverify\n</think>\n\nBuild fixed."},
|
| 1571 |
+
{"role": "user", "content": "Thanks"},
|
| 1572 |
+
]
|
| 1573 |
+
|
| 1574 |
+
# 92. Prefix KV cache stability across a full agentic session (default settings)
|
| 1575 |
+
execute_prefix_test(
|
| 1576 |
+
"92. Prefix KV cache stability across a full agentic session (default settings)",
|
| 1577 |
+
messages=agentic_session
|
| 1578 |
+
)
|
| 1579 |
+
|
| 1580 |
+
# 93. Prefix KV cache stability with tools and xhigh reasoning
|
| 1581 |
+
execute_prefix_test(
|
| 1582 |
+
"93. Prefix KV cache stability with tools and xhigh reasoning",
|
| 1583 |
+
messages=agentic_session,
|
| 1584 |
+
kwargs={"tools": tools_sample, "reasoning_effort": "xhigh"}
|
| 1585 |
+
)
|
| 1586 |
+
|
| 1587 |
+
# 94. chat_template_oneline.txt renders identically to chat_template.jinja
|
| 1588 |
+
execute_parity_test(
|
| 1589 |
+
"94. chat_template_oneline.txt renders identically to chat_template.jinja",
|
| 1590 |
+
cases=[
|
| 1591 |
+
("plain", [{"role": "user", "content": "Hello"}], {}),
|
| 1592 |
+
("system + tools", [
|
| 1593 |
+
{"role": "system", "content": "You are helpful."},
|
| 1594 |
+
{"role": "user", "content": "Weather?"}
|
| 1595 |
+
], {"tools": tools_sample}),
|
| 1596 |
+
("agentic session", agentic_session, {}),
|
| 1597 |
+
("json format", [
|
| 1598 |
+
{"role": "user", "content": "Call"},
|
| 1599 |
+
{"role": "assistant", "content": "", "tool_calls": [
|
| 1600 |
+
{"type": "function", "function": {"name": "f", "arguments": {"a": "1"}}}
|
| 1601 |
+
]}
|
| 1602 |
+
], {"tool_call_format": "json"}),
|
| 1603 |
+
]
|
| 1604 |
+
)
|
| 1605 |
+
|
| 1606 |
+
# ==========================================
|
| 1607 |
+
# 13. Success Envelopes & Input Shapes (v22.3)
|
| 1608 |
+
# ==========================================
|
| 1609 |
+
|
| 1610 |
+
# 95. JSON success envelope with error:null does not trigger a warning
|
| 1611 |
+
execute_test(
|
| 1612 |
+
"95. JSON success envelope with error:null does not trigger a warning",
|
| 1613 |
+
messages=[
|
| 1614 |
+
{"role": "user", "content": "Query"},
|
| 1615 |
+
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "api", "arguments": {}}}]},
|
| 1616 |
+
{"role": "tool", "content": '{"error": null, "data": {"rows": [1, 2, 3]}}\n' + "x" * 700}
|
| 1617 |
+
],
|
| 1618 |
+
expected_not_in=[
|
| 1619 |
+
"⚠️ SYSTEM WARNING"
|
| 1620 |
+
]
|
| 1621 |
+
)
|
| 1622 |
+
|
| 1623 |
+
# 96. JSON success envelope with error:false does not trigger a warning
|
| 1624 |
+
execute_test(
|
| 1625 |
+
"96. JSON success envelope with error:false does not trigger a warning",
|
| 1626 |
+
messages=[
|
| 1627 |
+
{"role": "user", "content": "Query"},
|
| 1628 |
+
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "api", "arguments": {}}}]},
|
| 1629 |
+
{"role": "tool", "content": '{"error": false, "result": "done"}'}
|
| 1630 |
+
],
|
| 1631 |
+
expected_not_in=[
|
| 1632 |
+
"⚠️ SYSTEM WARNING"
|
| 1633 |
+
]
|
| 1634 |
+
)
|
| 1635 |
+
|
| 1636 |
+
# 97. Inline effort tag overrides the reasoning_effort kwarg
|
| 1637 |
+
execute_test(
|
| 1638 |
+
"97. Inline effort tag overrides the reasoning_effort kwarg",
|
| 1639 |
+
messages=[{"role": "user", "content": "Answer fast <|think_low|>"}],
|
| 1640 |
+
kwargs={"reasoning_effort": "xhigh"},
|
| 1641 |
+
expected_in=[
|
| 1642 |
+
"Reasoning effort is set to low."
|
| 1643 |
+
],
|
| 1644 |
+
expected_not_in=[
|
| 1645 |
+
"Reasoning effort is set to xhigh."
|
| 1646 |
+
]
|
| 1647 |
+
)
|
| 1648 |
+
|
| 1649 |
+
# 98. Tool result with multipart text content is flattened
|
| 1650 |
+
execute_test(
|
| 1651 |
+
"98. Tool result with multipart text content is flattened",
|
| 1652 |
+
messages=[
|
| 1653 |
+
{"role": "user", "content": "Query"},
|
| 1654 |
+
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "api", "arguments": {}}}]},
|
| 1655 |
+
{"role": "tool", "content": [{"type": "text", "text": "RESULT_42"}]}
|
| 1656 |
+
],
|
| 1657 |
+
expected_in=[
|
| 1658 |
+
"<tool_response>\nRESULT_42\n</tool_response>"
|
| 1659 |
+
]
|
| 1660 |
+
)
|
| 1661 |
+
|
| 1662 |
+
# 99. Argument exactly at max_tool_arg_chars is not truncated
|
| 1663 |
+
execute_test(
|
| 1664 |
+
"99. Argument exactly at max_tool_arg_chars is not truncated",
|
| 1665 |
+
messages=[
|
| 1666 |
+
{"role": "user", "content": "Query"},
|
| 1667 |
+
{"role": "assistant", "content": "", "tool_calls": [
|
| 1668 |
+
{"type": "function", "function": {"name": "f", "arguments": {"k": "A" * 40}}}]}
|
| 1669 |
+
],
|
| 1670 |
+
kwargs={"max_tool_arg_chars": 40},
|
| 1671 |
+
expected_in=[
|
| 1672 |
+
"A" * 40
|
| 1673 |
+
],
|
| 1674 |
+
expected_not_in=[
|
| 1675 |
+
"[TRUNCATED"
|
| 1676 |
+
]
|
| 1677 |
+
)
|
| 1678 |
+
|
| 1679 |
+
# 100. Multiple images in a single message get sequential vision ids
|
| 1680 |
+
execute_test(
|
| 1681 |
+
"100. Multiple images in a single message get sequential vision ids",
|
| 1682 |
+
messages=[{"role": "user", "content": [
|
| 1683 |
+
{"type": "image", "image": "a"},
|
| 1684 |
+
{"type": "image", "image": "b"},
|
| 1685 |
+
{"type": "text", "text": "compare these"}
|
| 1686 |
+
]}],
|
| 1687 |
+
kwargs={"add_vision_id": True},
|
| 1688 |
+
expected_in=[
|
| 1689 |
+
"Picture 1: <|vision_start|><|image_pad|><|vision_end|>",
|
| 1690 |
+
"Picture 2: <|vision_start|><|image_pad|><|vision_end|>"
|
| 1691 |
+
]
|
| 1692 |
+
)
|
| 1693 |
+
|
| 1694 |
+
# ==========================================
|
| 1695 |
+
# 14. Property-Based Fuzzing (v22.3)
|
| 1696 |
+
# ==========================================
|
| 1697 |
+
|
| 1698 |
+
def run_fuzz_property_test(name, cases, seed):
|
| 1699 |
+
print(f"\n--- Running Test: {name} ---")
|
| 1700 |
+
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
| 1701 |
+
try:
|
| 1702 |
+
import fuzz_template
|
| 1703 |
+
failures = fuzz_template.run_fuzz(cases=cases, seed=seed,
|
| 1704 |
+
template_dir=TEMPLATE_DIR,
|
| 1705 |
+
template_file=TEMPLATE_FILE)
|
| 1706 |
+
except Exception:
|
| 1707 |
+
print(f"❌ FAILED with exception:\n{traceback.format_exc()}")
|
| 1708 |
+
return False
|
| 1709 |
+
if failures:
|
| 1710 |
+
first = failures[0]
|
| 1711 |
+
print(f"❌ FAILED: {len(failures)} invariant violation(s); "
|
| 1712 |
+
f"first: [{first['invariant']}] {first['detail']}")
|
| 1713 |
+
print(f"Repro (seed {seed}, case {first['case']}): {first['repro'][:600]}")
|
| 1714 |
+
return False
|
| 1715 |
+
print(f"✅ PASSED ({cases} generated conversations, seed {seed})")
|
| 1716 |
+
return True
|
| 1717 |
+
|
| 1718 |
+
def execute_fuzz_property_test(*args, **kwargs):
|
| 1719 |
+
global tests_passed, tests_total
|
| 1720 |
+
tests_total += 1
|
| 1721 |
+
if run_fuzz_property_test(*args, **kwargs):
|
| 1722 |
+
tests_passed += 1
|
| 1723 |
+
|
| 1724 |
+
# 101. Property fuzz: generated conversations uphold all structural invariants
|
| 1725 |
+
execute_fuzz_property_test(
|
| 1726 |
+
"101. Property fuzz: generated conversations uphold all structural invariants",
|
| 1727 |
+
cases=300,
|
| 1728 |
+
seed=20260820
|
| 1729 |
+
)
|
| 1730 |
+
|
| 1731 |
+
print("\n==========================================")
|
| 1732 |
+
print(f"Results: {tests_passed} / {tests_total} tests passed ({tests_passed/tests_total*100:.1f}%)")
|
| 1733 |
+
print("==========================================")
|
| 1734 |
+
|
| 1735 |
+
if tests_passed != tests_total:
|
| 1736 |
+
sys.exit(1)
|
scripts/verify_template.py
CHANGED
|
@@ -4,7 +4,7 @@
|
|
| 4 |
python3 scripts/verify_template.py
|
| 5 |
|
| 6 |
Run this before publishing, and after any edit to chat_template.jinja. It fetches current
|
| 7 |
-
upstream (v22.
|
| 8 |
splice silently sits on top of a template we haven't looked at.
|
| 9 |
|
| 10 |
The checks exist because each of them has a real failure mode:
|
|
@@ -17,6 +17,7 @@ The checks exist because each of them has a real failure mode:
|
|
| 17 |
from __future__ import annotations
|
| 18 |
|
| 19 |
import difflib
|
|
|
|
| 20 |
import pathlib
|
| 21 |
import sys
|
| 22 |
import urllib.request
|
|
@@ -27,6 +28,7 @@ HERE = pathlib.Path(__file__).resolve().parent.parent
|
|
| 27 |
UPSTREAM = ("https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates/"
|
| 28 |
"resolve/main/chat_template.jinja")
|
| 29 |
MARKER = "Never: open with preamble"
|
|
|
|
| 30 |
|
| 31 |
fails: list[str] = []
|
| 32 |
|
|
@@ -46,22 +48,26 @@ def main() -> int:
|
|
| 46 |
full = (HERE / "chat_template.jinja").read_text()
|
| 47 |
mini = (HERE / "chat_template_oneline.txt").read_text()
|
| 48 |
|
| 49 |
-
print("=== diff vs upstream v22.
|
| 50 |
with urllib.request.urlopen(UPSTREAM, timeout=60) as r:
|
| 51 |
up = r.read().decode()
|
| 52 |
diff = [l for l in difflib.unified_diff(up.splitlines(), full.splitlines(), lineterm="")
|
| 53 |
if l.startswith(("+", "-")) and not l.startswith(("+++", "---"))]
|
| 54 |
-
# v22.
|
| 55 |
-
# thinking-on/off terseness-lead split), so it is NO LONGER a pure-insertion diff on v22.
|
| 56 |
# the fixes conditionalize a few upstream tool-call lines. We therefore assert INTENT, not an
|
| 57 |
# exact insertion count -- the append must be present, the fast-mode fixes must be in, and the
|
| 58 |
-
# thinking-ON path must remain byte-identical to "upstream v22.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 59 |
ins = [l for l in diff if l.startswith("+")]
|
| 60 |
check(len(ins) >= 11, f"terseness + fixes inserted (>=11 added lines, got {len(ins)})")
|
| 61 |
check("Answer directly and concisely." in full,
|
| 62 |
"fast-mode terseness lead present (v22.1.1 split)")
|
| 63 |
-
check('template_version = "qwen3.8-froggeric-v22.
|
| 64 |
-
check("qwen3.8-froggeric-v22.
|
| 65 |
check("Nail" not in full and "Dagger" not in full, "no model-specific identity in template")
|
| 66 |
|
| 67 |
print("\n=== rendering")
|
|
@@ -70,7 +76,7 @@ def main() -> int:
|
|
| 70 |
"with system prompt": [{"role": "system", "content": "Be a pirate."},
|
| 71 |
{"role": "user", "content": "hi"}],
|
| 72 |
"multi-turn w/ think": [{"role": "user", "content": "Q1"},
|
| 73 |
-
{"role": "assistant", "content": "<think>
|
| 74 |
{"role": "user", "content": "Q2"}],
|
| 75 |
}
|
| 76 |
for name, msgs in cases.items():
|
|
@@ -78,7 +84,10 @@ def main() -> int:
|
|
| 78 |
check(out.count(MARKER) == 1, f"{name}: terseness appears exactly once")
|
| 79 |
check("Be a pirate." in render(full, cases["with system prompt"]),
|
| 80 |
"user's own system prompt is preserved")
|
| 81 |
-
|
|
|
|
|
|
|
|
|
|
| 82 |
"prior thinking is retained across turns")
|
| 83 |
|
| 84 |
# v22.1 defaults reasoning_effort to MEDIUM, which injects no steering line -- so out-of-the-box
|
|
@@ -100,6 +109,27 @@ def main() -> int:
|
|
| 100 |
out = render(full, [{"role": "user", "content": "weather?"}], tools=tools)
|
| 101 |
check("get_weather" in out and out.count(MARKER) == 1, "tool definitions still render")
|
| 102 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 103 |
print("\n=== minified round-trip")
|
| 104 |
for name, msgs in cases.items():
|
| 105 |
check(render(full, msgs) == render(mini, msgs), f"{name}: oneline == full")
|
|
|
|
| 4 |
python3 scripts/verify_template.py
|
| 5 |
|
| 6 |
Run this before publishing, and after any edit to chat_template.jinja. It fetches current
|
| 7 |
+
upstream (v22.3) fresh, so it also catches the case where froggeric ships a new version and our
|
| 8 |
splice silently sits on top of a template we haven't looked at.
|
| 9 |
|
| 10 |
The checks exist because each of them has a real failure mode:
|
|
|
|
| 17 |
from __future__ import annotations
|
| 18 |
|
| 19 |
import difflib
|
| 20 |
+
import re
|
| 21 |
import pathlib
|
| 22 |
import sys
|
| 23 |
import urllib.request
|
|
|
|
| 28 |
UPSTREAM = ("https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates/"
|
| 29 |
"resolve/main/chat_template.jinja")
|
| 30 |
MARKER = "Never: open with preamble"
|
| 31 |
+
BASE = "qwen3.8-froggeric-v22.3" # the upstream release this fork is rebased onto
|
| 32 |
|
| 33 |
fails: list[str] = []
|
| 34 |
|
|
|
|
| 48 |
full = (HERE / "chat_template.jinja").read_text()
|
| 49 |
mini = (HERE / "chat_template_oneline.txt").read_text()
|
| 50 |
|
| 51 |
+
print("=== diff vs upstream v22.3")
|
| 52 |
with urllib.request.urlopen(UPSTREAM, timeout=60) as r:
|
| 53 |
up = r.read().decode()
|
| 54 |
diff = [l for l in difflib.unified_diff(up.splitlines(), full.splitlines(), lineterm="")
|
| 55 |
if l.startswith(("+", "-")) and not l.startswith(("+++", "---"))]
|
| 56 |
+
# v22.3.1 is froggeric v22.3 + the terseness append + the fast-mode fixes (Fix 3 + the
|
| 57 |
+
# thinking-on/off terseness-lead split), so it is NO LONGER a pure-insertion diff on v22.3:
|
| 58 |
# the fixes conditionalize a few upstream tool-call lines. We therefore assert INTENT, not an
|
| 59 |
# exact insertion count -- the append must be present, the fast-mode fixes must be in, and the
|
| 60 |
+
# thinking-ON path must remain byte-identical to "upstream v22.3 + the terseness append".
|
| 61 |
+
up_ver = re.search(r'template_version = "([^"]+)"', up)
|
| 62 |
+
up_ver = up_ver.group(1) if up_ver else "?"
|
| 63 |
+
check(up_ver == BASE, f"upstream is still the rebase base {BASE} (live: {up_ver})")
|
| 64 |
+
|
| 65 |
ins = [l for l in diff if l.startswith("+")]
|
| 66 |
check(len(ins) >= 11, f"terseness + fixes inserted (>=11 added lines, got {len(ins)})")
|
| 67 |
check("Answer directly and concisely." in full,
|
| 68 |
"fast-mode terseness lead present (v22.1.1 split)")
|
| 69 |
+
check('template_version = "qwen3.8-froggeric-v22.3.1"' in full, "version is v22.3.1")
|
| 70 |
+
check("qwen3.8-froggeric-v22.3" in full, "built on the froggeric v22.3 base")
|
| 71 |
check("Nail" not in full and "Dagger" not in full, "no model-specific identity in template")
|
| 72 |
|
| 73 |
print("\n=== rendering")
|
|
|
|
| 76 |
"with system prompt": [{"role": "system", "content": "Be a pirate."},
|
| 77 |
{"role": "user", "content": "hi"}],
|
| 78 |
"multi-turn w/ think": [{"role": "user", "content": "Q1"},
|
| 79 |
+
{"role": "assistant", "content": "<think>PRIORTHOUGHT</think>A1"},
|
| 80 |
{"role": "user", "content": "Q2"}],
|
| 81 |
}
|
| 82 |
for name, msgs in cases.items():
|
|
|
|
| 84 |
check(out.count(MARKER) == 1, f"{name}: terseness appears exactly once")
|
| 85 |
check("Be a pirate." in render(full, cases["with system prompt"]),
|
| 86 |
"user's own system prompt is preserved")
|
| 87 |
+
# v22.2+ extracts in-content reasoning into a canonical block instead of passing the
|
| 88 |
+
# tags through, so match the thought itself inside a think block, not a tag layout.
|
| 89 |
+
mt = render(full, cases["multi-turn w/ think"])
|
| 90 |
+
check(any("PRIORTHOUGHT" in b for b in re.findall(r"<think>(.*?)</think>", mt, re.S)),
|
| 91 |
"prior thinking is retained across turns")
|
| 92 |
|
| 93 |
# v22.1 defaults reasoning_effort to MEDIUM, which injects no steering line -- so out-of-the-box
|
|
|
|
| 109 |
out = render(full, [{"role": "user", "content": "weather?"}], tools=tools)
|
| 110 |
check("get_weather" in out and out.count(MARKER) == 1, "tool definitions still render")
|
| 111 |
|
| 112 |
+
# The fast-mode fixes: with thinking off, nothing in the prompt may ask for a <think> block
|
| 113 |
+
# the generation prompt has already closed. Assert both directions -- the instructions must
|
| 114 |
+
# vanish when thinking is off AND survive when it is on, so a future edit can't "fix" this by
|
| 115 |
+
# deleting them outright. Tool-call FORMAT rules are not part of the fix and must stay.
|
| 116 |
+
print("\n=== fast mode (thinking off)")
|
| 117 |
+
weather = [{"role": "user", "content": "weather?"}]
|
| 118 |
+
for fmt in ("xml", "json"):
|
| 119 |
+
fast = render(full, weather, tools=tools, enable_thinking=False, tool_call_format=fmt)
|
| 120 |
+
slow = render(full, weather, tools=tools, tool_call_format=fmt)
|
| 121 |
+
body = fast.split("<|im_start|>assistant")[0] # ignore the empty-think prefill
|
| 122 |
+
check("think" not in body.lower(), f"{fmt}: no thinking reference anywhere in fast prompt")
|
| 123 |
+
check("IMMEDIATELY, with NO conversational" in fast,
|
| 124 |
+
f"{fmt}: tool-call immediacy rule rephrased without 'after thinking'")
|
| 125 |
+
check("IMMEDIATELY after thinking" in slow,
|
| 126 |
+
f"{fmt}: thinking-on keeps the original 'after thinking' wording")
|
| 127 |
+
check("Brief explanation of tool call" in slow,
|
| 128 |
+
f"{fmt}: thinking-on keeps the <think> stanza")
|
| 129 |
+
check("Function calls MUST follow the specified format" in fast,
|
| 130 |
+
f"{fmt}: fast mode still carries the tool-call format rules")
|
| 131 |
+
check("Answer directly and concisely" in fast, f"{fmt}: fast-mode terseness lead")
|
| 132 |
+
|
| 133 |
print("\n=== minified round-trip")
|
| 134 |
for name, msgs in cases.items():
|
| 135 |
check(render(full, msgs) == render(mini, msgs), f"{name}: oneline == full")
|