CalinR commited on
Commit
f0feac6
·
verified ·
1 Parent(s): 7d4c97e

Fix missing thinking channel in Gemma 4 chat template when using continue_final_message

Browse files

When `enable_thinking: false` is set, Gemma 4's chat template signals "no thinking" by inserting an empty thinking channel (<|channel>thought\n<channel|>) at the start of the model turn. However, this only happens in the `add_generation_prompt` code path. When using `continue_final_message: true` for assistant prefilling, the template skips this injection entirely, producing:
```
<|turn>model
[assistant prefill content]
```
Instead of the expected:
```
<|turn>model
<|channel>thought
<channel|>[assistant prefill content]
```
Without the empty thinking channel, the model is in a bad state and may produce thinking tokens in its response.

Solution
Added a conditional in the message loop to inject the empty thinking channel on the last model turn when thinking is disabled:
```
{%- if role == 'model' and not enable_thinking | default(false) and loop.last -%}
{{- '<|channel>thought\n<channel|>' -}}
{%- endif -%}
```

Files changed (1) hide show
  1. chat_template.jinja +5 -0
chat_template.jinja CHANGED
@@ -232,6 +232,11 @@
232
  {{- '<|turn>' + role + '\n' }}
233
  {%- endif -%}
234
 
 
 
 
 
 
235
  {#- Render reasoning/reasoning_content as thinking channel -#}
236
  {%- set thinking_text = message.get('reasoning') or message.get('reasoning_content') -%}
237
  {%- if thinking_text and loop.index0 > ns_turn.last_user_idx and message.get('tool_calls') -%}
 
232
  {{- '<|turn>' + role + '\n' }}
233
  {%- endif -%}
234
 
235
+ {#- Inject empty thinking channel for the last model turn when thinking is disabled -#}
236
+ {%- if role == 'model' and not enable_thinking | default(false) and loop.last -%}
237
+ {{- '<|channel>thought\n<channel|>' -}}
238
+ {%- endif -%}
239
+
240
  {#- Render reasoning/reasoning_content as thinking channel -#}
241
  {%- set thinking_text = message.get('reasoning') or message.get('reasoning_content') -%}
242
  {%- if thinking_text and loop.index0 > ns_turn.last_user_idx and message.get('tool_calls') -%}