Cloudbusting commited on
Commit
6495113
·
verified ·
1 Parent(s): 3d2b14b

Fix chat template: render assistant tool_calls history

Browse files

Thanks for LFM2.5. The 1.2B is a remarkably capable tool caller for its size, and we have enjoyed working with it.

Defect: the shipped message loop renders `content` but never renders a past assistant turn's `tool_calls`. Standard OpenAI-compatible clients replay history as an assistant `tool_calls` array followed by `role: tool` results. The current template therefore emits an empty assistant turn, and the model cannot see the actions it already took.

Measured effect: in seeded 4-step and 6-step runs with the official Q4_K_M GGUF, temperature 0.1, top_k 50, and llama.cpp, the shipped template scored ordered 0/6 and exact 0/6. With history rendered, the fixed 4-step runs scored ordered 3/3 and exact 0/3 because the model inserted one unrequested note call each run. The fixed 6-step runs restored coherent history but still fabricated the final read, so they scored ordered 0/3 and exact 0/3. This patch restores memory. It does not claim to fix the model's extra call or fabricated final.

Fix: port the `format_arg_value` and tool-call rendering macros already shipped by LFM2.5-8B-A1B into this template's message loop. A past tool-call turn then renders in the model family's native form:

<|im_start|>assistant
<|tool_call_start|>[get_weather(city='Paris')]<|tool_call_end|><|im_end|>

The change also tolerates `content: null` or missing content on pure tool-call turns. Everything outside the tool-call path stays byte-identical. The included regression test covers the defect, fixed rendering, tool-free byte equality, multimodal content, null content, hostile argument values, and malformed call shapes against the pinned 8B-family behavior.

The live route doctor's verdict changes from a template-history failure to healthy with the fix alone. The strict exact suite can still fail on this small model's own extra call, and the evidence reports that rather than hiding it.

GGUF users receive the fix through a re-converted GGUF or a runtime override such as `llama-server --jinja --chat-template-file <this file>`. We used the runtime override for the live proof.

Happy to adjust formatting, macro placement, or null-content handling to match your conventions.

Evidence, per-run ledgers, wire excerpts, regression tests, and reproductions:
https://github.com/graphometer/droplet/tree/main/public/evidence

Disclosure: prepared with heavy AI assistance; every claim above comes from recorded runs that can be inspected. Not affiliated with, endorsed by, or connected to Liquid AI.

Files changed (1) hide show
  1. chat_template.jinja +31 -3
chat_template.jinja CHANGED
@@ -1,5 +1,27 @@
1
  {{- bos_token -}}
2
  {%- set keep_past_thinking = keep_past_thinking | default(false) -%}
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  {%- set ns = namespace(system_prompt="") -%}
4
  {%- if messages[0]["role"] == "system" -%}
5
  {%- set ns.system_prompt = messages[0]["content"] -%}
@@ -29,8 +51,10 @@
29
  {%- endfor -%}
30
  {%- for message in messages -%}
31
  {{- "<|im_start|>" + message["role"] + "\n" -}}
32
- {%- set content = message["content"] -%}
33
- {%- if content is not string -%}
 
 
34
  {%- set content = content | tojson -%}
35
  {%- endif -%}
36
  {%- if message["role"] == "assistant" and not keep_past_thinking and loop.index0 != ns.last_assistant_index -%}
@@ -38,7 +62,11 @@
38
  {%- set content = content.split("</think>")[-1] | trim -%}
39
  {%- endif -%}
40
  {%- endif -%}
41
- {{- content + "<|im_end|>\n" -}}
 
 
 
 
42
  {%- endfor -%}
43
  {%- if add_generation_prompt -%}
44
  {{- "<|im_start|>assistant\n" -}}
 
1
  {{- bos_token -}}
2
  {%- set keep_past_thinking = keep_past_thinking | default(false) -%}
3
+ {%- macro format_arg_value(arg_value) -%}
4
+ {%- if arg_value is string -%}
5
+ {{- "'" + arg_value + "'" -}}
6
+ {%- elif arg_value is mapping -%}
7
+ {{- arg_value | tojson -}}
8
+ {%- else -%}
9
+ {{- arg_value | string -}}
10
+ {%- endif -%}
11
+ {%- endmacro -%}
12
+ {%- macro render_tool_calls(tool_calls) -%}
13
+ {%- set tool_calls_ns = namespace(tool_calls=[]) -%}
14
+ {%- for tool_call in tool_calls -%}
15
+ {%- set func_name = tool_call["function"]["name"] -%}
16
+ {%- set func_args = tool_call["function"]["arguments"] -%}
17
+ {%- set args_ns = namespace(arg_strings=[]) -%}
18
+ {%- for arg_name, arg_value in func_args.items() -%}
19
+ {%- set args_ns.arg_strings = args_ns.arg_strings + [arg_name + "=" + format_arg_value(arg_value)] -%}
20
+ {%- endfor -%}
21
+ {%- set tool_calls_ns.tool_calls = tool_calls_ns.tool_calls + [func_name + "(" + (args_ns.arg_strings | join(", ")) + ")"] -%}
22
+ {%- endfor -%}
23
+ {{- "<|tool_call_start|>[" + (tool_calls_ns.tool_calls | join(", ")) + "]<|tool_call_end|>" -}}
24
+ {%- endmacro -%}
25
  {%- set ns = namespace(system_prompt="") -%}
26
  {%- if messages[0]["role"] == "system" -%}
27
  {%- set ns.system_prompt = messages[0]["content"] -%}
 
51
  {%- endfor -%}
52
  {%- for message in messages -%}
53
  {{- "<|im_start|>" + message["role"] + "\n" -}}
54
+ {%- set content = message.get("content") -%}
55
+ {%- if not content -%}
56
+ {%- set content = "" -%}
57
+ {%- elif content is not string -%}
58
  {%- set content = content | tojson -%}
59
  {%- endif -%}
60
  {%- if message["role"] == "assistant" and not keep_past_thinking and loop.index0 != ns.last_assistant_index -%}
 
62
  {%- set content = content.split("</think>")[-1] | trim -%}
63
  {%- endif -%}
64
  {%- endif -%}
65
+ {{- content -}}
66
+ {%- if message["role"] == "assistant" and message.get("tool_calls") -%}
67
+ {{- render_tool_calls(message["tool_calls"]) -}}
68
+ {%- endif -%}
69
+ {{- "<|im_end|>\n" -}}
70
  {%- endfor -%}
71
  {%- if add_generation_prompt -%}
72
  {{- "<|im_start|>assistant\n" -}}