Broken Tool Calling?
Tool calling doesn't seem to work properly, maybe there is a template issue? Has anyone got this working?
Hey @And1mon I personnaly have no issue with thinking, it is reliably working. However, I can also observe no tool calling is working, at least with the llama-server webui (for this test, I used --tools all and tried to make it use get_datetime) or openwebui (trying to use get_current_timestamp).
In both case, it correctly sees the list of tools available, but can't call them correctly and output for example name="get_current_timestamp">
`
Example:
[
{
"type": "reasoning",
"id": "r_817333d3d4a7488e8b79a7ae",
"status": "completed",
"start_tag": "<think>",
"end_tag": "</think>",
"attributes": {
"type": "reasoning_content"
},
"content": [
{
"type": "output_text",
"text": "Okay, the user is asking me to demonstrate the get_current_timestamp tool. First, I need to check if I have the right function. Looking back at the tools provided, there's get_current_timestamp with no parameters, and calculate_timestamp with parameters days_ago, weeks_ago, etc.\n\nThe user wants to demonstrate using get_current_timestamp, so I should call that function. The parameters are all null in get_current_timestamp, but the description says it gets the current Unix timestamp in seconds. So I need to make sure to call it without any arguments.\n\nWait, the parameters for get_current_timestamp are listed as {\"properties\": {}, \"type\": \"object\"} so no parameters. So the tool call should be name \"get_current_timestamp\" with empty arguments.\n\nThe user said \"Can you use get_current_timestamp to demonstrate?\" so I should generate a tool call for that function. Since the instructions say to return the tool call in JSON within tool_call tags.\n\nI need to make sure I don't miss any steps. The user might want to see the actual timestamp, but since they didn't provide a time range, maybe the tool is called correctly by default. The calculate_timestamp tool can also be used, but the user is asking specifically about demonstrating get_current_timestamp.\n\nSo I should output the tool call for get_current_timestamp. Let me check the example structure. The example shows \"function name arguments\". Since get_current_timestamp has no parameters, the arguments should be empty object.\n\nSo the correct tool call is {\"name\": \"get_current_timestamp\", \"arguments\": {}}\n\nWait, in the parameters, the get_current_timestamp has \"properties\": {}, \"type\": \"object\", so no parameters. So the arguments should be an empty object.\n\nI should format the tool call as:\n\n{\"name\": \"get_current_timestamp\", \"arguments\": {}}\n\nYes, that's correct.\n\nI need to make sure not to call any other tools. The user only asked about demonstrating get_current_timestamp, so only that function.\n\nSo the response should be to generate that tool call.\n"
}
],
"summary": null,
"started_at": 1779800689.0132089,
"ended_at": 1779800696.767792,
"duration": 7
},
{
"type": "message",
"id": "msg_224367e09dd3464ea549ed1b",
"status": "completed",
"role": "assistant",
"content": [
{
"type": "output_text",
"text": "name=\"get_current_timestamp\">"
}
]
}
]
Yeah the thinking issue was user error on my side, however tool calling appears broken. I think it outputs a tool calling format llama.cpp cannot interpret.
Tool Calling
For tool / function calling, SGLang is the recommended backend. MiniCPM5-1B emits XML-style tool calls and SGLang's built-in minicpm5 parser converts them to OpenAI-compatible tool_calls natively:
python -m sglang.launch_server --model-path openbmb/MiniCPM5-1B --port 30000
--tool-call-parser minicpm5 # or: --tool-call-parser auto