# Tool calling with these APEX quants (reproducible)

granite's stock chat template can't emit tool calls; **use the `granite-3.1-tools.jinja` in this repo.**
Below is a full round-trip you can paste and run — every setting included.

## Server
```bash
llama-server --jinja --chat-template-file granite-3.1-tools.jinja --ctx-size 8192 -fa on \
  --temp 0.6 --top-p 0.9 --repeat-penalty 1.05 --model <this-quant>.gguf --host 127.0.0.1 --port 8090
```

## System prompt
```
You are a helpful assistant with access to tools. When a tool is needed, call it; when it returns a result, answer the user directly using that result.
```

## Turn 1 — request (user + tool schema)
```json
{
    "messages": [
        {
            "role": "system",
            "content": "You are a helpful assistant with access to tools. When a tool is needed, call it; when it returns a result, answer the user directly using that result."
        },
        {
            "role": "user",
            "content": "What is the weather in Paris right now? Use celsius."
        }
    ],
    "tools": [
        {
            "type": "function",
            "function": {
                "name": "get_weather",
                "description": "Get the current weather for a location.",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "location": {
                            "type": "string",
                            "description": "City name"
                        },
                        "unit": {
                            "type": "string",
                            "enum": [
                                "celsius",
                                "fahrenheit"
                            ]
                        }
                    },
                    "required": [
                        "location"
                    ]
                }
            }
        }
    ],
    "temperature": 0.6,
    "top_p": 0.9
}
```
## Turn 1 — model response (the tool call)
```json
{
    "choices": [
        {
            "finish_reason": "tool_calls",
            "index": 0,
            "message": {
                "role": "assistant",
                "content": "",
                "tool_calls": [
                    {
                        "type": "function",
                        "function": {
                            "name": "get_weather",
                            "arguments": "{\"location\": \"Paris\", \"unit\": \"celsius\"}"
                        },
                        "id": "eZB0TTBwOvxvL09vmJBizhJcTrGtTBPj"
                    }
                ]
            }
        }
    ],
    "created": 1785251947,
    "model": "granite-3.1-1b-a400m-instruct-APEX-quality.gguf",
    "system_fingerprint": "b1215-3fc4e105",
    "object": "chat.completion",
    "usage": {
        "completion_tokens": 33,
        "prompt_tokens": 197,
        "total_tokens": 230,
        "prompt_tokens_details": {
            "cached_tokens": 0
        }
    },
    "id": "chatcmpl-RESswmFpvUBlJHKzj8KhGIwxnKs5mYBJ",
    "timings": {
        "cache_n": 0,
        "prompt_n": 197,
        "prompt_ms": 185.403,
        "prompt_per_token_ms": 0.9411319796954314,
        "prompt_per_second": 1062.5502284213308,
        "predicted_n": 33,
        "predicted_ms": 88.299,
        "predicted_per_token_ms": 2.675727272727273,
        "predicted_per_second": 373.7301668195563
    }
}
```

## Turn 2 — feed the tool result back
```json
{
    "messages": [
        {
            "role": "system",
            "content": "You are a helpful assistant with access to tools. When a tool is needed, call it; when it returns a result, answer the user directly using that result."
        },
        {
            "role": "user",
            "content": "What is the weather in Paris right now? Use celsius."
        },
        {
            "role": "assistant",
            "content": "",
            "tool_calls": [
                {
                    "type": "function",
                    "function": {
                        "name": "get_weather",
                        "arguments": "{\"location\": \"Paris\", \"unit\": \"celsius\"}"
                    },
                    "id": "eZB0TTBwOvxvL09vmJBizhJcTrGtTBPj"
                }
            ]
        },
        {
            "role": "tool",
            "name": "get_weather",
            "content": "{\"location\":\"Paris\",\"temp_c\":14,\"conditions\":\"light rain\"}"
        }
    ]
}
```
## Turn 2 — model final answer
```json
{
    "choices": [
        {
            "finish_reason": "stop",
            "index": 0,
            "message": {
                "role": "assistant",
                "content": "The weather in Paris right now is light rain."
            }
        }
    ],
    "created": 1785251948,
    "model": "granite-3.1-1b-a400m-instruct-APEX-quality.gguf",
    "system_fingerprint": "b1215-3fc4e105",
    "object": "chat.completion",
    "usage": {
        "completion_tokens": 12,
        "prompt_tokens": 123,
        "total_tokens": 135,
        "prompt_tokens_details": {
            "cached_tokens": 39
        }
    },
    "id": "chatcmpl-jQgducSKaag4fO9sQpxX7xbjy3w8lqAJ",
    "timings": {
        "cache_n": 39,
        "prompt_n": 84,
        "prompt_ms": 13.654,
        "prompt_per_token_ms": 0.16254761904761905,
        "prompt_per_second": 6152.043357257946,
        "predicted_n": 12,
        "predicted_ms": 29.859,
        "predicted_per_token_ms": 2.4882500000000003,
        "predicted_per_second": 401.88887772530893
    }
}
```