Instructions to use LiquidAI/LFM2.5-1.2B-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use LiquidAI/LFM2.5-1.2B-Instruct with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="LiquidAI/LFM2.5-1.2B-Instruct") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("LiquidAI/LFM2.5-1.2B-Instruct") model = AutoModelForCausalLM.from_pretrained("LiquidAI/LFM2.5-1.2B-Instruct", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use LiquidAI/LFM2.5-1.2B-Instruct with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "LiquidAI/LFM2.5-1.2B-Instruct" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "LiquidAI/LFM2.5-1.2B-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/LiquidAI/LFM2.5-1.2B-Instruct
- SGLang
How to use LiquidAI/LFM2.5-1.2B-Instruct with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "LiquidAI/LFM2.5-1.2B-Instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "LiquidAI/LFM2.5-1.2B-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "LiquidAI/LFM2.5-1.2B-Instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "LiquidAI/LFM2.5-1.2B-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use LiquidAI/LFM2.5-1.2B-Instruct with Docker Model Runner:
docker model run hf.co/LiquidAI/LFM2.5-1.2B-Instruct
Fix chat template: render assistant tool_calls history
Browse filesThanks for LFM2.5. The 1.2B is a remarkably capable tool caller for its size, and we have enjoyed working with it.
Defect: the shipped message loop renders `content` but never renders a past assistant turn's `tool_calls`. Standard OpenAI-compatible clients replay history as an assistant `tool_calls` array followed by `role: tool` results. The current template therefore emits an empty assistant turn, and the model cannot see the actions it already took.
Measured effect: in seeded 4-step and 6-step runs with the official Q4_K_M GGUF, temperature 0.1, top_k 50, and llama.cpp, the shipped template scored ordered 0/6 and exact 0/6. With history rendered, the fixed 4-step runs scored ordered 3/3 and exact 0/3 because the model inserted one unrequested note call each run. The fixed 6-step runs restored coherent history but still fabricated the final read, so they scored ordered 0/3 and exact 0/3. This patch restores memory. It does not claim to fix the model's extra call or fabricated final.
Fix: port the `format_arg_value` and tool-call rendering macros already shipped by LFM2.5-8B-A1B into this template's message loop. A past tool-call turn then renders in the model family's native form:
<|im_start|>assistant
<|tool_call_start|>[get_weather(city='Paris')]<|tool_call_end|><|im_end|>
The change also tolerates `content: null` or missing content on pure tool-call turns. Everything outside the tool-call path stays byte-identical. The included regression test covers the defect, fixed rendering, tool-free byte equality, multimodal content, null content, hostile argument values, and malformed call shapes against the pinned 8B-family behavior.
The live route doctor's verdict changes from a template-history failure to healthy with the fix alone. The strict exact suite can still fail on this small model's own extra call, and the evidence reports that rather than hiding it.
GGUF users receive the fix through a re-converted GGUF or a runtime override such as `llama-server --jinja --chat-template-file <this file>`. We used the runtime override for the live proof.
Happy to adjust formatting, macro placement, or null-content handling to match your conventions.
Evidence, per-run ledgers, wire excerpts, regression tests, and reproductions:
https://github.com/graphometer/droplet/tree/main/public/evidence
Disclosure: prepared with heavy AI assistance; every claim above comes from recorded runs that can be inspected. Not affiliated with, endorsed by, or connected to Liquid AI.
- chat_template.jinja +31 -3
|
@@ -1,5 +1,27 @@
|
|
| 1 |
{{- bos_token -}}
|
| 2 |
{%- set keep_past_thinking = keep_past_thinking | default(false) -%}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
{%- set ns = namespace(system_prompt="") -%}
|
| 4 |
{%- if messages[0]["role"] == "system" -%}
|
| 5 |
{%- set ns.system_prompt = messages[0]["content"] -%}
|
|
@@ -29,8 +51,10 @@
|
|
| 29 |
{%- endfor -%}
|
| 30 |
{%- for message in messages -%}
|
| 31 |
{{- "<|im_start|>" + message["role"] + "\n" -}}
|
| 32 |
-
{%- set content = message
|
| 33 |
-
{%- if
|
|
|
|
|
|
|
| 34 |
{%- set content = content | tojson -%}
|
| 35 |
{%- endif -%}
|
| 36 |
{%- if message["role"] == "assistant" and not keep_past_thinking and loop.index0 != ns.last_assistant_index -%}
|
|
@@ -38,7 +62,11 @@
|
|
| 38 |
{%- set content = content.split("</think>")[-1] | trim -%}
|
| 39 |
{%- endif -%}
|
| 40 |
{%- endif -%}
|
| 41 |
-
{{- content
|
|
|
|
|
|
|
|
|
|
|
|
|
| 42 |
{%- endfor -%}
|
| 43 |
{%- if add_generation_prompt -%}
|
| 44 |
{{- "<|im_start|>assistant\n" -}}
|
|
|
|
| 1 |
{{- bos_token -}}
|
| 2 |
{%- set keep_past_thinking = keep_past_thinking | default(false) -%}
|
| 3 |
+
{%- macro format_arg_value(arg_value) -%}
|
| 4 |
+
{%- if arg_value is string -%}
|
| 5 |
+
{{- "'" + arg_value + "'" -}}
|
| 6 |
+
{%- elif arg_value is mapping -%}
|
| 7 |
+
{{- arg_value | tojson -}}
|
| 8 |
+
{%- else -%}
|
| 9 |
+
{{- arg_value | string -}}
|
| 10 |
+
{%- endif -%}
|
| 11 |
+
{%- endmacro -%}
|
| 12 |
+
{%- macro render_tool_calls(tool_calls) -%}
|
| 13 |
+
{%- set tool_calls_ns = namespace(tool_calls=[]) -%}
|
| 14 |
+
{%- for tool_call in tool_calls -%}
|
| 15 |
+
{%- set func_name = tool_call["function"]["name"] -%}
|
| 16 |
+
{%- set func_args = tool_call["function"]["arguments"] -%}
|
| 17 |
+
{%- set args_ns = namespace(arg_strings=[]) -%}
|
| 18 |
+
{%- for arg_name, arg_value in func_args.items() -%}
|
| 19 |
+
{%- set args_ns.arg_strings = args_ns.arg_strings + [arg_name + "=" + format_arg_value(arg_value)] -%}
|
| 20 |
+
{%- endfor -%}
|
| 21 |
+
{%- set tool_calls_ns.tool_calls = tool_calls_ns.tool_calls + [func_name + "(" + (args_ns.arg_strings | join(", ")) + ")"] -%}
|
| 22 |
+
{%- endfor -%}
|
| 23 |
+
{{- "<|tool_call_start|>[" + (tool_calls_ns.tool_calls | join(", ")) + "]<|tool_call_end|>" -}}
|
| 24 |
+
{%- endmacro -%}
|
| 25 |
{%- set ns = namespace(system_prompt="") -%}
|
| 26 |
{%- if messages[0]["role"] == "system" -%}
|
| 27 |
{%- set ns.system_prompt = messages[0]["content"] -%}
|
|
|
|
| 51 |
{%- endfor -%}
|
| 52 |
{%- for message in messages -%}
|
| 53 |
{{- "<|im_start|>" + message["role"] + "\n" -}}
|
| 54 |
+
{%- set content = message.get("content") -%}
|
| 55 |
+
{%- if not content -%}
|
| 56 |
+
{%- set content = "" -%}
|
| 57 |
+
{%- elif content is not string -%}
|
| 58 |
{%- set content = content | tojson -%}
|
| 59 |
{%- endif -%}
|
| 60 |
{%- if message["role"] == "assistant" and not keep_past_thinking and loop.index0 != ns.last_assistant_index -%}
|
|
|
|
| 62 |
{%- set content = content.split("</think>")[-1] | trim -%}
|
| 63 |
{%- endif -%}
|
| 64 |
{%- endif -%}
|
| 65 |
+
{{- content -}}
|
| 66 |
+
{%- if message["role"] == "assistant" and message.get("tool_calls") -%}
|
| 67 |
+
{{- render_tool_calls(message["tool_calls"]) -}}
|
| 68 |
+
{%- endif -%}
|
| 69 |
+
{{- "<|im_end|>\n" -}}
|
| 70 |
{%- endfor -%}
|
| 71 |
{%- if add_generation_prompt -%}
|
| 72 |
{{- "<|im_start|>assistant\n" -}}
|