ewinregirgojr/MiniCPM5-1B-Agentic-Tooluse-QLoRA-v2

MiniCPM5-1B Agentic Tooluse QLoRA v2

Latest Status: July 2026 Nemotron Repair

This is the current best adapter release for the MiniCPM5-1B Agentic Tooluse family. It was repaired with a short Nemotron SFT continuation followed by DPO preference optimization, starting from the previously trained DPO adapter.

External Team-ACE/ToolACE evaluation, 300 cases:

Metric	Base MiniCPM5-1B	This adapter	Delta
parseable_rate	0.0133	0.9933	+0.9800
valid_name_rate	0.0133	0.9700	+0.9567
expected_name_rate	0.0133	0.9267	+0.9133
args_exact_rate	0.1500	0.6533	+0.5033
arg_key_overlap	0.0033	0.7517	+0.7484
no_schema_copy_rate	1.0000	1.0000	+0.0000
no_repetition_rate	0.9967	1.0000	+0.0033
stopped_cleanly_rate	0.0000	0.1500	+0.1500

Use this adapter for first-call XML tool selection. In production, stop generation immediately after the first complete </function> call.

PEFT QLoRA adapter for openbmb/MiniCPM5-1B, tuned for MiniCPM5-style XML tool calling and function calling.

This repository is the adapter-only release. It is useful when you want a small LoRA/QLoRA checkpoint that can be loaded on top of the base model with PEFT.

Model Family

Use case	Repository
Adapter-only PEFT/LoRA loading	This repo: ewinregirgojr/MiniCPM5-1B-Agentic-Tooluse-QLoRA-v2
Standalone merged fp16 model for Transformers or vLLM	ewinregirgojr/MiniCPM5-1B-Agentic-Tooluse-Merged-FP16
Quantized GGUF files for llama.cpp-compatible runtimes	ewinregirgojr/MiniCPM5-1B-Agentic-Tooluse-GGUF

Quick Answer

This is a MiniCPM5-1B tool-calling adapter for experiments where the model must choose a provided tool and emit a MiniCPM5 XML function call.

The strongest observed behavior is first-call tool selection. The model should be used with a tool-calling runtime that stops generation after the first completed function call.

Model Details

• Base model: openbmb/MiniCPM5-1B
• Release type: PEFT adapter
• Training method: QLoRA / LoRA adapter tuning
• Primary task: tool calling, function calling, XML function call generation
• Native output format: MiniCPM5 XML-style function calls
• Recommended runtime behavior: stop generation after the first completed </function>

Native Tool Format

MiniCPM5 renders tool calls as XML:

<function name="tool_name"><param name="param_name">value</param></function>

For best results, use:

tokenizer.apply_chat_template(messages, tools=tools, add_generation_prompt=True)

Avoid hand-written prompt formats unless you have verified they match the model's tokenizer chat template.

Intended Use

This adapter is intended for:

• research on small-model tool calling
• first-call function selection experiments
• local MiniCPM5 tool-use prototypes
• comparing base MiniCPM5 against a tool-use adapter
• loading with PEFT in Transformers workflows

This adapter is not intended as a fully autonomous production agent by itself.

Loading

Use the base tokenizer from openbmb/MiniCPM5-1B. The adapter contains tokenizer files for convenience, but the base tokenizer is the safest loading path.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "openbmb/MiniCPM5-1B"
adapter_id = "ewinregirgojr/MiniCPM5-1B-Agentic-Tooluse-QLoRA-v2"

tokenizer = AutoTokenizer.from_pretrained(base_model_id, trust_remote_code=True)
base = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    dtype=torch.float16,
    device_map="auto",
    trust_remote_code=True,
)
model = PeftModel.from_pretrained(base, adapter_id)
model.eval()

Recommended Inference

Use a strict system instruction when tools are provided:

When tools are provided and the user request can be satisfied by a tool, call exactly one matching function. Do not answer normally.

Decode with skip_special_tokens=False when evaluating or parsing MiniCPM5 XML output.

In deployment, stop generation after the first completed function call:

</function>

This is runtime-controlled tool-calling behavior. It should not be confused with unconstrained free-running multi-turn generation.

Evaluation

External ToolACE-Derived Sanity Eval

This is an external first-tool-call sanity eval built from Team-ACE/ToolACE, which was not part of the xLAM/Glaive training mixture.

This is not the official ToolACE leaderboard evaluator. It converts the first assistant function call into MiniCPM5 XML-style tool selection with decoy tools.

n=300, greedy decoding, runtime stops after first completed function call.

Metric	Base MiniCPM5-1B	Fine-tuned adapter	Delta
parseable_rate	0.3567	0.9967	+0.6400
valid_name_rate	0.3467	0.9400	+0.5933
expected_name_rate	0.3467	0.8400	+0.4933
no_schema_copy_rate	0.3567	0.9967	+0.6400
no_repetition_rate	1.0000	1.0000	+0.0000

Held-Out Source-Mix Eval

This evaluation used a held-out split from the same source mixture used for training. It is not an external benchmark.

n=300:

Metric	Score
parseable_rate	0.7800
valid_name_rate	0.7733
expected_name_rate	0.7633
no_schema_copy_rate	0.7800
no_repetition_rate	0.7800

Training Data

The adapter was trained from a mixture of:

• Salesforce/xlam-function-calling-60k
• madroid/glaive-function-calling-openai

The original training pipeline validated that examples contained valid tool calls whose names appeared in the provided tool list. A later audit found that this was not strict enough: some Glaive-style examples contain assistant text, tool responses, follow-up user turns, or final assistant answers after a tool call.

That means this release is best described as a first-call tool-selection adapter, not a clean-stop multi-turn agent model.

Known Limitations

• Adapter-only release; it must be loaded with openbmb/MiniCPM5-1B through PEFT/Transformers.
• Strongest validated use case is first-call XML tool selection, not unrestricted autonomous agent operation.
• External ToolACE eval improved strongly after the July 2026 Nemotron SFT+DPO repair, but this is still not an official ToolACE/BFCL leaderboard submission.
• Function-name accuracy is high on the sampled ToolACE eval, but near-miss names or casing differences can still happen on unseen tool libraries.
• Argument exactness improved but is not perfect; validate required arguments, argument types, and allowed enum values before executing tools.
• Clean stopping is still runtime-sensitive. The evaluated model avoids repetition well, but production code should still stop immediately after the first complete </function> block.
• Multi-turn tool execution should be controlled by the application loop: model emits a call, app executes the tool, app sends the result back as a new turn.

Ethical and Safety Notes

This model can emit tool calls. Any real system using it should validate tool names, validate arguments, enforce an allowlist, require user confirmation for sensitive actions, and execute tools in a sandboxed or permissioned environment.

Do not let generated tool calls directly trigger irreversible actions without external validation.

Search Keywords

MiniCPM5, MiniCPM5-1B, OpenBMB, PEFT adapter, LoRA adapter, QLoRA, agentic tool calling, function calling, API calling, XML tool call, tool-use LLM, vLLM MiniCPM, llama.cpp source model, GGUF source model, Transformers text generation, Unsloth-compatible lineage.

July 2026 Nemotron SFT+DPO Repair Update

This adapter-only PEFT/QLoRA release now reflects the latest repair pass from the MiniCPM5-1B Agentic Tooluse model family.

The repair continued from the previous DPO adapter and added a targeted Nemotron SFT + DPO pass focused on:

• first-call tool selection
• valid function-name prediction
• argument-key selection
• schema-copy avoidance
• repeated-call suppression
• XML <function ...>...</function> tool-call formatting

Training/eval artifacts:

• SFT continuation: ./minicpm5_tooluse_nemotron_sft_from_repair_dpo
• DPO continuation: ./minicpm5_tooluse_nemotron_dpo_from_repair_dpo
• Merged fp16 export: ./minicpm5_tooluse_nemotron_merged_fp16
• External eval: Team-ACE/ToolACE, 300 cases

Datasets used in the repair:

• nvidia/Nemotron-SFT-Agentic-v2
• nvidia/Nemotron-RL-Agentic-Function-Calling-Pivot-v1
• nvidia/Nemotron-RL-Agentic-Conversational-Tool-Use-Pivot-v1

Dataset inspection note: nvidia/Nemotron-SFT-Agentic-v2/data/tool_calling.jsonl contains 8,444 physical lines and one malformed JSONL line. The final training pipeline reads the file directly with hf_hub_download, parses JSONL line by line, and skips invalid rows. The pipeline also removes oversized source policy text from SFT prompts while preserving tool schemas and recent dialogue context.

Latest External ToolACE Evaluation

Metric	Base MiniCPM5-1B	Nemotron repaired adapter/model	Delta
parseable_rate	0.0133	0.9933	+0.9800
valid_name_rate	0.0133	0.9700	+0.9567
expected_name_rate	0.0133	0.9267	+0.9133
args_exact_rate	0.1500	0.6533	+0.5033
arg_key_overlap	0.0033	0.7517	+0.7484
no_schema_copy_rate	1.0000	1.0000	+0.0000
no_repetition_rate	0.9967	1.0000	+0.0033
stopped_cleanly_rate	0.0000	0.1500	+0.1500

The repaired model beats base on 7/8 external ToolACE metrics.

The strongest improvements are in parseability, valid tool-name selection, expected tool-name selection, and argument-key overlap. The remaining weak point is clean stopping, so production runtimes should stop generation immediately after the first complete </function> call.

Runtime Contract

Use this as a first-call tool-selection model:

1. Render tools in the prompt.
2. Generate one tool call.
3. Parse the first complete <function ...>...</function> block.
4. Stop at </function>.
5. Execute the tool outside the model.
6. Feed the tool result back as the next turn.

Do not let the model keep generating fake tool responses or synthetic user turns after a function call.

Current Files vs Previous Versions

This repository keeps old versions in Hugging Face commit history. Files removed from the current visible file list are not permanently erased; they can still be found by opening the repository History tab and selecting an older commit.

Current adapter weights are:

• adapter_model.safetensors
• adapter_config.json

Older pre-Nemotron eval note files were removed from the current file list because the current evaluation is now stored in:

• EVAL_RESULTS.md
• external_toolace_base_vs_nemotron_dpo_eval.json

The removed old note files are still recoverable from the repository commit history.

Current Status

This is the current adapter-only release after the July 2026 Nemotron SFT+DPO repair. Use it when you want the smaller PEFT adapter and will load openbmb/MiniCPM5-1B separately.

Current primary artifacts:

• adapter_model.safetensors
• adapter_config.json
• tokenizer/chat-template files for prompt compatibility
• EVAL_RESULTS.md
• external_toolace_base_vs_nemotron_dpo_eval.json

The previous pre-Nemotron eval notes were removed from the current file list to avoid confusing users. They remain recoverable from Hugging Face commit history.