Scarletwolf/rwkv7-g1-toolcalling-states

RWKV-7 G1 tool-calling states

Trained initial states for RWKV-7 G1, for real-world tool calling: pick the right function out of 231, fill its arguments, and stay silent when no tool applies.

A state is not a fine-tuned model. It is the recurrent state each layer starts from, 0.2% of the parameters, trained with every weight frozen. You load the public checkpoint once and inject the state on top, so the artifact is 11-17 MB instead of several gigabytes, and it hot-swaps.

State	Base checkpoint	Score	Cost to train
rwkv-state-72b.pth (17 MB)	rwkv7-g1g-7.2b-20260523-ctx8192	71/82	~$3
rwkv-state-v3.pth (11 MB)	rwkv7-g1g-2.9b-20260526-ctx8192	61/82	$0.53

Measured on a frozen 82-case judge over the 231 real tool schemas of a self-hosted assistant. Same scorer for every model in the tables below. For reference, on the identical 82 cases: the raw 7.2B checkpoint scores 29, and Qwen3-30B-A3B scores 58 with all 231 schemas in context, 71 with the same 40-tool shortlist these states get.

The 2.9B state is byte-reproducible: a full independent re-run produced the same SHA-256 and the same 164 generations, character for character.

Usage

Tool calling needs the native rwkv runtime, not GGUF — the GGUF format cannot carry a trained initial state.

import torch
from rwkv.model import RWKV
from rwkv.utils import PIPELINE

model = RWKV(model="/path/to/rwkv7-g1g-7.2b-20260523-ctx8192", strategy="cuda bf16")
pipe = PIPELINE(model, "rwkv_vocab_v20230424")
sd = torch.load("rwkv-state-72b.pth", map_location="cpu", weights_only=True)

def injected_state(n_layer=32):
    _, st = model.forward([0], None)
    for i in range(len(st)):
        st[i] = torch.zeros_like(st[i])
    for i in range(n_layer):
        ts = sd[f"blocks.{i}.att.time_state"].float()
        tgt = st[i * 3 + 1]
        st[i * 3 + 1] = ts.to(dtype=tgt.dtype, device=tgt.device).reshape(tgt.shape)
    return st

out, state = None, injected_state()
# then prefill your prompt and decode greedily

Greedy decoding, no repetition penalty. Two things matter more than they look:

Prompt the model in the format it was trained on. These states expect the official G1x tool catalog — {"name": ..., "description": ..., "arguments": {...}} per tool, not the OpenAI parameters/properties wrapper — with the assistant turn opening directly on a JSON fence. Swapping that priming for another one costs this state 9 points and half its abstention.

Keep the tool list compact. One line or one flat object per tool. The same state scores 70 with full JSON schemas (34k characters) and 71 with a compact catalog (11k), and the eval runs 3.7x faster. Dropping tool descriptions entirely, though, costs 9 points — the model needs to know what a tool does in order to decide not to call it.

What these states do and do not learn

Trained in one epoch, they install dispositions. Abstention goes from 0/17 to 17/17: the raw checkpoint never declines to act, the state-tuned one declines correctly every time. Output format discipline comes along with it.

They do not install lookups. 135 examples built precisely against 7 tool-name confusions (linear_search_issues vs github_search_issues and friends) fixed none of them; the wrong answers moved to other wrong answers. A fixed-size state does not hold a 231-entry directory.

That distinction is the useful part of this work, and it shows up from the opposite direction too: a distilled hybrid inherits its teacher's content capability and loses its abstention entirely.

Reproduce, or attack

Everything is public: the 82-case judge, the tool schemas, the raw generations of 25 measurements including the unflattering ones, the training data, verbatim pod recipes, a working GRPO loop for RWKV-7, and the SHA-256 of every artifact.

Benchmark: https://codeberg.org/scarletwolf_ai/rwkv-toolcaller-bench (mirror: https://git.scarletwolf.cloud/kevin/rwkv-toolcaller-bench)

Write-ups, in order: parity for $0.53 · the ceiling and the variable that breaks it · what you measure when you measure a model

Honest limits

One benchmark, our own tools, no replication on a public tool set yet. The comparison against Qwen3-30B is asymmetric: it was never trained on our data. The 7.2B run is not replicated end-to-end the way the 2.9B one is. Multi-step sits at 0/2 for every model we have measured, including the 30B, because the harness is one turn and one call.

Found a hole in the protocol? Get in touch. The last one arrived in a three-word Discord message.

Lab: ScarletWolf AI. Apache-2.0.