sabeshbesh/uncle-rudy-ramble-sft-coreai

UncleRudy Ramble tool-caller — Qwen3-0.6B (Core AI / iOS)

A LoRA fine-tune of Qwen3-0.6B that turns one spoken line into exactly one to-do tool call, exported to Apple Core AI (.aimodel) for on-device inference on iOS 27+.

It backs the voice-capture feature in UncleRudy: you talk, Apple's speech-to-text transcribes, and this model decides whether that line creates, updates, or deletes a to-do — or is just chatter that should be ignored.

Results

Measured on 240 held-out records built from language pools the training data never saw, replayed through the real Core AI Swift runtime (not val loss).

	base Qwen3-0.6B (prompted)	this model
exact match (tool + every arg)	26.2%	90.0%
tool choice	53.8%	97.5%
title correct	11.2%	96.7%
due correct	7.7%	83.8%
target id correct	68.3%	96.9%
no_op recall (noise rejected)	97.4%	97.4%
false no_op (real to-dos dropped)	102	2
latency / generation (M-series, greedy)	278 ms	206 ms
prompt size	~676 tok	~219 tok

Per tool: create 87.6% · update 84.4% · delete 97.0% · no_op 97.4%.

Two things worth noting. It is faster, not slower — fine-tuning moves the schema into the weights, so the tool-definition block leaves the prompt (676 → 219 tokens) and every call prefills less. And the no_op tax disappeared: prompting the base model into a "do nothing" option cost it 102 false-positives on real to-dos; trained from data instead, that is 2, with identical noise rejection. That is the thing prompting could not buy at any price.

Schema

The model sees the current to-do list with 1-based integer ids and targets them by id — no fuzzy title matching.

System message:

You turn one spoken line into exactly one tool call for a to-do app. Reply with a single <tool_call> and nothing else.
- create_todo — the user wants a new task recorded.
- update_todo — change an existing task's title and/or due time. Target it by its [id].
- delete_todo — remove an existing task. Target it by its [id].
- no_op — the line is not a to-do instruction (greeting, question, chatter, or cut-off speech).
Titles are bare and imperative: drop openers and drop any date/time words, which go in `due` as ISO-8601. Omit `due` when no time was given.

Now: 2026-07-27T15:30 (Monday)
To-dos:
[1] Call mom
[2] Go on a run — due 2026-07-27T19:00

Empty list renders as To-dos: (none).

tool	args
create_todo	title, due?
update_todo	id, title?, due?
delete_todo	id
no_op	—

Output is a single <tool_call>{"name": …, "arguments": {…}}</tool_call>. due is ISO-8601 yyyy-MM-dd'T'HH:mm.

Send no tools block. The schema is in the weights; passing tool definitions re-inflates the prompt for nothing. Likewise do not pass enable_thinking — Qwen3's template injects <think></think> when you do, which the fine-tune never saw (measured 91.7% → 90.0%).

Usage

Swift, via coreai-models:

let bundle = try LanguageBundle(at: modelDirectory)
let engine = try await CoreAIRunner(from: bundle).makeInferenceEngine()
let tokenizer = try await bundle.loadTokenizer()

let ids = try tokenizer.applyChatTemplate(
    messages: [["role": "system", "content": systemPrompt],
               ["role": "user", "content": utterance]],
    tools: [],              // deliberately empty
    additionalContext: nil  // deliberately nil
)
let stream = try engine.generate(
    with: ids.map { Int32($0) },
    samplingConfiguration: .greedy,
    inferenceOptions: InferenceOptions(maxTokens: 64, includeLogits: false)
)

Known limits

• Only 5 to-dos are addressable. Training list lengths topped out at 5 and emitted ids 1–5. Rendering a longer list puts ids out of distribution, and a hallucinated in-range id silently targets the wrong row. Cap the window at 5.
• Relative date arithmetic is the weak spot — due is the only metric under 90%, and roughly half of all residual failures are off-by-one on relative days ("this weekend" → Friday, "next monday" → a week early). A 0.6B doing calendar math is the limitation. The fix is to have it emit the phrase and resolve it in application code; the model is excellent at spotting the time expression and only bad at converting it.
• Context is 1024. The exported graph carries 256/512/1024 KV shapes only.
• Occasionally wraps the JSON in a ```http fence instead of <tool_call>. Parse with a first-{-to-last-} fallback.

Training

4200 synthetic examples, 2 epochs, LoRA r=16 on all attention + MLP projections (10.1M trainable, 1.67%), completion-only loss masking. ~50 min on Apple silicon (MPS).

Data was composed so that labels are correct by construction: a language model supplied only the language (task phrasings, openers, chatter, real speech-to-text mis-hearings), while a script supplied the structure (which tool, which id, the resolved ISO date). Train and eval draw from disjoint pools. Speech-to-text noise is modelled from real observed substitutions ("need to" → "tried to", "I have to" → "how to", "PR" → "VR").

Files

qwen3_0_6b_ramble_sft_8bit.aimodel/   Core AI graph (8-bit weights)
tokenizer/                            Qwen3 tokenizer + chat template
metadata.json                         Core AI bundle manifest

8-bit quantized, ~615 MB total.