lewisdog/lfm2.5-230m-cogs-ingest-lora

lfm2.5-230m-cogs-ingest

A 230M ingest student for the Cogitarium wiki pipeline: distils one raw captured document into structured JSON (extract / suggest_links / contradiction / page_update).

Serving pins (important)

• temperature 0, repeat_penalty 1.0 (NO penalty). A repeat penalty produces schema-valid but empty/degenerate JSON on extraction at this size — the extract task copies input tokens and a penalty starves it.
• Cap max_new_tokens and/or constrain with a JSON grammar: the extract task can run away under pure greedy (well-formed but unterminated JSON). This is a decoding artifact, not a format defect — do NOT reach for a repeat penalty to fix it.

Sanity eval (5 samples / 4 task types, temp 0, rep 1.0)

quant	strict JSON / keys	note
F16 / Q8_0	4/5	suggest_links, page_update, contradiction perfect; extract may not terminate under greedy
Q4_K_M	3/5	quant cliff — page_update loses the section_md key

Recommended quant: Q8_0 (233 MB, 723 tok/s on GB10). Q4_K_M only where size dominates and page_update is not used. Training matched the Qwen3-1.7B token accuracy (0.752 vs 0.756) at 7x fewer params.

This repo

LoRA adapter only (r=16, alpha=32, targets = attn + hybrid-conv/MLP proj). Load on top of the base LiquidAI/LFM2.5-230M with PEFT, or use the pre-merged repo lfm2.5-230m-cogs-ingest.

Base model & license

Fine-tuned from LiquidAI/LFM2.5-230M. Use is governed by the LFM Open License v1.0 (lfm1.0) — see the LICENSE in the base repo. This derivative complies with and inherits those terms; attribution to LiquidAI is retained above.

Provenance

LoRA SFT (TRL) on the Cogitarium distillation datasets, DGX Spark (GB10). Full methodology, loss curves, eval harnesses and per-quant results: see the project RESULTS.md. This is the "fast/small tier" of the Cogitarium model picker; the Qwen3-1.7B students remain the quality tier.