suture-8b

PEFT LoRA adapter for policy-issuance QC: two page images in (underwriting binder, issued policy), structured JSON out (PASS/FLAG + discrepancy objects). This is a closed synthetic task, not a general VL model and not a claim that the 8B beats GPT-5 on insurance.

• Base: Qwen/Qwen3-VL-8B-Instruct (Apache-2.0)
• Adapter: QLoRA r=32, α=64, dropout 0.05, 4-bit train, 384px thumbs, thinking off
• Checkpoint name in this project: /checkpoints/sft-limithi/final (2026-08-21)
• Code + scorer: caiotheodoro/suture (9e9e476 and later)
• Do not use this for live carrier documents. Train and eval data are forge-rendered ACORD-style pages, not real submissions.

Held-out result (the number)

Seed-777, n=1000, split-line limit/deductible render (eval re-renders from structured fields). Same scorer as the repo. Zero train overlap with 777.

Model	Severity-w. recall	HIGH recall	Precision	Parse
this adapter (sft-limithi)	0.839	0.893	0.870	1.0
prior published (sft-restem on same render)	0.634	0.795	0.704	1.0
GPT-5.6 Luna zero-shot vision	0.373	0.388	0.344	0.972
8B base, no adapter	0.098	0.115	0.278	0.973

Contract in CONTRACTS.md / README is still >0.95 recall, >0.95 precision, 1.00 HIGH. This release is a satisfactory baseline that proves numeric reading, not the product bar.

Weak classes on 777: DEDUCTIBLE_LOWER 0.494, DEDUCTIBLE_HIGHER 0.507 (precision 0.407), LIMIT_HIGHER 0.716. Citation exact-match ≈ 1.0 is gold-string cloning, not a quality win. Full tables: docs/BENCHMARK.md.

Older restem 777 0.622 used a cramped one-line limit X / ded Y PNG. Compare restem-on-new-render (0.634), not that row, when judging this adapter.

Intended use

• Research / reproduction of the Suture benchmark (binder vs policy discrepancy listing).
• Starting point for further SFT on the same forge generator (next: deductible direction / FPs).

Out of scope: production underwriting, real PDFs, OCR pipelines, “beats frontier on insurance” marketing.

How to load

from peft import PeftModel
from transformers import AutoProcessor, Qwen3VLForConditionalGeneration

base_id = "Qwen/Qwen3-VL-8B-Instruct"
adapter_id = "caiotheodoro/suture-8b"

model = Qwen3VLForConditionalGeneration.from_pretrained(base_id, device_map="auto")
model = PeftModel.from_pretrained(model, adapter_id)
processor = AutoProcessor.from_pretrained(base_id)

Two images (binder page, then policy page). Output schema is in the repo CONTRACTS.md. Inference in this project used the Modal eval path in cloud/modal_eval.py (4-bit, 384px).

Training data

Synthetic only. Generator + oracle: suture_forge in the GitHub repo. Seeds used in the published chain: 7 (train/val), 11 (distill pool), 13 (numeric mix), 17 (LIMIT_HIGHER mix). Seed 777 is eval-only and was never in train. JSONL is gitignored on purpose; regenerate from seeds.

Chain: sft-vl → sft-distill → sft-restem → sft-numeric → sft-limithi. Continue-from-adapter SFT, 1 epoch per mix, Modal L4.

This Hub repo does not include training JSONL or a merged 16-bit 8B.

Limitations

• 8B QLoRA at 384px on synthetic pages. HIGH recall is 0.893, not 1.00.
• Deductible higher/lower is still ~coin-flip on 777; DEDUCTIBLE_HIGHER false positives drag precision.
• Luna (0.37) is a weaker type teacher than gold JSON; this adapter lists gold types, it does not copy Luna style.

Carbon / hardware

Modal L4 24GB, 4-bit QLoRA. This adapter: continue sft-numeric → sft-limithi, 680 steps. Earlier chain steps are documented in docs/DECISIONS.md.