soup_30: composite=89.45 — see model card for benchmark deltas vs v45

README.md

@@ -18,7 +18,7 @@ datasets:
 - juanquivilla/sotto-transcript-cleanup
 ---
 
-# SottoASR Transcript Cleanup — LFM2.5-350M (Full Precision, v55)
+# SottoASR Transcript Cleanup — LFM2.5-350M (Full Precision, soup_30)
 
 [sottoasr.app](https://sottoasr.app) · [MLX 5-bit (recommended)](https://huggingface.co/juanquivilla/sotto-cleanup-lfm25-350m-mlx-5bit) · [MLX 4-bit (smaller)](https://huggingface.co/juanquivilla/sotto-cleanup-lfm25-350m-mlx-4bit) · [Training Dataset](https://huggingface.co/datasets/juanquivilla/sotto-transcript-cleanup)
 
@@ -26,9 +26,9 @@ datasets:
 
 Full-precision bf16 fine-tune of [LiquidAI/LFM2.5-350M-Base](https://huggingface.co/LiquidAI/LFM2.5-350M-Base) for on-device speech-to-text transcript cleanup. This is the **training artifact** — for on-device deployment on Apple Silicon, use the [5-bit MLX variant](https://huggingface.co/juanquivilla/sotto-cleanup-lfm25-350m-mlx-5bit).
 
-## What's new in v55
+## What's new in soup_30
 
-v55 extends v45 with targeted training data covering five failure modes observed on real production transcripts:
+soup_30 extends v45 with targeted training data covering five failure modes observed on real production transcripts:
 
 - **Multi-number sentences** ("the score was 21 to 14")
 - **Year-with-context drift** ("twenty eighteen and twenty twenty seven" → preserve both years)
@@ -42,18 +42,16 @@ evaluation.
 
 ## Headline numbers
 
-Numbers below are at **`max_new_tokens=900`** to avoid mid-output truncation on long inputs (a 512-token cap inflates apparent deletion rate by ~3×). Use the same setting in production.
-
-| Capability | v36 | v45 | **v55 (this model)** |
+| Capability | v36 | v45 | **soup_30 (this model)** |
 |---|---:|---:|---:|
 | Number accuracy (171-sample stratified val) | 12.9% | 95.9% | **96.5%** |
 | 66-case adversarial benchmark (greedy) | n/a | 76% | **86%** |
-| 66-case adversarial benchmark (temp 0.7 × 4) | n/a | 77% | **83%** |
-| Loops on 264 sampling-mode probes | n/a | 0 | **1** |
-| Filler-free on 241 long inputs | 67.2% | 68.0% | **73.4%** |
-| Sub-deletion >15% on 241 long inputs (max=900) | 13.3% | 13.7% | **3.7%** |
+| 66-case adversarial benchmark (temp 0.7 × 4) | n/a | 77% | **85%** |
+| Loops on 264 sampling-mode probes | n/a | 0 | **2** |
+| Filler-free on 241 long inputs | 67.2% | 68.0% | **73.0%** |
+| Sub-deletion >15% on 241 long inputs | 13.3% | 13.7% | **4.6%** |
 
-Composite score (0.35×num + 0.30×bench_greedy + 0.15×bench_sample + 0.10×filler_long + 0.05×(1-sub15) + 0.05×(1-loops/N)): **89.32**
+Composite score (0.35×num + 0.30×bench_greedy + 0.15×bench_sample + 0.10×filler_long + 0.05×(1-sub15) + 0.05×(1-loops/N)): **89.45**
 
 ## Training pipeline
 
@@ -63,7 +61,7 @@ LiquidAI/LFM2.5-350M-Base
   → GRPO v36: full FT with substantive-deletion-aware reward
   → SFT v39: + 12.7K augmented number examples (ITN)
   → GRPO v40-v45: chained refinement, fixed reward + amplified filler penalty
-  → SFT/GRPO v55: + 5.5K gap-filling examples (multi-number, year-context,
+  → SFT/GRPO soup_30: + 5.5K gap-filling examples (multi-number, year-context,
                        disconnected, duplicates, long-form), Qwen-audited and cleaned
 ```
 
@@ -84,26 +82,13 @@ prompt = f"### Input:\n{text}\n\n### Output:\n"
 
 inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
 with torch.no_grad():
-    out = model.generate(
-        **inputs,
-        max_new_tokens=max(512, int(len(text.split()) * 1.5)),  # ≥1.5× input word count
-        do_sample=False,
-        repetition_penalty=1.05,                                # LFM2.5 official default
-    )
+    out = model.generate(**inputs, max_new_tokens=512, do_sample=False)
 output = tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
 if "###" in output:
     output = output[:output.index("###")]
 print(output.strip())
 ```
 
-### Inference recommendations
-
-Use these generation settings to match the production deployment:
-
-- **`repetition_penalty=1.05`** — LFM2.5's official default. **Critical** for long inputs: prevents the rare voicemail-style 5-gram loops that can occur with `repetition_penalty=1.0`.
-- **`max_new_tokens >= 1.5 × input_word_count`** (or 600 minimum) — long inputs (>200 words) need headroom; truncating mid-output looks like content deletion.
-- **`do_sample=False`** (greedy) for deterministic output. If sampling is needed, use `temperature=0.1, top_k=50`.
-
 ## All Variants
 
 | Variant | Size | Use Case |

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:901c6b1aae01a90a2831d05a2f867c9de392dfae8aa944202959cd838bfc4235
+oid sha256:6e96eeffdcdd60f881e13eb2019b339b39d1a74951446f062e7e641a82f6422e
 size 708984464