Revise model card

README.md

@@ -27,10 +27,10 @@ language:
 # Nemotron-3.5-30B-A3B-Antislop-FTPO
 
 A merged, ready-to-serve BF16 checkpoint of NVIDIA's Nemotron 3.5 30B-A3B, tuned with
-[Antislop](https://github.com/sam-paech/auto-antislop) / FTPO to suppress the overused
+[Antislop](https://github.com/sam-paech/auto-antislop) and FTPO to suppress the overused
 lexical patterns that make LLM prose recognisable as LLM prose.
 
-**~66% of overused patterns suppressed, with no statistically detectable loss of writing
+**66.4% of overused patterns suppressed, with no statistically detectable loss of writing
 quality and no capability regression.**
 
 The LoRA adapter that produced this checkpoint is released separately at
@@ -38,7 +38,7 @@ The LoRA adapter that produced this checkpoint is released separately at
 
 ## What this is
 
-Language models overuse a characteristic vocabulary — *shivers down the spine*, *testament
+Language models overuse a characteristic vocabulary. *Shivers down the spine*, *testament
 to*, *barely above a whisper*, and a long tail of names like Elara and Kael. Antislop
 identifies those patterns by comparing model output against a human-written baseline, then
 uses a backtracking sampler to catch the model reaching for one and record what it should
@@ -47,47 +47,32 @@ Preference Optimization)** trains the preference into the weights so it persists
 sampler switched off.
 
 FTPO adjusts only the specific token choices that need adjusting, holding the rest of the
-vocabulary steady. That precision is why capabilities survive.
-
-## Base checkpoint — please read
-
-This model was trained from **`nvidia_nemotron_3_5_nano_ea2_07202026_vv0.1`**, the EA-0720
-early-access intermediate checkpoint (Nemotron 3.5 Nano 30B-A3B BF16, intermediate
-checkpoint 2, produced after an initial stage of RL post-training).
-
-That checkpoint is **architecturally identical** to the now-public
-[`nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16`](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16)
-— same 52-layer Mamba-2/MoE/attention block pattern, same MTP layers, byte-identical
-tokenizer — but it is **not weight-identical**. It is a pre-release RL checkpoint, not the
-final Lightning release. Benchmark numbers below are all measured against the EA-0720
-checkpoint as baseline, not against public Lightning.
-
-The `base_model` tag points at Lightning because that is the closest public ancestor.
+vocabulary steady. That precision is what keeps capabilities intact.
 
 ## Results
 
-Evaluated on 400 held-out Reddit writing prompts (indices 1000–1399; the pipeline trained
-on 0–999), identical sampling for both models (temp 1.0, top_p 1.0, top_k 50, min_p 0.01),
-**Antislop sampler off** — so these numbers reflect what training changed in the weights,
-not what a sampler suppresses at inference.
+Evaluated on 400 held-out Reddit writing prompts (indices 1000 to 1399, with the pipeline
+trained on 0 to 999), identical sampling for both models (temp 1.0, top_p 1.0, top_k 50,
+min_p 0.01), **Antislop sampler off**. These numbers reflect what training changed in the
+weights, not what a sampler suppresses at inference.
 
 | Metric | Baseline | FTPO | Δ | Significant? |
 |---|---|---|---|---|
-| **Banlist suppression (prose only)** | 0% | **66.41%** | +66.41 | — |
-| Banlist suppression (all outputs) | 0% | 58.35% | +58.35 | — |
-| MMLU (600 q, thinking on) | 0.8383 | **0.8433** | +0.0050 | — |
-| GSM8K (250 q, thinking on) | 0.9240 | **0.9360** | +0.0120 | — |
-| Writing quality (0–100, claude-opus-5 judge, n=150 paired) | 54.30 | 53.34 | −0.96 | no (CI −2.36…+0.44) |
-| Lexical diversity (index, baseline=100) | 100.00 | 98.44 | −1.56 | — |
+| **Banlist suppression (prose only)** | 0% | **66.41%** | +66.41 | |
+| Banlist suppression (all outputs) | 0% | 58.35% | +58.35 | |
+| MMLU (600 q, thinking on) | 0.8383 | **0.8433** | +0.0050 | |
+| GSM8K (250 q, thinking on) | 0.9240 | **0.9360** | +0.0120 | |
+| Writing quality (0 to 100, claude-opus-5 judge, n=150 paired) | 54.30 | 53.34 | −0.96 | no (CI −2.36 to +0.44) |
+| Lexical diversity (index, baseline=100) | 100.00 | 98.44 | −1.56 | |
 | Refusal rate on writing prompts | 15.25% | 20.00% | +4.75pp | no (McNemar p=0.073) |
 
-Lexical diversity at 98.4% sits inside the 95–102% band the Antislop paper reports for
-FTPO, and well clear of the 74–92% collapse it measures for DPO.
+Lexical diversity at 98.4% sits inside the 95 to 102% band the Antislop paper reports for
+FTPO, and well clear of the 74 to 92% collapse it measures for DPO.
 
 ### Agentic and reasoning benchmarks
 
 Single H200, vLLM nightly, tool calling via the `qwen3_xml` parser, greedy decoding,
-identical harnesses. τ-bench user simulator was `claude-sonnet-5` for both models.
+identical harnesses. The τ-bench user simulator was `claude-sonnet-5` for both models.
 
 | Benchmark | n | Baseline | FTPO | Δ | Verdict |
 |---|---|---|---|---|---|
@@ -102,36 +87,6 @@ identical harnesses. τ-bench user simulator was `claude-sonnet-5` for both mode
 No benchmark difference is statistically distinguishable from zero. Every confidence
 interval crosses zero.
 
-## Limitations — the honest list
-
-These are real and we would rather you hear them from us.
-
-- **Consistency of tense/pronoun/POV regressed significantly.** Of six writing-quality
-  criteria, exactly one moved: −0.67 on a 0–20 scale, 95% CI [−1.11, −0.23]. This is a
-  plausible mechanism rather than a fluke — FTPO adjusts final-token logits, and
-  pronoun/tense agreement is precisely a final-token decision.
-- **τ-bench pass^k points down.** Average reward is flat, but pass^k (fraction of tasks
-  where *all* k trials succeed) fell from 0.539 to 0.504 on retail and 0.340 to 0.260 on
-  airline, widening with k and pointing the same way in both domains. There is no
-  significance test for it — it is a derived statistic over the same trials — so treat it
-  as directional. The plain reading: about as capable on any single attempt, marginally
-  less repeatable across attempts.
-- **Long-form writing was never evaluated.** The Antislop paper's ~30k-token multi-turn
-  benchmark is exactly where repetition damage becomes visible, and it did not fit the
-  deadline. The no-quality-loss claim is verified at ~1,000 tokens and **unverified beyond
-  that**.
-- **SWE-bench Verified and Terminal-Bench were not run** (per-instance Docker containers
-  were unavailable in the eval environment). Given the pass^k observation, long-horizon
-  agentic work is the most informative remaining test.
-- **The refusal detector was broken for the entire run**, so refusal boilerplate leaked
-  into the banlist. Several top remaining patterns (*safety guidelines prohibit*,
-  *harmless AI assistant*) are refusal text, not slop.
-- **The banlist contains ordinary vocabulary** that a manual pruning pass would remove.
-  It was not hand-reviewed.
-- Suppression is capped by the `lm_head`-only training constraint. The paper reaches
-  83–92% with full target modules; 66.41% matches its `lm_head`-only precedent on
-  Llama-3.3-70B.
-
 ## Usage
 
 Requires `trust_remote_code=True` for the `nemotron_h` architecture.
@@ -153,8 +108,8 @@ print(tok.decode(out[0], skip_special_tokens=True))
 
 > **Note:** on `transformers` 5.14.x, `apply_chat_template(..., tokenize=True)` under the
 > `TokenizersBackend` tokenizer class returns a truncated sequence. Render with
-> `tokenize=False` and tokenize the resulting string, as above. This affects the upstream
-> base checkpoint identically and is not specific to this model.
+> `tokenize=False` and tokenize the resulting string, as shown above. This affects the
+> upstream base model identically and is not specific to this checkpoint.
 
 ### vLLM
 
@@ -173,14 +128,14 @@ VLLM_USE_RUST_FRONTEND=1 vllm serve --model thoughtworks/Nemotron-3.5-30B-A3B-An
     --max-model-len 65536
 ```
 
-For tool calling, use the `qwen3_xml` parser — it handles this model's XML tool syntax
+For tool calling, use the `qwen3_xml` parser, which handles this model's XML tool syntax
 correctly.
 
 ## Training
 
 | | |
 |---|---|
-| Method | Antislop → FTPO (Final Token Preference Optimization) |
+| Method | Antislop, then FTPO (Final Token Preference Optimization) |
 | Target modules | `lm_head` only |
 | LoRA rank / alpha / dropout | 256 / 256 / 0.05 |
 | Steps / epochs | 265 / 1 |
@@ -190,14 +145,14 @@ correctly.
 | Generation | 1,000 prompts per iteration |
 | Preference examples | ~13,000 |
 | Banlist size | 4,267 patterns |
-| Hardware | 2× H100, ~12.7 h total (~4h15m for the FTPO stage) |
+| Hardware | 2x H100, about 12.7 h total, of which about 4 h 15 m was the FTPO stage |
 
 ## Model architecture
 
-Inherited unchanged from the base checkpoint: hybrid Latent Mixture-of-Experts with
+Inherited unchanged from the base model. A hybrid Latent Mixture-of-Experts design with
 interleaved Mamba-2 and MoE layers plus select attention layers, and Multi-Token
-Prediction (MTP) layers for speculative decoding. 30B total parameters, 3B active.
-Context length up to 262,144 in this checkpoint's config.
+Prediction (MTP) layers for speculative decoding. 30B total parameters, 3B active. Context
+length up to 262,144 in this checkpoint's config.
 
 ## License
 
@@ -215,6 +170,5 @@ The method:
 
 ## Acknowledgements
 
-Built on NVIDIA's Nemotron 3.5, with thanks to the NVIDIA team for early-access
-availability of the EA-0720 checkpoint and for the benchmark guidance that shaped the
-evaluation.
+Built on NVIDIA's Nemotron 3.5, with thanks to the NVIDIA team for the benchmark guidance
+that shaped the evaluation.