Update README to match LFM2.5 family format

README.md

@@ -1,42 +1,151 @@
 ---
-license: other
 base_model: LiquidAI/LFM2.5-8B-A1B
+base_model_relation: finetune
+license: other
+license_name: lfm1.0
+license_link: https://huggingface.co/LiquidAI/LFM2.5-8B-A1B/blob/main/LICENSE
+library_name: transformers
+pipeline_tag: text-generation
+language:
+- en
+- ar
+- zh
+- fr
+- de
+- ja
+- ko
+- es
+- pt
 tags:
-- abliteration
 - heretic
+- abliterated
 - decensored
+- uncensored
 - liquid
+- lfm2
+- lfm2.5
 - moe
+- edge
+- conversational
 ---
 
 # LFM2.5-8B-A1B-Uncensored
 
-This is a decensored version of [LiquidAI/LFM2.5-8B-A1B](https://huggingface.co/LiquidAI/LFM2.5-8B-A1B), made using [Heretic](https://github.com/p-e-w/heretic).
+An uncensored version of [`LiquidAI/LFM2.5-8B-A1B`](https://huggingface.co/LiquidAI/LFM2.5-8B-A1B),
+made with [Heretic](https://github.com/p-e-w/heretic).
+
+Heretic removes the model's safety alignment ("censorship") using **directional
+ablation** (abliteration), with parameters chosen automatically by a TPE
+optimizer that co-minimizes the refusal rate and the KL divergence from the
+original model. Hence, the model stops refusing while keeping as much of its
+original behavior as possible. No human prompt-engineering or fine-tuning data
+was involved.
+
+## Performance
+
+| Metric | This model | Original model |
+|---|---|---|
+| Refusals (/100 harmful prompts) | **0** | 0 |
+| KL divergence (harmless prompts) | **0.0481** | 0 (by definition) |
+
+Refusals are measured against `mlabonne/harmful_behaviors`; KL divergence is
+measured on `mlabonne/harmless_alpaca`. Lower is better for both.
+
+> **Note on the baseline.** Heretic's substring-based refusal detector
+> registered very few refusals on the base `LFM2.5-8B-A1B` for this benchmark
+> (0–2 / 100, depending on the run), suggesting either that its refusal
+> phrasing doesn't match Heretic's marker list or that this model is comparatively
+> compliant out of the box. The abliteration still applies real, measurable
+> changes to the attention output and dense MLP projections (KL ≈ 0.05),
+> targeting the directional component associated with refusals.
+
+## Usage
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_id = "zaakirio/LFM2.5-8B-A1B-Uncensored"
+
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    device_map="auto",
+    trust_remote_code=True,
+)
+
+messages = [{"role": "user", "content": "Who are you?"}]
+inputs = tokenizer.apply_chat_template(
+    messages,
+    add_generation_prompt=True,
+    tokenize=True,
+    return_dict=True,
+    return_tensors="pt",
+).to(model.device)
+
+outputs = model.generate(**inputs, max_new_tokens=256)
+print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
+```
+
+The export is a merged, full-precision **BF16** model in Hugging Face format
+(~16 GB across 4 safetensors shards) — no adapter merge or dequantization step
+is required at load time.
 
 ## Abliteration parameters
 
-Trial 10 from the optimization study, which produced the best balanced result.
+Selected from **trial 131 of 130** (the best refusal/KL trade-off found by the
+optimizer among trials that actually modify outputs). Parameter names follow
+Heretic's canonical scheme; for LFM2 these map onto the `out_proj` (attention
+output) and `w2` (dense MLP down) projections. The fused MoE expert tensors
+are not directly modified by abliteration.
 
 | Parameter | Value |
-| :-------- | :---: |
-| direction_index | 11.99 |
-| attn.o_proj.max_weight | 0.96 |
-| attn.o_proj.max_weight_position | 21.20 |
-| attn.o_proj.min_weight | 0.95 |
-| attn.o_proj.min_weight_distance | 3.49 |
-| mlp.down_proj.max_weight | 0.96 |
-| mlp.down_proj.max_weight_position | 15.91 |
-| mlp.down_proj.min_weight | 0.11 |
-| mlp.down_proj.min_weight_distance | 1.90 |
-
-## Results
-
-| Metric | Value |
-| :----- | :---: |
-| Refusals on 100 harmful prompts | 0 / 100 |
-| KL divergence vs base model | 0.0000 |
-
-## Notes
-
-- The model has an unusual MoE architecture (`Lfm2MoeForCausalLM`) which required patches to `transformers/models/lfm2_moe/modeling_lfm2_moe.py` to support CPU offloading during optimization.
-- Abliteration targets `out_proj` (attention output) and `w2` (dense MLP down-projection) modules. The fused MoE expert tensors were not modified.
+|---|---|
+| direction_scope | global |
+| direction_index | 12.64 |
+| attn.o_proj.max_weight | 0.9009 |
+| attn.o_proj.max_weight_position | 22.83 |
+| attn.o_proj.min_weight | 0.8831 |
+| attn.o_proj.min_weight_distance | 12.85 |
+| mlp.down_proj.max_weight | 1.1906 |
+| mlp.down_proj.max_weight_position | 14.92 |
+| mlp.down_proj.min_weight | 0.0391 |
+| mlp.down_proj.min_weight_distance | 8.44 |
+
+## Run details
+
+- **Base model:** `LiquidAI/LFM2.5-8B-A1B` @ commit `5492b17c7128ec966b5fc661e374ee7edba7423d`
+- **Architecture:** LFM2 MoE (`Lfm2MoeForCausalLM`), 24 layers (2 dense + 22 MoE), BF16, 32 experts, 4 active per token
+- **Trials:** 130 completed (60 startup) · **Seed:** 1355772479
+- **Quantization during Heretic run:** none (CPU offload via Accelerate)
+- **Row normalization:** full · **Orthogonalize direction:** true
+- **Harmful set:** `mlabonne/harmful_behaviors` · **Harmless set:** `mlabonne/harmless_alpaca`
+
+## Notes / reproducibility
+
+LFM2 MoE is not yet natively supported by upstream Heretic. This run used a
+local compatibility patch:
+
+- `heretic/src/heretic/model.py` extended `get_layer_modules` to recognise LFM2's
+  `conv.out_proj`, `self_attn.out_proj`, `feed_forward.w2`, and
+  `feed_forward.experts.down_proj` paths.
+- `transformers/models/lfm2_moe/modeling_lfm2_moe.py` had `Lfm2MoeShortConv.slow_forward`
+  patched to route through `self.conv(...)` rather than directly accessing
+  `self.conv.weight`, so Accelerate's pre-forward hook can materialise
+  CPU-offloaded weights before the kernel runs.
+- The merge step was performed via a standalone CPU script
+  (`merge_trial10.py`) because the in-process merge during Heretic's interactive
+  save flow hit GPU OOM at this model size on a 16 GB card.
+
+## Intended use & disclaimer
+
+This model has had its refusal behavior substantially removed and will comply
+with requests the original model would have declined. It is provided for
+research and unrestricted local use. **You are responsible for how you use it**
+and for complying with all applicable laws and with the base model's
+[lfm1.0 license](https://huggingface.co/LiquidAI/LFM2.5-8B-A1B/blob/main/LICENSE),
+which carries over to this derivative.
+
+## Acknowledgements
+
+- Base model: [LiquidAI/LFM2.5-8B-A1B](https://huggingface.co/LiquidAI/LFM2.5-8B-A1B)
+- Decensoring tool: [Heretic](https://github.com/p-e-w/heretic) by p-e-w