README.md

---
license: cc-by-nc-4.0
base_model: answerdotai/ModernBERT-base
library_name: transformers
pipeline_tag: text-classification
tags:
  - rag
  - governance
  - pyrrho
  - fitz-gov-v2
  - modernbert
  - multi-label-classification
---

# pyrrho-v2-nano-g1

`pyrrho-v2-nano-g1` is the Pyrrho v2 ModernBERT-base classifier for Fitz RAG
governance. It emits four native v2 heads:

- `evidence_verdict`: `INSUFFICIENT`, `DISPUTED`, `SUFFICIENT`
- `failure_mode`: `none`, `unresolved_conflict`, `missing_or_incomplete_evidence`, `wrong_scope_or_version`, `ambiguous_request`
- `retrieval_intents`: multi-label `needs_lookup`, `needs_temporal_resolution`, `needs_comparison_or_set`, `needs_broad_coverage`
- `evidence_kinds`: multi-label `needs_text`, `needs_table_or_record`, `needs_code_or_symbol`, `needs_config_or_setting`, `needs_log_or_run_result`, `needs_document_layout`

The model is intended for local governance in `fitz-sage`: decide whether
retrieved evidence is sufficient, insufficient, or disputed, and expose
actionable retrieval/failure metadata.

## Inputs

Governance input:

```text
Question: <user query>

Sources:
[1] <retrieved source text>
[2] <retrieved source text>
```

## Output Decoding

The 18 logits are not one flat softmax. Decode them by group:

```python
import torch
from transformers import AutoModelForSequenceClassification, PreTrainedTokenizerFast

model_id = "yafitzdev/pyrrho-v2-nano-g1"
tokenizer = PreTrainedTokenizerFast.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id).eval()

text = "Question: What is the capital of France?\n\nSources:\n[1] Paris is the capital of France."
encoded = tokenizer(text, return_tensors="pt", truncation=True, max_length=2048)
with torch.no_grad():
    logits = model(**encoded).logits[0]

verdict_labels = ["INSUFFICIENT", "DISPUTED", "SUFFICIENT"]
failure_labels = [
    "none",
    "unresolved_conflict",
    "missing_or_incomplete_evidence",
    "wrong_scope_or_version",
    "ambiguous_request",
]
intent_labels = [
    "needs_lookup",
    "needs_temporal_resolution",
    "needs_comparison_or_set",
    "needs_broad_coverage",
]
kind_labels = [
    "needs_text",
    "needs_table_or_record",
    "needs_code_or_symbol",
    "needs_config_or_setting",
    "needs_log_or_run_result",
    "needs_document_layout",
]

verdict = verdict_labels[int(torch.softmax(logits[0:3], dim=-1).argmax())]
failure = failure_labels[int(torch.softmax(logits[3:8], dim=-1).argmax())]
intents = [
    label for label, score in zip(intent_labels, torch.sigmoid(logits[8:12]))
    if float(score) >= 0.5
]
kinds = [
    label for label, score in zip(kind_labels, torch.sigmoid(logits[12:18]))
    if float(score) >= 0.5
]
```

## Training Snapshot

- Dataset: `fitz-gov-v2`
- Clean active training rows: 41,358
- Training source pointer: `fitz_gov_v2_41358_20260703`
- Base model: `answerdotai/ModernBERT-base`
- Seed: 42

## Local Evaluation

Held-out training eval from `outputs/modernbert_base_v2_alpha_41358_active_20260704_seed42`:

| Metric | Value |
| --- | ---: |
| overall score | 0.9497 |
| verdict accuracy | 0.9727 |
| false sufficient rate | 0.0455 |
| failure accuracy | 0.9601 |
| retrieval exact match | 0.8335 |
| retrieval macro F1 | 0.9300 |
| evidence-kind exact match | 0.9809 |
| evidence-kind macro F1 | 0.9950 |

Fitz-sage benchmark check for this release candidate:

| Benchmark | Result |
| --- | ---: |
| balanced fixed-evidence governance sanity suite | 120/120 |
| live fitz-sage benchmark | 86/120 |

The live benchmark result is the practical integration target; the fixed-evidence
suite is a minimal sanity check for the governance head.

## Artifacts

This repository contains:

- `model.safetensors`: Transformers checkpoint
- `model.onnx`: FP32 ONNX export
- `model_quantized.onnx`: INT8 dynamic ONNX export
- tokenizer/config files
- `manifest.json`: release metadata

## License

CC BY-NC 4.0. Free for research, evaluation, and personal use; commercial use
requires a separate license.