README.md

metadata

license: cc-by-nc-4.0
base_model: answerdotai/ModernBERT-base
library_name: transformers
pipeline_tag: text-classification
tags:
  - rag
  - governance
  - pyrrho
  - fitz-gov-v2
  - modernbert
  - multi-label-classification

pyrrho-v2-nano-g1

pyrrho-v2-nano-g1 is a ModernBERT-base classifier for Fitz RAG governance. It replaces the older Pyrrho g5 multitask shape with four v2 heads:

• evidence_verdict: INSUFFICIENT, DISPUTED, SUFFICIENT
• failure_mode: none, unresolved_conflict, missing_or_incomplete_evidence, wrong_scope_or_version, ambiguous_request
• retrieval_intents: multi-label needs_lookup, needs_temporal_resolution, needs_comparison_or_set, needs_broad_coverage
• evidence_kinds: multi-label needs_text, needs_table_or_record, needs_code_or_symbol, needs_config_or_setting, needs_log_or_run_result, needs_document_layout

The model is intended for local governance in fitz-sage: decide whether retrieved evidence is sufficient, insufficient, or disputed, and expose actionable retrieval/failure metadata.

Inputs

Post-retrieval governance input:

Question: <user query>

Sources:
[1] <retrieved source text>
[2] <retrieved source text>

Pre-retrieval planning input:

Question: <user query>

Output Decoding

The 18 logits are not one flat softmax. Decode them by group:

import torch
from transformers import AutoModelForSequenceClassification, PreTrainedTokenizerFast

model_id = "yafitzdev/pyrrho-v2-nano-g1"
tokenizer = PreTrainedTokenizerFast.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id).eval()

text = "Question: What is the capital of France?\n\nSources:\n[1] Paris is the capital of France."
encoded = tokenizer(text, return_tensors="pt", truncation=True, max_length=2048)
with torch.no_grad():
    logits = model(**encoded).logits[0]

verdict_labels = ["INSUFFICIENT", "DISPUTED", "SUFFICIENT"]
failure_labels = [
    "none",
    "unresolved_conflict",
    "missing_or_incomplete_evidence",
    "wrong_scope_or_version",
    "ambiguous_request",
]
intent_labels = [
    "needs_lookup",
    "needs_temporal_resolution",
    "needs_comparison_or_set",
    "needs_broad_coverage",
]
kind_labels = [
    "needs_text",
    "needs_table_or_record",
    "needs_code_or_symbol",
    "needs_config_or_setting",
    "needs_log_or_run_result",
    "needs_document_layout",
]

verdict = verdict_labels[int(torch.softmax(logits[0:3], dim=-1).argmax())]
failure = failure_labels[int(torch.softmax(logits[3:8], dim=-1).argmax())]
intents = [
    label for label, score in zip(intent_labels, torch.sigmoid(logits[8:12]))
    if float(score) >= 0.5
]
kinds = [
    label for label, score in zip(kind_labels, torch.sigmoid(logits[12:18]))
    if float(score) >= 0.5
]

Training Snapshot

• Dataset: fitz-gov-v2
• Clean active training rows: 41,358
• Training source pointer: fitz_gov_v2_41358_20260703
• Poisoned/quarantined later data is excluded.
• Base model: answerdotai/ModernBERT-base
• Seed: 42

Local Evaluation

Held-out training eval from outputs/modernbert_base_v2_alpha_41358_active_20260704_seed42:

Metric	Value
overall score	0.9497
verdict accuracy	0.9727
false sufficient rate	0.0455
failure accuracy	0.9601
retrieval exact match	0.8335
retrieval macro F1	0.9300
evidence-kind exact match	0.9809
evidence-kind macro F1	0.9950

Fitz-sage benchmark check for this release candidate:

Benchmark	Result
balanced fixed-evidence toy cases	120/120
live fitz-sage benchmark	86/120

The live benchmark result is the practical integration target; the fixed-evidence suite is a minimal sanity check for the governance head.

Artifacts

This repository contains:

• model.safetensors: Transformers checkpoint
• model.onnx: FP32 ONNX export
• model_quantized.onnx: INT8 dynamic ONNX export
• tokenizer/config files
• manifest.json: release metadata

License

CC BY-NC 4.0. Free for research, evaluation, and personal use; commercial use requires a separate license.