narcolepticchicken/legal-agent-router-v3

Legal Agent Router v3

Multi-head legal intake routing classifier. Maps user requests to practice areas, flags escalation signals, and predicts whether attorney review or source documents are needed.

No DV content. No domestic violence detection.

Architecture

• Base model: nlpaueb/legal-bert-small-uncased — 6 hidden layers, 512 hidden dim, ~35M params
• Heads: 11 route heads + 3 flag heads + 3 auxiliary heads = 17 outputs
• Activation: Sigmoid (multi-label classification)
• Dropout: 0.15

Outputs

Practice Area Routes (11)

criminal_law, civil_litigation, corporate_law, family_law, immigration_law, intellectual_property, employment_labor, real_estate, consumer_finance, health_benefits, traffic

Escalation Flags (3)

criminal_exposure, immigration_consequence, imminent_deadline

Auxiliary Decisions (3)

attorney_review_required, source_required, is_uncertain

Training

Parameter	Value
Training samples	135
Validation samples	29
Test samples	30
Epochs	30
Learning rate	1.5e-5
Schedule	Cosine with 10% warmup
Batch size	4 × 4 accumulation = 16
Loss	Weighted BCE (routes ×2, flags ×2)
Training time	~33s on A10G

Evaluation

Validation (29 samples)

Metric	Value
Route subset accuracy	0.000
Route macro F1	0.000
Flag macro F1	0.237
Missed escalation rate	0.000
Attorney review accuracy	1.000
Source required accuracy	0.690
Uncertainty accuracy	0.966

Test (30 samples)

Metric	Value
Route subset accuracy	0.000
Route macro F1	0.000
Flag macro F1	0.258
Missed escalation rate	0.000
Attorney review accuracy	0.967
Source required accuracy	0.800
Uncertainty accuracy	0.867

Per-Flag F1 Scores

Flag	Validation	Test
criminal_exposure	0.000	0.000
immigration_consequence	0.000	0.000
imminent_deadline	0.711	0.773

Per-Route F1 Scores

Route	Validation	Test
criminal_law	0.000	0.000
civil_litigation	0.000	0.000
corporate_law	0.000	0.000
family_law	0.000	0.000
immigration_law	0.000	0.000
intellectual_property	0.000	0.000
employment_labor	0.000	0.000
real_estate	0.000	0.000
consumer_finance	0.000	0.000
health_benefits	0.000	0.000
traffic	0.000	0.000

Inference Latency

• Average: 0.6ms per sample (batch size 8)
• P50: 0.6ms
• P99: 0.8ms
• Hardware: T4 (16GB)

Usage

from transformers import AutoTokenizer, BertConfig
from safetensors.torch import load_file
from huggingface_hub import hf_hub_download
import torch, torch.nn as nn

# Define model class (custom architecture)
class LegalRouterInference(nn.Module):
    def __init__(self, config):
        super().__init__()
        self.encoder = BertModel(config)
        h = config.hidden_size
        self.dropout = nn.Dropout(0.15)
        self.route_head = nn.Linear(h, 11)
        self.flag_head = nn.Linear(h, 3)
        self.attorney_head = nn.Linear(h, 1)
        self.source_head = nn.Linear(h, 1)
        self.uncertainty_head = nn.Linear(h, 1)
    
    def forward(self, input_ids, attention_mask):
        p = self.encoder(input_ids=input_ids, attention_mask=attention_mask)
        p = p.last_hidden_state[:, 0, :]
        p = self.dropout(p)
        # ... (see modeling code in repo)

# Load
config = BertConfig.from_pretrained("narcolepticchicken/legal-agent-router-v3")
model = LegalRouterInference(config)
sd = load_file(hf_hub_download("narcolepticchicken/legal-agent-router-v3", "model.safetensors"))
model.load_state_dict({k.replace("model.", ""): v for k, v in sd.items()})

tokenizer = AutoTokenizer.from_pretrained("narcolepticchicken/legal-agent-router-v3")
inputs = tokenizer("Request: I got a speeding ticket...", return_tensors="pt")
with torch.no_grad():
    logits = model(inputs["input_ids"], inputs["attention_mask"])
    probs = 1 / (1 + torch.exp(-logits))

Limitations

• Trained on 135 synthetic samples only — no real legal intake data
• Per-route training coverage is ~10-20 examples each
• May not generalize well to real-world legal intake patterns
• English only
• No coverage of niche practice areas (tax, environmental, maritime, etc.)
• Not for use in actual legal proceedings — experimental routing aid only

Dataset

narcolepticchicken/legal-agent-routing-v3 — 194 synthetic legal intake scenarios covering all 11 routes with multi-label cases and escalation flags.

License

Apache 2.0

---

Generated 2026-05-09 21:34 UTC

Generated by ML Intern

This model repository was generated by ML Intern, an agent for machine learning research and development on the Hugging Face Hub.

• Try ML Intern: https://smolagents-ml-intern.hf.space
• Source code: https://github.com/huggingface/ml-intern