---
base_model: cross-encoder/nli-MiniLM2-L6-H768
tags:
  - intent-detection
  - cross-encoder
  - reranker
  - text-classification
language:
  - en
license: apache-2.0
---

# intent-crossencoder-miniLM2-L6-H768

Cross-encoder reranker for enterprise intent detection (DLP / security).
Fine-tuned from `cross-encoder/nli-MiniLM2-L6-H768` on a synthetic intent-detection dataset.

## Intended use

Binary classification: given a `(user_input, intent_description)` pair,
predict whether the user input matches the intent.
Designed as **Stage-2** in a cascading firewall:
Stage-1 (fast heuristic) → **this model** (reranker) → Stage-2 LLM (Qwen).

## Input format

```
user_input [SEP] intent_description
```

## Performance (held-out test set, threshold=0.3)

| Metric     | Value         |
|------------|---------------|
| Recall     | 0.9933    |
| Precision  | 0.9922 |
| F1         | 0.9927        |
| AUC-ROC    | 0.9985       |
| PR-AUC     | 0.9980    |
| Best threshold (F1-optimal) | 0.3727 → F1=0.9933 |

## Training config

| Parameter           | Value                  |
|---------------------|------------------------|
| Base model          | `cross-encoder/nli-MiniLM2-L6-H768`         |
| Batch size          | 64    |
| Grad accum steps    | 1 |
| Effective batch     | 64 |
| Learning rate       | 2e-05            |
| Label smoothing     | 0.05 |
| Warmup ratio        | 0.06  |
| Max sequence length | 256              |
| Early stopping      | recall@0.3 (patience=3) |
| Epochs trained      | 7 |
| Training time       | 2.9 min |

## Inference snippet

```python
import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

model_id  = "aryasuneesh-quilr/intent-crossencoder-miniLM2-L6-H768"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model     = AutoModelForSequenceClassification.from_pretrained(model_id)
model.eval()

def score(user_input: str, intent_description: str) -> float:
    pair = f"{user_input} [SEP] {intent_description}"
    enc  = tokenizer(pair, return_tensors="pt", truncation=True, max_length=256)
    with torch.no_grad():
        logits = model(**enc).logits
    return torch.softmax(logits, dim=1)[0, 1].item()   # P(match)

# Example
s = score(
    "Our AWS_SECRET_ACCESS_KEY was found in a public repo",
    "Identify exposure of authentication credentials or API keys"
)
print(f"Match probability: {s:.4f}")   # use threshold 0.3727 for best F1
```

## Files in this repo

| File | Description |
|------|-------------|
| `model.safetensors` | HF-native weights |
| `best_model.pt` | Raw PyTorch state_dict (for resuming training) |
| `training_config.json` | Full hyperparameter record |
| `metrics/` | Per-epoch + test-set evaluation CSVs |

---
*Generated 2026-02-23 08:35 UTC by ablation_reranker_training.py*