erayyapagci/lfm2.5-pii-en-hi-hinglish-q8

LFM2.5 PII — English, Hindi, and Hinglish (FP32 + ONNX INT8)

Fine-tuned version of LiquidAI/LFM2.5-Encoder-350M-PII-Detector for high-recall PII detection in English, Romanized Hinglish, and Devanagari Hindi.

This repository contains:

• the original fine-tuned PyTorch FP32 model (model.safetensors);
• tokenizer, configuration, and custom Transformers model code;
• a CPU-oriented dynamically quantized ONNX INT8 export under onnx-q8/.

Evaluation

The following results are from a 900-record synthetic post-training holdout with 600 positive records and 300 hard negatives. Span matching uses overlap matching plus the same deterministic boundary/rule post-processing for every format.

Format	Precision	Recall	F1	Character coverage	Residual entities	Hard negatives flagged	CPU throughput	Peak RSS
Original LiquidAI base, ONNX FP32	0.6863	0.6783	0.6823	0.8101	166/600	87/300	36.0 records/s	3.54 GB
Fine-tuned model, ONNX FP32	0.7896	0.8817	0.8331	0.9309	57/600	70/300	37.8 records/s	3.49 GB
Fine-tuned model, ONNX INT8	0.7741	0.8850	0.8258	0.9240	62/600	81/300	46.9 records/s	2.00 GB

The original-base row evaluates the untouched upstream model revision shown below, exported to FP32 ONNX with the same graph exporter and scored with the same decoding and post-processing. The fine-tuned FP32 ONNX export had exact token-argmax parity with the uploaded PyTorch weights in prior parity checks.

Contextual person-name recall was 1.0000 for FP32 and 0.9900 for INT8 on this holdout.

Loading the original Transformers model

The repository includes custom model code inherited from the base architecture, so loading requires trust_remote_code=True.

from transformers import AutoModelForTokenClassification, AutoTokenizer

repo = "erayyapagci/lfm2.5-pii-en-hi-hinglish-q8"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForTokenClassification.from_pretrained(
    repo,
    trust_remote_code=True,
)

Review remote model code before enabling trust_remote_code in a sensitive environment.

Loading ONNX INT8 on CPU

import onnxruntime as ort
from transformers import AutoTokenizer

repo = "erayyapagci/lfm2.5-pii-en-hi-hinglish-q8"
tokenizer = AutoTokenizer.from_pretrained(repo)
session = ort.InferenceSession(
    "onnx-q8/model.int8.onnx",
    providers=["CPUExecutionProvider"],
)

Token logits require BIOES decoding and tokenizer offset mapping to recover text spans. Evaluation also used deterministic boundary repair and structured-PII rules; raw token argmax alone will not reproduce the table exactly.

Intended use and limitations

This is an experimental PII detector, not a complete security boundary. Test it on manually reviewed data from your own domain before use. In the holdout it performed well on names, addresses, dates, emails, phones, and URLs, but remained weak on flexible grouped account numbers and previously unseen arbitrary secret formats. Quantization can change borderline predictions. Use fail-closed handling for sensitive exports and consider deterministic recognizers for structured identifiers.

The training mixture used local synthetic data and permissively licensed/synthetic external sources, including NVIDIA Nemotron PII, Gretel synthetic PII, and AI4Privacy-derived data. Do not infer that the model is suitable for every locale, demographic, or regulatory context.

Base model and license

Base model revision: b8c9cf3d2d6ae52501b35a27ba46f271449c9ce2.

The model is distributed under the LFM Open License v1.0. See LICENSE and the base model repository.