sheltron-ai/privacy-filter-ettin-32m

Sheltron Privacy Filter Ettin 32M

Fast, compact privacy filter for lightweight deployment.

Sheltron Privacy Filter Ettin 32M is a compact token-classification model for detecting privacy-risk spans in text, with same output format as OpenAI Privacy Filter. Model architecture is based on jhu-clsp/ettin-encoder-32m.

This model is intended for fast and light-weight usecases such as screening of emails, messages, conversations, and commands.

This model repository is standalone. Loading the root checkpoint requires only standard Hugging Face Transformers dependencies; loading an ONNX variant requires ONNX Runtime. The root AutoModelForTokenClassification.from_pretrained(...) call loads the FP32 Safetensors checkpoint; select an ONNX file explicitly when using FP32, FP16, or INT8 ONNX Runtime inference.

Model Details

Field	Value
Architecture	ModernBERT token classifier
Parameters	32,043,681
Base model	jhu-clsp/ettin-encoder-32m
Output classes	33: O plus BIOES tags for eight span categories
Evaluated context window	512 tokens with stride 128
Root checkpoint precision	FP32
Training	Full-parameter fine-tuning
Decoder	Constrained BIOES Viterbi, zero transition biases

The eight decoded categories are account_number, private_address, private_date, private_email, private_person, private_phone, private_url, and secret.

Although the base architecture exposes a longer positional limit, this model was trained and evaluated at 512 tokens. Inputs longer than 512 tokens should be processed with overlapping 512-token windows and stride 128.

Files And Runtime Targets

File	Precision	Size	Intended runtime
model.safetensors	FP32	122.2 MiB	Canonical Transformers checkpoint
onnx/model.onnx	FP32	122.5 MiB	ONNX reference and compatibility path
onnx/model_fp16.onnx	FP16 weights/compute, FP32 output	61.4 MiB	GPU; validated with ONNX Runtime CUDA
onnx/model_quantized.onnx	Per-row QInt8 embedding plus selective dynamic QUInt8	33.6 MiB	Compact CPU path, one request per inference batch

Usage

Transformers

import torch
from transformers import AutoModelForTokenClassification, AutoTokenizer

model_id = "sheltron-ai/privacy-filter-ettin-32m"
tokenizer = AutoTokenizer.from_pretrained(model_id, use_fast=True)
model = AutoModelForTokenClassification.from_pretrained(model_id)
model.eval()

text = "Send the report to alice@example.com after review."
encoded = tokenizer(
    text,
    return_tensors="pt",
    return_offsets_mapping=True,
    return_overflowing_tokens=True,
    truncation=True,
    max_length=512,
    stride=128,
    padding="max_length",
)
offsets = encoded.pop("offset_mapping")
encoded.pop("overflow_to_sample_mapping", None)

with torch.inference_mode():
    logits = model(**encoded).logits

The logits are BIOES token scores. Do not independently argmax them when reproducing the reported span metrics. Apply the constrained Viterbi transitions and operating point in viterbi_calibration.json, then convert decoded token offsets into character spans and deduplicate exact spans from overlapping windows.

ONNX Runtime

Install the ordinary ONNX inference dependencies; Optimum, bitsandbytes, and a custom runtime plugin are not required:

pip install onnxruntime transformers huggingface-hub numpy

Hugging Face does not define one loader shared by every INT8 format. This repository's INT8 artifact is an ONNX model, so download that file explicitly and open it with ONNX Runtime:

from huggingface_hub import hf_hub_download
import numpy as np
import onnxruntime as ort
from transformers import AutoTokenizer

model_id = "sheltron-ai/privacy-filter-ettin-32m"
filename = "onnx/model_quantized.onnx"  # or model.onnx / model_fp16.onnx
model_path = hf_hub_download(repo_id=model_id, filename=filename)

providers = ["CPUExecutionProvider"]
# For model_fp16.onnx use: providers = ["CUDAExecutionProvider"]
session = ort.InferenceSession(model_path, providers=providers)
tokenizer = AutoTokenizer.from_pretrained(model_id, use_fast=True)

tokens = tokenizer(
    "Send the report to alice@gmail.com after review.",
    return_tensors="np",
    truncation=True,
    max_length=512,
    padding="max_length",
)
input_names = {item.name for item in session.get_inputs()}
inputs = {
    key: np.asarray(value, dtype=np.int64)
    for key, value in tokens.items()
    if key in input_names
}
logits = session.run(None, inputs)[0]

Use viterbi_calibration.json for the same post-processing contract as the Transformers path. ONNX Runtime returns raw BIOES logits; constrained Viterbi decoding is required to reproduce the reported span metrics for every precision variant, not only INT8.

License And Attribution

The Sheltron Privacy Filter Ettin 32M model is released under Apache-2.0. The Ettin base model is published under the MIT License.