ettin-encoder-32m-arxiv-classification-8192

jhu-clsp/ettin-encoder-32m (ModernBERT encoder, 8192 context) finetuned for arXiv paper topic classification (11 classes), 8192-token context on ccdv/arxiv-classification.

Results (held-out test)

metric	value
accuracy	0.783
macro-F1	0.7803
eval max_length	8192

Finetuned on a single RTX 3080 (bf16). See the project for the full training pipeline.

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

tok = AutoTokenizer.from_pretrained("vumichien/ettin-encoder-32m-arxiv-classification-8192")
model = AutoModelForSequenceClassification.from_pretrained("vumichien/ettin-encoder-32m-arxiv-classification-8192")

inputs = tok("your text here", truncation=True, max_length=8192, return_tensors="pt")
pred = model(**inputs).logits.argmax(-1).item()
print(model.config.id2label[pred])