README.md

metadata

library_name: transformers
pipeline_tag: text-classification
base_model: microsoft/BiomedNLP-BiomedBERT-base-uncased-abstract-fulltext
tags:
  - medical-triage
  - text-classification
  - peft
  - frozen-encoder
  - linear-probe
  - sortmed

Frozen Encoder BioMedBERT Medical Triage

This model is a frozen-encoder sequence classifier for three-class medical pre-triage classification. The pretrained encoder/backbone is kept frozen and only the classification head is trained.

Base model

microsoft/BiomedNLP-BiomedBERT-base-uncased-abstract-fulltext

Task

The classifier predicts one of three triage labels:

• self_monitor
• consult_gp
• urgent

Adaptation method

• Method: Frozen Encoder + Trainable Classification Head
• Base encoder: frozen
• Trainable: classification head (+ pooler for BERT-style models)
• Max sequence length: 128
• Epochs: 8 (early stopping patience 3), best checkpoint by validation macro F1

Loading

This is a standard Transformers model and loads with the plain API (no trust_remote_code required):

from transformers import AutoModelForSequenceClassification, AutoTokenizer
tok = AutoTokenizer.from_pretrained("cristian-untaru/frozen-encoder-biomedbert-medical-triage")
model = AutoModelForSequenceClassification.from_pretrained("cristian-untaru/frozen-encoder-biomedbert-medical-triage")

Test metrics

{
  "eval_loss": 0.8381163477897644,
  "eval_accuracy": 0.6038,
  "eval_precision": 0.6201,
  "eval_recall": 0.6058,
  "eval_f1": 0.6107,
  "eval_auc": 0.8076,
  "eval_specificity": 0.7995,
  "eval_iou": 0.4444
}

This model is intended for academic experimentation and prototyping, not for standalone clinical decision-making.