dlepighe1/resume-jd-matcher-mpnet

ResumeAI — Resume / Job-Description Matcher (MPNet, fine-tuned + calibrated)

Fine-tuned all-mpnet-base-v2 scoring resume-to-job-description fit on a 0-1 scale. Trained with a combined CoSENT + CosineSimilarity objective on 815 curated pairs from 255 real job postings with 3x augmentation, then calibrated with Platt scaling fitted on an external calibration split.

Evaluation

All numbers below are on 106 pairs from 53 job postings with zero overlap with training — verified, not assumed. The calibrator was fitted on a separate 106-pair external split, so the test pairs are untouched by any stage of fitting.

Metric	Value	95% CI
Spearman (production seed 43)	0.8163	[0.7396, 0.8638]
MAE	0.1270	[0.108, 0.1462]
Spearman across 3 seeds	0.8273 +/- 0.0236
MAE across 3 seeds	0.1194 +/- 0.0113
Base model before fine-tuning	0.6246 / 0.2138
Precision@1 over 53 unseen postings	84.9%	vs 25% random

Calibrator choice: Platt and isotonic are statistically tied on this test set (bootstrap 95% CI on the MAE difference: [-0.0037, 0.014]). Platt ships because a 2-parameter sigmoid cannot overfit a 106-pair calibration split, while an isotonic step function can.

Usage

from sentence_transformers import SentenceTransformer
import numpy as np, pickle

model = SentenceTransformer("dlepighe1/resume-jd-matcher-mpnet")
emb = model.encode([resume_text, jd_text])
raw = float(np.dot(emb[0], emb[1]) / (np.linalg.norm(emb[0])*np.linalg.norm(emb[1])))

# calibrated 0-1 score
platt = pickle.load(open("platt_calibrator.pkl", "rb"))
score = platt([raw])[0]

Apply smart_truncate_jd (see the research repo, src/text_utils.py) to the job description first — the model was trained on preprocessed JDs capped at 350 words, and skipping it degrades scores silently.

Limitations

• Match labels are synthetic, generated and hand-curated against a rubric rather than collected from recruiters. Scores reflect that rubric, not hiring outcomes.
• n = 106 on the final test — the confidence intervals above are wide, and differences smaller than their width are not meaningful.
• Not audited for use in automated hiring decisions. See the research repo's data card.

Research repo: https://github.com/dlepighe1/Resume-jd-matcher