CHANGELOG.md

Changelog

v1.0.0 — Production Model (2025-04-27)

Trained on Real Data

• 242,179 citation edges from Semantic Scholar API (50K sampled papers from 1.6M corpus)
• 90,993 training rows (1,857 queries, pre-2023 papers)
• 7,007 eval rows (143 queries, 2023+ papers)
• Strict time-split with verified no temporal leakage

Results

• nDCG@10: 0.8791 (vs heuristic 0.2641 → +232.8%)
• nDCG@5: 0.8250 (vs heuristic 0.1819 → +353.6%)
• MRR: 0.8795 (vs heuristic 0.2906 → +202.7%)
• HR@10: 1.0000 (vs heuristic 0.6638 → +50.6%)
• Latency: 0.371ms per 100 candidates (2.7× under 1ms budget)
• Model size: 948 KB

Key Features (by importance)

1. candidate_num_cited_by (75,203) — dominant signal
2. age_ratio (7,597)
3. candidate_position (6,765)
4. cosine_x_citations (2,383)
5. qdrant_cosine_score (2,353)

Model Details

• LightGBM lambdarank, 500 rounds with early stopping at iteration 141
• 37-feature schema (26 active + 11 zero-filled user features)
• Heuristic baseline: qdrant_cosine_score as proxy for ewma_longterm_similarity

---

v0.1.0 — Synthetic Proof of Concept (2025-04-27)

Pipeline Validation

• Full 3-script pipeline tested end-to-end on synthetic data
• 6-category test suite: data quality, model learning, fair comparison, prod readiness, feature analysis, honest verdict
• All tests passed

Synthetic Results

• nDCG@10: 0.9985 (vs heuristic 0.9111 → +9.6%)
• 91.4% per-query win rate vs heuristic
• 0.088ms latency, 286 KB model size
• Train-eval gap: 0.0008 (no overfitting)

---

Pipeline Design (2025-04-27)

Created

• 01_fetch_citation_edges.py — S2 API batch + bulk download + checkpoint/resume
• 02_generate_training_triples.py — Qdrant ANN + Turso metadata + time-split
• 03_train_lightgbm.py — lambdarank training + heuristic comparison + latency benchmark
• test_full_pipeline.py — comprehensive 6-category test suite
• 37-feature schema designed spanning content, user behavior, and cross features