docs: add changelog tracking project history
Browse files- CHANGELOG.md +55 -0
CHANGELOG.md
ADDED
|
@@ -0,0 +1,55 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Changelog
|
| 2 |
+
|
| 3 |
+
## v1.0.0 — Production Model (2025-04-27)
|
| 4 |
+
|
| 5 |
+
### Trained on Real Data
|
| 6 |
+
- **242,179 citation edges** from Semantic Scholar API (50K sampled papers from 1.6M corpus)
|
| 7 |
+
- **90,993 training rows** (1,857 queries, pre-2023 papers)
|
| 8 |
+
- **7,007 eval rows** (143 queries, 2023+ papers)
|
| 9 |
+
- Strict time-split with verified no temporal leakage
|
| 10 |
+
|
| 11 |
+
### Results
|
| 12 |
+
- nDCG@10: **0.8791** (vs heuristic 0.2641 → +232.8%)
|
| 13 |
+
- nDCG@5: **0.8250** (vs heuristic 0.1819 → +353.6%)
|
| 14 |
+
- MRR: **0.8795** (vs heuristic 0.2906 → +202.7%)
|
| 15 |
+
- HR@10: **1.0000** (vs heuristic 0.6638 → +50.6%)
|
| 16 |
+
- Latency: **0.371ms** per 100 candidates (2.7× under 1ms budget)
|
| 17 |
+
- Model size: **948 KB**
|
| 18 |
+
|
| 19 |
+
### Key Features (by importance)
|
| 20 |
+
1. `candidate_num_cited_by` (75,203) — dominant signal
|
| 21 |
+
2. `age_ratio` (7,597)
|
| 22 |
+
3. `candidate_position` (6,765)
|
| 23 |
+
4. `cosine_x_citations` (2,383)
|
| 24 |
+
5. `qdrant_cosine_score` (2,353)
|
| 25 |
+
|
| 26 |
+
### Model Details
|
| 27 |
+
- LightGBM lambdarank, 500 rounds with early stopping at iteration 141
|
| 28 |
+
- 37-feature schema (26 active + 11 zero-filled user features)
|
| 29 |
+
- Heuristic baseline: qdrant_cosine_score as proxy for ewma_longterm_similarity
|
| 30 |
+
|
| 31 |
+
---
|
| 32 |
+
|
| 33 |
+
## v0.1.0 — Synthetic Proof of Concept (2025-04-27)
|
| 34 |
+
|
| 35 |
+
### Pipeline Validation
|
| 36 |
+
- Full 3-script pipeline tested end-to-end on synthetic data
|
| 37 |
+
- 6-category test suite: data quality, model learning, fair comparison, prod readiness, feature analysis, honest verdict
|
| 38 |
+
- All tests passed
|
| 39 |
+
|
| 40 |
+
### Synthetic Results
|
| 41 |
+
- nDCG@10: 0.9985 (vs heuristic 0.9111 → +9.6%)
|
| 42 |
+
- 91.4% per-query win rate vs heuristic
|
| 43 |
+
- 0.088ms latency, 286 KB model size
|
| 44 |
+
- Train-eval gap: 0.0008 (no overfitting)
|
| 45 |
+
|
| 46 |
+
---
|
| 47 |
+
|
| 48 |
+
## Pipeline Design (2025-04-27)
|
| 49 |
+
|
| 50 |
+
### Created
|
| 51 |
+
- `01_fetch_citation_edges.py` — S2 API batch + bulk download + checkpoint/resume
|
| 52 |
+
- `02_generate_training_triples.py` — Qdrant ANN + Turso metadata + time-split
|
| 53 |
+
- `03_train_lightgbm.py` — lambdarank training + heuristic comparison + latency benchmark
|
| 54 |
+
- `test_full_pipeline.py` — comprehensive 6-category test suite
|
| 55 |
+
- 37-feature schema designed spanning content, user behavior, and cross features
|