# Changelog

## v5.0.0 — Graph + Metadata Model (2026-05-11)

### V5: Combined Graph Topology + Turso Paper Metadata
Added real paper metadata features from Turso DB (1.6M papers):
- Citation counts, influential citations, publication dates
- Primary category matching (same field = relevance signal)
- Recency scores, citation-per-year velocity
- Citation count ratios (candidate vs survey)

**This broke through the graph-only ceiling.** The model now combines structural (who cites whom) with content (what kind of paper is this) signals.

| Metric | CitationCount | V4 (graph) | **V5 (graph+meta)** | Oracle |
|--------|---------------|------------|---------------------|--------|
| nDCG@10 | 0.837 | 0.946 | **1.000** | 1.000 |
| Hard Neg AUC | 0.628 | 0.778 | **0.837** | 1.000 |
| MRR | 0.876 | 0.902 | **0.938** | 0.938 |

**Hard Neg AUC: 0.837** — model can distinguish expert-included from expert-excluded papers 83.7% of the time.

### Top Features (V5)
1. `n_surveys_citing` (33,610) — cross-survey consensus
2. `cand_cited_by` (5,473) — total incoming citations from graph
3. `hub_score` (1,091) — cited by well-connected papers
4. `survey_refs` (1,036) — complexity of the survey paper
5. `cand_recency` (971) — newer papers get slight boost
6. `cand_influential_cit` (716) — Semantic Scholar influential citations
7. `same_category` (607) — category match matters!
8. `cited_by_survey_refs` (584) — 2-hop citation paths
9. `authority` (579) — PageRank-like authority
10. `cit_ratio` (572) — relative citation strength

### Model Details
- **25 features** (12 graph + 5 advanced graph + 8 metadata)
- **200 trees**, 127 leaves, lr=0.05
- **2.7 MB** model file
- Trained on 58,749 rows, 486 queries

---

## v4.0.0 — Advanced Graph Features (2026-05-11)

### V4: Extended Graph Topology
Added 12 new features beyond V3:
- Authority score (PageRank-like)
- Year difference and recency
- Clustering coefficient
- Preferential attachment
- Indirect citation paths

| Metric | V3 | **V4** |
|--------|-----|--------|
| Hard Neg AUC | 0.738 | **0.778** |

---

## v3.0.0 — Cross-Survey Authority Model + Eval V2 (2026-05-10)

### New Evaluation Framework (Eval V2)
The old eval tested "can you find a cited paper?" which inflated metrics. The new eval tests "can you rank papers an expert considers essential above ones they deliberately excluded?"

- **Label scheme**: 5 tiers (essential=4, relevant=3, useful=2, niche=1, hard_negative=0)
- **Hard negatives**: Papers cited by OTHER surveys in same field but NOT this one (expert exclusions)
- **Key metric**: Hard Negative AUC (can model tell cited from expert-excluded?)
- **Data**: 500 comprehensive papers (≥30 references each), 60,574 rows
- **Design doc**: `EVAL_V2_DESIGN.md`

### V3 Model: Cross-Survey Authority
Trained on **cross-survey consensus labels** with 13 graph-based features.

| Metric | Random | CitationCount | V3 LightGBM | Oracle |
|--------|--------|---------------|-------------|--------|
| Hard Neg AUC | 0.504 | 0.628 | **0.738** | 1.000 |

### Key Insight
V1/V2 models are essentially popularity rankers. V3+ moves beyond this by learning cross-survey authority.

---

## v2.0.0 — V2 LightGBM (2025-04-27)

- nDCG@10: **0.9186**, MRR: **1.0000**
- 26,494 training rows, 532 queries
- Model size: 353 KB

---

## v1.0.0 — Production Model (2025-04-27)

- nDCG@10: **0.8791** (vs heuristic 0.2641, +232.8%)
- 90,993 training rows, 1,857 queries
- Model size: 948 KB

---

## v0.1.0 — Synthetic Proof of Concept (2025-04-27)

- Full pipeline tested on synthetic data
- nDCG@10: 0.9985

---

## v0.0.1 — Pipeline Design (2025-04-27)

- 3-script pipeline created
- 37-feature schema designed