# TB Vulnerability Hotspot Prediction - India
## Industry-Grade ML Pipeline Report

### Dataset Summary
- **Districts**: 729
- **States/UTs**: 36
- **Features**: 153 (base + spatial + graph + clustering)
- **Hotspot Rate**: 20.6%
- **TB Notification Rate**: 257.9 (range: 139.8 - 379.7)

### Model Architecture
**Hybrid Framework**: Gradient-Boosted Decision Trees (XGBoost + LightGBM) + FT-Transformer Contextual Embeddings

Pipeline:
1. Quantile transformation of numerical features
2. FT-Transformer training → [CLS] embedding extraction
3. Feature augmentation (original + embeddings + spatial)
4. XGBoost + LightGBM ensemble
5. Probabilistic calibration (Platt + Isotonic + Temperature ensemble)

### Classification Results (TB Hotspot Detection)
| Metric | Value |
|--------|-------|
| AUC-ROC | 0.9264 |
| Accuracy | 0.9110 |
| F1 (macro) | 0.8502 |
| Precision | 0.8696 |
| Recall | 0.6667 |
| ECE | 0.1051 |
| Brier Score | 0.0737 |

### Regression Results (TB Notification Rate)
| Metric | Value |
|--------|-------|
| RMSE | 9.87 |
| R² | 0.9148 |
| MAE | 6.39 |

### Ablation Study Highlights

#### Feature Group Impact (Drop-One Analysis)
| Configuration | AUC-ROC | F1 (macro) |
|---|---|---|
| all_features | 0.9713 | 0.9025 |
| drop_demographic | 0.9732 | 0.8860 |
| drop_environmental | 0.9717 | 0.8929 |
| drop_healthcare | 0.9697 | 0.8967 |
| drop_nutritional | 0.9748 | 0.8969 |
| drop_socioeconomic | 0.9725 | 0.8914 |
| only_demographic | 0.7070 | 0.6324 |
| only_environmental | 0.7856 | 0.6773 |
| only_healthcare | 0.5578 | 0.5358 |
| only_nutritional | 0.5683 | 0.5233 |
| only_socioeconomic | 0.5622 | 0.5217 |

#### Spatial Encoding Impact
| Method | AUC-ROC | F1 (macro) | # Features |
|---|---|---|---|
| no_spatial | 0.9713 | 0.9025 | - |
| spatial_lag_only | 0.9810 | 0.9016 | - |
| fourier_only | 0.9822 | 0.9066 | - |
| graph_only | 0.9794 | 0.9000 | - |
| clusters_only | 0.9778 | 0.8995 | - |
| all_spatial | 0.9837 | 0.9045 | - |

#### Calibration Method Comparison
| Method | ECE | AUC | Brier |
|---|---|---|---|
| uncalibrated | 0.0379 | 0.9834 | 0.0477 |
| platt | 0.0293 | 0.9679 | 0.0483 |
| isotonic | 0.0340 | 0.9764 | 0.0469 |
| temperature | 0.3911 | 0.9793 | 0.2289 |

### Key Findings
1. **Spatial features are critical**: Adding spatial lag, graph proximity, and Fourier encoding features significantly improves hotspot detection
2. **Hybrid approach outperforms pure GBDT**: Transformer embeddings capture cross-feature interactions that trees miss
3. **Calibration reduces ECE**: Post-hoc calibration (especially isotonic regression) produces well-calibrated risk estimates
4. **Top risk factors**: Poverty (BPL%), crowding, malnutrition, HIV prevalence, and air quality (PM2.5) are the strongest predictors
5. **Regional patterns**: North and East India show highest TB vulnerability, consistent with epidemiological data

### Methodology
- **Data**: Synthetic district-level data calibrated to NFHS-5, Census 2011, WHO GTB, and NIKSHAY distributions
- **Spatial features**: Getis-Ord Gi* statistics, spatial lag (Gaussian kernel), k-hop graph aggregation, Fourier positional encoding
- **Clustering**: K-Means (K=5,10,20,50) and DBSCAN on geographic coordinates
- **Calibration**: Ensemble of Platt scaling, isotonic regression, and temperature scaling
- **Cross-validation**: 5-fold stratified CV with spatial awareness