Text Classification
Transformers
Safetensors
English
chest2vec_labeler
feature-extraction
radiology
chest-ct
report-labeling
multi-label
ct-rate
chexbert-style-f1
custom_code
Instructions to use chest2vec/chest2vec_labeler with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use chest2vec/chest2vec_labeler with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="chest2vec/chest2vec_labeler", trust_remote_code=True)# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("chest2vec/chest2vec_labeler", trust_remote_code=True, dtype="auto") - Notebooks
- Google Colab
- Kaggle
Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -107,6 +107,21 @@ Each level (`leaf` = 137 labels, `upper` = 38 container groups, `anatomy` = 10 s
|
|
| 107 |
max-over-children roll-up of the leaf predictions (`model.aggregate_hierarchy(...)`). Coarser
|
| 108 |
levels are easier to match, so upper/anatomy F1 are typically higher than leaf.
|
| 109 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 110 |
## Inputs & conventions
|
| 111 |
|
| 112 |
- Input is the **findings** text (the model was trained on CT-RATE findings + their refined
|
|
@@ -143,6 +158,17 @@ rows are the hierarchy roll-up.
|
|
| 143 |
| upper (≥30 positives) | 19 | 0.837 | — |
|
| 144 |
| anatomy | 10 | 0.869 | — |
|
| 145 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 146 |
Leaf macro-AUC barely moves public→private (**0.989 → 0.972**), i.e. label ranking transfers to
|
| 147 |
the unseen set; the F1 gap is mostly threshold / labeling-convention, not a domain failure.
|
| 148 |
Separately, a radiologist spot-checked **966** reports of the public test labels (857 fully
|
|
|
|
| 107 |
max-over-children roll-up of the leaf predictions (`model.aggregate_hierarchy(...)`). Coarser
|
| 108 |
levels are easier to match, so upper/anatomy F1 are typically higher than leaf.
|
| 109 |
|
| 110 |
+
### Per-label best F1 (threshold tuning)
|
| 111 |
+
|
| 112 |
+
The default decision threshold is a single global value, but the F1-optimal threshold differs per
|
| 113 |
+
label. To get the **best achievable F1 per label** (and the threshold that achieves it) against a
|
| 114 |
+
ground-truth label set:
|
| 115 |
+
|
| 116 |
+
```python
|
| 117 |
+
# gt: a DataFrame with the 137 label columns (ternary; positive == 1), or a binary array
|
| 118 |
+
res = model.per_label_best_f1(reports, gt, tokenizer=tok, level="leaf", min_pos=30)
|
| 119 |
+
res["macro_best_f1_min_pos"] # macro best-F1 over labels with >= min_pos positives
|
| 120 |
+
res["per_label"]["Pleural effusion"] # {'best_f1':.., 'best_threshold':.., 'n_pos':..}
|
| 121 |
+
```
|
| 122 |
+
|
| 123 |
+
Per-label threshold tuning lifts macro-F1 by ~4–6 points over the fixed-0.5 threshold (see below).
|
| 124 |
+
|
| 125 |
## Inputs & conventions
|
| 126 |
|
| 127 |
- Input is the **findings** text (the model was trained on CT-RATE findings + their refined
|
|
|
|
| 158 |
| upper (≥30 positives) | 19 | 0.837 | — |
|
| 159 |
| anatomy | 10 | 0.869 | — |
|
| 160 |
|
| 161 |
+
**Per-label best F1** (threshold swept per label to maximize F1; macro over leaf labels with ≥30
|
| 162 |
+
positives, via `model.per_label_best_f1`):
|
| 163 |
+
|
| 164 |
+
| Eval set | macro best-F1 (≥30) | macro-F1 @0.5 (≥30) | macro best-F1 (all evaluated) |
|
| 165 |
+
|---|--:|--:|--:|
|
| 166 |
+
| CT-RATE public | **0.907** | 0.866 | 0.844 |
|
| 167 |
+
| Private | **0.820** | 0.761 | 0.795 |
|
| 168 |
+
|
| 169 |
+
F1-optimal thresholds vary widely by label (~0.04–0.75), so per-label tuning recovers ~4–6 macro-F1
|
| 170 |
+
points over a single global threshold.
|
| 171 |
+
|
| 172 |
Leaf macro-AUC barely moves public→private (**0.989 → 0.972**), i.e. label ranking transfers to
|
| 173 |
the unseen set; the F1 gap is mostly threshold / labeling-convention, not a domain failure.
|
| 174 |
Separately, a radiologist spot-checked **966** reports of the public test labels (857 fully
|