Hugging Face's logo

Spaces:
HTW-KI-Werkstatt
/
RamanBench
Running

App Files Community
Koddenbrock commited on
Commit
3ba03f4
·
1 Parent(s): 9ae710b

final results

Browse files
Files changed (9) hide show
  1. data/leaderboard_clf.csv +21 -21
  2. data/leaderboard_overall.csv +25 -25
  3. figures/efficiency_overview.png +2 -2
  4. figures/elo_ranking.png +2 -2
  5. figures/figure1.png +2 -2
  6. figures/improvability_vs_time.png +2 -2
  7. figures/metrics_vs_time.png +2 -2
  8. figures/pairwise_win_rates.png +2 -2
  9. figures/raman_examples.png +2 -2
data/leaderboard_clf.csv CHANGED
@@ -1,29 +1,29 @@
1
  model,Rank,Model,Category,F1,Bal. Acc.,Accuracy,MCC,Avg Rank,Score,Improvability %,# Datasets,Elo
2
- AUTOGLUON,1,"AutoGluon 1.5 (extreme, 4h)",AutoML,82.9%,83.7%,84.1%,70.5%,6.0,0.69,34.8,21,1454
3
- REALTABPFN-V2.5,2,TabPFN v2.5 (default),Tabular Foundation,82.4%,83.2%,83.5%,69.4%,7.0,0.66,38.9,21,1397
4
- TABICL,3,TabICL (default),Tabular Foundation,82.5%,83.3%,83.9%,70.3%,7.7,0.60,42.6,21,1360
5
- REZERONET,4,ReZeroNet (default),Raman-Specific,82.0%,83.0%,83.5%,69.8%,8.3,0.52,48.2,21,1338
6
- ROCKET,5,ROCKET (default),TS Classification,81.4%,81.5%,82.2%,68.0%,10.4,0.38,55.5,21,1257
7
- ARSENAL,6,Arsenal (default),TS Classification,80.7%,81.6%,82.2%,67.7%,10.4,0.42,58.3,21,1236
8
- REALTABPFN-V2,7,TabPFN v2 (default),Tabular Foundation,80.9%,81.7%,82.8%,69.8%,10.5,0.41,62.6,21,1256
9
- DEEPCNN,8,DeepCNN (default),Raman-Specific,79.0%,79.5%,80.0%,63.6%,12.1,0.38,52.2,21,1181
10
- TABM,9,TabM (default),Tabular Foundation,76.8%,77.4%,78.5%,60.3%,13.1,0.31,64.7,21,1153
11
- RAMANNET,10,RamanNet (default),Raman-Specific,77.9%,77.9%,79.6%,62.8%,13.2,0.33,59.1,21,1167
12
- TABDPT,11,TabDPT (default),Tabular Foundation,79.4%,80.1%,80.5%,64.2%,13.3,0.32,64.7,21,1151
13
- MITRA,12,MITRA (default),Tabular Foundation,78.0%,79.3%,79.9%,64.0%,13.4,0.33,66.5,21,1140
14
- NN_TORCH,13,NN (Torch) (default),Deep Learning,76.7%,77.2%,78.1%,58.9%,13.4,0.24,68.7,21,1145
15
  LR,14,Logistic Reg. (default),Traditional ML,76.3%,77.7%,78.0%,60.7%,14.0,0.33,58.6,21,1131
16
- FCRESNEXT,15,FC-ResNeXt (default),Deep Learning,78.7%,79.1%,79.8%,65.3%,14.5,0.24,65.0,21,1107
17
- REALMLP,16,RealMLP (default),Deep Learning,77.0%,76.9%,77.9%,59.9%,14.6,0.20,68.8,21,1113
18
  SANET,17,SANet (default),Raman-Specific,57.0%,61.5%,63.4%,43.9%,17.1,0.24,68.5,21,1001
19
- CAT,18,CatBoost (default),Gradient Boosting,74.3%,74.1%,75.6%,56.2%,17.2,0.05,73.3,21,1021
20
- FASTAI,19,FastAI (default),Deep Learning,74.5%,75.1%,76.8%,58.6%,17.3,0.15,69.7,21,1010
21
- COATNET,20,CoAtNet (default),Deep Learning,70.6%,72.2%,73.1%,57.1%,17.9,0.12,74.9,21,1015
22
- XGB,21,XGBoost (default),Gradient Boosting,73.4%,73.6%,75.1%,56.1%,17.9,0.09,71.9,21,981
23
  RF,22,Random Forest (default),Tree-based,74.0%,74.1%,75.6%,58.3%,18.1,0.09,72.4,21,1000
24
  RAMANFORMER,23,RamanFormer (default),Raman-Specific,53.4%,54.7%,56.8%,31.2%,18.2,0.20,68.8,21,935
25
- GBM,24,LightGBM (default),Gradient Boosting,72.0%,74.2%,76.1%,59.4%,18.4,0.05,73.9,21,972
26
  XT,25,Extra Trees (default),Tree-based,72.7%,72.5%,74.2%,55.7%,19.0,0.09,73.6,21,951
27
  KNN,26,KNN (default),Traditional ML,71.7%,72.9%,73.8%,55.7%,19.1,0.04,77.7,21,957
28
  PLS,27,PLS (default),Traditional ML,65.8%,67.6%,69.3%,51.7%,19.5,0.20,75.7,21,914
29
- RAMANTRANSFORMER,28,RamanTransformer (default),Raman-Specific,24.4%,35.5%,36.0%,8.0%,24.4,0.03,83.0,21,688
 
1
  model,Rank,Model,Category,F1,Bal. Acc.,Accuracy,MCC,Avg Rank,Score,Improvability %,# Datasets,Elo
2
+ AUTOGLUON,1,"AutoGluon 1.5 (extreme, 4h)",AutoML,83.4%,84.1%,84.5%,71.0%,5.6,0.71,33.5,21,1472
3
+ REALTABPFN-V2.5,2,TabPFN v2.5 (default),Tabular Foundation,82.4%,83.2%,83.5%,69.4%,7.1,0.65,38.9,21,1397
4
+ TABICL,3,TabICL (default),Tabular Foundation,82.5%,83.3%,83.9%,70.3%,7.7,0.60,42.6,21,1361
5
+ REZERONET,4,ReZeroNet (default),Raman-Specific,82.0%,83.0%,83.5%,69.8%,8.3,0.51,48.2,21,1338
6
+ REALTABPFN-V2,5,TabPFN v2 (default),Tabular Foundation,81.3%,82.1%,83.2%,70.2%,10.2,0.41,61.9,21,1265
7
+ ROCKET,6,ROCKET (default),TS Classification,81.4%,81.5%,82.2%,68.0%,10.5,0.38,55.5,21,1255
8
+ ARSENAL,7,ARSENAL (default),TS Classification,80.7%,81.6%,82.2%,67.7%,10.5,0.42,58.3,21,1234
9
+ DEEPCNN,8,DeepCNN (default),Raman-Specific,79.0%,79.5%,80.0%,63.6%,12.1,0.37,52.2,21,1182
10
+ TABM,9,TabM (default),Tabular Foundation,76.8%,77.4%,78.5%,60.3%,13.1,0.30,64.7,21,1154
11
+ RAMANNET,10,RamanNet (default),Raman-Specific,77.9%,77.9%,79.6%,62.8%,13.2,0.33,59.1,21,1168
12
+ MITRA,11,MITRA (default),Tabular Foundation,78.0%,79.3%,79.9%,64.0%,13.4,0.33,66.5,21,1140
13
+ NN_TORCH,12,NN (Torch) (default),Deep Learning,76.7%,77.2%,78.1%,58.9%,13.4,0.24,68.7,21,1145
14
+ TABDPT,13,TabDPT (default),Tabular Foundation,79.4%,80.1%,80.5%,64.2%,13.4,0.32,64.7,21,1149
15
  LR,14,Logistic Reg. (default),Traditional ML,76.3%,77.7%,78.0%,60.7%,14.0,0.33,58.6,21,1131
16
+ FCRESNEXT,15,FC-ResNeXt (default),Deep Learning,78.7%,79.1%,79.8%,65.3%,14.5,0.23,65.0,21,1108
17
+ REALMLP,16,RealMLP (default),Deep Learning,77.0%,76.9%,77.9%,59.9%,14.7,0.19,68.8,21,1113
18
  SANET,17,SANet (default),Raman-Specific,57.0%,61.5%,63.4%,43.9%,17.1,0.24,68.5,21,1001
19
+ CAT,18,CatBoost (default),Gradient Boosting,74.3%,74.1%,75.6%,56.2%,17.3,0.05,73.3,21,1019
20
+ FASTAI,19,FastAI (default),Deep Learning,74.5%,75.1%,76.8%,58.6%,17.3,0.15,69.7,21,1011
21
+ COATNET,20,CoAtNet (default),Deep Learning,70.6%,72.2%,73.1%,57.1%,18.0,0.11,74.9,21,1013
22
+ XGB,21,XGBoost (default),Gradient Boosting,73.4%,73.6%,75.1%,56.1%,18.0,0.09,71.9,21,978
23
  RF,22,Random Forest (default),Tree-based,74.0%,74.1%,75.6%,58.3%,18.1,0.09,72.4,21,1000
24
  RAMANFORMER,23,RamanFormer (default),Raman-Specific,53.4%,54.7%,56.8%,31.2%,18.2,0.20,68.8,21,935
25
+ GBM,24,LightGBM (default),Gradient Boosting,72.0%,74.2%,76.1%,59.4%,18.4,0.05,73.9,21,974
26
  XT,25,Extra Trees (default),Tree-based,72.7%,72.5%,74.2%,55.7%,19.0,0.09,73.6,21,951
27
  KNN,26,KNN (default),Traditional ML,71.7%,72.9%,73.8%,55.7%,19.1,0.04,77.7,21,957
28
  PLS,27,PLS (default),Traditional ML,65.8%,67.6%,69.3%,51.7%,19.5,0.20,75.7,21,914
29
+ RAMANTRANSFORMER,28,RamanTransformer (default),Raman-Specific,24.4%,35.5%,36.0%,8.0%,24.5,0.03,83.0,21,684
data/leaderboard_overall.csv CHANGED
@@ -1,29 +1,29 @@
1
  model_id,Rank,Model,Category,Elo,Score,Avg Rank,Improvability,Train Time s,Infer. s/1K
2
- AUTOGLUON,1,"AutoGluon 1.5 (extreme, 4h)",AutoML,1558,0.75,4.8,23.0,1776.3,220.7
3
  REALTABPFN-V2.5,2,TabPFN v2.5 (default),Tabular Foundation,1529,0.71,5.5,27.3,89.6,100.8
4
- TABICL,3,TabICL (default),Tabular Foundation,1443,0.62,6.5,33.3,39.9,130.3
5
- REALTABPFN-V2,4,TabPFN v2 (default),Tabular Foundation,1402,0.54,8.1,44.0,77.6,146.4
6
- MITRA,5,MITRA (default),Tabular Foundation,1312,0.42,10.3,49.5,827.4,626.5
7
- ROCKET,6,ROCKET (default),TS Classification,1244,0.38,10.4,55.5,1270.3,140.5
8
- ARSENAL,7,Arsenal (default),TS Classification,1240,0.42,10.4,58.3,2876.0,2790.3
9
- TABM,8,TabM (default),Tabular Foundation,1156,0.24,12.5,54.6,52.1,114.2
10
- TABDPT,9,TabDPT (default),Tabular Foundation,1136,0.3,12.5,52.9,12.3,80.2
11
- REZERONET,10,ReZeroNet (default),Raman-Specific,1133,0.39,10.3,45.2,44.6,23.1
12
- REALMLP,11,RealMLP (default),Deep Learning,1111,0.21,13.7,56.4,1935.5,71.1
13
- CAT,12,CatBoost (default),Gradient Boosting,1069,0.08,15.4,60.5,537.5,22.4
14
- NN_TORCH,13,NN (Torch) (default),Deep Learning,1066,0.21,13.8,57.0,1378.2,53.8
15
- XT,14,Extra Trees (default),Tree-based,1057,0.11,16.3,60.4,22.2,7.2
16
- RAMANNET,15,RamanNet (default),Raman-Specific,1028,0.22,14.3,54.0,73.9,13.3
17
- DEEPCNN,16,DeepCNN (default),Raman-Specific,1006,0.25,13.9,49.9,1280.1,63.5
18
- PLS,17,PLS (default),Traditional ML,1004,0.21,17.1,60.8,29.7,19.2
19
  LR,18,Logistic Reg. (default),Traditional ML,1002,0.27,14.6,53.3,37.2,41.2
20
- RF,19,Random Forest (default),Tree-based,1000,0.1,16.6,60.8,53.9,13.3
21
  KNN,20,KNN (default),Traditional ML,986,0.1,17.3,62.4,8.2,10.1
22
- RAMANFORMER,21,RamanFormer (default),Raman-Specific,979,0.18,17.2,59.1,501.2,102.7
23
- FASTAI,22,FastAI (default),Deep Learning,970,0.12,16.8,59.8,255.5,19.0
24
- COATNET,23,CoAtNet (default),Deep Learning,963,0.12,17.0,62.0,61.5,16.2
25
- GBM,24,LightGBM (default),Gradient Boosting,950,0.04,17.6,62.9,1453.5,7.3
26
- FCRESNEXT,25,FC-ResNeXt (default),Deep Learning,933,0.15,16.2,59.1,16.2,5.4
27
- XGB,26,XGBoost (default),Gradient Boosting,923,0.06,17.9,62.3,57.9,3.7
28
- SANET,27,SANet (default),Raman-Specific,798,0.14,19.1,64.6,37.9,39.7
29
- RAMANTRANSFORMER,28,RamanTransformer (default),Raman-Specific,710,0.04,23.1,74.0,429.8,134.8
 
1
  model_id,Rank,Model,Category,Elo,Score,Avg Rank,Improvability,Train Time s,Infer. s/1K
2
+ AUTOGLUON,1,"AutoGluon 1.5 (extreme, 4h)",AutoML,1562,0.76,4.6,22.4,1785.5,225.8
3
  REALTABPFN-V2.5,2,TabPFN v2.5 (default),Tabular Foundation,1529,0.71,5.5,27.3,89.6,100.8
4
+ TABICL,3,TabICL (default),Tabular Foundation,1444,0.62,6.5,33.3,40.0,130.7
5
+ REALTABPFN-V2,4,TabPFN v2 (default),Tabular Foundation,1404,0.54,8.0,43.7,120.6,151.0
6
+ MITRA,5,MITRA (default),Tabular Foundation,1312,0.42,10.3,49.5,827.6,626.5
7
+ ROCKET,6,ROCKET (default),TS Classification,1240,0.38,10.5,55.5,1270.3,140.5
8
+ ARSENAL,7,ARSENAL (default),TS Classification,1236,0.42,10.5,58.3,2876.0,2790.3
9
+ TABM,8,TabM (default),Tabular Foundation,1156,0.24,12.5,54.6,52.1,114.3
10
+ TABDPT,9,TabDPT (default),Tabular Foundation,1135,0.3,12.5,52.9,12.3,80.3
11
+ REZERONET,10,ReZeroNet (default),Raman-Specific,1133,0.38,10.3,45.2,46.7,23.1
12
+ REALMLP,11,RealMLP (default),Deep Learning,1111,0.2,13.7,56.4,1917.2,71.6
13
+ CAT,12,CatBoost (default),Gradient Boosting,1069,0.08,15.4,60.5,539.1,22.5
14
+ NN_TORCH,13,NN (Torch) (default),Deep Learning,1066,0.21,13.8,57.0,1393.9,53.5
15
+ XT,14,Extra Trees (default),Tree-based,1057,0.11,16.3,60.4,22.1,7.2
16
+ RAMANNET,15,RamanNet (default),Raman-Specific,1029,0.22,14.3,54.0,75.3,13.3
17
+ DEEPCNN,16,DeepCNN (default),Raman-Specific,1006,0.25,13.9,49.9,1284.0,63.4
18
+ PLS,17,PLS (default),Traditional ML,1004,0.21,17.1,60.8,29.9,19.2
19
  LR,18,Logistic Reg. (default),Traditional ML,1002,0.27,14.6,53.3,37.2,41.2
20
+ RF,19,Random Forest (default),Tree-based,1000,0.1,16.6,60.8,53.6,13.3
21
  KNN,20,KNN (default),Traditional ML,986,0.1,17.3,62.4,8.2,10.1
22
+ RAMANFORMER,21,RamanFormer (default),Raman-Specific,979,0.18,17.2,59.1,501.2,102.8
23
+ FASTAI,22,FastAI (default),Deep Learning,970,0.11,16.8,59.8,254.2,19.1
24
+ COATNET,23,CoAtNet (default),Deep Learning,963,0.12,17.0,62.0,62.8,16.2
25
+ GBM,24,LightGBM (default),Gradient Boosting,951,0.04,17.6,62.9,1458.0,7.3
26
+ FCRESNEXT,25,FC-ResNeXt (default),Deep Learning,933,0.14,16.2,59.1,16.3,5.4
27
+ XGB,26,XGBoost (default),Gradient Boosting,922,0.06,18.0,62.3,57.9,3.7
28
+ SANET,27,SANet (default),Raman-Specific,798,0.14,19.1,64.6,38.4,39.7
29
+ RAMANTRANSFORMER,28,RamanTransformer (default),Raman-Specific,710,0.04,23.1,74.0,449.5,135.0
figures/efficiency_overview.png CHANGED

Git LFS Details

  • SHA256: f0793f33a57f6c61e322c2d7536f464dbdeeda0f02c37b2a80e56429145aedf6
  • Pointer size: 131 Bytes
  • Size of remote file: 106 kB

Git LFS Details

  • SHA256: da1848646f04dff9884fb0fd9507511fcb7fa54aac808e9bc4c257d75b431eba
  • Pointer size: 131 Bytes
  • Size of remote file: 106 kB
figures/elo_ranking.png CHANGED

Git LFS Details

  • SHA256: 8729329c967751efcdc2ba2da25490cc35a0a9d2f5747ea3c8cd773b813810d6
  • Pointer size: 131 Bytes
  • Size of remote file: 156 kB

Git LFS Details

  • SHA256: dedb73781e546be33fcd12476a0589402fb2e085bf836e8f40d0d69e2c4b76f1
  • Pointer size: 131 Bytes
  • Size of remote file: 139 kB
figures/figure1.png CHANGED

Git LFS Details

  • SHA256: b2d8f9f31b708b17041af6adb85f0826fcaf07a033e31fd99b47a8e18f91db54
  • Pointer size: 131 Bytes
  • Size of remote file: 138 kB

Git LFS Details

  • SHA256: 9e39a90d5f6684c92272c2bfe912f76667eba8fb4dd49c69867ca6af20d27293
  • Pointer size: 131 Bytes
  • Size of remote file: 138 kB
figures/improvability_vs_time.png CHANGED

Git LFS Details

  • SHA256: bc09cef763a2f4b341af91989719e45b8e86de62b401b2afdac90f5a11406b25
  • Pointer size: 131 Bytes
  • Size of remote file: 162 kB

Git LFS Details

  • SHA256: 23387a859320bfa03dc2e31743f3561bbd2e3f337b14ebae2d2bc8e62e87b29a
  • Pointer size: 131 Bytes
  • Size of remote file: 162 kB
figures/metrics_vs_time.png CHANGED

Git LFS Details

  • SHA256: 7586af49ab084e536d414d45c0f37bac04e5ae17428bb60b19b77c48a7e7db47
  • Pointer size: 131 Bytes
  • Size of remote file: 162 kB

Git LFS Details

  • SHA256: bea4c31e5bc4b9e602cb789b3d857105e51246d7b81cf731cea10f7c01efa6e8
  • Pointer size: 131 Bytes
  • Size of remote file: 163 kB
figures/pairwise_win_rates.png CHANGED

Git LFS Details

  • SHA256: 3452eb6580488ec217fbaa30398941c1657995ac2d0cd1f7503c528957a66a22
  • Pointer size: 131 Bytes
  • Size of remote file: 281 kB

Git LFS Details

  • SHA256: 16abb66ab7e207faf3a99d452ffd0b28ac4ca4e4620d1552b1ebc04b384c588e
  • Pointer size: 131 Bytes
  • Size of remote file: 281 kB
figures/raman_examples.png CHANGED

Git LFS Details

  • SHA256: 73baa220482d4f07dcf9506b3e6a776df936c82cca8f8b9a54d20395dd5f7017
  • Pointer size: 131 Bytes
  • Size of remote file: 441 kB

Git LFS Details

  • SHA256: d68d5b47f0c0b7b4ae220c86170d7f093e3fefe5eeefc7ec9815e288867580e8
  • Pointer size: 131 Bytes
  • Size of remote file: 443 kB