Hugging Face's logo

capemox
/
splade-ettin-encoder-17m-gooaq

Feature Extraction
sentence-transformers
Safetensors
English
modernbert
sparse-encoder
sparse
splade
Generated from Trainer
dataset_size:99000
loss:SpladeLoss
loss:SparseMultipleNegativesRankingLoss
loss:FlopsLoss
Eval Results (legacy)
text-embeddings-inference
Model card Files
xet
 Community
capemox commited on
Commit
6153c7b
·
verified ·
1 Parent(s): 3909fdc

Add new SparseEncoder model

Browse files
Files changed (9) hide show
  1. 1_SpladePooling/config.json +5 -0
  2. README.md +1763 -0
  3. config.json +64 -0
  4. config_sentence_transformers.json +14 -0
  5. model.safetensors +3 -0
  6. modules.json +14 -0
  7. sentence_bert_config.json +10 -0
  8. tokenizer.json +0 -0
  9. tokenizer_config.json +16 -0
1_SpladePooling/config.json ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ {
2
+ "pooling_strategy": "max",
3
+ "activation_function": "relu",
4
+ "embedding_dimension": 50368
5
+ }
README.md ADDED
@@ -0,0 +1,1763 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ license: mit
5
+ tags:
6
+ - sentence-transformers
7
+ - sparse-encoder
8
+ - sparse
9
+ - splade
10
+ - generated_from_trainer
11
+ - dataset_size:99000
12
+ - loss:SpladeLoss
13
+ - loss:SparseMultipleNegativesRankingLoss
14
+ - loss:FlopsLoss
15
+ base_model: jhu-clsp/ettin-encoder-17m
16
+ widget:
17
+ - text: In addition to proof of identity, such as passport, birth certificate or adoption
18
+ certificate, and proof of your address, or a letter from your university or college
19
+ confirming your place, and bring it along with your other identity and address
20
+ verification documents to your local branch.
21
+ - text: An abiotic factor is a non-living part of an ecosystem that shapes its environment.
22
+ In a terrestrial ecosystem, examples might include temperature, light, and water.
23
+ In a marine ecosystem, abiotic factors would include salinity and ocean currents.
24
+ Abiotic and biotic factors work together to create a unique ecosystem.
25
+ - text: how many 16 oz bottles of water is a gallon?
26
+ - text: A vet assistant and vet tech both provide general animal care and assist with
27
+ the treatment of sick and injured animals. Depending on the setting, a vet assistant
28
+ may have more administrative responsibilities, while a vet tech may have more
29
+ clinical responsibilities.
30
+ - text: The two main stars are Alpha Centauri A and Alpha Centauri B, which form a
31
+ binary pair. They are an average of 4.3 light-years from Earth. The third star
32
+ is Proxima Centauri. It is about 4.22 light-years from Earth and is the closest
33
+ star other than the sun.
34
+ datasets:
35
+ - sentence-transformers/gooaq
36
+ pipeline_tag: feature-extraction
37
+ library_name: sentence-transformers
38
+ metrics:
39
+ - dot_accuracy@1
40
+ - dot_accuracy@3
41
+ - dot_accuracy@5
42
+ - dot_accuracy@10
43
+ - dot_precision@1
44
+ - dot_precision@3
45
+ - dot_precision@5
46
+ - dot_precision@10
47
+ - dot_recall@1
48
+ - dot_recall@3
49
+ - dot_recall@5
50
+ - dot_recall@10
51
+ - dot_ndcg@10
52
+ - dot_mrr@10
53
+ - dot_map@100
54
+ - query_active_dims
55
+ - query_sparsity_ratio
56
+ - corpus_active_dims
57
+ - corpus_sparsity_ratio
58
+ - avg_flops
59
+ model-index:
60
+ - name: splade-ettin-encoder-17m trained on GooAQ
61
+ results:
62
+ - task:
63
+ type: sparse-information-retrieval
64
+ name: Sparse Information Retrieval
65
+ dataset:
66
+ name: NanoMSMARCO 256
67
+ type: NanoMSMARCO_256
68
+ metrics:
69
+ - type: dot_accuracy@1
70
+ value: 0.2
71
+ name: Dot Accuracy@1
72
+ - type: dot_accuracy@3
73
+ value: 0.34
74
+ name: Dot Accuracy@3
75
+ - type: dot_accuracy@5
76
+ value: 0.44
77
+ name: Dot Accuracy@5
78
+ - type: dot_accuracy@10
79
+ value: 0.58
80
+ name: Dot Accuracy@10
81
+ - type: dot_precision@1
82
+ value: 0.2
83
+ name: Dot Precision@1
84
+ - type: dot_precision@3
85
+ value: 0.11333333333333333
86
+ name: Dot Precision@3
87
+ - type: dot_precision@5
88
+ value: 0.08800000000000001
89
+ name: Dot Precision@5
90
+ - type: dot_precision@10
91
+ value: 0.057999999999999996
92
+ name: Dot Precision@10
93
+ - type: dot_recall@1
94
+ value: 0.2
95
+ name: Dot Recall@1
96
+ - type: dot_recall@3
97
+ value: 0.34
98
+ name: Dot Recall@3
99
+ - type: dot_recall@5
100
+ value: 0.44
101
+ name: Dot Recall@5
102
+ - type: dot_recall@10
103
+ value: 0.58
104
+ name: Dot Recall@10
105
+ - type: dot_ndcg@10
106
+ value: 0.36748190085250804
107
+ name: Dot Ndcg@10
108
+ - type: dot_mrr@10
109
+ value: 0.30210317460317454
110
+ name: Dot Mrr@10
111
+ - type: dot_map@100
112
+ value: 0.3224995402831516
113
+ name: Dot Map@100
114
+ - type: query_active_dims
115
+ value: 252.0
116
+ name: Query Active Dims
117
+ - type: query_sparsity_ratio
118
+ value: 0.9949968233799238
119
+ name: Query Sparsity Ratio
120
+ - type: corpus_active_dims
121
+ value: 256.0
122
+ name: Corpus Active Dims
123
+ - type: corpus_sparsity_ratio
124
+ value: 0.9949174078780179
125
+ name: Corpus Sparsity Ratio
126
+ - type: avg_flops
127
+ value: 104.76126861572266
128
+ name: Avg Flops
129
+ - type: dot_accuracy@1
130
+ value: 0.2
131
+ name: Dot Accuracy@1
132
+ - type: dot_accuracy@3
133
+ value: 0.34
134
+ name: Dot Accuracy@3
135
+ - type: dot_accuracy@5
136
+ value: 0.42
137
+ name: Dot Accuracy@5
138
+ - type: dot_accuracy@10
139
+ value: 0.6
140
+ name: Dot Accuracy@10
141
+ - type: dot_precision@1
142
+ value: 0.2
143
+ name: Dot Precision@1
144
+ - type: dot_precision@3
145
+ value: 0.11333333333333333
146
+ name: Dot Precision@3
147
+ - type: dot_precision@5
148
+ value: 0.084
149
+ name: Dot Precision@5
150
+ - type: dot_precision@10
151
+ value: 0.06000000000000001
152
+ name: Dot Precision@10
153
+ - type: dot_recall@1
154
+ value: 0.2
155
+ name: Dot Recall@1
156
+ - type: dot_recall@3
157
+ value: 0.34
158
+ name: Dot Recall@3
159
+ - type: dot_recall@5
160
+ value: 0.42
161
+ name: Dot Recall@5
162
+ - type: dot_recall@10
163
+ value: 0.6
164
+ name: Dot Recall@10
165
+ - type: dot_ndcg@10
166
+ value: 0.37143757052695625
167
+ name: Dot Ndcg@10
168
+ - type: dot_mrr@10
169
+ value: 0.302047619047619
170
+ name: Dot Mrr@10
171
+ - type: dot_map@100
172
+ value: 0.3209443084622113
173
+ name: Dot Map@100
174
+ - type: query_active_dims
175
+ value: 252.10000610351562
176
+ name: Query Active Dims
177
+ - type: query_sparsity_ratio
178
+ value: 0.9949948378711977
179
+ name: Query Sparsity Ratio
180
+ - type: corpus_active_dims
181
+ value: 256.0
182
+ name: Corpus Active Dims
183
+ - type: corpus_sparsity_ratio
184
+ value: 0.9949174078780179
185
+ name: Corpus Sparsity Ratio
186
+ - type: avg_flops
187
+ value: 104.66670989990234
188
+ name: Avg Flops
189
+ - task:
190
+ type: sparse-information-retrieval
191
+ name: Sparse Information Retrieval
192
+ dataset:
193
+ name: NanoNFCorpus 256
194
+ type: NanoNFCorpus_256
195
+ metrics:
196
+ - type: dot_accuracy@1
197
+ value: 0.24
198
+ name: Dot Accuracy@1
199
+ - type: dot_accuracy@3
200
+ value: 0.32
201
+ name: Dot Accuracy@3
202
+ - type: dot_accuracy@5
203
+ value: 0.4
204
+ name: Dot Accuracy@5
205
+ - type: dot_accuracy@10
206
+ value: 0.52
207
+ name: Dot Accuracy@10
208
+ - type: dot_precision@1
209
+ value: 0.24
210
+ name: Dot Precision@1
211
+ - type: dot_precision@3
212
+ value: 0.18666666666666668
213
+ name: Dot Precision@3
214
+ - type: dot_precision@5
215
+ value: 0.18
216
+ name: Dot Precision@5
217
+ - type: dot_precision@10
218
+ value: 0.14400000000000002
219
+ name: Dot Precision@10
220
+ - type: dot_recall@1
221
+ value: 0.007416693841083107
222
+ name: Dot Recall@1
223
+ - type: dot_recall@3
224
+ value: 0.01728407183582485
225
+ name: Dot Recall@3
226
+ - type: dot_recall@5
227
+ value: 0.03449216239510752
228
+ name: Dot Recall@5
229
+ - type: dot_recall@10
230
+ value: 0.04713697363333782
231
+ name: Dot Recall@10
232
+ - type: dot_ndcg@10
233
+ value: 0.16244101089890484
234
+ name: Dot Ndcg@10
235
+ - type: dot_mrr@10
236
+ value: 0.3040714285714286
237
+ name: Dot Mrr@10
238
+ - type: dot_map@100
239
+ value: 0.04634431882731895
240
+ name: Dot Map@100
241
+ - type: query_active_dims
242
+ value: 255.6199951171875
243
+ name: Query Active Dims
244
+ - type: query_sparsity_ratio
245
+ value: 0.9949249524476416
246
+ name: Query Sparsity Ratio
247
+ - type: corpus_active_dims
248
+ value: 256.0
249
+ name: Corpus Active Dims
250
+ - type: corpus_sparsity_ratio
251
+ value: 0.9949174078780177
252
+ name: Corpus Sparsity Ratio
253
+ - type: avg_flops
254
+ value: 101.3379135131836
255
+ name: Avg Flops
256
+ - type: dot_accuracy@1
257
+ value: 0.22
258
+ name: Dot Accuracy@1
259
+ - type: dot_accuracy@3
260
+ value: 0.34
261
+ name: Dot Accuracy@3
262
+ - type: dot_accuracy@5
263
+ value: 0.38
264
+ name: Dot Accuracy@5
265
+ - type: dot_accuracy@10
266
+ value: 0.52
267
+ name: Dot Accuracy@10
268
+ - type: dot_precision@1
269
+ value: 0.22
270
+ name: Dot Precision@1
271
+ - type: dot_precision@3
272
+ value: 0.19333333333333333
273
+ name: Dot Precision@3
274
+ - type: dot_precision@5
275
+ value: 0.18
276
+ name: Dot Precision@5
277
+ - type: dot_precision@10
278
+ value: 0.14
279
+ name: Dot Precision@10
280
+ - type: dot_recall@1
281
+ value: 0.006928888963034326
282
+ name: Dot Recall@1
283
+ - type: dot_recall@3
284
+ value: 0.018744633333151052
285
+ name: Dot Recall@3
286
+ - type: dot_recall@5
287
+ value: 0.030603858996218224
288
+ name: Dot Recall@5
289
+ - type: dot_recall@10
290
+ value: 0.04577448609594998
291
+ name: Dot Recall@10
292
+ - type: dot_ndcg@10
293
+ value: 0.15706743654657598
294
+ name: Dot Ndcg@10
295
+ - type: dot_mrr@10
296
+ value: 0.29643650793650794
297
+ name: Dot Mrr@10
298
+ - type: dot_map@100
299
+ value: 0.046568619637846746
300
+ name: Dot Map@100
301
+ - type: query_active_dims
302
+ value: 255.6199951171875
303
+ name: Query Active Dims
304
+ - type: query_sparsity_ratio
305
+ value: 0.9949249524476416
306
+ name: Query Sparsity Ratio
307
+ - type: corpus_active_dims
308
+ value: 256.0
309
+ name: Corpus Active Dims
310
+ - type: corpus_sparsity_ratio
311
+ value: 0.9949174078780177
312
+ name: Corpus Sparsity Ratio
313
+ - type: avg_flops
314
+ value: 101.26585388183594
315
+ name: Avg Flops
316
+ - task:
317
+ type: sparse-information-retrieval
318
+ name: Sparse Information Retrieval
319
+ dataset:
320
+ name: NanoNQ 256
321
+ type: NanoNQ_256
322
+ metrics:
323
+ - type: dot_accuracy@1
324
+ value: 0.1
325
+ name: Dot Accuracy@1
326
+ - type: dot_accuracy@3
327
+ value: 0.32
328
+ name: Dot Accuracy@3
329
+ - type: dot_accuracy@5
330
+ value: 0.42
331
+ name: Dot Accuracy@5
332
+ - type: dot_accuracy@10
333
+ value: 0.52
334
+ name: Dot Accuracy@10
335
+ - type: dot_precision@1
336
+ value: 0.1
337
+ name: Dot Precision@1
338
+ - type: dot_precision@3
339
+ value: 0.10666666666666666
340
+ name: Dot Precision@3
341
+ - type: dot_precision@5
342
+ value: 0.084
343
+ name: Dot Precision@5
344
+ - type: dot_precision@10
345
+ value: 0.05800000000000001
346
+ name: Dot Precision@10
347
+ - type: dot_recall@1
348
+ value: 0.09
349
+ name: Dot Recall@1
350
+ - type: dot_recall@3
351
+ value: 0.29
352
+ name: Dot Recall@3
353
+ - type: dot_recall@5
354
+ value: 0.38
355
+ name: Dot Recall@5
356
+ - type: dot_recall@10
357
+ value: 0.51
358
+ name: Dot Recall@10
359
+ - type: dot_ndcg@10
360
+ value: 0.2892021613044471
361
+ name: Dot Ndcg@10
362
+ - type: dot_mrr@10
363
+ value: 0.2285793650793651
364
+ name: Dot Mrr@10
365
+ - type: dot_map@100
366
+ value: 0.22850363025154985
367
+ name: Dot Map@100
368
+ - type: query_active_dims
369
+ value: 256.0
370
+ name: Query Active Dims
371
+ - type: query_sparsity_ratio
372
+ value: 0.9949174078780177
373
+ name: Query Sparsity Ratio
374
+ - type: corpus_active_dims
375
+ value: 256.0
376
+ name: Corpus Active Dims
377
+ - type: corpus_sparsity_ratio
378
+ value: 0.9949174078780176
379
+ name: Corpus Sparsity Ratio
380
+ - type: avg_flops
381
+ value: 114.13877868652344
382
+ name: Avg Flops
383
+ - type: dot_accuracy@1
384
+ value: 0.1
385
+ name: Dot Accuracy@1
386
+ - type: dot_accuracy@3
387
+ value: 0.28
388
+ name: Dot Accuracy@3
389
+ - type: dot_accuracy@5
390
+ value: 0.4
391
+ name: Dot Accuracy@5
392
+ - type: dot_accuracy@10
393
+ value: 0.54
394
+ name: Dot Accuracy@10
395
+ - type: dot_precision@1
396
+ value: 0.1
397
+ name: Dot Precision@1
398
+ - type: dot_precision@3
399
+ value: 0.09333333333333334
400
+ name: Dot Precision@3
401
+ - type: dot_precision@5
402
+ value: 0.08000000000000002
403
+ name: Dot Precision@5
404
+ - type: dot_precision@10
405
+ value: 0.05800000000000001
406
+ name: Dot Precision@10
407
+ - type: dot_recall@1
408
+ value: 0.09
409
+ name: Dot Recall@1
410
+ - type: dot_recall@3
411
+ value: 0.25
412
+ name: Dot Recall@3
413
+ - type: dot_recall@5
414
+ value: 0.36
415
+ name: Dot Recall@5
416
+ - type: dot_recall@10
417
+ value: 0.52
418
+ name: Dot Recall@10
419
+ - type: dot_ndcg@10
420
+ value: 0.29254729188799355
421
+ name: Dot Ndcg@10
422
+ - type: dot_mrr@10
423
+ value: 0.2322698412698413
424
+ name: Dot Mrr@10
425
+ - type: dot_map@100
426
+ value: 0.23046052440382125
427
+ name: Dot Map@100
428
+ - type: query_active_dims
429
+ value: 256.0
430
+ name: Query Active Dims
431
+ - type: query_sparsity_ratio
432
+ value: 0.9949174078780177
433
+ name: Query Sparsity Ratio
434
+ - type: corpus_active_dims
435
+ value: 256.0
436
+ name: Corpus Active Dims
437
+ - type: corpus_sparsity_ratio
438
+ value: 0.9949174078780176
439
+ name: Corpus Sparsity Ratio
440
+ - type: avg_flops
441
+ value: 114.126953125
442
+ name: Avg Flops
443
+ - task:
444
+ type: sparse-nano-beir
445
+ name: Sparse Nano BEIR
446
+ dataset:
447
+ name: NanoBEIR mean 256
448
+ type: NanoBEIR_mean_256
449
+ metrics:
450
+ - type: dot_accuracy@1
451
+ value: 0.18000000000000002
452
+ name: Dot Accuracy@1
453
+ - type: dot_accuracy@3
454
+ value: 0.32666666666666666
455
+ name: Dot Accuracy@3
456
+ - type: dot_accuracy@5
457
+ value: 0.42
458
+ name: Dot Accuracy@5
459
+ - type: dot_accuracy@10
460
+ value: 0.54
461
+ name: Dot Accuracy@10
462
+ - type: dot_precision@1
463
+ value: 0.18000000000000002
464
+ name: Dot Precision@1
465
+ - type: dot_precision@3
466
+ value: 0.13555555555555554
467
+ name: Dot Precision@3
468
+ - type: dot_precision@5
469
+ value: 0.11733333333333335
470
+ name: Dot Precision@5
471
+ - type: dot_precision@10
472
+ value: 0.08666666666666667
473
+ name: Dot Precision@10
474
+ - type: dot_recall@1
475
+ value: 0.0991388979470277
476
+ name: Dot Recall@1
477
+ - type: dot_recall@3
478
+ value: 0.21576135727860826
479
+ name: Dot Recall@3
480
+ - type: dot_recall@5
481
+ value: 0.2848307207983692
482
+ name: Dot Recall@5
483
+ - type: dot_recall@10
484
+ value: 0.3790456578777793
485
+ name: Dot Recall@10
486
+ - type: dot_ndcg@10
487
+ value: 0.27304169101861997
488
+ name: Dot Ndcg@10
489
+ - type: dot_mrr@10
490
+ value: 0.27825132275132275
491
+ name: Dot Mrr@10
492
+ - type: dot_map@100
493
+ value: 0.1991158297873401
494
+ name: Dot Map@100
495
+ - type: query_active_dims
496
+ value: 254.53999837239584
497
+ name: Query Active Dims
498
+ - type: query_sparsity_ratio
499
+ value: 0.9949463945685276
500
+ name: Query Sparsity Ratio
501
+ - type: corpus_active_dims
502
+ value: 256.0
503
+ name: Corpus Active Dims
504
+ - type: corpus_sparsity_ratio
505
+ value: 0.9949174078780177
506
+ name: Corpus Sparsity Ratio
507
+ - type: avg_flops
508
+ value: 105.0486068725586
509
+ name: Avg Flops
510
+ - type: dot_accuracy@1
511
+ value: 0.2730612244897959
512
+ name: Dot Accuracy@1
513
+ - type: dot_accuracy@3
514
+ value: 0.47042386185243334
515
+ name: Dot Accuracy@3
516
+ - type: dot_accuracy@5
517
+ value: 0.556734693877551
518
+ name: Dot Accuracy@5
519
+ - type: dot_accuracy@10
520
+ value: 0.6553532182103611
521
+ name: Dot Accuracy@10
522
+ - type: dot_precision@1
523
+ value: 0.2730612244897959
524
+ name: Dot Precision@1
525
+ - type: dot_precision@3
526
+ value: 0.20167451596023023
527
+ name: Dot Precision@3
528
+ - type: dot_precision@5
529
+ value: 0.16646153846153847
530
+ name: Dot Precision@5
531
+ - type: dot_precision@10
532
+ value: 0.12314913657770801
533
+ name: Dot Precision@10
534
+ - type: dot_recall@1
535
+ value: 0.12970938218659442
536
+ name: Dot Recall@1
537
+ - type: dot_recall@3
538
+ value: 0.26896230737703214
539
+ name: Dot Recall@3
540
+ - type: dot_recall@5
541
+ value: 0.3370186758223455
542
+ name: Dot Recall@5
543
+ - type: dot_recall@10
544
+ value: 0.4261361667823346
545
+ name: Dot Recall@10
546
+ - type: dot_ndcg@10
547
+ value: 0.345879757383274
548
+ name: Dot Ndcg@10
549
+ - type: dot_mrr@10
550
+ value: 0.3924966360170442
551
+ name: Dot Mrr@10
552
+ - type: dot_map@100
553
+ value: 0.2731986853504099
554
+ name: Dot Map@100
555
+ - type: query_active_dims
556
+ value: 249.2619424828763
557
+ name: Query Active Dims
558
+ - type: query_sparsity_ratio
559
+ value: 0.9950511844329163
560
+ name: Query Sparsity Ratio
561
+ - type: corpus_active_dims
562
+ value: 254.11144146927373
563
+ name: Corpus Active Dims
564
+ - type: corpus_sparsity_ratio
565
+ value: 0.9949549030839169
566
+ name: Corpus Sparsity Ratio
567
+ - type: avg_flops
568
+ value: 109.02598571777344
569
+ name: Avg Flops
570
+ - task:
571
+ type: sparse-information-retrieval
572
+ name: Sparse Information Retrieval
573
+ dataset:
574
+ name: NanoClimateFEVER 256
575
+ type: NanoClimateFEVER_256
576
+ metrics:
577
+ - type: dot_accuracy@1
578
+ value: 0.2
579
+ name: Dot Accuracy@1
580
+ - type: dot_accuracy@3
581
+ value: 0.38
582
+ name: Dot Accuracy@3
583
+ - type: dot_accuracy@5
584
+ value: 0.42
585
+ name: Dot Accuracy@5
586
+ - type: dot_accuracy@10
587
+ value: 0.48
588
+ name: Dot Accuracy@10
589
+ - type: dot_precision@1
590
+ value: 0.2
591
+ name: Dot Precision@1
592
+ - type: dot_precision@3
593
+ value: 0.12666666666666665
594
+ name: Dot Precision@3
595
+ - type: dot_precision@5
596
+ value: 0.092
597
+ name: Dot Precision@5
598
+ - type: dot_precision@10
599
+ value: 0.066
600
+ name: Dot Precision@10
601
+ - type: dot_recall@1
602
+ value: 0.085
603
+ name: Dot Recall@1
604
+ - type: dot_recall@3
605
+ value: 0.17833333333333332
606
+ name: Dot Recall@3
607
+ - type: dot_recall@5
608
+ value: 0.2
609
+ name: Dot Recall@5
610
+ - type: dot_recall@10
611
+ value: 0.26
612
+ name: Dot Recall@10
613
+ - type: dot_ndcg@10
614
+ value: 0.215122111282969
615
+ name: Dot Ndcg@10
616
+ - type: dot_mrr@10
617
+ value: 0.29888888888888887
618
+ name: Dot Mrr@10
619
+ - type: dot_map@100
620
+ value: 0.1741526011558945
621
+ name: Dot Map@100
622
+ - type: query_active_dims
623
+ value: 256.0
624
+ name: Query Active Dims
625
+ - type: query_sparsity_ratio
626
+ value: 0.9949174078780177
627
+ name: Query Sparsity Ratio
628
+ - type: corpus_active_dims
629
+ value: 256.0
630
+ name: Corpus Active Dims
631
+ - type: corpus_sparsity_ratio
632
+ value: 0.9949174078780177
633
+ name: Corpus Sparsity Ratio
634
+ - type: avg_flops
635
+ value: 129.39976501464844
636
+ name: Avg Flops
637
+ - task:
638
+ type: sparse-information-retrieval
639
+ name: Sparse Information Retrieval
640
+ dataset:
641
+ name: NanoDBPedia 256
642
+ type: NanoDBPedia_256
643
+ metrics:
644
+ - type: dot_accuracy@1
645
+ value: 0.4
646
+ name: Dot Accuracy@1
647
+ - type: dot_accuracy@3
648
+ value: 0.66
649
+ name: Dot Accuracy@3
650
+ - type: dot_accuracy@5
651
+ value: 0.74
652
+ name: Dot Accuracy@5
653
+ - type: dot_accuracy@10
654
+ value: 0.84
655
+ name: Dot Accuracy@10
656
+ - type: dot_precision@1
657
+ value: 0.4
658
+ name: Dot Precision@1
659
+ - type: dot_precision@3
660
+ value: 0.3733333333333333
661
+ name: Dot Precision@3
662
+ - type: dot_precision@5
663
+ value: 0.34800000000000003
664
+ name: Dot Precision@5
665
+ - type: dot_precision@10
666
+ value: 0.324
667
+ name: Dot Precision@10
668
+ - type: dot_recall@1
669
+ value: 0.043815756303188326
670
+ name: Dot Recall@1
671
+ - type: dot_recall@3
672
+ value: 0.08533510688544765
673
+ name: Dot Recall@3
674
+ - type: dot_recall@5
675
+ value: 0.13559343564169452
676
+ name: Dot Recall@5
677
+ - type: dot_recall@10
678
+ value: 0.20974054299022687
679
+ name: Dot Recall@10
680
+ - type: dot_ndcg@10
681
+ value: 0.3792423494760472
682
+ name: Dot Ndcg@10
683
+ - type: dot_mrr@10
684
+ value: 0.5518571428571429
685
+ name: Dot Mrr@10
686
+ - type: dot_map@100
687
+ value: 0.2586818991199033
688
+ name: Dot Map@100
689
+ - type: query_active_dims
690
+ value: 253.9600067138672
691
+ name: Query Active Dims
692
+ - type: query_sparsity_ratio
693
+ value: 0.9949579096506935
694
+ name: Query Sparsity Ratio
695
+ - type: corpus_active_dims
696
+ value: 256.0
697
+ name: Corpus Active Dims
698
+ - type: corpus_sparsity_ratio
699
+ value: 0.9949174078780179
700
+ name: Corpus Sparsity Ratio
701
+ - type: avg_flops
702
+ value: 108.96640014648438
703
+ name: Avg Flops
704
+ - task:
705
+ type: sparse-information-retrieval
706
+ name: Sparse Information Retrieval
707
+ dataset:
708
+ name: NanoFEVER 256
709
+ type: NanoFEVER_256
710
+ metrics:
711
+ - type: dot_accuracy@1
712
+ value: 0.34
713
+ name: Dot Accuracy@1
714
+ - type: dot_accuracy@3
715
+ value: 0.7
716
+ name: Dot Accuracy@3
717
+ - type: dot_accuracy@5
718
+ value: 0.78
719
+ name: Dot Accuracy@5
720
+ - type: dot_accuracy@10
721
+ value: 0.92
722
+ name: Dot Accuracy@10
723
+ - type: dot_precision@1
724
+ value: 0.34
725
+ name: Dot Precision@1
726
+ - type: dot_precision@3
727
+ value: 0.2333333333333333
728
+ name: Dot Precision@3
729
+ - type: dot_precision@5
730
+ value: 0.15600000000000003
731
+ name: Dot Precision@5
732
+ - type: dot_precision@10
733
+ value: 0.09399999999999999
734
+ name: Dot Precision@10
735
+ - type: dot_recall@1
736
+ value: 0.30666666666666664
737
+ name: Dot Recall@1
738
+ - type: dot_recall@3
739
+ value: 0.6566666666666666
740
+ name: Dot Recall@3
741
+ - type: dot_recall@5
742
+ value: 0.7366666666666666
743
+ name: Dot Recall@5
744
+ - type: dot_recall@10
745
+ value: 0.8766666666666667
746
+ name: Dot Recall@10
747
+ - type: dot_ndcg@10
748
+ value: 0.5965342963229342
749
+ name: Dot Ndcg@10
750
+ - type: dot_mrr@10
751
+ value: 0.5312936507936507
752
+ name: Dot Mrr@10
753
+ - type: dot_map@100
754
+ value: 0.4997229758545548
755
+ name: Dot Map@100
756
+ - type: query_active_dims
757
+ value: 256.0
758
+ name: Query Active Dims
759
+ - type: query_sparsity_ratio
760
+ value: 0.9949174078780177
761
+ name: Query Sparsity Ratio
762
+ - type: corpus_active_dims
763
+ value: 256.0
764
+ name: Corpus Active Dims
765
+ - type: corpus_sparsity_ratio
766
+ value: 0.9949174078780179
767
+ name: Corpus Sparsity Ratio
768
+ - type: avg_flops
769
+ value: 120.7999038696289
770
+ name: Avg Flops
771
+ - task:
772
+ type: sparse-information-retrieval
773
+ name: Sparse Information Retrieval
774
+ dataset:
775
+ name: NanoFiQA2018 256
776
+ type: NanoFiQA2018_256
777
+ metrics:
778
+ - type: dot_accuracy@1
779
+ value: 0.18
780
+ name: Dot Accuracy@1
781
+ - type: dot_accuracy@3
782
+ value: 0.3
783
+ name: Dot Accuracy@3
784
+ - type: dot_accuracy@5
785
+ value: 0.4
786
+ name: Dot Accuracy@5
787
+ - type: dot_accuracy@10
788
+ value: 0.44
789
+ name: Dot Accuracy@10
790
+ - type: dot_precision@1
791
+ value: 0.18
792
+ name: Dot Precision@1
793
+ - type: dot_precision@3
794
+ value: 0.14
795
+ name: Dot Precision@3
796
+ - type: dot_precision@5
797
+ value: 0.124
798
+ name: Dot Precision@5
799
+ - type: dot_precision@10
800
+ value: 0.07400000000000001
801
+ name: Dot Precision@10
802
+ - type: dot_recall@1
803
+ value: 0.0821904761904762
804
+ name: Dot Recall@1
805
+ - type: dot_recall@3
806
+ value: 0.17335714285714285
807
+ name: Dot Recall@3
808
+ - type: dot_recall@5
809
+ value: 0.24057936507936506
810
+ name: Dot Recall@5
811
+ - type: dot_recall@10
812
+ value: 0.2906349206349207
813
+ name: Dot Recall@10
814
+ - type: dot_ndcg@10
815
+ value: 0.2236717246585837
816
+ name: Dot Ndcg@10
817
+ - type: dot_mrr@10
818
+ value: 0.25316666666666665
819
+ name: Dot Mrr@10
820
+ - type: dot_map@100
821
+ value: 0.18633896011707374
822
+ name: Dot Map@100
823
+ - type: query_active_dims
824
+ value: 240.22000122070312
825
+ name: Query Active Dims
826
+ - type: query_sparsity_ratio
827
+ value: 0.9952307020088011
828
+ name: Query Sparsity Ratio
829
+ - type: corpus_active_dims
830
+ value: 256.0
831
+ name: Corpus Active Dims
832
+ - type: corpus_sparsity_ratio
833
+ value: 0.9949174078780176
834
+ name: Corpus Sparsity Ratio
835
+ - type: avg_flops
836
+ value: 116.01726531982422
837
+ name: Avg Flops
838
+ - task:
839
+ type: sparse-information-retrieval
840
+ name: Sparse Information Retrieval
841
+ dataset:
842
+ name: NanoHotpotQA 256
843
+ type: NanoHotpotQA_256
844
+ metrics:
845
+ - type: dot_accuracy@1
846
+ value: 0.58
847
+ name: Dot Accuracy@1
848
+ - type: dot_accuracy@3
849
+ value: 0.7
850
+ name: Dot Accuracy@3
851
+ - type: dot_accuracy@5
852
+ value: 0.78
853
+ name: Dot Accuracy@5
854
+ - type: dot_accuracy@10
855
+ value: 0.84
856
+ name: Dot Accuracy@10
857
+ - type: dot_precision@1
858
+ value: 0.58
859
+ name: Dot Precision@1
860
+ - type: dot_precision@3
861
+ value: 0.32
862
+ name: Dot Precision@3
863
+ - type: dot_precision@5
864
+ value: 0.21599999999999994
865
+ name: Dot Precision@5
866
+ - type: dot_precision@10
867
+ value: 0.12599999999999997
868
+ name: Dot Precision@10
869
+ - type: dot_recall@1
870
+ value: 0.29
871
+ name: Dot Recall@1
872
+ - type: dot_recall@3
873
+ value: 0.48
874
+ name: Dot Recall@3
875
+ - type: dot_recall@5
876
+ value: 0.54
877
+ name: Dot Recall@5
878
+ - type: dot_recall@10
879
+ value: 0.63
880
+ name: Dot Recall@10
881
+ - type: dot_ndcg@10
882
+ value: 0.5600780523888909
883
+ name: Dot Ndcg@10
884
+ - type: dot_mrr@10
885
+ value: 0.6586666666666666
886
+ name: Dot Mrr@10
887
+ - type: dot_map@100
888
+ value: 0.49395504821298336
889
+ name: Dot Map@100
890
+ - type: query_active_dims
891
+ value: 256.0
892
+ name: Query Active Dims
893
+ - type: query_sparsity_ratio
894
+ value: 0.9949174078780177
895
+ name: Query Sparsity Ratio
896
+ - type: corpus_active_dims
897
+ value: 256.0
898
+ name: Corpus Active Dims
899
+ - type: corpus_sparsity_ratio
900
+ value: 0.9949174078780176
901
+ name: Corpus Sparsity Ratio
902
+ - type: avg_flops
903
+ value: 114.6017074584961
904
+ name: Avg Flops
905
+ - task:
906
+ type: sparse-information-retrieval
907
+ name: Sparse Information Retrieval
908
+ dataset:
909
+ name: NanoQuoraRetrieval 256
910
+ type: NanoQuoraRetrieval_256
911
+ metrics:
912
+ - type: dot_accuracy@1
913
+ value: 0.16
914
+ name: Dot Accuracy@1
915
+ - type: dot_accuracy@3
916
+ value: 0.38
917
+ name: Dot Accuracy@3
918
+ - type: dot_accuracy@5
919
+ value: 0.4
920
+ name: Dot Accuracy@5
921
+ - type: dot_accuracy@10
922
+ value: 0.46
923
+ name: Dot Accuracy@10
924
+ - type: dot_precision@1
925
+ value: 0.16
926
+ name: Dot Precision@1
927
+ - type: dot_precision@3
928
+ value: 0.12666666666666665
929
+ name: Dot Precision@3
930
+ - type: dot_precision@5
931
+ value: 0.08
932
+ name: Dot Precision@5
933
+ - type: dot_precision@10
934
+ value: 0.04800000000000001
935
+ name: Dot Precision@10
936
+ - type: dot_recall@1
937
+ value: 0.16
938
+ name: Dot Recall@1
939
+ - type: dot_recall@3
940
+ value: 0.354
941
+ name: Dot Recall@3
942
+ - type: dot_recall@5
943
+ value: 0.374
944
+ name: Dot Recall@5
945
+ - type: dot_recall@10
946
+ value: 0.444
947
+ name: Dot Recall@10
948
+ - type: dot_ndcg@10
949
+ value: 0.31465958594721516
950
+ name: Dot Ndcg@10
951
+ - type: dot_mrr@10
952
+ value: 0.2770238095238095
953
+ name: Dot Mrr@10
954
+ - type: dot_map@100
955
+ value: 0.2826009380599248
956
+ name: Dot Map@100
957
+ - type: query_active_dims
958
+ value: 229.0
959
+ name: Query Active Dims
960
+ - type: query_sparsity_ratio
961
+ value: 0.9954534625158831
962
+ name: Query Sparsity Ratio
963
+ - type: corpus_active_dims
964
+ value: 234.99563598632812
965
+ name: Corpus Active Dims
966
+ - type: corpus_sparsity_ratio
967
+ value: 0.9953344259056082
968
+ name: Corpus Sparsity Ratio
969
+ - type: avg_flops
970
+ value: 119.71977996826172
971
+ name: Avg Flops
972
+ - task:
973
+ type: sparse-information-retrieval
974
+ name: Sparse Information Retrieval
975
+ dataset:
976
+ name: NanoSCIDOCS 256
977
+ type: NanoSCIDOCS_256
978
+ metrics:
979
+ - type: dot_accuracy@1
980
+ value: 0.34
981
+ name: Dot Accuracy@1
982
+ - type: dot_accuracy@3
983
+ value: 0.5
984
+ name: Dot Accuracy@3
985
+ - type: dot_accuracy@5
986
+ value: 0.58
987
+ name: Dot Accuracy@5
988
+ - type: dot_accuracy@10
989
+ value: 0.72
990
+ name: Dot Accuracy@10
991
+ - type: dot_precision@1
992
+ value: 0.34
993
+ name: Dot Precision@1
994
+ - type: dot_precision@3
995
+ value: 0.22
996
+ name: Dot Precision@3
997
+ - type: dot_precision@5
998
+ value: 0.18
999
+ name: Dot Precision@5
1000
+ - type: dot_precision@10
1001
+ value: 0.13799999999999998
1002
+ name: Dot Precision@10
1003
+ - type: dot_recall@1
1004
+ value: 0.07266666666666667
1005
+ name: Dot Recall@1
1006
+ - type: dot_recall@3
1007
+ value: 0.13766666666666666
1008
+ name: Dot Recall@3
1009
+ - type: dot_recall@5
1010
+ value: 0.18566666666666667
1011
+ name: Dot Recall@5
1012
+ - type: dot_recall@10
1013
+ value: 0.2836666666666667
1014
+ name: Dot Recall@10
1015
+ - type: dot_ndcg@10
1016
+ value: 0.2672568147859396
1017
+ name: Dot Ndcg@10
1018
+ - type: dot_mrr@10
1019
+ value: 0.4424126984126984
1020
+ name: Dot Mrr@10
1021
+ - type: dot_map@100
1022
+ value: 0.19041553956973106
1023
+ name: Dot Map@100
1024
+ - type: query_active_dims
1025
+ value: 256.0
1026
+ name: Query Active Dims
1027
+ - type: query_sparsity_ratio
1028
+ value: 0.9949174078780177
1029
+ name: Query Sparsity Ratio
1030
+ - type: corpus_active_dims
1031
+ value: 256.0
1032
+ name: Corpus Active Dims
1033
+ - type: corpus_sparsity_ratio
1034
+ value: 0.9949174078780177
1035
+ name: Corpus Sparsity Ratio
1036
+ - type: avg_flops
1037
+ value: 123.3700942993164
1038
+ name: Avg Flops
1039
+ - task:
1040
+ type: sparse-information-retrieval
1041
+ name: Sparse Information Retrieval
1042
+ dataset:
1043
+ name: NanoArguAna 256
1044
+ type: NanoArguAna_256
1045
+ metrics:
1046
+ - type: dot_accuracy@1
1047
+ value: 0.04
1048
+ name: Dot Accuracy@1
1049
+ - type: dot_accuracy@3
1050
+ value: 0.32
1051
+ name: Dot Accuracy@3
1052
+ - type: dot_accuracy@5
1053
+ value: 0.42
1054
+ name: Dot Accuracy@5
1055
+ - type: dot_accuracy@10
1056
+ value: 0.48
1057
+ name: Dot Accuracy@10
1058
+ - type: dot_precision@1
1059
+ value: 0.04
1060
+ name: Dot Precision@1
1061
+ - type: dot_precision@3
1062
+ value: 0.10666666666666666
1063
+ name: Dot Precision@3
1064
+ - type: dot_precision@5
1065
+ value: 0.084
1066
+ name: Dot Precision@5
1067
+ - type: dot_precision@10
1068
+ value: 0.04800000000000001
1069
+ name: Dot Precision@10
1070
+ - type: dot_recall@1
1071
+ value: 0.04
1072
+ name: Dot Recall@1
1073
+ - type: dot_recall@3
1074
+ value: 0.32
1075
+ name: Dot Recall@3
1076
+ - type: dot_recall@5
1077
+ value: 0.42
1078
+ name: Dot Recall@5
1079
+ - type: dot_recall@10
1080
+ value: 0.48
1081
+ name: Dot Recall@10
1082
+ - type: dot_ndcg@10
1083
+ value: 0.2567073862056186
1084
+ name: Dot Ndcg@10
1085
+ - type: dot_mrr@10
1086
+ value: 0.18450000000000003
1087
+ name: Dot Mrr@10
1088
+ - type: dot_map@100
1089
+ value: 0.18984106283244218
1090
+ name: Dot Map@100
1091
+ - type: query_active_dims
1092
+ value: 256.0
1093
+ name: Query Active Dims
1094
+ - type: query_sparsity_ratio
1095
+ value: 0.9949174078780177
1096
+ name: Query Sparsity Ratio
1097
+ - type: corpus_active_dims
1098
+ value: 256.0
1099
+ name: Corpus Active Dims
1100
+ - type: corpus_sparsity_ratio
1101
+ value: 0.9949174078780177
1102
+ name: Corpus Sparsity Ratio
1103
+ - type: avg_flops
1104
+ value: 150.7188262939453
1105
+ name: Avg Flops
1106
+ - task:
1107
+ type: sparse-information-retrieval
1108
+ name: Sparse Information Retrieval
1109
+ dataset:
1110
+ name: NanoSciFact 256
1111
+ type: NanoSciFact_256
1112
+ metrics:
1113
+ - type: dot_accuracy@1
1114
+ value: 0.3
1115
+ name: Dot Accuracy@1
1116
+ - type: dot_accuracy@3
1117
+ value: 0.44
1118
+ name: Dot Accuracy@3
1119
+ - type: dot_accuracy@5
1120
+ value: 0.64
1121
+ name: Dot Accuracy@5
1122
+ - type: dot_accuracy@10
1123
+ value: 0.7
1124
+ name: Dot Accuracy@10
1125
+ - type: dot_precision@1
1126
+ value: 0.3
1127
+ name: Dot Precision@1
1128
+ - type: dot_precision@3
1129
+ value: 0.1533333333333333
1130
+ name: Dot Precision@3
1131
+ - type: dot_precision@5
1132
+ value: 0.14
1133
+ name: Dot Precision@5
1134
+ - type: dot_precision@10
1135
+ value: 0.078
1136
+ name: Dot Precision@10
1137
+ - type: dot_recall@1
1138
+ value: 0.275
1139
+ name: Dot Recall@1
1140
+ - type: dot_recall@3
1141
+ value: 0.42
1142
+ name: Dot Recall@3
1143
+ - type: dot_recall@5
1144
+ value: 0.61
1145
+ name: Dot Recall@5
1146
+ - type: dot_recall@10
1147
+ value: 0.68
1148
+ name: Dot Recall@10
1149
+ - type: dot_ndcg@10
1150
+ value: 0.4730952763241869
1151
+ name: Dot Ndcg@10
1152
+ - type: dot_mrr@10
1153
+ value: 0.41774603174603164
1154
+ name: Dot Mrr@10
1155
+ - type: dot_map@100
1156
+ value: 0.40792257335430643
1157
+ name: Dot Map@100
1158
+ - type: query_active_dims
1159
+ value: 256.0
1160
+ name: Query Active Dims
1161
+ - type: query_sparsity_ratio
1162
+ value: 0.9949174078780177
1163
+ name: Query Sparsity Ratio
1164
+ - type: corpus_active_dims
1165
+ value: 256.0
1166
+ name: Corpus Active Dims
1167
+ - type: corpus_sparsity_ratio
1168
+ value: 0.9949174078780179
1169
+ name: Corpus Sparsity Ratio
1170
+ - type: avg_flops
1171
+ value: 129.66539001464844
1172
+ name: Avg Flops
1173
+ - task:
1174
+ type: sparse-information-retrieval
1175
+ name: Sparse Information Retrieval
1176
+ dataset:
1177
+ name: NanoTouche2020 256
1178
+ type: NanoTouche2020_256
1179
+ metrics:
1180
+ - type: dot_accuracy@1
1181
+ value: 0.4897959183673469
1182
+ name: Dot Accuracy@1
1183
+ - type: dot_accuracy@3
1184
+ value: 0.7755102040816326
1185
+ name: Dot Accuracy@3
1186
+ - type: dot_accuracy@5
1187
+ value: 0.8775510204081632
1188
+ name: Dot Accuracy@5
1189
+ - type: dot_accuracy@10
1190
+ value: 0.9795918367346939
1191
+ name: Dot Accuracy@10
1192
+ - type: dot_precision@1
1193
+ value: 0.4897959183673469
1194
+ name: Dot Precision@1
1195
+ - type: dot_precision@3
1196
+ value: 0.4217687074829932
1197
+ name: Dot Precision@3
1198
+ - type: dot_precision@5
1199
+ value: 0.4
1200
+ name: Dot Precision@5
1201
+ - type: dot_precision@10
1202
+ value: 0.34693877551020413
1203
+ name: Dot Precision@10
1204
+ - type: dot_recall@1
1205
+ value: 0.03395351363569543
1206
+ name: Dot Recall@1
1207
+ - type: dot_recall@3
1208
+ value: 0.08240644615901016
1209
+ name: Dot Recall@3
1210
+ - type: dot_recall@5
1211
+ value: 0.12813279263988042
1212
+ name: Dot Recall@5
1213
+ - type: dot_recall@10
1214
+ value: 0.21928688511591962
1215
+ name: Dot Recall@10
1216
+ - type: dot_ndcg@10
1217
+ value: 0.38901694962865085
1218
+ name: Dot Ndcg@10
1219
+ - type: dot_mrr@10
1220
+ value: 0.6561467444120505
1221
+ name: Dot Mrr@10
1222
+ - type: dot_map@100
1223
+ value: 0.2699778587746352
1224
+ name: Dot Map@100
1225
+ - type: query_active_dims
1226
+ value: 216.85714721679688
1227
+ name: Query Active Dims
1228
+ - type: query_sparsity_ratio
1229
+ value: 0.9956945452029702
1230
+ name: Query Sparsity Ratio
1231
+ - type: corpus_active_dims
1232
+ value: 255.92410278320312
1233
+ name: Corpus Active Dims
1234
+ - type: corpus_sparsity_ratio
1235
+ value: 0.9949189147319091
1236
+ name: Corpus Sparsity Ratio
1237
+ - type: avg_flops
1238
+ value: 86.41167449951172
1239
+ name: Avg Flops
1240
+ ---
1241
+
1242
+ # splade-ettin-encoder-17m trained on GooAQ
1243
+
1244
+ This is a [SPLADE Sparse Encoder](https://www.sbert.net/docs/sparse_encoder/usage/usage.html) model finetuned from [jhu-clsp/ettin-encoder-17m](https://huggingface.co/jhu-clsp/ettin-encoder-17m) on the [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) dataset using the [sentence-transformers](https://www.SBERT.net) library. It maps sentences & paragraphs to a 50368-dimensional sparse vector space and can be used for semantic search and sparse retrieval.
1245
+ ## Model Details
1246
+
1247
+ ### Model Description
1248
+ - **Model Type:** SPLADE Sparse Encoder
1249
+ - **Base model:** [jhu-clsp/ettin-encoder-17m](https://huggingface.co/jhu-clsp/ettin-encoder-17m) <!-- at revision 987607455c61e7a5bbc85f7758e0512ea6d0ae4c -->
1250
+ - **Maximum Sequence Length:** 256 tokens
1251
+ - **Output Dimensionality:** 50368 dimensions
1252
+ - **Similarity Function:** Dot Product
1253
+ - **Supported Modality:** Text
1254
+ - **Training Dataset:**
1255
+ - [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq)
1256
+ - **Language:** en
1257
+ - **License:** mit
1258
+
1259
+ ### Model Sources
1260
+
1261
+ - **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
1262
+ - **Documentation:** [Sparse Encoder Documentation](https://www.sbert.net/docs/sparse_encoder/usage/usage.html)
1263
+ - **Repository:** [Sentence Transformers on GitHub](https://github.com/huggingface/sentence-transformers)
1264
+ - **Hugging Face:** [Sparse Encoders on Hugging Face](https://huggingface.co/models?library=sentence-transformers&other=sparse-encoder)
1265
+
1266
+ ### Full Model Architecture
1267
+
1268
+ ```
1269
+ SparseEncoder(
1270
+ (0): Transformer({'transformer_task': 'fill-mask', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'logits'}}, 'module_output_name': 'token_embeddings', 'architecture': 'ModernBertForMaskedLM'})
1271
+ (1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'embedding_dimension': 50368})
1272
+ )
1273
+ ```
1274
+
1275
+ ## Usage
1276
+
1277
+ ### Direct Usage (Sentence Transformers)
1278
+
1279
+ First install the Sentence Transformers library:
1280
+
1281
+ ```bash
1282
+ pip install -U sentence-transformers
1283
+ ```
1284
+ Then you can load this model and run inference.
1285
+ ```python
1286
+ from sentence_transformers import SparseEncoder
1287
+
1288
+ # Download from the 🤗 Hub
1289
+ model = SparseEncoder("capemox/splade-ettin-encoder-17m-gooaq")
1290
+ # Run inference
1291
+ sentences = [
1292
+ 'how close is the closest star?',
1293
+ 'The two main stars are Alpha Centauri A and Alpha Centauri B, which form a binary pair. They are an average of 4.3 light-years from Earth. The third star is Proxima Centauri. It is about 4.22 light-years from Earth and is the closest star other than the sun.',
1294
+ 'One gallon can of paint will cover up to 400 square feet, which is enough to cover a small room like a bathroom. Two gallon cans of paint cover up to 800 square feet, which is enough to cover an average size room. This is the most common amount needed, especially when considering second coat coverage.',
1295
+ ]
1296
+ embeddings = model.encode(sentences)
1297
+ print(embeddings.shape)
1298
+ # [3, 50368]
1299
+
1300
+ # Get the similarity scores for the embeddings
1301
+ similarities = model.similarity(embeddings, embeddings)
1302
+ print(similarities)
1303
+ # tensor([[144.9772, 158.3885, 140.6785],
1304
+ # [158.3885, 496.0056, 344.2902],
1305
+ # [140.6785, 344.2902, 528.3322]])
1306
+ ```
1307
+ <!--
1308
+ ### Direct Usage (Transformers)
1309
+
1310
+ <details><summary>Click to see the direct usage in Transformers</summary>
1311
+
1312
+ </details>
1313
+ -->
1314
+
1315
+ <!--
1316
+ ### Downstream Usage (Sentence Transformers)
1317
+
1318
+ You can finetune this model on your own dataset.
1319
+
1320
+ <details><summary>Click to expand</summary>
1321
+
1322
+ </details>
1323
+ -->
1324
+
1325
+ <!--
1326
+ ### Out-of-Scope Use
1327
+
1328
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
1329
+ -->
1330
+
1331
+ ## Evaluation
1332
+
1333
+ ### Metrics
1334
+
1335
+ #### Sparse Information Retrieval
1336
+
1337
+ * Datasets: `NanoMSMARCO_256`, `NanoNFCorpus_256`, `NanoNQ_256`, `NanoClimateFEVER_256`, `NanoDBPedia_256`, `NanoFEVER_256`, `NanoFiQA2018_256`, `NanoHotpotQA_256`, `NanoMSMARCO_256`, `NanoNFCorpus_256`, `NanoNQ_256`, `NanoQuoraRetrieval_256`, `NanoSCIDOCS_256`, `NanoArguAna_256`, `NanoSciFact_256` and `NanoTouche2020_256`
1338
+ * Evaluated with [<code>SparseInformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sparse_encoder/evaluation.html#sentence_transformers.sparse_encoder.evaluation.SparseInformationRetrievalEvaluator) with these parameters:
1339
+ ```json
1340
+ {
1341
+ "max_active_dims": 256
1342
+ }
1343
+ ```
1344
+
1345
+ | Metric | NanoMSMARCO_256 | NanoNFCorpus_256 | NanoNQ_256 | NanoClimateFEVER_256 | NanoDBPedia_256 | NanoFEVER_256 | NanoFiQA2018_256 | NanoHotpotQA_256 | NanoQuoraRetrieval_256 | NanoSCIDOCS_256 | NanoArguAna_256 | NanoSciFact_256 | NanoTouche2020_256 |
1346
+ |:----------------------|:----------------|:-----------------|:-----------|:---------------------|:----------------|:--------------|:-----------------|:-----------------|:-----------------------|:----------------|:----------------|:----------------|:-------------------|
1347
+ | dot_accuracy@1 | 0.2 | 0.22 | 0.1 | 0.2 | 0.4 | 0.34 | 0.18 | 0.58 | 0.16 | 0.34 | 0.04 | 0.3 | 0.4898 |
1348
+ | dot_accuracy@3 | 0.34 | 0.34 | 0.28 | 0.38 | 0.66 | 0.7 | 0.3 | 0.7 | 0.38 | 0.5 | 0.32 | 0.44 | 0.7755 |
1349
+ | dot_accuracy@5 | 0.42 | 0.38 | 0.4 | 0.42 | 0.74 | 0.78 | 0.4 | 0.78 | 0.4 | 0.58 | 0.42 | 0.64 | 0.8776 |
1350
+ | dot_accuracy@10 | 0.6 | 0.52 | 0.54 | 0.48 | 0.84 | 0.92 | 0.44 | 0.84 | 0.46 | 0.72 | 0.48 | 0.7 | 0.9796 |
1351
+ | dot_precision@1 | 0.2 | 0.22 | 0.1 | 0.2 | 0.4 | 0.34 | 0.18 | 0.58 | 0.16 | 0.34 | 0.04 | 0.3 | 0.4898 |
1352
+ | dot_precision@3 | 0.1133 | 0.1933 | 0.0933 | 0.1267 | 0.3733 | 0.2333 | 0.14 | 0.32 | 0.1267 | 0.22 | 0.1067 | 0.1533 | 0.4218 |
1353
+ | dot_precision@5 | 0.084 | 0.18 | 0.08 | 0.092 | 0.348 | 0.156 | 0.124 | 0.216 | 0.08 | 0.18 | 0.084 | 0.14 | 0.4 |
1354
+ | dot_precision@10 | 0.06 | 0.14 | 0.058 | 0.066 | 0.324 | 0.094 | 0.074 | 0.126 | 0.048 | 0.138 | 0.048 | 0.078 | 0.3469 |
1355
+ | dot_recall@1 | 0.2 | 0.0069 | 0.09 | 0.085 | 0.0438 | 0.3067 | 0.0822 | 0.29 | 0.16 | 0.0727 | 0.04 | 0.275 | 0.034 |
1356
+ | dot_recall@3 | 0.34 | 0.0187 | 0.25 | 0.1783 | 0.0853 | 0.6567 | 0.1734 | 0.48 | 0.354 | 0.1377 | 0.32 | 0.42 | 0.0824 |
1357
+ | dot_recall@5 | 0.42 | 0.0306 | 0.36 | 0.2 | 0.1356 | 0.7367 | 0.2406 | 0.54 | 0.374 | 0.1857 | 0.42 | 0.61 | 0.1281 |
1358
+ | dot_recall@10 | 0.6 | 0.0458 | 0.52 | 0.26 | 0.2097 | 0.8767 | 0.2906 | 0.63 | 0.444 | 0.2837 | 0.48 | 0.68 | 0.2193 |
1359
+ | **dot_ndcg@10** | **0.3714** | **0.1571** | **0.2925** | **0.2151** | **0.3792** | **0.5965** | **0.2237** | **0.5601** | **0.3147** | **0.2673** | **0.2567** | **0.4731** | **0.389** |
1360
+ | dot_mrr@10 | 0.302 | 0.2964 | 0.2323 | 0.2989 | 0.5519 | 0.5313 | 0.2532 | 0.6587 | 0.277 | 0.4424 | 0.1845 | 0.4177 | 0.6561 |
1361
+ | dot_map@100 | 0.3209 | 0.0466 | 0.2305 | 0.1742 | 0.2587 | 0.4997 | 0.1863 | 0.494 | 0.2826 | 0.1904 | 0.1898 | 0.4079 | 0.27 |
1362
+ | query_active_dims | 252.1 | 255.62 | 256.0 | 256.0 | 253.96 | 256.0 | 240.22 | 256.0 | 229.0 | 256.0 | 256.0 | 256.0 | 216.8571 |
1363
+ | query_sparsity_ratio | 0.995 | 0.9949 | 0.9949 | 0.9949 | 0.995 | 0.9949 | 0.9952 | 0.9949 | 0.9955 | 0.9949 | 0.9949 | 0.9949 | 0.9957 |
1364
+ | corpus_active_dims | 256.0 | 256.0 | 256.0 | 256.0 | 256.0 | 256.0 | 256.0 | 256.0 | 234.9956 | 256.0 | 256.0 | 256.0 | 255.9241 |
1365
+ | corpus_sparsity_ratio | 0.9949 | 0.9949 | 0.9949 | 0.9949 | 0.9949 | 0.9949 | 0.9949 | 0.9949 | 0.9953 | 0.9949 | 0.9949 | 0.9949 | 0.9949 |
1366
+ | avg_flops | 104.6667 | 101.2659 | 114.127 | 129.3998 | 108.9664 | 120.7999 | 116.0173 | 114.6017 | 119.7198 | 123.3701 | 150.7188 | 129.6654 | 86.4117 |
1367
+
1368
+ #### Sparse Nano BEIR
1369
+
1370
+ * Dataset: `NanoBEIR_mean_256`
1371
+ * Evaluated with [<code>SparseNanoBEIREvaluator</code>](https://sbert.net/docs/package_reference/sparse_encoder/evaluation.html#sentence_transformers.sparse_encoder.evaluation.SparseNanoBEIREvaluator) with these parameters:
1372
+ ```json
1373
+ {
1374
+ "dataset_names": [
1375
+ "msmarco",
1376
+ "nfcorpus",
1377
+ "nq"
1378
+ ],
1379
+ "dataset_id": "sentence-transformers/NanoBEIR-en",
1380
+ "max_active_dims": 256
1381
+ }
1382
+ ```
1383
+
1384
+ | Metric | Value |
1385
+ |:----------------------|:----------|
1386
+ | dot_accuracy@1 | 0.18 |
1387
+ | dot_accuracy@3 | 0.3267 |
1388
+ | dot_accuracy@5 | 0.42 |
1389
+ | dot_accuracy@10 | 0.54 |
1390
+ | dot_precision@1 | 0.18 |
1391
+ | dot_precision@3 | 0.1356 |
1392
+ | dot_precision@5 | 0.1173 |
1393
+ | dot_precision@10 | 0.0867 |
1394
+ | dot_recall@1 | 0.0991 |
1395
+ | dot_recall@3 | 0.2158 |
1396
+ | dot_recall@5 | 0.2848 |
1397
+ | dot_recall@10 | 0.379 |
1398
+ | **dot_ndcg@10** | **0.273** |
1399
+ | dot_mrr@10 | 0.2783 |
1400
+ | dot_map@100 | 0.1991 |
1401
+ | query_active_dims | 254.54 |
1402
+ | query_sparsity_ratio | 0.9949 |
1403
+ | corpus_active_dims | 256.0 |
1404
+ | corpus_sparsity_ratio | 0.9949 |
1405
+ | avg_flops | 105.0486 |
1406
+
1407
+ #### Sparse Nano BEIR
1408
+
1409
+ * Dataset: `NanoBEIR_mean_256`
1410
+ * Evaluated with [<code>SparseNanoBEIREvaluator</code>](https://sbert.net/docs/package_reference/sparse_encoder/evaluation.html#sentence_transformers.sparse_encoder.evaluation.SparseNanoBEIREvaluator) with these parameters:
1411
+ ```json
1412
+ {
1413
+ "dataset_names": [
1414
+ "climatefever",
1415
+ "dbpedia",
1416
+ "fever",
1417
+ "fiqa2018",
1418
+ "hotpotqa",
1419
+ "msmarco",
1420
+ "nfcorpus",
1421
+ "nq",
1422
+ "quoraretrieval",
1423
+ "scidocs",
1424
+ "arguana",
1425
+ "scifact",
1426
+ "touche2020"
1427
+ ],
1428
+ "dataset_id": "sentence-transformers/NanoBEIR-en",
1429
+ "max_active_dims": 256
1430
+ }
1431
+ ```
1432
+
1433
+ | Metric | Value |
1434
+ |:----------------------|:-----------|
1435
+ | dot_accuracy@1 | 0.2731 |
1436
+ | dot_accuracy@3 | 0.4704 |
1437
+ | dot_accuracy@5 | 0.5567 |
1438
+ | dot_accuracy@10 | 0.6554 |
1439
+ | dot_precision@1 | 0.2731 |
1440
+ | dot_precision@3 | 0.2017 |
1441
+ | dot_precision@5 | 0.1665 |
1442
+ | dot_precision@10 | 0.1231 |
1443
+ | dot_recall@1 | 0.1297 |
1444
+ | dot_recall@3 | 0.269 |
1445
+ | dot_recall@5 | 0.337 |
1446
+ | dot_recall@10 | 0.4261 |
1447
+ | **dot_ndcg@10** | **0.3459** |
1448
+ | dot_mrr@10 | 0.3925 |
1449
+ | dot_map@100 | 0.2732 |
1450
+ | query_active_dims | 249.2619 |
1451
+ | query_sparsity_ratio | 0.9951 |
1452
+ | corpus_active_dims | 254.1114 |
1453
+ | corpus_sparsity_ratio | 0.995 |
1454
+ | avg_flops | 109.026 |
1455
+
1456
+ <!--
1457
+ ## Bias, Risks and Limitations
1458
+
1459
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
1460
+ -->
1461
+
1462
+ <!--
1463
+ ### Recommendations
1464
+
1465
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
1466
+ -->
1467
+
1468
+ ## Training Details
1469
+
1470
+ ### Training Dataset
1471
+
1472
+ #### gooaq
1473
+
1474
+ * Dataset: [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) at [b089f72](https://huggingface.co/datasets/sentence-transformers/gooaq/tree/b089f728748a068b7bc5234e5bcf5b25e3c8279c)
1475
+ * Size: 99,000 training samples
1476
+ * Columns: <code>question</code> and <code>answer</code>
1477
+ * Approximate statistics based on the first 1000 samples:
1478
+ | | question | answer |
1479
+ |:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
1480
+ | type | string | string |
1481
+ | details | <ul><li>min: 8 tokens</li><li>mean: 11.93 tokens</li><li>max: 21 tokens</li></ul> | <ul><li>min: 15 tokens</li><li>mean: 58.27 tokens</li><li>max: 137 tokens</li></ul> |
1482
+ * Samples:
1483
+ | question | answer |
1484
+ |:-----------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
1485
+ | <code>what are the 5 characteristics of a star?</code> | <code>Key Concept: Characteristics used to classify stars include color, temperature, size, composition, and brightness.</code> |
1486
+ | <code>are copic markers alcohol ink?</code> | <code>Copic Ink is alcohol-based and flammable. Keep away from direct sunlight and extreme temperatures.</code> |
1487
+ | <code>what is the difference between appellate term and appellate division?</code> | <code>Appellate terms An appellate term is an intermediate appellate court that hears appeals from the inferior courts within their designated counties or judicial districts, and are intended to ease the workload on the Appellate Division and provide a less expensive forum closer to the people.</code> |
1488
+ * Loss: [<code>SpladeLoss</code>](https://sbert.net/docs/package_reference/sparse_encoder/losses.html#spladeloss) with these parameters:
1489
+ ```json
1490
+ {
1491
+ "loss": "SparseMultipleNegativesRankingLoss(scale=1.0, similarity_fct='dot_score', gather_across_devices=False, directions=('query_to_doc',), partition_mode='joint', hardness_mode=None, hardness_strength=0.0)",
1492
+ "document_regularizer_weight": 3e-05,
1493
+ "query_regularizer_weight": 5e-05
1494
+ }
1495
+ ```
1496
+
1497
+ ### Evaluation Dataset
1498
+
1499
+ #### gooaq
1500
+
1501
+ * Dataset: [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) at [b089f72](https://huggingface.co/datasets/sentence-transformers/gooaq/tree/b089f728748a068b7bc5234e5bcf5b25e3c8279c)
1502
+ * Size: 1,000 evaluation samples
1503
+ * Columns: <code>question</code> and <code>answer</code>
1504
+ * Approximate statistics based on the first 1000 samples:
1505
+ | | question | answer |
1506
+ |:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
1507
+ | type | string | string |
1508
+ | details | <ul><li>min: 8 tokens</li><li>mean: 12.05 tokens</li><li>max: 23 tokens</li></ul> | <ul><li>min: 15 tokens</li><li>mean: 58.98 tokens</li><li>max: 186 tokens</li></ul> |
1509
+ * Samples:
1510
+ | question | answer |
1511
+ |:-----------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
1512
+ | <code>should you take ibuprofen with high blood pressure?</code> | <code>In general, people with high blood pressure should use acetaminophen or possibly aspirin for over-the-counter pain relief. Unless your health care provider has said it's OK, you should not use ibuprofen, ketoprofen, or naproxen sodium. If aspirin or acetaminophen doesn't help with your pain, call your doctor.</code> |
1513
+ | <code>how old do you have to be to work in sc?</code> | <code>The general minimum age of employment for South Carolina youth is 14, although the state allows younger children who are performers to work in show business. If their families are agricultural workers, children younger than age 14 may also participate in farm labor.</code> |
1514
+ | <code>how to write a topic proposal for a research paper?</code> | <code>['Write down the main topic of your paper. ... ', 'Write two or three short sentences under the main topic that explain why you chose that topic. ... ', 'Write a thesis sentence that states the angle and purpose of your research paper. ... ', 'List the items you will cover in the body of the paper that support your thesis statement.']</code> |
1515
+ * Loss: [<code>SpladeLoss</code>](https://sbert.net/docs/package_reference/sparse_encoder/losses.html#spladeloss) with these parameters:
1516
+ ```json
1517
+ {
1518
+ "loss": "SparseMultipleNegativesRankingLoss(scale=1.0, similarity_fct='dot_score', gather_across_devices=False, directions=('query_to_doc',), partition_mode='joint', hardness_mode=None, hardness_strength=0.0)",
1519
+ "document_regularizer_weight": 3e-05,
1520
+ "query_regularizer_weight": 5e-05
1521
+ }
1522
+ ```
1523
+
1524
+ ### Training Hyperparameters
1525
+ #### Non-Default Hyperparameters
1526
+
1527
+ - `per_device_train_batch_size`: 32
1528
+ - `num_train_epochs`: 1
1529
+ - `learning_rate`: 2e-05
1530
+ - `bf16`: True
1531
+ - `per_device_eval_batch_size`: 32
1532
+ - `load_best_model_at_end`: True
1533
+ - `batch_sampler`: no_duplicates
1534
+
1535
+ #### All Hyperparameters
1536
+ <details><summary>Click to expand</summary>
1537
+
1538
+ - `per_device_train_batch_size`: 32
1539
+ - `num_train_epochs`: 1
1540
+ - `max_steps`: -1
1541
+ - `learning_rate`: 2e-05
1542
+ - `lr_scheduler_type`: linear
1543
+ - `lr_scheduler_kwargs`: None
1544
+ - `warmup_steps`: 0
1545
+ - `optim`: adamw_torch_fused
1546
+ - `optim_args`: None
1547
+ - `weight_decay`: 0.0
1548
+ - `adam_beta1`: 0.9
1549
+ - `adam_beta2`: 0.999
1550
+ - `adam_epsilon`: 1e-08
1551
+ - `optim_target_modules`: None
1552
+ - `gradient_accumulation_steps`: 1
1553
+ - `average_tokens_across_devices`: True
1554
+ - `max_grad_norm`: 1.0
1555
+ - `label_smoothing_factor`: 0.0
1556
+ - `bf16`: True
1557
+ - `fp16`: False
1558
+ - `bf16_full_eval`: False
1559
+ - `fp16_full_eval`: False
1560
+ - `tf32`: None
1561
+ - `gradient_checkpointing`: False
1562
+ - `gradient_checkpointing_kwargs`: None
1563
+ - `torch_compile`: False
1564
+ - `torch_compile_backend`: None
1565
+ - `torch_compile_mode`: None
1566
+ - `use_liger_kernel`: False
1567
+ - `liger_kernel_config`: None
1568
+ - `use_cache`: False
1569
+ - `neftune_noise_alpha`: None
1570
+ - `torch_empty_cache_steps`: None
1571
+ - `auto_find_batch_size`: False
1572
+ - `log_on_each_node`: True
1573
+ - `logging_nan_inf_filter`: True
1574
+ - `include_num_input_tokens_seen`: no
1575
+ - `log_level`: passive
1576
+ - `log_level_replica`: warning
1577
+ - `disable_tqdm`: False
1578
+ - `project`: huggingface
1579
+ - `trackio_space_id`: trackio
1580
+ - `per_device_eval_batch_size`: 32
1581
+ - `prediction_loss_only`: True
1582
+ - `eval_on_start`: False
1583
+ - `eval_do_concat_batches`: True
1584
+ - `eval_use_gather_object`: False
1585
+ - `eval_accumulation_steps`: None
1586
+ - `include_for_metrics`: []
1587
+ - `batch_eval_metrics`: False
1588
+ - `save_only_model`: False
1589
+ - `save_on_each_node`: False
1590
+ - `enable_jit_checkpoint`: False
1591
+ - `push_to_hub`: False
1592
+ - `hub_private_repo`: None
1593
+ - `hub_model_id`: None
1594
+ - `hub_strategy`: every_save
1595
+ - `hub_always_push`: False
1596
+ - `hub_revision`: None
1597
+ - `load_best_model_at_end`: True
1598
+ - `ignore_data_skip`: False
1599
+ - `restore_callback_states_from_checkpoint`: False
1600
+ - `full_determinism`: False
1601
+ - `seed`: 42
1602
+ - `data_seed`: None
1603
+ - `use_cpu`: False
1604
+ - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
1605
+ - `parallelism_config`: None
1606
+ - `dataloader_drop_last`: False
1607
+ - `dataloader_num_workers`: 0
1608
+ - `dataloader_pin_memory`: True
1609
+ - `dataloader_persistent_workers`: False
1610
+ - `dataloader_prefetch_factor`: None
1611
+ - `remove_unused_columns`: True
1612
+ - `label_names`: None
1613
+ - `train_sampling_strategy`: random
1614
+ - `length_column_name`: length
1615
+ - `ddp_find_unused_parameters`: None
1616
+ - `ddp_bucket_cap_mb`: None
1617
+ - `ddp_broadcast_buffers`: False
1618
+ - `ddp_backend`: None
1619
+ - `ddp_timeout`: 1800
1620
+ - `fsdp`: []
1621
+ - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
1622
+ - `deepspeed`: None
1623
+ - `debug`: []
1624
+ - `skip_memory_metrics`: True
1625
+ - `do_predict`: False
1626
+ - `resume_from_checkpoint`: None
1627
+ - `warmup_ratio`: None
1628
+ - `local_rank`: -1
1629
+ - `prompts`: None
1630
+ - `batch_sampler`: no_duplicates
1631
+ - `multi_dataset_batch_sampler`: proportional
1632
+ - `router_mapping`: {}
1633
+ - `learning_rate_mapping`: {}
1634
+
1635
+ </details>
1636
+
1637
+ ### Training Logs
1638
+ | Epoch | Step | Training Loss | Validation Loss | NanoMSMARCO_256_dot_ndcg@10 | NanoNFCorpus_256_dot_ndcg@10 | NanoNQ_256_dot_ndcg@10 | NanoBEIR_mean_256_dot_ndcg@10 | NanoClimateFEVER_256_dot_ndcg@10 | NanoDBPedia_256_dot_ndcg@10 | NanoFEVER_256_dot_ndcg@10 | NanoFiQA2018_256_dot_ndcg@10 | NanoHotpotQA_256_dot_ndcg@10 | NanoQuoraRetrieval_256_dot_ndcg@10 | NanoSCIDOCS_256_dot_ndcg@10 | NanoArguAna_256_dot_ndcg@10 | NanoSciFact_256_dot_ndcg@10 | NanoTouche2020_256_dot_ndcg@10 |
1639
+ |:----------:|:--------:|:-------------:|:---------------:|:---------------------------:|:----------------------------:|:----------------------:|:-----------------------------:|:--------------------------------:|:---------------------------:|:-------------------------:|:----------------------------:|:----------------------------:|:----------------------------------:|:---------------------------:|:---------------------------:|:---------------------------:|:------------------------------:|
1640
+ | 0.0323 | 100 | 954.7106 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1641
+ | 0.0646 | 200 | 17.8241 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1642
+ | 0.0970 | 300 | 4.5136 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1643
+ | 0.1293 | 400 | 2.5427 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1644
+ | 0.1616 | 500 | 1.4733 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1645
+ | 0.1939 | 600 | 1.0940 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1646
+ | 0.1972 | 610 | - | 0.8522 | 0.1440 | 0.0453 | 0.0771 | 0.0888 | - | - | - | - | - | - | - | - | - | - |
1647
+ | 0.2262 | 700 | 0.7541 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1648
+ | 0.2586 | 800 | 0.7425 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1649
+ | 0.2909 | 900 | 0.5966 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1650
+ | 0.3232 | 1000 | 0.5606 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1651
+ | 0.3555 | 1100 | 0.5440 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1652
+ | 0.3878 | 1200 | 0.4032 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1653
+ | 0.3943 | 1220 | - | 0.4165 | 0.2494 | 0.0613 | 0.2065 | 0.1724 | - | - | - | - | - | - | - | - | - | - |
1654
+ | 0.4202 | 1300 | 0.3995 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1655
+ | 0.4525 | 1400 | 0.2976 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1656
+ | 0.4848 | 1500 | 0.2971 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1657
+ | 0.5171 | 1600 | 0.2716 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1658
+ | 0.5495 | 1700 | 0.2577 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1659
+ | 0.5818 | 1800 | 0.2370 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1660
+ | 0.5915 | 1830 | - | 0.2104 | 0.3406 | 0.1173 | 0.2414 | 0.2331 | - | - | - | - | - | - | - | - | - | - |
1661
+ | 0.6141 | 1900 | 0.2360 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1662
+ | 0.6464 | 2000 | 0.2238 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1663
+ | 0.6787 | 2100 | 0.2237 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1664
+ | 0.7111 | 2200 | 0.2162 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1665
+ | 0.7434 | 2300 | 0.2044 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1666
+ | 0.7757 | 2400 | 0.2202 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1667
+ | 0.7886 | 2440 | - | 0.1736 | 0.3932 | 0.1545 | 0.2717 | 0.2731 | - | - | - | - | - | - | - | - | - | - |
1668
+ | 0.8080 | 2500 | 0.1672 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1669
+ | 0.8403 | 2600 | 0.2122 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1670
+ | 0.8727 | 2700 | 0.1704 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1671
+ | 0.9050 | 2800 | 0.1870 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1672
+ | 0.9373 | 2900 | 0.1671 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1673
+ | 0.9696 | 3000 | 0.1386 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1674
+ | **0.9858** | **3050** | **-** | **0.17** | **0.3714** | **0.1571** | **0.2925** | **0.2737** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** |
1675
+ | 1.0 | 3094 | - | 0.1706 | 0.3675 | 0.1624 | 0.2892 | 0.2730 | - | - | - | - | - | - | - | - | - | - |
1676
+ | -1 | -1 | - | - | 0.3714 | 0.1571 | 0.2925 | 0.3459 | 0.2151 | 0.3792 | 0.5965 | 0.2237 | 0.5601 | 0.3147 | 0.2673 | 0.2567 | 0.4731 | 0.3890 |
1677
+
1678
+ * The bold row denotes the saved checkpoint.
1679
+
1680
+ ### Training Time
1681
+ - **Training**: 7.7 minutes
1682
+ - **Evaluation**: 2.0 minutes
1683
+ - **Total**: 9.7 minutes
1684
+
1685
+ ### Framework Versions
1686
+ - Python: 3.12.3
1687
+ - Sentence Transformers: 5.4.1
1688
+ - Transformers: 5.5.4
1689
+ - PyTorch: 2.11.0+cu130
1690
+ - Accelerate: 1.13.0
1691
+ - Datasets: 4.8.4
1692
+ - Tokenizers: 0.22.2
1693
+
1694
+ ## Citation
1695
+
1696
+ ### BibTeX
1697
+
1698
+ #### Sentence Transformers
1699
+ ```bibtex
1700
+ @inproceedings{reimers-2019-sentence-bert,
1701
+ title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
1702
+ author = "Reimers, Nils and Gurevych, Iryna",
1703
+ booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
1704
+ month = "11",
1705
+ year = "2019",
1706
+ publisher = "Association for Computational Linguistics",
1707
+ url = "https://arxiv.org/abs/1908.10084",
1708
+ }
1709
+ ```
1710
+
1711
+ #### SpladeLoss
1712
+ ```bibtex
1713
+ @misc{formal2022distillationhardnegativesampling,
1714
+ title={From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effective},
1715
+ author={Thibault Formal and Carlos Lassance and Benjamin Piwowarski and Stéphane Clinchant},
1716
+ year={2022},
1717
+ eprint={2205.04733},
1718
+ archivePrefix={arXiv},
1719
+ primaryClass={cs.IR},
1720
+ url={https://arxiv.org/abs/2205.04733},
1721
+ }
1722
+ ```
1723
+
1724
+ #### SparseMultipleNegativesRankingLoss
1725
+ ```bibtex
1726
+ @misc{oord2019representationlearningcontrastivepredictive,
1727
+ title={Representation Learning with Contrastive Predictive Coding},
1728
+ author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
1729
+ year={2019},
1730
+ eprint={1807.03748},
1731
+ archivePrefix={arXiv},
1732
+ primaryClass={cs.LG},
1733
+ url={https://arxiv.org/abs/1807.03748},
1734
+ }
1735
+ ```
1736
+
1737
+ #### FlopsLoss
1738
+ ```bibtex
1739
+ @article{paria2020minimizing,
1740
+ title={Minimizing flops to learn efficient sparse representations},
1741
+ author={Paria, Biswajit and Yeh, Chih-Kuan and Yen, Ian EH and Xu, Ning and Ravikumar, Pradeep and P{'o}czos, Barnab{'a}s},
1742
+ journal={arXiv preprint arXiv:2004.05665},
1743
+ year={2020}
1744
+ }
1745
+ ```
1746
+
1747
+ <!--
1748
+ ## Glossary
1749
+
1750
+ *Clearly define terms in order to be accessible across audiences.*
1751
+ -->
1752
+
1753
+ <!--
1754
+ ## Model Card Authors
1755
+
1756
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
1757
+ -->
1758
+
1759
+ <!--
1760
+ ## Model Card Contact
1761
+
1762
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
1763
+ -->
config.json ADDED
@@ -0,0 +1,64 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "ModernBertForMaskedLM"
4
+ ],
5
+ "attention_bias": false,
6
+ "attention_dropout": 0.0,
7
+ "bos_token_id": 50281,
8
+ "causal_mask": false,
9
+ "classifier_activation": "gelu",
10
+ "classifier_bias": false,
11
+ "classifier_dropout": 0.0,
12
+ "classifier_pooling": "mean",
13
+ "cls_token_id": 50281,
14
+ "decoder_bias": true,
15
+ "deterministic_flash_attn": false,
16
+ "dtype": "float32",
17
+ "embedding_dropout": 0.0,
18
+ "eos_token_id": 50282,
19
+ "global_attn_every_n_layers": 3,
20
+ "gradient_checkpointing": false,
21
+ "hidden_activation": "gelu",
22
+ "hidden_size": 256,
23
+ "initializer_cutoff_factor": 2.0,
24
+ "initializer_range": 0.02,
25
+ "intermediate_size": 384,
26
+ "is_causal": false,
27
+ "layer_norm_eps": 1e-05,
28
+ "layer_types": [
29
+ "full_attention",
30
+ "sliding_attention",
31
+ "sliding_attention",
32
+ "full_attention",
33
+ "sliding_attention",
34
+ "sliding_attention",
35
+ "full_attention"
36
+ ],
37
+ "local_attention": 128,
38
+ "max_position_embeddings": 7999,
39
+ "mlp_bias": false,
40
+ "mlp_dropout": 0.0,
41
+ "model_type": "modernbert",
42
+ "norm_bias": false,
43
+ "norm_eps": 1e-05,
44
+ "num_attention_heads": 4,
45
+ "num_hidden_layers": 7,
46
+ "pad_token_id": 50283,
47
+ "position_embedding_type": "sans_pos",
48
+ "rope_parameters": {
49
+ "full_attention": {
50
+ "rope_theta": 160000.0,
51
+ "rope_type": "default"
52
+ },
53
+ "sliding_attention": {
54
+ "rope_theta": 160000.0,
55
+ "rope_type": "default"
56
+ }
57
+ },
58
+ "sep_token_id": 50282,
59
+ "sparse_pred_ignore_index": -100,
60
+ "sparse_prediction": false,
61
+ "tie_word_embeddings": true,
62
+ "transformers_version": "5.5.4",
63
+ "vocab_size": 50368
64
+ }
config_sentence_transformers.json ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "__version__": {
3
+ "pytorch": "2.11.0+cu130",
4
+ "sentence_transformers": "5.4.1",
5
+ "transformers": "5.5.4"
6
+ },
7
+ "default_prompt_name": null,
8
+ "model_type": "SparseEncoder",
9
+ "prompts": {
10
+ "document": "",
11
+ "query": ""
12
+ },
13
+ "similarity_fn_name": "dot"
14
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1d5f9b54c854ccba6df900d2c003e527eee8fa691cebda6dfe9f704fc222d4a5
3
+ size 67659080
modules.json ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "idx": 0,
4
+ "name": "0",
5
+ "path": "",
6
+ "type": "sentence_transformers.base.modules.transformer.Transformer"
7
+ },
8
+ {
9
+ "idx": 1,
10
+ "name": "1",
11
+ "path": "1_SpladePooling",
12
+ "type": "sentence_transformers.sparse_encoder.modules.splade_pooling.SpladePooling"
13
+ }
14
+ ]
sentence_bert_config.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "transformer_task": "fill-mask",
3
+ "modality_config": {
4
+ "text": {
5
+ "method": "forward",
6
+ "method_output_name": "logits"
7
+ }
8
+ },
9
+ "module_output_name": "token_embeddings"
10
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "backend": "tokenizers",
3
+ "clean_up_tokenization_spaces": true,
4
+ "cls_token": "[CLS]",
5
+ "is_local": false,
6
+ "mask_token": "[MASK]",
7
+ "model_input_names": [
8
+ "input_ids",
9
+ "attention_mask"
10
+ ],
11
+ "model_max_length": 256,
12
+ "pad_token": "[PAD]",
13
+ "sep_token": "[SEP]",
14
+ "tokenizer_class": "TokenizersBackend",
15
+ "unk_token": "[UNK]"
16
+ }