Hugging Face's logo

tomaarsen
/
reranker-msmarco-MiniLM-L12-H384-uncased-adrmse

Text Ranking
sentence-transformers
Safetensors
English
bert
cross-encoder
reranker
Generated from Trainer
dataset_size:9000
loss:ADRMSELoss
Eval Results (legacy)
text-embeddings-inference
Model card Files
xet
 Community
tomaarsen HF Staff commited on
Commit
6bf556f
·
verified ·
1 Parent(s): 1455c43

Add new CrossEncoder model

Browse files
Files changed (8) hide show
  1. README.md +511 -0
  2. config.json +34 -0
  3. config_sentence_transformers.json +11 -0
  4. model.safetensors +3 -0
  5. modules.json +8 -0
  6. sentence_bert_config.json +10 -0
  7. tokenizer.json +0 -0
  8. tokenizer_config.json +14 -0
README.md ADDED
@@ -0,0 +1,511 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ tags:
5
+ - sentence-transformers
6
+ - cross-encoder
7
+ - reranker
8
+ - generated_from_trainer
9
+ - dataset_size:9000
10
+ - loss:ADRMSELoss
11
+ base_model: microsoft/MiniLM-L12-H384-uncased
12
+ datasets:
13
+ - sentence-transformers/msmarco
14
+ pipeline_tag: text-ranking
15
+ library_name: sentence-transformers
16
+ metrics:
17
+ - map
18
+ - mrr@10
19
+ - ndcg@10
20
+ model-index:
21
+ - name: CrossEncoder based on microsoft/MiniLM-L12-H384-uncased
22
+ results:
23
+ - task:
24
+ type: cross-encoder-reranking
25
+ name: Cross Encoder Reranking
26
+ dataset:
27
+ name: NanoMSMARCO R100
28
+ type: NanoMSMARCO_R100
29
+ metrics:
30
+ - type: map
31
+ value: 0.5412
32
+ name: Map
33
+ - type: mrr@10
34
+ value: 0.5341
35
+ name: Mrr@10
36
+ - type: ndcg@10
37
+ value: 0.6073
38
+ name: Ndcg@10
39
+ - task:
40
+ type: cross-encoder-reranking
41
+ name: Cross Encoder Reranking
42
+ dataset:
43
+ name: NanoNFCorpus R100
44
+ type: NanoNFCorpus_R100
45
+ metrics:
46
+ - type: map
47
+ value: 0.36
48
+ name: Map
49
+ - type: mrr@10
50
+ value: 0.6053
51
+ name: Mrr@10
52
+ - type: ndcg@10
53
+ value: 0.4303
54
+ name: Ndcg@10
55
+ - task:
56
+ type: cross-encoder-reranking
57
+ name: Cross Encoder Reranking
58
+ dataset:
59
+ name: NanoNQ R100
60
+ type: NanoNQ_R100
61
+ metrics:
62
+ - type: map
63
+ value: 0.5805
64
+ name: Map
65
+ - type: mrr@10
66
+ value: 0.595
67
+ name: Mrr@10
68
+ - type: ndcg@10
69
+ value: 0.65
70
+ name: Ndcg@10
71
+ - task:
72
+ type: cross-encoder-nano-beir
73
+ name: Cross Encoder Nano BEIR
74
+ dataset:
75
+ name: NanoBEIR R100 mean
76
+ type: NanoBEIR_R100_mean
77
+ metrics:
78
+ - type: map
79
+ value: 0.4939
80
+ name: Map
81
+ - type: mrr@10
82
+ value: 0.5781
83
+ name: Mrr@10
84
+ - type: ndcg@10
85
+ value: 0.5625
86
+ name: Ndcg@10
87
+ ---
88
+
89
+ # CrossEncoder based on microsoft/MiniLM-L12-H384-uncased
90
+
91
+ This is a [Cross Encoder](https://www.sbert.net/docs/cross_encoder/usage/usage.html) model finetuned from [microsoft/MiniLM-L12-H384-uncased](https://huggingface.co/microsoft/MiniLM-L12-H384-uncased) on the [msmarco](https://huggingface.co/datasets/sentence-transformers/msmarco) dataset using the [sentence-transformers](https://www.SBERT.net) library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.
92
+
93
+ ## Model Details
94
+
95
+ ### Model Description
96
+ - **Model Type:** Cross Encoder
97
+ - **Base model:** [microsoft/MiniLM-L12-H384-uncased](https://huggingface.co/microsoft/MiniLM-L12-H384-uncased) <!-- at revision 44acabbec0ef496f6dbc93adadea57f376b7c0ec -->
98
+ - **Maximum Sequence Length:** 512 tokens
99
+ - **Number of Output Labels:** 1 label
100
+ - **Supported Modality:** Text
101
+ - **Training Dataset:**
102
+ - [msmarco](https://huggingface.co/datasets/sentence-transformers/msmarco)
103
+ - **Language:** en
104
+ <!-- - **License:** Unknown -->
105
+
106
+ ### Model Sources
107
+
108
+ - **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
109
+ - **Documentation:** [Cross Encoder Documentation](https://www.sbert.net/docs/cross_encoder/usage/usage.html)
110
+ - **Repository:** [Sentence Transformers on GitHub](https://github.com/huggingface/sentence-transformers)
111
+ - **Hugging Face:** [Cross Encoders on Hugging Face](https://huggingface.co/models?library=sentence-transformers&other=cross-encoder)
112
+
113
+ ### Full Model Architecture
114
+
115
+ ```
116
+ CrossEncoder(
117
+ (0): Transformer({'transformer_task': 'sequence-classification', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'logits'}}, 'module_output_name': 'scores', 'architecture': 'BertForSequenceClassification'})
118
+ )
119
+ ```
120
+
121
+ ## Usage
122
+
123
+ ### Direct Usage (Sentence Transformers)
124
+
125
+ First install the Sentence Transformers library:
126
+
127
+ ```bash
128
+ pip install -U sentence-transformers
129
+ ```
130
+
131
+ Then you can load this model and run inference.
132
+ ```python
133
+ from sentence_transformers import CrossEncoder
134
+
135
+ # Download from the 🤗 Hub
136
+ model = CrossEncoder("tomaarsen/reranker-msmarco-MiniLM-L12-H384-uncased-adrmse")
137
+ # Get scores for pairs of inputs
138
+ pairs = [
139
+ ['herbs for lowering blood sugar', 'Ten Herbs To Lower Blood Sugar Naturally. 1. Cinnamon â\x80\x93 As little as half a teaspoon a day can help improve your insulin sensitivity and control of blood sugar. If you donâ\x80\x99t enjoy sprinkling cinnamon in your daily yogurt or oatmeal, you can easily find capsules of cinnamon to make a cinnamon regimen easier. 2. Sage â\x80\x93 In one German study, blood sugar levels were reduced when sage infusions were consumed by diabetics on an empty stomach. Sage is one of many powerful herbs.'],
140
+ ['herbs for lowering blood sugar', 'Ten Herbs To Lower Blood Sugar Naturally. 1. Cinnamon â\x80\x93 As little as half a teaspoon a day can help improve your insulin sensitivity and control of blood sugar. If you donâ\x80\x99t enjoy sprinkling cinnamon in your daily yogurt or oatmeal, you can easily find capsules of cinnamon to make a cinnamon regimen easier.'],
141
+ ['herbs for lowering blood sugar', 'Cinnamon, people who have diabetes, is commonly used to reduce blood sugar and cholesterol level in blood. Onions contain falconoid and high sulfur which if consumed two ounces daily by diabetics reduces blood sugar significantly. Garlic is a beneficial herb is another of the foods that lower blood sugar. Garlic helps to elevate production of insulin and its sensitivity. Nuts.'],
142
+ ['herbs for lowering blood sugar', '#7: Ginseng. Ginseng is a popular herb that has many healing properties, but for diabetes patients, its daily use can help to lower blood sugar levels. This is because of its ability to increase the use of glucose in the cells and to reduce the carbohydrates absorption rates. In turn, this helps to lower the blood sugar levels in the body.'],
143
+ ['herbs for lowering blood sugar', 'Cinnamon, people who have diabetes, is commonly used to reduce blood sugar and cholesterol level in blood. Onions contain falconoid and high sulfur which if consumed two ounces daily by diabetics reduces blood sugar significantly. Garlic is a beneficial herb is another of the foods that lower blood sugar.'],
144
+ ]
145
+ scores = model.predict(pairs)
146
+ print(scores)
147
+ # [0.7691 0.7417 0.48 0.7066 0.4656]
148
+
149
+ # Or rank different texts based on similarity to a single text
150
+ ranks = model.rank(
151
+ 'herbs for lowering blood sugar',
152
+ [
153
+ 'Ten Herbs To Lower Blood Sugar Naturally. 1. Cinnamon â\x80\x93 As little as half a teaspoon a day can help improve your insulin sensitivity and control of blood sugar. If you donâ\x80\x99t enjoy sprinkling cinnamon in your daily yogurt or oatmeal, you can easily find capsules of cinnamon to make a cinnamon regimen easier. 2. Sage â\x80\x93 In one German study, blood sugar levels were reduced when sage infusions were consumed by diabetics on an empty stomach. Sage is one of many powerful herbs.',
154
+ 'Ten Herbs To Lower Blood Sugar Naturally. 1. Cinnamon â\x80\x93 As little as half a teaspoon a day can help improve your insulin sensitivity and control of blood sugar. If you donâ\x80\x99t enjoy sprinkling cinnamon in your daily yogurt or oatmeal, you can easily find capsules of cinnamon to make a cinnamon regimen easier.',
155
+ 'Cinnamon, people who have diabetes, is commonly used to reduce blood sugar and cholesterol level in blood. Onions contain falconoid and high sulfur which if consumed two ounces daily by diabetics reduces blood sugar significantly. Garlic is a beneficial herb is another of the foods that lower blood sugar. Garlic helps to elevate production of insulin and its sensitivity. Nuts.',
156
+ '#7: Ginseng. Ginseng is a popular herb that has many healing properties, but for diabetes patients, its daily use can help to lower blood sugar levels. This is because of its ability to increase the use of glucose in the cells and to reduce the carbohydrates absorption rates. In turn, this helps to lower the blood sugar levels in the body.',
157
+ 'Cinnamon, people who have diabetes, is commonly used to reduce blood sugar and cholesterol level in blood. Onions contain falconoid and high sulfur which if consumed two ounces daily by diabetics reduces blood sugar significantly. Garlic is a beneficial herb is another of the foods that lower blood sugar.',
158
+ ]
159
+ )
160
+ # [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]
161
+ ```
162
+
163
+ <!--
164
+ ### Direct Usage (Transformers)
165
+
166
+ <details><summary>Click to see the direct usage in Transformers</summary>
167
+
168
+ </details>
169
+ -->
170
+
171
+ <!--
172
+ ### Downstream Usage (Sentence Transformers)
173
+
174
+ You can finetune this model on your own dataset.
175
+
176
+ <details><summary>Click to expand</summary>
177
+
178
+ </details>
179
+ -->
180
+
181
+ <!--
182
+ ### Out-of-Scope Use
183
+
184
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
185
+ -->
186
+
187
+ ## Evaluation
188
+
189
+ ### Metrics
190
+
191
+ #### Cross Encoder Reranking
192
+
193
+ * Datasets: `NanoMSMARCO_R100`, `NanoNFCorpus_R100` and `NanoNQ_R100`
194
+ * Evaluated with [<code>CrossEncoderRerankingEvaluator</code>](https://sbert.net/docs/package_reference/cross_encoder/evaluation.html#sentence_transformers.cross_encoder.evaluation.CrossEncoderRerankingEvaluator) with these parameters:
195
+ ```json
196
+ {
197
+ "at_k": 10,
198
+ "always_rerank_positives": true
199
+ }
200
+ ```
201
+
202
+ | Metric | NanoMSMARCO_R100 | NanoNFCorpus_R100 | NanoNQ_R100 |
203
+ |:------------|:---------------------|:---------------------|:---------------------|
204
+ | map | 0.5412 (+0.0516) | 0.3600 (+0.0990) | 0.5805 (+0.1609) |
205
+ | mrr@10 | 0.5341 (+0.0566) | 0.6053 (+0.1054) | 0.5950 (+0.1683) |
206
+ | **ndcg@10** | **0.6073 (+0.0669)** | **0.4303 (+0.1053)** | **0.6500 (+0.1493)** |
207
+
208
+ #### Cross Encoder Nano BEIR
209
+
210
+ * Dataset: `NanoBEIR_R100_mean`
211
+ * Evaluated with [<code>CrossEncoderNanoBEIREvaluator</code>](https://sbert.net/docs/package_reference/cross_encoder/evaluation.html#sentence_transformers.cross_encoder.evaluation.CrossEncoderNanoBEIREvaluator) with these parameters:
212
+ ```json
213
+ {
214
+ "dataset_names": [
215
+ "msmarco",
216
+ "nfcorpus",
217
+ "nq"
218
+ ],
219
+ "dataset_id": "sentence-transformers/NanoBEIR-en",
220
+ "rerank_k": 100,
221
+ "at_k": 10,
222
+ "always_rerank_positives": true
223
+ }
224
+ ```
225
+
226
+ | Metric | Value |
227
+ |:------------|:---------------------|
228
+ | map | 0.4939 (+0.1038) |
229
+ | mrr@10 | 0.5781 (+0.1101) |
230
+ | **ndcg@10** | **0.5625 (+0.1071)** |
231
+
232
+ <!--
233
+ ## Bias, Risks and Limitations
234
+
235
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
236
+ -->
237
+
238
+ <!--
239
+ ### Recommendations
240
+
241
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
242
+ -->
243
+
244
+ ## Training Details
245
+
246
+ ### Training Dataset
247
+
248
+ #### msmarco
249
+
250
+ * Dataset: [msmarco](https://huggingface.co/datasets/sentence-transformers/msmarco) at [2a16a97](https://huggingface.co/datasets/sentence-transformers/msmarco/tree/2a16a97b87802876f28636ca673e872248336951)
251
+ * Size: 9,000 training samples
252
+ * Columns: <code>query</code>, <code>docs</code>, and <code>scores</code>
253
+ * Approximate statistics based on the first 1000 samples:
254
+ | | query | docs | scores |
255
+ |:--------|:---------------------------------------------------------------------------------|:------------------------------------|:------------------------------------|
256
+ | type | string | list | list |
257
+ | details | <ul><li>min: 4 tokens</li><li>mean: 8.94 tokens</li><li>max: 33 tokens</li></ul> | <ul><li>size: 20 elements</li></ul> | <ul><li>size: 20 elements</li></ul> |
258
+ * Samples:
259
+ | query | docs | scores |
260
+ |:--------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------|
261
+ | <code>what is medical d. o.</code> | <code>['Medical Definition of D.O. D.O.: Abbreviation for Doctor of Osteopathy, an osteopathic physician. Osteopathy is a system of therapy founded in the 19th century based on the concept that the body can formulate its own remedies against diseases when the body is in a normal structural relationship, has a normal environment and enjoys good nutrition.', 'Doctor of Osteopathic Medicine (D.O.) is a professional doctoral degree for physicians and surgeons offered by medical schools in the United States. A D.O. degree graduate may become licensed as an osteopathic physician, having equivalent rights, privileges, and responsibilities as a physician who has earned the Doctor of Medicine (M.D.) degree. D.O. physicians are licensed to practice the full scope of medicine and surgery in sixty five countries, and all fifty states in the US.', 'Doctor of Osteopathic Medicine (D.O.) is a professional doctoral degree for physicians and surgeons offered by medical schools in the United States. A D.O. degree graduate may become licensed as an osteopathic physician, having equivalent rights, privileges, and responsibilities as a physician who has earned the Doctor of Medicine (M.D.) degree. D.O. physicians are licensed to practice the full scope of medicine and surgery in sixty-five countries, and in all fifty states.', 'Doctor of Osteopathic Medicine (D.O.) is a professional doctoral degree for physicians and surgeons offered by medical schools in the United States. A D.O. degree graduate may become licensed as an osteopathic physician, having equivalent rights, privileges, and responsibilities as a physician who has earned the Doctor of Medicine (M.D.) degree.', 'A doctor of osteopathic medicine (D.O.) is a fully licensed medical doctor offering all the techniques and treatments of modern medicine with the added benefits of hands-on diagnostics and a holistic philosophy.', ...]</code> | <code>[20, 19, 18, 17, 16, ...]</code> |
262
+ | <code>pending sale definition</code> | <code>['Sale pending means that there is a transaction that is going through Escrow but that has not closed yet. Translation it is too late to put in an offer now. If it falls out of escrow and the seller decides if he wants to put it back on the market then you can put in your offer.', 'Answers. 1 Sale pending means that there is a transaction that is going through Escrow but that has not closed yet. Translation it is too late to put in an offer now. If it falls out of escrow and the seller decides if he wants to put it back on the market then you can put in your offer.', '· just now. 1 Sale pending means that there is a transaction that is going through Escrow but that has not closed yet. 2 Sale pending can mean anything from an offer being accepted or contracts signed. 3 It means the buyer and seller have agreed on a contract, but closing has not happened yet.', "Thereâ\x80\x99s just one problem: It's sale pending. But what does sale pending mean exactly? Are you too late or do you still have a shot? The short answer: If a home you love is pending sale, donâ\x80\x99t give up hope. What does sale pending mean? A pending sale means the seller has an offer but hasn't closed yet. (This is different from a contingent sale.) A property is placed in pending status the minute a contract is executed.", "What does pending mean in real estate? Definition of Pending A home's status is changed to Pending or Pending Offer once it's set to close and all contingencies have been satisfied or waived. This is when the lender and the escrow agent or attorney process the loan and title documents to make sure the deal closes on time.", ...]</code> | <code>[20, 19, 18, 17, 16, ...]</code> |
263
+ | <code>diseases and symptoms list a-z</code> | <code>['Diseases A to Z. Find disease information including general information, medical symptoms and treatments. Disease information also contains links to the primary and secondary symptoms of the disease. A : B : C : D : E : F : G : H : I : J : K : L : M : N : O : P : Q : R : S : T : U : V : W : X : Y : Z.', 'The A to Z index of medical diseases comprises links to topics with information about that particular health condition. The information included in these topics comprises symptoms, causes, diagnosis, prevention, and treatment measures for the respective condition.', 'A to Z List of Medical Diseases, Disorders and Medical Conditions. Below is an alphabetical list of diseases with information on a wide variety of common medical conditions, diseases, disorders, syndromes, illnesses, and injuries.', "Diseases & Conditions A-Z List. Search by Letter: Click Here ». Find relevant and reliable medical information on diseases and conditions. Find your medical topic by using the comprehensive A-Z list above, click on the browse health centers below or one of our categorized listings of health and medical conditions. 1 Allergies. Alzheimer's.", 'Find relevant and reliable medical information on diseases and conditions. Find your medical topic by using the comprehensive A-Z list above, click on the browse health centers below or one of our categorized listings of health and medical conditions.', ...]</code> | <code>[20, 19, 18, 17, 16, ...]</code> |
264
+ * Loss: [<code>ADRMSELoss</code>](https://sbert.net/docs/package_reference/cross_encoder/losses.html#adrmseloss) with these parameters:
265
+ ```json
266
+ {
267
+ "alpha": 1.0,
268
+ "activation_fn": "torch.nn.modules.linear.Identity",
269
+ "mini_batch_size": 16
270
+ }
271
+ ```
272
+
273
+ ### Evaluation Dataset
274
+
275
+ #### msmarco
276
+
277
+ * Dataset: [msmarco](https://huggingface.co/datasets/sentence-transformers/msmarco) at [2a16a97](https://huggingface.co/datasets/sentence-transformers/msmarco/tree/2a16a97b87802876f28636ca673e872248336951)
278
+ * Size: 1,000 evaluation samples
279
+ * Columns: <code>query</code>, <code>docs</code>, and <code>scores</code>
280
+ * Approximate statistics based on the first 1000 samples:
281
+ | | query | docs | scores |
282
+ |:--------|:---------------------------------------------------------------------------------|:------------------------------------|:------------------------------------|
283
+ | type | string | list | list |
284
+ | details | <ul><li>min: 4 tokens</li><li>mean: 8.97 tokens</li><li>max: 29 tokens</li></ul> | <ul><li>size: 20 elements</li></ul> | <ul><li>size: 20 elements</li></ul> |
285
+ * Samples:
286
+ | query | docs | scores |
287
+ |:---------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------|
288
+ | <code>herbs for lowering blood sugar</code> | <code>['Ten Herbs To Lower Blood Sugar Naturally. 1. Cinnamon â\x80\x93 As little as half a teaspoon a day can help improve your insulin sensitivity and control of blood sugar. If you donâ\x80\x99t enjoy sprinkling cinnamon in your daily yogurt or oatmeal, you can easily find capsules of cinnamon to make a cinnamon regimen easier. 2. Sage â\x80\x93 In one German study, blood sugar levels were reduced when sage infusions were consumed by diabetics on an empty stomach. Sage is one of many powerful herbs.', 'Ten Herbs To Lower Blood Sugar Naturally. 1. Cinnamon â\x80\x93 As little as half a teaspoon a day can help improve your insulin sensitivity and control of blood sugar. If you donâ\x80\x99t enjoy sprinkling cinnamon in your daily yogurt or oatmeal, you can easily find capsules of cinnamon to make a cinnamon regimen easier.', 'Cinnamon, people who have diabetes, is commonly used to reduce blood sugar and cholesterol level in blood. Onions contain falconoid and high sulfur which if consumed two ounces daily by diabetics reduces blood sugar significantly. Garlic is a beneficial herb is another of the foods that lower blood sugar. Garlic helps to elevate production of insulin and its sensitivity. Nuts.', '#7: Ginseng. Ginseng is a popular herb that has many healing properties, but for diabetes patients, its daily use can help to lower blood sugar levels. This is because of its ability to increase the use of glucose in the cells and to reduce the carbohydrates absorption rates. In turn, this helps to lower the blood sugar levels in the body.', 'Cinnamon, people who have diabetes, is commonly used to reduce blood sugar and cholesterol level in blood. Onions contain falconoid and high sulfur which if consumed two ounces daily by diabetics reduces blood sugar significantly. Garlic is a beneficial herb is another of the foods that lower blood sugar.', ...]</code> | <code>[20, 19, 18, 17, 16, ...]</code> |
289
+ | <code>define oxygen saturation</code> | <code>['Definition: Oxygen Saturation. Oxygen saturation refers to the extent to which hemoglobin is saturated with oxygen. Hemoglobin is an element in the blood that binds with oxygen to carry it through the bloodstream to the organs, tissues and cells of the body. Normal oxygen saturation is usually between 96% and 98%.', 'Oxygen saturation is a term referring to the fraction of oxygen-saturated hemoglobin relative to total hemoglobin (unsaturated + saturated) in the blood. The human body requires and regulates a very precise and specific balance of oxygen in the blood. Normal blood oxygen levels in humans are considered 95-100 percent.', 'Oxygen saturation (medicine) Oxygen saturation is a term referring to the fraction of oxygen-saturated hemoglobin relative to total hemoglobin (unsaturated + saturated) in the blood. The human body requires and regulates a very precise and specific balance of oxygen in the blood. Normal blood oxygen levels in humans are considered 95-100 percent.', 'Oxygen saturation (medicine) Oxygen saturation is a term referring to the fraction of oxygen-saturated hemoglobin relative to total hemoglobin (unsaturated + saturated) in the blood. The human body requires and regulates a very precise and specific balance of oxygen in the blood. Normal blood oxygen levels in humans are considered 95-100 percent. If the level is below 90 percent, it is considered low resulting in hypoxemia. Blood oxygen levels below 80 percent may compromise organ function, such as the brain and heart, and should be promptly addressed. Continued low oxygen levels may lead to respiratory or cardiac arrest.', 'Oxygen saturation is a term referring to the fraction of oxygen-saturated hemoglobin relative to total hemoglobin (unsaturated + saturated) in the blood. The human body requires and regulates a very precise and specific balance of oxygen in the blood. Normal blood oxygen levels in humans are considered 95-100 percent.n medicine, oxygen saturation (SO 2), commonly referred to as sats, measures the percentage of hemoglobin binding sites in the bloodstream occupied by oxygen. At low partial pressures of oxygen, most hemoglobin is deoxygenated.', ...]</code> | <code>[20, 19, 18, 17, 16, ...]</code> |
290
+ | <code>what type of chemical is copper</code> | <code>['Copper is a chemical element with symbol Cu (from Latin: cuprum) and atomic number 29. It is a ductile metal with very high thermal and electrical conductivity. Pure copper is soft and malleable; a freshly exposed surface has a reddish-orange color.It is used as a conductor of heat and electricity, a building material, and a constituent of various metal alloys.ts compounds are commonly encountered as copper(II) salts, which often impart blue or green colors to minerals such as azurite and turquoise and have been widely used historically as pigments. Architectural structures built with copper corrode to give green verdigris (or patina).', 'Copper is a chemical element with symbol Cu (from Latin: cuprum) and atomic number 29. It is a ductile metal with very high thermal and electrical conductivity. Pure copper is soft and malleable; a freshly exposed surface has a reddish-orange color.It is used as a conductor of heat and electricity, a building material, and a constituent of various metal alloys.ttrium barium copper oxide (YBa 2 Cu 3 O 7) consists of both Cu(II) and Cu(III) centres. Like oxide, fluoride is a highly basic anion and is known to stabilize metal ions in high oxidation states. Indeed, both copper(III) and even copper(IV) fluorides are known, K3CuF6 and Cs2CuF6, respectively.', 'When it joins with other atoms, copper behaves chemically in two quite different ways to form compounds that are either described as copper (I), also known as cuprous, or copper (II), also known as cupric.The cupric compounds are more stable; cuprous ones generally turn into cupric ones.The two most important copper compounds are copper (II) sulphate, which is bright blue and used in agriculture and medicine, and copper (II) chloride, which is used as a wood preservative and in the printing and dyeing industries.he cupric compounds are more stable; cuprous ones generally turn into cupric ones. The two most important copper compounds are copper (II) sulphate, which is bright blue and used in agriculture and medicine, and copper (II) chloride, which is used as a wood preservative and in the printing and dyeing industries.', 'Copper is a chemical element. It is the 29th element in the periodic table and has 29 protons. Its mass number is 63.55. It is a transition metal in the middle of the periodic table. The symbol for copper is Cu, which comes from the Latin word cuprum, which, in turn, came from the Latin word for the island of Cyprus, where copper was found.', 'Copper is a chemical element. It is the 29th element in the periodic table and has 29 protons. Its mass number is 63.55. It is a transition metal in the middle of the periodic table.', ...]</code> | <code>[20, 19, 18, 17, 16, ...]</code> |
291
+ * Loss: [<code>ADRMSELoss</code>](https://sbert.net/docs/package_reference/cross_encoder/losses.html#adrmseloss) with these parameters:
292
+ ```json
293
+ {
294
+ "alpha": 1.0,
295
+ "activation_fn": "torch.nn.modules.linear.Identity",
296
+ "mini_batch_size": 16
297
+ }
298
+ ```
299
+
300
+ ### Training Hyperparameters
301
+ #### Non-Default Hyperparameters
302
+
303
+ - `per_device_train_batch_size`: 16
304
+ - `num_train_epochs`: 1
305
+ - `learning_rate`: 2e-05
306
+ - `warmup_steps`: 0.1
307
+ - `bf16`: True
308
+ - `per_device_eval_batch_size`: 16
309
+ - `load_best_model_at_end`: True
310
+ - `seed`: 12
311
+
312
+ #### All Hyperparameters
313
+ <details><summary>Click to expand</summary>
314
+
315
+ - `per_device_train_batch_size`: 16
316
+ - `num_train_epochs`: 1
317
+ - `max_steps`: -1
318
+ - `learning_rate`: 2e-05
319
+ - `lr_scheduler_type`: linear
320
+ - `lr_scheduler_kwargs`: None
321
+ - `warmup_steps`: 0.1
322
+ - `optim`: adamw_torch_fused
323
+ - `optim_args`: None
324
+ - `weight_decay`: 0.0
325
+ - `adam_beta1`: 0.9
326
+ - `adam_beta2`: 0.999
327
+ - `adam_epsilon`: 1e-08
328
+ - `optim_target_modules`: None
329
+ - `gradient_accumulation_steps`: 1
330
+ - `average_tokens_across_devices`: True
331
+ - `max_grad_norm`: 1.0
332
+ - `label_smoothing_factor`: 0.0
333
+ - `bf16`: True
334
+ - `fp16`: False
335
+ - `bf16_full_eval`: False
336
+ - `fp16_full_eval`: False
337
+ - `tf32`: None
338
+ - `gradient_checkpointing`: False
339
+ - `gradient_checkpointing_kwargs`: None
340
+ - `torch_compile`: False
341
+ - `torch_compile_backend`: None
342
+ - `torch_compile_mode`: None
343
+ - `use_liger_kernel`: False
344
+ - `liger_kernel_config`: None
345
+ - `use_cache`: False
346
+ - `neftune_noise_alpha`: None
347
+ - `torch_empty_cache_steps`: None
348
+ - `auto_find_batch_size`: False
349
+ - `log_on_each_node`: True
350
+ - `logging_nan_inf_filter`: True
351
+ - `include_num_input_tokens_seen`: no
352
+ - `log_level`: passive
353
+ - `log_level_replica`: warning
354
+ - `disable_tqdm`: False
355
+ - `project`: huggingface
356
+ - `trackio_space_id`: trackio
357
+ - `per_device_eval_batch_size`: 16
358
+ - `prediction_loss_only`: True
359
+ - `eval_on_start`: False
360
+ - `eval_do_concat_batches`: True
361
+ - `eval_use_gather_object`: False
362
+ - `eval_accumulation_steps`: None
363
+ - `include_for_metrics`: []
364
+ - `batch_eval_metrics`: False
365
+ - `save_only_model`: False
366
+ - `save_on_each_node`: False
367
+ - `enable_jit_checkpoint`: False
368
+ - `push_to_hub`: False
369
+ - `hub_private_repo`: None
370
+ - `hub_model_id`: None
371
+ - `hub_strategy`: every_save
372
+ - `hub_always_push`: False
373
+ - `hub_revision`: None
374
+ - `load_best_model_at_end`: True
375
+ - `ignore_data_skip`: False
376
+ - `restore_callback_states_from_checkpoint`: False
377
+ - `full_determinism`: False
378
+ - `seed`: 12
379
+ - `data_seed`: None
380
+ - `use_cpu`: False
381
+ - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
382
+ - `parallelism_config`: None
383
+ - `dataloader_drop_last`: False
384
+ - `dataloader_num_workers`: 0
385
+ - `dataloader_pin_memory`: True
386
+ - `dataloader_persistent_workers`: False
387
+ - `dataloader_prefetch_factor`: None
388
+ - `remove_unused_columns`: True
389
+ - `label_names`: None
390
+ - `train_sampling_strategy`: random
391
+ - `length_column_name`: length
392
+ - `ddp_find_unused_parameters`: None
393
+ - `ddp_bucket_cap_mb`: None
394
+ - `ddp_broadcast_buffers`: False
395
+ - `ddp_backend`: None
396
+ - `ddp_timeout`: 1800
397
+ - `fsdp`: []
398
+ - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
399
+ - `deepspeed`: None
400
+ - `debug`: []
401
+ - `skip_memory_metrics`: True
402
+ - `do_predict`: False
403
+ - `resume_from_checkpoint`: None
404
+ - `warmup_ratio`: None
405
+ - `local_rank`: -1
406
+ - `prompts`: None
407
+ - `batch_sampler`: batch_sampler
408
+ - `multi_dataset_batch_sampler`: proportional
409
+ - `router_mapping`: {}
410
+ - `learning_rate_mapping`: {}
411
+
412
+ </details>
413
+
414
+ ### Training Logs
415
+ | Epoch | Step | Training Loss | Validation Loss | NanoMSMARCO_R100_ndcg@10 | NanoNFCorpus_R100_ndcg@10 | NanoNQ_R100_ndcg@10 | NanoBEIR_R100_mean_ndcg@10 |
416
+ |:-------:|:-------:|:-------------:|:---------------:|:------------------------:|:-------------------------:|:--------------------:|:--------------------------:|
417
+ | -1 | -1 | - | - | 0.0300 (-0.5104) | 0.2528 (-0.0723) | 0.0168 (-0.4839) | 0.0999 (-0.3555) |
418
+ | 0.0018 | 1 | 14.3181 | - | - | - | - | - |
419
+ | 0.0515 | 29 | 14.3139 | - | - | - | - | - |
420
+ | 0.1030 | 58 | 14.3144 | - | - | - | - | - |
421
+ | 0.1545 | 87 | 14.2601 | - | - | - | - | - |
422
+ | 0.2007 | 113 | - | 12.6358 | 0.4474 (-0.0930) | 0.3621 (+0.0371) | 0.5767 (+0.0761) | 0.4621 (+0.0067) |
423
+ | 0.2060 | 116 | 13.2220 | - | - | - | - | - |
424
+ | 0.2575 | 145 | 12.7249 | - | - | - | - | - |
425
+ | 0.3091 | 174 | 12.5317 | - | - | - | - | - |
426
+ | 0.3606 | 203 | 11.9536 | - | - | - | - | - |
427
+ | 0.4014 | 226 | - | 11.6810 | 0.6019 (+0.0615) | 0.4339 (+0.1088) | 0.6231 (+0.1225) | 0.5530 (+0.0976) |
428
+ | 0.4121 | 232 | 11.7751 | - | - | - | - | - |
429
+ | 0.4636 | 261 | 11.9774 | - | - | - | - | - |
430
+ | 0.5151 | 290 | 11.4589 | - | - | - | - | - |
431
+ | 0.5666 | 319 | 11.7365 | - | - | - | - | - |
432
+ | 0.6021 | 339 | - | 11.0220 | 0.6192 (+0.0788) | 0.4210 (+0.0960) | 0.6315 (+0.1308) | 0.5572 (+0.1019) |
433
+ | 0.6181 | 348 | 11.4237 | - | - | - | - | - |
434
+ | 0.6696 | 377 | 11.4196 | - | - | - | - | - |
435
+ | 0.7211 | 406 | 11.2971 | - | - | - | - | - |
436
+ | 0.7726 | 435 | 11.2319 | - | - | - | - | - |
437
+ | 0.8028 | 452 | - | 10.8140 | 0.6179 (+0.0774) | 0.4177 (+0.0927) | 0.6458 (+0.1452) | 0.5605 (+0.1051) |
438
+ | 0.8242 | 464 | 11.4555 | - | - | - | - | - |
439
+ | 0.8757 | 493 | 11.2641 | - | - | - | - | - |
440
+ | 0.9272 | 522 | 11.1994 | - | - | - | - | - |
441
+ | 0.9787 | 551 | 11.1511 | - | - | - | - | - |
442
+ | **1.0** | **563** | **-** | **10.7337** | **0.6073 (+0.0669)** | **0.4303 (+0.1053)** | **0.6500 (+0.1493)** | **0.5625 (+0.1071)** |
443
+ | -1 | -1 | - | - | 0.6073 (+0.0669) | 0.4303 (+0.1053) | 0.6500 (+0.1493) | 0.5625 (+0.1071) |
444
+
445
+ * The bold row denotes the saved checkpoint.
446
+
447
+ ### Training Time
448
+ - **Training**: 11.3 minutes
449
+ - **Evaluation**: 5.0 minutes
450
+ - **Total**: 16.4 minutes
451
+
452
+ ### Framework Versions
453
+ - Python: 3.11.6
454
+ - Sentence Transformers: 5.5.0.dev0
455
+ - Transformers: 5.5.0
456
+ - PyTorch: 2.10.0+cu128
457
+ - Accelerate: 1.13.0.dev0
458
+ - Datasets: 4.8.4
459
+ - Tokenizers: 0.22.2
460
+
461
+ ## Additional Resources
462
+
463
+ - [Training and Finetuning Reranker Models with Sentence Transformers](https://huggingface.co/blog/train-reranker): the end-to-end guide for training or finetuning Cross Encoder (reranker) models.
464
+ - [Multimodal Embedding & Reranker Models with Sentence Transformers](https://huggingface.co/blog/multimodal-sentence-transformers): use text, image, audio, and video reranker models through the same API.
465
+ - [Training and Finetuning Multimodal Embedding & Reranker Models with Sentence Transformers](https://huggingface.co/blog/train-multimodal-sentence-transformers): training multimodal Cross Encoders, including Any-to-Any and Feature Extraction architectures.
466
+
467
+ ## Citation
468
+
469
+ ### BibTeX
470
+
471
+ #### Sentence Transformers
472
+ ```bibtex
473
+ @inproceedings{reimers-2019-sentence-bert,
474
+ title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
475
+ author = "Reimers, Nils and Gurevych, Iryna",
476
+ booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
477
+ month = "11",
478
+ year = "2019",
479
+ publisher = "Association for Computational Linguistics",
480
+ url = "https://arxiv.org/abs/1908.10084",
481
+ }
482
+ ```
483
+
484
+ #### ADRMSELoss
485
+ ```bibtex
486
+ @inproceedings{schlatt2025rankdistillm,
487
+ title={Rank-DistiLLM: Closing the Effectiveness Gap Between Cross-Encoders and LLMs for Passage Re-ranking},
488
+ author={Schlatt, Ferdinand and Fröbe, Maik and Scells, Harrisen and Zhuang, Shengyao and Koopman, Bevan and Zuccon, Guido and Stein, Benno and Potthast, Martin and Hagen, Matthias},
489
+ booktitle={Advances in Information Retrieval (ECIR 2025)},
490
+ year={2025},
491
+ doi={10.1007/978-3-031-88714-7_31},
492
+ }
493
+ ```
494
+
495
+ <!--
496
+ ## Glossary
497
+
498
+ *Clearly define terms in order to be accessible across audiences.*
499
+ -->
500
+
501
+ <!--
502
+ ## Model Card Authors
503
+
504
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
505
+ -->
506
+
507
+ <!--
508
+ ## Model Card Contact
509
+
510
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
511
+ -->
config.json ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_cross_attention": false,
3
+ "architectures": [
4
+ "BertForSequenceClassification"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "bos_token_id": null,
8
+ "classifier_dropout": null,
9
+ "dtype": "float32",
10
+ "eos_token_id": null,
11
+ "hidden_act": "gelu",
12
+ "hidden_dropout_prob": 0.1,
13
+ "hidden_size": 384,
14
+ "id2label": {
15
+ "0": "LABEL_0"
16
+ },
17
+ "initializer_range": 0.02,
18
+ "intermediate_size": 1536,
19
+ "is_decoder": false,
20
+ "label2id": {
21
+ "LABEL_0": 0
22
+ },
23
+ "layer_norm_eps": 1e-12,
24
+ "max_position_embeddings": 512,
25
+ "model_type": "bert",
26
+ "num_attention_heads": 12,
27
+ "num_hidden_layers": 12,
28
+ "pad_token_id": 0,
29
+ "tie_word_embeddings": true,
30
+ "transformers_version": "5.5.0",
31
+ "type_vocab_size": 2,
32
+ "use_cache": false,
33
+ "vocab_size": 30522
34
+ }
config_sentence_transformers.json ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "__version__": {
3
+ "pytorch": "2.10.0+cu128",
4
+ "sentence_transformers": "5.5.0.dev0",
5
+ "transformers": "5.5.0"
6
+ },
7
+ "activation_fn": "torch.nn.modules.activation.Sigmoid",
8
+ "default_prompt_name": null,
9
+ "model_type": "CrossEncoder",
10
+ "prompts": {}
11
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5dd17a35d8b5b027da8b7beccebaeb8eed8d46ef91ef1b8377f2c8ce8e400137
3
+ size 133464812
modules.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "idx": 0,
4
+ "name": "0",
5
+ "path": "",
6
+ "type": "sentence_transformers.base.modules.transformer.Transformer"
7
+ }
8
+ ]
sentence_bert_config.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "transformer_task": "sequence-classification",
3
+ "modality_config": {
4
+ "text": {
5
+ "method": "forward",
6
+ "method_output_name": "logits"
7
+ }
8
+ },
9
+ "module_output_name": "scores"
10
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "backend": "tokenizers",
3
+ "cls_token": "[CLS]",
4
+ "do_lower_case": true,
5
+ "is_local": false,
6
+ "mask_token": "[MASK]",
7
+ "model_max_length": 512,
8
+ "pad_token": "[PAD]",
9
+ "sep_token": "[SEP]",
10
+ "strip_accents": null,
11
+ "tokenize_chinese_chars": true,
12
+ "tokenizer_class": "BertTokenizer",
13
+ "unk_token": "[UNK]"
14
+ }