tomaarsen/reranker-msmarco-MiniLM-L12-H384-uncased-adrmse

CrossEncoder based on microsoft/MiniLM-L12-H384-uncased

This is a Cross Encoder model finetuned from microsoft/MiniLM-L12-H384-uncased on the msmarco dataset using the sentence-transformers library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.

Model Details

Model Description

• Model Type: Cross Encoder
• Base model: microsoft/MiniLM-L12-H384-uncased
• Maximum Sequence Length: 512 tokens
• Number of Output Labels: 1 label
• Supported Modality: Text
• Training Dataset:
  • msmarco
• Language: en

Model Sources

• Documentation: Sentence Transformers Documentation
• Documentation: Cross Encoder Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Cross Encoders on Hugging Face

Full Model Architecture

CrossEncoder(
  (0): Transformer({'transformer_task': 'sequence-classification', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'logits'}}, 'module_output_name': 'scores', 'architecture': 'BertForSequenceClassification'})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import CrossEncoder

# Download from the 🤗 Hub
model = CrossEncoder("tomaarsen/reranker-msmarco-MiniLM-L12-H384-uncased-adrmse")
# Get scores for pairs of inputs
pairs = [
    ['herbs for lowering blood sugar', 'Ten Herbs To Lower Blood Sugar Naturally. 1. Cinnamon â\x80\x93 As little as half a teaspoon a day can help improve your insulin sensitivity and control of blood sugar. If you donâ\x80\x99t enjoy sprinkling cinnamon in your daily yogurt or oatmeal, you can easily find capsules of cinnamon to make a cinnamon regimen easier. 2. Sage â\x80\x93 In one German study, blood sugar levels were reduced when sage infusions were consumed by diabetics on an empty stomach. Sage is one of many powerful herbs.'],
    ['herbs for lowering blood sugar', 'Ten Herbs To Lower Blood Sugar Naturally. 1. Cinnamon â\x80\x93 As little as half a teaspoon a day can help improve your insulin sensitivity and control of blood sugar. If you donâ\x80\x99t enjoy sprinkling cinnamon in your daily yogurt or oatmeal, you can easily find capsules of cinnamon to make a cinnamon regimen easier.'],
    ['herbs for lowering blood sugar', 'Cinnamon, people who have diabetes, is commonly used to reduce blood sugar and cholesterol level in blood. Onions contain falconoid and high sulfur which if consumed two ounces daily by diabetics reduces blood sugar significantly. Garlic is a beneficial herb is another of the foods that lower blood sugar. Garlic helps to elevate production of insulin and its sensitivity. Nuts.'],
    ['herbs for lowering blood sugar', '#7: Ginseng. Ginseng is a popular herb that has many healing properties, but for diabetes patients, its daily use can help to lower blood sugar levels. This is because of its ability to increase the use of glucose in the cells and to reduce the carbohydrates absorption rates. In turn, this helps to lower the blood sugar levels in the body.'],
    ['herbs for lowering blood sugar', 'Cinnamon, people who have diabetes, is commonly used to reduce blood sugar and cholesterol level in blood. Onions contain falconoid and high sulfur which if consumed two ounces daily by diabetics reduces blood sugar significantly. Garlic is a beneficial herb is another of the foods that lower blood sugar.'],
]
scores = model.predict(pairs)
print(scores)
# [0.7691 0.7417 0.48   0.7066 0.4656]

# Or rank different texts based on similarity to a single text
ranks = model.rank(
    'herbs for lowering blood sugar',
    [
        'Ten Herbs To Lower Blood Sugar Naturally. 1. Cinnamon â\x80\x93 As little as half a teaspoon a day can help improve your insulin sensitivity and control of blood sugar. If you donâ\x80\x99t enjoy sprinkling cinnamon in your daily yogurt or oatmeal, you can easily find capsules of cinnamon to make a cinnamon regimen easier. 2. Sage â\x80\x93 In one German study, blood sugar levels were reduced when sage infusions were consumed by diabetics on an empty stomach. Sage is one of many powerful herbs.',
        'Ten Herbs To Lower Blood Sugar Naturally. 1. Cinnamon â\x80\x93 As little as half a teaspoon a day can help improve your insulin sensitivity and control of blood sugar. If you donâ\x80\x99t enjoy sprinkling cinnamon in your daily yogurt or oatmeal, you can easily find capsules of cinnamon to make a cinnamon regimen easier.',
        'Cinnamon, people who have diabetes, is commonly used to reduce blood sugar and cholesterol level in blood. Onions contain falconoid and high sulfur which if consumed two ounces daily by diabetics reduces blood sugar significantly. Garlic is a beneficial herb is another of the foods that lower blood sugar. Garlic helps to elevate production of insulin and its sensitivity. Nuts.',
        '#7: Ginseng. Ginseng is a popular herb that has many healing properties, but for diabetes patients, its daily use can help to lower blood sugar levels. This is because of its ability to increase the use of glucose in the cells and to reduce the carbohydrates absorption rates. In turn, this helps to lower the blood sugar levels in the body.',
        'Cinnamon, people who have diabetes, is commonly used to reduce blood sugar and cholesterol level in blood. Onions contain falconoid and high sulfur which if consumed two ounces daily by diabetics reduces blood sugar significantly. Garlic is a beneficial herb is another of the foods that lower blood sugar.',
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

Evaluation

Metrics

Cross Encoder Reranking

• Datasets: NanoMSMARCO_R100, NanoNFCorpus_R100 and NanoNQ_R100
• Evaluated with CrossEncoderRerankingEvaluator with these parameters:

  {
      "at_k": 10,
      "always_rerank_positives": true
  }
  

Metric	NanoMSMARCO_R100	NanoNFCorpus_R100	NanoNQ_R100
map	0.5412 (+0.0516)	0.3600 (+0.0990)	0.5805 (+0.1609)
mrr@10	0.5341 (+0.0566)	0.6053 (+0.1054)	0.5950 (+0.1683)
ndcg@10	0.6073 (+0.0669)	0.4303 (+0.1053)	0.6500 (+0.1493)

Cross Encoder Nano BEIR

• Dataset: NanoBEIR_R100_mean
• Evaluated with CrossEncoderNanoBEIREvaluator with these parameters:

  {
      "dataset_names": [
          "msmarco",
          "nfcorpus",
          "nq"
      ],
      "dataset_id": "sentence-transformers/NanoBEIR-en",
      "rerank_k": 100,
      "at_k": 10,
      "always_rerank_positives": true
  }
  

Metric	Value
map	0.4939 (+0.1038)
mrr@10	0.5781 (+0.1101)
ndcg@10	0.5625 (+0.1071)

Training Details

Training Dataset

msmarco

• Dataset: msmarco at 2a16a97
• Size: 9,000 training samples
• Columns: query, docs, and scores
• Approximate statistics based on the first 1000 samples:

  	query	docs	scores
  type	string	list	list
  details	min: 4 tokens mean: 8.94 tokens max: 33 tokens	size: 20 elements	size: 20 elements

• Samples:

  query	docs	scores
  what is medical d. o.	['Medical Definition of D.O. D.O.: Abbreviation for Doctor of Osteopathy, an osteopathic physician. Osteopathy is a system of therapy founded in the 19th century based on the concept that the body can formulate its own remedies against diseases when the body is in a normal structural relationship, has a normal environment and enjoys good nutrition.', 'Doctor of Osteopathic Medicine (D.O.) is a professional doctoral degree for physicians and surgeons offered by medical schools in the United States. A D.O. degree graduate may become licensed as an osteopathic physician, having equivalent rights, privileges, and responsibilities as a physician who has earned the Doctor of Medicine (M.D.) degree. D.O. physicians are licensed to practice the full scope of medicine and surgery in sixty five countries, and all fifty states in the US.', 'Doctor of Osteopathic Medicine (D.O.) is a professional doctoral degree for physicians and surgeons offered by medical schools in the United States. A D.O. degree graduate may become licensed as an osteopathic physician, having equivalent rights, privileges, and responsibilities as a physician who has earned the Doctor of Medicine (M.D.) degree. D.O. physicians are licensed to practice the full scope of medicine and surgery in sixty-five countries, and in all fifty states.', 'Doctor of Osteopathic Medicine (D.O.) is a professional doctoral degree for physicians and surgeons offered by medical schools in the United States. A D.O. degree graduate may become licensed as an osteopathic physician, having equivalent rights, privileges, and responsibilities as a physician who has earned the Doctor of Medicine (M.D.) degree.', 'A doctor of osteopathic medicine (D.O.) is a fully licensed medical doctor offering all the techniques and treatments of modern medicine with the added benefits of hands-on diagnostics and a holistic philosophy.', ...]	[20, 19, 18, 17, 16, ...]
  pending sale definition	['Sale pending means that there is a transaction that is going through Escrow but that has not closed yet. Translation it is too late to put in an offer now. If it falls out of escrow and the seller decides if he wants to put it back on the market then you can put in your offer.', 'Answers. 1 Sale pending means that there is a transaction that is going through Escrow but that has not closed yet. Translation it is too late to put in an offer now. If it falls out of escrow and the seller decides if he wants to put it back on the market then you can put in your offer.', '· just now. 1 Sale pending means that there is a transaction that is going through Escrow but that has not closed yet. 2 Sale pending can mean anything from an offer being accepted or contracts signed. 3 It means the buyer and seller have agreed on a contract, but closing has not happened yet.', "Thereâ\x80\x99s just one problem: It's sale pending. But what does sale pending mean exactly? Are you too late or do you still have a shot? The short answer: If a home you love is pending sale, donâ\x80\x99t give up hope. What does sale pending mean? A pending sale means the seller has an offer but hasn't closed yet. (This is different from a contingent sale.) A property is placed in pending status the minute a contract is executed.", "What does pending mean in real estate? Definition of Pending A home's status is changed to Pending or Pending Offer once it's set to close and all contingencies have been satisfied or waived. This is when the lender and the escrow agent or attorney process the loan and title documents to make sure the deal closes on time.", ...]	[20, 19, 18, 17, 16, ...]
  diseases and symptoms list a-z	['Diseases A to Z. Find disease information including general information, medical symptoms and treatments. Disease information also contains links to the primary and secondary symptoms of the disease. A : B : C : D : E : F : G : H : I : J : K : L : M : N : O : P : Q : R : S : T : U : V : W : X : Y : Z.', 'The A to Z index of medical diseases comprises links to topics with information about that particular health condition. The information included in these topics comprises symptoms, causes, diagnosis, prevention, and treatment measures for the respective condition.', 'A to Z List of Medical Diseases, Disorders and Medical Conditions. Below is an alphabetical list of diseases with information on a wide variety of common medical conditions, diseases, disorders, syndromes, illnesses, and injuries.', "Diseases & Conditions A-Z List. Search by Letter: Click Here ». Find relevant and reliable medical information on diseases and conditions. Find your medical topic by using the comprehensive A-Z list above, click on the browse health centers below or one of our categorized listings of health and medical conditions. 1 Allergies. Alzheimer's.", 'Find relevant and reliable medical information on diseases and conditions. Find your medical topic by using the comprehensive A-Z list above, click on the browse health centers below or one of our categorized listings of health and medical conditions.', ...]	[20, 19, 18, 17, 16, ...]

• Loss: ADRMSELoss with these parameters:

  {
      "alpha": 1.0,
      "activation_fn": "torch.nn.modules.linear.Identity",
      "mini_batch_size": 16
  }
  

Evaluation Dataset

msmarco

• Dataset: msmarco at 2a16a97
• Size: 1,000 evaluation samples
• Columns: query, docs, and scores
• Approximate statistics based on the first 1000 samples:

  	query	docs	scores
  type	string	list	list
  details	min: 4 tokens mean: 8.97 tokens max: 29 tokens	size: 20 elements	size: 20 elements

• Samples:

  query	docs	scores
  herbs for lowering blood sugar	['Ten Herbs To Lower Blood Sugar Naturally. 1. Cinnamon â\x80\x93 As little as half a teaspoon a day can help improve your insulin sensitivity and control of blood sugar. If you donâ\x80\x99t enjoy sprinkling cinnamon in your daily yogurt or oatmeal, you can easily find capsules of cinnamon to make a cinnamon regimen easier. 2. Sage â\x80\x93 In one German study, blood sugar levels were reduced when sage infusions were consumed by diabetics on an empty stomach. Sage is one of many powerful herbs.', 'Ten Herbs To Lower Blood Sugar Naturally. 1. Cinnamon â\x80\x93 As little as half a teaspoon a day can help improve your insulin sensitivity and control of blood sugar. If you donâ\x80\x99t enjoy sprinkling cinnamon in your daily yogurt or oatmeal, you can easily find capsules of cinnamon to make a cinnamon regimen easier.', 'Cinnamon, people who have diabetes, is commonly used to reduce blood sugar and cholesterol level in blood. Onions contain falconoid and high sulfur which if consumed two ounces daily by diabetics reduces blood sugar significantly. Garlic is a beneficial herb is another of the foods that lower blood sugar. Garlic helps to elevate production of insulin and its sensitivity. Nuts.', '#7: Ginseng. Ginseng is a popular herb that has many healing properties, but for diabetes patients, its daily use can help to lower blood sugar levels. This is because of its ability to increase the use of glucose in the cells and to reduce the carbohydrates absorption rates. In turn, this helps to lower the blood sugar levels in the body.', 'Cinnamon, people who have diabetes, is commonly used to reduce blood sugar and cholesterol level in blood. Onions contain falconoid and high sulfur which if consumed two ounces daily by diabetics reduces blood sugar significantly. Garlic is a beneficial herb is another of the foods that lower blood sugar.', ...]	[20, 19, 18, 17, 16, ...]
  define oxygen saturation	['Definition: Oxygen Saturation. Oxygen saturation refers to the extent to which hemoglobin is saturated with oxygen. Hemoglobin is an element in the blood that binds with oxygen to carry it through the bloodstream to the organs, tissues and cells of the body. Normal oxygen saturation is usually between 96% and 98%.', 'Oxygen saturation is a term referring to the fraction of oxygen-saturated hemoglobin relative to total hemoglobin (unsaturated + saturated) in the blood. The human body requires and regulates a very precise and specific balance of oxygen in the blood. Normal blood oxygen levels in humans are considered 95-100 percent.', 'Oxygen saturation (medicine) Oxygen saturation is a term referring to the fraction of oxygen-saturated hemoglobin relative to total hemoglobin (unsaturated + saturated) in the blood. The human body requires and regulates a very precise and specific balance of oxygen in the blood. Normal blood oxygen levels in humans are considered 95-100 percent.', 'Oxygen saturation (medicine) Oxygen saturation is a term referring to the fraction of oxygen-saturated hemoglobin relative to total hemoglobin (unsaturated + saturated) in the blood. The human body requires and regulates a very precise and specific balance of oxygen in the blood. Normal blood oxygen levels in humans are considered 95-100 percent. If the level is below 90 percent, it is considered low resulting in hypoxemia. Blood oxygen levels below 80 percent may compromise organ function, such as the brain and heart, and should be promptly addressed. Continued low oxygen levels may lead to respiratory or cardiac arrest.', 'Oxygen saturation is a term referring to the fraction of oxygen-saturated hemoglobin relative to total hemoglobin (unsaturated + saturated) in the blood. The human body requires and regulates a very precise and specific balance of oxygen in the blood. Normal blood oxygen levels in humans are considered 95-100 percent.n medicine, oxygen saturation (SO 2), commonly referred to as sats, measures the percentage of hemoglobin binding sites in the bloodstream occupied by oxygen. At low partial pressures of oxygen, most hemoglobin is deoxygenated.', ...]	[20, 19, 18, 17, 16, ...]
  what type of chemical is copper	['Copper is a chemical element with symbol Cu (from Latin: cuprum) and atomic number 29. It is a ductile metal with very high thermal and electrical conductivity. Pure copper is soft and malleable; a freshly exposed surface has a reddish-orange color.It is used as a conductor of heat and electricity, a building material, and a constituent of various metal alloys.ts compounds are commonly encountered as copper(II) salts, which often impart blue or green colors to minerals such as azurite and turquoise and have been widely used historically as pigments. Architectural structures built with copper corrode to give green verdigris (or patina).', 'Copper is a chemical element with symbol Cu (from Latin: cuprum) and atomic number 29. It is a ductile metal with very high thermal and electrical conductivity. Pure copper is soft and malleable; a freshly exposed surface has a reddish-orange color.It is used as a conductor of heat and electricity, a building material, and a constituent of various metal alloys.ttrium barium copper oxide (YBa 2 Cu 3 O 7) consists of both Cu(II) and Cu(III) centres. Like oxide, fluoride is a highly basic anion and is known to stabilize metal ions in high oxidation states. Indeed, both copper(III) and even copper(IV) fluorides are known, K3CuF6 and Cs2CuF6, respectively.', 'When it joins with other atoms, copper behaves chemically in two quite different ways to form compounds that are either described as copper (I), also known as cuprous, or copper (II), also known as cupric.The cupric compounds are more stable; cuprous ones generally turn into cupric ones.The two most important copper compounds are copper (II) sulphate, which is bright blue and used in agriculture and medicine, and copper (II) chloride, which is used as a wood preservative and in the printing and dyeing industries.he cupric compounds are more stable; cuprous ones generally turn into cupric ones. The two most important copper compounds are copper (II) sulphate, which is bright blue and used in agriculture and medicine, and copper (II) chloride, which is used as a wood preservative and in the printing and dyeing industries.', 'Copper is a chemical element. It is the 29th element in the periodic table and has 29 protons. Its mass number is 63.55. It is a transition metal in the middle of the periodic table. The symbol for copper is Cu, which comes from the Latin word cuprum, which, in turn, came from the Latin word for the island of Cyprus, where copper was found.', 'Copper is a chemical element. It is the 29th element in the periodic table and has 29 protons. Its mass number is 63.55. It is a transition metal in the middle of the periodic table.', ...]	[20, 19, 18, 17, 16, ...]

• Loss: ADRMSELoss with these parameters:

  {
      "alpha": 1.0,
      "activation_fn": "torch.nn.modules.linear.Identity",
      "mini_batch_size": 16
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 16
• num_train_epochs: 1
• learning_rate: 2e-05
• warmup_steps: 0.1
• bf16: True
• per_device_eval_batch_size: 16
• load_best_model_at_end: True
• seed: 12

All Hyperparameters

Click to expand

• per_device_train_batch_size: 16
• num_train_epochs: 1
• max_steps: -1
• learning_rate: 2e-05
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_steps: 0.1
• optim: adamw_torch_fused
• optim_args: None
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• optim_target_modules: None
• gradient_accumulation_steps: 1
• average_tokens_across_devices: True
• max_grad_norm: 1.0
• label_smoothing_factor: 0.0
• bf16: True
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• use_liger_kernel: False
• liger_kernel_config: None
• use_cache: False
• neftune_noise_alpha: None
• torch_empty_cache_steps: None
• auto_find_batch_size: False
• log_on_each_node: True
• logging_nan_inf_filter: True
• include_num_input_tokens_seen: no
• log_level: passive
• log_level_replica: warning
• disable_tqdm: False
• project: huggingface
• trackio_space_id: trackio
• per_device_eval_batch_size: 16
• prediction_loss_only: True
• eval_on_start: False
• eval_do_concat_batches: True
• eval_use_gather_object: False
• eval_accumulation_steps: None
• include_for_metrics: []
• batch_eval_metrics: False
• save_only_model: False
• save_on_each_node: False
• enable_jit_checkpoint: False
• push_to_hub: False
• hub_private_repo: None
• hub_model_id: None
• hub_strategy: every_save
• hub_always_push: False
• hub_revision: None
• load_best_model_at_end: True
• ignore_data_skip: False
• restore_callback_states_from_checkpoint: False
• full_determinism: False
• seed: 12
• data_seed: None
• use_cpu: False
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• dataloader_prefetch_factor: None
• remove_unused_columns: True
• label_names: None
• train_sampling_strategy: random
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• ddp_backend: None
• ddp_timeout: 1800
• fsdp: []
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• deepspeed: None
• debug: []
• skip_memory_metrics: True
• do_predict: False
• resume_from_checkpoint: None
• warmup_ratio: None
• local_rank: -1
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	Validation Loss	NanoMSMARCO_R100_ndcg@10	NanoNFCorpus_R100_ndcg@10	NanoNQ_R100_ndcg@10	NanoBEIR_R100_mean_ndcg@10
-1	-1	-	-	0.0300 (-0.5104)	0.2528 (-0.0723)	0.0168 (-0.4839)	0.0999 (-0.3555)
0.0018	1	14.3181	-	-	-	-	-
0.0515	29	14.3139	-	-	-	-	-
0.1030	58	14.3144	-	-	-	-	-
0.1545	87	14.2601	-	-	-	-	-
0.2007	113	-	12.6358	0.4474 (-0.0930)	0.3621 (+0.0371)	0.5767 (+0.0761)	0.4621 (+0.0067)
0.2060	116	13.2220	-	-	-	-	-
0.2575	145	12.7249	-	-	-	-	-
0.3091	174	12.5317	-	-	-	-	-
0.3606	203	11.9536	-	-	-	-	-
0.4014	226	-	11.6810	0.6019 (+0.0615)	0.4339 (+0.1088)	0.6231 (+0.1225)	0.5530 (+0.0976)
0.4121	232	11.7751	-	-	-	-	-
0.4636	261	11.9774	-	-	-	-	-
0.5151	290	11.4589	-	-	-	-	-
0.5666	319	11.7365	-	-	-	-	-
0.6021	339	-	11.0220	0.6192 (+0.0788)	0.4210 (+0.0960)	0.6315 (+0.1308)	0.5572 (+0.1019)
0.6181	348	11.4237	-	-	-	-	-
0.6696	377	11.4196	-	-	-	-	-
0.7211	406	11.2971	-	-	-	-	-
0.7726	435	11.2319	-	-	-	-	-
0.8028	452	-	10.8140	0.6179 (+0.0774)	0.4177 (+0.0927)	0.6458 (+0.1452)	0.5605 (+0.1051)
0.8242	464	11.4555	-	-	-	-	-
0.8757	493	11.2641	-	-	-	-	-
0.9272	522	11.1994	-	-	-	-	-
0.9787	551	11.1511	-	-	-	-	-
1.0	563	-	10.7337	0.6073 (+0.0669)	0.4303 (+0.1053)	0.6500 (+0.1493)	0.5625 (+0.1071)
-1	-1	-	-	0.6073 (+0.0669)	0.4303 (+0.1053)	0.6500 (+0.1493)	0.5625 (+0.1071)

• The bold row denotes the saved checkpoint.

Training Time

• Training: 11.3 minutes
• Evaluation: 5.0 minutes
• Total: 16.4 minutes

Framework Versions

• Python: 3.11.6
• Sentence Transformers: 5.5.0.dev0
• Transformers: 5.5.0
• PyTorch: 2.10.0+cu128
• Accelerate: 1.13.0.dev0
• Datasets: 4.8.4
• Tokenizers: 0.22.2

Additional Resources

• Training and Finetuning Reranker Models with Sentence Transformers: the end-to-end guide for training or finetuning Cross Encoder (reranker) models.
• Multimodal Embedding & Reranker Models with Sentence Transformers: use text, image, audio, and video reranker models through the same API.
• Training and Finetuning Multimodal Embedding & Reranker Models with Sentence Transformers: training multimodal Cross Encoders, including Any-to-Any and Feature Extraction architectures.

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

ADRMSELoss

@inproceedings{schlatt2025rankdistillm,
    title={Rank-DistiLLM: Closing the Effectiveness Gap Between Cross-Encoders and LLMs for Passage Re-ranking},
    author={Schlatt, Ferdinand and Fröbe, Maik and Scells, Harrisen and Zhuang, Shengyao and Koopman, Bevan and Zuccon, Guido and Stein, Benno and Potthast, Martin and Hagen, Matthias},
    booktitle={Advances in Information Retrieval (ECIR 2025)},
    year={2025},
    doi={10.1007/978-3-031-88714-7_31},
}