---
language:
- en
license: apache-2.0
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- dense
- generated_from_trainer
- dataset_size:90000
- loss:CachedMultipleNegativesBidirectionalRankingLoss
base_model: microsoft/mpnet-base
widget:
- source_sentence: what is the difference between trojan virus and worm?
sentences:
- Worms spread from computer to computer, but unlike a virus, it has the capability
to travel without any help from a person. ... A Trojan horse is not a virus. It
is a destructive program that looks as a genuine application. Unlike viruses,
Trojan horses do not replicate themselves but they can be just as destructive.
- You're usually no longer infectious 24 hours after starting a course of antibiotics,
but this time period can sometimes vary. For example, the antibiotics may take
longer to work if your body takes longer to absorb them, or if you're taking other
medicine that interacts with the antibiotics.
- Eating salt raises the amount of sodium in your bloodstream and wrecks the delicate
balance, reducing the ability of your kidneys to remove the water. The result
is a higher blood pressure due to the extra fluid and extra strain on the delicate
blood vessels leading to the kidneys.
- source_sentence: which are the neighbouring countries of pakistan?
sentences:
- Pakistan is bordered by India on the east, the Arabian Sea on the south, Iran
on the southwest, and Afghanistan on the west and north; in the northeast is the
disputed territory (with India) of Kashmir, of which the part occupied by Pakistan
borders on China.
- Age is a big factor when it comes to how much sleep a dog needs. Just as human
babies need a lot of sleep, the AKC notes your puppy needs 15-20 hours of sleep
a day to help his central nervous system, immune system and muscles develop properly.
- 'Step 1: Connect your iPhone to your computer using n USB cable through any of
the USB ports available on your computer. Step 2: Open iTunes, click the "Files"
tab and check the boxes to sync or transfer your files. Step 3: Select your desired
destination folder for the files and click "Sync" to complete the transfer.'
- source_sentence: what can you do with 1gb of data?
sentences:
- You could even contact your email provider, complain that somebody else is using
your email address, and say that you are worried about your account being compromised.
They're very unlikely to do anything, but if something goes wrong, at least you
can prove you forewarned them.
- 1) Under Section 80CCD(1), investment in Atal Pension Yojana or NPS up to ₹ 1.5
lakh qualifies for income tax deduction. But remember that the total amount of
deduction under sections 80C, 80CCC and 80CCD cannot exceed ₹ 1.5 lakh.
- 1GB (or 1024MB) of data lets you send or receive about 1,000 emails and browse
the Internet for about 20 hours every month. (This limit relates only to your
1GB mobile data allocation; if you are an 'inclusive mobile broadband customer'
you also get 2000 BT Wi-fi wi-fi minutes every month.)
- source_sentence: how many carbon atoms are in carbon dioxide?
sentences:
- For CO2 there is one atom of carbon and two atoms of oxygen. For H2O, there is
one atom of oxygen and two atoms of hydrogen. A molecule can be made of only one
type of atom.
- Avian influenza refers to the disease caused by infection with avian (bird) influenza
(flu) Type A viruses. These viruses occur naturally among wild aquatic birds worldwide
and can infect domestic poultry and other bird and animal species. Avian flu viruses
do not normally infect humans.
- At the end of "Inception," Dom Cobb (Leonardo DiCaprio) finally returns home to
his kids after spending a long time in the dream world. Cobb carries a little
top with him. If the top keeps spinning, that means he is in a dream. ... The
final shot shows the top spinning, but it never reveals whether it falls over.
- source_sentence: is duchenne muscular dystrophy a dominant or recessive trait?
sentences:
- Duchenne muscular dystrophy is inherited in an X-linked recessive pattern. Males
have only one copy of the X chromosome from their mother and one copy of the Y
chromosome from their father. If their X chromosome has a DMD gene mutation, they
will have Duchenne muscular dystrophy.
- An automatic transmission will downshift for you when you drive uphill. However,
for moderately steep slopes, it's wise to shift to the gear range marked D2, 2,
or L to ascend and descend the hill. For steep slopes that you can't ascend at
a speed faster than 10 mph (about 15 kph), shift to D1 or 1.
- The dream suggests captivity and it refers to your fear of punishment. Another
interpretation of this dream refers to a need to do what you feel is right in
waking life. Being in jail suggests that your feelings may be trapped by a limited
mind and body. ... Jail also suggests repressed feelings.
datasets:
- sentence-transformers/gooaq
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
- cosine_accuracy@1
- cosine_accuracy@3
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@3
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@3
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@10
- cosine_mrr@10
- cosine_map@100
co2_eq_emissions:
emissions: 23.455178658284066
energy_consumed: 0.08764293913909942
source: codecarbon
training_type: fine-tuning
on_cloud: false
cpu_model: 13th Gen Intel(R) Core(TM) i7-13700K
ram_total_size: 31.777088165283203
hours_used: 0.24
hardware_used: 1 x NVIDIA GeForce RTX 3090
model-index:
- name: MPNet base trained on GooAQ triplets using CachedMultipleNegativesBidirectionalRankingLoss
(refactored)
results:
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: gooaq dev
type: gooaq-dev
metrics:
- type: cosine_accuracy@1
value: 0.759
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.8924
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.931
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.9636
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.759
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.2974666666666666
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.18620000000000003
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.09636000000000002
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.759
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.8924
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.931
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.9636
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.8653161814709974
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.8333427777777741
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.8350324253726423
name: Cosine Map@100
---
# MPNet base trained on GooAQ triplets using CachedMultipleNegativesBidirectionalRankingLoss (refactored)
This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [microsoft/mpnet-base](https://huggingface.co/microsoft/mpnet-base) on the [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
## Model Details
### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [microsoft/mpnet-base](https://huggingface.co/microsoft/mpnet-base)
- **Maximum Sequence Length:** 512 tokens
- **Output Dimensionality:** 768 dimensions
- **Similarity Function:** Cosine Similarity
- **Training Dataset:**
- [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq)
- **Language:** en
- **License:** apache-2.0
### Model Sources
- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/huggingface/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
### Full Model Architecture
```
SentenceTransformer(
(0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'MPNetModel'})
(1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)
```
## Usage
### Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
```bash
pip install -U sentence-transformers
```
Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("tomaarsen/mpnet-base-gooaq-cmnbrl-1024bs")
# Run inference
queries = [
"is duchenne muscular dystrophy a dominant or recessive trait?",
]
documents = [
'Duchenne muscular dystrophy is inherited in an X-linked recessive pattern. Males have only one copy of the X chromosome from their mother and one copy of the Y chromosome from their father. If their X chromosome has a DMD gene mutation, they will have Duchenne muscular dystrophy.',
'The dream suggests captivity and it refers to your fear of punishment. Another interpretation of this dream refers to a need to do what you feel is right in waking life. Being in jail suggests that your feelings may be trapped by a limited mind and body. ... Jail also suggests repressed feelings.',
"An automatic transmission will downshift for you when you drive uphill. However, for moderately steep slopes, it's wise to shift to the gear range marked D2, 2, or L to ascend and descend the hill. For steep slopes that you can't ascend at a speed faster than 10 mph (about 15 kph), shift to D1 or 1.",
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 768] [3, 768]
# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.9487, 0.7892, 0.7462]])
```
## Evaluation
### Metrics
#### Information Retrieval
* Dataset: `gooaq-dev`
* Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
| Metric | Value |
|:--------------------|:-----------|
| cosine_accuracy@1 | 0.759 |
| cosine_accuracy@3 | 0.8924 |
| cosine_accuracy@5 | 0.931 |
| cosine_accuracy@10 | 0.9636 |
| cosine_precision@1 | 0.759 |
| cosine_precision@3 | 0.2975 |
| cosine_precision@5 | 0.1862 |
| cosine_precision@10 | 0.0964 |
| cosine_recall@1 | 0.759 |
| cosine_recall@3 | 0.8924 |
| cosine_recall@5 | 0.931 |
| cosine_recall@10 | 0.9636 |
| **cosine_ndcg@10** | **0.8653** |
| cosine_mrr@10 | 0.8333 |
| cosine_map@100 | 0.835 |
## Training Details
### Training Dataset
#### gooaq
* Dataset: [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) at [b089f72](https://huggingface.co/datasets/sentence-transformers/gooaq/tree/b089f728748a068b7bc5234e5bcf5b25e3c8279c)
* Size: 90,000 training samples
* Columns: question and answer
* Approximate statistics based on the first 1000 samples:
| | question | answer |
|:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
| type | string | string |
| details | - min: 8 tokens
- mean: 11.83 tokens
- max: 20 tokens
| - min: 15 tokens
- mean: 60.45 tokens
- max: 180 tokens
|
* Samples:
| question | answer |
|:--------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| how long does halifax take to transfer mortgage funds? | Bear in mind that the speed of application will vary depending on your own personal circumstances and the lender's present day-to-day performance. In some cases, applications can be approved by the lender within 24 hours, while some can take weeks or even months. |
| can you get a false pregnancy test? | In very rare cases, you can have a false-positive result. This means you're not pregnant but the test says you are. You could have a false-positive result if you have blood or protein in your pee. Certain drugs, such as tranquilizers, anticonvulsants, hypnotics, and fertility drugs, could cause false-positive results. |
| are ahead of its time? | Definition of ahead of one's/its time : too advanced or modern to be understood or appreciated during the time when one lives or works As a director, he was ahead of his time. |
* Loss: [CachedMultipleNegativesBidirectionalRankingLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cachedmultiplenegativesbidirectionalrankingloss) with these parameters:
```json
{
"temperature": 0.01,
"similarity_fct": "cos_sim",
"mini_batch_size": 64,
"gather_across_devices": false
}
```
### Evaluation Dataset
#### gooaq
* Dataset: [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) at [b089f72](https://huggingface.co/datasets/sentence-transformers/gooaq/tree/b089f728748a068b7bc5234e5bcf5b25e3c8279c)
* Size: 10,000 evaluation samples
* Columns: question and answer
* Approximate statistics based on the first 1000 samples:
| | question | answer |
|:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
| type | string | string |
| details | - min: 8 tokens
- mean: 11.93 tokens
- max: 25 tokens
| - min: 14 tokens
- mean: 60.84 tokens
- max: 127 tokens
|
* Samples:
| question | answer |
|:-----------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| should you take ibuprofen with high blood pressure? | In general, people with high blood pressure should use acetaminophen or possibly aspirin for over-the-counter pain relief. Unless your health care provider has said it's OK, you should not use ibuprofen, ketoprofen, or naproxen sodium. If aspirin or acetaminophen doesn't help with your pain, call your doctor. |
| how old do you have to be to work in sc? | The general minimum age of employment for South Carolina youth is 14, although the state allows younger children who are performers to work in show business. If their families are agricultural workers, children younger than age 14 may also participate in farm labor. |
| how to write a topic proposal for a research paper? | ['Write down the main topic of your paper. ... ', 'Write two or three short sentences under the main topic that explain why you chose that topic. ... ', 'Write a thesis sentence that states the angle and purpose of your research paper. ... ', 'List the items you will cover in the body of the paper that support your thesis statement.'] |
* Loss: [CachedMultipleNegativesBidirectionalRankingLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cachedmultiplenegativesbidirectionalrankingloss) with these parameters:
```json
{
"temperature": 0.01,
"similarity_fct": "cos_sim",
"mini_batch_size": 64,
"gather_across_devices": false
}
```
### Training Hyperparameters
#### Non-Default Hyperparameters
- `eval_strategy`: steps
- `per_device_train_batch_size`: 1024
- `per_device_eval_batch_size`: 1024
- `learning_rate`: 8e-05
- `num_train_epochs`: 1
- `warmup_ratio`: 0.1
- `bf16`: True
- `batch_sampler`: no_duplicates
#### All Hyperparameters
Click to expand
- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: steps
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 1024
- `per_device_eval_batch_size`: 1024
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 1
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 8e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 1
- `max_steps`: -1
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.1
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `bf16`: True
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: False
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `parallelism_config`: None
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch_fused
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `project`: huggingface
- `trackio_space_id`: trackio
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: None
- `hub_always_push`: False
- `hub_revision`: None
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `include_for_metrics`: []
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`:
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: no
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `use_liger_kernel`: False
- `liger_kernel_config`: None
- `eval_use_gather_object`: False
- `average_tokens_across_devices`: True
- `prompts`: None
- `batch_sampler`: no_duplicates
- `multi_dataset_batch_sampler`: proportional
- `router_mapping`: {}
- `learning_rate_mapping`: {}