--- tags: - sentence-transformers - sentence-similarity - feature-extraction - dense - generated_from_trainer - dataset_size:230357 - loss:MultipleNegativesRankingLoss base_model: Snowflake/snowflake-arctic-embed-l widget: - source_sentence: post-discharge oral hygiene follow-up plan sentences: - The patient is advised to continue regular oral hygiene measures and follow-up with their primary care dentist. - 'PLAN: Follow-up with the patient as clinically indicated.' - The patient's last available report revealed significant improvement in kidney function. No further information is currently available. The patient is advised to follow up with his primary care physician for routine care. - source_sentence: Genetic mutations associated with PKD sentences: - We identified an insertion of C at 650 bp of coding sequence (exon 2) leading to frameshift mutation [p.R217Pfs*8] in the PKD patient, but not in the control subject. In addition, we also identified a novel point mutation [c.244C > T;] at exon 2 in the patient, but not in the control subject. - Following the National Comprehensive Cancer Network guidelines, the patient will be monitored with careful history and physical, blood pressure measurement, and urine and plasma catecholamine assessment every three months through the first year and every 6–12 months thereafter through 10 years. - 'Genetic analysis showed two variants in NR2E3: a missense variation c.1118T>C, which leads to substitution of leucine with proline in amino acid position 373, and c.349+5G>C, which involves a gene sequence near a splicing site.' - source_sentence: What was the patient's diagnosis based on the MRI results? sentences: - Magnetic resonance imaging of the orbit showed hyperintensity and increased thickness of the preseptal orbital tissue, and optic nerve thickening in the right eye. - 'Probands #1 and #2 presented with significant and consistent delays in cognitive function, language and sensorimotor development.' - An MRI was performed, which revealed the presence of a declivous purulent material inside the lateral ventricles with a moderate dilatation of these structures and semi-recent ischemic strokes. - source_sentence: Post-operative complications of extended left hepatectomy sentences: - The patient was referred to the Oncology Department for further management and treatment. A multidisciplinary team approach would be used to develop an optimal treatment strategy. - However, on postoperative day 15, the patient was readmitted with symptoms of abdominal pain, fevers, elevated total bilirubin, and leukocytosis. A CT scan revealed the formation of a collection in the hepatic fossa and a dilated right posterior bile duct, which prompted the placement of a drain. - Post-operatively, the patient recovered well, and was discharged on the 7th post-operative day without any complications. Follow-up assessments did not report any issues, and the patient was advised to maintain ample rest and a nutritious diet with frequent walkthroughs to avoid stress on the abdomen. - source_sentence: moderate level of depression diagnosis sentences: - The right upper limb was examined and revealed a morphological variant of the coracobrachialis muscle (CBM) that corresponded to the classical description, originating from the apex of the coracoid process together with the short head of the biceps brachii. - The patient's symptoms are suggestive of depression. The Beck Depression Inventory score was 24, indicating a moderate level of depression. - The patient was admitted due to symptoms of Major Depressive Disorder (MDD) which included restlessness, sleep problems, depressed mood, decreased social function, and suicidal thoughts as reported by the patient. datasets: - abhinand/MedEmbed-training-triplets-v1 pipeline_tag: sentence-similarity library_name: sentence-transformers metrics: - cosine_accuracy model-index: - name: SentenceTransformer based on Snowflake/snowflake-arctic-embed-l results: - task: type: triplet name: Triplet dataset: name: medembed triplets dev 300 type: medembed-triplets-dev-300 metrics: - type: cosine_accuracy value: 0.9599999785423279 name: Cosine Accuracy --- # SentenceTransformer based on Snowflake/snowflake-arctic-embed-l This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [Snowflake/snowflake-arctic-embed-l](https://huggingface.co/Snowflake/snowflake-arctic-embed-l) on the [med_embed-training-triplets-v1](https://huggingface.co/datasets/abhinand/MedEmbed-training-triplets-v1) dataset. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. ## Model Details ### Model Description - **Model Type:** Sentence Transformer - **Base model:** [Snowflake/snowflake-arctic-embed-l](https://huggingface.co/Snowflake/snowflake-arctic-embed-l) - **Maximum Sequence Length:** 512 tokens - **Output Dimensionality:** 1024 dimensions - **Similarity Function:** Cosine Similarity - **Training Dataset:** - [med_embed-training-triplets-v1](https://huggingface.co/datasets/abhinand/MedEmbed-training-triplets-v1) ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) ### Full Model Architecture ``` SentenceTransformer( (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'BertModel'}) (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) (2): Normalize() ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the 🤗 Hub model = SentenceTransformer("lion-ai/arctic-l-medembed") # Run inference queries = [ "moderate level of depression diagnosis", ] documents = [ "The patient's symptoms are suggestive of depression. The Beck Depression Inventory score was 24, indicating a moderate level of depression.", 'The patient was admitted due to symptoms of Major Depressive Disorder (MDD) which included restlessness, sleep problems, depressed mood, decreased social function, and suicidal thoughts as reported by the patient.', 'The right upper limb was examined and revealed a morphological variant of the coracobrachialis muscle (CBM) that corresponded to the classical description, originating from the apex of the coracoid process together with the short head of the biceps brachii.', ] query_embeddings = model.encode_query(queries) document_embeddings = model.encode_document(documents) print(query_embeddings.shape, document_embeddings.shape) # [1, 1024] [3, 1024] # Get the similarity scores for the embeddings similarities = model.similarity(query_embeddings, document_embeddings) print(similarities) # tensor([[0.5307, 0.3000, 0.0426]]) ``` ## Evaluation ### Metrics #### Triplet * Dataset: `medembed-triplets-dev-300` * Evaluated with [TripletEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.TripletEvaluator) | Metric | Value | |:--------------------|:---------| | **cosine_accuracy** | **0.96** | ## Training Details ### Training Dataset #### med_embed-training-triplets-v1 * Dataset: [med_embed-training-triplets-v1](https://huggingface.co/datasets/abhinand/MedEmbed-training-triplets-v1) at [0b344f0](https://huggingface.co/datasets/abhinand/MedEmbed-training-triplets-v1/tree/0b344f0f578f1cf3435338dbc53a2bf3449d5c88) * Size: 230,357 training samples * Columns: anchor, positive, and negative * Approximate statistics based on the first 1000 samples: | | anchor | positive | negative | |:--------|:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------| | type | string | string | string | | details | | | | * Samples: | anchor | positive | negative | |:------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | MTHFR homozygous A223V mutation symptoms | The patient was admitted to our institution where MTHFR homozygous A223V mutation was identified. Folic acid intake was increased to 800 mcg/d, and no other coagulation tests were abnormal. | The patient had several symptoms of MM, including hypercalcemia, bone fractures, anemia, and renal insufficiency. A biopsy showed atypical clonal plasma cells with Cluster of Differentiation (CD)138 positive infiltration. | | Causes of spindle cell malignancy in the duodenal wall | Histological analysis revealed a spindle cell malignancy that was positive for CD21, CD23, and vimentin, but negative for CD20, CD34, CD35, CD117, DOG 1, and smooth muscle actin. | Based on immunohistochemical analysis of the tumor cells, the primary buttock tumor was diagnosed as a skeletal muscle metastasis of the primary small intestine gastrointestinal stromal tumor (GIST). | | What was the patient's main complaint during hospital admission? | This 27-year-old pregnant woman was admitted to the hospital at 36 weeks gestation with acute vision loss in her left eye and severe onset headache. | The patient was discharged the next day | * Loss: [MultipleNegativesRankingLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters: ```json { "scale": 20.0, "similarity_fct": "cos_sim", "gather_across_devices": false } ``` ### Evaluation Dataset #### med_embed-training-triplets-v1 * Dataset: [med_embed-training-triplets-v1](https://huggingface.co/datasets/abhinand/MedEmbed-training-triplets-v1) at [0b344f0](https://huggingface.co/datasets/abhinand/MedEmbed-training-triplets-v1/tree/0b344f0f578f1cf3435338dbc53a2bf3449d5c88) * Size: 300 evaluation samples * Columns: anchor, positive, and negative * Approximate statistics based on the first 300 samples: | | anchor | positive | negative | |:--------|:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------| | type | string | string | string | | details | | | | * Samples: | anchor | positive | negative | |:---------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | What was the initial presentation of the patient? | The 45-year-old female patient presented to the department with an enlarging lesion in her upper abdomen. | The patient was transferred to this hospital for further evaluation. | | giant omphalocele symptoms | The patient, a 9-year-old female, presented to the hospital with a large lump in the anterior abdominal wall extending from the xiphisternum to the level of iliac crest. | The patient presented with bilateral nasovestibular lumps which grew in size over several months, occluding nasal entrance and protruding outside the nose. | | granulomatous lymphocytic interstitial lung disease treatment | The patient had clubbing and chronic lung findings, and thorax CT revealed extended and severe bronchiectasis with thickened bronchial walls, some granulomatous nodules and mosaic appearance, compatible with granulomatous lymphocytic interstitial lung disease (GLILD). Regular intravenous immunoglobulin (IVIG) replacement was started. | The patient was treated with methylprednisolone pulse therapy followed by oral prednisolone (PSL) and cyclophosphamide intravenously. After treatment, arthralgia, renal function, proteinuria, and skin manifestations improved. | * Loss: [MultipleNegativesRankingLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters: ```json { "scale": 20.0, "similarity_fct": "cos_sim", "gather_across_devices": false } ``` ### Training Hyperparameters #### Non-Default Hyperparameters - `eval_strategy`: steps - `per_device_train_batch_size`: 16 - `per_device_eval_batch_size`: 16 - `gradient_accumulation_steps`: 8 - `learning_rate`: 1e-05 - `weight_decay`: 0.01 - `num_train_epochs`: 2 - `warmup_ratio`: 0.1 - `dataloader_num_workers`: 4 - `load_best_model_at_end`: True - `ddp_find_unused_parameters`: False - `batch_sampler`: no_duplicates #### All Hyperparameters
Click to expand - `overwrite_output_dir`: False - `do_predict`: False - `eval_strategy`: steps - `prediction_loss_only`: True - `per_device_train_batch_size`: 16 - `per_device_eval_batch_size`: 16 - `per_gpu_train_batch_size`: None - `per_gpu_eval_batch_size`: None - `gradient_accumulation_steps`: 8 - `eval_accumulation_steps`: None - `torch_empty_cache_steps`: None - `learning_rate`: 1e-05 - `weight_decay`: 0.01 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `max_grad_norm`: 1.0 - `num_train_epochs`: 2 - `max_steps`: -1 - `lr_scheduler_type`: linear - `lr_scheduler_kwargs`: {} - `warmup_ratio`: 0.1 - `warmup_steps`: 0 - `log_level`: passive - `log_level_replica`: warning - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `save_safetensors`: True - `save_on_each_node`: False - `save_only_model`: False - `restore_callback_states_from_checkpoint`: False - `no_cuda`: False - `use_cpu`: False - `use_mps_device`: False - `seed`: 42 - `data_seed`: None - `jit_mode_eval`: False - `bf16`: False - `fp16`: False - `fp16_opt_level`: O1 - `half_precision_backend`: auto - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: None - `local_rank`: 0 - `ddp_backend`: None - `tpu_num_cores`: None - `tpu_metrics_debug`: False - `debug`: [] - `dataloader_drop_last`: False - `dataloader_num_workers`: 4 - `dataloader_prefetch_factor`: None - `past_index`: -1 - `disable_tqdm`: False - `remove_unused_columns`: True - `label_names`: None - `load_best_model_at_end`: True - `ignore_data_skip`: False - `fsdp`: [] - `fsdp_min_num_params`: 0 - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} - `fsdp_transformer_layer_cls_to_wrap`: None - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `parallelism_config`: None - `deepspeed`: None - `label_smoothing_factor`: 0.0 - `optim`: adamw_torch_fused - `optim_args`: None - `adafactor`: False - `group_by_length`: False - `length_column_name`: length - `project`: huggingface - `trackio_space_id`: trackio - `ddp_find_unused_parameters`: False - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `skip_memory_metrics`: True - `use_legacy_prediction_loop`: False - `push_to_hub`: False - `resume_from_checkpoint`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_private_repo`: None - `hub_always_push`: False - `hub_revision`: None - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `include_inputs_for_metrics`: False - `include_for_metrics`: [] - `eval_do_concat_batches`: True - `fp16_backend`: auto - `push_to_hub_model_id`: None - `push_to_hub_organization`: None - `mp_parameters`: - `auto_find_batch_size`: False - `full_determinism`: False - `torchdynamo`: None - `ray_scope`: last - `ddp_timeout`: 1800 - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `include_tokens_per_second`: False - `include_num_input_tokens_seen`: no - `neftune_noise_alpha`: None - `optim_target_modules`: None - `batch_eval_metrics`: False - `eval_on_start`: False - `use_liger_kernel`: False - `liger_kernel_config`: None - `eval_use_gather_object`: False - `average_tokens_across_devices`: True - `prompts`: None - `batch_sampler`: no_duplicates - `multi_dataset_batch_sampler`: proportional - `router_mapping`: {} - `learning_rate_mapping`: {}
### Training Logs
Click to expand | Epoch | Step | Training Loss | Validation Loss | medembed-triplets-dev-300_cosine_accuracy | |:----------:|:--------:|:-------------:|:---------------:|:-----------------------------------------:| | -1 | -1 | - | - | 0.5767 | | 0.0056 | 10 | 2.6703 | - | - | | 0.0111 | 20 | 2.5114 | - | - | | 0.0167 | 30 | 2.3725 | - | - | | 0.0222 | 40 | 2.2162 | - | - | | 0.0278 | 50 | 2.0325 | - | - | | 0.0333 | 60 | 1.8674 | - | - | | 0.0389 | 70 | 1.7283 | - | - | | 0.0445 | 80 | 1.5936 | - | - | | 0.0500 | 90 | 1.5153 | - | - | | 0.0556 | 100 | 1.421 | 1.3269 | 0.6367 | | 0.0611 | 110 | 1.3861 | - | - | | 0.0667 | 120 | 1.2706 | - | - | | 0.0722 | 130 | 1.286 | - | - | | 0.0778 | 140 | 1.2037 | - | - | | 0.0833 | 150 | 1.1301 | - | - | | 0.0889 | 160 | 1.1587 | - | - | | 0.0945 | 170 | 1.1741 | - | - | | 0.1000 | 180 | 1.0915 | - | - | | 0.1056 | 190 | 1.0786 | - | - | | 0.1111 | 200 | 1.0518 | 1.0083 | 0.7767 | | 0.1167 | 210 | 1.0562 | - | - | | 0.1222 | 220 | 1.042 | - | - | | 0.1278 | 230 | 0.9928 | - | - | | 0.1334 | 240 | 1.022 | - | - | | 0.1389 | 250 | 1.044 | - | - | | 0.1445 | 260 | 0.9621 | - | - | | 0.1500 | 270 | 0.98 | - | - | | 0.1556 | 280 | 1.0073 | - | - | | 0.1611 | 290 | 0.9346 | - | - | | 0.1667 | 300 | 0.995 | 0.8794 | 0.8100 | | 0.1722 | 310 | 0.9057 | - | - | | 0.1778 | 320 | 1.0023 | - | - | | 0.1834 | 330 | 0.8486 | - | - | | 0.1889 | 340 | 0.8888 | - | - | | 0.1945 | 350 | 0.9358 | - | - | | 0.2000 | 360 | 0.8503 | - | - | | 0.2056 | 370 | 0.9181 | - | - | | 0.2111 | 380 | 0.8965 | - | - | | 0.2167 | 390 | 0.8807 | - | - | | 0.2223 | 400 | 0.8833 | 0.8012 | 0.8267 | | 0.2278 | 410 | 0.8564 | - | - | | 0.2334 | 420 | 0.838 | - | - | | 0.2389 | 430 | 0.8775 | - | - | | 0.2445 | 440 | 0.7808 | - | - | | 0.2500 | 450 | 0.7795 | - | - | | 0.2556 | 460 | 0.8087 | - | - | | 0.2611 | 470 | 0.7418 | - | - | | 0.2667 | 480 | 0.7921 | - | - | | 0.2723 | 490 | 0.7526 | - | - | | 0.2778 | 500 | 0.6838 | 0.7306 | 0.8567 | | 0.2834 | 510 | 0.7908 | - | - | | 0.2889 | 520 | 0.8013 | - | - | | 0.2945 | 530 | 0.7176 | - | - | | 0.3000 | 540 | 0.7432 | - | - | | 0.3056 | 550 | 0.7148 | - | - | | 0.3112 | 560 | 0.7034 | - | - | | 0.3167 | 570 | 0.7405 | - | - | | 0.3223 | 580 | 0.7185 | - | - | | 0.3278 | 590 | 0.6609 | - | - | | 0.3334 | 600 | 0.6246 | 0.6693 | 0.8767 | | 0.3389 | 610 | 0.6743 | - | - | | 0.3445 | 620 | 0.6826 | - | - | | 0.3500 | 630 | 0.7143 | - | - | | 0.3556 | 640 | 0.706 | - | - | | 0.3612 | 650 | 0.6787 | - | - | | 0.3667 | 660 | 0.6784 | - | - | | 0.3723 | 670 | 0.6938 | - | - | | 0.3778 | 680 | 0.6647 | - | - | | 0.3834 | 690 | 0.6732 | - | - | | 0.3889 | 700 | 0.6409 | 0.6291 | 0.9033 | | 0.3945 | 710 | 0.6773 | - | - | | 0.4001 | 720 | 0.6767 | - | - | | 0.4056 | 730 | 0.6656 | - | - | | 0.4112 | 740 | 0.6423 | - | - | | 0.4167 | 750 | 0.6341 | - | - | | 0.4223 | 760 | 0.6369 | - | - | | 0.4278 | 770 | 0.5883 | - | - | | 0.4334 | 780 | 0.6497 | - | - | | 0.4389 | 790 | 0.6106 | - | - | | 0.4445 | 800 | 0.6423 | 0.5550 | 0.9067 | | 0.4501 | 810 | 0.6054 | - | - | | 0.4556 | 820 | 0.6238 | - | - | | 0.4612 | 830 | 0.5986 | - | - | | 0.4667 | 840 | 0.6196 | - | - | | 0.4723 | 850 | 0.5767 | - | - | | 0.4778 | 860 | 0.6286 | - | - | | 0.4834 | 870 | 0.587 | - | - | | 0.4890 | 880 | 0.5661 | - | - | | 0.4945 | 890 | 0.51 | - | - | | 0.5001 | 900 | 0.6048 | 0.5459 | 0.9167 | | 0.5056 | 910 | 0.5621 | - | - | | 0.5112 | 920 | 0.5954 | - | - | | 0.5167 | 930 | 0.5542 | - | - | | 0.5223 | 940 | 0.5846 | - | - | | 0.5279 | 950 | 0.5864 | - | - | | 0.5334 | 960 | 0.6152 | - | - | | 0.5390 | 970 | 0.5696 | - | - | | 0.5445 | 980 | 0.5639 | - | - | | 0.5501 | 990 | 0.555 | - | - | | 0.5556 | 1000 | 0.5465 | 0.4860 | 0.9200 | | 0.5612 | 1010 | 0.5618 | - | - | | 0.5667 | 1020 | 0.5228 | - | - | | 0.5723 | 1030 | 0.5117 | - | - | | 0.5779 | 1040 | 0.5305 | - | - | | 0.5834 | 1050 | 0.5048 | - | - | | 0.5890 | 1060 | 0.5261 | - | - | | 0.5945 | 1070 | 0.4963 | - | - | | 0.6001 | 1080 | 0.5034 | - | - | | 0.6056 | 1090 | 0.4615 | - | - | | 0.6112 | 1100 | 0.5192 | 0.4642 | 0.9267 | | 0.6168 | 1110 | 0.5226 | - | - | | 0.6223 | 1120 | 0.5417 | - | - | | 0.6279 | 1130 | 0.5214 | - | - | | 0.6334 | 1140 | 0.5256 | - | - | | 0.6390 | 1150 | 0.5327 | - | - | | 0.6445 | 1160 | 0.5413 | - | - | | 0.6501 | 1170 | 0.5261 | - | - | | 0.6556 | 1180 | 0.4876 | - | - | | 0.6612 | 1190 | 0.4694 | - | - | | 0.6668 | 1200 | 0.4795 | 0.4229 | 0.9500 | | 0.6723 | 1210 | 0.4896 | - | - | | 0.6779 | 1220 | 0.4796 | - | - | | 0.6834 | 1230 | 0.4732 | - | - | | 0.6890 | 1240 | 0.486 | - | - | | 0.6945 | 1250 | 0.4942 | - | - | | 0.7001 | 1260 | 0.4403 | - | - | | 0.7057 | 1270 | 0.4581 | - | - | | 0.7112 | 1280 | 0.4494 | - | - | | 0.7168 | 1290 | 0.4676 | - | - | | 0.7223 | 1300 | 0.4575 | 0.4201 | 0.9500 | | 0.7279 | 1310 | 0.4395 | - | - | | 0.7334 | 1320 | 0.4197 | - | - | | 0.7390 | 1330 | 0.4471 | - | - | | 0.7445 | 1340 | 0.4724 | - | - | | 0.7501 | 1350 | 0.43 | - | - | | 0.7557 | 1360 | 0.4691 | - | - | | 0.7612 | 1370 | 0.425 | - | - | | 0.7668 | 1380 | 0.4244 | - | - | | 0.7723 | 1390 | 0.417 | - | - | | 0.7779 | 1400 | 0.4398 | 0.3899 | 0.9467 | | 0.7834 | 1410 | 0.452 | - | - | | 0.7890 | 1420 | 0.476 | - | - | | 0.7946 | 1430 | 0.4433 | - | - | | 0.8001 | 1440 | 0.4021 | - | - | | 0.8057 | 1450 | 0.461 | - | - | | 0.8112 | 1460 | 0.4191 | - | - | | 0.8168 | 1470 | 0.4068 | - | - | | 0.8223 | 1480 | 0.4274 | - | - | | 0.8279 | 1490 | 0.4238 | - | - | | 0.8334 | 1500 | 0.4403 | 0.3585 | 0.9433 | | 0.8390 | 1510 | 0.3875 | - | - | | 0.8446 | 1520 | 0.4004 | - | - | | 0.8501 | 1530 | 0.4194 | - | - | | 0.8557 | 1540 | 0.3883 | - | - | | 0.8612 | 1550 | 0.3861 | - | - | | 0.8668 | 1560 | 0.4131 | - | - | | 0.8723 | 1570 | 0.4006 | - | - | | 0.8779 | 1580 | 0.4195 | - | - | | 0.8835 | 1590 | 0.4144 | - | - | | 0.8890 | 1600 | 0.4454 | 0.3131 | 0.9533 | | 0.8946 | 1610 | 0.3751 | - | - | | 0.9001 | 1620 | 0.3655 | - | - | | 0.9057 | 1630 | 0.3875 | - | - | | 0.9112 | 1640 | 0.3922 | - | - | | 0.9168 | 1650 | 0.4043 | - | - | | 0.9224 | 1660 | 0.3686 | - | - | | 0.9279 | 1670 | 0.3745 | - | - | | 0.9335 | 1680 | 0.3681 | - | - | | 0.9390 | 1690 | 0.3858 | - | - | | 0.9446 | 1700 | 0.3817 | 0.3126 | 0.9533 | | 0.9501 | 1710 | 0.3787 | - | - | | 0.9557 | 1720 | 0.3849 | - | - | | 0.9612 | 1730 | 0.352 | - | - | | 0.9668 | 1740 | 0.3549 | - | - | | 0.9724 | 1750 | 0.3634 | - | - | | 0.9779 | 1760 | 0.3548 | - | - | | 0.9835 | 1770 | 0.3662 | - | - | | 0.9890 | 1780 | 0.3915 | - | - | | 0.9946 | 1790 | 0.3713 | - | - | | 1.0 | 1800 | 0.3647 | 0.3077 | 0.9533 | | 1.0056 | 1810 | 0.3365 | - | - | | 1.0111 | 1820 | 0.3084 | - | - | | 1.0167 | 1830 | 0.3261 | - | - | | 1.0222 | 1840 | 0.3036 | - | - | | 1.0278 | 1850 | 0.3054 | - | - | | 1.0333 | 1860 | 0.3384 | - | - | | 1.0389 | 1870 | 0.3129 | - | - | | 1.0445 | 1880 | 0.3092 | - | - | | 1.0500 | 1890 | 0.3043 | - | - | | 1.0556 | 1900 | 0.2904 | 0.3005 | 0.9533 | | 1.0611 | 1910 | 0.2917 | - | - | | 1.0667 | 1920 | 0.3196 | - | - | | 1.0722 | 1930 | 0.3401 | - | - | | 1.0778 | 1940 | 0.312 | - | - | | 1.0833 | 1950 | 0.321 | - | - | | 1.0889 | 1960 | 0.3052 | - | - | | 1.0945 | 1970 | 0.3136 | - | - | | 1.1000 | 1980 | 0.3243 | - | - | | 1.1056 | 1990 | 0.2843 | - | - | | 1.1111 | 2000 | 0.2994 | 0.2905 | 0.9533 | | 1.1167 | 2010 | 0.3144 | - | - | | 1.1222 | 2020 | 0.3029 | - | - | | 1.1278 | 2030 | 0.3085 | - | - | | 1.1334 | 2040 | 0.3053 | - | - | | 1.1389 | 2050 | 0.3009 | - | - | | 1.1445 | 2060 | 0.3083 | - | - | | 1.1500 | 2070 | 0.2993 | - | - | | 1.1556 | 2080 | 0.2932 | - | - | | 1.1611 | 2090 | 0.2994 | - | - | | 1.1667 | 2100 | 0.2796 | 0.2704 | 0.9467 | | 1.1722 | 2110 | 0.3033 | - | - | | 1.1778 | 2120 | 0.3083 | - | - | | 1.1834 | 2130 | 0.3118 | - | - | | 1.1889 | 2140 | 0.2978 | - | - | | 1.1945 | 2150 | 0.2697 | - | - | | 1.2000 | 2160 | 0.2918 | - | - | | 1.2056 | 2170 | 0.2814 | - | - | | 1.2111 | 2180 | 0.2615 | - | - | | 1.2167 | 2190 | 0.289 | - | - | | 1.2223 | 2200 | 0.2819 | 0.2454 | 0.9533 | | 1.2278 | 2210 | 0.2751 | - | - | | 1.2334 | 2220 | 0.3228 | - | - | | 1.2389 | 2230 | 0.2852 | - | - | | 1.2445 | 2240 | 0.2909 | - | - | | 1.2500 | 2250 | 0.285 | - | - | | 1.2556 | 2260 | 0.2936 | - | - | | 1.2611 | 2270 | 0.3052 | - | - | | 1.2667 | 2280 | 0.2874 | - | - | | 1.2723 | 2290 | 0.3008 | - | - | | 1.2778 | 2300 | 0.3161 | 0.2529 | 0.9567 | | 1.2834 | 2310 | 0.2764 | - | - | | 1.2889 | 2320 | 0.2779 | - | - | | 1.2945 | 2330 | 0.2547 | - | - | | 1.3000 | 2340 | 0.2645 | - | - | | 1.3056 | 2350 | 0.3015 | - | - | | 1.3112 | 2360 | 0.2835 | - | - | | 1.3167 | 2370 | 0.279 | - | - | | 1.3223 | 2380 | 0.3158 | - | - | | 1.3278 | 2390 | 0.2933 | - | - | | 1.3334 | 2400 | 0.2791 | 0.2412 | 0.9567 | | 1.3389 | 2410 | 0.2783 | - | - | | 1.3445 | 2420 | 0.2533 | - | - | | 1.3500 | 2430 | 0.2943 | - | - | | 1.3556 | 2440 | 0.2808 | - | - | | 1.3612 | 2450 | 0.2828 | - | - | | 1.3667 | 2460 | 0.2789 | - | - | | 1.3723 | 2470 | 0.2814 | - | - | | 1.3778 | 2480 | 0.2675 | - | - | | 1.3834 | 2490 | 0.2746 | - | - | | 1.3889 | 2500 | 0.2907 | 0.2419 | 0.9567 | | 1.3945 | 2510 | 0.278 | - | - | | 1.4001 | 2520 | 0.2594 | - | - | | 1.4056 | 2530 | 0.2659 | - | - | | 1.4112 | 2540 | 0.2859 | - | - | | 1.4167 | 2550 | 0.2755 | - | - | | 1.4223 | 2560 | 0.2834 | - | - | | 1.4278 | 2570 | 0.26 | - | - | | 1.4334 | 2580 | 0.2693 | - | - | | 1.4389 | 2590 | 0.2678 | - | - | | **1.4445** | **2600** | **0.2624** | **0.2236** | **0.96** | | 1.4501 | 2610 | 0.257 | - | - | | 1.4556 | 2620 | 0.2876 | - | - | | 1.4612 | 2630 | 0.2263 | - | - | | 1.4667 | 2640 | 0.2508 | - | - | | 1.4723 | 2650 | 0.2446 | - | - | | 1.4778 | 2660 | 0.2532 | - | - | | 1.4834 | 2670 | 0.2383 | - | - | | 1.4890 | 2680 | 0.2441 | - | - | | 1.4945 | 2690 | 0.241 | - | - | | 1.5001 | 2700 | 0.2594 | 0.2242 | 0.9567 | | 1.5056 | 2710 | 0.2463 | - | - | | 1.5112 | 2720 | 0.2797 | - | - | | 1.5167 | 2730 | 0.2419 | - | - | | 1.5223 | 2740 | 0.2933 | - | - | | 1.5279 | 2750 | 0.2724 | - | - | | 1.5334 | 2760 | 0.2434 | - | - | | 1.5390 | 2770 | 0.2635 | - | - | | 1.5445 | 2780 | 0.2317 | - | - | | 1.5501 | 2790 | 0.2698 | - | - | | 1.5556 | 2800 | 0.2645 | 0.2309 | 0.9567 | | 1.5612 | 2810 | 0.2585 | - | - | | 1.5667 | 2820 | 0.2456 | - | - | | 1.5723 | 2830 | 0.2211 | - | - | | 1.5779 | 2840 | 0.2547 | - | - | | 1.5834 | 2850 | 0.2673 | - | - | | 1.5890 | 2860 | 0.2569 | - | - | | 1.5945 | 2870 | 0.2477 | - | - | | 1.6001 | 2880 | 0.2277 | - | - | | 1.6056 | 2890 | 0.223 | - | - | | 1.6112 | 2900 | 0.2415 | 0.2254 | 0.9600 | | 1.6168 | 2910 | 0.2412 | - | - | | 1.6223 | 2920 | 0.2564 | - | - | | 1.6279 | 2930 | 0.2735 | - | - | | 1.6334 | 2940 | 0.2589 | - | - | | 1.6390 | 2950 | 0.2678 | - | - | | 1.6445 | 2960 | 0.2549 | - | - | | 1.6501 | 2970 | 0.227 | - | - | | 1.6556 | 2980 | 0.2598 | - | - | | 1.6612 | 2990 | 0.2593 | - | - | | 1.6668 | 3000 | 0.2248 | 0.2247 | 0.9533 | | 1.6723 | 3010 | 0.2432 | - | - | | 1.6779 | 3020 | 0.2469 | - | - | | 1.6834 | 3030 | 0.2394 | - | - | | 1.6890 | 3040 | 0.2525 | - | - | | 1.6945 | 3050 | 0.2436 | - | - | | 1.7001 | 3060 | 0.2247 | - | - | | 1.7057 | 3070 | 0.2564 | - | - | | 1.7112 | 3080 | 0.2267 | - | - | | 1.7168 | 3090 | 0.2667 | - | - | | 1.7223 | 3100 | 0.2365 | 0.2198 | 0.9600 | | 1.7279 | 3110 | 0.2354 | - | - | | 1.7334 | 3120 | 0.2396 | - | - | | 1.7390 | 3130 | 0.2357 | - | - | | 1.7445 | 3140 | 0.2397 | - | - | | 1.7501 | 3150 | 0.2371 | - | - | | 1.7557 | 3160 | 0.2508 | - | - | | 1.7612 | 3170 | 0.2247 | - | - | | 1.7668 | 3180 | 0.2561 | - | - | | 1.7723 | 3190 | 0.2309 | - | - | | 1.7779 | 3200 | 0.2275 | 0.2126 | 0.9567 | | 1.7834 | 3210 | 0.2234 | - | - | | 1.7890 | 3220 | 0.2332 | - | - | | 1.7946 | 3230 | 0.2611 | - | - | | 1.8001 | 3240 | 0.2227 | - | - | | 1.8057 | 3250 | 0.2582 | - | - | | 1.8112 | 3260 | 0.2574 | - | - | | 1.8168 | 3270 | 0.2144 | - | - | | 1.8223 | 3280 | 0.2353 | - | - | | 1.8279 | 3290 | 0.241 | - | - | | 1.8334 | 3300 | 0.2406 | 0.2087 | 0.9567 | | 1.8390 | 3310 | 0.2536 | - | - | | 1.8446 | 3320 | 0.2579 | - | - | | 1.8501 | 3330 | 0.2273 | - | - | | 1.8557 | 3340 | 0.2662 | - | - | | 1.8612 | 3350 | 0.2356 | - | - | | 1.8668 | 3360 | 0.2177 | - | - | | 1.8723 | 3370 | 0.2268 | - | - | | 1.8779 | 3380 | 0.2337 | - | - | | 1.8835 | 3390 | 0.2686 | - | - | | 1.8890 | 3400 | 0.2239 | 0.2087 | 0.9567 | | 1.8946 | 3410 | 0.2559 | - | - | | 1.9001 | 3420 | 0.2135 | - | - | | 1.9057 | 3430 | 0.2456 | - | - | | 1.9112 | 3440 | 0.2461 | - | - | | 1.9168 | 3450 | 0.2349 | - | - | | 1.9224 | 3460 | 0.2227 | - | - | | 1.9279 | 3470 | 0.2443 | - | - | | 1.9335 | 3480 | 0.2425 | - | - | | 1.9390 | 3490 | 0.2427 | - | - | | 1.9446 | 3500 | 0.2469 | 0.2081 | 0.9600 | | 1.9501 | 3510 | 0.2504 | - | - | | 1.9557 | 3520 | 0.2518 | - | - | | 1.9612 | 3530 | 0.2241 | - | - | | 1.9668 | 3540 | 0.241 | - | - | | 1.9724 | 3550 | 0.2204 | - | - | | 1.9779 | 3560 | 0.2558 | - | - | | 1.9835 | 3570 | 0.2109 | - | - | | 1.9890 | 3580 | 0.2067 | - | - | | 1.9946 | 3590 | 0.2556 | - | - | | 2.0 | 3600 | 0.2603 | 0.2081 | 0.9600 | * The bold row denotes the saved checkpoint.
### Framework Versions - Python: 3.12.7 - Sentence Transformers: 5.1.1 - Transformers: 4.57.1 - PyTorch: 2.9.0+cu128 - Accelerate: 1.10.1 - Datasets: 3.6.0 - Tokenizers: 0.22.1 ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ``` #### MultipleNegativesRankingLoss ```bibtex @misc{henderson2017efficient, title={Efficient Natural Language Response Suggestion for Smart Reply}, author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil}, year={2017}, eprint={1705.00652}, archivePrefix={arXiv}, primaryClass={cs.CL} } ```