OmmWankhedee369/bge-base-medical-v1

bge-base medical fine-tune (MNRL)

This is a sentence-transformers model finetuned from BAAI/bge-base-en-v1.5. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: BAAI/bge-base-en-v1.5
• Maximum Sequence Length: 192 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text
• Language: en
• License: apache-2.0

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
  (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'cls', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("OmmWankhedee369/bge-base-medical-v1")
# Run inference
sentences = [
    'Represent this sentence for searching relevant passages: Adoptive immunity is by:',
    "Injection of lymphocytes. Ans. is 'c' i.e., Injection of lymphocytes(Ref: Jawetz, 27th/e, p. 130)* An extract of immunologically competent T - lymphocytes, known as transfer factor can be used in the treatment of lepromatous leprosy when CMI is low.",
    'Nonalcoholic fatty liver disease (NAFLD) is likely to replace Hepatitis C as the leading cause of cirrhosis resulting in liver transplantation (LT) within a few years. Unfortunately, due to the lack of established guidelines for the screening of NAFLD in high-risk populations, many patients present with portal hypertension complications as their first manifestation of NAFLD require a LT evaluation. We aimed to investigate what proportion of patients who underwent LT for NAFLD-cirrhosis had knowledge of their liver disease prior to presenting with portal hypertension complications and to identify differences in clinical parameters between those with and without knowledge of preexisting NAFLD. Consecutive patients who underwent LT for NAFLD-cirrhosis at a tertiary referral center were included in the study. Demographic and clinical data at the time of the first LT evaluation visit were collected, and patient knowledge of previous NAFLD was documented. Ascites, variceal bleeding, hepatic encephalopathy, and thrombocytopenia leading to diagnosis of underlying cirrhosis were considered as the presenting symptoms of portal hypertension. A p < 0.05 was considered statistically significant. A total of 124 subjects who received LT for NAFLD-cirrhosis were included, 58 % (n = 72) were male. At the time of the first LT evaluation visit, 60 % had diabetes, the mean body mass index was 33.2 [28.6, 37.6] kg/m(2), and the mean Model for End-Stage Liver Disease (MELD) score was 14.0 [11.0, 19.0]. More importantly, 85/124 patients (68.5 %) had no knowledge of preexisting NAFLD prior to presentation with symptoms of portal hypertension. The presenting symptoms were new-onset ascites in 61 %, hepatic encephalopathy in 25 %, variceal bleeding in 18 %, thrombocytopenia in 9 %, and other in 9 % (non-exclusive). Patients with no prior knowledge of NAFLD were less likely to have a diagnosis of hypercholesterolemia (30 vs. 50 %, p = 0.035) and had a trend toward having higher MELD scores at the time of the first LT evaluation visit (15 vs. 13.5, p = 0.05) and presenting with encephalopathy (25 vs. 10 %, p = 0.06) compared to those with previous knowledge of NAFLD diagnosis.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000,  0.2966, -0.1118],
#         [ 0.2966,  1.0000, -0.0079],
#         [-0.1118, -0.0079,  1.0000]])

Training Details

Training Dataset

Unnamed Dataset

• Size: 374,340 training samples
• Columns: anchor and positive
• Approximate statistics based on the first 100 samples:

  	anchor	positive
  type	string	string
  modality	text	text
  details	min: 15 tokens mean: 31.23 tokens max: 63 tokens	min: 8 tokens mean: 159.06 tokens max: 192 tokens

• Samples:

  anchor	positive
  Represent this sentence for searching relevant passages: Is the Thr92Ala 5 ' type 2 deiodinase gene polymorphism associated with a delayed triiodothyronine secretion in response to the thyrotropin-releasing hormone-stimulation test : a pharmacogenomic study?	The common Thr92Ala D2 polymorphism has been associated with changes in pituitary-thyroid axis homeostasis, but published results are conflicting. To investigate the effects of the Thr92Ala polymorphism on intrathyroidal thyroxine (T4) to triiodothyronine (T3) conversion, we designed prospective pharmacogenomic intervention aimed to detect differences in T3 levels after thyrotropin (TSH)-releasing hormone (TRH)-mediated TSH stimulation of the thyroid gland. Eighty-three healthy volunteers were screened and genotyped for the Thr92Ala polymorphism. Fifteen volunteers of each genotype (Thr/Thr, Thr/Ala, and Ala/Ala) underwent a 500 mcg intravenous TRH stimulation test with serial measurements of serum total T3 (TT3), free T4, and TSH over 180 minutes. No differences in baseline thyroid hormone levels were seen among the study groups. Compared to the Thr/Thr group, the Ala/Ala group showed a significantly lower TRH-stimulated increase in serum TT3 at 60 minutes (12.07 ± 2.67 vs. 21.07 ± 2....
  Represent this sentence for searching relevant passages: Do neuroticism and low educational level predict the risk of posttraumatic stress disorder in women after miscarriage or stillbirth?	This study aimed to determine whether neuroticism and educational level predict posttraumatic stress disorder (PTSD) in women following an unsuccessful pregnancy. Via advertisements, pregnant women with a gestational period shorter than 12 weeks were asked to participate in a study regarding their perception of pregnancy. After they had agreed, they were sent questionnaires, including a scale for neuroticism and their highest attained educational level. Every other month during the pregnancy and 1 month after the expected date of birth, they were sent brief questionnaires about the pregnancy. Participants for whom the pregnancy had ended unsuccessfully were contacted by phone and asked to participate in a follow-up study with a PTSD scale. Of the 1339 women studied, 126 (9%) experienced an unsuccessful pregnancy; 8 of these dropped out of the study (response rate, 94%); 1 had not indicated her educational level. The remaining 117 women filled out the PTSD scale after about 1 month. Thi...
  Represent this sentence for searching relevant passages: Does transfection of promyelocytic leukemia in retrovirus vector inhibit growth of human bladder cancer cells?	To construct a recombinant retrovirus vector carrying human promyelocytic leukemia (PML) cDNA and identify its expression and biology role in bladder cancer UM-UC-2 cells for future gene therapy. PML full-length cDNA was inserted into the EcoR I and BamH I site of pLXSN vector containing the long terminal repeat (LTR) promoter. The vector was identified by restriction enzyme digestion and then transfected into PA317 packaging cell line by calcium phosphate coprecipitation. PML cDNA was detected by polymerase chain reaction (PCR) and the protein was identified by laser confocal microscopy and Western blot in bladder cancer cells, respectively. The morphology was observed by inverted phase contrast microscope, and MTT assay determined growth curve of the bladder cancer cells. Restriction enzyme digestion proved that a 2.1 kb PML cDNA was inserted into the pLXSN vector. PCR assay demonstrated that 304 bp fragments were found in UM-UC-2/pLPMLSN transfects. Laser confocal microscopy showed ...

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false,
      "directions": [
          "query_to_doc"
      ],
      "partition_mode": "joint",
      "hardness_mode": null,
      "hardness_strength": 0.0
  }
  

Evaluation Dataset

Unnamed Dataset

• Size: 2,000 evaluation samples
• Columns: anchor and positive
• Approximate statistics based on the first 100 samples:

  	anchor	positive
  type	string	string
  modality	text	text
  details	min: 15 tokens mean: 31.72 tokens max: 111 tokens	min: 9 tokens mean: 148.59 tokens max: 192 tokens

• Samples:

  anchor	positive
  Represent this sentence for searching relevant passages: Are baseline comorbidities and treatment strategy in elderly patients associated with outcome of cardiogenic shock in a community-based population?	Cardiogenic shock (CGS) historically results in high inhospital mortality, particularly in elderly patients. Factors that contribute to increased mortality and treatment strategies that improve short- and long-term outcomes in patients with CGS remain to be established. The study consisted of 1263 consecutive patients with acute myocardial infarction admitted from Olmsted County, Minn, during the period 1988 to 2000; of these, 73 (6%) developed cardiogenic shock. Short- and long-term mortality was compared between the elderly and younger populations in both shock and nonshock groups. In patients with acute myocardial infarction, age of > or =65 years was associated with increased long-term mortality for nonshock patients (unadjusted relative risk [RR] 5.23, 95% CI 4.10-6.67, P <.001) and to a lesser degree in patients with cardiogenic shock (unadjusted RR 2.02, 95% CI 1.12-3.65, P =.02). Among cardiogenic shock patients, estimated survival at 1 and 5 years for elderly patients was 38% ...
  Represent this sentence for searching relevant passages: Irvin Moore sign is positive in	Chronic tonsillitis. Irwin-Moore sign:- Expression of cheesy material from the tonsil, on pressing anterior pillar seen in case of chronic tonsillitis. Ref:- Textbook of ENT, Hazarika; pg num:- 480
  Represent this sentence for searching relevant passages: Do three-dimensional echo-planar MR spectroscopic imaging at short echo times in the human brain?	To demonstrate the feasibility of three-dimensional echo-planar spectroscopic imaging (EPSI) at short echo time (13 msec) with a conventional clinical imager in the human brain. Periodic inversions of a readout gradient were used during data acquisition to simultaneously encode chemical shift and one spatial dimension in one excitation. Aliasing artifacts were avoided with a modified acquisition-and-processing method based on oversampling. A double outer-volume suppression technique that adapts to the ovoid brain shape was used to strongly reduce extracranial lipid resonances. Three-dimensional spatial encoding in vivo of eight sections with 32 x 32 voxels each (0.75 cm3) was performed in 34 minutes with four signal averages. The spectral resolution and signal-to-noise ratio (S/N) of resonances of inositol, choline, creatine, glutamate and glutamine, and N-acetyl aspartate were consistent with those previously recorded with conventional phase encoding.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false,
      "directions": [
          "query_to_doc"
      ],
      "partition_mode": "joint",
      "hardness_mode": null,
      "hardness_strength": 0.0
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 16
• num_train_epochs: 1
• learning_rate: 2e-05
• warmup_steps: 0.1
• fp16: True
• per_device_eval_batch_size: 16
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• per_device_train_batch_size: 16
• num_train_epochs: 1
• max_steps: -1
• learning_rate: 2e-05
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_steps: 0.1
• optim: adamw_torch_fused
• optim_args: None
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• optim_target_modules: None
• gradient_accumulation_steps: 1
• average_tokens_across_devices: True
• max_grad_norm: 1.0
• label_smoothing_factor: 0.0
• bf16: False
• fp16: True
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• use_liger_kernel: False
• liger_kernel_config: None
• use_cache: False
• neftune_noise_alpha: None
• torch_empty_cache_steps: None
• auto_find_batch_size: False
• log_on_each_node: True
• logging_nan_inf_filter: True
• include_num_input_tokens_seen: no
• log_level: passive
• log_level_replica: warning
• disable_tqdm: False
• project: huggingface
• trackio_space_id: None
• trackio_bucket_id: None
• trackio_static_space_id: None
• per_device_eval_batch_size: 16
• prediction_loss_only: True
• eval_on_start: False
• eval_do_concat_batches: True
• eval_use_gather_object: False
• eval_accumulation_steps: None
• include_for_metrics: []
• batch_eval_metrics: False
• save_only_model: False
• save_on_each_node: False
• enable_jit_checkpoint: False
• push_to_hub: False
• hub_private_repo: None
• hub_model_id: None
• hub_strategy: every_save
• hub_always_push: False
• hub_revision: None
• load_best_model_at_end: False
• ignore_data_skip: False
• restore_callback_states_from_checkpoint: False
• full_determinism: False
• seed: 42
• data_seed: None
• use_cpu: False
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• dataloader_drop_last: True
• dataloader_num_workers: 0
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• dataloader_prefetch_factor: None
• remove_unused_columns: True
• label_names: None
• train_sampling_strategy: random
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• ddp_static_graph: None
• ddp_backend: None
• ddp_timeout: 1800
• fsdp: None
• fsdp_config: None
• deepspeed: None
• debug: []
• skip_memory_metrics: True
• do_predict: False
• resume_from_checkpoint: None
• warmup_ratio: None
• local_rank: -1
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Click to expand

Epoch	Step	Training Loss	Validation Loss
0.0085	100	0.2854	-
0.0171	200	0.1561	-
0.0256	300	0.1099	-
0.0342	400	0.0774	-
0.0427	500	0.0777	-
0.0513	600	0.0587	-
0.0598	700	0.0632	-
0.0684	800	0.0605	-
0.0769	900	0.0666	-
0.0855	1000	0.0651	-
0.0940	1100	0.0758	-
0.1026	1200	0.0610	-
0.1111	1300	0.0499	-
0.1197	1400	0.0509	-
0.1282	1500	0.0611	-
0.1368	1600	0.0487	-
0.1453	1700	0.0503	-
0.1539	1800	0.0508	-
0.1624	1900	0.0556	-
0.1710	2000	0.0454	0.0327
0.1795	2100	0.0443	-
0.1881	2200	0.0505	-
0.1966	2300	0.0454	-
0.2052	2400	0.0433	-
0.2137	2500	0.0417	-
0.2223	2600	0.0441	-
0.2308	2700	0.0373	-
0.2394	2800	0.0481	-
0.2479	2900	0.0405	-
0.2565	3000	0.0473	-
0.2650	3100	0.0374	-
0.2736	3200	0.0254	-
0.2821	3300	0.0408	-
0.2906	3400	0.0359	-
0.2992	3500	0.0382	-
0.3077	3600	0.0297	-
0.3163	3700	0.0407	-
0.3248	3800	0.0343	-
0.3334	3900	0.0373	-
0.3419	4000	0.0452	0.0210
0.3505	4100	0.0485	-
0.3590	4200	0.0393	-
0.3676	4300	0.0375	-
0.3761	4400	0.0356	-
0.3847	4500	0.0407	-
0.3932	4600	0.0343	-
0.4018	4700	0.0401	-
0.4103	4800	0.0381	-
0.4189	4900	0.0317	-
0.4274	5000	0.0277	-
0.4360	5100	0.0307	-
0.4445	5200	0.0289	-
0.4531	5300	0.0334	-
0.4616	5400	0.0348	-
0.4702	5500	0.0319	-
0.4787	5600	0.0382	-
0.4873	5700	0.0255	-
0.4958	5800	0.0298	-
0.5044	5900	0.0335	-
0.5129	6000	0.0302	0.0233
0.5215	6100	0.0308	-
0.5300	6200	0.0313	-
0.5386	6300	0.0298	-
0.5471	6400	0.0318	-
0.5557	6500	0.0314	-
0.5642	6600	0.0385	-
0.5727	6700	0.0304	-
0.5813	6800	0.0293	-
0.5898	6900	0.0232	-
0.5984	7000	0.0291	-
0.6069	7100	0.0348	-
0.6155	7200	0.0256	-
0.6240	7300	0.0314	-
0.6326	7400	0.0290	-
0.6411	7500	0.0294	-
0.6497	7600	0.0298	-
0.6582	7700	0.0271	-
0.6668	7800	0.0338	-
0.6753	7900	0.0259	-
0.6839	8000	0.0294	0.0194
0.6924	8100	0.0250	-
0.7010	8200	0.0282	-
0.7095	8300	0.0295	-
0.7181	8400	0.0302	-
0.7266	8500	0.0254	-
0.7352	8600	0.0229	-
0.7437	8700	0.0240	-
0.7523	8800	0.0274	-
0.7608	8900	0.0230	-
0.7694	9000	0.0247	-
0.7779	9100	0.0262	-
0.7865	9200	0.0269	-
0.7950	9300	0.0337	-
0.8036	9400	0.0283	-
0.8121	9500	0.0270	-
0.8207	9600	0.0193	-
0.8292	9700	0.0215	-
0.8378	9800	0.0283	-
0.8463	9900	0.0226	-
0.8548	10000	0.0224	0.0175
0.8634	10100	0.0267	-
0.8719	10200	0.0341	-
0.8805	10300	0.0250	-
0.8890	10400	0.0222	-
0.8976	10500	0.0218	-
0.9061	10600	0.0230	-
0.9147	10700	0.0235	-
0.9232	10800	0.0293	-
0.9318	10900	0.0292	-
0.9403	11000	0.0223	-
0.9489	11100	0.0259	-
0.9574	11200	0.0352	-
0.9660	11300	0.0243	-
0.9745	11400	0.0237	-
0.9831	11500	0.0281	-
0.9916	11600	0.0271	-
1.0	11698	-	0.0160

Training Time

• Training: 1.0 hours
• Evaluation: 28.5 seconds
• Total: 1.0 hours

Framework Versions

• Python: 3.12.13
• Sentence Transformers: 5.6.1
• Transformers: 5.14.1
• PyTorch: 2.10.0+cu128
• Accelerate: 1.14.0
• Datasets: 5.0.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}