Instructions to use dgwon/e5-small-v2.nfcorpus.meta-llama-Llama-3.1-8B-Instruct.top10pct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use dgwon/e5-small-v2.nfcorpus.meta-llama-Llama-3.1-8B-Instruct.top10pct with sentence-transformers:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("dgwon/e5-small-v2.nfcorpus.meta-llama-Llama-3.1-8B-Instruct.top10pct")

sentences = [
    "query: What Foods Cause Cancer?",
    "passage: Observational and ecological studies are generally used to determine the presence of effect of cancer risk-modifying factors. Researchers generally agree that environmental factors such as smoking, alcohol consumption, poor diet, lack of physical activity, and low serum 25-hdyroxyvitamin D levels are important cancer risk factors. This ecological study used age-adjusted incidence rates for 21 cancers for 157 countries (87 with high-quality data) in 2008 with respect to dietary supply and other factors, including per capita gross domestic product, life expectancy, lung cancer incidence rate (an index for smoking), and latitude (an index for solar ultraviolet-B doses). The factors found to correlate strongly with multiple types of cancer were lung cancer (direct correlation with 12 types of cancer), energy derived from animal products (direct correlation with 12 types of cancer, inverse with two), latitude (direct correlation with six types, inverse correlation with three), and per capita gross national product (five types). Life expectancy and sweeteners directly correlated with three cancers, animal fat with two, and alcohol with one. Consumption of animal products correlated with cancer incidence with a lag time of 15–25 years. Types of cancer which correlated strongly with animal product consumption, tended to correlate weakly with latitude; this occurred for 11 cancers for the entire set of countries. Regression results were somewhat different for the 87 high-quality country data set and the 157-country set. Single-country ecological studies have inversely correlated nearly all of these cancers with solar ultraviolet-B doses. These results can provide guidance for prevention of cancer.",
    "passage: Most of the early studies published on soy and breast cancer were not designed to test the effect of soy; the assessment of soy intake was usually crude and few potential confounders were considered in the analysis. In this review, we focused on studies with relatively complete assessment of dietary soy exposure in the targeted populations and appropriate consideration for potential confounders in the statistical analysis of study data. Meta-analysis of the 8 (1 cohort, 7 case–control) studies conducted in high-soy-consuming Asians show a significant trend of decreasing risk with increasing soy food intake. Compared to the lowest level of soy food intake (⩽5 mg isoflavones per day), risk was intermediate (OR=0.88, 95% confidence interval (CI)=0.78–0.98) among those with modest (∼10 mg isoflavones per day) intake and lowest (OR=0.71, 95% CI=0.60–0.85) among those with high intake (⩾20 mg isoflavones per day). In contrast, soy intake was unrelated to breast cancer risk in studies conducted in the 11 low-soy-consuming Western populations whose average highest and lowest soy isoflavone intake levels were around 0.8 and 0.15 mg per day, respectively. Thus, the evidence to date, based largely on case–control studies, suggest that soy food intake in the amount consumed in Asian populations may have protective effects against breast cancer.",
    "passage: Nineteen unselected patients with mild to moderate essential hypertension, whose average supine blood pressure after two months' observation on no treatment was 156/98 mm Hg, were advised not to add salt to food and to avoid sodium-laden foods. After 2 weeks of sodium restriction patients were entered into an 8-week double-blind randomised crossover study of 'Slow Sodium' (Ciba) versus slow sodium placebo. The mean supine blood pressure was 7.1 mm Hg (6.1%) lower in the fourth week of placebo than that in the fourth week of slow sodium (p less than 0.001). Urinary sodium excretion in the fourth week of slow sodium was 162 +/- 9 mmol/24 h and that in the fourth week of placebo was 86 mmol +/- 9 mmol/24 h (p less than 0.001). There was no difference in potassium excretion. These results suggest that moderate sodium restriction achieved by not adding salt and avoiding sodium-laden foods should, if not already, become part of the management of essential hypertension."
]
embeddings = model.encode(sentences)

similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [4, 4]

Notebooks
Google Colab
Kaggle

SentenceTransformer based on intfloat/e5-small-v2

This is a sentence-transformers model finetuned from intfloat/e5-small-v2. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: intfloat/e5-small-v2
Maximum Sequence Length: 512 tokens
Output Dimensionality: 384 dimensions
Similarity Function: Cosine Similarity

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'query: Does Drinking Tea Improve Blood Flow?',
    'passage: Background Tea consumption is associated with a lower risk of cardiovascular disease including stroke. Direct effects of tea components on the vasculature, particularly the endothelium, may partly explain this association. Objective We performed a meta-analysis of controlled human intervention studies on the effect of tea on flow-mediated dilation (FMD) of the brachial artery, a measurement of endothelial function, which is suggested to be associated with cardiovascular risk. Methods Human intervention studies were identified by systematic search of the databases Medline, Embase, Chemical s and Biosis through March 2009 and by hand-searching related articles. Studies were selected based on predefined criteria: intervention with tea as the sole experimental variable, placebo-controlled design, and no missing data on FMD outcome or its variability. A random effects model was used to calculate the pooled overall effect on FMD due to the intake of tea. The impact of various subject and treatment characteristics was investigated in the presence of heterogeneity. Results In total, 9 studies from different research groups were included with 15 relevant study arms. The overall absolute increase in FMD of tea vs. placebo was 2.6% of the arterial diameter (95% CI: 1.8-3.3%; P-value <0.001) for a median daily dose of 500 mL of tea (2–3 cups). This is a relative increase of approximately 40% compared to the average FMD of 6.3% measured under placebo or baseline conditions. There was significant heterogeneity between studies (P-value <0.001) that might partly be explained by the cuff position either distal or proximal to the area of FMD measurement. No indication for publication bias was found. Conclusion Moderate consumption of tea substantially enhances endothelial-dependent vasodilation. This may provide a mechanistic explanation for the reduced risk of cardiovascular events and stroke observed among tea drinkers.',
    'passage: BACKGROUND: The role of diet in renal cell carcinoma risk has been inconclusive. This study uses an integrative approach to assess the role of food groups and food items in renal cell carcinoma risk. DESIGN: A case-control study was conducted from 2003-2006. SUBJECTS/SETTING: Incident cases (n=335) were identified from hospital records and the Florida cancer registry, and population controls (n=337) frequency matched by age (+/-5 years), sex, and race were identified through random-digit dialing. Eating habits were assessed through the use of the 70-item Block food frequency questionnaire. STATISTICAL ANALYSES: Odds ratios (ORs), 95% confidence intervals (CIs), and tests for trends were calculated using logistic regression, controlled for age, sex, race, income, body mass index, and pack-years of smoking. RESULTS: Decreased renal cell carcinoma risk was observed among the total sample and for men for vegetable consumption (all subjects: OR 0.56, 95% CI 0.35, 0.88; men: OR 0.49, 95% CI 0.25, 0.96) but not for fruit consumption. Tomato consumption decreased renal cell carcinoma risk for the total population and for men (all subjects: OR 0.50, 95% CI 0.31, 0.81; men: OR 0.47, 95% CI 0.24, 0.95). Increased risk of renal cell carcinoma was observed among all subjects and among women with increased consumption of red meat (all subjects: OR 4.43, 95% CI 2.02, 9.75; women: OR 3.04, 95% CI 1.60, 5.79). White bread consumption increased renal cell carcinoma risk among women only (OR 3.05, 95% CI 1.50, 6.20), as did total dairy consumption (OR 2.36, 95% CI 1.21, 4.60). CONCLUSIONS: The protective role of vegetables and the increased risk of renal cell carcinoma with meat consumption are supported. The protective role of fruits is not. Novel findings include the increased risk of renal cell carcinoma with white bread and white potato consumption and the decreased risk of renal cell carcinoma with tomato consumption.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000,  0.6244, -0.0561],
#         [ 0.6244,  1.0000,  0.0099],
#         [-0.0561,  0.0099,  1.0000]])

Training Details

Training Dataset

Unnamed Dataset

Size: 3,267 training samples
Columns: anchor and positive
Approximate statistics based on the first 1000 samples:
anchor positive
type string string
details
min: 9 tokens
mean: 13.64 tokens
max: 27 tokens

min: 23 tokens
mean: 332.11 tokens
max: 512 tokens

	anchor	positive
type	string	string
details	min: 9 tokens mean: 13.64 tokens max: 27 tokens	min: 23 tokens mean: 332.11 tokens max: 512 tokens

Samples:

anchor	positive
`query: How Accurate is the ESBL NDP Test?`	passage: Enterobacterial strains producing clavulanic-acid-inhibited extended-spectrum β-lactamases (ESBLs) are increasingly reported worldwide. Conventional detection of ESBL production remains time-consuming (24 to 48 h). Therefore, the ESBL NDP (Nordmann/Dortet/Poirel) test was developed for a rapid identification of ESBLs in Enterobacteriaceae. This biochemical test was based on the in vitro detection of a cephalosporin (cefotaxime) hydrolysis that is inhibited by tazobactam addition. The ESBL activity was evidenced by a color change (red to yellow) of a pH indicator (red phenol) due to carboxyl-acid formation resulting from cefotaxime hydrolysis that was reversed by addition of tazobactam (positive test). The ESBL NDP test was applied to cultured strains (215 ESBL producers and 40 ESBL nonproducers). Its sensitivity and specificity were 92.6% and 100%, respectively. Its sensitivity (100%) was excellent for detection of CTX-M producers. A few ESBL producers (n = 16) that remained s...
`query: Does Curcumin Extend Lifespan?`	passage: Over the last ten years curcumin has been reported to be effective against a wide variety of diseases and is characterized as having anti-carcinogenic, hepatoprotective, thrombosuppressive, cardioprotective, anti-arthritic, and anti-infectious properties. Recent studies performed in both vertebrate and invertebrate models have been conducted to determine whether curcumin was also neuroprotective. The efficacy of curcumin in several pre-clinical trials for neurodegenerative diseases has created considerable excitement mainly due to its lack of toxicity and low cost. This suggests that curcumin could be a worthy candidate for nutraceutical intervention. Since aging is a common risk factor for neurodegenerative diseases, it is possible that some compounds that target aging mechanisms could also prevent these kinds of diseases. One potential mechanism to explain several of the general health benefits associated with curcumin is that it may prevent aging-associated changes in cellu...
`query: Does Boiling Sweet Potatoes Make Them Healthier?`	passage: The effects of baking and boiling on the nutritional and antioxidant properties of three sweet potato cultivars (Beniazuma, Koganesengan, Kotobuki) cultivated in Turkey were investigated. The samples were analyzed for proximate composition, total phenolic content, ascorbic acid, β-carotene, antiradical activity, and free sugars. The dry matter, protein, and starch contents of the sweet potatoes were significantly changed by the treatments while the ash and crude fiber contents did not differ as significantly. The β-carotene contents of baked and boiled sweet potatoes were lower than those of fresh sweet potatoes; however, the total phenolic and ascorbic acid contents of the baked and boiled sweet potatoes were higher than those of the fresh samples. Generally, the antiradical activity of the sweet potatoes increased with the treatments. Sucrose, glucose, and fructose were quantified as free sugars in all fresh sweet potatoes; however, maltose was determined in the treated samp...

Loss: CachedMultipleNegativesRankingLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "cos_sim",
    "mini_batch_size": 16,
    "gather_across_devices": false
}

Evaluation Dataset

Unnamed Dataset

Size: 363 evaluation samples
Columns: anchor and positive
Approximate statistics based on the first 363 samples:
anchor positive
type string string
details
min: 9 tokens
mean: 13.42 tokens
max: 27 tokens

min: 21 tokens
mean: 334.46 tokens
max: 512 tokens

	anchor	positive
type	string	string
details	min: 9 tokens mean: 13.42 tokens max: 27 tokens	min: 21 tokens mean: 334.46 tokens max: 512 tokens

Samples:

anchor	positive
`query: What Causes Non-Celiac Gluten Sensitivity?`	passage: Celiac disease is an immune-mediated inflammatory disorder of the small intestine caused by sensitivity to dietary gluten and related proteins in genetically predisposed individuals. Over the past several years, the concept of non-celiac gluten sensitivity (NCGS) has gained significant interest from the scientific community and mass media and the number of individuals embracing a gluten-free diet is rapidly growing. This condition is characterized by gastrointestinal or extraintestinal symptoms that respond to gluten withdrawal without evidence for underlying celiac disease or wheat allergy. Symptoms display significant overlap with the irritable bowel syndrome. Many important factors regarding this relatively novel condition remain to be elucidated; no discriminative markers to support a diagnosis of gluten sensitivity have been identified yet and its pathogenesis remains obscure. Here we review the current knowledge on NCGS, and outline potential pathogenic pathways of diffe...
`query: What Foods Contain Antibiotics?`	passage: Antibiotic entry into the water environment has been of growing concern. However, few investigations have been performed to examine the potential for indirect human exposure to environmental antibiotic residues. We evaluated the contribution of drinking water and major food consumption to inadvertent intake of antibiotic residues among general human population in Korea. We estimated daily human intake of six antibiotics, i.e., sulfamethazine (SMZ), sulfamethoxazole (SMX), sulfathiazole (STZ), trimethoprim (TMP), enrofloxacin (EFX), and roxithromycin (RTM), by measuring the concentrations of the antibiotics and their major metabolites in urine from general population in Korea (n=541). In addition, we measured antibiotics from source water of drinking water as well as in tap water samples, and surveyed water consumption rates among the study population. To assess the contribution of dietary factor, we also surveyed consumption pattern for several major foods which are suspected ...
`query: Does Red Pepper Help IBS Symptoms?`	passage: BACKGROUND: Abdominal pain, that characterizes irritable bowel syndrome (IBS) together with bloating and disordered defecation, is mainly related to a visceral hypersensitivity due to an increase of TRPV(1) nociceptive nerve fiber activity. AIM: As capsaicin contained in red pepper is able to desensitize the TRPV(1) fibres, we evaluated whether the red pepper oral administration can decrease the symptoms of visceral hypersensitivity in IBS patients. METHODS: The study was performed on 50 patients with IBS diagnosed following Rome II criteria. After a 2-week washout period, 23 patients were planned to receive 4 pills/day, for 6 weeks randomly and in a double blind manner, each containing 150 mg of red pepper powder with a coat that dissolves in the colon, and 27 patients placebo. The patients scored each day in a diary the abdominal pain and bloating intensities following the 5-point Likert scale. The weekly symptom mean scores and the final patient subjective evaluation on tre...

Loss: CachedMultipleNegativesRankingLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "cos_sim",
    "mini_batch_size": 16,
    "gather_across_devices": false
}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: epoch
per_device_train_batch_size: 128
learning_rate: 2e-05
num_train_epochs: 30
warmup_ratio: 0.1
fp16: True
load_best_model_at_end: True
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: epoch
prediction_loss_only: True
per_device_train_batch_size: 128
per_device_eval_batch_size: 8
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 2e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 30
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
bf16: False
fp16: True
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: True
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
project: huggingface
trackio_space_id: trackio
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
hub_revision: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: no
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
liger_kernel_config: None
eval_use_gather_object: False
average_tokens_across_devices: True
prompts: None
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	Validation Loss
1.0	26	3.0073	0.2560
2.0	52	1.0125	0.0271
3.0	78	0.255	0.0142
4.0	104	0.1718	0.0119
5.0	130	0.123	0.0122
6.0	156	0.1083	0.0133
7.0	182	0.0944	0.0124

The bold row denotes the saved checkpoint.

Framework Versions

Python: 3.12.12
Sentence Transformers: 5.1.2
Transformers: 4.57.1
PyTorch: 2.8.0+cu126
Accelerate: 1.11.0
Datasets: 4.0.0
Tokenizers: 0.22.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

CachedMultipleNegativesRankingLoss

@misc{gao2021scaling,
    title={Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup},
    author={Luyu Gao and Yunyi Zhang and Jiawei Han and Jamie Callan},
    year={2021},
    eprint={2101.06983},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}