SentenceTransformer based on BAAI/bge-large-en-v1.5

This is a sentence-transformers model finetuned from BAAI/bge-large-en-v1.5. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: BAAI/bge-large-en-v1.5
Maximum Sequence Length: 512 tokens
Output Dimensionality: 1024 dimensions
Similarity Function: Cosine Similarity

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'The weather is lovely today.',
    "It's so sunny outside!",
    'He drove to the stadium.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.8286, 0.3838],
#         [0.8286, 1.0000, 0.3545],
#         [0.3838, 0.3545, 1.0000]])

Training Details

Training Dataset

Unnamed Dataset

Size: 2,076 training samples
Columns: sentence_0, sentence_1, and sentence_2

Approximate statistics based on the first 1000 samples:

	sentence_0	sentence_1	sentence_2
type	string	string	string
details	min: 10 tokens mean: 178.83 tokens max: 470 tokens	min: 11 tokens mean: 188.98 tokens max: 473 tokens	min: 16 tokens mean: 198.35 tokens max: 316 tokens

Samples:

sentence_0	sentence_1	sentence_2
A wealthy businessman's teenage daughter is abducted while walking home from school by a group of masked individuals. The kidnappers contact the family demanding five million dollars in unmarked bills and threaten to harm the girl if police are involved. Despite his wife's pleas to comply, the businessman decides to publicly announce that he will not pay the ransom and instead offers the money as a bounty for the kidnappers' capture. The strategy backfires when the lead kidnapper, feeling cornered and betrayed by his accomplices who want to collect the bounty, decides to eliminate the only witness to their crime. In a climactic confrontation at an abandoned warehouse, the businessman arrives alone to negotiate his daughter's release. The kidnapper attempts to kill both father and daughter, but is ultimately overpowered when the daughter manages to free herself and assist her father. The film concludes with the family reunited and the surviving kidnappers arrested by authorities who had...	A prominent surgeon's young son is seized while leaving his piano lesson by a crew of hooded assailants. The abductors reach out to the family requesting three million dollars in untraceable currency and warn that any contact with law enforcement will result in the boy's death. Against his spouse's desperate appeals to meet their demands, the surgeon chooses to make a televised statement refusing to pay and instead places the entire sum as a reward for the criminals' apprehension. The plan goes awry when the primary captor, now desperate and abandoned by his partners who seek to claim the reward money, resolves to silence the sole witness to their actions. During a final showdown at a derelict factory, the surgeon comes alone attempting to secure his son's freedom. The captor tries to murder both father and child, but is defeated when the boy breaks free from his restraints and helps his father subdue the attacker. The story ends with the family's emotional reunion and the remaining cr...	A struggling marine biologist discovers that her research partner has been secretly selling protected coral specimens to wealthy collectors on the black market. When she threatens to expose the illegal operation, her partner frames her for the theft and disappears with their shared research data worth years of work. Desperate to clear her name and recover her life's work, the biologist begins tracking down the buyers herself, posing as a potential customer in the underground exotic specimen trade. During a tense meeting with the largest collector, she realizes he has been unknowingly purchasing from multiple poachers who are systematically destroying an entire reef ecosystem. Rather than pursuing revenge against her former partner, she convinces the collector to become a whistleblower and help authorities dismantle the entire smuggling network. The collector agrees to cooperate in exchange for immunity, leading to multiple arrests but allowing her former partner to escape to another co...
A seasoned detective is tasked with solving a series of seemingly unrelated murders in a bustling metropolis. As he delves deeper, he discovers a cryptic pattern linking the victims to a powerful underground crime syndicate. With the help of a reluctant informant, he uncovers a conspiracy involving high-ranking officials and a planned heist targeting a government vault. The detective’s investigation leads him to a confrontation with the syndicate’s enigmatic leader, who reveals a personal connection to his past. Amid escalating tension, the detective must navigate a web of betrayal, including a mole within his own department. In a climactic showdown, he thwarts the heist but at a great personal cost, leaving the city safe but his own life in shambles. The film ends with the detective walking away, haunted by the choices he made.	A veteran investigator, Clara Hayes, is assigned to investigate a string of seemingly random killings in the sprawling city of Newhaven. As she digs deeper, she uncovers a hidden thread connecting the victims to a notorious smuggling ring operating in the shadows. With assistance from a hesitant insider, Clara exposes a plot involving corrupt city officials and an imminent attack on a high-security treasury. Her pursuit leads to a tense encounter with the smuggling ring’s elusive mastermind, who discloses a long-buried tie to her own history. As trust erodes, Clara discovers a traitor within her own unit, complicating her mission. In a dramatic final confrontation, she successfully prevents the treasury raid but suffers devastating personal losses. The city is saved, but Clara is left broken and disillusioned. The story concludes with her walking away into the night, burdened by the weight of her sacrifices.	In a remote desert town, a reclusive inventor named Elias struggles to repair a malfunctioning weather machine that once brought prosperity to the region. When a sudden sandstorm devastates the town’s crops, the desperate mayor demands answers, accusing Elias of negligence. As Elias investigates, he discovers that the machine’s core component has been stolen, leading him to suspect a rival inventor from a neighboring settlement. With the help of a young scavenger named Kira, he tracks the missing part to an abandoned mining outpost, where he uncovers a plot to monopolize the region’s water supply. Elias confronts the rival, who reveals the theft was orchestrated by a secret coalition of merchants seeking to control the town’s resources. In a daring act of sabotage, Elias overrides the weather machine to trigger a massive rainstorm, restoring balance but destroying the machine in the process. The town celebrates, but Elias, knowing his life’s work is gone, quietly departs, leaving Kira ...
A war veteran returns to his small hometown after years of service, struggling to adjust to civilian life. Haunted by memories of the battlefield, he isolates himself, spending most of his days repairing an old motorcycle in his garage. When a local teenager asks for help after being bullied, the veteran reluctantly agrees to teach him self-defense, forming an unexpected bond in the process. As their friendship grows, the veteran begins to confront his own trauma, sharing stories from his past and finding solace in mentorship. When the bullies escalate their harassment, the veteran steps in, using his skills to defuse the conflict without violence. His actions earn him the respect of the community, and he begins to reintegrate into society, finding purpose in guiding others. The film ends with the veteran riding his repaired motorcycle into the countryside, symbolizing his journey toward healing.	A former firefighter returns to his quiet coastal town after years of service, grappling with the emotional scars left by a devastating blaze that claimed his crew. Withdrawn and burdened by guilt, he spends his days restoring a weathered fishing boat in a dilapidated shed by the harbor. When a shy teenager from the neighborhood seeks his advice on dealing with a group of bullies, he hesitantly agrees to teach the boy how to stand his ground, sparking an unexpected friendship. Through their time together, the firefighter begins to open up about his past, finding catharsis in sharing his experiences and helping the boy grow in confidence. When the bullies become more aggressive, the firefighter intervenes, resolving the situation calmly and without confrontation. His quiet courage earns him newfound admiration from the townspeople, and he starts to reconnect with the community, discovering a renewed sense of purpose. The film concludes with him sailing the restored boat into the open se...	A reclusive lighthouse keeper, burdened by decades of solitude and the weight of his estranged family, tends to a crumbling beacon on a stormy coastline. One evening, a shipwreck survivor—a young artist—washes ashore, seeking shelter. Initially cold and dismissive, the keeper reluctantly allows the artist to stay in the lighthouse. Over time, the artist begins painting murals on the walls, capturing the swirling tempest outside and the quiet anguish within. The keeper, moved by the vivid depictions, shares fragments of his life, revealing the regret that drove him to isolation. When another storm threatens to destroy the lighthouse, the pair collaborate to reinforce its foundation, their efforts blending practicality and artistry. As dawn breaks, the keeper watches the artist depart, leaving behind the murals as a testament to their fleeting connection. The lighthouse, scarred but standing, becomes a symbol of fragile resilience.

Loss: TripletLoss with these parameters:

{
    "distance_metric": "TripletDistanceMetric.COSINE",
    "triplet_margin": 0.5
}

Training Hyperparameters

Non-Default Hyperparameters

num_train_epochs: 10
fp16: True
multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

do_predict: False
eval_strategy: no
prediction_loss_only: True
per_device_train_batch_size: 8
per_device_eval_batch_size: 8
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 5e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1
num_train_epochs: 10
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: None
warmup_ratio: None
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
enable_jit_checkpoint: False
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
use_cpu: False
seed: 42
data_seed: None
bf16: False
fp16: True
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: -1
ddp_backend: None
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
group_by_length: False
length_column_name: length
project: huggingface
trackio_space_id: trackio
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
hub_revision: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_for_metrics: []
eval_do_concat_batches: True
auto_find_batch_size: False
full_determinism: False
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_num_input_tokens_seen: no
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
liger_kernel_config: None
eval_use_gather_object: False
average_tokens_across_devices: True
use_cache: False
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: round_robin
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss
1.9231	500	0.1024
3.8462	1000	0.0087
5.7692	1500	0.0016
7.6923	2000	0.0003
9.6154	2500	0.0001

Framework Versions

Python: 3.11.7
Sentence Transformers: 5.2.2
Transformers: 5.0.0
PyTorch: 2.5.1+cu121
Accelerate: 1.10.1
Datasets: 4.2.0
Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

TripletLoss

@misc{hermans2017defense,
    title={In Defense of the Triplet Loss for Person Re-Identification},
    author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
    year={2017},
    eprint={1703.07737},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}