SentenceTransformer based on BAAI/bge-large-en-v1.5

This is a sentence-transformers model finetuned from BAAI/bge-large-en-v1.5. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: BAAI/bge-large-en-v1.5
Maximum Sequence Length: 512 tokens
Output Dimensionality: 1024 dimensions
Similarity Function: Cosine Similarity

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'The weather is lovely today.',
    "It's so sunny outside!",
    'He drove to the stadium.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.7888, 0.2997],
#         [0.7888, 1.0000, 0.2630],
#         [0.2997, 0.2630, 1.0000]])

Training Details

Training Dataset

Unnamed Dataset

Size: 2,076 training samples
Columns: sentence_0, sentence_1, and sentence_2

Approximate statistics based on the first 1000 samples:

	sentence_0	sentence_1	sentence_2
type	string	string	string
details	min: 24 tokens mean: 179.54 tokens max: 512 tokens	min: 24 tokens mean: 189.56 tokens max: 512 tokens	min: 19 tokens mean: 199.04 tokens max: 330 tokens

Samples:

sentence_0	sentence_1	sentence_2
In a medieval kingdom, dragons, long thought to be extinct, emerge from their hidden sanctuary beneath the mountains. Their leader, a colossal, ancient dragon named Vyrak, declares vengeance on humanity for centuries of persecution. Towns are razed, and the king’s army is overwhelmed as the dragons reclaim territories once stolen from them. A young blacksmith, Elara, discovers she has a rare ability to communicate with dragons and convinces a renegade dragon, Kael, to help broker peace. Together, they infiltrate Vyrak’s lair, where Elara pleads for coexistence, revealing humanity’s dwindling knowledge of the natural world. Vyrak reluctantly agrees, leading to a fragile truce between dragons and humans. The film ends with Elara standing atop a ridge, watching dragons soar freely over the kingdom as both sides cautiously rebuild their worlds.	In an ancient desert empire, phoenixes, believed to have vanished centuries ago, reappear from their hidden oasis deep within the dunes. Their leader, an immense and fiery elder phoenix named Seryth, vows retribution against humanity for generations of exploitation and habitat destruction. Villages are incinerated, and the emperor’s elite forces are no match for the phoenixes as they reclaim sacred lands. A young glassblower, Amira, discovers she has the rare gift of understanding phoenix song and persuades a rogue phoenix, Rynar, to aid her in seeking peace. Together, they venture into Seryth’s volcanic sanctuary, where Amira appeals for harmony, emphasizing humanity’s forgotten respect for the balance of nature. Seryth begrudgingly accepts, initiating an uneasy accord between the phoenixes and the empire. The story concludes with Amira standing on a sunlit dune, watching phoenixes streak across the sky as both sides cautiously strive for renewal.	In a futuristic urban sprawl, genetically engineered creatures known as "Skyforms" break free from their corporate labs and begin to wreak havoc on the city. Led by their most advanced prototype, a towering, biomechanical entity called Zyrak, the Skyforms seek revenge for years of exploitation and experimentation. Amid the chaos, a data courier named Lina discovers she can interface with the Skyforms' neural systems, a skill she honed while hacking surveillance drones. Teaming up with an injured Skyform named Ryl, Lina devises a plan to shut down Zyrak's command signal and free the remaining creatures from their programmed aggression. In the climactic confrontation atop a ruined skyscraper, Lina overrides Zyrak’s systems, but instead of destroying him, she reprograms him to guide the Skyforms to an uninhabited zone outside the city. The story ends with Lina watching the skyline as the Skyforms vanish into the horizon, leaving humanity to repair the damage caused by its hubris.
In a future where humanity has mastered interstellar travel, a crew of scientists embarks on a mission to investigate a distant exoplanet believed to harbor signs of extraterrestrial life. Upon arrival, they discover an ancient alien artifact buried beneath the planet's surface, which emits a mysterious energy signal. As they attempt to decipher its purpose, the artifact activates, projecting holographic images of a long-lost alien civilization that once thrived in the region. The crew learns that the artifact is a warning device, designed to alert others of a catastrophic event that wiped out the aliens. While debating whether to deactivate it or let it continue broadcasting, the crew faces sabotage from within, revealing a traitor with a hidden agenda. A tense confrontation ensues, leading to the destruction of the artifact and the crew's desperate escape as the planet begins to destabilize. Returning to Earth, they must confront the implications of their discovery and the fragile na...	In a distant future where humanity has colonized multiple star systems, a team of researchers is sent to explore a remote moon, Zephyria, rumored to contain evidence of ancient alien activity. Upon landing, they uncover a colossal structure embedded in the moon’s icy crust, emitting an enigmatic pulsating light. As they study the structure, it awakens, projecting visions of an advanced civilization that once flourished in the area. The team deciphers that the structure is a beacon, warning of a cosmic disaster that annihilated its creators. While debating whether to preserve or dismantle the beacon, internal betrayal surfaces, exposing a crew member who seeks to exploit its technology for personal gain. A violent clash erupts, resulting in the beacon’s destruction and the moon’s rapid geological collapse. The team narrowly escapes, returning to their homeworld to grapple with the moral and existential ramifications of their encounter and the dangers of venturing into the unknown.	In a remote desert town, a reclusive inventor named Elias creates a machine capable of translating the whispers of the wind into coherent messages. Believing the wind carries the voices of forgotten ancestors, Elias becomes obsessed with uncovering their secrets. When the machine unexpectedly broadcasts a cryptic warning about an impending natural disaster, the townspeople dismiss it as a hoax. However, a young geologist named Mira investigates and discovers evidence of an ancient fault line beneath the town. As tensions rise, Elias and Mira struggle to convince the community to evacuate before it’s too late. In a climactic moment, a minor earthquake shakes the town, finally spurring the residents into action. Though the disaster is averted, Elias is left grappling with the ethical dilemma of whether to continue his experiments, fearing the unintended consequences of meddling with forces beyond human understanding.
In an ancient era, a once-prosperous kingdom is forgotten by history, its citizens scattered and its former glory hidden beneath the sands of time. A young adventurer, driven by a map and a legend, embarks on a quest to uncover the lost kingdom. Along the way, they encounter a wise old sage who warns of the kingdom's dark secrets and the dangers that lie within. The adventurer discovers the kingdom, now a labyrinth of ruins filled with ancient treasures and perils. After overcoming trials and facing ancient curses, they unlock the truth behind the kingdom's fall and restore a measure of its former grandeur.	In a forgotten age, a once-prosperous realm is lost to time, its people vanished and its splendor buried beneath the dunes. A young explorer, guided by an ancient map and a myth, sets out on a quest to find the lost realm. On their journey, they meet an old guardian who cautions of the realm's hidden terrors and the hazards that await. The explorer uncovers the realm, now a maze of ruins teeming with ancient riches and dangers. After surmounting trials and confronting ancient curses, they reveal the truth behind the realm's downfall and revive a fraction of its former magnificence.	In a remote valley, a group of aspiring filmmakers sets out to uncover the truth behind a mysterious, long-abandoned village. Guided by an old, weathered map and a local legend, they trek through dense forests and treacherous terrain. They encounter an eccentric, reclusive artist who warns them of the village's dark past and the hidden dangers that still linger. Inside the village, they discover remnants of a once-thriving community, now overgrown with wild flora and filled with eerie artifacts. As they delve deeper, they face their own internal conflicts and moral dilemmas, ultimately restoring the village to a state of semi-life through their film, preserving its legacy and confronting their own fears.

Loss: TripletLoss with these parameters:

{
    "distance_metric": "TripletDistanceMetric.COSINE",
    "triplet_margin": 0.4
}

Training Hyperparameters

Non-Default Hyperparameters

num_train_epochs: 6
fp16: True
multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

do_predict: False
eval_strategy: no
prediction_loss_only: True
per_device_train_batch_size: 8
per_device_eval_batch_size: 8
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 5e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1
num_train_epochs: 6
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: None
warmup_ratio: None
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
enable_jit_checkpoint: False
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
use_cpu: False
seed: 42
data_seed: None
bf16: False
fp16: True
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: -1
ddp_backend: None
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
group_by_length: False
length_column_name: length
project: huggingface
trackio_space_id: trackio
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
hub_revision: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_for_metrics: []
eval_do_concat_batches: True
auto_find_batch_size: False
full_determinism: False
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_num_input_tokens_seen: no
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
liger_kernel_config: None
eval_use_gather_object: False
average_tokens_across_devices: True
use_cache: False
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: round_robin
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss
1.9231	500	0.0403
3.8462	1000	0.0027
5.7692	1500	0.0009

Framework Versions

Python: 3.11.7
Sentence Transformers: 5.2.2
Transformers: 5.0.0
PyTorch: 2.5.1+cu121
Accelerate: 1.10.1
Datasets: 4.2.0
Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

TripletLoss

@misc{hermans2017defense,
    title={In Defense of the Triplet Loss for Person Re-Identification},
    author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
    year={2017},
    eprint={1703.07737},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}