ahmedHamdi/IR-pt-en-gemma

SentenceTransformer based on google/embeddinggemma-300m

This is a sentence-transformers model finetuned from google/embeddinggemma-300m. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: google/embeddinggemma-300m
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'Gemma3TextModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("ahmedHamdi/IR-pt-en-gemma")
# Run inference
sentences = [
    "Tom Brand (Kevin Spacey) is a business tycoon who has separated his daughter Rebecca (Malina Weissman) and his wife Lara (Jennifer Garner). In a rush to get a last-minute gift for his daughter's birthday, he goes to a mysterious pet shop run by Felix Grant (Christopher Walken) and buys a cat. On the way home, Tom is involved in an accident and becomes trapped inside the cat's body. Felix tells him that he must reconcile with his family within a week or he will remain a cat forever.",
    "Tom Brand is a major business tycoon in New York City whose workaholic attitude ruined his first marriage with Madison, and his adult son David now works for him and strives for his approval. Tom and David now live with their second wife and stepmother Lara, who is a little more tolerant of the fact that Tom is never at home, and their daughter and half-sister Rebecca. Tom’s eponymous company FireBrand is nearing completion on its greatest achievement to date: the tallest skyscraper in the northern hemisphere that will be the new headquarters. In the opening scene, he skydives out of a plane and lands on the top of the new skyscraper, while David remains in the plane, refusing to jump. Rebecca's 11th birthday comes up and she has always wanted a cat, but Tom has always refused as he hates cats. Not wanting to disappoint her, he rushes to get a last-minute present for his daughter's birthday, but as he is distracted, the GPS redirects him to a mysterious pet store called Purrkins Pet Shop, brimming with odd and exotic cats. The store's eccentric owner Felix Perkins tells him he does not pick the cat, the cat picks him. The tomcat that picks him is called Mr. Fuzzypants, who, Perkins says, has used up seven of his nine lives. Tom buys the cat. On his way home for the party, Tom meets with Ian Cox, one of the top managers of the company in charge of the new building, and learns that another building in Chicago will be taller. They have an argument while on the roof that ends with Tom firing Ian, but lightning strikes the antenna. Tom is blown off the building along with the cat, and without any effort from Ian to save him, he plummets off the side of the skyscraper, but his leg gets snagged on some stray equipment and gets flung back inside through a window as he passes out. When he wakes up, he realizes that his human body is in the hospital in a state of coma, and his consciousness is trapped inside the cat's body. Tom is visited by Felix, who is able to talk to him and knows what happened. Felix warns Tom to reevaluate his priorities, connect with his family, and avoid past mistakes within one week or else remain as a cat forever. When Lara and Rebecca take Mr. Fuzzypants home, he acts in an odd and stubborn way to try to convince his wife and daughter that he is actually Tom. This only drives them mad and he slowly comes to see how much he has ignored his family. He suspects that Lara is cheating on him with model Josh Myers and they had been looking at houses together as a prelude to divorcing him. This motivates him to try to make her happy. He also learns that Ian is trying to make the company public with the help of the board of directors to take power from Tom, even though David is trying to stop him. Ian fires David from the company and plans to announce that the company will be sold at the party for the new tower opening. Eventually, Rebecca realizes that Mr. Fuzzypants is really her father. Meanwhile, Tom's body is in crisis at the hospital and Lara, David, and Rebecca all go there. Rebecca has the cat hidden in her backpack. In a moment alone, David apologizes to Tom for failing to save the company and takes Tom's ID badge, implying he will commit suicide at the tower. At the hospital, Lara and Dr. Cole plan for Lara to sign a DNR, which is a legal form, indicating that if Tom's heart stops beating, they won't save him. Rebecca calls to the cat to come prove he is really Tom, but Tom remembers what Felix said about love being sacrifice and decides to save David instead, sacrificing his only chance at regaining his humanity. At the tower, David jumps off the building and the cat jumps after him pulling a cable. It is then revealed that David is wearing a BASE jumping parachute. He lands in the middle of the party, presents the articles of incorporation of the company, and announces that he now controls his father's 51% of the stock. He says the company will remain a family company and fires Ian. Although the cat is not seen landing, Tom wakes up from his coma just in time to see David's announcement on the television. Meanwhile, Ian passes Felix who tells him to hang up his cell phone. Ian ignores him, but is then hit by a car, and his consciousness transfers into a cat that Felix takes back to his shop. Tom and Rebecca return to Felix where Tom asks if he has any dogs. Felix says he does not, but presents Mr. Fuzzypants, who has one life left.",
    "Aging Greek Theo Tomasis rose from his peasant roots to become a mogul who owns oil tankers, airlines, and Mediterranean islands and longs to be elected President of Greece. He eventually falls in love with the considerably younger Liz Cassidy. The two first meet when she is visiting his island estate with her husband James, the Senator from the state of Massachusetts. Theo immediately is attracted to her and, despite the fact she is happily married, begins to woo her aboard his yacht while James converses with the former Prime Minister of the United Kingdom.  Meanwhile, Theo's relationship with his son, Nico, is tenuous at best. He encourages Nico to join the business and make money in the same manner he did. Nico wants to manage his own affairs and convinces his father to jumpstart his ventures. The plan is to buy older freighters and convert them to oil tankers and sell them to the Saudi family. This would allow Saudi oil magnates to control the delivery of oil outside of American Corporate controls. Theo is impressed and helps bankroll the purchase of a fleet of boats.  Upon their return to the States, the Cassidys begin their run for President of the United States, eventually winning the primary and the general election. The First Lady later suffers a miscarriage and becomes depressed and tired of the public life. Tomasis, who has a good relationship with the First Family, offers a vacation on his yacht as a change of pace. The President does not care for Tomasis due to his shady dealings in the shipping business and Arab oil states, so Lizzie goes by herself, much to his anger. In Greece, the yacht is full of guests and celebrities. Tomasis seems to spend a lot of time with Lizzie, who is a bit restrained. During a dinner however, Lizzie joins in a traditional sensual dance with plates being thrown and the other guests clapping. Lizzie is attracted to Tomasis' free spirit and joie de vivre and leaves the vacation early to go back to the states-much to his protests. Back in the US, Lizzie and James walk on the family beach-front property, where the President is assassinated by a sharpshooter. After the funeral, Lizzie becomes depressed and stays outside of public eyes even though the former President's brother John (the attorney general) requests she help promote his campaign. She refuses, and Tomasis, re-invites her to Greece for a longer stay. In Greece, the relationship blooms and Lizzie cheers up. Tomasis' business dealings with Nico are becoming scrutinized by American politicians due to the OPEC embargoes and anger over oil prices. Freighters are getting confiscated and the investment of hundreds of millions at risk. Upon his return from the US, Theo asks Lizzie to marry him. Lizzie wants to but is worried about appearances in her country but he outlines the arrangement that is fair and financially attractive enough to abandon the Cassidy family fortune and control. After the wedding, there are growing pains for the relationship, as Lizzie is accustomed to being a voice in her husband's affairs in the past. After Tomasis berates her for interfering in his business, he apologizes for his Greek chauvinism, and they learn to live with each other's cultural differences. The marriage is loving until Niko dies in an accident, and Tomasis gets depressed. Lizzie manages to coax him out of his moods, and things appear to look up as his lawyer and representative cops him a cheap plea deal on the oil manipulation charges. However, his health declines and a doctor gives him a bad prognosis. He holds his medical information secret as his energy and health declines. When Liz wants to shop in Paris, he encourages her to go alone and enjoy herself.  She leaves the boat on her vacation, waving, and Tomasis visits his old peasant home on the coast for the last time visiting his old haunts and ending with traditional Greek dance.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.8029, 0.0491],
#         [0.8029, 1.0000, 0.0805],
#         [0.0491, 0.0805, 1.0000]])

Training Details

Training Dataset

Unnamed Dataset

• Size: 5,804 training samples
• Columns: sentence_0 and sentence_1
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1
  type	string	string
  details	min: 10 tokens mean: 121.93 tokens max: 256 tokens	min: 16 tokens mean: 235.34 tokens max: 256 tokens

• Samples:

  sentence_0	sentence_1
  Steve Grayson (Elvis Presley) is a champion race car driver with a generous soul, sharing all his winnings with people in need. Because of his generosity and his boss's (Bill Bixby) passion for gambling and women, Steve finds himself drowning in debt.	Steve Grayson (Presley) is a generous NASCAR race car driver with a heart of gold who feels compelled to bail friends and acquaintances out of financial hardship. However, Steve's manager and best friend Kenny Donford, a compulsive gambler, had been mismanaging Steve's winnings to support his gambling habits, landing Steve in deep trouble with the IRS for nonpayment of back taxes and causing many of Steve's valuable possessions to be repossessed. This proves to be a problem for Steve in his efforts to continue racing competitively and support those who depend on his intense generosity. Enter Susan Jacks (Sinatra), an IRS agent assigned to keep tabs on Steve and apply his future prize money toward his $150,000 debt, but she ends up taking a romantic interest in him as well.
  The lives of three New York couples take incredible turns after two of them, Rebecca (Marley Shelton) and Dag (Ron Eldard), share a forbidden night of love. More betrayal, passion, revenge, and death follow the couple's unmasking, but what if nothing more than a kiss had happened that night?	Dag is a successful director of television commercials who lives with his girlfriend, Halley. Dag, however, has a serious case of wandering eye and is given to frequent flings with other women. Halley tries to turn a blind eye to Dag's infidelity, but when she discovers Dag had a one-night stand with Rebecca, a beautiful but troubled ballet dancer who is dating Dag's close friend Peter, she decides things have gone too far. To add to her feeling of betrayal, she hears a message on the answering machine for Dag from another woman looking to meet up. Halley breaks up with Dag; soon after, she makes the acquaintance of Andre, a handsome and well-mannered classical musician. Andre, however, is married to Colleen, a woman with exotic sexual tastes who meets up with Peter, now suddenly without a girlfriend, on an airline flight. Peter and Colleen have sex in the plane’s bathroom. Just before the plane lands, Peter makes a cell phone call, which causes interference with the signals from the p...
  A horror and suspense thriller following the formula of Steven Spielberg's 1971 film Duel. The peaceful village of Santa Ynez, in the Southwestern United States, is shaken when a mysterious, all-black, extremely fast car runs over and kills people without any apparent reason. The episode unsettles Sheriff Wade Parent (Brolin), Sheriff Everett Peck (Marley), and Deputy Luke Johnson (Cox). The car reappears and kills Everett. An eyewitness, an old Native American woman, claims that the vehicle has no driver, and the assumption that it is a ghost car seems to be confirmed when the car threatens a school parade and Wade's girlfriend, Lauren (Lloyd), manages to lead students and teachers to a cemetery: in confrontation with Christian symbols, the car retreats and flees. The road suspense continues with a thrilling chase, culminating in a direct confrontation between Wade and the Devil's machine. And the plot unfolds with other climaxes until an epilogue." Apocalyptic, with special effects b...	Two cyclists riding through a canyon are followed down the road by a mysterious matte black car. At a bridge, the car proceeds to crush one cyclist against the wall and ram the other from behind, catapulting him off the bridge. A hitchhiker, hoping to get a ride, encounters the car and insults it after it purposely tries to run him down. It halts to a stop, reverses backwards, and runs him over several times before driving off. The local sheriff's office, called to a series of hit and runs, gets a lead on the car. It appears to be heavily customized and has no license plates, as pointed out by Amos Clemens after he saw it run over the hitchhiker. That night, after intentionally swerving around Amos, the car instead runs over Sheriff Everett Peck, leaving Chief Deputy Wade Parent in charge. During the resulting investigation, an eyewitness to the accident states there was no driver, furthering Wade's confusion. Wade asks his girlfriend, Lauren, a teacher at the local school, to cancel t...

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 4
• per_device_eval_batch_size: 4
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 4
• per_device_eval_batch_size: 4
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss
0.3446	500	0.0194
0.6892	1000	0.0846
1.0338	1500	0.0424
1.3784	2000	0.0198
1.7229	2500	0.0171
2.0675	3000	0.0169
2.4121	3500	0.0039
2.7567	4000	0.003

Framework Versions

• Python: 3.9.18
• Sentence Transformers: 5.1.2
• Transformers: 4.57.6
• PyTorch: 2.8.0+cu128
• Accelerate: 1.10.1
• Datasets: 4.5.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}