KingTechnician/osmosis-joint-setfit

SentenceTransformer based on sentence-transformers/paraphrase-MiniLM-L6-v2

This is a sentence-transformers model finetuned from sentence-transformers/paraphrase-MiniLM-L6-v2. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/paraphrase-MiniLM-L6-v2
• Maximum Sequence Length: 128 tokens
• Output Dimensionality: 384 tokens
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("KingTechnician/osmosis-joint-setfit")
# Run inference
sentences = [
    'Objective:\nWhere can i get help for Basic of creating Macros in Visual Basics? Where can i get help to create macros for my daily activities which are to be done everyday and time consuming.\\nSo wanted to know the basic of creating macros in Visual Basic for my activities in Excel\nResponse: Where can i get help for Basic of creating Macros in Visual Basics? I am an Excel developer for a major financial company in NYC and I use MSDN daily to research the Excel Object Model.',
    "Objective:\nI started working out about 1 month ago, specifically working on my abs.How can I get rid of the belly fat? I'm working out 5-6 days a week, and have started eating healthier. I'm feeling some results but I'm not really seeing them yet. When can I expect to see some changes in my stomache?\nResponse: I started working out about 1 month ago, specifically working on my abs.How can I get rid of the belly fat? You have to do cardio to burn belly fat.  Just lifting weights and doing sit ups wont cut it.  Cardio, cardio, cardio!  Everytime you do cardio be sure to throw some stomach crunches in there too.  you should see results in about a month.",
    'Objective:\nAssess student perceptions of question-asking encouragement and answer quality.\nResponse: that reminds me of group projects... we had to work together a lot and it was kinda hard to get everyone on the same page, i mean we had to use this one tool to communicate and it was really annoying to use, the interface was all weird and stuff',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Implementation

To load later: encoder = SentenceTransformer('KingTechnician/osmosis-joint-setfit') head = joblib.load('head.joblib') # download from repo first model = SetFitClassifier(encoder, head)

Training Details

Training Dataset

Unnamed Dataset

• Size: 40,000 training samples
• Columns: sentence_0, sentence_1, and label
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1	label
  type	string	string	float
  details	min: 29 tokens mean: 108.6 tokens max: 128 tokens	min: 33 tokens mean: 110.11 tokens max: 128 tokens	min: 0.0 mean: 0.5 max: 1.0

• Samples:

  sentence_0	sentence_1	label
  Objective: What is the best way to lose 15 lbs in 30 days? I'm looking to lose just a little bit of weight before I had to the Philippines. I weigh 180lbs but I want to get to at least 165lb before my trip. Thanks Response: What is the best way to lose 15 lbs in 30 days? I would do the South Beach Diet, you stick to it and and you can lose 7-15lbs in two weeks. I did.	Objective: whats the best lawyer in michigan for federal drug cases? third time felony on felony probation Response: whats the best lawyer in michigan for federal drug cases? the one who got caught with drugs ED....something he helped my brother	0.0
  Objective: what second lenguage would you study? Spanish\nFrench\nJapanese\nPortuguese\nChinise\nOthers... Response: what second lenguage would you study? Spanish:\nAs a 40 something Engineer, I have found that I should have learned Spanish instead of German. Why because many of the operators in the plants that I have worked are Mexican Americans.	Objective: Should I do something dirty? At a party the other night there was a guy who was flirting with me, and I flirted back. My friends are having another party in a couple of weeks and he is invited. For the past couple of days doing something, well, dirty, with him has been the only thing on my mind. I also want a relationship, but I know we would never work. Should I initiate something, and satisfy those thoughts or leave it alone? Response: Should I do something dirty? do something dirty!!! chances are only once in a blue moon!!! being good all the time is boring. have some fun while you can because you cant do it again once you are old. were not getting any younger, that is the adveture in life. dont miss it.	0.0
  Objective: How does elinks (links) detect double-click when using PuTTY? How can the text browser links / elinks detect double-click when using PuTTY. Response: How does elinks (links) detect double-click when using PuTTY? PuTTY allows the server to send control codes that let it take over the mouse and use it for purposes other than copy and paste.\nApplications which use this feature include the text-mode web browser links', the Usenet newsreader trn' version 4, and the file manager `mc' (Midnight Commander).	Objective: Is there a way to invest into a home when you have no capital? Since I was a teenager I have always dreamed of owning my own home. My ex wife and I when we were married were on far opposite ends of this very issue. She likes to rent because all the repairs are the land lords responcibility and I felt that was a lazy poor excuse not to buy a home. Now that I am divorced and my credit is shot all to hell, I wonder if there is a way to invest "Now that I can do this without the wrath of someone!", in a home that will be my foot in the door and a fresh start on the road to my own Home! I am ready to go after my dreams regardless of what people say or think! HA!!!!!!!!!! Response: Is there a way to invest into a home when you have no capital? Yes you can. In this instance I would suggest finding someone in your area who buys properties for rental purposes. See if they will talk to you about how they purchase propery. There are a few who constantly buy property with little out of pocket capital. There are also books by Robert Kyosocki, and Cources by Carleton Sheets that do work that can teach you how to do this. \n\nYou can even find small trailer park communities in your area. and purchase them for $1,000 to $6,000 clean them up and resell them at double the price and offer financing with 10% interestes for couples trying to find the first home. you can quickly flip several of these and creat a significent down payment for a home of your choice and just use a bit of "Sweat equity".	1.0

• Loss: CosineSimilarityLoss with these parameters:

  {
      "loss_fct": "torch.nn.modules.loss.MSELoss"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: False
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• eval_use_gather_object: False
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin

Training Logs

Epoch	Step	Training Loss
0.2	500	0.3074
0.4	1000	0.2529
0.6	1500	0.2518
0.8	2000	0.2524
1.0	2500	0.2514
1.2	3000	0.2472
1.4	3500	0.2451
1.6	4000	0.2439
1.8	4500	0.24
2.0	5000	0.2369
2.2	5500	0.2249
2.4	6000	0.2235
2.6	6500	0.2178
2.8	7000	0.217
3.0	7500	0.2111

Framework Versions

• Python: 3.12.13
• Sentence Transformers: 3.0.1
• Transformers: 4.44.2
• PyTorch: 2.10.0+cu128
• Accelerate: 1.13.0
• Datasets: 4.0.0
• Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}