Hyphonical/Qwen3-VL-Embedding-2B-NSFW

SentenceTransformer based on Qwen/Qwen3-VL-Embedding-2B

This is a sentence-transformers model finetuned from Qwen/Qwen3-VL-Embedding-2B on the json dataset. It maps sentences & paragraphs to a 2048-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: Qwen/Qwen3-VL-Embedding-2B
• Maximum Sequence Length: 262144 tokens
• Output Dimensionality: 2048 dimensions
• Similarity Function: Cosine Similarity
• Supported Modalities: Text, Image, Video, Message
• Training Dataset:
  • json

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}, 'image': {'method': 'forward', 'method_output_name': 'last_hidden_state'}, 'video': {'method': 'forward', 'method_output_name': 'last_hidden_state'}, 'message': {'method': 'forward', 'method_output_name': 'last_hidden_state', 'format': 'structured'}}, 'module_output_name': 'token_embeddings', 'processing_kwargs': {'chat_template': {'add_generation_prompt': True}}, 'unpad_inputs': False, 'architecture': 'Qwen3VLModel'})
  (1): Pooling({'embedding_dimension': 2048, 'pooling_mode': 'lasttoken', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
queries = [
    'May sits with her legs spread, her massive breasts barely contained by her bikini as she grips her huge cock and thick, long penis, precum leaking out while she looks over her shoulder with a smile.',
]
documents = [
    './Images/11897115.jpg',
    './Images/12313417.jpg',
    './Images/12110929.jpg',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 2048] [3, 2048]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[ 0.3335,  0.1907, -0.0211]])

Evaluation

Metrics

Information Retrieval

• Dataset: nsfw-ir-eval
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.1863
cosine_accuracy@3	0.3174
cosine_accuracy@5	0.3702
cosine_accuracy@10	0.4382
cosine_precision@1	0.1863
cosine_precision@3	0.1058
cosine_precision@5	0.074
cosine_precision@10	0.0438
cosine_recall@1	0.1863
cosine_recall@3	0.3174
cosine_recall@5	0.3702
cosine_recall@10	0.4382
cosine_ndcg@10	0.3066
cosine_mrr@10	0.2651
cosine_map@100	0.2746

Training Details

Training Dataset

json

• Dataset: json
• Size: 42,977 training samples
• Columns: query, positive, negative0, negative1, negative2, and negative3
• Approximate statistics based on the first 100 samples:

  	query	positive	negative0	negative1	negative2	negative3
  type	string	string	string	string	string	string
  modality	text	image	image	image	image	image
  details	min: 46 tokens mean: 74.73 tokens max: 108 tokens

• Samples:

  query	positive	negative0	negative1	negative2	negative3
  Jane Doe kneels with her skirt hiked, her large breasts jiggling as she deepthroats a thick penis while licking the shaft, her curvaceous body pressed against the viewer’s face and her smooth thighs spread wide to show her dripping pussy and the thick tail pressed against her ass.	./Images/12354799.jpg	./Images/12371705.jpg	./Images/11812526.jpg	./Images/11742962.jpg	./Images/11978446.jpg
  Isabelle from Animal Crossing stands with her tied blonde hair and green bikini top and bottom, her big breasts spilling through the cleavage as green hearts hover around her head and her thick thighs and yellow fur draw a female-only, animal-eyed focus.	./Images/11651564.jpg	./Images/13035434.jpg	./Images/13000601.jpg	./Images/11979417.jpg	./Images/11790567.jpg
  Ambrosius Goldenloin fucks the hairy, piercing-adorned Nimona with a thick, juicy cock as she moans and clenches around his balls.	./Images/10880184.jpg	./Images/11855995.jpg	./Images/12893082.jpg	./Images/11948541.jpg	./Images/12449120.jpg

• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "CachedMultipleNegativesRankingLoss",
      "matryoshka_dims": [
          2048,
          1536,
          1024,
          512,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  

Evaluation Dataset

json

• Dataset: json
• Size: 4,776 evaluation samples
• Columns: query, positive, negative0, negative1, negative2, and negative3
• Approximate statistics based on the first 100 samples:

  	query	positive	negative0	negative1	negative2	negative3
  type	string	string	string	string	string	string
  modality	text	image	image	image	image	image
  details	min: 46 tokens mean: 75.79 tokens max: 133 tokens

• Samples:

  query	positive	negative0	negative1	negative2	negative3
  May sits with her legs spread, her massive breasts barely contained by her bikini as she grips her huge cock and thick, long penis, precum leaking out while she looks over her shoulder with a smile.	./Images/11897115.jpg	./Images/12829919.jpg	./Images/12813676.jpg	./Images/12136766.jpg	./Images/12403205.jpg
  On a sunny beach, Mythra from Xenoblade Chronicles 2 wears a skimpy sling bikini, her blonde hair and large breasts on full display as she kneels with arms up, cheeks flushed in an ahegao, enduring deep penetration from a big-penised ugly man while cum trickles from her lactating nipples and a thick toe curls between her fat ass cheeks.	./Images/12313417.jpg	./Images/12710154.jpg	./Images/12739902.jpg	./Images/11840408.jpg	./Images/12339885.jpg
  Lisa from Genshin Impact kneels in the center of a gangbang, taking all three large penises deep in her mouth as thick saliva and lipstick marks glisten on their dark skin and her own light skin, while the tattooed, ugly man and his companions thrust into her mouth and onto her face, overwhelming her with big balls and big breasts.	./Images/12110929.jpg	./Images/12259516.jpg	./Images/12317214.jpg	./Images/11592249.jpg	./Images/12261530.jpg

• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "CachedMultipleNegativesRankingLoss",
      "matryoshka_dims": [
          2048,
          1536,
          1024,
          512,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 48
• num_train_epochs: 1
• learning_rate: 2e-05
• warmup_steps: 0.1
• bf16: True
• per_device_eval_batch_size: 48
• dataloader_num_workers: 8
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• per_device_train_batch_size: 48
• num_train_epochs: 1
• max_steps: -1
• learning_rate: 2e-05
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_steps: 0.1
• optim: adamw_torch_fused
• optim_args: None
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• optim_target_modules: None
• gradient_accumulation_steps: 1
• average_tokens_across_devices: True
• max_grad_norm: 1.0
• label_smoothing_factor: 0.0
• bf16: True
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• use_liger_kernel: False
• liger_kernel_config: None
• use_cache: False
• neftune_noise_alpha: None
• torch_empty_cache_steps: None
• auto_find_batch_size: False
• log_on_each_node: True
• logging_nan_inf_filter: True
• include_num_input_tokens_seen: no
• log_level: passive
• log_level_replica: warning
• disable_tqdm: False
• project: huggingface
• trackio_space_id: None
• trackio_bucket_id: None
• trackio_static_space_id: None
• per_device_eval_batch_size: 48
• prediction_loss_only: True
• eval_on_start: False
• eval_do_concat_batches: True
• eval_use_gather_object: False
• eval_accumulation_steps: None
• include_for_metrics: []
• batch_eval_metrics: False
• save_only_model: False
• save_on_each_node: False
• enable_jit_checkpoint: False
• push_to_hub: False
• hub_private_repo: None
• hub_model_id: None
• hub_strategy: every_save
• hub_always_push: False
• hub_revision: None
• load_best_model_at_end: False
• ignore_data_skip: False
• restore_callback_states_from_checkpoint: False
• full_determinism: False
• seed: 42
• data_seed: None
• use_cpu: False
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• dataloader_drop_last: False
• dataloader_num_workers: 8
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• dataloader_prefetch_factor: None
• remove_unused_columns: True
• label_names: None
• train_sampling_strategy: random
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• ddp_static_graph: None
• ddp_backend: None
• ddp_timeout: 1800
• fsdp: None
• fsdp_config: None
• deepspeed: None
• debug: []
• skip_memory_metrics: True
• do_predict: False
• resume_from_checkpoint: None
• warmup_ratio: None
• local_rank: -1
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	Validation Loss	nsfw-ir-eval_cosine_ndcg@10
0.3125	280	14.1777	-	-
0.3237	290	13.0856	-	-
0.3348	300	13.0595	-	-
0.3460	310	13.2660	-	-
0.3571	320	13.9253	-	-
0.3683	330	13.0266	-	-
0.3795	340	12.9428	-	-
0.3906	350	13.0304	-	-
0.4018	360	13.0024	-	-
0.4129	370	12.4377	-	-
0.4241	380	12.8503	-	-
0.4353	390	11.5343	-	-
0.4464	400	13.6251	-	-
0.4576	410	12.6396	-	-
0.4688	420	12.0347	-	-
0.4799	430	12.5446	-	-
0.4911	440	12.1837	-	-
0.5022	450	13.3048	-	-
0.5134	460	13.1663	-	-
0.5246	470	12.7172	-	-
0.5357	480	13.5921	-	-
0.5469	490	12.6573	-	-
0.5580	500	12.0380	-	-
0.5692	510	12.3582	-	-
0.5804	520	12.2693	-	-
0.5915	530	12.0021	-	-
0.6027	540	13.1601	-	-
0.6138	550	11.7692	-	-
0.625	560	12.3722	-	-
0.6362	570	11.3196	-	-
0.6473	580	11.6348	-	-
0.6585	590	11.8007	-	-
0.6696	600	12.9399	-	-
0.6808	610	12.2507	-	-
0.6920	620	12.3191	-	-
0.7031	630	11.2547	-	-
0.7143	640	12.8182	-	-
0.7254	650	11.6674	-	-
0.7366	660	10.7994	-	-
0.7478	670	11.8420	-	-
0.7589	680	11.9707	-	-
0.7701	690	12.0791	-	-
0.7812	700	11.6418	-	-
0.7924	710	11.9785	-	-
0.8036	720	11.1946	-	-
0.8147	730	12.6947	-	-
0.8259	740	12.2069	-	-
0.8371	750	11.7020	-	-
0.8482	760	12.0197	-	-
0.8594	770	12.8374	-	-
0.8705	780	11.4478	-	-
0.8817	790	11.5673	-	-
0.8929	800	11.5145	-	-
0.9040	810	11.6466	-	-
0.9152	820	11.0412	-	-
0.9263	830	11.7764	-	-
0.9375	840	11.4838	-	-
0.9487	850	11.5468	-	-
0.9598	860	12.9759	-	-
0.9710	870	11.7273	-	-
0.9821	880	12.1682	-	-
0.9933	890	13.0278	-	-
1.0	896	-	12.2338	0.3066
-1	-1	-	-	0.3066

Training Time

• Training: 1.3 hours
• Evaluation: 24.0 minutes
• Total: 1.7 hours

Framework Versions

• Python: 3.11.15
• Sentence Transformers: 5.5.1
• Transformers: 5.10.2
• PyTorch: 2.11.0+cu128
• Accelerate: 1.13.0
• Datasets: 4.8.5
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

CachedMultipleNegativesRankingLoss

@misc{gao2021scaling,
    title={Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup},
    author={Luyu Gao and Yunyi Zhang and Jiawei Han and Jamie Callan},
    year={2021},
    eprint={2101.06983},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}