--- tags: - sentence-transformers - sentence-similarity - feature-extraction - generated_from_trainer - dataset_size:42977 - loss:MatryoshkaLoss - loss:CachedMultipleNegativesRankingLoss base_model: Qwen/Qwen3-VL-Embedding-2B widget: - source_sentence: A muscular, nude man with big, veiny cock and large pecs flexes his biceps and traps while his bara chest, complete with visible male nipples and pubic hair, rises and falls with a hard erection. sentences: - ./Images/12604035.jpg - ./Images/11882380.jpg - ./Images/12535412.jpg - ./Images/12450474.jpg - ./Images/12604909.jpg - source_sentence: 'Kafka from Honkai: Star Rail rides her thick dickgirl cock like a cowgirl, grinding against her futanari girlfriend Wanila, friends with benefits exchanging sweaty smiles as their coworker bodies move in perfect rhythm.' sentences: - ./Images/11902307.jpg - ./Images/12562944.jpg - ./Images/10856318.jpg - ./Images/12110183.jpg - ./Images/12323048.jpg - source_sentence: Camilla from Fire Emblem lies completely nude as a hand squeezes her huge breasts together, her long wavy hair and purple eyes framed by a hair ornament while her tongue pouts from an open mouth, taking a thick penis in her mouth and swallowing his cum on her tongue and breasts. sentences: - ./Images/12281117.jpg - ./Images/12315235.jpg - ./Images/12297876.jpg - ./Images/12516461.jpg - ./Images/13030938.jpg - source_sentence: Shuten Douji from Fate stands in a rear view pose, her light-skinned body adorned with oni horns, accentuating her big, fleshy ass as it fills the high-heeled boots detail. sentences: - ./Images/12368295.jpg - ./Images/12454623.jpg - ./Images/11608710.jpg - ./Images/12431272.jpg - ./Images/12912872.jpg - source_sentence: A young futanari with long black hair and blue eyes stands outdoors in a bedroom, her big breasts bouncing as she masturbates her large penis and full package, throwing her head back in an ahegao expression while cum spurts from her erection and pussy. sentences: - ./Images/12428280.jpg - ./Images/12565353.jpg - ./Images/12438214.jpg - ./Images/11823231.jpg - ./Images/12013212.jpg pipeline_tag: sentence-similarity library_name: sentence-transformers metrics: - cosine_accuracy@1 - cosine_accuracy@3 - cosine_accuracy@5 - cosine_accuracy@10 - cosine_precision@1 - cosine_precision@3 - cosine_precision@5 - cosine_precision@10 - cosine_recall@1 - cosine_recall@3 - cosine_recall@5 - cosine_recall@10 - cosine_ndcg@10 - cosine_mrr@10 - cosine_map@100 model-index: - name: SentenceTransformer based on Qwen/Qwen3-VL-Embedding-2B results: - task: type: information-retrieval name: Information Retrieval dataset: name: nsfw ir eval type: nsfw-ir-eval metrics: - type: cosine_accuracy@1 value: 0.18634840871021777 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.3174204355108878 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.37018425460636517 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.4382328308207705 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.18634840871021777 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.10580681183696257 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.07403685092127303 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.04382328308207705 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.18634840871021777 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.3174204355108878 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.37018425460636517 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.4382328308207705 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.3065599946661936 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.2651106059397514 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.27459570794346067 name: Cosine Map@100 --- # SentenceTransformer based on Qwen/Qwen3-VL-Embedding-2B This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [Qwen/Qwen3-VL-Embedding-2B](https://huggingface.co/Qwen/Qwen3-VL-Embedding-2B) on the json dataset. It maps sentences & paragraphs to a 2048-dimensional dense vector space and can be used for retrieval. ## Model Details ### Model Description - **Model Type:** Sentence Transformer - **Base model:** [Qwen/Qwen3-VL-Embedding-2B](https://huggingface.co/Qwen/Qwen3-VL-Embedding-2B) - **Maximum Sequence Length:** 262144 tokens - **Output Dimensionality:** 2048 dimensions - **Similarity Function:** Cosine Similarity - **Supported Modalities:** Text, Image, Video, Message - **Training Dataset:** - json ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/huggingface/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) ### Full Model Architecture ``` SentenceTransformer( (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}, 'image': {'method': 'forward', 'method_output_name': 'last_hidden_state'}, 'video': {'method': 'forward', 'method_output_name': 'last_hidden_state'}, 'message': {'method': 'forward', 'method_output_name': 'last_hidden_state', 'format': 'structured'}}, 'module_output_name': 'token_embeddings', 'processing_kwargs': {'chat_template': {'add_generation_prompt': True}}, 'unpad_inputs': False, 'architecture': 'Qwen3VLModel'}) (1): Pooling({'embedding_dimension': 2048, 'pooling_mode': 'lasttoken', 'include_prompt': True}) (2): Normalize({}) ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the 🤗 Hub model = SentenceTransformer("sentence_transformers_model_id") # Run inference queries = [ 'May sits with her legs spread, her massive breasts barely contained by her bikini as she grips her huge cock and thick, long penis, precum leaking out while she looks over her shoulder with a smile.', ] documents = [ './Images/11897115.jpg', './Images/12313417.jpg', './Images/12110929.jpg', ] query_embeddings = model.encode_query(queries) document_embeddings = model.encode_document(documents) print(query_embeddings.shape, document_embeddings.shape) # [1, 2048] [3, 2048] # Get the similarity scores for the embeddings similarities = model.similarity(query_embeddings, document_embeddings) print(similarities) # tensor([[ 0.3335, 0.1907, -0.0211]]) ``` ## Evaluation ### Metrics #### Information Retrieval * Dataset: `nsfw-ir-eval` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.sentence_transformer.evaluation.InformationRetrievalEvaluator) | Metric | Value | |:--------------------|:-----------| | cosine_accuracy@1 | 0.1863 | | cosine_accuracy@3 | 0.3174 | | cosine_accuracy@5 | 0.3702 | | cosine_accuracy@10 | 0.4382 | | cosine_precision@1 | 0.1863 | | cosine_precision@3 | 0.1058 | | cosine_precision@5 | 0.074 | | cosine_precision@10 | 0.0438 | | cosine_recall@1 | 0.1863 | | cosine_recall@3 | 0.3174 | | cosine_recall@5 | 0.3702 | | cosine_recall@10 | 0.4382 | | **cosine_ndcg@10** | **0.3066** | | cosine_mrr@10 | 0.2651 | | cosine_map@100 | 0.2746 | ## Training Details ### Training Dataset #### json * Dataset: json * Size: 42,977 training samples * Columns: query, positive, negative0, negative1, negative2, and negative3 * Approximate statistics based on the first 100 samples: | | query | positive | negative0 | negative1 | negative2 | negative3 | |:---------|:------------------------------------------------------------------------------------|:---------|:----------|:----------|:----------|:----------| | type | string | string | string | string | string | string | | modality | text | image | image | image | image | image | | details | | | | | | | * Samples: | query | positive | negative0 | negative1 | negative2 | negative3 | |:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------|:-----------------------------------|:-----------------------------------|:-----------------------------------|:-----------------------------------| | Jane Doe kneels with her skirt hiked, her large breasts jiggling as she deepthroats a thick penis while licking the shaft, her curvaceous body pressed against the viewer’s face and her smooth thighs spread wide to show her dripping pussy and the thick tail pressed against her ass. | ./Images/12354799.jpg | ./Images/12371705.jpg | ./Images/11812526.jpg | ./Images/11742962.jpg | ./Images/11978446.jpg | | Isabelle from Animal Crossing stands with her tied blonde hair and green bikini top and bottom, her big breasts spilling through the cleavage as green hearts hover around her head and her thick thighs and yellow fur draw a female-only, animal-eyed focus. | ./Images/11651564.jpg | ./Images/13035434.jpg | ./Images/13000601.jpg | ./Images/11979417.jpg | ./Images/11790567.jpg | | Ambrosius Goldenloin fucks the hairy, piercing-adorned Nimona with a thick, juicy cock as she moans and clenches around his balls. | ./Images/10880184.jpg | ./Images/11855995.jpg | ./Images/12893082.jpg | ./Images/11948541.jpg | ./Images/12449120.jpg | * Loss: [MatryoshkaLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters: ```json { "loss": "CachedMultipleNegativesRankingLoss", "matryoshka_dims": [ 2048, 1536, 1024, 512, 256, 128, 64 ], "matryoshka_weights": [ 1, 1, 1, 1, 1, 1, 1 ], "n_dims_per_step": -1 } ``` ### Evaluation Dataset #### json * Dataset: json * Size: 4,776 evaluation samples * Columns: query, positive, negative0, negative1, negative2, and negative3 * Approximate statistics based on the first 100 samples: | | query | positive | negative0 | negative1 | negative2 | negative3 | |:---------|:------------------------------------------------------------------------------------|:---------|:----------|:----------|:----------|:----------| | type | string | string | string | string | string | string | | modality | text | image | image | image | image | image | | details | | | | | | | * Samples: | query | positive | negative0 | negative1 | negative2 | negative3 | |:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------|:-----------------------------------|:-----------------------------------|:-----------------------------------|:-----------------------------------| | May sits with her legs spread, her massive breasts barely contained by her bikini as she grips her huge cock and thick, long penis, precum leaking out while she looks over her shoulder with a smile. | ./Images/11897115.jpg | ./Images/12829919.jpg | ./Images/12813676.jpg | ./Images/12136766.jpg | ./Images/12403205.jpg | | On a sunny beach, Mythra from Xenoblade Chronicles 2 wears a skimpy sling bikini, her blonde hair and large breasts on full display as she kneels with arms up, cheeks flushed in an ahegao, enduring deep penetration from a big-penised ugly man while cum trickles from her lactating nipples and a thick toe curls between her fat ass cheeks. | ./Images/12313417.jpg | ./Images/12710154.jpg | ./Images/12739902.jpg | ./Images/11840408.jpg | ./Images/12339885.jpg | | Lisa from Genshin Impact kneels in the center of a gangbang, taking all three large penises deep in her mouth as thick saliva and lipstick marks glisten on their dark skin and her own light skin, while the tattooed, ugly man and his companions thrust into her mouth and onto her face, overwhelming her with big balls and big breasts. | ./Images/12110929.jpg | ./Images/12259516.jpg | ./Images/12317214.jpg | ./Images/11592249.jpg | ./Images/12261530.jpg | * Loss: [MatryoshkaLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters: ```json { "loss": "CachedMultipleNegativesRankingLoss", "matryoshka_dims": [ 2048, 1536, 1024, 512, 256, 128, 64 ], "matryoshka_weights": [ 1, 1, 1, 1, 1, 1, 1 ], "n_dims_per_step": -1 } ``` ### Training Hyperparameters #### Non-Default Hyperparameters - `per_device_train_batch_size`: 48 - `num_train_epochs`: 1 - `learning_rate`: 2e-05 - `warmup_steps`: 0.1 - `bf16`: True - `per_device_eval_batch_size`: 48 - `dataloader_num_workers`: 8 - `batch_sampler`: no_duplicates #### All Hyperparameters
Click to expand - `per_device_train_batch_size`: 48 - `num_train_epochs`: 1 - `max_steps`: -1 - `learning_rate`: 2e-05 - `lr_scheduler_type`: linear - `lr_scheduler_kwargs`: None - `warmup_steps`: 0.1 - `optim`: adamw_torch_fused - `optim_args`: None - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `optim_target_modules`: None - `gradient_accumulation_steps`: 1 - `average_tokens_across_devices`: True - `max_grad_norm`: 1.0 - `label_smoothing_factor`: 0.0 - `bf16`: True - `fp16`: False - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: None - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `use_liger_kernel`: False - `liger_kernel_config`: None - `use_cache`: False - `neftune_noise_alpha`: None - `torch_empty_cache_steps`: None - `auto_find_batch_size`: False - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `include_num_input_tokens_seen`: no - `log_level`: passive - `log_level_replica`: warning - `disable_tqdm`: False - `project`: huggingface - `trackio_space_id`: None - `trackio_bucket_id`: None - `trackio_static_space_id`: None - `per_device_eval_batch_size`: 48 - `prediction_loss_only`: True - `eval_on_start`: False - `eval_do_concat_batches`: True - `eval_use_gather_object`: False - `eval_accumulation_steps`: None - `include_for_metrics`: [] - `batch_eval_metrics`: False - `save_only_model`: False - `save_on_each_node`: False - `enable_jit_checkpoint`: False - `push_to_hub`: False - `hub_private_repo`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_always_push`: False - `hub_revision`: None - `load_best_model_at_end`: False - `ignore_data_skip`: False - `restore_callback_states_from_checkpoint`: False - `full_determinism`: False - `seed`: 42 - `data_seed`: None - `use_cpu`: False - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `parallelism_config`: None - `dataloader_drop_last`: False - `dataloader_num_workers`: 8 - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `dataloader_prefetch_factor`: None - `remove_unused_columns`: True - `label_names`: None - `train_sampling_strategy`: random - `length_column_name`: length - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `ddp_static_graph`: None - `ddp_backend`: None - `ddp_timeout`: 1800 - `fsdp`: None - `fsdp_config`: None - `deepspeed`: None - `debug`: [] - `skip_memory_metrics`: True - `do_predict`: False - `resume_from_checkpoint`: None - `warmup_ratio`: None - `local_rank`: -1 - `prompts`: None - `batch_sampler`: no_duplicates - `multi_dataset_batch_sampler`: proportional - `router_mapping`: {} - `learning_rate_mapping`: {}
### Training Logs | Epoch | Step | Training Loss | Validation Loss | nsfw-ir-eval_cosine_ndcg@10 | |:------:|:----:|:-------------:|:---------------:|:---------------------------:| | 0.3125 | 280 | 14.1777 | - | - | | 0.3237 | 290 | 13.0856 | - | - | | 0.3348 | 300 | 13.0595 | - | - | | 0.3460 | 310 | 13.2660 | - | - | | 0.3571 | 320 | 13.9253 | - | - | | 0.3683 | 330 | 13.0266 | - | - | | 0.3795 | 340 | 12.9428 | - | - | | 0.3906 | 350 | 13.0304 | - | - | | 0.4018 | 360 | 13.0024 | - | - | | 0.4129 | 370 | 12.4377 | - | - | | 0.4241 | 380 | 12.8503 | - | - | | 0.4353 | 390 | 11.5343 | - | - | | 0.4464 | 400 | 13.6251 | - | - | | 0.4576 | 410 | 12.6396 | - | - | | 0.4688 | 420 | 12.0347 | - | - | | 0.4799 | 430 | 12.5446 | - | - | | 0.4911 | 440 | 12.1837 | - | - | | 0.5022 | 450 | 13.3048 | - | - | | 0.5134 | 460 | 13.1663 | - | - | | 0.5246 | 470 | 12.7172 | - | - | | 0.5357 | 480 | 13.5921 | - | - | | 0.5469 | 490 | 12.6573 | - | - | | 0.5580 | 500 | 12.0380 | - | - | | 0.5692 | 510 | 12.3582 | - | - | | 0.5804 | 520 | 12.2693 | - | - | | 0.5915 | 530 | 12.0021 | - | - | | 0.6027 | 540 | 13.1601 | - | - | | 0.6138 | 550 | 11.7692 | - | - | | 0.625 | 560 | 12.3722 | - | - | | 0.6362 | 570 | 11.3196 | - | - | | 0.6473 | 580 | 11.6348 | - | - | | 0.6585 | 590 | 11.8007 | - | - | | 0.6696 | 600 | 12.9399 | - | - | | 0.6808 | 610 | 12.2507 | - | - | | 0.6920 | 620 | 12.3191 | - | - | | 0.7031 | 630 | 11.2547 | - | - | | 0.7143 | 640 | 12.8182 | - | - | | 0.7254 | 650 | 11.6674 | - | - | | 0.7366 | 660 | 10.7994 | - | - | | 0.7478 | 670 | 11.8420 | - | - | | 0.7589 | 680 | 11.9707 | - | - | | 0.7701 | 690 | 12.0791 | - | - | | 0.7812 | 700 | 11.6418 | - | - | | 0.7924 | 710 | 11.9785 | - | - | | 0.8036 | 720 | 11.1946 | - | - | | 0.8147 | 730 | 12.6947 | - | - | | 0.8259 | 740 | 12.2069 | - | - | | 0.8371 | 750 | 11.7020 | - | - | | 0.8482 | 760 | 12.0197 | - | - | | 0.8594 | 770 | 12.8374 | - | - | | 0.8705 | 780 | 11.4478 | - | - | | 0.8817 | 790 | 11.5673 | - | - | | 0.8929 | 800 | 11.5145 | - | - | | 0.9040 | 810 | 11.6466 | - | - | | 0.9152 | 820 | 11.0412 | - | - | | 0.9263 | 830 | 11.7764 | - | - | | 0.9375 | 840 | 11.4838 | - | - | | 0.9487 | 850 | 11.5468 | - | - | | 0.9598 | 860 | 12.9759 | - | - | | 0.9710 | 870 | 11.7273 | - | - | | 0.9821 | 880 | 12.1682 | - | - | | 0.9933 | 890 | 13.0278 | - | - | | 1.0 | 896 | - | 12.2338 | 0.3066 | | -1 | -1 | - | - | 0.3066 | ### Training Time - **Training**: 1.3 hours - **Evaluation**: 24.0 minutes - **Total**: 1.7 hours ### Framework Versions - Python: 3.11.15 - Sentence Transformers: 5.5.1 - Transformers: 5.10.2 - PyTorch: 2.11.0+cu128 - Accelerate: 1.13.0 - Datasets: 4.8.5 - Tokenizers: 0.22.2 ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ``` #### MatryoshkaLoss ```bibtex @misc{kusupati2024matryoshka, title={Matryoshka Representation Learning}, author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi}, year={2024}, eprint={2205.13147}, archivePrefix={arXiv}, primaryClass={cs.LG} } ``` #### CachedMultipleNegativesRankingLoss ```bibtex @misc{gao2021scaling, title={Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup}, author={Luyu Gao and Yunyi Zhang and Jiawei Han and Jamie Callan}, year={2021}, eprint={2101.06983}, archivePrefix={arXiv}, primaryClass={cs.LG} } ```