--- tags: - unsloth - sentence-transformers - sentence-similarity - feature-extraction - dense - generated_from_trainer - dataset_size:1137 - loss:MultipleNegativesRankingLoss base_model: unsloth/Qwen3-Embedding-4B widget: - source_sentence: a-na a-la-hi-im KIŠIB a-mur-IŠTAR sentences: - 'From Ennānum to Idnaya and Aššur- : In accordance with what I wrote to you with my orders that leads - urgent, pay attention and guard my goods and my donkeys like your own life as you are a gentleman. When Aššur-imittī went to the City he brought some 2 or 3 minas of silver on his own, and in the City Aššur-imittī took 0.5 mina of silver, the working capital of halgiaššu. Clear what textiles and tin he brings on his own and let it remain with him. Send me word. Also, carry out in accordance with my instructions I sent to you. Also, with respect to the interest(?) of Aššur-imittī he must not lie and make me angry at you.' - To Ali-ahum; seal of Amur-Ištar. - 'To Ennam-Aššur from Ali-ahum and Amur-Ištar: Sadly, our father has died. It is not Šalim-Aššur who is our father, it is you who are our father. Take care there of our father''s instructions and clear up the affairs. You shall not transfer any consignment of our father''s to this place. One or two of our investors are staying here. Our dear father and lord, clear it up.' - source_sentence: 8 ma-na KÙ ṣa-ru-pá-am i-ṣé-er i-dí-a-bi4-im ku-la i-šu iš-tù ha-mu-uš-tim ša ha-pì-ah-šu ù sú-kà-lí-a a-na 25 ha-am-ša-tim a-na a-lim{ki} i-lá-ak-ma iš-tí um-me-a-ni-šu 2 ma-na KÙ.GI šu-mì ku-lá i-lá-pá-at šu-ma i-na ma-lá u4-me-šu a-na a-lim{ki} lá i-ta-lá-ak KÙ.BABBAR a-na-ku lá i-ša-qal šu-ma lá iš-qú-ul a-na 1 ma-na-im 1½ GÍN.TA i-na 1 ITU.KAM ú-ṣa-áb IGI a-šùr-ba-ni IGI i-dí-sú-in sentences: - If Šalim-Ašsur's investors raise claim against Ennānum concerning tin textiles belonging to Ennānum concerning tin textiles belonging to Ennānum and Ennam-Aššur, then Ennam-Aššur will release Ennānum son of Amriya with Šalim-Aššur's investors for as much tin and ordinary textiles as they clear at the gate of the God. Witnessed by Turaya, by Lā-qēp. - Of 1.5 talents 4 minas of good copper and 1 talent 14 minas of black copper that Ali-ahum left, Aššur-dugul took 31 minas of good copper as the price of Enna-Suen took 4 minas of copper. Zikur-Adad took 8 minas of good copper. 15 minas of copper were paid to his account as the price of a kutānu. 1 mina of black copper minas of copper Um-x took. - Iddin-abum owes 8 minas of refined silver to Kula. Reckoned from the week of happi-ahšu and Sukkalliya he must go to the City within 25 weeks and together with his investors he must book Kula's name for 2 minas of gold. If he has not left for the City within his stipulated term, he must repay the silver to Kula. If he has not paid (in time), he must pay interest at the rate 1.5 shekel per mina per month. Witnessed by Aššur-bāni, by Iddin-Suen. - source_sentence: um-ma e-lá-ma-ma a-na a-šùr-DU10 qí-bi-ma a-na-kam-ma ú-na-hi-id-kà um-ma a-na-ku-ma KÙ.BABBAR 1 ma-na ma-lá tù-tù-ú pá-ni-a-ma šé-bí-lam KÙ.BABBAR ma-lá i qá-tí-kà i-ba-ší-ú x-kà-ma a-ta ki-lá 6 TÚG.HI.A káb-tù-tim SIG5 big_gap ša šu-bi-ri-im ku-ni-lúm na-áš-a-kum mu-dí-a-am ZI-lá-tí-a-am ṣa-ba-at-ma ṭù-ri-šu-nu lu-ša-dí-id-ma a-ni-ib-ra-re lu-sí-ma-ku-šu-nu-ma ma-lá KÙ.BABBAR 1 GÍN tù-šé-lu-ú lu-bu-šé ra-qá-tim ù šu-lu-up-ki-e iš-tí wa-ar-ki-ú-tim ú-šé-ba-lá-kum lu šu-ul-hu lu ma-ku-hu ša a-ma-kam ṭá-áb-ú-ku-um a-mu-ur-ma té-er-ta-kà li-li-kam-ma i ha-ra-an ha-ra-ma ta xx a-ta ANŠE gap li-ta-lu-ku-ni-ma KÙ.BABBAR 1 GÍN li-li-am sentences: - 'Thus Elamma, say to Aššur-ṭāb: It was here that I gave you the following instruction: ''Send me every mina of silver as soon as you find it. All the silver you have already available you must retain .'' 6.3333 heavy textiles, of good quality, of Šubarean make, Kūn-ilum is bringing you. Get hold of a specialist from Zilat and let him stretch their yarns and make them suitable for you for / as -garments, so that you can realize at least some silver. With the next shipment I will send you garments, thin textiles and Šulupkian textiles. Look out for - or -textiles over there that please you and let your report come to me. And let with every single caravan regularly come to me, so that at least some silver becomes available to me.' - Dan-Aššur son of Šalim-ahum Šalim-Aššur has taken a donkey owned by us both; he has 0.3333 mina of silver of his own funds. Of the that Šu-Ištar son of Dadānum conducted here, Pūšu-kēn has given me 6 on his own. Aduda and Šu-Ištar sent them. - Of the 1 mina 33 shekels of tin, of the balance payment of hinnāya, at 7.5 shekels (of tin) per (shekel of silver) its amount in silver is 12 5 / 12 shekels. From what Amur-Aššur (owes?), 16 shekels of silver, of the -tax, thereof Pilah-Ištar took 12 5 / 12 shekels, Elamma took 3 7 / 12 shekels. - source_sentence: um-ma i-ku-pí-a-ma a-na en-um-a-šùr i-dí-a-bi-im ù lá-qé-ep qí-bi-ma 10 ma-na KÙ.BABBAR ku-nu-ki-a en-um-a-šùr na-áš-a-ku-nu-tí a-ṣé-er 40 ma-na AN.NA ša en-um-a-šùr ú-šé-ba-lá-ni ú i-na KÙ.BABBAR-pí-a ša 2 ma-na AN.NA-ak-šu a-ṣé-er ma-na-im ša-a-ma a-dí gap ší-mì ta-ša-a-ma-ni AN.NA na-ší-ra-ma iš-tí pá-ni-im-ma šé-bi-lá-nim a-dí a-lá-ak ṣú-ha-ri-a ší-tí KÙ.BABBAR-pí-a sà-áp-tí-ni x-bi SIG5 ša tal-ha-at pì-ri-kà-ni na-ar-bu-tim ší-ma-am ša ba-lá-ṭí-a ša-ma-nim-ma šé-bi-lá-nim e-ma-ri-šu a-na big_gap i ša-tim iš-tí tám-kà-ri-a ú big_gap a-na big_gap ih-da-ma big_gap sentences: - 'From Ikūn-pīya to Ennam-Aššur, Iddin-abum and Lā-qēp: Ennam-Aššur brings you 10 minas of silver under my seal. In addition to 40 minas of tin which Ennam-Aššur will send, add my silver and buy worth 2 minas of his tin - in addition to the one mina, and until you will make purchases for me , set aside the tin and send it as soon as possible. Until my servant arrives, with the rest of my silver make a profitable purchase of good from Talhat, (and) soft and send them to me. His donkeys per year together with my customers ' - in sum[mer they will ], at Anna's festival they will pay the silver. They will weigh it out by our weight, in the correct amount. - Seal of Kula brother of Šukkutum, seal of Puzur-Ti'amtum son of Ištar-pālil, seal of Ennam-Aššur son of Karria, seal of Enna-Suen son of Iddin-abum, seal of Anuli his brother. As to the goods belonging to Šalim-Aššur that were in hahhum in the house of Issu-arik, and which the sons of Iddin-abum seized - in accordance with the tablet with a verdict of the City that Kakuwa brought, Enna-Suen and Anuli must not raise claim against Šalim-Aššur with respect to these goods. - source_sentence: 37 ku-ta-ni 34 {túg}šu-ru-tum a-na lá-qé-pí-im áp-qí-id IGI i-dí-{d}IŠKUR IGI i-dí-a-šur DUMU sú-e-ta-ta 7 TÚG.HI.A ša li-wi-tim IGI a-šùr-SIPA a-dí-šu-um sentences: - 'To Ešarra and Ab-šalim from Ennam-Aššur: 10 shekels of silver and an undergarment sealed by me is for Ešarra. 10 shekels of silver and 2 sashes are for Ab-šalim and the girl. 2 shekels of silver is for sister Ištar-lamassī ' - 'To Šalim-Aššur from Šu-Tammuzi, Adida, Elaya and Lamassī: When you left you said: Lulu''s son Aššur-ṭāb at least 10 minas of silver " the king will receive its / his tithe. Later the man went to the City assembly and said: "He gave it to me for purchases. I was entrusted with a credit for 5 years." He asked for a favour from the City assembly, and his representative will set his terms for a year and his representative whether you entrusted on credit ' - 37 -textiles (and) 34 dark textiles I entrusted to Lā-qēpum in the presence of Iddin-Adad and of Iddin-Aššur, son of Suettata. 7 textiles for wrapping I gave him in the presence of Aššur-rē'ī. pipeline_tag: sentence-similarity library_name: sentence-transformers metrics: - pearson_cosine - spearman_cosine model-index: - name: SentenceTransformer based on unsloth/Qwen3-Embedding-4B results: - task: type: semantic-similarity name: Semantic Similarity dataset: name: akkadian val type: akkadian_val metrics: - type: pearson_cosine value: 0.9123573791412815 name: Pearson Cosine - type: spearman_cosine value: 0.8625063164293781 name: Spearman Cosine --- # SentenceTransformer This model was finetuned with [Unsloth](https://github.com/unslothai/unsloth). [](https://github.com/unslothai/unsloth) based on unsloth/Qwen3-Embedding-4B This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [unsloth/Qwen3-Embedding-4B](https://huggingface.co/unsloth/Qwen3-Embedding-4B). It maps sentences & paragraphs to a 2560-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. ## Model Details ### Model Description - **Model Type:** Sentence Transformer - **Base model:** [unsloth/Qwen3-Embedding-4B](https://huggingface.co/unsloth/Qwen3-Embedding-4B) - **Maximum Sequence Length:** 1024 tokens - **Output Dimensionality:** 2560 dimensions - **Similarity Function:** Cosine Similarity ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/huggingface/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) ### Full Model Architecture ``` SentenceTransformer( (0): Transformer({'max_seq_length': 1024, 'do_lower_case': False, 'architecture': 'PeftModelForFeatureExtraction'}) (1): Pooling({'word_embedding_dimension': 2560, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': True, 'include_prompt': True}) (2): Normalize() ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the 🤗 Hub model = SentenceTransformer("sentence_transformers_model_id") # Run inference sentences = [ '37 ku-ta-ni 34 {túg}šu-ru-tum a-na lá-qé-pí-im áp-qí-id IGI i-dí-{d}IŠKUR IGI i-dí-a-šur DUMU sú-e-ta-ta 7 TÚG.HI.A ša li-wi-tim IGI a-šùr-SIPA a-dí-šu-um', "37 -textiles (and) 34 dark textiles I entrusted to Lā-qēpum in the presence of Iddin-Adad and of Iddin-Aššur, son of Suettata. 7 textiles for wrapping I gave him in the presence of Aššur-rē'ī.", 'To Ešarra and Ab-šalim from Ennam-Aššur: 10 shekels of silver and an undergarment sealed by me is for Ešarra. 10 shekels of silver and 2 sashes are for Ab-šalim and the girl. 2 shekels of silver is for sister Ištar-lamassī ', ] embeddings = model.encode(sentences) print(embeddings.shape) # [3, 2560] # Get the similarity scores for the embeddings similarities = model.similarity(embeddings, embeddings) print(similarities) # tensor([[1.0000, 0.7541, 0.0110], # [0.7541, 1.0000, 0.0221], # [0.0110, 0.0221, 1.0000]]) ``` ## Evaluation ### Metrics #### Semantic Similarity * Dataset: `akkadian_val` * Evaluated with [EmbeddingSimilarityEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.EmbeddingSimilarityEvaluator) | Metric | Value | |:--------------------|:-----------| | pearson_cosine | 0.9124 | | **spearman_cosine** | **0.8625** | ## Training Details ### Training Dataset #### Unnamed Dataset * Size: 1,137 training samples * Columns: anchor and positive * Approximate statistics based on the first 1000 samples: | | anchor | positive | |:--------|:-------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------| | type | string | string | | details |
  • min: 13 tokens
  • mean: 229.79 tokens
  • max: 579 tokens
|
  • min: 7 tokens
  • mean: 137.38 tokens
  • max: 442 tokens
| * Samples: | anchor | positive | |:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | 1 ma-na KÙ.BABBAR big_gap 4 {túg}ku-ta-ni big_gap ni-ik-na-x- big_gap KÙ.BABBAR a-ha-ma big_gap 2 GÍN KÙ.BABBAR big_gap ša áb-na-tim kà-ú-nam big_gap áb-na-tim big_gap uk-ta-in big_gap KÙ.BABBAR i-za-az big_gap ŠU.NÍGIN 1 ma-na 2 GÍN KÙ.BABBAR i li-bi big_gap IGI pì-lá-ah- big_gap IGI a-šur-na-da šu-ma KÙ.BABBAR a-na big_gap lá iš-ta-qá-al big_gap iš-tù ha-mu-uš-tim ša a-šur-be-el-a-wa-tim 1 ma-na-um 3 GÍN.TA ṣí-ib-tám ú-ṣa-áb i-na ITU.KAM a-ma-nu-šu-um | 1 mina of silver 4 kutānu-textiles silver; further, 2 shekels of silver of the stones confirm He has confirmed the stones. The silver stands ready. In all: 1 mina 2 shekels of silver is owed by Witnessed by Pilah- , by Aššur-nādā. If he has not paid the silver in I shall count interest for him reckoned from the week of Aššur-bēl-awātim at the rate 3 shekels per mina per month. | | ŠU.NÍGIN KÙ.BABBAR-pì-kà 15 ma-na 10 GÍN lu ša AN.NA ú ṣú-ba-tí-kà ku-nu-ki-ni ṣí-li-a na-áš-a-ku-um | Total of your silver: 15 minas 10 shekels, Ṣilliya brings you under our seal - both that from the tin and that from your textiles. | | 1 ma-na 7.5 GÍN KÙ.BABBAR ṣa-ru-pá-am i-ṣé-er a-mur-IŠTAR DUMU da-da e-la-ma i-šu iš-tù ha-muš-tim ša a-la-hi-im ú {d}MAR.TU-ba-ni a-na 11 ha-am-ša-tim i-ša-qal šu-ma lá iš-qú-ul 1½ GÍN.TA ṣí-ib-tám a-na ma-na-im i-na ITU.1.KAM ú-ṣa-áb ITU.KAM ša sà-ra-tim li-mu-um ša qá-té DINGIR-šu-GAL DUMU ba-zi-a IGI im-dí-lim DUMU šu-lá-ba-an IGI e-me-me-i DUMU a-zu-ta-a | 1 mina 7.5 shekels of refined silver Āmur-Ištar, son of Dada, owes to Elamma. From the week of Ali-ahum and Amurrum-bāni he will pay in 11 weeks; if he does not pay he will add 1.5 shekel as interest per mina per month. Month II, eponymy of the successor of Ilšu-rabi, son of Baziya. In the presence of Imdī-ilum, son of Šu-Labān, of Ememe'i, son of Azutaya. | * Loss: [MultipleNegativesRankingLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters: ```json { "scale": 20.0, "similarity_fct": "cos_sim", "gather_across_devices": false } ``` ### Training Hyperparameters #### Non-Default Hyperparameters - `eval_strategy`: steps - `per_device_train_batch_size`: 32 - `learning_rate`: 2e-05 - `lr_scheduler_type`: cosine - `warmup_ratio`: 0.1 - `bf16`: True - `dataloader_pin_memory`: False - `batch_sampler`: no_duplicates #### All Hyperparameters
Click to expand - `overwrite_output_dir`: False - `do_predict`: False - `eval_strategy`: steps - `prediction_loss_only`: True - `per_device_train_batch_size`: 32 - `per_device_eval_batch_size`: 8 - `per_gpu_train_batch_size`: None - `per_gpu_eval_batch_size`: None - `gradient_accumulation_steps`: 1 - `eval_accumulation_steps`: None - `torch_empty_cache_steps`: None - `learning_rate`: 2e-05 - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `max_grad_norm`: 1.0 - `num_train_epochs`: 3 - `max_steps`: -1 - `lr_scheduler_type`: cosine - `lr_scheduler_kwargs`: None - `warmup_ratio`: 0.1 - `warmup_steps`: 0 - `log_level`: passive - `log_level_replica`: warning - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `save_safetensors`: True - `save_on_each_node`: False - `save_only_model`: False - `restore_callback_states_from_checkpoint`: False - `no_cuda`: False - `use_cpu`: False - `use_mps_device`: False - `seed`: 42 - `data_seed`: None - `jit_mode_eval`: False - `bf16`: True - `fp16`: False - `fp16_opt_level`: O1 - `half_precision_backend`: auto - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: None - `local_rank`: 0 - `ddp_backend`: None - `tpu_num_cores`: None - `tpu_metrics_debug`: False - `debug`: [] - `dataloader_drop_last`: False - `dataloader_num_workers`: 0 - `dataloader_prefetch_factor`: None - `past_index`: -1 - `disable_tqdm`: False - `remove_unused_columns`: True - `label_names`: None - `load_best_model_at_end`: False - `ignore_data_skip`: False - `fsdp`: [] - `fsdp_min_num_params`: 0 - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} - `fsdp_transformer_layer_cls_to_wrap`: None - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `parallelism_config`: None - `deepspeed`: None - `label_smoothing_factor`: 0.0 - `optim`: adamw_torch_fused - `optim_args`: None - `adafactor`: False - `group_by_length`: False - `length_column_name`: length - `project`: huggingface - `trackio_space_id`: trackio - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `dataloader_pin_memory`: False - `dataloader_persistent_workers`: False - `skip_memory_metrics`: True - `use_legacy_prediction_loop`: False - `push_to_hub`: False - `resume_from_checkpoint`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_private_repo`: None - `hub_always_push`: False - `hub_revision`: None - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `include_inputs_for_metrics`: False - `include_for_metrics`: [] - `eval_do_concat_batches`: True - `fp16_backend`: auto - `push_to_hub_model_id`: None - `push_to_hub_organization`: None - `mp_parameters`: - `auto_find_batch_size`: False - `full_determinism`: False - `torchdynamo`: None - `ray_scope`: last - `ddp_timeout`: 1800 - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `include_tokens_per_second`: False - `include_num_input_tokens_seen`: no - `neftune_noise_alpha`: None - `optim_target_modules`: None - `batch_eval_metrics`: False - `eval_on_start`: False - `use_liger_kernel`: False - `liger_kernel_config`: None - `eval_use_gather_object`: False - `average_tokens_across_devices`: True - `prompts`: None - `batch_sampler`: no_duplicates - `multi_dataset_batch_sampler`: proportional - `router_mapping`: {} - `learning_rate_mapping`: {}
### Training Logs | Epoch | Step | Training Loss | akkadian_val_spearman_cosine | |:------:|:----:|:-------------:|:----------------------------:| | 0.1389 | 5 | 2.402 | - | | 0.2778 | 10 | 2.3992 | - | | 0.4167 | 15 | 2.1648 | - | | 0.5556 | 20 | 1.8975 | - | | 0.6944 | 25 | 1.4115 | 0.7776 | | 0.8333 | 30 | 1.0211 | - | | 0.9722 | 35 | 0.6742 | - | | 1.1111 | 40 | 0.4176 | - | | 1.25 | 45 | 0.2966 | - | | 1.3889 | 50 | 0.2419 | 0.8580 | | 1.5278 | 55 | 0.2028 | - | | 1.6667 | 60 | 0.1523 | - | | 1.8056 | 65 | 0.1445 | - | | 1.9444 | 70 | 0.106 | - | | 2.0833 | 75 | 0.0906 | 0.8614 | | 2.2222 | 80 | 0.1198 | - | | 2.3611 | 85 | 0.0625 | - | | 2.5 | 90 | 0.1019 | - | | 2.6389 | 95 | 0.0474 | - | | 2.7778 | 100 | 0.0945 | 0.8625 | | 2.9167 | 105 | 0.1227 | - | ### Framework Versions - Python: 3.12.3 - Sentence Transformers: 5.2.0 - Transformers: 4.57.6 - PyTorch: 2.9.0+cu128 - Accelerate: 1.12.0 - Datasets: 4.3.0 - Tokenizers: 0.22.2 ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ``` #### MultipleNegativesRankingLoss ```bibtex @misc{henderson2017efficient, title={Efficient Natural Language Response Suggestion for Smart Reply}, author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil}, year={2017}, eprint={1705.00652}, archivePrefix={arXiv}, primaryClass={cs.CL} } ```