Hugging Face's logo

dhammanana
/
harrier-tipitaka-v1

Sentence Similarity
sentence-transformers
ONNX
Safetensors
gemma3_text
feature-extraction
Generated from Trainer
dataset_size:744489
loss:MultipleNegativesRankingLoss
text-embeddings-inference
Model card Files
xet
 Community

SentenceTransformer based on microsoft/harrier-oss-v1-270m

This is a sentence-transformers model finetuned from microsoft/harrier-oss-v1-270m. It maps sentences & paragraphs to a 640-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

  • Model Type: Sentence Transformer
  • Base model: microsoft/harrier-oss-v1-270m
  • Maximum Sequence Length: 128 tokens
  • Output Dimensionality: 640 dimensions
  • Similarity Function: Cosine Similarity
  • Supported Modality: Text

Model Sources

Full Model Architecture 

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'Gemma3TextModel'})
  (1): Pooling({'embedding_dimension': 640, 'pooling_mode': 'lasttoken', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'Instruct: Represent this Buddhist passage for semantic retrieval.\nQuery: Bhagavā taṃ paṭiggahetvā ānandattheraṃ āṇāpesi – ‘‘ imaṃ phalaṃ madditvā pānaṃ dehī ’’ ti. Thero tathā akāsi. Bhagavā ambarasaṃ pivitvā ambaṭṭhiṃ uyyānapālassa datvā ‘‘ imaṃ ropehī ’’ ti āha. So vālukaṃ viyūhitvā taṃ ropesi, ānandatthero kuṇḍikāya udakaṃ āsiñci.',
    'The Blessed One accepted it and instructed Venerable Ānanda: “Crush this fruit and give it as a drink.” The Elder did so. The Blessed One, having drunk the mango juice, gave the mango seed to the gardener and said, “Plant this.” He dug up the sand and planted it, and Venerable Ānanda poured water from a vessel.',
    '“The foremost of my disciples, monks, who are very learned. Who are mindful. Who are clear-headed. Who are resolute. Who are attendants, is Ānanda” –',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 640]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.8314, 0.1940],
#         [0.8314, 1.0000, 0.2902],
#         [0.1940, 0.2902, 1.0000]])

Training Details

Training Dataset

Unnamed Dataset

  • Size: 744,489 training samples
  • Columns: anchor and positive
  • Approximate statistics based on the first 1000 samples:
    anchor positive
    type string string
    details
    • min: 22 tokens
    • mean: 90.93 tokens
    • max: 128 tokens
    • min: 9 tokens
    • mean: 55.09 tokens
    • max: 128 tokens
  • Samples:
    anchor positive
    Instruct: Represent this Buddhist passage for semantic retrieval.
    Query: Namo tassa bhagavato arahato sammāsambuddhassa    		 Homage to the Blessed One, the Perfected One, the Fully Self-Awakened Buddha
    Instruct: Represent this Buddhist passage for semantic retrieval.
    Query: 1. Evaṃ me sutaṃ – ekaṃ samayaṃ bhagavā sāvatthiyaṃ viharati jetavane anāthapiṇḍikassa ārāme. Tatra kho bhagavā bhikkhū āmantesi – ‘‘ bhikkhavo ’’ ti. ‘‘ Bhadante ’’ ti te bhikkhū bhagavato paccassosuṃ. Bhagavā etadavoca –    		 Thus have I heard — on one occasion the Blessed One was dwelling at Sāvatthī in Jeta's Grove, in Anāthapiṇḍika’s Park. There the Blessed One addressed the monks: “Monks!” “Venerable sir,” those monks replied to the Blessed One. The Blessed One said:
    Instruct: Represent this Buddhist passage for semantic retrieval.
    Query: 2.‘‘ Nāhaṃ, bhikkhave, aññaṃ ekasaddampi samanupassāmi yaṃ evaṃ purisassa cittaṃ pariyādāya tiṭṭhati yathayidaṃ, bhikkhave, itthisaddo. Itthisaddo, bhikkhave, purisassa cittaṃ pariyādāya tiṭṭhatī ’’ ti. Dutiyaṃ.    		 “Monks, I do not see a single sound that invades his mind and remains in a person as much as the sound of a woman. The sound of a woman, monks, overpowers a man’s mind.” The second.
  • Loss: MultipleNegativesRankingLoss with these parameters:
    {
    "scale": 20.0,
    "similarity_fct": "cos_sim",
    "gather_across_devices": false,
    "directions": [
        "query_to_doc"
    ],
    "partition_mode": "joint",
    "hardness_mode": null,
    "hardness_strength": 0.0
}

Evaluation Dataset

Unnamed Dataset

  • Size: 15,180 evaluation samples
  • Columns: anchor and positive
  • Approximate statistics based on the first 1000 samples:
    anchor positive
    type string string
    details
    • min: 21 tokens
    • mean: 82.69 tokens
    • max: 128 tokens
    • min: 7 tokens
    • mean: 58.92 tokens
    • max: 128 tokens
  • Samples:
    anchor positive
    Instruct: Represent this Buddhist passage for semantic retrieval.
    Query: ‘‘ Nāhaṃ, bhikkhave, aññaṃ ekarūpampi samanupassāmi yaṃ evaṃ purisassa cittaṃ pariyādāya tiṭṭhati yathayidaṃ, bhikkhave, itthirūpaṃ. Itthirūpaṃ, bhikkhave, purisassa cittaṃ pariyādāya tiṭṭhatī ’’ ti. Paṭhamaṃ.    		 “Monks, I do not see a single sight that invades his mind and remains in a person as much as the sight of a woman. The sight of a woman, monks, overpowers a man’s mind.” The first.
    Instruct: Represent this Buddhist passage for semantic retrieval.
    Query: Paṇihitaacchavaggo pañcamo.    		 The Chapter on the Directed and Clear is the fifth.
    Instruct: Represent this Buddhist passage for semantic retrieval.
    Query: 8. Kalyāṇamittādivaggo    		 8. Chapter on Good Friendship
  • Loss: MultipleNegativesRankingLoss with these parameters:
    {
    "scale": 20.0,
    "similarity_fct": "cos_sim",
    "gather_across_devices": false,
    "directions": [
        "query_to_doc"
    ],
    "partition_mode": "joint",
    "hardness_mode": null,
    "hardness_strength": 0.0
}

Training Hyperparameters

Non-Default Hyperparameters

  • per_device_train_batch_size: 16
  • gradient_accumulation_steps: 2
  • learning_rate: 2e-05
  • num_train_epochs: 1
  • lr_scheduler_type: cosine
  • warmup_steps: 0.05
  • fp16: True
  • remove_unused_columns: False
  • load_best_model_at_end: True
  • batch_sampler: no_duplicates

All Hyperparameters

Click to expand
  • do_predict: False
  • prediction_loss_only: True
  • per_device_train_batch_size: 16
  • per_device_eval_batch_size: 8
  • gradient_accumulation_steps: 2
  • eval_accumulation_steps: None
  • torch_empty_cache_steps: None
  • learning_rate: 2e-05
  • weight_decay: 0.0
  • adam_beta1: 0.9
  • adam_beta2: 0.999
  • adam_epsilon: 1e-08
  • max_grad_norm: 1.0
  • num_train_epochs: 1
  • max_steps: -1
  • lr_scheduler_type: cosine
  • lr_scheduler_kwargs: None
  • warmup_ratio: None
  • warmup_steps: 0.05
  • log_level: passive
  • log_level_replica: warning
  • log_on_each_node: True
  • logging_nan_inf_filter: True
  • enable_jit_checkpoint: False
  • save_on_each_node: False
  • save_only_model: False
  • restore_callback_states_from_checkpoint: False
  • use_cpu: False
  • seed: 42
  • data_seed: None
  • bf16: False
  • fp16: True
  • bf16_full_eval: False
  • fp16_full_eval: False
  • tf32: None
  • local_rank: -1
  • ddp_backend: None
  • debug: []
  • dataloader_drop_last: False
  • dataloader_num_workers: 0
  • dataloader_prefetch_factor: None
  • disable_tqdm: False
  • remove_unused_columns: False
  • label_names: None
  • load_best_model_at_end: True
  • ignore_data_skip: False
  • fsdp: []
  • fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
  • accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
  • parallelism_config: None
  • deepspeed: None
  • label_smoothing_factor: 0.0
  • optim: adamw_torch_fused
  • optim_args: None
  • group_by_length: False
  • length_column_name: length
  • project: huggingface
  • trackio_space_id: trackio
  • ddp_find_unused_parameters: None
  • ddp_bucket_cap_mb: None
  • ddp_broadcast_buffers: False
  • dataloader_pin_memory: True
  • dataloader_persistent_workers: False
  • skip_memory_metrics: True
  • push_to_hub: False
  • resume_from_checkpoint: None
  • hub_model_id: None
  • hub_strategy: every_save
  • hub_private_repo: None
  • hub_always_push: False
  • hub_revision: None
  • gradient_checkpointing: False
  • gradient_checkpointing_kwargs: None
  • include_for_metrics: []
  • eval_do_concat_batches: True
  • auto_find_batch_size: False
  • full_determinism: False
  • ddp_timeout: 1800
  • torch_compile: False
  • torch_compile_backend: None
  • torch_compile_mode: None
  • include_num_input_tokens_seen: no
  • neftune_noise_alpha: None
  • optim_target_modules: None
  • batch_eval_metrics: False
  • eval_on_start: False
  • use_liger_kernel: False
  • liger_kernel_config: None
  • eval_use_gather_object: False
  • average_tokens_across_devices: True
  • use_cache: False
  • prompts: None
  • batch_sampler: no_duplicates
  • multi_dataset_batch_sampler: proportional
  • router_mapping: {}
  • learning_rate_mapping: {}

Training Logs

Click to expand
Epoch Step Training Loss Validation Loss
0.0000 1 2.4117 -
0.0021 50 2.2977 -
0.0043 100 1.4074 -
0.0064 150 0.9612 -
0.0086 200 0.6430 -
0.0107 250 0.4462 -
0.0129 300 0.3157 -
0.0150 350 0.2660 -
0.0172 400 0.2222 -
0.0193 450 0.1691 -
0.0215 500 0.1565 -
0.0236 550 0.1109 -
0.0258 600 0.1512 -
0.0279 650 0.1317 -
0.0301 700 0.1211 -
0.0322 750 0.0998 -
0.0344 800 0.0884 -
0.0365 850 0.0988 -
0.0387 900 0.0813 -
0.0408 950 0.0744 -
0.0430 1000 0.0733 -
0.0451 1050 0.0689 -
0.0473 1100 0.0577 -
0.0494 1150 0.0637 -
0.0516 1200 0.0645 -
0.0537 1250 0.0491 -
0.0559 1300 0.0614 -
0.0580 1350 0.0512 -
0.0602 1400 0.0452 -
0.0623 1450 0.0577 -
0.0645 1500 0.0518 -
0.0666 1550 0.0366 -
0.0688 1600 0.0529 -
0.0709 1650 0.0412 -
0.0731 1700 0.0313 -
0.0752 1750 0.0397 -
0.0774 1800 0.0361 -
0.0795 1850 0.0428 -
0.0817 1900 0.0423 -
0.0838 1950 0.0288 -
0.0860 2000 0.0490 0.0196
0.0881 2050 0.0379 -
0.0903 2100 0.0309 -
0.0924 2150 0.0347 -
0.0946 2200 0.0327 -
0.0967 2250 0.0442 -
0.0989 2300 0.0244 -
0.1010 2350 0.0337 -
0.1032 2400 0.0308 -
0.1053 2450 0.0271 -
0.1075 2500 0.0342 -
0.1096 2550 0.0344 -
0.1118 2600 0.0285 -
0.1139 2650 0.0334 -
0.1161 2700 0.0359 -
0.1182 2750 0.0309 -
0.1203 2800 0.0455 -
0.1225 2850 0.0301 -
0.1246 2900 0.0229 -
0.1268 2950 0.0236 -
0.1289 3000 0.0327 -
0.1311 3050 0.0148 -
0.1332 3100 0.0189 -
0.1354 3150 0.0213 -
0.1375 3200 0.0246 -
0.1397 3250 0.0230 -
0.1418 3300 0.0246 -
0.1440 3350 0.0204 -
0.1461 3400 0.0207 -
0.1483 3450 0.0339 -
0.1504 3500 0.0202 -
0.1526 3550 0.0268 -
0.1547 3600 0.0252 -
0.1569 3650 0.0225 -
0.1590 3700 0.0279 -
0.1612 3750 0.0233 -
0.1633 3800 0.0204 -
0.1655 3850 0.0212 -
0.1676 3900 0.0256 -
0.1698 3950 0.0211 -
0.1719 4000 0.0209 0.0132
0.1741 4050 0.0245 -
0.1762 4100 0.0176 -
0.1784 4150 0.0184 -
0.1805 4200 0.0293 -
0.1827 4250 0.0256 -
0.1848 4300 0.0185 -
0.1870 4350 0.0115 -
0.1891 4400 0.0199 -
0.1913 4450 0.0145 -
0.1934 4500 0.0158 -
0.1956 4550 0.0238 -
0.1977 4600 0.0267 -
0.1999 4650 0.0222 -
0.2020 4700 0.0166 -
0.2042 4750 0.0175 -
0.2063 4800 0.0168 -
0.2085 4850 0.0188 -
0.2106 4900 0.0182 -
0.2128 4950 0.0108 -
0.2149 5000 0.0202 -
0.2171 5050 0.0128 -
0.2192 5100 0.0148 -
0.2214 5150 0.0182 -
0.2235 5200 0.0124 -
0.2257 5250 0.0137 -
0.2278 5300 0.0099 -
0.2300 5350 0.0203 -
0.2321 5400 0.0128 -
0.2343 5450 0.0168 -
0.2364 5500 0.0185 -
0.2386 5550 0.0143 -
0.2407 5600 0.0148 -
0.2428 5650 0.0290 -
0.2450 5700 0.0133 -
0.2471 5750 0.0146 -
0.2493 5800 0.0192 -
0.2514 5850 0.0229 -
0.2536 5900 0.0234 -
0.2557 5950 0.0114 -
0.2579 6000 0.0098 0.0111
0.2600 6050 0.0163 -
0.2622 6100 0.0119 -
0.2643 6150 0.0187 -
0.2665 6200 0.0184 -
0.2686 6250 0.0129 -
0.2708 6300 0.0131 -
0.2729 6350 0.0129 -
0.2751 6400 0.0138 -
0.2772 6450 0.0122 -
0.2794 6500 0.0198 -
0.2815 6550 0.0231 -
0.2837 6600 0.0150 -
0.2858 6650 0.0173 -
0.2880 6700 0.0156 -
0.2901 6750 0.0212 -
0.2923 6800 0.0159 -
0.2944 6850 0.0250 -
0.2966 6900 0.0144 -
0.2987 6950 0.0181 -
0.3009 7000 0.0123 -
0.3030 7050 0.0222 -
0.3052 7100 0.0155 -
0.3073 7150 0.0263 -
0.3095 7200 0.0216 -
0.3116 7250 0.0143 -
0.3138 7300 0.0092 -
0.3159 7350 0.0070 -
0.3181 7400 0.0203 -
0.3202 7450 0.0174 -
0.3224 7500 0.0262 -
0.3245 7550 0.0239 -
0.3267 7600 0.0126 -
0.3288 7650 0.0132 -
0.3310 7700 0.0145 -
0.3331 7750 0.0170 -
0.3353 7800 0.0125 -
0.3374 7850 0.0124 -
0.3396 7900 0.0205 -
0.3417 7950 0.0108 -
0.3439 8000 0.0156 0.0099
0.3460 8050 0.0151 -
0.3482 8100 0.0188 -
0.3503 8150 0.0059 -
0.3525 8200 0.0152 -
0.3546 8250 0.0202 -
0.3568 8300 0.0131 -
0.3589 8350 0.0145 -
0.3610 8400 0.0135 -
0.3632 8450 0.0147 -
0.3653 8500 0.0080 -
0.3675 8550 0.0118 -
0.3696 8600 0.0064 -
0.3718 8650 0.0204 -
0.3739 8700 0.0101 -
0.3761 8750 0.0125 -
0.3782 8800 0.0129 -
0.3804 8850 0.0062 -
0.3825 8900 0.0120 -
0.3847 8950 0.0112 -
0.3868 9000 0.0131 -
0.3890 9050 0.0166 -
0.3911 9100 0.0114 -
0.3933 9150 0.0157 -
0.3954 9200 0.0133 -
0.3976 9250 0.0145 -
0.3997 9300 0.0042 -
0.4019 9350 0.0142 -
0.4040 9400 0.0138 -
0.4062 9450 0.0214 -
0.4083 9500 0.0095 -
0.4105 9550 0.0067 -
0.4126 9600 0.0094 -
0.4148 9650 0.0063 -
0.4169 9700 0.0120 -
0.4191 9750 0.0116 -
0.4212 9800 0.0105 -
0.4234 9850 0.0222 -
0.4255 9900 0.0142 -
0.4277 9950 0.0121 -
0.4298 10000 0.0091 0.0080
0.4320 10050 0.0173 -
0.4341 10100 0.0098 -
0.4363 10150 0.0195 -
0.4384 10200 0.0117 -
0.4406 10250 0.0091 -
0.4427 10300 0.0146 -
0.4449 10350 0.0143 -
0.4470 10400 0.0132 -
0.4492 10450 0.0125 -
0.4513 10500 0.0116 -
0.4535 10550 0.0106 -
0.4556 10600 0.0099 -
0.4578 10650 0.0118 -
0.4599 10700 0.0051 -
0.4621 10750 0.0079 -
0.4642 10800 0.0086 -
0.4664 10850 0.0094 -
0.4685 10900 0.0065 -
0.4707 10950 0.0182 -
0.4728 11000 0.0160 -
0.4750 11050 0.0094 -
0.4771 11100 0.0129 -
0.4793 11150 0.0119 -
0.4814 11200 0.0183 -
0.4835 11250 0.0208 -
0.4857 11300 0.0125 -
0.4878 11350 0.0063 -
0.4900 11400 0.0106 -
0.4921 11450 0.0136 -
0.4943 11500 0.0086 -
0.4964 11550 0.0085 -
0.4986 11600 0.0115 -
0.5007 11650 0.0137 -
0.5029 11700 0.0141 -
0.5050 11750 0.0064 -
0.5072 11800 0.0123 -
0.5093 11850 0.0094 -
0.5115 11900 0.0090 -
0.5136 11950 0.0053 -
0.5158 12000 0.0086 0.0076
0.5179 12050 0.0093 -
0.5201 12100 0.0063 -
0.5222 12150 0.0121 -
0.5244 12200 0.0103 -
0.5265 12250 0.0066 -
0.5287 12300 0.0112 -
0.5308 12350 0.0127 -
0.5330 12400 0.0161 -
0.5351 12450 0.0071 -
0.5373 12500 0.0096 -
0.5394 12550 0.0088 -
0.5416 12600 0.0095 -
0.5437 12650 0.0075 -
0.5459 12700 0.0113 -
0.5480 12750 0.0121 -
0.5502 12800 0.0059 -
0.5523 12850 0.0099 -
0.5545 12900 0.0064 -
0.5566 12950 0.0094 -
0.5588 13000 0.0108 -
0.5609 13050 0.0074 -
0.5631 13100 0.0038 -
0.5652 13150 0.0043 -
0.5674 13200 0.0082 -
0.5695 13250 0.0112 -
0.5717 13300 0.0178 -
0.5738 13350 0.0073 -
0.5760 13400 0.0059 -
0.5781 13450 0.0042 -
0.5803 13500 0.0121 -
0.5824 13550 0.0093 -
0.5846 13600 0.0087 -
0.5867 13650 0.0169 -
0.5889 13700 0.0064 -
0.5910 13750 0.0047 -
0.5932 13800 0.0099 -
0.5953 13850 0.0086 -
0.5975 13900 0.0173 -
0.5996 13950 0.0141 -
0.6017 14000 0.0071 0.0070
0.6039 14050 0.0066 -
0.6060 14100 0.0069 -
0.6082 14150 0.0143 -
0.6103 14200 0.0084 -
0.6125 14250 0.0119 -
0.6146 14300 0.0142 -
0.6168 14350 0.0067 -
0.6189 14400 0.0186 -
0.6211 14450 0.0164 -
0.6232 14500 0.0060 -
0.6254 14550 0.0175 -
0.6275 14600 0.0060 -
0.6297 14650 0.0137 -
0.6318 14700 0.0161 -
0.6340 14750 0.0064 -
0.6361 14800 0.0067 -
0.6383 14850 0.0153 -
0.6404 14900 0.0056 -
0.6426 14950 0.0149 -
0.6447 15000 0.0052 -
0.6469 15050 0.0153 -
0.6490 15100 0.0069 -
0.6512 15150 0.0155 -
0.6533 15200 0.0060 -
0.6555 15250 0.0079 -
0.6576 15300 0.0075 -
0.6598 15350 0.0174 -
0.6619 15400 0.0084 -
0.6641 15450 0.0053 -
0.6662 15500 0.0117 -
0.6684 15550 0.0110 -
0.6705 15600 0.0131 -
0.6727 15650 0.0080 -
0.6748 15700 0.0099 -
0.6770 15750 0.0073 -
0.6791 15800 0.0030 -
0.6813 15850 0.0085 -
0.6834 15900 0.0118 -
0.6856 15950 0.0075 -
0.6877 16000 0.0113 0.0060

Training Time

  • Training: 6.1 hours

Framework Versions

  • Python: 3.12.13
  • Sentence Transformers: 5.4.1
  • Transformers: 5.0.0
  • PyTorch: 2.10.0+cu128
  • Accelerate: 1.13.0
  • Datasets: 5.0.0
  • Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers 

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss 

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}
Downloads last month
145
Safetensors
Model size
0.3B params
Tensor type
F32
·
Inference Providers NEW
Sentence Similarity
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for dhammanana/harrier-tipitaka-v1

Base model

microsoft/harrier-oss-v1-270m
Quantized
(12)
this model

Papers for dhammanana/harrier-tipitaka-v1