Instructions to use llm-semantic-router/mmbert-embed-medical with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use llm-semantic-router/mmbert-embed-medical with sentence-transformers:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("llm-semantic-router/mmbert-embed-medical")

sentences = [
"A 60-year-old man is rushed to the emergency room after he was found unconscious in bed that afternoon. The patient’s wife says he has been confused and irritable for the past several days. She says he has a history of chronic daily alcohol abuse and has been hospitalized multiple times with similar symptoms His temperature is 37°C (98.6°F), the blood pressure is 110/80 mm Hg, the pulse is 90/min, and the respiratory rate is 14/min. On physical examination, the patient is minimally responsive to painful stimuli. His abdomen is distended with positive shifting dullness. Laboratory results are as follows:\nComplete blood count\nHematocrit 35%\nPlatelets 100,000/mm3\nWhite blood cells 5000/mm3\nLiver function studies\nSerum Albumin 2 g/dL\nAlkaline phosphatase (ALP) 200 IU/L\nAspartate aminotransferase (AST) 106 IU/L\nAlanine aminotransferase (ALT) 56 IU/L\nThe patient is admitted to the hospital and started on the appropriate treatment to improve his mental status. Which of the following best describes the mechanism of action of the drug that is most likely used to treat this patient’s symptoms?",
"A 60-year-old man is rushed to the emergency room after he was found unconscious in bed that afternoon. The patient’s wife says he has been confused and irritable for the past several days. She says he has a history of chronic daily alcohol abuse and has been hospitalized multiple times with similar symptoms His temperature is 37°C (98.6°F), the blood pressure is 110/80 mm Hg, the pulse is 90/min, and the respiratory rate is 14/min. On physical examination, the patient is minimally responsive to painful stimuli. His abdomen is distended with positive shifting dullness. Laboratory results are as follows:\nComplete blood count\nHematocrit 35%\nPlatelets 100,000/mm3\nWhite blood cells 5000/mm3\nLiver function studies\nSerum Albumin 2 g/dL\nAlkaline phosphatase (ALP) 200 IU/L\nAspartate aminotransferase (AST) 106 IU/L\nAlanine aminotransferase (ALT) 56 IU/L\nThe patient is admitted to the hospital and started on the appropriate treatment to improve his mental status. Which of the following best describes the mechanism of action of the drug that is most likely used to treat this patient’s symptoms?\n\nAnswer: Decreases pH in the gastrointestinal lumen",
"A 65-year-old man is brought to the emergency department after loss of consciousness. He is accompanied by his wife. He is started on intravenous fluids, and his vital signs are assessed. His blood pressure is 85/50 mm Hg, pulse 50/min, and respiratory rate 10/min. He has been admitted in the past for a heart condition. His wife is unable to recall the name of the condition, but she does know that the doctor recommended some medications at that time in case his condition worsened. She has brought with her the test reports from previous medical visits over the last few months. She says that she has noticed that he often has difficulty breathing and requires three pillows to sleep at night to avoid being short of breath. He can only walk for a few kilometers before he has to stop and rest. His wife also reports that he has had occasional severe coughing spells with pinkish sputum production. She also mentions that he has been drinking alcohol for the past 30 years. Which of the following medications will improve the prognosis of this patient?\n\nAnswer: Enalapril",
"A 71-year-old African American man is brought to the emergency department with a worsening productive cough and dyspnea for 2 days. He has had generalized bone pain for 2 months. He was admitted for pyelonephritis last month. He also received outpatient treatment for pneumonia almost 2 months ago. Over the past 2 months, he has been taking over-the-counter ibuprofen for pain as needed. He appears anxious. The vital signs include: temperature 38.8°C (101.8°F), pulse 95/min, respiratory rate 20/min, and blood pressure 155/90 mm Hg. The conjunctivae are pale. Crackles are heard in the right lower lobe. The cardiac examination shows no abnormalities. The laboratory studies show the following:\nHemoglobin 9 g/dL\nMean corpuscular volume 95 μm3\nLeukocyte count 13,500/mm3\nSegmented neutrophils 75%\nLymphocytes 25%\nPlatelet count 240,000/mm3\nESR 85 mm/hr\nSerum\nNa+ 135 mEq/L\nK+ 4.2 mEq/L\nCl− 113 mEq/L\nHCO3− 20 mEq/L\nCa+ 12.4 mg/dL\nAlbumin 4 g/dL\nUrea nitrogen 38 mg/dL\nCreatinine 2.2 mg/dL\nA chest X-ray shows a right lower lobe opacity and blurring of the ipsilateral diaphragmatic dome. Skull and pelvic X-rays are performed (see image). Which of the following is the most likely underlying cause of this patient’s recent infections?\n\nAnswer: Hypogammaglobulinemia"
]
embeddings = model.encode(sentences)

similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [4, 4]

Notebooks
Google Colab
Kaggle

HuaminChen commited on Mar 5

Commit

52c6063

verified ·

1 Parent(s): 7b7368e

Add config.json to onnx/

Browse files

Files changed (1) hide show

onnx/config.json +79 -0

onnx/config.json ADDED Viewed

	@@ -0,0 +1,79 @@

+{
+  "architectures": [
+    "ModernBertModel"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 2,
+  "classifier_activation": "gelu",
+  "classifier_bias": false,
+  "classifier_dropout": 0.0,
+  "classifier_pooling": "mean",
+  "cls_token_id": 1,
+  "decoder_bias": true,
+  "deterministic_flash_attn": false,
+  "dtype": "bfloat16",
+  "embedding_dropout": 0.0,
+  "eos_token_id": 1,
+  "global_attn_every_n_layers": 3,
+  "gradient_checkpointing": false,
+  "hidden_activation": "gelu",
+  "hidden_size": 768,
+  "initializer_cutoff_factor": 2.0,
+  "initializer_range": 0.02,
+  "intermediate_size": 1152,
+  "layer_norm_eps": 1e-05,
+  "layer_types": [
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention"
+  ],
+  "local_attention": 128,
+  "mask_token_id": 4,
+  "max_position_embeddings": 32768,
+  "mlp_bias": false,
+  "mlp_dropout": 0.0,
+  "model_type": "modernbert",
+  "norm_bias": false,
+  "norm_eps": 1e-05,
+  "num_attention_heads": 12,
+  "num_hidden_layers": 22,
+  "pad_token_id": 0,
+  "position_embedding_type": "sans_pos",
+  "repad_logits_with_grad": false,
+  "rope_parameters": {
+    "full_attention": {
+      "rope_theta": 160000,
+      "rope_type": "default"
+    },
+    "sliding_attention": {
+      "rope_theta": 160000,
+      "rope_type": "default"
+    }
+  },
+  "sep_token_id": 1,
+  "sparse_pred_ignore_index": -100,
+  "sparse_prediction": false,
+  "tie_word_embeddings": true,
+  "transformers_version": "5.0.0",
+  "vocab_size": 256000
+}