Feature Extraction
sentence-transformers
Safetensors
Transformers
English
distilbert
fill-mask
learned sparse
opensearch
retrieval
passage-retrieval
document-expansion
bag-of-words
sparse-encoder
sparse
asymmetric
inference-free
splade
text-embeddings-inference
Instructions to use opensearch-project/opensearch-neural-sparse-encoding-doc-v3-distill with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use opensearch-project/opensearch-neural-sparse-encoding-doc-v3-distill with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("opensearch-project/opensearch-neural-sparse-encoding-doc-v3-distill") sentences = [ "The weather is lovely today.", "It's so sunny outside!", "He drove to the stadium." ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] - Transformers
How to use opensearch-project/opensearch-neural-sparse-encoding-doc-v3-distill with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="opensearch-project/opensearch-neural-sparse-encoding-doc-v3-distill")# Load model directly from transformers import AutoTokenizer, AutoModelForMaskedLM tokenizer = AutoTokenizer.from_pretrained("opensearch-project/opensearch-neural-sparse-encoding-doc-v3-distill") model = AutoModelForMaskedLM.from_pretrained("opensearch-project/opensearch-neural-sparse-encoding-doc-v3-distill", device_map="auto") - Inference
- Notebooks
- Google Colab
- Kaggle
Commit ·
504c07e
1
Parent(s): 3535a59
Update README.md
Browse files
README.md
CHANGED
|
@@ -80,7 +80,6 @@ for i in range(len(decoded_query)):
|
|
| 80 |
if doc_score != 0:
|
| 81 |
print(f"Token: {query_token}, Query score: {query_score:.4f}, Document score: {doc_score:.4f}")
|
| 82 |
|
| 83 |
-
# Top tokens 8 for each text:
|
| 84 |
# Token: ny, Query score: 5.7729, Document score: 0.8049
|
| 85 |
# Token: weather, Query score: 4.5684, Document score: 0.9710
|
| 86 |
# Token: now, Query score: 3.5895, Document score: 0.4720
|
|
|
|
| 80 |
if doc_score != 0:
|
| 81 |
print(f"Token: {query_token}, Query score: {query_score:.4f}, Document score: {doc_score:.4f}")
|
| 82 |
|
|
|
|
| 83 |
# Token: ny, Query score: 5.7729, Document score: 0.8049
|
| 84 |
# Token: weather, Query score: 4.5684, Document score: 0.9710
|
| 85 |
# Token: now, Query score: 3.5895, Document score: 0.4720
|