Instructions to use qikp/small-german-tokenizer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use qikp/small-german-tokenizer with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("qikp/small-german-tokenizer", dtype="auto") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -5,6 +5,8 @@ datasets:
|
|
| 5 |
- deutsche-telekom/Ger-RAG-eval
|
| 6 |
tags:
|
| 7 |
- tokenization
|
|
|
|
|
|
|
| 8 |
---
|
| 9 |
|
| 10 |
# Small German Tokenizer
|
|
|
|
| 5 |
- deutsche-telekom/Ger-RAG-eval
|
| 6 |
tags:
|
| 7 |
- tokenization
|
| 8 |
+
language:
|
| 9 |
+
- de
|
| 10 |
---
|
| 11 |
|
| 12 |
# Small German Tokenizer
|