Token Classification
Transformers
PyTorch
Literary Chinese
roberta
classical chinese
literary chinese
ancient chinese
sentence segmentation
Instructions to use KoichiYasuoka/roberta-classical-chinese-base-sentence-segmentation with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use KoichiYasuoka/roberta-classical-chinese-base-sentence-segmentation with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("token-classification", model="KoichiYasuoka/roberta-classical-chinese-base-sentence-segmentation")# Load model directly from transformers import AutoTokenizer, AutoModelForTokenClassification tokenizer = AutoTokenizer.from_pretrained("KoichiYasuoka/roberta-classical-chinese-base-sentence-segmentation") model = AutoModelForTokenClassification.from_pretrained("KoichiYasuoka/roberta-classical-chinese-base-sentence-segmentation", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Commit ·
d6bd290
1
Parent(s): a0a997a
Reference
Browse files
README.md
CHANGED
|
@@ -31,3 +31,7 @@ p=[model.config.id2label[q] for q in torch.argmax(model(tokenizer.encode(s,retur
|
|
| 31 |
print("".join(c+"。" if q=="E" or q=="S" else c for c,q in zip(s,p)))
|
| 32 |
```
|
| 33 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 31 |
print("".join(c+"。" if q=="E" or q=="S" else c for c,q in zip(s,p)))
|
| 32 |
```
|
| 33 |
|
| 34 |
+
## Reference
|
| 35 |
+
|
| 36 |
+
Koichi Yasuoka: Sentence Segmentation of Classical Chinese Texts Using Transformers and BERT/RoBERTa Models, IPSJ Symposium Series, Vol.2021, No.1 (December 2021), pp.104-109.
|
| 37 |
+
|