universal-dependencies/universal_dependencies
Updated • 6.71k • 36
How to use KoichiYasuoka/roberta-base-japanese-juman-ud-goeswith with Transformers:
# Use a pipeline as a high-level helper
from transformers import pipeline
pipe = pipeline("token-classification", model="KoichiYasuoka/roberta-base-japanese-juman-ud-goeswith") # Load model directly
from transformers import AutoTokenizer, AutoModelForTokenClassification
tokenizer = AutoTokenizer.from_pretrained("KoichiYasuoka/roberta-base-japanese-juman-ud-goeswith")
model = AutoModelForTokenClassification.from_pretrained("KoichiYasuoka/roberta-base-japanese-juman-ud-goeswith")This is a RoBERTa model pretrained on Japanese Wikipedia and CC-100 texts for POS-tagging and dependency-parsing (using goeswith for subwords), derived from roberta-base-japanese.
from transformers import pipeline
nlp=pipeline("universal-dependencies","KoichiYasuoka/roberta-base-japanese-juman-ud-goeswith",trust_remote_code=True,aggregation_strategy="simple")
print(nlp("全学年にわたって小学校の国語の教科書に挿し絵が用いられている"))
fugashi is required.
Base model
nlp-waseda/roberta-base-japanese