Add tokenizer.json
#3
by jonatanklosko - opened
Generated with:
from transformers import BlenderbotTokenizerFast
tokenizer = BlenderbotTokenizerFast.from_pretrained("/Users/jonatanklosko/git/hf/blenderbot-1B-distill/")
tokenizer.save_pretrained("/Users/jonatanklosko/git/hf/blenderbot-1B-distill/")
jonatanklosko changed pull request status to open
Thanks!
patrickvonplaten changed pull request status to merged