ZombitX64
/

zombitx64-thaitokenizer

text-processing

Model card Files Files and versions

zombitx64-thaitokenizer / special_tokens_map.json

JonusNattapong's picture

Upload Thai tokenizer v1.0.0

6a65062 verified 9 months ago

history blame contribute delete

103 Bytes

	{
	"pad_token": "[PAD]",
	"unk_token": "[UNK]",
	"bos_token": "[BOS]",
	"eos_token": "[EOS]"
	}