SentencePiece

SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing

概要 Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates

Neural Machine Translation of Rare Words with Subword Units (2016)