ngram_tokenizer | Ngrams tokenizer |
pack | Pack prettified data.frame of tokens |
polano | Whole text of 'Porano no Hiroba' written by Miyazawa Kenji from Aozora Bunko |
read_rewrite_def | Read a rewrite.def file |
strj_fill_iter_mark | Fill Japanese iteration marks |
strj_hiraganize | Hiraganize Japanese characters |
strj_katakanize | Katakanize Japanese characters |
strj_normalize | Convert text following the rules of 'NEologd' |
strj_rewrite_as_def | Rewrite text using rewrite.def |
strj_romanize | Romanize Japanese Hiragana and Katakana |
strj_segment | Segment text into phrases |
strj_tokenize | Simply tokenize sentence |
strj_transcribe_num | Transcribe Arabic to Kansuji |