as_tokens | Create a list of tokens |
bind_lr | Bind importance of bigrams |
bind_tf_idf2 | Bind term frequency and inverse document frequency |
build_sys_dic | Build system dictionary |
build_user_dic | Build user dictionary |
collapse_tokens | Collapse sequences of tokens by condition |
dictionary_info | Get dictionary information |
gbs_tokenize | Tokenize sentences using 'MeCab' |
get_dict_features | Get dictionary features |
ginga | Whole text of 'Ginga Tetsudo no Yoru' written by Miyazawa Kenji from Aozora Bunko |
is_blank | Check if scalars are blank |
lex_density | Calculate lexical density |
mute_tokens | Mute tokens by condition |
ngram_tokenizer | Ngrams tokenizer |
pack | Pack a data.frame of tokens |
prettify | Prettify tokenized output |
tokenize | Tokenize sentences using 'MeCab' |