as_tokens {gibasa} | R Documentation |
Create a list of tokens
Description
Create a list of tokens
Usage
as_tokens(
tbl,
token_field = "token",
pos_field = get_dict_features()[1],
nm = NULL
)
Arguments
tbl |
A tibble of tokens out of |
token_field |
< |
pos_field |
Column containing features
that will be kept as the names of tokens.
If you don't need them, give a |
nm |
Names of returned list.
If left with |
Value
A named list of tokens.
Examples
## Not run:
tokenize(
data.frame(
doc_id = seq_along(5:8),
text = ginga[5:8]
)
) |>
prettify(col_select = "POS1") |>
as_tokens()
## End(Not run)
[Package gibasa version 1.1.1 Index]