tokenify {klsh}

Function to token a string into its k components

Description

Function to token a string into its k components

Usage

tokenify(string, k)

Arguments

string

A string or record

k

A parameter k, which is the number of shingle, tokens, or grams to break the string into

Value

Computes the tokenized or grammed version of a string

Examples

tokenify("Alexander",2)
tokenify("Alexander Smith", 2)