BaseTokenizer {fastai}R Documentation

BaseTokenizer

Description

Basic tokenizer that just splits on spaces

Usage

BaseTokenizer(split_char = " ")

Arguments

split_char

separator

Value

None

[Package fastai version 2.2.2 Index]