tok_normalizer {tok}

Generic class for normalizers

Description

Generic class for normalizers

Generic class for normalizers

Public fields

.normalizer

Internal pointer to normalizer object

Methods

Public methods

• tok_normalizer$new()

• tok_normalizer$clone()

Method new()

Initializes a tokenizer

Usage

tok_normalizer$new(normalizer)

Arguments

normalizer

a raw pointer to a tokenizer

Method clone()

The objects of this class are cloneable with this method.

Usage

tok_normalizer$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.

See Also

Other normalizers: normalizer_nfc, normalizer_nfkc