tok_processor {tok}

Generic class for processors

Description

Generic class for processors

Generic class for processors

Public fields

.processor

Internal pointer to processor object

Methods

Public methods

• tok_processor$new()

• tok_processor$clone()

Method new()

Initializes a tokenizer

Usage

tok_processor$new(processor)

Arguments

processor

a raw pointer to a processor

Method clone()

The objects of this class are cloneable with this method.

Usage

tok_processor$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.

See Also

Other processors: processor_byte_level