read_text_batches {TheOpenAIR} | R Documentation |
Load data from text files in batches
Description
This function reads in data from text files in batches using the read_lines_chunked function from the readr package.
Usage
read_text_batches(text, batch_size = 3500)
Arguments
text |
character string, either containing a path to a text file to read in or containing the text. |
batch_size |
integer indicating the number of lines to read in per batch (default is 3500) |
Value
a numeric value indicating the total number of tokens in the text file
Author(s)
Ulrich Matter umatter@protonmail.com
Examples
## Not run:
data_path <- system.file("text", "lorem.txt", package = "OpenAIR")
text_data <- read_text_batches(data_path)
## End(Not run)
[Package TheOpenAIR version 0.1.0 Index]