create_chunk_mapper {disk.frame}

Create function that applies to each chunk if disk.frame

Description

A function to make it easier to create functions like filter

Usage

create_chunk_mapper(chunk_fn, warning_msg = NULL, as.data.frame = FALSE)

Arguments

chunk_fn

The dplyr function to create a mapper for

warning_msg

The warning message to display when invoking the mapper

as.data.frame

force the input chunk of a data.frame; needed for dtplyr

Examples

filter = create_chunk_mapper(dplyr::filter)

#' example: creating a function that keeps only the first and last n row
first_and_last <- function(chunk, n, ...) {
  nr = nrow(chunk)
  print(nr-n+1:nr)
  chunk[c(1:n, (nr-n+1):nr), ]
}

#' create the function for use with disk.frame
first_and_last_df = create_chunk_mapper(first_and_last)

mtcars.df = as.disk.frame(mtcars)

#' the operation is lazy
lazy_mtcars.df = mtcars.df %>%
  first_and_last_df(2)

#' bring into R
collect(lazy_mtcars.df)

#' clean up
delete(mtcars.df)